Compare commits

..
5675 changed files with 309259 additions and 731118 deletions

No files matched your search

-21
View File
@@ -1,21 +0,0 @@
{
"name": "rea",
"interface": {
"displayName": "REA"
},
"plugins": [
{
"name": "rea",
"source": {
"source": "npm",
"package": "rea-agents",
"version": "6.8.0"
},
"policy": {
"installation": "AVAILABLE",
"authentication": "ON_INSTALL"
},
"category": "Developer Tools"
}
]
}
@@ -10,16 +10,11 @@ equivalent completed investigations. A useful outcome can also be a measured
decision to leave an implementation alone.
Follow the checkout's [performance priorities](../../../AGENTS.md#performance),
[tool design](../../../docs/development/tool-design.md), and
[testing guide](../../../docs/development/testing.md). This skill supplies an optimization
[tool design](../../../docs/tool-design.md), and
[testing guide](../../../docs/testing.md). This skill supplies an optimization
workflow, not additional permission to execute targets, publish, or merge.
Use the current contracts, provider prerequisites, and user authorization.
Reserve performance measurements for changes that target performance or have a
concrete regression concern. Routine cleanup uses correctness checks and does
not require this optimization workflow or synthetic benchmarks merely because
helper calls, imports, or representations change.
Read [the Kuna case study](references/kuna-case-study.md) when selecting a
reuse strategy or interpreting a benchmark. Its pinned observations illustrate
tradeoffs; they are not current-provider guarantees or universal speedups.
+4 -5
View File
@@ -6,7 +6,7 @@ description: Prepare or rewrite REA release changelogs and GitHub release notes
# REA changelog update
Use this for release-note preparation and contributor attribution. Read
[docs/development/releasing.md](../../../docs/development/releasing.md) from the repository root for
[docs/releasing.md](../../../docs/releasing.md) from the repository root for
publication authority and the existing checkpoint checks. This skill prepares
notes; it does not authorize merging, tagging, or publishing.
@@ -68,10 +68,9 @@ closing-issue reporters. Grouping preserves all those credits in one line.
When Highlights, detailed entries or migrations reference the same work,
put its thanks in one of those entries, not all of them. Include issue links.
Direct-commit bullets carry the commit link and known human credit.
Exclude GitHub bots and known agent accounts. REA's observed Anthropic, Cursor,
Codex, and Copilot co-author trailers resolve to `@claude`, `@cursoragent`,
`@codex`, and `@Copilot` with GitHub type `User`; the credit policy excludes
them while retaining their metadata.
Exclude GitHub bots and known agent accounts. REA's observed Anthropic, Cursor
and Codex co-author trailers resolve to `@claude`, `@cursoragent` and `@codex`
with GitHub type `User`; the credit policy excludes them while retaining their metadata.
At the REA maintainer's request, omit `@morluto` self-thanks while preserving
authorship in the inventory. Other human contributors remain eligible.
Use explicit GitHub links for cross-repository references to avoid ambiguous
+1 -1
View File
@@ -9,7 +9,7 @@ Establish which checks still catch real regressions and which costs can be
removed without losing that protection. The outcome may be a verified fix,
an evidence-backed recommendation, or an explicit decision to stop.
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/development/testing.md),
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/testing.md),
and [contribution rules](../../../CONTRIBUTING.md). Use their current commands
and pinned toolchain, rather than commands remembered from an earlier audit.
Preserve the requested scope: investigation alone does not authorize edits;
+3 -19
View File
@@ -1,36 +1,20 @@
---
name: rea-tool-design
description: Audit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
description: Design or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
---
# REA tool design
Use the [tool-design guide](../../../docs/development/tool-design.md) for tool boundaries,
Use the [tool-design guide](../../../docs/tool-design.md) for tool boundaries,
contract semantics, provider ownership, and discoverability. Inspect the current
contract and nearest existing tool relevant to the analyst question; source,
documentation, and live catalogs may describe different versions.
For provider-backed work, exercise the analyst's task through REA and compare
relevant native provider and independent artifact observations where available.
Check applicable partial results, recovery and session boundaries; report what
was not exercised. For failure guidance, follow [design recovery around the
failure](../../../docs/development/tool-design.md#design-recovery-around-the-failure).
Use [workflow verification](../../../docs/development/testing.md#provider-backed-workflow-verification)
to select evidence for the proposed contract instead of seeking provider API
parity.
For an audit, follow [audit existing tools](../../../docs/development/tool-design.md#audit-existing-tools):
scope representative tasks, exercise the live workflow, and distinguish
reproduced bugs, supported limitations, design opportunities and unverified
claims. Check relevant rendered MCP prompts, guided workflows and packaged skill
advice against the live contracts and exercise their suggested handoffs. Report
working alternatives and evidence limits as well as findings.
For a design request, return the proposed boundary, input/result shape, Evidence
semantics, nearest alternative, and verification approach. Implement when
requested, preserving the user's scope and existing authorization.
For implementation or verification, consult [testing](../../../docs/development/testing.md)
For implementation or verification, consult [testing](../../../docs/testing.md)
for the affected consumer and provider lanes, and
[contributing](../../../CONTRIBUTING.md) for generated-file ownership. Read
provider/workflow guides only as the task requires.
+63 -153
View File
@@ -1,8 +1,8 @@
---
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "61"
version: "46"
---
# REA
@@ -17,9 +17,6 @@ readiness or provider commands.
Installing this skill supplies instructions; it does not register the REA MCP
server, install analysis engines, or add tools to an already-running agent.
The exception is a Claude Code or Codex plugin install: the plugin bundles
this skill and registers its own `rea` server, outside the state that
`rea setup` and `rea doctor` manage.
If REA tools are available and their registration is not known to be stale,
proceed directly to the target. Do not run diagnostics or setup before every
@@ -40,23 +37,42 @@ repeat or known not to have started. If it may already have changed the
target, or it fails again, tell the user and ask them to restart the MCP
client. If REA could not start its connection, retry once, then tell the user
and ask them to restart the MCP client. Do not retry a startup failure that
returns the same unchanged result. For plugin installs, follow
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin).
For setup-owned registrations, run `rea doctor` when REA could not start
returns the same unchanged result. Run `rea doctor` when REA could not start
or tools stay missing after that restart. For `cleanup_incomplete`, a retryable
capacity limit, or `configuration_invalid`, follow the recovery guide's ladder
before asking for a restart. Read that guide before a second repair attempt.
Choose the installation owner before repairing registration. Plugin installs use
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin);
only setup-owned registrations use the guide's
[registration repair plan](references/connection-and-recovery.md#repair-registration-when-needed).
A plugin-only doctor result of `missing` does not call for `rea setup`.
1. Diagnose without changing REA or client configuration. `npx` may download
the package and write to npm's cache. For Codex, run
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
`commandcode`, `vscode`, `grok_build`, `omp`, or `grok_bot`. `grok_bot` has no local
file registration: doctor reports the Grok Bot chat step, and setup does not
write `mcp.json`. If the client is unknown, use `doctor --json` and
inspect its registration results before choosing a setup scope.
2. Distinguish the reason. Missing, malformed, or stale registration needs a
scoped configuration repair. An aligned registration with no tools in the
active session needs a restart/reconnection; doctor cannot prove that the
current agent has connected. A missing provider affects only tasks requiring
that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
and Android inspection has separate bring-your-own JADX/Java prerequisites.
3. For a configuration repair, prepare the read-only plan:
`npx -y rea-agents@latest setup --client codex --dry-run --json`.
Use the current client's ID, show its exact proposed paths, backups, and
changes, and obtain approval before setup writes configuration or installs
Hopper. Setup normally installs the matching bundled skill too; include that
replacement in the reviewed plan. After approval, apply the same scope with
`npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
only if that separate installation was needed and explicitly approved.
4. Restart/reconnect the affected agent. Verify that REA tools actually appear
in the session, then resume the original investigation. If they remain
absent, inspect the client's MCP launch error rather than repeating setup.
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
JADX, Kuna, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
software to repair MCP registration. For an unsupported client, use manual stdio
registration with an existing installation’s absolute runtime and entry script, or continue through the CLI.
registration with a version-pinned `rea-agents` package or continue through the CLI.
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
@@ -69,11 +85,8 @@ graph, limitations, and unknowns with the same care as an MCP result; the CLI
returns the Evidence record directly. This fallback does not establish that
MCP is configured. Native CLI tasks still require their selected engine.
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
is within the authorized scope, use `npx -y rea-agents@latest` in its place from a
trusted directory outside the target project, with absolute target paths. Managed
setup binds the current installation directly; retain its runtime and package
cache entry, or restore that installation before repairing its registration.
For setup-owned registration, configuration repair starts with a read-only setup plan; show paths, backups and
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
Configuration repair starts with a read-only setup plan; show paths, backups and
skill replacement before applying authorized changes. Preserve existing approval
and target-execution boundaries.
@@ -94,9 +107,6 @@ explicit path or endpoint and do not need it.
than extra calls and schemas. Runtime options are metadata, not files.
Compact delivery still inventories and hashes members. See the
[Python packaged applications guide](https://github.com/morluto/rea/blob/main/docs/python-packaged-applications.md).
- Bun `--compile` executable: `open_binary`, then `inspect_artifact`; the Bun
module graph is the application layer. Extract stored modules and analyze the
JavaScript. See [Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
`open_bundle_resources`, then the selected resource tool. This does not
require `CFBundleExecutable` or a deep provider. Resource sessions have no
@@ -108,9 +118,7 @@ explicit path or endpoint and do not need it.
- Firmware image: `inspect_firmware_regions` when advertised. Use
`extract_firmware` when extraction is requested, with a caller-selected new
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md).
- ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact`
without a provider; see the ESP section of `references/native-and-artifacts.md`.
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
optional metadata recovery and its supported host/layout boundary.
@@ -121,29 +129,10 @@ explicit path or endpoint and do not need it.
cleanup effects, and do not require a preliminary static session unless their
own contract says so.
- User-owned browser page already open: `list_browser_targets`.
- Passive traffic capture of a SPA: the default scope is the page's own origin,
and data APIs usually live on another origin. Discover API origins from
absolute URLs in loaded bundles or script text via JavaScript analysis tools
(another passive capture without those origins still scopes to the page
origin, and `completeness.excluded` counts lack URLs). Pass both the page
origin and those API origins in `allowed_origins` — the list replaces the
default scope — and read `completeness.excluded` before reporting that an
API is absent. For long captures with body shapes, use
`export_evidence_bundle` instead of expecting an inline result.
- Browser action the agent must perform (click, type, navigate):
`capture_browser_scenario` with declared actions. See
[runtime observation](references/runtime-observation.md) for connect/launch
and the focused follow-up browser tools.
- Agent environment (AgentEnv bundle) of a web application: read
[environments](references/environments.md) before recording; it records with
`capture_browser_scenario` and writes the bundle with `export_environment_bundle`.
- Task for a desktop application's exported environment, from a
`capture-desktop-application` folder: read
[desktop application tasks](references/environments.md#tasks-for-a-desktop-application-environment).
- Environment of a Linux desktop app with a task, from a request such as
"build me an Inkscape environment where the agent changes a poster's
headline": read [desktop builds](references/desktop-build.md). You drive the
app on a build computer from `start_build_computer`, then export and witness.
- Fiddler Everywhere traffic or a native `.saz` capture: read the
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
@@ -154,7 +143,7 @@ explicit path or endpoint and do not need it.
old-generation heap is `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` (default 192 MiB,
floor 8, maximum 16384), set where REA starts. Host memory and `NODE_OPTIONS` do not
raise it. See
[historical captures](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md) for the exact upstream
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
or establish network loss. Prefer detail summary, then kind qlog trace/stream
@@ -166,10 +155,8 @@ explicit path or endpoint and do not need it.
and field names/types before choosing fields for `inspect_packet_capture`.
Query/page the relevant records; registrations do not establish observed traffic.
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
display filters and decode-as rules. Requires Linux, util-linux prlimit and
TShark 4.x on an absolute `PATH` entry or via the optional absolute
`REA_TSHARK_COMMAND` override; see
[capture guide](https://github.com/morluto/rea/blob/main/docs/guides/data/packet-captures.md).
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
Packet limits count input packets before filtering. Output defaults to 50 matching
frames; use `match_offset` / `match_limit` for later pages, or explicit null
limit for complete matching output. Preserve the source digest across pages.
@@ -177,7 +164,7 @@ explicit path or endpoint and do not need it.
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
and capinfos, with Linux prlimit or Windows Job Objects; see
[packet summaries](https://github.com/morluto/rea/blob/main/docs/guides/data/pcap-analysis.md).
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
Select an exact DNS question for address pages; page exhaustion does not prove
network completeness. This reads recordings and does not acquire traffic.
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
@@ -185,7 +172,7 @@ explicit path or endpoint and do not need it.
relevant details before choosing settings; this does not scan hardware.
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/guides/signals/signal-recordings.md).
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
Output defaults to 50 literal lines; select annotation classes and page with
`line_offset` / `line_limit`, or explicit null for all remaining output.
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
@@ -194,7 +181,7 @@ explicit path or endpoint and do not need it.
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
Select JavaScript glue and candidate paths explicitly and preserve URL/path
ambiguity. No implicit fetch or target execution; WAT is not original source.
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/guides/wasm/wasm-artifacts.md).
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/wasm-artifacts.md).
- Explicit local Android APK (or ZIP/AAB) signing: `inspect_apk_signature`.
Prefer its summary or selected signer/digest/lineage page; retain complete
Evidence and use `inspect_analysis_view` kind `apk-signature` for follow-ups.
@@ -213,17 +200,13 @@ explicit path or endpoint and do not need it.
Evidence stays retained under its original identity. Use the returned
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
Candidate coverage does not establish interface completeness. See the
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/guides/evm/evm-bytecode.md).
- Embedded Go compiler, module, build-setting metadata, or static `(ptr, len)`
string literals: `inspect_go_binary` with the explicit local file (pass
`include_string_literals: true` and optional `max_string_literals` for bounded
inferred static string-header recovery, and page retained rows with
`inspect_analysis_view` kind `go` collection `string_literals`). This built-in
target-free inspection requires no Go installation or disassembler. Preserve
raw module bytes and source ranges; absent metadata does not prove a binary is
not Go, static string recovery excludes compiler-inlined stack or immediate
strings, and unparsed text records remain explicit. See the
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/go-binaries.md).
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
with the explicit local file. This built-in target-free inspection requires no
Go installation or disassembler. Preserve raw module bytes and source ranges;
absent metadata does not prove a binary is not Go, and unparsed text records
remain explicit. See the
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
- Embedded CUDA/HIP code objects: `inspect_gpu_code_objects`; inventory targets
first, then decode selected payloads and query retained GPU Evidence.
- Bundled GGUF or safetensors model declarations: `inspect_model_artifact`
@@ -276,31 +259,6 @@ explicit path or endpoint and do not need it.
proof the image never imported APIs through another table. Bound snapshots
and loaded IAT contents remain unobserved. See the
[PE import guide](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
- Windows Shell Link shortcuts: `inspect_windows_shortcut` with the explicit
local `.lnk` file. This built-in target-free inspection requires no Windows
SDK or COM. Preserve header flags and FILETIMEs, IDList items, LinkInfo
volume and network names, StringData arguments and icon location, ExtraData
signatures, and Tracker droid facts labelled derived. Unknown IDList items
and ExtraData signatures stay raw with SHA-256. The target path is not
resolved or executed. See the
[Windows shortcut guide](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
- PE TLS callbacks and load configuration: `inspect_pe_load_config` with the
explicit local PE32/PE32+ file. This built-in target-free inspection requires
no Windows SDK. Preserve callback VAs as strings with their RVA, file offset
and section, and the size-covered security cookie, SafeSEH, CFG,
EH-continuation and XFG fields. Guard-flag mitigation claims are inferred
linker declarations, not proof of enforcement. A missing directory is observed
absence of that declaration. See the
[PE load configuration guide](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
- ELF, PE, and Mach-O hardening: `inspect_binary_hardening` with the explicit
local file. This built-in target-free inspection requires no pwntools or other
host tool. One image per Mach-O slice. Each property is `enabled`, `disabled`,
`partial`, `unknown`, or `not_applicable`, with the header field or import
behind it. A missing stack-protector or fortify import is `unknown`, not
proof the protection was omitted. Authenticode and Mach-O code signatures
report presence, not validity. Hardened runtime stays unknown and names
`inspect_signature`. The image is not executed. See the
[binary hardening guide](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
- ELF note tables: `inspect_elf_notes` with the explicit local ELF32/ELF64
file. This built-in target-free inspection requires no disassembler. Preserve
PT_NOTE and SHT_NOTE ranges, owners, types, descriptor SHA-256, and GNU
@@ -315,16 +273,6 @@ explicit path or endpoint and do not need it.
body; other types, including `"288"` stackshots, return no crash result.
Opened Mach-O and dSYM UUID joins remain unobserved. See the
[Apple IPS guide](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
- Linux kernel module metadata: `inspect_kernel_module` with the explicit local
ELF32/ELF64 relocatable file. This built-in target-free inspection requires
no Linux host or `modinfo`. Preserve `.modinfo` key order and duplicate keys,
`__versions` CRC hex, section file ranges, and the appended signature
trailer's PKCS#7 SHA-256. A missing section is observed absence when section
names are known, not proof the module never carried that metadata.
`struct module` is not decoded. The signature is not validated. gzip, xz, and
zstd containers are unsupported; decompress into a separate file with an
external codec utility, then inspect the decompressed `ET_REL` image. See the
[kernel module guide](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
- CPython 3.7–3.13 `.pyc` header, nested marshal code objects and bytecode
identity: `inspect_python_pyc` with the explicit local file. This built-in
target-free inspection requires no Python installation. Preserve magic,
@@ -348,17 +296,17 @@ explicit path or endpoint and do not need it.
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
report truncation and have unspecified order. Use a quiescent snapshot rather
than assuming a live database/WAL pair was copied atomically. See the
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
- Explicit Linux ELF file for offline layout, symbols, relocations or static
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
caller-supplied pwntools without opening a disassembler database. Preserve its
raw locations and inference/coverage limits; see the
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
- Supplied Linux x86-64 ELF core: `inspect_recorded_crash`. Read every recorded
thread, raw note and source-bound register without opening a live target.
Historical PIDs do not select live processes. Request `include_debugger_context`
only when core-only mapping candidates help; see the
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md).
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md).
- PE resource type/name/language trees and static icon group references:
`inspect_pe_resources` with an explicit path. This portable operation requires
no active target or engine; opaque payloads retain file ranges and SHA-256.
@@ -374,12 +322,7 @@ The legacy attached profile binds the already-open GUI input by SHA-256 and
leaves its database open. The modern headless profile creates and closes an
owned private database without saving. Installation and registration reuse
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md).
For migration to the separate native IDA MCP plugin, follow the guide's
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md#migrate-a-legacy-attached-registration).
Replace the upstream registration explicitly; modern mrexodia tools are not the
native profile. Native attachment preserves the database but does not provide
direct callers; check session availability and the guide's verification limits.
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
Proceed directly to `analyze_function`, pseudocode, or function/string searches
after opening; IDA does not supply `binary_overview`. Consult session availability
when composing broader workflows. Modern direct callers and unsupported dossier
@@ -410,28 +353,18 @@ native target whose analysis tool operates on the active target. Do not call an
unadvertised tool; inspect `binary_session` with `{}` and its
`result.tool_availability` for availability reasons and remediation. Expand each
group's `names` into entries, inheriting the inventory-level client requirements
unless the group overrides them. The grouped name union is the complete tool
inventory; use `tools/list` for canonical tool schemas. The default summary keeps
each provider candidate's availability and target-support status, code and
reason; `omitted_fields` lists what it leaves out. Request `{"detail": "complete"}`
only when you need an unavailable candidate's remediation, provider
diagnostics, candidate capability sets, or tool surface, effect and annotation
flags; in those closed flag vocabularies, listed
flags are true and omitted known flags are false. Provider candidate facts
describe possible providers, while top-level session capabilities describe the
current route and `analysis_provider_binding` names the selected provider. Do
not treat those scopes as interchangeable.
For editable Ghidra analysis that must survive a later agent process, select
the [owned Program checkpoint workflow](references/native-and-artifacts.md#continue-editable-ghidra-analysis-across-processes)
before making metadata edits. It is separate from Evidence/Unknown persistence.
unless the group overrides them. In its closed effect and annotation vocabularies,
listed flags are true and omitted known flags are false. The grouped name union
is the complete tool inventory; use `tools/list` for canonical tool schemas.
Provider candidate facts describe possible providers, while top-level session
capabilities describe the current route and `analysis_provider_binding` names the
selected provider. Do not treat those scopes as interchangeable.
For a Ghidra startup timeout on a large native target, follow
[deadline recovery](references/native-and-artifacts.md#ghidra-startup-deadline-recovery)
before abandoning deep analysis or repeating the same cold import. The default
330 seconds can be changed for new startup attempts through
`update_analysis_settings` using `ghidra.startup_timeout_ms`. Changing the
server's launch environment instead requires a new REA process.
330 seconds is configurable; changing the server's environment requires a new
REA process.
## Work summary-first
@@ -443,18 +376,9 @@ call `analyze_javascript_application` or `inspect_binary_layout` with
a flag: its default reply is the summary, and the complete dossier stays
retained under `parent_evidence_id` for facet paging. Pass that ID as a
`retained-evidence` reference to `compare_functions`, or inside
`match_native_functions` `left_observations`/`right_observations` or
`build_call_path.functions`; request `"detail": "complete"` only when the
dossier must travel inline.
Use the same exact-reference form in other advertised logical Evidence inputs,
including optional sources, arrays and nested sources across domains. Each
source can independently be inline or retained; preserve its source identity,
coverage, partial facts and unknowns. A summary or inspection page has its own
Evidence ID: use `normalized_result.parent_evidence_id` when the consumer needs
the complete source. For portable CLI follow-ups, commands advertising
`--evidence-bundle` resolve references from that selected exported bundle.
See [Evidence composition](references/evidence-workflows.md) for scope and
recovery; file selectors are supported only where explicitly advertised.
`match_native_functions` `left_observations`/`right_observations`; request
`"detail": "complete"` only when the dossier must travel inline, as
`build_call_path` requires.
Page module identities and select the returned `node_id` before requesting an
item; a single file path can represent several graph roles. Do not repeat an
identical analysis or rejected call without a changed input or resolved blocker.
@@ -473,10 +397,7 @@ output also streams without a single MCP frame.
When the retained parent ID is unknown, use `get_evidence_bundle` with
`detail: "summary"` and exact filters for operation, target SHA-256,
analysis-profile digest or canonical procedure address. Select its returned
Evidence ID for the next view instead of requesting the complete bundle. When
that record has no supported view (for example an annotation), read it with
`detail: "records"` and an `evidence_id` filter; `missing_evidence_links`
names linked sources this session does not retain.
Evidence ID for the next view instead of requesting the complete bundle.
`retention: "complete-record"` means the whole record is stored; native
value-flow truncation and unknown metadata remain separate. For an exported
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
@@ -550,21 +471,10 @@ as complete while required questions remain open.
[references/runtime-observation.md](references/runtime-observation.md)
- Evidence paging, comparisons, residual unknowns, and verification:
[references/evidence-workflows.md](references/evidence-workflows.md)
- Agent environments (AgentEnv bundles) of a web application, recorded from
a live instance or a HAR:
[references/environments.md](references/environments.md)
- Tasks for a desktop application's exported environment:
[references/environments.md](references/environments.md#tasks-for-a-desktop-application-environment)
- A desktop app environment from a request, built and witnessed on build
computers: [references/desktop-build.md](references/desktop-build.md)
## Finish the task
Explain findings in plain language and tie them to returned evidence. When the
user asks to build something, use normal coding tools and separate observed
behavior from design choices. Close an opened native session with
`close_binary` when its engine is no longer needed. Evidence references and Unknown
history remain in memory on the same MCP connection. End the investigation with
`reset_investigation`; supply `save_path` to checkpoint the complete bundle before
clearing. Import the saved bundle to restore original references and Unknown
history in a new process. Retention is bounded and is not automatic persistence.
`close_binary` when the investigation is complete.
@@ -13,27 +13,6 @@ own APK path and explicit selectors; do not open a native session first.
Choose an `overload_index` from the class inventory when methods are ambiguous
instead of guessing.
For JNI implementations, discover `inspect_android_jni_bindings`. Select raw
DEX class/method identities and optional descriptors, ABI or logical library
paths. Prefer `detail: "summary"`; complete Evidence remains retained and
`android-jni` views page declarations, libraries, candidates or binding rows
with exact fields. Global native-pair ambiguity survives class selection.
Preserve slot/relocation provenance and source coverage: matched means observed
candidate presence; partial scans do not establish uniqueness or absence.
This reads DEX/ELF bytes without loading the APK into the decompiler or executing
target code. Runtime registration and loading remain unknown. See the
[JNI guide](https://github.com/morluto/rea/blob/main/docs/guides/android/android-analysis.md#link-native-declarations-to-jni-candidates).
For an R8/ProGuard release build with its `mapping.txt`, pass the absolute
`mapping_path` to `inspect_android_class` or `inspect_android_method`. Prefer
`detail: "summary"` to receive normalized symbols or the selected body without
duplicate raw class metadata. Complete Evidence stays retained under the linked
parent ID and can be exported; CLI summary delivery requires `--evidence_output`.
Omitted or complete detail preserves the full inline response. Keep
selecting by observed names; the separate `mapping` block reports each symbol
as `mapped`, `ambiguous` or `unmapped` after checking DEX types and prototypes.
REA does not verify that the mapping belongs to the APK build.
The provider requires a separately supplied headless JADX MCP JAR 0.7.1 and
Java 17 or newer. `REA_JADX_MCP_JAR` selects the absolute JAR path; `JAVA_HOME`
can select Java. Use `rea doctor --provider jadx --json` to check the JADX
@@ -49,11 +28,8 @@ stdio. A full JDK, including `jdk.compiler`, is required on every host; Windows
verification coverage. Calls use one worker and provider deadlines; optional
`REA_JADX_HEAP_MIB` selects the JVM heap and `REA_JADX_OPERATION_TIMEOUT_MS`
the per-operation deadline (integer milliseconds from 1 to 2147483647, default
120000). To change that deadline on a running MCP server, set
`jadx.operation_timeout_ms` through `update_analysis_settings`, then retry after
work and cleanup settle. Newly accepted operations use the override; accepted
work keeps its budget. Changing the launch environment requires a new process.
The MCP client's tool timeout must also allow the selected deadline. The APK is never launched,
120000; a change applies on the next process start; the MCP client's tool
timeout must also allow it). The APK is never launched,
and no emulator is required.
Manifest entries and decompiled text are static observations, not execution or
@@ -11,18 +11,18 @@ main and an older release expose the same arguments. Tool-name prefixes belong
to the client: find the canonical REA operation in its actual tool listing and
server association before concluding that it is absent. A shared prefix alone
does not establish which server an approval wildcard covers. See
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/getting-started/client-configuration.md#client-tool-names-and-discovery).
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/client-configuration.md#client-tool-names-and-discovery).
## Choose the remedy from the observed failure
| Observation | Meaning and next step |
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| No REA tools in the active session | If REA was installed as a client plugin, follow the plugin section below. Otherwise inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
| MCP connection dropped or REA could not start | If the connection drops, retry that call once only when it is safe to repeat or known not to have started. If it may already have changed the target, or the same failure returns, tell the user REA's connection failed and ask them to restart the MCP client. If REA could not start its connection, retry once, then tell the user and ask them to restart the MCP client. Do not retry a startup failure that returns the same unchanged result. Run `rea doctor` when REA could not start or tools stay missing after restart. Doctor cannot restore a live connection by itself. |
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. Request `{"detail": "complete"}` only for provider diagnostics. |
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. |
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
If the client cannot launch its shell/sandbox helper or read the installed skill,
@@ -116,32 +116,6 @@ is not a recommendation to change it. Do not invent a setting. Host memory and
- `invalid_request`: correct the reported fields from the current schema. Do
not repeat the unchanged call.
## REA installed as a client plugin
A Claude Code or Codex plugin owns its `rea` server and this skill. Doctor's
`--client` and `--skill` checks and `rea setup` read and write only
setup-owned state, so for a plugin-only install they report `missing`, and
applying setup would add a second `rea` server. If this skill was loaded from
a plugin, or the client lists a `rea` plugin, do not prepare a setup plan for
missing tools. Instead:
- Check that the plugin is installed and enabled with the client's own
listing: `claude plugin list` and `/mcp` in Claude Code, `codex plugin list`
in Codex.
- Restart or reconnect the client, then read its launch error for the `rea`
server if tools are still absent. The plugin starts its bundled runtime; a missing build or version mismatch
requires a reinstall, or a fresh build when developing from source.
- Check analysis engines through the connected server's
`binary_session.result.tool_availability`. `rea doctor --provider <id> --json`
from a shell reports that shell's environment, which can differ from the
environment the client gave the plugin server. If you still run it through
the package runner, use the version pinned in the plugin manifest's server
entry (`rea-agents@<version>`), not `@latest`, so it matches the server the
plugin starts.
Use the setup repair below only when the user wants a setup-owned registration
instead of the plugin, and remove or disable the plugin first.
## Repair registration when needed
For Codex:
@@ -154,12 +128,9 @@ rea setup --client codex --dry-run --json
Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
`cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`,
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `trae`, `grok_build`, `omp`, `pi`,
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, `freebuff`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
For unsupported clients, use manual stdio registration with the selected
installation’s absolute runtime and entry script, or continue through the CLI.
Acquire packages from a trusted directory outside the target project. Keep a
package-runner installation and its runtime available while registrations use it.
For unsupported clients, use version-pinned manual stdio registration or CLI.
Show the plan's exact paths, backups, changes and bundled-skill replacement.
Obtain approval for writes when that concrete scope is not already authorized,
@@ -204,17 +175,6 @@ writing. Managed policy can constrain client settings. Consult the
for the installed client's supported settings. This is an optional recovery
choice, not a prerequisite for every REA task.
## Repair an editable REA limit
If `details.limit` names an editable setting and its bounds, call
`update_analysis_settings` directly with a `changes` entry containing `setting`,
`action: "set"` and `value`. `inspect_analysis_settings` is optional when those
facts are already present. Retry the same input after work and cleanup settle.
Runtime defaults affect new work only; explicit per-call arguments take
precedence and must be corrected directly when selected. Reset restores launch
values. Updates survive target close/open but disappear on REA restart. They do
not edit host deadlines, response buffers, registrations or fixed limits.
## Continue through an installed CLI
Check the command environment only if the launcher fails:
@@ -1,438 +0,0 @@
# Desktop app environments from a request
Use this when someone asks for an environment of a Linux desktop app with a
task, such as "Build me an Inkscape environment where the agent changes a
poster's headline and saves it". You are the building agent: you drive the
app on a build computer through REA's MCP tools, write the task, export the
bundle and witness it in a fresh computer. REA runs no model. The tools are
MCP-only and need Docker, `REA_AGENT_ENV_COMMAND` and
`REA_BUILD_COMPUTER_REA_ARCHIVE` where the REA server starts; when a start
reports one missing, tell the person what to set and stop.
For the whole build:
- Drive the app only with `act_on_build_computer`. Never script it, run
commands in the computer or write its files another way.
- Ask the upfront questions once, then work on your own. Pause only when
blocked.
- The seed's content is synthetic. Never put real personal data in it.
- Close every computer you start, on success and on failure.
- Note the time you start; the report gives the build time.
## Ask once, upfront
In one message, ask only what the request leaves open, and keep the list for
the report:
1. The app and the task, if either is unclear: what changes, in which
document.
2. Whether they have a starting file of their own; otherwise you make one.
3. Whether they want solver runs after the witness: how many (the budget)
and with which model.
4. If so, whether they want to review the task before those runs.
## When you are blocked
Stop, say what you need and wait only for:
- a licence the app or its installer asks someone to accept: never accept it
yourself; name it and where to read it, and go on only once the person
says they accept it;
- credentials: they may only be typed as handles you never see, and the
build tools have none yet, so say the app needs a sign-in and stop. Never
ask for a password in chat;
- a signed-in state in the seed: ask before seeding one;
- an install this flow cannot do, below.
Decide everything else yourself and note it in the report.
## Choose the install
The computer is Ubuntu 24.04 (noble) on this host's architecture only. Use
the first source that has the app:
| Source | `install` |
| ---------------------------- | --------------------------------------------------------------------------------- |
| Ubuntu archive | `{"kind": "archive", "package": "inkscape"}` |
| PPA or vendor apt repository | `{"kind": "apt_repository", "package", "uri", "suite", "components", "key_file"}` |
| `.deb` by URL | `{"kind": "package_file", "url", "sha256", "architecture", "dependencies"}` |
- A PPA's `uri` is `https://ppa.launchpadcontent.net/OWNER/NAME/ubuntu`,
`suite` `noble`, `components` `["main"]`. `key_file` is the repository's
signing key saved on this host; for a PPA, fetch it from
keyserver.ubuntu.com by the fingerprint its Launchpad page shows.
- For a `.deb`, ask the requester for its URL and SHA-256; do not fetch a
vendor's download page yourself. List its `Depends` as `dependencies`:
it installs with the network off.
- Flatpak, AppImage and snaps are not supported yet: say so and stop. An
archive package that only installs a snap, such as noble's `firefox`,
counts as a snap. A package with no build for this host's architecture
(some PPAs are amd64 only) cannot be installed either.
- `desktop_entry` is the package's `.desktop` file name under
`/usr/share/applications` without `.desktop`, such as
`org.inkscape.Inkscape`. Add `window_class` when you know it; Inkscape's
is `Inkscape`. AbiWord's entry is `abiword` and its class `Abiword`.
- To find the entry, read the package's file list: its page on
packages.ubuntu.com, or `apt-get download PKG && dpkg -c PKG_*.deb` where
apt runs. The entry may ship in a companion `-data` or `-common` package,
as Planner's in `planner-data`: read that package's list too. The entry's `StartupWMClass` may not be the window's real
class, so do not take `window_class` from it: MuseScore 3's says
`mscore3`, its window's is `MuseScore3`. The export uses the class the
capture saw.
## Drive the app
- Pass `computer_id` and the action's fields at the top level: click
`{x, y, count}`, scroll `{x, y, direction, amount}`, a screenshot region
`[x0, y0, x1, y1]`. The rest are in
[the build-computer guide](https://github.com/morluto/rea/blob/main/docs/desktop-build-computer.md).
- Take a `screenshot` before pointing: points are pixels of the latest
frame. If a response is too large, ask for a `region`, or `jpeg` with a
lower `quality`. A region crops the latest frame and becomes it: take a
full screenshot before pointing again.
- A drawn caret does not prove keyboard focus. After using a toolbar
control, click into the document, then type a few characters and look
before a long text.
- Confirm a dialog with `key` Return, not a trailing `\n` in `type`.
- An action's own screenshot is taken at once. After opening a menu, a
dialog or a window, look with
`{"action": "wait", "ms": 500, "screenshot": true}`.
- On a computer started from a request the app is not open at first, and
`start_build_capture` opens it: never open it yourself before that. Do not
close it while capturing; the capture ends when the app exits. When the
app is closed at any other time, open it from Applications → Other → the
environment's name.
- Prefer shortcuts and typing to long pointer paths: `key` takes `keys`, an
array holding one chord's keys, such as
`{"action": "key", "keys": ["Control", "s"]}`, and `type` turns `\n` into
Enter.
- Before `type`, check in a screenshot that a text cursor or a focused
field is there. In apps with single-key shortcuts, such as Inkscape, text
typed without one runs as commands.
- Save As differs by app. Check the dialog's current folder first: most
open in the home, so click Documents in the sidebar, but Scribus opens in
`~/Documents` and its sidebar has no Documents entry. Type only the file
name. A dialog taller than the 1280×800 screen hides its buttons:
maximize it from its title bar, or confirm it with `key` Return, as when
HomeBank's Save button is off-screen.
- Answer the app's save questions, such as keeping the format, then check
the saved path in the title or status bar, or the dialog's folder when
the app shows only the name, and that it shows no unsaved mark, before
you mark a checkpoint or check. Where the title never shows one, as in
HomeBank, a greyed Save or Revert, or the file reopened from recent
files, confirms the save.
- Qt spin boxes change their value on scroll: move the pointer off them
before you scroll.
- What the app saved under the home before `seed`, such as a document in
`~/Documents` or its settings, is part of the seed. A file you move to the
Trash is left out, and the export lists it in `limitations`. Trash a stray
file rather than starting over.
- Dismiss first-run dialogs, such as a welcome screen, before `seed`. A
choice the app saves only later, such as Inkscape's "show this every
time", may not reach the seed, so expect the dialog in the witness too.
- Give what the task changes a stable name where the app has one, such as an
id in Inkscape's Object Properties, so a check can select it.
- Expect apps to split text into runs: Writer's revision ids split an
edited paragraph into spans and may give it an automatic style such as
`P1`, and AbiWord wraps each insertion in `<c>`. Check such text whole
with `text_equals` (see Write the task), and find the paragraph by
something the edit keeps, such as AbiWord's `xid`.
## Build the seed and the edit
1. `start_build_computer` with
`{"source": {"kind": "request", "desktop_entry": ..., "install": ...}}`.
The first start of an app builds its image, and starts at the same time
on this host wait for each other; either can take minutes. A failed
start returns the build's error: fix the request, or stop when the
install cannot work, and close any `computer_id` the error names.
2. `start_build_capture`. It returns once the app's window is open. A
`package_lists_error` may stop the capture finding the package's
sources; the export says when that matters. When the app exits before
its window opens, the error gives its exit code and the end of its
stderr, which may name what it lacks, such as a Java runtime: try
another install source, or stop as for an install this flow cannot do.
- REA counts a file as the app's document only when the app's window
title shows its file name or `~/` path. When the title shows something
else, such as MuseScore's score title, pass the path you will save the
document at in `state_paths`, relative to the home:
`{"computer_id": ..., "state_paths": ["Documents/MuseScore3/Scores/Morning_Song.mscz"]}`.
It need not exist yet. Add the app's settings folder when its name
differs from the app's, such as `.config/MuseScore`, so the seed keeps
the dismissed first-run dialogs. Do not pass the document as a launch
argument to make REA see it: `state_paths` is the way.
- An app whose document is a folder of pages, such as a Zim notebook
(`Notebooks/Notes`, a `.txt` file per page), never shows a file in its
title: pass the folder in `state_paths`. Every page in it is then a
document, so the bundle opens none at launch and the app starts where
it keeps its place; check the page the task changes.
3. Make the seed through the app and save it from the app under
`~/Documents`: build the document in the app, or write its source where
the app edits its format as text, then save it again from the app. Make
it look real but synthetic: the thing the task changes, plus a few things
the change must leave alone. Keep it to one document: the bundle's app
opens the seed's document at launch only when there is exactly one, and
otherwise starts on its own screen. For a starting file of the requester's,
read it here and rebuild it in the app with synthetic data in place of
personal data; REA cannot copy a file into the computer.
- An app that keeps its own data and saves after every change, such as
Osmo's contacts in `~/.local/share/osmo/contacts_records.xml`, has no
document: that state is the seed. Build it in the app and mark `seed`
once the app has written. To see whether it has, mark a throwaway
checkpoint, such as `probe-1`, and read its `changed_files`: up to ten
files that changed since the checkpoint before, those under
`state_paths` first, then those the app held open, caches last, with
`more_changed_files` counting the rest.
- The app's own compressed format works when it is gzip, such as
Gnumeric's `.gnumeric`, Dia's `.dia` or Inkscape's `.svgz`: REA reads
the document inside, and checks point into it as into the same
document uncompressed. A format REA reads as binary, such as one
compressed with xz, has no view: save in a format the app offers that
REA reads, then reopen the file once and check that a plain Ctrl+S
keeps that format.
4. Only once the seed is saved, `mark_build_checkpoint` with name `seed`.
5. Make the change the task asks for, as a user would, and save it.
6. Only once the change is saved, `mark_build_checkpoint` with name `end`.
7. `stop_build_capture`. Read `draft.files` (each file that changed, with
its net change and its `values`), `draft.task.checks` (suggested
checks), `draft.guards`, `draft.notes` and `limitations`. When `draft`
is null, read `draft_refusal` first: a missing `seed` or `end`
checkpoint means capturing again on a new build computer, while a failed
draft command names its own cause to fix. When `draft.files` is empty,
`draft.notes` says why:
- `no_change`: nothing was saved between `seed` and `end`. Capture again
on a new build computer, and save before marking `end`.
- `unowned_changes` with `no_document`: the app saved your document
where REA did not count it as the app's. Its `files` list what changed,
with their roles; the document is among them. Close the computer and
capture again on a new one with that path in `state_paths`; the export
refuses a check on a file outside the owned paths.
- `uncheckable_changes`: the app saved the change in dconf, as gedit does
its preferences, and no check reads dconf. Saving again does not help:
choose a change the app saves in a file, and capture again on a new
build computer.
- `unreadable_changes`: a dconf database changed but REA could not read
it; `error` says why. This is not evidence the app saved there: report
the error rather than changing the task.
A note with a `file` says why that file's suggestions are thin:
- `by_position`: only position names what changed; its `elements` lack
a naming attribute. Check an ancestor's text, or give the element a
name in the app, such as an id, and capture again.
- `view_state_only`: every change is view state, a renumbered id or a
time the app stamps, and its `pointers` are the checks kept. The edit
was likely not saved: capture again and save it before `end`.
- `long_text`: the element's text is too long to check whole; check a
phrase in it with `text_contains`.
- `cut_view` or `unreadable_view`: REA's view holds only part of the
file, or none a check can read, such as a binary format (step 3).
## Write the task
Write `{"name", "prompt", "checks"}` from the draft and what you did.
- `name`: lowercase letters, digits and hyphens, not `smoke`.
- `prompt`: what the requester wants, in their words and the app's terms,
naming the document as a user would and ending with saving it: "Change the
poster's headline in Documents/poster.svg to Summer Bake Sale and save
it." Leave saving out for an app that saves on its own, after every change
or a few seconds after the last one, such as Osmo or Gnote: grading waits
until the app's files have held for 3 s, so the solving agent need not
wait for the save. A save more than 3 s after the agent stops is still
missed; name such an app in the report. Never say how the file stores it,
and never give the checks.
- Checks are file-based: content checks that prove the edit, never only that
a file exists or changed. Start from the suggestions; keep those that say
what the user did and drop what follows from how the app saves.
- Compare exact values, so a typo or a wrong number fails.
- Read each file's `values` before writing checks: every value the change
added, changed and removed, with `before` and `after` (a scalar, or an
element as compact XML, cut at 600 characters) at the pointer a check
names it by, so copy the pointer and the `after` value into a check.
Only XML files, archives, JSON and key files have values: a text file's
or SQLite database's `change` already lists its lines and rows.
`refers_to` says what a number stands for, such as `category="44"`
naming `<cat key="44" name="Groceries"/>`; use it for the prompt's words.
Skip entries whose `note` is `by_position` (no check can use that
pointer) or noise (`view_state`, `renumbered`, `volatile`,
`timestamp`, `upkeep`). `values_cut` means a limit left some out.
- Select elements by what they hold, never by position: SVG text by id,
`/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0`,
or an ODF paragraph by style,
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Title]/children/0`.
A selector must match exactly one element, so give the target a unique id
or style in the seed. The draft's pointers already do this: a name-like
attribute such as Scribus's `ANNAME` before a coordinate, and two
attributes together where one is not enough, as a spreadsheet cell's
`[@Col=3][@Row=2]`. Only an element's text is read by position, as
`children/0`.
- When an app may split an element's text into runs, check it whole:
`text_equals` (or `text_contains` for a value inside a sentence) on the
element compares all its text, however it is split. Add
`normalize_whitespace: true` when the app may reflow spaces. Example, an
AbiWord paragraph by its stable id:
`{"kind": "text_contains", "file": "Documents/memo.abw", "pointer": "/root/children/[name=section]/children/[name=p][@xid=6]", "value": "at 3:30 PM"}`.
Point a text check at the paragraph or another element the edit keeps,
never at a run such as AbiWord's `[name=c]`: a reopened document splits
the text into several runs. The draft lists only what changed; an
unchanged element is in `draft.guards` when it is a like sibling of a
changed one, and otherwise use the app's usual numbering, such as another
paragraph's `xid`, and let the export confirm it.
- In an ODS spreadsheet no attribute names a row or a cell. Name the sheet
in the seed, and check its whole text: `text_equals` on
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:spreadsheet]/children/[name=table:table][@table:name=Q1 Sales]`,
or a stretch of it with `text_contains`. Cell texts join with nothing
between, so a row reads `February23100`.
- For a line added to free text, such as an item on a note's list, the
draft compares the whole text with `text_equals`. This rejects longer or
negated items, but can reject a correct edit with different ordering or
spacing. Report that limitation when the task permits such variations.
- Some apps keep text in an attribute, such as Scribus's `<ITEXT CH=...>`.
`text_equals` cannot join it, so check the attribute with `value_equals`:
`/root/children/[name=DOCUMENT]/children/[name=PAGEOBJECT][@ANNAME=Headline]/children/[name=StoryText]/children/[name=ITEXT]/attributes/CH`.
A partial edit the app restyles splits the text into several runs and
fails that check; list it in the report's limitations.
- When the draft shows no values or no checks, a scratch export can expose
`envs/<env>/seed/` only after the task has at least one valid check: it must
pass on the captured end state and fail on the seed. Before the first
successful export, use `act_on_build_computer` to open the saved file in an
available GUI text editor or viewer and inspect its actual representation
without changing it. Use the visible saved content and the task's edit to
write a check; an export refusal can also name the end state's value, such
as `is "44", not "1"`. If the file cannot be inspected or expressed by a
supported check, report the limitation instead of guessing a check.
- Where easy, add one check that the rest is untouched, such as another
text's value the edit must leave alone. `draft.guards` offers them at
their seed values, which the end holds too: a field on the nearest like
siblings, one the user sets, such as Planner's `work`, before a default
they all share; the text an addition kept; a text file's lines beside its
new one. Copy one or two into `checks`; they pass on the seed, which is
fine, since the export needs only one check to fail there.
- The draft suggests a decimal the app writes as text, such as `146.83`,
as a number with `tolerance: 0.001`; keep the tolerance for any number
you check that way.
- When the draft shows an XML document's changes as `lines`, REA read it
as text, for example because its DOCTYPE declares entities. Selectors are
impossible there, and `line_count` with `contains` only approximates a
value: `3:30` also matches `13:30`. Bound it with neighbouring text and
name the check as approximate in the report's limitations.
- Some apps save a document reloaded from disk differently from one typed
in the same session: AbiWord wraps each new insertion in its own markup.
Your build's end state hides this, which is why whole-text checks matter;
expect the witness to show the real format.
The kinds and pointer rules are in
[the desktop tasks section](environments.md#pick-the-checks).
## Export
`export_build_environment` with `task` and `directory`, the absolute path of
a new or empty folder here, such as `<home>/rea-builds/poster/bundle`. The
export seeds what was saved at `seed`, replaces the computer's host name,
`user@host` and home path (a bare user name, such as AbiWord's author
`user`, stays), and checks the task against `seed` and `end`. It refuses a task
unless at least one check fails on the seed and every check passes at the
end, and says why, such as a pointer that names nothing at the end. A
check that the rest is untouched passes on the seed too, which is fine. On
a refusal, fix the task and export again, into another folder if that one
is no longer empty. Keep
`seed_files`, `scrubbed_files`, `unscrubbed_files` and `limitations` for the
report.
The limitations name the folders the app wrote that the bundle neither owns
nor seeds, such as `~/.config/MuseScore`, so the app starts without them:
expect its first-run wizard, a tour or a "restore session?" question at every
boot. If the witness shows one, capture again on a new build computer with
those folders in `state_paths`, and dismiss the dialogs before `seed`. Lock
files the app holds while it runs, such as LibreOffice's `.lock` and
`.~lock.<name>#`, are left out of the seed and listed after "Left out of the
seed", so you need not close the document or delete them before `seed`. The
limitations also name seeded WebKit or Chromium storage and caches; when the
witness's boot is clean, list them in the report.
Keep the build computer open until the witness holds: a task fix exports
again from it.
## Witness
Witness in a fresh computer of the bundle, from the prompt, as a solving
agent would. Do not replay the build's actions or coordinates, and do not
check between stages to steer the edit: a solving agent never sees the
checks. Give every `check_build_task` the task exactly as exported.
1. `start_build_computer` with
`{"source": {"kind": "bundle", "path": "<bundle>"}}`. The app starts
with the seed's files in place and opens the seed's document when the
seed holds exactly one and the app takes a file. When the export's
`limitations` say it opens no document, open it from the prompt's path,
as a solving agent would. Take a `wait` screenshot first, since the
desktop may not have drawn yet. When the start fails because the
computer never got ready, and its error names lock files in the bundle's
seed, delete them from the bundle and start again.
2. `check_build_task` with `stage: "seed"` and `directory`, the absolute
path of a new or empty folder for the record. A check must fail.
Recording starts. Later stages take no `directory`.
3. Do the task from the prompt with `act_on_build_computer` and save.
`check_build_task` with `stage: "after_edit"`: every check must pass.
Each check waits until the app's files have held for 3 s, as the
bundle's verifier does, so an app that saves on its own needs no wait of
yours. A result with `settled: false` means the files were still
changing at the 15 s bound, and `read_cut: true` that a slow read was
still running there.
4. Make a near miss on top of it: a plausible wrong edit, such as a typo or
a wrong number, and save. `check_build_task` with `stage: "near_miss"`:
a check must fail.
5. `act_on_build_computer` with `action: "reset"`, then `check_build_task`
with `stage: "after_reset"`: a check must fail again.
6. The last stage ends the witness: `witness.record` has the verdict, the
path of `witness.json` and the recording's segments in `directory`.
`close_build_computer` on the witness computer.
A stage that does not hold ends the witness as `failed`. Read each check's
result and decide:
- `after_edit` failed though the saved file is right: the check is brittle.
Fix it, export again and witness in a new computer.
- `after_edit` failed because your edit was wrong: witness again in a new
computer; a computer is witnessed once.
- `near_miss` passed: the checks are too loose. Tighten them, export again
and witness again.
- `after_reset` passed: the reset did not restore the seed. Report it as a
limitation.
Stop after three witnesses and report what still fails.
## Solver runs
Only if the requester asked for them. If they wanted to review the task
first, show them its prompt and checks and wait for their approval. Then run
the bundle within the budget, one run at a time, with the runner
`REA_AGENT_ENV_COMMAND` names:
`agent-env run <bundle> --task <name> --agent claude-code --model <model>`.
Count runs reported `passed`. A run `scored below 1` is a miss; one reported
`failed` did not finish, so run it again if the budget allows instead of
counting it. Never run past the budget.
## Close and report
`close_build_computer` on every computer still open, also after a failure.
If a close fails, call it again; if it still fails, tell the requester what
is left.
Report:
- the bundle path and the environment name;
- the task's name and prompt;
- each check, and why it proves the edit and fails the near miss;
- each witness stage with its result, the verdict, and the paths of
`witness.json` and the recording;
- the seed's contents: `seed_files` and what each holds;
- `scrubbed_files` and `unscrubbed_files`; an unscrubbed file still names
the host or user, so it needs review before sharing. `recompressed_files`
are gzip seeds whose headers were cleared, their content unchanged: give
their count;
- limitations: the capture's, the export's and anything you could not do;
- solver runs, if any: the model, the runs and the passes;
- the build time and the questions you asked.
Never publish or share the bundle yourself.
@@ -1,391 +0,0 @@
# Agent environments from a live application
Use this when the user asks for an agent environment (an AgentEnv bundle) of a
web application: a folder `agent-env` deploys, whose MCP tools answer from what
one recording of the application observed and serve the writes that recording
confirmed. The workflow records the application once with
`capture_browser_scenario`, checks what the environment would serve, records
again if a hint names a gap, then writes the bundle with
`export_environment_bundle`. For a HAR the user already has, skip to
[export](#export-and-hand-off) through `inspect_web_network_capture`, or use
`rea environment-build <capture.har> <dir>`. For a task in a desktop
application's exported environment, read
[its section](#tasks-for-a-desktop-application-environment).
## Before recording
- Record only against an instance the user says is disposable or seeded for
this purpose: every write step creates, edits or deletes its data. If the
user has not said so, ask before any scenario that writes. REA does not
start, seed or reset applications.
- Credentials come from environment variables the user names. Declare each as a
scenario secret, `{secret_id, environment_variable}`; the variable must be set
in the environment REA's MCP server started with. Never ask for a password in
chat. Declared secrets are redacted as the capture records.
- Map the application first with `inspect_web_page` (and `capture_web_screenshot`
when layout matters): the login form's locators, and the API calls behind the
pages that list and edit the entities the user asked about.
- `inspect_web_page` cannot sign in. To see pages behind sign-in, run a short
`capture_browser_scenario` that signs in and opens them, with
`"capture": {"after_each_step": ["accessibility"]}` (`"dom"` for markup).
Read a step's tree with `inspect_analysis_view`, `view: {kind:
"browser-scenario", collection: "text", step_index, artifact:
"accessibility", offset: 0, limit: 2048}`, paging with `offset`. Capture
options are only `after_each_step`, `at_end`, `events`, `network` and
`limits`.
## Record a session an environment can learn writes from
One `capture_browser_scenario` records the whole session. Start from this
shape, with the application's own locators, paths and headers:
```json
{
"browser": { "mode": "launch", "executable_path": "/usr/bin/chromium" },
"start_url": { "url": "http://127.0.0.1:3000/login" },
"secrets": [
{ "secret_id": "admin_password", "environment_variable": "APP_PASSWORD" }
],
"capture": {
"network": {
"request_body": true,
"response_body": true,
"header_values": true
}
},
"actions": [
{
"step_id": "user",
"action": "fill",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=username]" },
"value": { "source": "literal", "value": "admin" }
},
{
"step_id": "password",
"action": "fill",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=password]" },
"value": { "source": "secret", "secret_id": "admin_password" }
},
{
"step_id": "sign-in",
"action": "press",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=password]" },
"key": "Enter"
},
{
"step_id": "signed-in",
"action": "wait_for",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "nav a[href='/settings']" },
"state": "visible"
},
{
"step_id": "folders-1",
"action": "request",
"timeout_ms": 15000,
"method": "GET",
"path": "/api/folders",
"expected_status": [200]
},
{
"step_id": "new-folder-1",
"action": "click",
"timeout_ms": 15000,
"locator": { "kind": "role", "role": "button", "name": "New folder" }
},
{
"step_id": "folder-title-1",
"action": "fill",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=title]" },
"value": { "source": "literal", "value": "Scratch folder 1" }
},
{
"step_id": "create-folder-1",
"action": "click",
"timeout_ms": 15000,
"locator": { "kind": "role", "role": "button", "name": "Create" },
"save_response": {
"method": "POST",
"path": "/api/folders",
"save": [{ "name": "folder_1", "json_pointer": "/id" }]
}
},
{
"step_id": "read-folder-1",
"action": "request",
"timeout_ms": 15000,
"method": "GET",
"path": "/api/folders/{saved:folder_1}",
"expected_status": [200]
}
]
}
```
Give each step that accepts `timeout_ms` a deadline. Without one, a step waits
until it completes or the scenario is cancelled, so a locator that never
matches hangs the capture and the client with it. The `wait_for_timeout` pause
instead uses `duration_ms` and does not accept `timeout_ms`. Match `wait_for`
locators to exactly one element.
1. Sign in through the UI: `fill` the user and password fields (the password
as a secret reference), submit, then `wait_for` an element only a signed-in
user sees.
2. Read with `request` steps: each collection the environment needs, twice
before any write, so fields that change on their own show up; then each
object a list shows by id, up to ten per collection. An object known only
from a list that leaves out fields its own read shows cannot be read by id
or updated in the environment.
3. Write the way the application's UI does: fill its forms and click its
buttons, so each request carries the fields the UI sends, including ones
the server would otherwise derive. Give the click or key press that submits
a create `save_response` to keep the new object's ID, then `goto` its page
(`/items/{saved:item_1}`) or read it with a `request` step. Never guess an
ID, and never create a probe object to learn the next one. Write with
`request` steps instead only when the UI cannot reach the operation (an
API-only resource, or a UI that does not call the JSON API), or when coverage
withholds the UI-shaped write as `unexplained-change` or `contradicted`;
that counts as one of the two re-recordings. A `request` step write that
coverage withholds that way usually lacks fields the UI sends: record it
through the UI instead. If a journey cannot be scripted, leave the write
unserved and report the limitation.
Record two numbered scratch objects per write type,
with varied free text. After each write, and before the next one, read the
object back by id and then its list: create 1, read 1, list, create 2,
read 2, list; never several writes before their reads.
4. Leave more than a second between two creates and between a create and its
first edit (`wait_for_timeout` with `duration_ms: 1100`), so lists sorted
by creation time show them in order.
5. Update both objects and read each back, changing the fields a person
would change (a name, an email address, a status, a due date) rather than
only a description: a served update accepts only the members a recorded
update stored. Run an action that turns something on or off (star,
bookmark, archive) twice in a row on one object, reading it back after
each: the second call shows whether it sets or flips the field, and the
environment serves the action only once a call has shown it. Delete both
and read each back (expect 404, or the object marked deleted) and the list.
6. Vary what the two creates send, however you write them: the first every
field the application takes, the second only what a person must give (its
title or name), with the other fields left out or empty. A served create
requires each member both creates filled in and lets a caller leave out one
a create left out or sent empty, so an agent that names only a title is not
refused for fields it had no reason to set.
About `request` steps:
- `path` is on the start URL's origin; the page must be on that origin.
- Header values are literals, declared secrets, a cookie the browser holds
(`{source: "cookie", name}`), a local storage entry
(`{source: "local_storage", key, json_pointer}`), a saved value
(`{source: "saved", name}`), or Basic credentials
(`{source: "basic_credentials", parts: [username, password]}`, sent as the
base64 of `username:password` and redacted), with an optional `prefix` such as
`"Bearer "` or `"Basic "`. Each credential part is a literal, secret, saved
value, cookie or local storage entry, so a header the frontend builds from a
login token (the username and the token) can be copied too; the password is
a secret or saved value, never a literal. Copy the
headers the application's own requests send (a CSRF header such as
`X-CSRFToken` from the `csrftoken` cookie, a bearer token). Values read from the page are redacted from records made after they
are read, not from the login response that returned them, and values
shorter than 8 characters are not redacted at all.
- `save: [{name, json_pointer, secret}]` keeps a string or number from the
step's JSON response, such as the created object's `/id`, for later steps;
`secret: true` redacts it, as for a token the API returns, so mark every
saved token, key or session value secret. This also protects the selected
source response before its body or headers are retained. Compressed or
unknown-size UI responses cannot supply saved values. On a `click` or `press`,
`save_response: {method, path, save}` keeps values from the response to the
new request observed while the step waits (`{*}` in `path` matches one segment).
Matching does not prove UI causality: choose a path distinct from polling;
multiple observed matches fail as ambiguous. The step
fails unless a 2xx JSON response arrives before its `timeout_ms`. Later
steps send a saved value as `{saved:name}` in a request `path` (and its
query) or a `goto` URL; in the `json` body, where a string that is only
`{saved:name}` keeps the saved JSON type; or as `{source: "saved", name}` in
a header or credential part.
- Reading a page value makes later bodies costlier to keep: after a CSRF
token, bodies over about 47 KiB are kept as `unavailable`. When the API
returns larger responses, raise `capture.limits`
(`max_network_body_allocation_bytes`, `max_total_event_allocation_bytes`).
- Request steps follow no redirects: a redirect fails the step, and
`expected_status` cannot name a 3xx status. Use the path the redirect names,
such as the one with the trailing slash.
- Set `expected_status` on every step, so a refused call fails its step and
the later steps report not having run, instead of recording read-backs of
nothing.
Keep it to about 60 to 160 requests. A scenario with no `request` step is
recorded exactly as before.
The capture returns its whole Evidence, every recorded body included, which is
often more than a client shows inline. Before checking coverage, confirm that
every entry of `normalized_result.steps` has `status: "completed"`; if the
client saved the result to a file, search it for `"status":"failed"` and read
that step's `error`. A failed sign-in or write cancels every step after it.
Use the result's `evidence_id` for coverage and export; do not paste the
Evidence back into another tool.
## Check coverage
`inspect_analysis_view` with `source: {kind: "retained-evidence",
evidence_id}` (the capture's id) and `view: {kind: "environment-coverage",
template_rules: ["after-uid-literal", "opaque-token", "sibling-variance",
"prefix-variance"]}`. It reports read tools
answered from the state or replayed (`reads.recorded_reasons`), the served write
tools, and each withheld write with its `reason` and `hint`. It does not apply
the serve-time catalog and state size checks; the export reports those.
Re-record a withheld write at most twice, and only for a recording gap. Stop at
once when the reason is application behaviour, and report the write as
withheld with its hint:
| Reason | What to do |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
| `no-observation` | Re-record: read the entity back by id after the write; read it first for an update or delete. |
| `unconfirmed` | Re-record: read the entity and its list after the write. |
| `incomplete-entities` | Re-record: read the entity through every view that lists it. |
| `unexplained-change` | Re-record once with a second, different call of the same write. |
| `no-call` | Re-record once with a write that succeeds. |
| `side-effect` | Stop: the application changes other entities. |
| `contradicted` | Stop: a later read disagrees with the modelled effect. |
| `body-unusable` | Re-record a UI write once with `request` steps (its body was not exposed); else stop. |
Use new scratch names on each attempt, and delete what an earlier attempt
created before recording again (through the UI, or a `request` step with the
id that capture shows): the environment starts from what the recording first
reads, so a leftover object becomes seeded data that write tasks then find.
New browser scenario captures include request times, so a fully timed capture
can learn a write clock and serve updates that stamp modification times. Older
captures or ones missing a request start keep record order and learn no clock.
When coverage reports that limitation, record again with the current version
before concluding that the timestamp change cannot be served.
## Export and hand off
`export_environment_bundle` with `source: {kind: "retained-evidence",
evidence_id}` (or `{kind: "evidence-file", path}`), an absolute
`output_directory` that does not exist yet and whose parent does, and a `name`
(required when the application runs on `localhost` or an IP address). It returns
the coverage again, `tool_count`, the written files and warnings; `dry_run:
true` reports without writing. The bundle installs `rea-agents` of this REA's
version from the registry; when the user runs an unreleased REA build, pass the
`package_tarball` they give you (a packed `rea-agents` tarball) instead.
Tell the user how many tools answer from the state, which writes are served and
which are withheld and why, and the run command (`agent-env run <folder>`). Say
that the bundle ships the recorded responses and the capture's local paths
(for a browser scenario, the browser executable's path), and
list any `credential-named-members` warning pointers for review. Never publish
or share the bundle yourself.
See
[agent environments](https://github.com/morluto/rea/blob/main/docs/agent-environments.md)
for what the state, replay and the bundle establish, and
[evidence workflows](evidence-workflows.md) for the manifest and state views.
## Tasks for a desktop application environment
Use this when the user has a folder from `rea capture-desktop-application` and
wants a task for the environment `rea export-desktop-application` makes from
it: a prompt for a computer-use agent and checks that grade what the agent
left in the app's files. You write the prompt and the checks; REA validates
the task against the capture and exports it. These commands are CLI only. To
build the environment and its task from a request instead, follow
[desktop builds](desktop-build.md).
### Describe the change
Run `rea describe-desktop-application-task CAPTURE --format json`. Read:
- `owned_paths`: the only paths whose files the app serves, so the only files
a check can read.
- `files`: each file there whose state differs between the capture's first
and last version, with its `role`, the `baseline` and `final` versions and
the net `change`. Text changes list each added and removed line once, and
`line_counts` gives lines found at both ends whose count changed.
- `values` in each of those files: what the change added, changed and
removed, `before` and `after`, at the pointer a check names it by, with
`refers_to` naming the element a key such as `category="44"` stands for.
Skip entries noted `by_position` or as noise. Only XML files, archives,
JSON and key files have values: a text file's or database's change
already lists its lines and rows.
- `task`: a draft whose `checks` are suggestions from the net change of the
documents the user worked on, or of the app's own state when no document
changed.
- `guards`: optional checks that the rest kept its seed values, such as a
like sibling's field or a text file's neighbouring lines; add one or two
to show the rest is untouched.
- `notes`: why `files` is empty, or why a file's suggestions are thin, such
as `by_position` when no attribute names the changed element.
### Write the prompt
Say what the user wants done, in their words and the app's terms: "Add buy
milk to my to-do list and save it". Never say how the file stores it: no
paths the user would not name, line formats, XML, keys or SQL, and never the
checks. Replace the draft's placeholder; the export refuses it.
### Pick the checks
Checks must express only the observed change:
- Keep a suggestion only if it says what the user did. Drop those that follow
from how the app saves: a format rewrite, renumbered lines, counters. A
LibreCAD drawing gaining four holes was rewritten from DXF R12 to 2007, and
the draft suggests counts of `SECTION`, `ENDSEC` and `CIRCLE`; only
`CIRCLE` equals 9 is the user's change.
- Never check unchanged content, whole files, or values the app stamps:
times, ids it assigns, window sizes. REA refuses ISO 8601 and `asctime`
date-times (`Thu Nov 5 09:20:31 2026`); the
rest is your judgment.
- Prefer a few robust checks: whole lines over positions, rows by meaningful
columns over generated ids, `at_least` when the user may add more.
Kinds: `file_present` and `file_absent`; `lines_present` and `lines_absent`
(`lines`); `line_count` (`exact` or `contains`, and `equals`, `at_least` or
`at_most`); `value_equals` and `value_absent` (a JSON `pointer` into the view,
such as `/groups/GROUP/KEY` for a key file); `text_equals` and
`text_contains` (an XML element's whole text at a `pointer`, optional
`normalize_whitespace`); `row_present` and `row_absent` (`table`, `where` by
column) and `row_count`. Each names its `file` as a path under the home. A
document saved as a zip archive, such as FreeCAD's `.FCStd`, reads as its
entries: point into an entry's view, and name array members by selector
rather than position, as
`/entries/[name=Document.xml]/view/root/children/[name=Object][@name=Hole]`,
so the check survives an app version that writes them in another order. A
gzip-compressed document, such as Gnumeric's `.gnumeric`, reads as the
document inside: point into it as into the same document uncompressed. Give
`value_equals` a `tolerance` to compare a number the app writes as text, such
as `"5.0000000000000000"`, with a number. Check what the user wrote in a
document with `text_equals` on its paragraph element, not `value_equals` on a
text node: apps such as AbiWord and Writer split a paragraph into runs as they
save, so its first text node holds only part of it; use `text_contains` for a
value within a sentence, such as a time. The
[desktop application tasks](https://github.com/morluto/rea/blob/main/docs/desktop-application-tasks.md)
guide has the exact rules.
### Export the task
Write the task as JSON with a `name`, the `prompt` and the `checks`, then run
`rea export-desktop-application CAPTURE DIRECTORY --task TASK.json`. REA keeps
the task only when its checks fail on the seed and all pass on the captured
end state. A refusal names the check: one that fails on the end state does not
match what the user left; "every check already passes on the seed" means no
check captures the change.
When the export refuses an application that no apt repository installs, ask
the user for the `.deb` file they installed it from and capture with
`--package-file`, then ask where the computer may download that file and pass
it as `--package-url`. Never fetch a vendor's download page yourself: such
sites often refuse automated downloads, and the copy and its licence are the
user's to choose.
Tell the user the task's name, prompt and checks, and how to measure a solve
rate: run `agent-env run DIRECTORY --task NAME --agent claude-code` with
`--model MODEL` three times and count the runs reported `passed`. Do not run
agents unless asked. The bundle's seed is the user's real data: keep it local.
@@ -11,44 +11,6 @@ returned inline. When complete `inspect_binary_layout` or
`inspect_analysis_view` for a summary, one section/symbol/module, or a stable
page instead of repeating the producer.
For any logical Evidence source that advertises retained references, pass
`{kind: "retained-evidence", evidence_id}` instead of copying the complete
record back into a call. The same contract covers individual, optional, array
and nested sources across native, process, JavaScript, managed, mobile and WASM
workflows. Sources can independently mix inline records and exact references.
Resolution preserves the source's identity, provenance, coverage, partial facts
and unknowns; it neither reruns analysis nor makes incomplete facts complete.
Keep domain role and identity requirements: a native dossier cannot substitute
for a managed member observation, for example.
When starting from a summary or inspection page, use its
`normalized_result.parent_evidence_id` to reference the complete source.
The view's own `evidence_id` names
only that view. `compare_artifacts` explicitly supports artifact summary inputs
and resolves their parents; do not assume every consumer does. When a workflow
follows source links, keep those linked records available too.
MCP references belong to the current connection's ledger. If the ID is known,
pass it directly; no discovery call or provider setup is needed. If it is missing,
import the exported bundle or supply complete inline Evidence. Do not retry an
unchanged reference or rerun a producer merely to move an existing record.
Export before closing the session when the source must survive it. For CLI JSON
workflows that advertise `--evidence-bundle`, the same reference JSON resolves
against one explicitly selected authenticated bundle:
```bash
rea trace-application-feature request.json --evidence-bundle /absolute/path/evidence.json --json
```
This includes JavaScript traces/comparisons and runtime reconciliation, managed
reconstruction/native verification/graph projection, Android/Apple/Harmony graph
projection, WASM interface comparison and retained analysis views. The managed
member comparison CLI still takes two assembly paths; process comparison takes
two capture Evidence JSON paths. Check command help for the installed version.
An input bundle grants no implicit file lookup: `evidence-file` and historical
graph file selectors are accepted only by contracts advertising them. See the
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/getting-started/cli.md#import-export-and-compare-evidence).
Completed `inspect_artifact` inventory also supports a summary and
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
Omit `logical_paths` to page all available occurrences; `[]` selects none.
@@ -73,14 +35,6 @@ Parent Evidence ID plus source index preserves same-address
alternatives. Follow `coverage.next_offset` only as far as the question needs;
exhausting retained rows does not establish complete program discovery.
`batch_decompile` returns every body unless you select `detail: "summary"` when
the connected tool advertises it. Use its parent Evidence ID with
`view: {kind: "batch-decompile", collection: "items"}` for body-free rows and
exact `where.indices`, `addresses`, `procedure_addresses` or `statuses`. Add
`include_pseudocode: true` for several chosen bodies, or use
`collection: "pseudocode"` with one row's `index` to page a large body. Batch
limitations aggregate every item, so do not attribute them to one successful row.
Use `record_unknown` to track unresolved questions in the user's investigation
and name the authority or environment still required. Supply supporting and
contradicting evidence IDs. It updates the session registry directly.
@@ -92,22 +46,18 @@ Use comparisons with complete, compatible page sets when claiming equivalence
or absence:
- `compare_artifacts` compares inventory pages by occurrence path, content,
metadata, and graph relations. Name each side as
`{kind: "retained-evidence", evidence_id}` (an `inspect_artifact` result,
its nested inventory or its summary) or
`{kind: "evidence-file", path, evidence_id?}` instead of resending Evidence.
metadata, and graph relations.
- `compare_functions` compares explicit function Evidence, not fuzzy
whole-binary matches. It rejects an `analyze_function` summary reply; pass
the summary's `parent_evidence_id` as a `retained-evidence` reference or
the complete dossier.
- `compare_bundles` compares canonical bundle membership and unknown history.
- `find_changed_behavior` combines existing comparisons; static differences
remain candidates, not causal proof. Its `comparisons` accept exact references.
remain candidates, not causal proof.
- `build_call_path` needs explicit function dossiers from one artifact/provider;
pass complete parent IDs in `functions`. An incomplete frontier makes absence
unknown.
an incomplete frontier makes absence unknown.
- `correlate_static_and_runtime` uses explicit mappings; matching patterns are
hypotheses, not causality. Both comparison arrays accept exact references.
hypotheses, not causality.
- `verify_reconstruction` evaluates a finite typed specification. A pass covers
only declared comparable claims, not global equivalence.
@@ -156,13 +106,6 @@ each plan states why in its `reason`. It replays the capture instead with
`--replay-only` or when the state is unavailable (not derivable, or a catalog
over the discovery budget), and its stopped document's `state` says which
applied.
Use `view: {kind: "environment-coverage"}` with the same rules to see what an
environment built from a capture would serve: read tools answered from the
state or replayed with their reason, and each write tool served or withheld
with a hint about what to record next; it does not apply the serve-time
catalog and state size checks. `rea environment-build <capture.har> <dir>`
inspects, projects and exports in one step and prints the same coverage, or
`null` when the bundle serves replay.
For a PE/ELF export question, `inspect_native_exports` accepts `detail: "summary"`
and retains the complete table. Query `inspect_analysis_view` with
@@ -6,7 +6,7 @@ For captured or bundled JavaScript that needs readable source recovery, use
`additional_paths` when advertised, then analyze the returned `analysis_input`
directory together. Each input retains its own snapshot, digest and byte-range
provenance; inputs and recovered code are never executed. Recovery requires a
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/guides/javascript/javascript-recovery.md).
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
extracted tree. When the question concerns known source paths and the connected
@@ -24,13 +24,9 @@ not an unfiltered request. `detail: "summary"` reduces delivery
and does not by itself reduce analysis work.
ASAR header/index admission defaults to 16 MiB / 16777216 bytes;
an admission error names `asar.max_header_bytes` and its bounds (at most
256 MiB). Call `update_analysis_settings` with that setting, `action: "set"`
and enough bytes, then retry the same full-archive request. No inspection or
REA restart is required. `REA_ASAR_MAX_HEADER_BYTES` seeds future REA processes.
`source_paths`, `detail`, and `max_heap_mb` do not bypass or raise header admission.
Runtime JavaScript timeout/heap defaults are also editable; explicit per-call
`analysis_timeout_ms` and `max_heap_mb` take precedence.
`REA_ASAR_MAX_HEADER_BYTES` (at most 256 MiB, set where REA starts) changes it.
`source_paths`, `detail`, and `max_heap_mb` do not bypass or raise it. Or
analyze an extracted directory.
Exact `#alias` specifiers resolve through the importer's
enclosing package `imports` map to package-local files; nested `node_modules`
@@ -78,15 +74,6 @@ classification, and unavailable sources never establish route reachability or
the complete application route denominator. See the
[route inventory guide](https://github.com/morluto/rea/blob/main/docs/javascript-route-inventory.md).
For captured endpoint observations and inference facts, `analyze_web_bundle`
also accepts `view: {kind: "web-bundle", collection: "endpoints", limit: 32}`
with `detail: "summary"`. Read later `endpoints`, `inferences`, or `summary`
pages from the retained parent. Filter before requesting detail; retain
`source_index` together with parent ID and collection, and request `location`
or `basis` when tracing sources. Page exhaustion does not mean producer
completeness: check `completeness`, structured `source_unknowns`, and limitations.
Endpoint candidates remain inferences rather than confirmed network behavior.
## Read a module without repeating analysis
Use the actual returned parent ID, on the same connection, as `source.evidence_id`.
@@ -169,10 +156,7 @@ leaving it, and `both` when the question needs both. Trace traversal includes
containment relationships, so even a narrow seed can expand many nodes; a
bidirectional trace can reach unrelated sibling modules. A module item or page
is preferable when its observations already answer the question. Do not treat
all reached nodes as participants in the feature. For a broad trace, pass
`view: {"kind": "feature-trace", "collection": "paths"}` (or `summary`) to
retain the complete trace and read terminal paths first; page `nodes` or `edges`
with exact `where.node_ids` through `inspect_analysis_view` as needed.
all reached nodes as participants in the feature.
Reuse retained parent Evidence when the server advertises it, or supply complete
Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest
@@ -239,19 +223,10 @@ This notation is
a template: replace it with the actual returned ID. Versions before 4.1.0 accept only full
inline Evidence. Use the exact ID
returned by the producer on the same MCP connection. Resolution does not run
analysis or select a provider; findings remain inline. For summary or inspection
replies, select the complete `normalized_result.parent_evidence_id`. Native
observation arrays and runtime reconciliation's `static_layers[].analysis` and
`runtime_observations[]` also accept exact references where advertised. Each
source can independently be inline or retained, preserving identity, coverage
and unknowns. CLI workflows advertising `--evidence-bundle` resolve these inputs
from the explicitly selected bundle. `close_binary` releases the engine while
retaining references and complete Unknown history on the same MCP connection.
`reset_investigation` explicitly clears them; optional `save_path` checkpoints the
complete bundle before clearing. Import that bundle in another connection to
restore its original references. Retention is bounded, in-memory and never silently
evicted; `binary_session.investigation` reports the budget. A missing reference
includes its ID and recovery guidance.
analysis or select a provider; findings remain inline. Native observation
arrays still take full Evidence. `close_binary` clears retained references;
export a bundle before closing, or supply portable inline Evidence in another
connection. A missing reference includes its ID and recovery guidance.
For exact file/native metadata or processing gaps, use
`view: {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"], limit: 32}`.
@@ -1,11 +1,8 @@
# Native, managed, and packaged artifacts
For release comparisons on one MCP connection, retain `list_procedures` Evidence
from each target and reuse its exact IDs after target switches or `close_binary`.
For a handoff or new process, export the complete bundle and import it there,
including its source closure. `reset_investigation` explicitly clears retained
records; optional `save_path` checkpoints them before clearing.
`match_native_functions` pairs unique exact names and, when supplied,
For release comparisons, export `list_procedures` Evidence from each target before
closing it. Close clears session records: import both bundles after collecting the
two targets, or supply portable inline Evidence. `match_native_functions` pairs unique exact names and, when supplied,
complete observed body bytes. Reuse optional same-context `list_names`,
`procedure_info`, `read_bytes` and existing `analyze_function` dossiers as
`left_observations`/`right_observations`; an MCP `analyze_function` summary
@@ -16,20 +13,6 @@ Select columns, batch IDs and page individual facts; aliases remain unresolved.
Unpaired observations do not prove additions/removals. Existing matched dossier
pairs include inputs for `compare_functions`; otherwise analyze just chosen pairs.
Aggregate completed comparisons with `find_changed_behavior`, selecting each
comparison's exact `retained-evidence` ID instead of resending full records.
Inline comparison Evidence remains accepted. References belong to the current
server: after handoff, import the original bundle including its source closure.
Do not substitute dossier IDs or summary views. Static differences remain
behavior candidates, with runtime behavior unknown unless separately observed.
For a matching representation refusal, inspect `details.limit` and raise
`native.matching_representation_bytes` with `update_analysis_settings` before
retrying the unchanged MCP request. CLI launch seed:
`REA_NATIVE_MATCHING_REPRESENTATION_BYTES`. The default is 96 MiB; the policy
range is 1 byte to 1 GiB and does not bound process RSS. Keep the complete
inventories: splitting them changes global name/body uniqueness.
## Apple app resources
Use `open_bundle_resources` with the selected app path, then `inspect_plist`,
@@ -67,18 +50,11 @@ See [desktop installations](https://github.com/morluto/rea/blob/main/docs/deskto
Use `inspect_go_binary` for an explicit local ELF, PE or Mach-O (universal
files need `architecture`) when the
question concerns the embedded compiler version, module/dependency versions,
local replacements, build settings, or static Go string literals packed without
NUL terminators. Pass `include_string_literals: true`
(`--include-string-literals`, optional `max_string_literals` /
`--max-string-literals`) to recover bounded inferred word-aligned `(ptr, len)`
UTF-8 string slices from read-only data and page them through `go`
`string_literals` views. No `open_binary`, Go installation or
local replacements or build settings. No `open_binary`, Go installation or
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
Preserve artifact SHA-256, original file ranges, framed module bytes and every
unparsed module-text line. A missing record is an unknown, not proof of a
non-Go binary; observed metadata does not authenticate its claims, and static
`(ptr, len)` string recovery excludes compiler-inlined stack/immediate strings
without proving runtime reachability. Function,
non-Go binary; observed metadata does not authenticate its claims. Function,
source and type recovery remain a separate native-analysis question.
## Model files
@@ -149,45 +125,6 @@ table or overlay. Bound snapshots, loaded IAT contents, and export-forward
resolution remain unobserved. See
[PE import and delay-load tables](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
## Windows Shell Link shortcuts
Use `inspect_windows_shortcut` for an explicit local `.lnk` file when the
question concerns the recorded target path, arguments, working directory, icon
location, volume serial, network share names, ExtraData blocks, or Tracker
droid identifiers. No `open_binary`, Windows SDK, COM, or target execution is
required. The CLI equivalent is `inspect-windows-shortcut PATH`. Preserve
artifact SHA-256, FILETIME raw decimals, IDList type bytes, and ExtraData
signatures. Unknown IDList items and ExtraData signatures remain raw with
SHA-256; they are not guessed. Version-1 UUID MAC and timestamp values are
labelled derived. A missing optional section is observed absence of that
structure. See
[Windows Shell Link shortcuts](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
## PE TLS callbacks and load configuration
Use `inspect_pe_load_config` for an explicit local PE32/PE32+ image when the
question concerns code that runs before the entry point through TLS callbacks,
or the security cookie, SafeSEH, CFG, EH-continuation and XFG declarations in
the load configuration. No `open_binary`, Windows SDK, or target execution is
required. The CLI equivalent is `inspect-pe-load-config PATH`. Preserve artifact
SHA-256, directory and callback-array file ranges, and callback VAs as
hexadecimal strings. Fields outside the declared `Size` are `null`. Guard-flag
mitigations carry `evidence_kind: "inferred"`; enforcement remains unobserved.
See
[PE TLS callbacks and load configuration](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
## Binary hardening
Use `inspect_binary_hardening` for an explicit local ELF, PE, or Mach-O file
when the question concerns PIE or ASLR, NX, RELRO, stack-protector or FORTIFY
signs, control-flow flags, or stripped state. No `open_binary`, pwntools, or
target execution is required. The CLI equivalent is
`inspect-binary-hardening PATH`. Universal Mach-O files return one image per
slice. A missing import is `unknown`, not proof the protection was omitted.
Authenticode and Mach-O code-signature rows report presence, not validity.
`hardened_runtime` stays unknown and names `inspect_signature`. See
[binary hardening](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
## ELF note tables
Use `inspect_elf_notes` for an explicit local ELF32/ELF64 image when the
@@ -211,25 +148,6 @@ uppercase image UUIDs, and integer literals that exceed IEEE-754 range. Only
no crash result. Opened Mach-O and dSYM UUID joins remain unobserved. See
[Apple IPS crash reports](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
## Linux kernel modules
Use `inspect_kernel_module` for an explicit local ELF32/ELF64 relocatable
object when the question concerns `.modinfo` license, vermagic, alias,
depends, or parm strings, the `__versions` symbol CRCs, or an appended module
signature trailer. No `open_binary`, Linux host, or `modinfo` program is
required. The CLI equivalent is `inspect-kernel-module PATH`. Preserve
artifact SHA-256, key order, duplicate keys, CRC hex, and section file ranges.
A missing section is observed absence when section names are known, not proof
the module never carried that metadata. `.gnu.linkonce.this_module` is reported
as present or absent; `struct module` stays unknown because its layout changes
with the kernel version. `__ksymtab` records are not decoded. The PKCS#7 range
and its SHA-256 are observed; certificate, signer, and digest coverage remain
unvalidated. An absent trailer is not proof the module is unsigned in another
container. gzip, xz, and zstd whole-file containers are unsupported and name
`external_decompression` as the next probe: use an external codec utility to
decompress into a separate file, then call `inspect_kernel_module` on that file. See
[Linux kernel modules](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
## CPython pyc files
Use `inspect_python_pyc` for an explicit local CPython 3.7–3.13 `.pyc` when the
@@ -267,7 +185,7 @@ Choose `row_limit` when a finite sample is useful and keep truncation and
unspecified row order explicit. Virtual tables, shadow tables and views are
schema facts, not row-read targets. REA inspects private copies and leaves
the source files untouched; obtain a coherent quiescent database/WAL snapshot
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
## Recorded Linux crashes
@@ -282,7 +200,7 @@ The optional `include_debugger_context` facet adds mapping candidates through
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
establish current file identity; zero reported flags leave permissions unknown.
Requested unavailable context fails with setup guidance. See
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md)
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md)
for exact upstream profiles, bounds and verification coverage.
## Offline ELF layout
@@ -341,18 +259,6 @@ cross-references. Addresses and recovered pseudocode are analysis observations,
not original source. Provider unavailability and unsupported metadata remain
unknown rather than false.
For large pseudocode, request `procedure_pseudo_code` with `detail: "summary"`
on the first MCP call. It retains the complete observation and returns the
original selection, text availability and length without the body. Use
`normalized_result.parent_evidence_id` with `inspect_analysis_view` and
`view: {kind: "native", facet: "pseudocode", offset: 0, limit: 20000}`;
follow `coverage.next_offset` for later pages. Choose a smaller limit if one
page exceeds the client or transport budget. Offsets and limits count UTF-16
code units, not lines or tokens. Pages reuse the same observation without
repeating decompilation. Null and empty text remain distinct. Omitted or
`complete` detail preserves full inline delivery. Summary delivery requires a
retaining session and does not reduce the work or memory of decompilation.
For a completed `analyze_function` dossier, use `inspect_analysis_view` with
`view: {kind: "native", facet: "value_flow_operations", offset: 0, limit: 32}`
and the exact retained Evidence ID to read selected facts without reanalysis.
@@ -378,84 +284,40 @@ run, use `observe_native_calls` with explicit breakpoints and a bounded
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
runs and may change files or use the network. Hardened-runtime targets need the
`get-task-allow` entitlement.
Function breakpoints use exact LLDB runtime names: pass an `inspect_macho` export
or import `runtime_name` (`main`), not its symbol-table `name` (`_main`). REA does
not translate breakpoint names. Set `module` to the image
Function breakpoints use exact LLDB runtime names: a Darwin C `_main` static symbol
may be `main` at runtime. REA does not translate names. Set `module` to the image
file name when selecting that image only; otherwise loaded system images also
match. Inspect retained locations and module paths before attributing a call.
Use `binary_session` with no arguments to check the open target, selected
provider, and alignment. Its default summary `result.tool_availability` carries
the complete tool inventory grouped by current status. Expand each group's
`names` and inherit the inventory-level client requirements unless a group
overrides them; availability, reasons, remediation, and modes remain inline.
Provider candidates keep availability and target-support status, code, and
reason. `omitted_fields` names what the summary leaves out; `{"detail":
"complete"}` adds each unavailable candidate's `remediation`, provider
diagnostics, candidate capability sets, and tool surface, `effects`, and
`annotations`. Those arrays name true flags from fixed
vocabularies, so an omitted known flag is false. `tools/list` remains the
canonical source for full tool schemas when target or provider state changes.
provider, and alignment. Its default `result.tool_availability` carries a
complete inventory grouped by shared facts. Expand each group's `names` and
inherit the inventory-level client requirements unless a group overrides them;
availability, reasons, remediation, modes, and flags remain inline. The
`effects` and `annotations` arrays name true flags from fixed vocabularies, so an
omitted known flag is false. `tools/list` remains the canonical source for full
tool schemas when target or provider state changes.
Provider capability groups carry provider identity and ordered limitations.
Prepend each set's `shared_limitations` to a group's `limitations` to recover the
operation's full ordered list. Candidate capabilities, returned with complete
detail, describe possible providers. Top-level session capabilities describe the current route, and
operation's full ordered list. Candidate capabilities describe possible
providers. Top-level session capabilities describe the current route, and
`analysis_provider_binding` identifies the selected provider.
## Continue editable Ghidra analysis across processes
For a task spanning agent sessions, check `binary_session` after selecting
Ghidra: the current provider's `capabilities[].shared_limitations` states whether
Program checkpoints are enabled or the session is ephemeral. Successful native
queries distinguish initial analysis from an actually restored packed Program;
do not infer that restoration happened merely because checkpoint mode is enabled.
In ephemeral mode, choose a private absolute directory and set
`REA_GHIDRA_CHECKPOINT_DIR` in the environment starting REA. Restart the MCP server
after changing that launch configuration; no current tool call or runtime
analysis-setting update enables it. Stock ELF/PE/Mach-O imports on Linux/macOS
are supported; see the [checkpoint setup and limitations](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program).
With the setting enabled, normal opens restore compatible saved Programs
automatically. Initial completed analysis is saved, and `close_binary` saves
later metadata edits; no separate save/restore tool is needed. Repair a reported
save failure and retry close. Cancellation or process failure can lose edits
after the last successful save; unfinished initial analysis cannot resume.
Keep the same checkpoint configuration for later processes. Evidence/Unknown
history and immutable REA query snapshots have separate persistence workflows.
## Reuse caller-selected Ghidra analysis
When the caller supplies an expanded project, configure `REA_GHIDRA_PROJECT_PATH`
(the `.gpr` marker) and optionally `REA_GHIDRA_PROGRAM_NAME` (one root program),
then restart/reconnect before `open_binary`. CLI uses the same configuration.
REA opens only a fingerprint-bound owned copy with `-process -noanalysis -readOnly`;
original project files are not opened by the engine or saved. Database edits,
import overrides and repair scripts are unavailable in this mode. Treat original
analysis completeness/settings as unknown and prior edits as caller-project state.
The program's recorded import digest must match the target; changed project content
requires reopening for a fresh profile. See the
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#reuse-an-existing-ghidra-project).
## Ghidra startup deadline recovery
`provider_timeout` during the first Ghidra query can cover import, auto-analysis,
bridge startup, and readiness. The default 330,000 ms is configurable through
`REA_GHIDRA_STARTUP_TIMEOUT_MS`; a smaller query still needs the same cold import.
When the error names `ghidra.startup_timeout_ms`, call
`update_analysis_settings` with `action: "set"` and a larger value within the
reported bounds. Resolve retained cleanup before the next startup attempt.
No REA restart is needed; an already accepted startup keeps its original
budget. Follow the error's session-specific restart guidance, and preserve useful Evidence before closing
and reopening the target when remediation requires that lifecycle step.
For a per-function timeout, update `ghidra.decompile_timeout_seconds` and retry
the same function on the existing Program. Explicit `null` removes that
per-function deadline. Inspection is optional when the error supplies the
setting and bounds. Runtime updates disappear when REA exits; environment
changes only seed future REA processes.
Preserve useful Evidence before closing the session or restarting REA. Check
`close_binary` and resolve reported cleanup failures. Set a larger startup budget
in the environment that launches REA, then restart REA and reconnect the MCP
client before reopening the same target. Closing and reopening a binary in the
existing server keeps its original configuration and starts import again.
Environment changes in an agent's shell do not update an already running MCP
server. If its launch configuration cannot be changed, report that constraint
and the exact retry setting instead of claiming a fixed technical limit.
The client's tool-call deadline must allow startup plus the requested query;
server settings and progress notifications cannot override it. CLI analysis
@@ -470,7 +332,7 @@ and memory controls in place. Disable specific expensive analyzers only when
their missing coverage is acceptable and retain the resulting limitations.
Exported C and embedded resources remain useful independent evidence, but a
timeout does not establish complete analysis or absence of code. See the
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
## Bridge request ceiling
@@ -501,21 +363,6 @@ encoded in bounded batches.
The 25000 ms Cutter request deadline and the 256-candidate discovery budget
still have no runtime override.
## Selected native facts and call graphs
For a large `get_call_graph`, or repeated `procedure_address`,
`procedure_callees`, `procedure_callers` and `resolve_containing_procedure`
queries, select `detail: "summary"` when the first 64 recorded observations
suffice. Read further `native-observation` pages from the returned parent Evidence
ID. Graph message IDs index that page's exact-text dictionary; keep level,
source index, failed nodes, and partial/unknown relationship coverage.
Read referenced source context through `native-context` facets `overview`,
`analysis-profile`, `limitations` and `result-limitations`. Reuse inspected
context/profile values only for the same digest; until caveats are read,
interpretation beyond the returned facts remains unknown. Complete Evidence
remains retained and exportable. Small complete replies can cost fewer tokens
than additional context/page calls; count the completed task and discovery.
## Static memory and original file bytes
For Ghidra, `read_bytes` reads initialized analysis memory, including loader
@@ -543,14 +390,6 @@ loaded bytes, file mappings, relocations and the entry against the immutable
snapshot. Keep `mismatch` and `unsupported` explicit. Verified import covers the
reported static image, not runtime DOS or PC-98 hardware behavior.
For a large load-image report, select `detail: "summary"` and query its
`parent_evidence_id` with `inspect_analysis_view` kind `native-load-image`.
Choose a collection such as `relocations` or `mappings`, select only needed
fields, and page with offset/limit. Original indices preserve same-address
alternatives. Null collections remain unrecorded or inapplicable; paging does
not upgrade unsupported verification. CLI requires `--evidence_output` for
summary delivery so complete source Evidence remains accessible.
For Ghidra decompilation, `procedure_pseudo_code` also accepts `procedure: "file-offset:0x..."`; CLI `rea decompile <path> --file-offset=0x... --provider ghidra` resolves the source byte offset internally. An unresolved 0x-prefixed numeric procedure selector also falls back to a unique file mapping. Existing symbols and analysis addresses win; multiple mappings fail as ambiguous. Preserve the returned file-offset and address locations and raw selection observations. ELF virtual addresses are distinct from file offsets; never assume an arbitrary offset plus image base is an analysis address.
Complete function body ranges use inclusive ends and can contain gaps. Do not
@@ -594,11 +433,6 @@ names), coverage and per-collection counts. Page rows with
`view: {kind: "managed-members", collection: "call_edges", offset: 0, limit: 64}`
over `types`, `exported_types`, `fields`, `methods`, `member_refs`, `call_edges`,
`field_accesses` or `user_strings`.
Use exact `names` and `declaring_types` filters before paging supported
collections. For a compact method lookup, request `fields: ["name",
"declaring_type", "signature"]`; omit `body` to avoid anchors. Identity tokens,
original `source_index`, module/MVID and source coverage remain. Check
`unresolved_filter_rows` before interpreting an empty match as absence.
### .NET NativeAOT
@@ -621,7 +455,7 @@ broader metadata recovery on Linux; it requires an explicitly configured
NativeAOT JAR. These are separate routes. The built-in parser also runs on
Windows Ghidra P0; read frozen literals from its report, because
`search_strings` does not include them. See the
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-nativeaot.md)
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/ghidra-nativeaot.md)
for setup, supported layouts, and host restrictions. Preserve binary identity
and distinguish derived metadata from Ghidra database observations. Recovered
types and native pseudocode are not original C# source. Windows Ghidra P0 has
@@ -650,22 +484,12 @@ every executable in the bundle. It also lists missing or weak loads and earlier
provides them.
Use `extract_artifact` when materialized files from a supported format are needed.
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md).
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md).
It materializes all regular files into a fresh temporary directory chosen by
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
The result reports the materialized directory; callers do not choose the
destination. No separate permission grant is needed.
For classic iNES cartridges, `open_binary` recognizes the byte signature without
an executable architecture. `inspect_artifact` retains header facts and each
PRG/CHR file unit's offset, length and SHA-256; its nested inventory's
`ines.reconstruction` verifies the actual ordered region bytes against the
whole-file digest. Extraction includes the original header, optional trainer
and uninterpreted trailing bytes. These file units are not mapper/CPU bank
maps. NES 2.0, UNIF, dirty legacy headers and truncated payloads are explicitly
refused. No emulator or 6502 analysis is performed. See the
[iNES guide](https://github.com/morluto/rea/blob/main/docs/ines-containers.md).
Native DMG traversal automatically uses a read-only mount on supported macOS
hosts, with an owned temporary mount directory and cleanup. It needs no approval
flag. Unsupported hosts or mount failures remain explicit limitations; do not
@@ -678,7 +502,7 @@ temporary, not a retained export of the image's contents.
For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
`--target-format dos-com`). Inspect the load image before trusting function
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-dos.md).
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
@@ -692,34 +516,11 @@ otherwise. Big-endian ARM, Thumb-only, and Cortex-M variants are unsupported.
`0x0000..0xffff`. 65C02, 65816, cartridge containers, bank switching, and mapper
semantics are unsupported. `m68k` requires an even entry and decodes the 68040
superset of the 68000 family; Amiga/Atari containers and vector-table entry
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/providers/raw-binary.md).
Intel HEX, S-record, UF2 and DfuSe files are containers, not raw images:
`inspect_artifact` lists address regions (UF2 per familyID, DfuSe per element
with its alternate setting), and `extract_artifact` writes each region. Open an
extracted region as `raw-binary` with its start address as `entry_point`. UF2
numbering gaps leave coverage unknown; a DfuSe suffix CRC mismatch follows
`integrity_policy`. See [UF2 and DfuSe](https://github.com/morluto/rea/blob/main/docs/uf2-dfuse.md).
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
### Function annotations in Ghidra
Fresh-import sessions admit annotations; existing-project reuse is read-only.
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits. Normal close saves them when owned Program checkpoint mode is enabled; ephemeral mode discards them. CLI `annotate-native-function` follows the same setting when closing after its updated result. Select the [checkpoint workflow](#continue-editable-ghidra-analysis-across-processes) before edits that must survive a later process. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier. Ghidra rejects batch addresses that request the same name or a name another selected address holds; `raw_result.failures` gives each failure's reason.
A provider switch or different target can retire the active Ghidra database and discard
its edits, even if the new provider fails on a later call. Same-target,
same-profile reopen keeps the active database. Retained annotation Evidence
records what happened; it does not restore the edits.
For `compare_functions`, obtain `analyze_function` Evidence before and after the
edit while the MCP session is open; annotation Evidence is not a comparison
input. Export the needed Evidence before close if comparison will happen later.
Authenticated same-artifact coordinates preserve renamed entries as
`matched / same-entry`, with the canonical `entry_address` inline; name and
exact-text changes remain separate dimensions. This requires compatible
architecture, provider-owned mapping and selected-image commitments. Equal
numeric addresses alone cannot establish correspondence; inspect retained
source Evidence and coverage when the match remains ambiguous.
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier.
## Offline PE resource inventory
@@ -731,7 +532,7 @@ code pages, payload hashes and static icon-group references. Same-language icon
resolution is explicit; missing references and size mismatch remain visible.
Other payloads are opaque, and this operation does not decode images or simulate
Windows resource loading. Consult the budgets and supported tree profile in
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
## Offline PE and ELF export tables
@@ -741,15 +542,13 @@ directory or ELF dynamic symbol table on any host without opening a native
target. Ordinal-only, forwarded and hidden-version exports stay distinct;
partial or unsupported coverage means the inventory is incomplete, not that
exports are absent. Use `inspect_macho` for Mach-O. See
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
For exact core-module import/export differences, use `compare_wasm_interfaces`
with two complete inspection Evidence records (or current-session retained
references) and explicit scope. Preserve unknown types/duplicate names; static
interface equality is not runtime compatibility. CLI accepts complete inline
Evidence in `rea compare-wasm-interfaces input.json --json`, or exact references
with `--evidence-bundle /absolute/path/evidence.json`. Use the complete parent
ID when the inspection returned a summary.
Evidence in `rea compare-wasm-interfaces input.json --json`.
## Godot PCK
@@ -769,91 +568,3 @@ page cannot prove absence in unknown coverage. Retained views are historical,
not fresh filesystem checks. No decryption, Godot resource decoding, GDScript
decompilation or target execution is provided. See
[Godot PCK](https://github.com/morluto/rea/blob/main/docs/godot-pck.md).
UnityFS AssetBundle containers (`.bundle`, `.ab`, YooAsset packages and other
byte-identified files) list `CAB-*` SerializedFiles and `.resS` members with
header and node facts; none/LZ4/LZ4HC member blocks stream on demand. Bundles
declaring asset-bundle encryption keep header facts with unknown directory
coverage; LZMA member bytes retain named occurrences with unavailable hashes.
Virtual member offsets are not physical carrier locations. Format 6–8 readers
cache one decoded block and support retained filtered/paged views. No
decryption or SerializedFile object decoding is provided. See
[UnityFS](https://github.com/morluto/rea/blob/main/docs/unityfs.md).
WeChat mini-program packages (`.wxapkg` files with `0xBE`/`0xED` marks) list
members with declared names and absolute carrier offsets; member bytes are
stored verbatim. `V1MMWX` PC-client packages report `encrypted` directory
status with unknown coverage and are refused on extraction; no decryption is
attempted. See
[wxapkg](https://github.com/morluto/rea/blob/main/docs/wxapkg.md).
U-Boot legacy uImages use the same inspection/extraction workflows. Prefer a
retained summary and selected inventory views for header facts or component
IDs. Header addresses/CPU/OS are declarations; payloads remain as stored.
Multi-file components retain their aligned physical ranges; the full payload
is also materialized. Unknown name text retains its original 32 bytes. See
[U-Boot uImage](https://github.com/morluto/rea/blob/main/docs/uimage.md).
## ESP images
For an ESP32-family app or bootloader image, `open_binary` then
`inspect_artifact` with `detail: "summary"`. `root_format_metadata` holds the
observed `header` and `descriptor` (its `kind` is the app or bootloader role)
and the derived `chip`, `checksum`, `appended_sha256` and
`undescribed_trailing` ranges. Segment rows (`segment-NN.bin`) carry
`load_addr`, `classification` (`mapped`, `loaded`, `not-loaded` or `unknown`,
per ESP-IDF `should_map`/`should_load`; RAM ranges are not checked) and
`source_location`; select them with `inspect_analysis_view` and
`logical_paths`. Signature blocks are `unverified`; `public_key_sha256`
matches eFuse key digests.
Unknown `source_coverage` means truncation or undescribed trailing bytes, such
as the flash segments of a `--ram-only-header` image; do not infer absence
from it. Erased flash is described. Integrity mismatches are facts, not
refusals. ESP8266 images are refused, and flash-encrypted images are not
recognized. Segments are not disassembled, and RISC-V or Xtensa segments
cannot be opened as raw-binary. See
[ESP images](https://github.com/morluto/rea/blob/main/docs/esp-image.md).
## ESP flash images
A whole-flash dump opens as `esp-flash-image`. `root_format_metadata` holds
`partition_table` (its `method`: `default`, `caller` or a derived
`scan`), `bootloader` and `apps` ESP image facts, the `otadata` records,
`partitions_outside_dump` and `boot_selection`. Every table has a
`boot_selection`: its `basis` names the bootloader's branch (`no-otadata`
without otadata, `none` when the bootloader rejects the table, `unknown`
when a record is missing), and it lists its `assumptions`;
`running_partition` is always `unknown`. Each image names its entry
`path`. With `detail: "summary"` the root kind is `esp-flash-image-summary`
and each image is a compact `summary` (chip, entry point, segment count,
classified load ranges, signature block count, descriptor identity,
integrity); complete detail keeps every segment. Partition rows
(`partitions/<label>.bin`) carry the declared `type`, `subtype`, names,
`flash_address`, `size` and flags; `encrypted` is a storage flag, not
proof of ciphertext. `written_offset`/`written_end` bound the bytes that
are not erased: an app slot whose facts say `erased` can still hold an older
image, and a limitation says so. For a dump read from another flash address, call
`open_binary` with `format: "esp-flash-image"` and `flash_offset`; an
`esp-image` whose `flash_dump` is set names that offset in a next probe; a
relocated table can be named with `partition_table_offset`. No table found
means unknown, often flash encryption. Extraction writes `nvs_keys` as
ordinary data. See
[ESP flash images](https://github.com/morluto/rea/blob/main/docs/esp-flash-image.md).
## Bun standalone executables
A `bun build --compile` executable stores its module graph in Mach-O
`__BUN,__bun`, a PE or ELF `.bun` section, or an appended trailer.
`open_binary` then `inspect_artifact` reports root format `bun-standalone` with
the carrier, structural layout, module count, entry point, raw flags and
compile exec-argv; each module keeps its `/$bunfs/...` source name, loader,
encoding, sizes and SHA-256, with sourcemap, bytecode and module-info regions as
separate children. Use summary root metadata `entry_point_logical_path` as the
exact inventory `logical_paths` selector, then read the selected content node by
its artifact ID; the reader owns virtual-name normalization and indexed fallback. `extract-artifact` writes stored contents through the safe
output tree; route extracted JavaScript to `analyze_javascript_application`.
Layout labels describe record shapes, not a detected compiler version; unknown
or ambiguous layouts are unsupported. Bytecode is retained, not decoded, and the
executable is never run. See
[Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
@@ -41,16 +41,10 @@ behavior unknown. Record truncation, unmatched breakpoints, changed target
identity and cleanup uncertainty as limitations, rather than treating no retained
event as proof that the behavior never occurs.
For same-session MCP comparisons, pass each capture inline or reuse it as
`{"kind":"retained-evidence","evidence_id":"ev_..."}`. Default comparison
keeps capture-local timing as provenance without treating scheduler or observer
latency as a behavior change; use an exact `trace_spec` event when timing is part
of the analyst question.
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/guides/runtime/frida-instrumentation.md),
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/frida-instrumentation.md),
[GDB/Rizin/Cutter](https://github.com/morluto/rea/blob/main/docs/reverse-engineering-tools.md),
[process capture](https://github.com/morluto/rea/blob/main/docs/guides/runtime/process-capture.md),
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/guides/native/native-investigation.md#native-desktop-observation).
[process capture](https://github.com/morluto/rea/blob/main/docs/process-capture.md),
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/native-investigation.md#native-desktop-observation).
## Browser and Electron
@@ -92,8 +86,6 @@ Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
browser and profile. Check `browser.cleanup` for the outcome.
Request only the DOM, accessibility, or network details needed for the question.
The default final capture contains the URL; step outcomes are reported separately.
For multi-step flows with trees or events, select `detail: "summary"` and page
exact steps, transactions, text slices or events with `inspect_analysis_view` kind `browser-scenario`.
Add a wait action for the expected page state before capturing it. Report that
REA performed the declared actions.
@@ -136,5 +128,5 @@ Inspect native `.saz` snapshots with `inspect_web_network_capture` and
traffic analysis, connect Everywhere's separate MCP server and use its
advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or
`AgentCalls`). Preserve existing sessions and filters when inspecting a
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md)
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md)
for the workflow and Telerik's public setup and traffic-debugging skills.
-21
View File
@@ -1,21 +0,0 @@
{
"$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
"name": "rea",
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"owner": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"plugins": [
{
"name": "rea",
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"source": {
"source": "npm",
"package": "rea-agents",
"version": "6.8.0"
},
"category": "development"
}
]
}
-29
View File
@@ -1,29 +0,0 @@
{
"name": "rea",
"version": "6.8.0",
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"author": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"homepage": "https://rea.tools/",
"repository": "https://github.com/morluto/rea",
"license": "MIT",
"keywords": [
"reverse-engineering",
"binary-analysis",
"decompiler",
"mcp-server"
],
"skills": "./skills",
"mcpServers": {
"rea": {
"command": "node",
"args": [
"${CLAUDE_PLUGIN_ROOT}/scripts/plugin-mcp-launcher.mjs",
"6.8.0"
]
}
},
"icon": "./assets/rea-plugin.png"
}
-13
View File
@@ -1,13 +0,0 @@
{
"mcpServers": {
"rea": {
"command": "node",
"args": [
"./scripts/plugin-mcp-launcher.mjs",
"6.8.0"
],
"cwd": ".",
"startup_timeout_sec": 30
}
}
}
-40
View File
@@ -1,40 +0,0 @@
{
"name": "rea",
"version": "6.8.0",
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"author": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"homepage": "https://rea.tools/",
"repository": "https://github.com/morluto/rea",
"license": "MIT",
"keywords": [
"reverse-engineering",
"binary-analysis",
"decompiler",
"mcp-server"
],
"skills": "./skills",
"mcpServers": "./.codex-plugin/mcp.json",
"interface": {
"displayName": "REA",
"shortDescription": "Reverse engineer anything",
"longDescription": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"developerName": "morluto",
"category": "Developer Tools",
"capabilities": [
"Interactive",
"Read",
"Write"
],
"websiteURL": "https://rea.tools/",
"defaultPrompt": [
"Use REA to explain how this binary implements a feature.",
"Use REA to compare two versions of this application.",
"Use REA to recover the sources from this JavaScript bundle."
],
"composerIcon": "./assets/rea-plugin.png",
"logo": "./assets/rea-plugin.png"
}
}
-12
View File
@@ -1,12 +0,0 @@
# Review changes to security-sensitive automation and dependency inputs.
/.github/CODEOWNERS @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/.github/dependabot.yml @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/.github/workflows/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/.github/actions/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/scripts/verify-tools/skills/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/scripts/verify-tools/skills/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/scripts/fixtures/http3-lab-requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/website/requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/third_party/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
+1 -1
View File
@@ -27,7 +27,7 @@ body:
attributes:
label: Documentation location
description: Link the page, section, command, API, skill, or inline comment.
placeholder: README.md#quick-start or docs/reference/mcp-contracts.md
placeholder: README.md#quick-start or docs/mcp-contracts.md
validations:
required: true
-21
View File
@@ -1,21 +0,0 @@
name: REA production code
# Match the existing default-setup scan's local-input threat model.
threat-models: [local]
# Retain root traversal, including future production roots. Hidden directories
# need their own root; use a directory so Python can share this configuration.
paths:
- .
- docs/.vitepress
# Keep the default query suite and all production roots, including bridge/,
# install.sh, runtime scripts, release controllers, and GitHub Actions.
# Exclude test inputs at extraction time so they cannot taint production callers.
paths-ignore:
- tests
- scripts/fixtures
- "**/*.test.*"
- "**/*.fixture.*"
- "**/test_*.py"
- "**/*_test.py"
-33
View File
@@ -1,33 +0,0 @@
version: 2
updates:
- package-ecosystem: npm
directories:
- /
- /scripts/verify-tools/skills
schedule:
interval: weekly
cooldown:
default-days: 7
open-pull-requests-limit: 5
groups:
npm-minor-and-patch:
update-types:
- minor
- patch
- package-ecosystem: pip
directory: /website
schedule:
interval: weekly
open-pull-requests-limit: 5
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
actions-minor-and-patch:
update-types:
- minor
- patch
+2 -10
View File
@@ -36,9 +36,6 @@ unchecked. Maintainers can help resolve open questions.
Use "none" or "not applicable" only when established. -->
- Semantic owner and earliest changed stage:
<!-- Name the producer of parsed facts or resources and its affected consumers.
Summarize superseded paths removed and any distinct policy, lifecycle,
lazy-loading, or compatibility responsibility retained. -->
- Contribution scope fit and linked maintainer decision:
- CLI and MCP/tool-catalog contract:
- Provider, bridge, target-format, or platform compatibility:
@@ -71,10 +68,6 @@ source, link, or rendered-document checks. -->
- User-visible CLI/MCP output (if applicable):
- Remaining proof gaps:
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#boundary-regressions)
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#lifecycle-regressions)
covered by the evidence above, and explain relevant gaps.
For evidence-bearing changes:
- [ ] Observed, derived, and inferred claims remain distinguishable.
@@ -99,15 +92,14 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
its new behavior and migration, and why compatibility/deprecation cannot
preserve it. Every release increments minor, including breaking changes;
keep breaking markers and migration notes so callers can assess compatibility.
See docs/development/releasing.md. -->
See docs/releasing.md. -->
- Real Hopper/Ghidra, browser, or OS coverage:
- Package or release metadata impact:
- Security, privacy, process, or containment review:
## Author readiness
<!-- The human owner completes these declarations before requesting review,
unless the PR author is exempt as listed in CONTRIBUTING.md.
<!-- The human owner completes these declarations before requesting review.
The PR author is the owner by default. A different owner must explicitly accept
ownership on this PR.
AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA.
+19 -95
View File
@@ -61,9 +61,6 @@ jobs:
qualification_run_id: ${{ steps.qualification.outputs.run_id }}
code: ${{ steps.plan.outputs.code }}
reverse_tools: ${{ steps.plan.outputs.reverse_tools }}
python: ${{ steps.plan.outputs.python }}
packet_capture: ${{ steps.plan.outputs.packet_capture }}
signal_recording: ${{ steps.plan.outputs.signal_recording }}
sqlite: ${{ steps.plan.outputs.sqlite }}
runtime: ${{ steps.plan.outputs.runtime }}
package: ${{ steps.plan.outputs.package }}
@@ -147,9 +144,6 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Validate source architecture before dependency installation
if: needs.changes.outputs.code == 'true'
run: node scripts/verify/architecture-guards.mjs
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
@@ -270,27 +264,6 @@ jobs:
if-no-files-found: ignore
retention-days: 7
python:
name: Python producer tests
needs: changes
if: needs.changes.outputs.python == 'true' && needs.changes.outputs.tier == 'fast'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
# Analysis-efficiency checks import compiled process modules.
- name: Reuse portable build outputs across runs
uses: ./.github/actions/portable-build-cache
- run: npm run test:prepare
- run: npm run test:python
test-source:
name: Source tests
needs: changes
@@ -418,9 +391,7 @@ jobs:
needs.changes.outputs.web_network == 'true' ||
needs.changes.outputs.web_runtime == 'true' ||
needs.changes.outputs.web_source_map == 'true' ||
needs.changes.outputs.agentenv == 'true' ||
needs.changes.outputs.packet_capture == 'true' ||
needs.changes.outputs.signal_recording == 'true'
needs.changes.outputs.agentenv == 'true'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
@@ -779,23 +750,9 @@ jobs:
npm run verify:legacy-simulator
npm run verify:apple-dispatch
npm run verify:interface-builder
- name: Verify portable iNES CLI, MCP and source boundaries
run: >-
npm run test:focused --
tests/acceptance/inesContainer.test.ts
tests/boundary/filesystem/inesArtifact.test.ts
tests/boundary/filesystem/inesCancellation.test.ts
- name: Run native Apple and process boundary regressions
run: >-
npm run test:focused --
tests/boundary/providers/hopper/hopperMacLaunchArguments.test.ts
tests/acceptance/providers/hopperStartupRecovery.test.ts
tests/boundary/providers/hopper/hopperClient.helperFailure.test.ts
tests/boundary/providers/hopper/bridgeBootstrapCleanup.test.ts
tests/boundary/providers/hopper/bridgeTargetLease.integration.test.ts
tests/acceptance/artifactDmgAttachmentRecoveryCli.test.ts
tests/boundary/providers/native/nativeDmgArtifactReader.test.ts
tests/process-global/dmgCancellationReason.test.ts
tests/boundary/filesystem/installScript.test.ts
tests/boundary/filesystem/nibNativeHierarchy.test.ts
tests/boundary/filesystem/interfaceBuilderBundle.test.ts
@@ -805,13 +762,9 @@ jobs:
tests/boundary/cli/nativeDemangleCli.test.ts
tests/boundary/cli/nativeDispatchPropertyAttributes.test.ts
tests/boundary/mcp/nativeDemangleMcp.test.ts
tests/boundary/mcp/nativeMachoSummaryMcp.test.ts
tests/boundary/providers/native/nativeSignatureTargetBinding.test.ts
tests/boundary/providers/native/nativeUiOffscreenWindow.test.ts
tests/boundary/process/nativeUiVerifierLane.test.ts
tests/boundary/cli/nativeSignatureCli.test.ts
tests/boundary/cli/nativeMachoArm64_32Cli.test.ts
tests/boundary/cli/nativeMachoEncryptionCli.test.ts
tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts
src/process/capture/FilesystemSnapshot.test.ts
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
@@ -899,32 +852,12 @@ jobs:
# The matching native artifact establishes real Job Object and private
# runtime boundaries for these fixtures. Ghidra and Java are not needed;
# real engine claims remain in the separate controlled-provider workflow.
# Run native proofs before the broad curated suite so an unrelated
# fixture failure cannot hide the real control and recovery checks.
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-process-boundary-fixture
path: .cache/windows-process-boundary/
# Passing the fixture adds the breakaway, caller/child token, environment
# semantics and ancestor reparse observations, which the lane skips
# entirely when it is absent.
- run: >-
npm run verify:windows-native -- .
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-native-inspection-fixture
path: .cache/windows-native-inspection-fixture/
- name: Verify native inspection failure ownership and recovery
run: node scripts/verify-windows-native-inspection.mjs
- name: Run curated Windows capability tests
run: >-
npx vitest run --coverage.enabled=false
src/process/capture/FilesystemSnapshot.test.ts
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
tests/boundary/filesystem/typescriptCompiler.test.ts
tests/boundary/filesystem/releaseProposal.test.ts
tests/acceptance/byteEntropyWindowsIdentity.test.ts
tests/boundary/filesystem/binaryTarget.test.ts
src/contracts/sessionPathInputs.test.ts
tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts
@@ -934,7 +867,6 @@ jobs:
tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts
tests/boundary/filesystem/setupUnavailableProvider.test.ts
tests/boundary/filesystem/windowsNpxRegistration.test.ts
tests/boundary/filesystem/claudeDesktopSkillUninstallWindows.test.ts
tests/boundary/filesystem/clientConfigurationEnvironment.test.ts
tests/boundary/filesystem/clientConfigurationLayeredEnvironment.test.ts
tests/acceptance/setup/registrationRuntimeEnvironmentCli.test.ts
@@ -946,7 +878,6 @@ jobs:
tests/boundary/process/packageNpmInvocation.test.ts
tests/boundary/process/packedBridgeVerification.test.ts
tests/boundary/process/windowsOutputCapture.test.ts
tests/boundary/process/windowsExecutableSelection.test.ts
tests/acceptance/webModuleCli.test.ts
tests/acceptance/webSourceLocationCli.test.ts
tests/boundary/cli/referenceSourceImportStatus.test.ts
@@ -971,6 +902,22 @@ jobs:
tests/process-global/runnerCancellation.test.ts
tests/process-global/windowsOwnedCommand.test.ts
tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-process-boundary-fixture
path: .cache/windows-process-boundary/
# Passing the fixture adds the breakaway, caller/child token, environment
# semantics and ancestor reparse observations, which the lane skips
# entirely when it is absent.
- run: >-
npm run verify:windows-native -- .
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-native-inspection-fixture
path: .cache/windows-native-inspection-fixture/
- name: Verify native inspection failure ownership and recovery
run: node scripts/verify-windows-native-inspection.mjs
- name: Verify real Node Inspector discovery and observation
timeout-minutes: 10
env:
@@ -1017,8 +964,8 @@ jobs:
uses: ./.github/actions/npm-cache
- run: npm ci
- run: npm run docs:check
- name: Verify published routes, source links and local anchors
run: npm run verify:docs:routes
- name: Render documentation and validate page links
run: npm run docs:build
- name: Generation leaves authored files unchanged
run: git diff --exit-code
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
@@ -1131,26 +1078,6 @@ jobs:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
packet_capture:
needs: [changes, static, build]
if: needs.changes.outputs.packet_capture == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-packet-capture.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
signal_recording:
needs: [changes, static, build]
if: needs.changes.outputs.signal_recording == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-signal-recording.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
nativeaot:
needs: [changes, static]
if: needs.changes.outputs.nativeaot == 'true'
@@ -1177,9 +1104,6 @@ jobs:
static-deep,
build,
reverse-tools-quality,
python,
packet_capture,
signal_recording,
test-shard,
test-source,
sqlite-runtime,
-61
View File
@@ -1,61 +0,0 @@
name: CodeQL production
on:
push:
branches: [main]
pull_request:
branches: [main]
schedule:
- cron: "37 5 * * 1"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: codeql-production-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
analyze:
# A repository administrator must disable default setup before enabling
# this workflow; default setup rejects advanced CodeQL result uploads.
if: vars.REA_CODEQL_ADVANCED == 'true'
name: Analyze production (${{ matrix.language }})
runs-on: ubuntu-latest
timeout-minutes: 45
permissions:
contents: read
security-events: write
strategy:
fail-fast: false
max-parallel: 1
matrix:
language: [javascript-typescript, python, actions]
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
with:
persist-credentials: false
- name: Initialize production analysis
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
with:
languages: ${{ matrix.language }}
build-mode: none
config-file: .github/codeql/production.yml
threads: 2
ram: 4096
- name: Analyze production code
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
with:
category: /language:${{ matrix.language }}
output: ${{ runner.temp }}/codeql-production-results
threads: 2
ram: 4096
- name: Retain complete analysis evidence
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: codeql-production-${{ matrix.language }}
path: ${{ runner.temp }}/codeql-production-results
if-no-files-found: error
retention-days: 7
@@ -62,64 +62,3 @@ jobs:
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
cat "${probe_root}/output.log"
grep -F 'REA_ARM_CONTEXT_OK arm=0 thumb=1 unavailable=null (injected provider value)' "${probe_root}/output.log"
# Each probe imports source-owned bytes at 0x1000, away from the Cortex-M
# vector symbols that ARMCortex.pspec defines at 0, and decodes them from
# the language's default context, so M-profile Thumb state must hold at
# the first byte. The bytes are never executed.
- name: Decode M-profile ARM and Xtensa bytes in their default context
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
probe_root="${RUNNER_TEMP}/rea-decode"
mkdir -p "${probe_root}"
# mrs r0, msp; bx lr
printf '\357\363\010\200\160\107' > "${probe_root}/cortex.bin"
# mrs r0, msplim (ARMv8-M only); bx lr
printf '\357\363\012\200\160\107' > "${probe_root}/v8m.bin"
# entry a1, 32; retw.n
printf '\066\101\000\035\360' > "${probe_root}/xtensa.bin"
decode() {
local name="$1" language="$2"
shift 2
mkdir -p "${probe_root}/${name}-project"
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/${name}-project" Decode \
-import "${probe_root}/${name}.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
-processor "${language}" -noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra" \
-postScript ReaDecodeProbe.java "${language}" "$@" \
> "${probe_root}/${name}.log" 2>&1 || { cat "${probe_root}/${name}.log"; exit 1; }
cat "${probe_root}/${name}.log"
grep -F "REA_DECODE_OK ${language} $*" "${probe_root}/${name}.log"
}
decode cortex ARM:LE:32:Cortex mrs/4/msp bx/2/lr
decode v8m ARM:LE:32:v8-m mrs/4/msplim bx/2/lr
decode xtensa Xtensa:LE:32:default entry/3/a1 retw.n/2
# A call through a writable slot keeps Ghidra's recorded target as an
# observed callee while callee coverage stays partial; a direct call is
# the complete control. The source-owned x86-64 bytes are never executed.
- name: Keep computed-call targets non-exhaustive in callee coverage
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
probe_root="${RUNNER_TEMP}/rea-computed-call"
mkdir -p "${probe_root}/project"
{
printf '\270\001\000\000\000\303\314\314' # 0x1000: mov eax, 1; ret
printf '\270\002\000\000\000\303\314\314' # 0x1008: mov eax, 2; ret
printf '\377\025\052\000\000\000\303\314' # 0x1010: call [rip + 0x2a]; ret
printf '\377\320\303\314\314\314\314\314' # 0x1018: call rax; ret
printf '\350\333\377\377\377\303\314\314' # 0x1020: call 0x1000; ret
printf '\314%.0s' $(seq 24) # 0x1028: int3 padding
printf '\000\020\000\000\000\000\000\000' # 0x1040: slot = 0x1000
} > "${probe_root}/computed-call.bin"
test "$(wc -c < "${probe_root}/computed-call.bin")" -eq 72
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/project" ComputedCall \
-import "${probe_root}/computed-call.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
-processor x86:LE:64:default -noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
-postScript ReaComputedCallCoverageProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
cat "${probe_root}/output.log"
grep -F 'REA_COMPUTED_CALL_COVERAGE_OK slot=observed-complete-discovery-unknown register=partial direct=complete callers=discovery-unknown' "${probe_root}/output.log"
+1 -1
View File
@@ -27,4 +27,4 @@ jobs:
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- run: npm run verify:docs:routes
- run: npm run docs:build
+7 -8
View File
@@ -54,11 +54,10 @@ jobs:
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
# The pinned framework builds its gateway and service-db images only when pulling them from
# ghcr.io fails, from public ECR base images; public ECR caps anonymous pulls by data per source
# IP and hosted runners share IPs, so such a build can fail here for hours. mirror.gcr.io serves
# the same Docker official images under the same digests, so each base image those Dockerfiles
# name is pulled there and named as its Dockerfile names it, as the framework's own CI does.
# Public ECR caps anonymous pulls by data per source IP and hosted runners share IPs, so the
# framework's image builds can fail here for hours. mirror.gcr.io serves the same Docker
# official images under the same digests, so each base image the pinned framework builds from
# is pulled there and named as its Dockerfile names it, as the framework's own CI does.
# Digest-pinned names are set in containerd, whose images Docker reads only on the containerd
# image store; switching stores restarts dockerd, so it happens before anything uses Docker.
- name: Use the containerd image store
@@ -73,16 +72,16 @@ jobs:
run: |
runner_root="${RUNNER_TEMP}/rea-agentenv-runner"
python -m venv "${runner_root}"
"${runner_root}/bin/python" -m pip install --require-virtualenv --no-cache-dir 'agentenv-framework==0.9.1431'
"${runner_root}/bin/python" -m pip install --require-virtualenv --no-cache-dir 'agentenv-framework==0.9.1334'
"${runner_root}/bin/agent-env" --version
docker version --format '{{.Server.Version}}'
echo "REA_AGENT_ENV_COMMAND=${runner_root}/bin/agent-env" >> "${GITHUB_ENV}"
echo "REA_DOCKER_COMMAND=$(command -v docker)" >> "${GITHUB_ENV}"
- name: Pull the gateway and service-db public ECR base images from mirror.gcr.io
- name: Pull the pinned framework's public ECR base images from mirror.gcr.io
shell: bash
run: |
package="$("${REA_AGENT_ENV_COMMAND%/agent-env}/python" -c 'import agent_env, pathlib; print(pathlib.Path(agent_env.__file__).parent)')"
grep -rhoE '^FROM +public\.ecr\.aws/docker/library/[^ ]+' --include='Dockerfile*' "${package}/env/gateway" "${package}/env/envs/service_db" | awk '{print $2}' | sort -u |
grep -rhoE '^FROM +public\.ecr\.aws/docker/library/[^ ]+' --include='Dockerfile*' "${package}" | awk '{print $2}' | sort -u |
while read -r image; do
name=${image#public.ecr.aws/docker/library/}
source=mirror.gcr.io/library/$name
+1 -9
View File
@@ -11,7 +11,6 @@ on:
- tests/conformance/android/**
- tests/fixtures/android/static-initializers/**
- tests/fixtures/android/direct-invokes/**
- tests/fixtures/android/external-invokes/**
- .nvmrc
workflow_dispatch:
workflow_call:
@@ -45,8 +44,7 @@ jobs:
"$JAVA_HOME/bin/javac" -J-Xmx512m -J-XX:ActiveProcessorCount=2 \
-classpath "$jar" -d "$classes" \
bridge/android/ReaJadxBridge.java \
tests/conformance/android/ReaJadxMetadataRegression.java \
tests/conformance/android/ReaJadxExternalReferencesRegression.java
tests/conformance/android/ReaJadxMetadataRegression.java
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
-classpath "$classes:$jar" ReaJadxMetadataRegression \
"$android_root/ApiDemos-debug.apk" \
@@ -55,9 +53,3 @@ jobs:
| tee "$android_root/metadata.log"
grep -F 'RAW_CLASS_SELECTOR_REGRESSION_PASS nested=true metadata=true references=true' \
"$android_root/metadata.log"
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
-classpath "$classes:$jar" ReaJadxExternalReferencesRegression \
"$GITHUB_WORKSPACE/tests/fixtures/android/external-invokes" "$classes" \
| tee "$android_root/external-references.log"
grep -F 'EXTERNAL_REFERENCE_REGRESSION_PASS prototypes=2 using_nodes=3 sites=4' \
"$android_root/external-references.log"
-6
View File
@@ -55,12 +55,6 @@ jobs:
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- name: Install AArch64 and ELF32 cross toolchains for source-owned fixtures
run: |
sudo apt-get update
sudo apt-get install --yes --no-install-recommends \
gcc-aarch64-linux-gnu gcc-i686-linux-gnu gcc-arm-linux-gnueabi gcc-mips-linux-gnu gcc-mipsel-linux-gnu \
libc6-dev-arm64-cross libc6-dev-i386-cross libc6-dev-armel-cross libc6-dev-mips-cross libc6-dev-mipsel-cross
- name: Acquire exact upstream libraries in an isolated runner environment
shell: bash
run: |
+1 -7
View File
@@ -48,13 +48,7 @@ jobs:
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
# Own a fresh bus so Chromium never depends on an inherited session address.
- run: dbus-run-session -- npm run verify:browser
env:
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
REA_BROWSER_NO_SANDBOX: "true"
# The preceding verifier builds the runtime used by this request journey.
- run: dbus-run-session -- node scripts/verify-browser-request.mjs
- run: npm run verify:browser
env:
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
REA_BROWSER_NO_SANDBOX: "true"
+32 -38
View File
@@ -1,35 +1,39 @@
name: Real packet capture
on:
workflow_call:
inputs:
build_cache_artifact:
description: Portable build cache artifact from the calling CI run.
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
pull_request:
paths:
- "tests/boundary/captures/**"
- "tests/boundary/cli/captureCleanup.test.ts"
- "tests/boundary/mcp/packetCaptureMcp.test.ts"
- "src/captures/**"
- "src/pcap/**"
- "src/domain/pcap/**"
- "src/domain/analysisView/pcapCaptureView.ts"
- "src/application/pcap/**"
- "src/contracts/pcap/**"
- "src/server/registerPcapTools.ts"
- "tests/boundary/pcap/**"
- "src/domain/captures/**"
- "src/application/captures/**"
- "src/contracts/captures/**"
- "src/composition/captures.ts"
- "src/cli/captureCommands.ts"
- "src/server/registerCaptureTools.ts"
- "scripts/verify-packet-capture.mjs"
- ".github/workflows/real-packet-capture.yml"
permissions:
contents: read
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
concurrency:
group: real-packet-capture-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
tshark:
runs-on: ubuntu-24.04
@@ -44,18 +48,8 @@ jobs:
uses: ./.github/actions/npm-cache
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- run: npm run build:cached
- name: Run focused tests and require Linux resource and cleanup cases
run: |
report=".cache/rea-packet-required-tests.json"
mkdir -p .cache
rm -f "$report"
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts tests/acceptance/pcapVerifier.test.ts
node scripts/ci/require-provider-tests.mjs packet "$report"
- run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
- run: node scripts/verify-packet-capture.mjs
env:
REA_TSHARK_COMMAND: /usr/bin/tshark
+25 -38
View File
@@ -1,35 +1,32 @@
name: Real signal recording
on:
workflow_call:
inputs:
build_cache_artifact:
description: Portable build cache artifact from the calling CI run.
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
pull_request:
paths:
- "tests/boundary/signals/**"
- "tests/boundary/cli/signalCleanup.test.ts"
- "tests/boundary/mcp/signalRecordingMcp.test.ts"
- "src/signals/**"
- "src/domain/signals/**"
- "src/application/signals/**"
- "src/contracts/signals/**"
- "src/composition/signals.ts"
- "src/cli/signalCommands.ts"
- "src/server/registerSignalTools.ts"
- "scripts/verify-signal-recording.mjs"
- ".github/workflows/real-signal-recording.yml"
permissions:
contents: read
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
concurrency:
group: real-signal-recording-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
sigrok:
runs-on: ubuntu-24.04
@@ -44,18 +41,8 @@ jobs:
uses: ./.github/actions/npm-cache
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- run: npm run build:cached
- name: Run focused tests and require Linux resource and cleanup cases
run: |
report=".cache/rea-signal-required-tests.json"
mkdir -p .cache
rm -f "$report"
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
node scripts/ci/require-provider-tests.mjs signal "$report"
- run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
- run: node scripts/verify-signal-recording.mjs
env:
REA_SIGROK_COMMAND: /usr/bin/sigrok-cli
+1 -2
View File
@@ -44,8 +44,7 @@ jobs:
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
# Own a fresh bus so Chromium never depends on an inherited session address.
- run: dbus-run-session -- npm run verify:browser:runtime
- run: env -u DBUS_SESSION_BUS_ADDRESS npm run verify:browser:runtime
env:
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
REA_BROWSER_NO_SANDBOX: "true"
+1 -2
View File
@@ -46,8 +46,7 @@ jobs:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- name: Install the pinned compiler into an isolated fixture directory
run: npm install --prefix "$RUNNER_TEMP/rea-web-source-map-compiler" --no-audit --no-fund --package-lock=false [email protected]
# Own a fresh bus so Chromium never depends on an inherited session address.
- run: dbus-run-session -- npm run verify:browser:source-maps
- run: npm run verify:browser:source-maps
env:
# Retain browser launch argv/PID/stderr in the verifier's captured
# CLI failure diagnostics without changing its deadline or retrying.
-4
View File
@@ -21,10 +21,6 @@ jobs:
python-version: "3.12"
cache: pip
cache-dependency-path: website/requirements.txt
- name: Set up Node.js for the website Worker checks
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Install sharing-image dependencies
run: |
sudo apt-get update
+8 -50
View File
@@ -54,10 +54,6 @@ jobs:
python-version: "3.12"
cache: pip
cache-dependency-path: website/requirements.txt
- name: Set up Node.js for the website Worker checks
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Install sharing-image dependencies
run: |
sudo apt-get update
@@ -79,21 +75,10 @@ jobs:
PY
- name: Configure Pages
uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
# upload-pages-artifact excludes dot directories, including .well-known.
- name: Package the website including agent discovery
run: |
tar --dereference --hard-dereference --directory website/_site \
--exclude=.git --exclude=.github -cf "$RUNNER_TEMP/artifact.tar" .
tar -tf "$RUNNER_TEMP/artifact.tar" \
./.well-known/ard.json ./.well-known/ai-catalog.json \
./.well-known/agent-skills/index.json > /dev/null
- name: Upload the website
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0
with:
name: github-pages
path: ${{ runner.temp }}/artifact.tar
retention-days: 1
if-no-files-found: error
path: website/_site
deploy:
needs: prepare
@@ -145,24 +130,14 @@ jobs:
PUBLISH_TARGET: ${{ inputs.target }}
run: |
python3 -u - <<'PY'
import hashlib
import json
import os
import sys
import time
from urllib.error import URLError
from urllib.parse import urljoin, urlsplit
from urllib.parse import urljoin
from urllib.request import Request, urlopen
def open_public(url, accept=None):
headers = {
"Cache-Control": "no-cache",
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
}
if accept:
headers["Accept"] = accept
return urlopen(Request(url, headers=headers), timeout=10)
expected = os.environ["GITHUB_SHA"]
pages_url = os.environ["PAGES_URL"]
if not pages_url:
@@ -178,7 +153,11 @@ jobs:
for name, base in list(pending.items()):
url = urljoin(base, "deployment.json") + "?commit=" + expected
try:
with open_public(url) as response:
request = Request(url, headers={
"Cache-Control": "no-cache",
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
})
with urlopen(request, timeout=10) as response:
actual = json.load(response).get("commit")
if actual == expected:
print(f"{name} serves {expected}: {base}")
@@ -193,27 +172,6 @@ jobs:
time.sleep(5)
if pending:
sys.exit("Could not verify published revision on: " + ", ".join(pending))
for name, base in hosts.items():
with open_public(urljoin(base, ".well-known/agent-skills/index.json")) as response:
skill, = json.load(response)["skills"]
# Fetch from each selected host, including the /rea/ mirror.
artifact_url = urljoin(base, urlsplit(skill["url"]).path.lstrip("/"))
with open_public(artifact_url) as response:
digest = "sha256:" + hashlib.sha256(response.read()).hexdigest()
if digest != skill["digest"]:
sys.exit(name + " serves a skill archive with the wrong digest")
for manifest in ("ard.json", "ai-catalog.json"):
with open_public(urljoin(base, ".well-known/" + manifest)) as response:
if not json.load(response).get("entries"):
sys.exit(name + " serves an empty discovery manifest")
if name == "Cloudflare":
with open_public(base, accept="text/markdown") as response:
vary = {value.strip().lower() for value in response.headers.get("Vary", "").split(",")}
if response.headers.get_content_type() != "text/markdown" or "accept" not in vary:
sys.exit("Cloudflare did not negotiate Markdown with Vary: Accept")
if 'rel="ard"' not in response.headers.get("Link", ""):
sys.exit("Cloudflare is missing its discovery Link header")
print(name + " serves the discovery manifests and verified skill archive")
with open(os.environ["GITHUB_STEP_SUMMARY"], "a", encoding="utf-8") as summary:
links = " and ".join(f"[{name}]({url})" for name, url in hosts.items())
summary.write(f"Published commit `{expected}` to {links}.\n")
+1 -1
View File
@@ -50,7 +50,7 @@ Thumbs.db
# Local reference sources and analysis artifacts
/_reference/
/experiments/analysis-efficiency/
/experiments/analysis-efficiency/results/
# Generated website download
/website/public/examples/notes-example.zip
+1 -4
View File
@@ -10,10 +10,7 @@
"node_modules/**",
"CHANGELOG.md",
"package-lock.json",
".claude-plugin/plugin.json",
".codex-plugin/plugin.json",
"_reference/**",
"third_party/**",
".codex-plugin/mcp.json"
"third_party/**"
]
}
+3 -95
View File
@@ -1,13 +1,7 @@
{
"$schema": "./node_modules/oxlint/configuration_schema.json",
"plugins": ["typescript", "import", "unicorn", "oxc", "vitest"],
"plugins": ["typescript", "import", "unicorn", "oxc"],
"rules": {
"rea/no-buffer-deep-equality": "error",
"rea/cancellation-cause": "error",
"no-async-promise-executor": "error",
"no-unreachable": "error",
"no-unsafe-optional-chaining": "error",
"no-useless-escape": ["warn", { "allowRegexCharacters": ["-"] }],
"complexity": ["warn", 20],
"max-depth": ["warn", 4],
"max-lines": [
@@ -29,38 +23,7 @@
"max-params": ["warn", 4],
"no-var": "warn",
"no-var-requires": "error",
"typescript/no-import-type-side-effects": "error",
"vitest/no-standalone-expect": "off",
"vitest/no-conditional-expect": "off",
"vitest/expect-expect": "off",
"vitest/no-disabled-tests": "off",
"vitest/require-mock-type-parameters": "off",
"vitest/require-to-throw-message": "off",
"import/no-duplicates": [
"warn",
{
"preferInline": false
}
],
"vitest/valid-expect": [
"warn",
{
"maxArgs": 2
}
],
"rea/no-evidence-json-preadmission": "error",
"rea/no-positional-tool-contract": "error",
"rea/no-fixture-root-cleanup": "error",
"typescript/no-floating-promises": "error",
"typescript/no-misused-promises": "warn",
"typescript/return-await": ["error", "error-handling-correctness-only"],
"typescript/switch-exhaustiveness-check": [
"error",
{
"considerDefaultExhaustiveForUnions": true
}
],
"rea/no-shared-module-mock": "error"
"typescript/no-import-type-side-effects": "error"
},
"overrides": [
{
@@ -73,12 +36,6 @@
"no-var-requires": "off"
}
},
{
"files": ["src/**/*.ts"],
"rules": {
"typescript/no-misused-promises": "error"
}
},
{
"files": ["tests/**/*.ts", "src/**/*.test.ts", "src/**/*.fixture.ts"],
"rules": {
@@ -90,8 +47,7 @@
"skipBlankLines": true,
"skipComments": true
}
],
"typescript/no-misused-promises": "warn"
]
}
},
{
@@ -135,45 +91,6 @@
}
]
}
},
{
"files": ["tests/**/*.test.ts", "src/**/*.test.ts"],
"rules": {
"vitest/valid-expect": [
"warn",
{
"maxArgs": 2
}
],
"vitest/valid-expect-in-promise": "error",
"vitest/no-focused-tests": "error"
}
},
{
"files": [
"scripts/lib/http3-capture-assertions.test.mjs",
"scripts/lib/http3-verifier-lifecycle.test.mjs"
],
"rules": {
"typescript/no-floating-promises": [
"error",
{
"allowForKnownSafeCalls": [
{
"from": "package",
"package": "@types/node",
"name": "test"
}
]
}
]
}
},
{
"files": ["scripts/fixtures/javascript-recovery/code-split/entry.mjs"],
"rules": {
"typescript/no-floating-promises": "off"
}
}
],
"env": {
@@ -192,14 +109,5 @@
".devin",
".factory",
"third_party"
],
"options": {
"reportUnusedDisableDirectives": "error"
},
"jsPlugins": [
{
"name": "rea",
"specifier": "./scripts/lint/index.mjs"
}
]
}
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "6.8.0"
".": "6.7.0"
}
+13 -36
View File
@@ -14,13 +14,13 @@ Preserve caller-selected inputs and local evidence, including output, URLs, path
## Architecture and Cleanup
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture/architecture.mermaid](docs/architecture/architecture.mermaid) when changing composition.
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture.mermaid](docs/architecture.mermaid) when changing composition.
Give each interpretation and resource a clear owner. Consumers must use that owner's parsed facts rather than reinterpret raw source. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
For lifecycle changes, follow [ownership and completion invariants](docs/development/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/development/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
For lifecycle changes, follow [ownership and completion invariants](docs/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
Before adding complexity for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode, likelihood, and complexity cost to the user and get their buy-in. Do not silently expand scope for theoretical robustness.
@@ -28,10 +28,6 @@ Assess brittleness in touched code during implementation, debugging, review, and
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
After confirming a bug, search sibling adapters, consumers and similar workflows
for the same failure. Verify suspected matches, record working comparisons, and
group findings by root cause instead of filing duplicate reports.
When addressing a review thread, inspect the producing boundary and affected consumers for brittleness. Complete directly related refactoring in the same change. Before resolving the thread, explain the root cause, structural fix, validation, and any remaining cleanup.
Parse unknown boundary values and model expected failures with the tagged error algebra and `Result`. Preserve partial facts and meaningful failure reasons. Keep identity and lookup values separate from display formatting; preserve source values and explicit unknowns when normalization loses information. Absolute paths, file URLs, and HTTP paths have different semantics.
@@ -57,23 +53,14 @@ Optimize in this order:
3. CPU and memory. Avoid unnecessary work and copies; bound concurrency,
retained state and serialization growth.
Reserve performance measurements for changes that target performance or have a
concrete regression concern. Routine cleanup uses correctness checks; changes
to helper calls, imports, or representations alone do not require synthetic
benchmarks. When measurement is warranted, measure representative completed
tasks and compare equivalent workflows before and after where available. Report
token usage, elapsed time, CPU time and peak process-family memory. Separate cold
For changes affecting these costs, measure representative completed tasks and
compare equivalent workflows before and after where available. Report token
usage, elapsed time, CPU time and peak process-family memory. Separate cold
and warm runs, label estimates, and state unmeasured costs. Fewer bytes alone
do not establish fewer tokens or faster analysis.
## Development
Documentation ownership and authoring conventions live in
[docs/development/documentation.md](docs/development/documentation.md). A feature
uses `docs/specs/<domain>/<feature-slug>/spec.md`, one `plan.md` and `tasks.md`;
individual architectural decisions use `docs/adr/<domain>/<decision-slug>.md`.
Consult the [documentation directory](docs/README.md) before adding a guide.
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`. Prefix `PATH="/usr/local/bin:$PATH"` before `node` and `npm`.
- `npm ci`: install locked dependencies.
@@ -85,24 +72,15 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
When changing installers or setup recovery, follow
[Checking installers and publisher downloads](docs/development/testing.md#checking-installers-and-publisher-downloads).
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/development/testing.md#developer-commands).
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/testing.md#developer-commands).
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
## Tool and Test Changes
For tool design, follow [docs/development/tool-design.md](docs/development/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
For user- or agent-facing tool problems, prioritize fixing the owning behavior,
contract, and actionable result or recovery at the point of use. Document the
result in product docs when explanation is needed. Add skill guidance only for
workflow judgment or context that the tool cannot reasonably convey; do not use
a skill as the primary repair for a discoverability or contract defect.
For test changes or provider claims, use [docs/development/testing.md](docs/development/testing.md). Select applicable [boundary regressions](docs/development/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
@@ -130,12 +108,11 @@ distinguish source-derived conclusions, proposed checks, and unavailable coverag
Address actionable automated review findings and explain disagreements.
Leave the template's **Author readiness** declarations unchecked for the human
owner to complete. Never attest to human review on their behalf. Draft status
is optional; preserve the PR's current draft status unless the user asks to change
it. Later pushes do not invalidate the readiness declaration or require another
human sign-off. Keep the PR body and affected validation current as work changes.
The readiness check validates the filled declaration or the author exemption
listed in CONTRIBUTING.md; CI and automated review results do not establish that
owner to complete. Never attest to human review on their behalf. Keep the PR in
draft until the human declares the PR initially ready. Later pushes do not
invalidate that declaration or require another human sign-off. Keep the PR body
and affected validation current as work changes. The readiness check validates
the filled declaration; CI and automated review results do not establish that
human review occurred, maintainer scope approval, or merge permission.
<!-- BEGIN:turborepo-agent-rules -->
+9 -111
View File
@@ -1,107 +1,5 @@
# Changelog
## [6.8.0](https://github.com/morluto/rea/compare/rea-agents-6.7.0...rea-agents-6.8.0) (2026-10-25)
### Highlights
- Build local agent environments from network captures, browser scenarios, or a prompt, then serve derived reads and evidence-gated writes and compare them with fresh application captures.
- Inspect more shipped formats, including GPU code objects, BEAM, Python bytecode, Unity IL2CPP metadata, Godot PCK, PyInstaller, Node.js SEA, devicetree, and additional PE/ELF tables.
- Keep large investigations focused with retained summaries, selected fact queries, configurable recovery limits, and row-oriented CLI exports. Complete Evidence remains available.
### ⚠ BREAKING CHANGES
- **Function analysis:** `analyze_function` now defaults to a retained summary. Set `detail: "complete"` for the previous inline dossier; the CLI retains its complete default and requires `--evidence_output` when selecting a summary. [#5660](https://github.com/morluto/rea/pull/5660). Thanks @CryptoJones, @gensokyo-dev, @RishTheFish21.
- **Accessibility:** Accept JSON numbers and booleans in node `value`, alongside strings and null. Update consumers that previously assumed text-only values. [cc90771](https://github.com/morluto/rea/commit/cc90771af14f045538651615147d8bf86b576549), [#6759](https://github.com/morluto/rea/pull/6759), [#6467](https://github.com/morluto/rea/issues/6467). Thanks @coygeek, @gensokyo-dev, @mvanhorn.
- **JSON Lines:** `--format jsonl` now emits object-collection rows followed by a `$rea: "jsonl/v1"`, `type: "metadata"` record; scalar selections and unsupported reports are refused. Parse control records separately, select a single supported collection when necessary, and use `--format json` for the previous whole-document output. See [JSON Lines framing](https://github.com/morluto/rea/blob/rea-agents-6.8.0/docs/cli.md#json-lines-record-output). [#5629](https://github.com/morluto/rea/pull/5629), [#5487](https://github.com/morluto/rea/issues/5487). Thanks @gensokyo-dev, @jayykernel, @vijaykiran06.
- **Interface Builder:** Keyed-archive graph identities use decimal table UIDs instead of authored `objectID`/`object-id`. Treat authored IDs as attributes and update persisted graph lookups; ibtool and native NIBArchive identities keep their source semantics. [6fbe67d](https://github.com/morluto/rea/commit/6fbe67d66e3d716db69929503a3a232a7ada0860).
- **Frida:** `instrument-with-frida` JSON returns the canonical Evidence directly. Read `normalized_result.cleanup_error` instead of the former top-level wrapper field. [48f700e](https://github.com/morluto/rea/commit/48f700ee35e0988ed973e5486dc9db33f2a5d944).
- **Reconstruction proof:** ID-array proof records no longer establish verified or ready ledgers. Supply implementation/revision bindings and individual claim outcomes; handle inapplicable, unavailable, and failed proof diagnostics. [f8b65e7](https://github.com/morluto/rea/commit/f8b65e78723b1e2169f94dd1226198ea3139707e).
- **Mach-O imports:** Handle a null import `name` when identity is ambiguous; use `raw_name` and `name_status` to distinguish observed expressions from exact symbols. [5d0d243](https://github.com/morluto/rea/commit/5d0d243a45c9132e1aff75d454ab53d86a74cc31), [169512a](https://github.com/morluto/rea/commit/169512a35bb015ff209168fb8bc1c9b31a2c678b).
- **Node.js:** Use a runtime satisfying `^22.23.4 || ^24.22.1 || >=26.12.1`; Node 23 and 25 are unsupported. Upgrade Node before installing or updating REA; the installer does not upgrade it. [#6315](https://github.com/morluto/rea/pull/6315), [#6283](https://github.com/morluto/rea/issues/6283). Thanks @GamerDubz, @gensokyo-dev, @ShikamaruNaaraa.
- **Browser storage:** Historical hashed storage remains readable, but comparisons stay unknown until both captures are recaptured in the new representation. [#6341](https://github.com/morluto/rea/pull/6341), [#6289](https://github.com/morluto/rea/issues/6289). Thanks @coygeek, @gongfpp.
- **Environment serving:** `environment-serve` serves derived state by default, including write tools whose effects are confirmed by recorded evidence. Use `--replay-only` for stateless replay of manifest tools. Read stop-summary `state` instead of `state_evidence_id`. [#5351](https://github.com/morluto/rea/pull/5351), [#6158](https://github.com/morluto/rea/pull/6158). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
- **MCP schemas:** Input advertisements default to compact. Set `REA_MCP_INPUT_SCHEMA_PROFILE=full` for complete advertised guidance and nested constraints. [#5975](https://github.com/morluto/rea/pull/5975). Thanks @gensokyo-dev.
- **Browser console arguments:** Accept null `argument_types` and `text_capture.status: "unavailable"` when the protocol argument inventory is omitted or malformed; preserve that uncertainty. [#6016](https://github.com/morluto/rea/pull/6016), [#5934](https://github.com/morluto/rea/issues/5934). Thanks @coygeek, @kaoru0822-kitauji.
- **Reconstruction summaries:** `summary.by_evidence_authority` is keyed by Evidence authority; read the former source-state counts from required `summary.by_source_state`. Regenerate ledgers whose closure digests depended on the old projection. [#5875](https://github.com/morluto/rea/pull/5875), [#5838](https://github.com/morluto/rea/issues/5838). Thanks @coygeek, @kaoru0822-kitauji.
- **Swift bindings:** Conformance `type_name` and `protocol_name` may be null; consume typed `type_binding`/`protocol_binding` evidence. Replace the former Objective-C-specific import-binding type/schema with `MachoImportBinding`. [d0067ba](https://github.com/morluto/rea/commit/d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f).
- **WebSocket frames:** Accept null opcodes and preserve unknown frame facts; numeric opcodes are restricted to the four-bit protocol range. [0bc2b42](https://github.com/morluto/rea/commit/0bc2b42001f684beffcf47b70e507443b1a5d320).
- **Resource-only inspection:** Evidence subjects and executable digests may be null. Use `resource_root` and each resource's reported provenance instead of requiring an executable target. [6a6adf7](https://github.com/morluto/rea/commit/6a6adf703fa44100643947d20fbc670f86260b1b).
- **Browser timestamps:** Console and session-timeline timestamps may be null when the protocol supplies no finite/comparable time. Keep unknown time explicit in sorting and comparisons. [690fc51](https://github.com/morluto/rea/commit/690fc518540bae982038a74f47ef7781201785aa), [2a0410f](https://github.com/morluto/rea/commit/2a0410f1717b3126e8a95063d77dea66a597010a).
- **Reference sources:** Read failed root enumeration from `rootFailure` rather than a synthetic failed child at `"."`; preserve the top-level source-graph limitation. [6296fdf](https://github.com/morluto/rea/commit/6296fdf560f1ac4aa55985558edaa98300aacdbc).
- **Filesystem capture:** Accept `entry_unavailable` and `identity_changed` enumeration omission reasons in addition to the previous reason literals. [c85b44e](https://github.com/morluto/rea/commit/c85b44e1b14851c54e65327a4807f79ad5cd5a61). Thanks @rudrakshagarwal4200-ai.
- **APK locales:** Replace numeric legacy selectors such as `es-r419` with `es-419` or `b+es+419`. Two-letter regions remain supported. [29910d1](https://github.com/morluto/rea/commit/29910d16b98fe4b607a7aa15928789a73614b0b9). Thanks @mertkayacs.
- **XML plist scalars:** Recognize `$plist_type: "unknown"` and read `element`, `literal`, and `reason`; unresolved XML scalars use this wrapper instead of the former real/null unknown marker or malformed scalar prefixes. Supported extended UTC years preserve their timestamp. [3537e58](https://github.com/morluto/rea/commit/3537e58c1ab29309f06eabf50bc52adba9817a0c).
### Changes
- **Binary layouts:** Inspect ELF32 i386, little-endian ARM, and little- or big-endian MIPS `EXEC`/`DYN`/`REL` layouts on Linux x64 using a caller-supplied pwntools environment. [#7052](https://github.com/morluto/rea/pull/7052), [#6934](https://github.com/morluto/rea/issues/6934). Thanks @roblennon, @salmanalmutairi1.
- **Agent environments:** Project network captures and Evidence files into manifests, preserve template path pairs and positions, export AgentEnv bundles, and serve manifests as replay environments. [#4954](https://github.com/morluto/rea/pull/4954), [#5293](https://github.com/morluto/rea/pull/5293), [#5664](https://github.com/morluto/rea/pull/5664), [#4999](https://github.com/morluto/rea/pull/4999), [#4975](https://github.com/morluto/rea/pull/4975). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
- **Capture to environment:** Build environments from browser scenarios or a prompt, extract the initial observed state, and compare the environment with application probe captures. [#5182](https://github.com/morluto/rea/pull/5182), [#6946](https://github.com/morluto/rea/pull/6946), [#5319](https://github.com/morluto/rea/pull/5319), [#5699](https://github.com/morluto/rea/pull/5699). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
- **State-backed tools:** Render read filters, search, sorting and paging from state; apply confirmed write effects with broader state targets and recording-based write admission. [#5352](https://github.com/morluto/rea/pull/5352), [293832b](https://github.com/morluto/rea/commit/293832bafc9a054914ec514fe906dd4936c7a372), [#5800](https://github.com/morluto/rea/pull/5800), [#5787](https://github.com/morluto/rea/pull/5787), [#5600](https://github.com/morluto/rea/pull/5600), [5375ff0](https://github.com/morluto/rea/commit/5375ff0f9aadb4fd0d46333815aa0431e3df1010), [#6169](https://github.com/morluto/rea/pull/6169), [#6385](https://github.com/morluto/rea/pull/6385), [#6897](https://github.com/morluto/rea/pull/6897). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
- **State learning:** Model application-assigned values, preserve view-specific clock spellings, learn write clocks from scenario timing, derive fields, and learn creates across sorted arrays and referenced entities. [7ec9b6e](https://github.com/morluto/rea/commit/7ec9b6e56e1b3762285dca881db6d900a389ab9c), [#6136](https://github.com/morluto/rea/pull/6136), [957bcfa](https://github.com/morluto/rea/commit/957bcfa149c1a0593bb0aa2466cc56d1d4f62e7d), [#6138](https://github.com/morluto/rea/pull/6138), [#6967](https://github.com/morluto/rea/pull/6967), [#7025](https://github.com/morluto/rea/pull/7025), [#7034](https://github.com/morluto/rea/pull/7034). Thanks @earakely-scale.
- **Recorded writes:** Serve creates whose compatible read-backs add omitted fields or express timestamps at a different precision, and whole-object updates that regenerate observed slugs or modification times. Conflicting observations still withhold writes. State views and bundles carrying the new read-back data need REA 6.8.0 or later to serve. [#7080](https://github.com/morluto/rea/pull/7080). Thanks @earakely-scale.
- **Native investigations:** Page native inventory summaries, match functions across builds, export pseudocode to resumable directories, and select objdump disassembly by symbol. [#5378](https://github.com/morluto/rea/pull/5378), [#5390](https://github.com/morluto/rea/pull/5390), [#6170](https://github.com/morluto/rea/pull/6170), [#5171](https://github.com/morluto/rea/pull/5171), [#4249](https://github.com/morluto/rea/issues/4249), [#5343](https://github.com/morluto/rea/issues/5343), [#5159](https://github.com/morluto/rea/issues/5159). Thanks @coygeek, @CryptoJones, @gensokyo-dev, @kaoru0822-kitauji.
- **Engine workflows:** Bound a single Ghidra decompilation without closing its session, analyze GameCube/Wii DOL executables, and inspect static CUDA/HIP code objects. [#5662](https://github.com/morluto/rea/pull/5662), [#6240](https://github.com/morluto/rea/pull/6240), [#5813](https://github.com/morluto/rea/pull/5813), [#5340](https://github.com/morluto/rea/issues/5340), [#5345](https://github.com/morluto/rea/issues/5345). Thanks @arielfikru, @coygeek, @gensokyo-dev, @RishTheFish21.
- **PE/ELF:** Inspect export tables without an engine, PE imports/delay-loads, Authenticode certificates, TLS callbacks/load configuration, ELF notes, Linux kernel module metadata, and Zstandard section-name tables. [#6278](https://github.com/morluto/rea/pull/6278), [#6509](https://github.com/morluto/rea/pull/6509), [#4830](https://github.com/morluto/rea/pull/4830), [#7008](https://github.com/morluto/rea/pull/7008), [#4837](https://github.com/morluto/rea/pull/4837), [#7010](https://github.com/morluto/rea/pull/7010), [#6987](https://github.com/morluto/rea/pull/6987), [#4057](https://github.com/morluto/rea/issues/4057). Thanks @gensokyo-dev, @kaoru0822-kitauji, @mvanhorn, @Rinkia, @ShuaixinHuang.
- **Apple/native metadata:** Inspect IPS crash reports, decode Swift nominal declarations, and select universal Mach-O slices in Go/Rust metadata readers. [#6511](https://github.com/morluto/rea/pull/6511), [#6257](https://github.com/morluto/rea/pull/6257), [#6219](https://github.com/morluto/rea/pull/6219), [#5341](https://github.com/morluto/rea/issues/5341), [#4102](https://github.com/morluto/rea/issues/4102). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @mvanhorn, @Rinkia.
- **Packaged applications:** Triage desktop installation directories and inspect/extract Godot PCK, PyInstaller archives, and Node.js single-executable application blobs. [#5207](https://github.com/morluto/rea/pull/5207), [#3634](https://github.com/morluto/rea/pull/3634), [afa5da7](https://github.com/morluto/rea/commit/afa5da745e0b139e68029a96b776ae30e48ea2b2), [#2433](https://github.com/morluto/rea/pull/2433), [#7000](https://github.com/morluto/rea/pull/7000), [#4399](https://github.com/morluto/rea/issues/4399). Thanks @0xthearchitect, @coygeek, @gensokyo-dev, @mvanhorn.
- **Unity and managed code:** Inventory IL2CPP images/types/methods and metadata versions 24/27; read member names and literals, link native methods and generic instances, and export Ghidra seeds. Project managed call/field-access edges and page managed member summaries. [#3807](https://github.com/morluto/rea/pull/3807), [#5326](https://github.com/morluto/rea/pull/5326), [52d1a72](https://github.com/morluto/rea/commit/52d1a72af047a73b787b8935b032786a34966036), [#5327](https://github.com/morluto/rea/pull/5327), [#5785](https://github.com/morluto/rea/pull/5785), [#6260](https://github.com/morluto/rea/pull/6260), [#5167](https://github.com/morluto/rea/pull/5167), [#3784](https://github.com/morluto/rea/issues/3784), [#5112](https://github.com/morluto/rea/issues/5112), [#4056](https://github.com/morluto/rea/issues/4056), [#5148](https://github.com/morluto/rea/issues/5148). Thanks @8rulerstar, @alexlanghart, @elricfd, @gensokyo-dev, @jaelliot, @kaoru0822-kitauji, @Rinkia.
- **Android:** Inspect APK signing declarations and structured manifest components; report static-final DEX field values. [e93e3b1](https://github.com/morluto/rea/commit/e93e3b1e7d6e7e4f49c2de24b92452ec29e853f8), [#4158](https://github.com/morluto/rea/pull/4158), [#5025](https://github.com/morluto/rea/pull/5025), [#6981](https://github.com/morluto/rea/pull/6981), [#4537](https://github.com/morluto/rea/issues/4537), [#6397](https://github.com/morluto/rea/issues/6397). Thanks @Abelo9996, @BaaRai, @FerroLx, @gensokyo-dev, @sserrgo12.
- **Bytecode and firmware:** Inspect and query CPython pyc and BEAM facts, inspect flattened devicetrees, and attribute MMIO peripheral regions in native disassembly. [#4826](https://github.com/morluto/rea/pull/4826), [#6928](https://github.com/morluto/rea/pull/6928), [#4833](https://github.com/morluto/rea/pull/4833), [#6943](https://github.com/morluto/rea/pull/6943), [#7009](https://github.com/morluto/rea/pull/7009), [#5348](https://github.com/morluto/rea/pull/5348), [#6922](https://github.com/morluto/rea/issues/6922), [#6938](https://github.com/morluto/rea/issues/6938). Thanks @gensokyo-dev, @liziye627-design, @mvanhorn.
- **JavaScript:** Retain queryable static route inventories, link assignment values to writes, and accept source selections as a JSON array or file. [#6019](https://github.com/morluto/rea/pull/6019), [#6206](https://github.com/morluto/rea/pull/6206), [#5029](https://github.com/morluto/rea/pull/5029), [#4772](https://github.com/morluto/rea/issues/4772), [#6193](https://github.com/morluto/rea/issues/6193). Thanks @gensokyo-dev, @IvenKooLab, @jacobmalibiran, @JoshOsullivan-au, @kaoru0822-kitauji, @KJLavender.
- **Data inspection:** Decode selected protobuf/gRPC bodies, inspect GGUF/safetensors declarations, and produce byte-entropy profiles. [#5031](https://github.com/morluto/rea/pull/5031), [#5603](https://github.com/morluto/rea/pull/5603), [#2331](https://github.com/morluto/rea/pull/2331), [#4986](https://github.com/morluto/rea/issues/4986), [#3323](https://github.com/morluto/rea/issues/3323). Thanks @AayushJB03, @gensokyo-dev, @kaoru0822-kitauji, @KJLavender, @michaelkillgta, @mvanhorn.
- **CLI input/output:** Accept structured request documents on stdin and expose version/runtime output formats. Row-oriented JSONL exports use the migration above. [#7040](https://github.com/morluto/rea/pull/7040), [#5486](https://github.com/morluto/rea/issues/5486), [#6530](https://github.com/morluto/rea/pull/6530), [169512a](https://github.com/morluto/rea/commit/169512a35bb015ff209168fb8bc1c9b31a2c678b), [#5483](https://github.com/morluto/rea/issues/5483). Thanks @Abelo9996, @ahmed-husssain, @CassieFox23, @DeryFerd, @eco-null, @ezrabrilliant, @GamerDubz, @gensokyo-dev, @gongfpp, @hifzabuildsai, @jayykernel, @mhamzasakhi, @pablodiazjorge, @Plastic-time, @raidnouredden-ux, @SandstromPL, @ShuaixinHuang, @Sriram-PR, @XiaoBinGan.
- **MCP and setup:** Add opt-in Streamable HTTP and FreeBuff setup registration. FreeBuff support first appears in 6.8.0; installed native FreeBuff loading and skill discovery remain unverified. [#4955](https://github.com/morluto/rea/pull/4955), [#2736](https://github.com/morluto/rea/pull/2736), [#3243](https://github.com/morluto/rea/issues/3243). Thanks @0xLLVEL, @codebuff-team, @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
- **Recovery controls:** Adjust per-call WASM/SQLite/firmware deadlines, MCP startup budgets, regex matching deadlines/worker heap, capture decoder heap, ASAR header admission, Apple dispatch input, and the bridge request ceiling. MCP runtime limit recovery reports actionable settings. [#5798](https://github.com/morluto/rea/pull/5798), [#5818](https://github.com/morluto/rea/pull/5818), [aa53c06](https://github.com/morluto/rea/commit/aa53c065387c58b4189d82c47f1f3445fc505e30), [#5863](https://github.com/morluto/rea/pull/5863), [#6304](https://github.com/morluto/rea/pull/6304), [#6267](https://github.com/morluto/rea/pull/6267), [#6301](https://github.com/morluto/rea/pull/6301), [#6299](https://github.com/morluto/rea/pull/6299), [#6209](https://github.com/morluto/rea/pull/6209), [#6233](https://github.com/morluto/rea/pull/6233), [#6376](https://github.com/morluto/rea/pull/6376), [#7049](https://github.com/morluto/rea/pull/7049), [#5797](https://github.com/morluto/rea/issues/5797), [#5812](https://github.com/morluto/rea/issues/5812), [#5851](https://github.com/morluto/rea/issues/5851), [#6176](https://github.com/morluto/rea/issues/6176), [#3917](https://github.com/morluto/rea/issues/3917). Thanks @gensokyo-dev, @jaychou813, @kaoru0822-kitauji.
- **APK output:** Set `string_tables_only` or CLI `--string-tables-only` to omit duplicate primary strings and avoid their output-capacity charge. [7c458f0](https://github.com/morluto/rea/commit/7c458f04b9155707c8ba406f8b455b10016f1b99), [#4990](https://github.com/morluto/rea/pull/4990), [#4982](https://github.com/morluto/rea/issues/4982). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
- **Retained analysis:** Reuse successful validation of authenticated immutable native/imported-JavaScript parents and captures. Query retained Go build identity, Rust dependencies, JavaScript source-to-bundle mappings, and native export facts without repeating full output. [8f7d12e](https://github.com/morluto/rea/commit/8f7d12e30d63f3d3b4cfe2a90336362841438f56), [#4962](https://github.com/morluto/rea/pull/4962), [#5349](https://github.com/morluto/rea/pull/5349), [#5524](https://github.com/morluto/rea/pull/5524), [#5815](https://github.com/morluto/rea/pull/5815), [#6128](https://github.com/morluto/rea/pull/6128), [#6907](https://github.com/morluto/rea/pull/6907), [#4810](https://github.com/morluto/rea/issues/4810), [#5499](https://github.com/morluto/rea/issues/5499), [#5802](https://github.com/morluto/rea/issues/5802), [#6112](https://github.com/morluto/rea/issues/6112), [#6900](https://github.com/morluto/rea/issues/6900). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
- **Compact results:** Use retained Frida/GDB/objdump summaries, omit redundant Ghidra raw results, and defer the analysis runtime during MCP catalog discovery. [#5661](https://github.com/morluto/rea/pull/5661), [#5614](https://github.com/morluto/rea/pull/5614), [#5355](https://github.com/morluto/rea/pull/5355), [#4424](https://github.com/morluto/rea/pull/4424), [#5640](https://github.com/morluto/rea/issues/5640), [#5584](https://github.com/morluto/rea/issues/5584), [#5337](https://github.com/morluto/rea/issues/5337). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @leeguooooo.
- **Analysis work:** Reuse Hopper literal-string records, JavaScript workers and captured prefix selections, Interface Builder validation and plist scalar schemas; bound stable reads to one snapshot buffer and avoid decoding omitted GGUF strings. [#5017](https://github.com/morluto/rea/pull/5017), [#5819](https://github.com/morluto/rea/pull/5819), [31de177](https://github.com/morluto/rea/commit/31de177404461d122761d8243abe4fd24564f4da), [#5885](https://github.com/morluto/rea/pull/5885), [85e38ac](https://github.com/morluto/rea/commit/85e38ac199dff15e8c43343d9170db7e0e46d284), [#6966](https://github.com/morluto/rea/pull/6966), [#5781](https://github.com/morluto/rea/pull/5781), [#5010](https://github.com/morluto/rea/issues/5010), [#5855](https://github.com/morluto/rea/issues/5855), [#6954](https://github.com/morluto/rea/issues/6954), [#5774](https://github.com/morluto/rea/issues/5774). Thanks @gensokyo-dev, @kaoru0822-kitauji.
- **Native scheduling:** Batch modern IDA callee lookups, schedule decompilation reads on demand, index NativeAOT type closure, and retain call-path ancestry once per visited node. [0c6fed9](https://github.com/morluto/rea/commit/0c6fed90b74c75a9fd23d1ec1b510a7274f85887), [06fe7ee](https://github.com/morluto/rea/commit/06fe7eecbc8c387414b4c1aa0d88ad6314f54f0c), [ce61680](https://github.com/morluto/rea/commit/ce61680c04c95a9cdfad5335632a99542af26d37), [5e4ec0f](https://github.com/morluto/rea/commit/5e4ec0fa1b678aa3bf20704ec9110224ba8c1676).
- **Selected processing:** Prepare CLI positional schemas only for the selected command, separate entropy reads from report resolution, prune typed-array reference escapes, index state-write cutoffs, decode only selected replay query values, and skip payload text during WAT framing. [315b234](https://github.com/morluto/rea/commit/315b23403f0819967878cee418c3f9cb4b1a69d9), [#6950](https://github.com/morluto/rea/pull/6950), [#5139](https://github.com/morluto/rea/pull/5139), [#5350](https://github.com/morluto/rea/pull/5350), [96ea32b](https://github.com/morluto/rea/commit/96ea32b7e4e28d3c9cd79a1de4b8e1ba3b4244e9), [70835d3](https://github.com/morluto/rea/commit/70835d3d5f030deadf606e1c79c70a60883e50c9), [#5766](https://github.com/morluto/rea/pull/5766), [#6947](https://github.com/morluto/rea/issues/6947), [#5760](https://github.com/morluto/rea/issues/5760). Thanks @galleonli, @gensokyo-dev, @kaoru0822-kitauji.
### Fixes
- **IPS report fields:** Preserve the recorded terminating process and PID, termination flags, parent PID, and build/platform metadata. [#7079](https://github.com/morluto/rea/pull/7079), [#7078](https://github.com/morluto/rea/issues/7078). Thanks @rodricksz4h5.
- **Ghidra language selection:** Accept stock language IDs containing `LEBE` or slash-separated processor names, and report conflicting DOL language overrides as configuration errors naming `REA_GHIDRA_LANGUAGE_ID`. [#7070](https://github.com/morluto/rea/pull/7070), [#7072](https://github.com/morluto/rea/issues/7072), [#7081](https://github.com/morluto/rea/pull/7081), [#7082](https://github.com/morluto/rea/issues/7082). Thanks @Adolanium.
- **Setup and Hopper:** Preserve the Linux embedding runtime, supported x86_64 host restriction, readiness-based provider choices, orderly demo-helper shutdown, and installed apt dependency versions. Keep cleanup and skill publication ownership through retries. [#5041](https://github.com/morluto/rea/pull/5041), [#3422](https://github.com/morluto/rea/pull/3422), [f5d1012](https://github.com/morluto/rea/commit/f5d1012fabfe99af82523c3078ce9d93eb3780c1), [#4966](https://github.com/morluto/rea/pull/4966), [#6974](https://github.com/morluto/rea/pull/6974), [211e353](https://github.com/morluto/rea/commit/211e3533b34f81ae0910ad4f53f2a3d3c8587d3e), [#6920](https://github.com/morluto/rea/pull/6920), [#5015](https://github.com/morluto/rea/issues/5015), [#3419](https://github.com/morluto/rea/issues/3419), [#4921](https://github.com/morluto/rea/issues/4921), [#6975](https://github.com/morluto/rea/issues/6975). Thanks @Abel-I-Akhigbe, @Adolanium, @DavidFB-creator, @DeryFerd, @gensokyo-dev, @kaoru0822-kitauji, @sx4im.
- **Client setup:** Treat empty APPDATA as unset, distinguish MSIX Claude Desktop process views, preserve aliased Hermes server tables, reuse client dialect/alias bindings, and materialize the pinned package-runner cache before the first start. [#5126](https://github.com/morluto/rea/pull/5126), [#4903](https://github.com/morluto/rea/pull/4903), [#3785](https://github.com/morluto/rea/pull/3785), [b7eeded](https://github.com/morluto/rea/commit/b7eededf5af5061a834f93f57f2c888acf870344), [#7032](https://github.com/morluto/rea/pull/7032), [#5104](https://github.com/morluto/rea/issues/5104), [#4278](https://github.com/morluto/rea/issues/4278), [#3320](https://github.com/morluto/rea/issues/3320), [#7027](https://github.com/morluto/rea/issues/7027). Thanks @CmsChase, @coygeek, @myso-kr, @pablodiazjorge, @vaidehi-snk, @wxj006007.
- **Target admission:** Preserve thin arm64_32 Mach-O support, distinguish JVM identity before FAT admission, validate FAT format/CPU identity, preserve original DOS header padding, and reject NULs in local objdump/rizin paths. [#5788](https://github.com/morluto/rea/pull/5788), [6843dbe](https://github.com/morluto/rea/commit/6843dbea9bce4709ee2e8bf46c38f0bbf640716b), [497037e](https://github.com/morluto/rea/commit/497037e6e3975cbb37cfd89ea0ad300faa2a4769), [#7013](https://github.com/morluto/rea/pull/7013), [#3600](https://github.com/morluto/rea/pull/3600), [#5713](https://github.com/morluto/rea/issues/5713), [#6510](https://github.com/morluto/rea/issues/6510). Thanks @coygeek, @dhruvac29, @kaoru0822-kitauji, @Lew1sWong, @qorvr.
- **Mach-O and dylibs:** Preserve container framing, facts when dyld rejects a missing UUID, verified-signature snapshot identity, unsigned diagnostics, absolute paths inside the selected root, legacy platform rules, lazy-load payloads, and reasons for rejecting present dylibs. [cf3288d](https://github.com/morluto/rea/commit/cf3288d8cf7a09685acf89103b36c32359be56fa), [2b263a7](https://github.com/morluto/rea/commit/2b263a7fe9e9d4c185c7d6af6b4e67b21a73af30), [eb7acc7](https://github.com/morluto/rea/commit/eb7acc70bf06f119e8639e30a78081d5ba82b0bf), [1502cfb](https://github.com/morluto/rea/commit/1502cfbc8ca0d1d1d86eebf334018dfb418bbac0), [90062c0](https://github.com/morluto/rea/commit/90062c092228dea9d33fadc0d7e1b968105b8b2c), [#5650](https://github.com/morluto/rea/pull/5650), [3b67b36](https://github.com/morluto/rea/commit/3b67b362aa2cd79eac5af57d0965e81a7d0b8c2b), [#5551](https://github.com/morluto/rea/issues/5551). Thanks @coygeek.
- **Native facts:** Admit direct dispatch-metadata providers; preserve record-local Objective-C omissions, snapshot profiles, class-import binding identity, stripped class labels, version-like names, and qualified Swift protocol identities. [50b6975](https://github.com/morluto/rea/commit/50b6975d861069e2c6cfa202780621449f863802), [9eec57b](https://github.com/morluto/rea/commit/9eec57b469087fe9bff3d29dcc02f47074159e91), [0ba4d6f](https://github.com/morluto/rea/commit/0ba4d6f43ad610dacc861a1548be1a5c4712cd61), [4d7de00](https://github.com/morluto/rea/commit/4d7de00d4cc9a95ec2b7a3123b04e01db2daafa0), [22c84ce](https://github.com/morluto/rea/commit/22c84ce64aa6c7601aa81af86d4117b5a4c2feac), [4852c83](https://github.com/morluto/rea/commit/4852c830af1e3b9df437235b7a8fa8970a6d1c59), [aa9043f](https://github.com/morluto/rea/commit/aa9043f6b61c453e4735795cfe6a559bf6b6cc16).
- **Apple resources:** Preserve independent plist scalars, processing instructions, nonstandard table widths, unsupported NIB connector coverage, and strict IPS JSON surrogate validation; classify Interface Builder objects by their declared superclass chains. [fd728f2](https://github.com/morluto/rea/commit/fd728f271e7de7bcfa63890b573715720d5f948c), [#5806](https://github.com/morluto/rea/pull/5806), [4526862](https://github.com/morluto/rea/commit/4526862728af01f1135714079c0d8a8c754eb364), [14cd78c](https://github.com/morluto/rea/commit/14cd78c3224edcf79fca152915077112f547b9ed), [#6990](https://github.com/morluto/rea/pull/6990), [#5805](https://github.com/morluto/rea/issues/5805), [#5228](https://github.com/morluto/rea/pull/5228), [#5218](https://github.com/morluto/rea/issues/5218). Thanks @coygeek, @DeryFerd, @kaoru0822-kitauji, @Mangolycheematcha, @mhamzasakhi, @Shreyash021104.
- **Archives and files:** Complete positional ZIP reads, preserve bounded ASAR signatures, report stable symlink loops as resolution failures, retain target applicability, and report unknown traversal coverage. Retain reported integrity contradictions for members present in only one compared artifact. [#4414](https://github.com/morluto/rea/pull/4414), [e8da7e1](https://github.com/morluto/rea/commit/e8da7e1edd075feec0e6beedec93d5645fed8125), [#5571](https://github.com/morluto/rea/pull/5571), [#5763](https://github.com/morluto/rea/pull/5763), [#5867](https://github.com/morluto/rea/pull/5867), [#5521](https://github.com/morluto/rea/issues/5521), [#5731](https://github.com/morluto/rea/issues/5731), [#5837](https://github.com/morluto/rea/issues/5837), [#7076](https://github.com/morluto/rea/pull/7076). Thanks @Afloat16, @coygeek, @jdotc1, @kaoru0822-kitauji, @WenyuChiou.
- **Extraction and file ownership:** Restrict destination-case recovery to owned collisions, retain physical-read failures through late cancellation/cleanup, preserve cleanup capabilities for retry, and retain independent firmware extraction failures. [932fbb7](https://github.com/morluto/rea/commit/932fbb7d84a9a995b9aee3c1be7da05a8c4d7bbb), [9499667](https://github.com/morluto/rea/commit/949966712e68d8857a50a165ecddf04c2d7e6d3b), [56679dd](https://github.com/morluto/rea/commit/56679dd2ae31dda58e181f8bf99775014f7bb82b), [b3c912a](https://github.com/morluto/rea/commit/b3c912ab79ad147a37b6cfbe1552a1a5eecec629), [7298bd0](https://github.com/morluto/rea/commit/7298bd08d134860b84b4073f6703021916a5a4c5).
- **Processes and cancellation:** Retain partial samples and shutdown gaps, unresolved descendants and caller abort reasons; bind cleanup errors to their owners, propagate comparison contradictions, and report no normal exit code for signalled targets. [ba866a2](https://github.com/morluto/rea/commit/ba866a26ecbe179a717e746cbc0d1605ae616ad5), [#5270](https://github.com/morluto/rea/pull/5270), [#6144](https://github.com/morluto/rea/pull/6144), [a721c42](https://github.com/morluto/rea/commit/a721c42c07dc674c32ab939627355b8a1bee97a5), [#6298](https://github.com/morluto/rea/pull/6298), [#6486](https://github.com/morluto/rea/pull/6486), [#5935](https://github.com/morluto/rea/issues/5935), [#5944](https://github.com/morluto/rea/issues/5944), [#6100](https://github.com/morluto/rea/issues/6100), [#6101](https://github.com/morluto/rea/issues/6101), [#6110](https://github.com/morluto/rea/issues/6110), [#6113](https://github.com/morluto/rea/issues/6113), [#6121](https://github.com/morluto/rea/issues/6121), [#6131](https://github.com/morluto/rea/issues/6131), [#6132](https://github.com/morluto/rea/issues/6132), [#6218](https://github.com/morluto/rea/issues/6218), [#6445](https://github.com/morluto/rea/issues/6445). Thanks @agorevski, @coygeek, @gongfpp, @kwakhyun, @mhamzasakhi.
- **Provider cleanup:** Preserve Windows native handles until close is confirmed, native/Inspector/Electron cleanup ownership, Kuna probe identity and cleanup, IDA registration configuration, and Hopper preparation ownership. [#5772](https://github.com/morluto/rea/pull/5772), [87ab054](https://github.com/morluto/rea/commit/87ab054ea384b3fcbae2998b1f94685edcd5f624), [ff78165](https://github.com/morluto/rea/commit/ff78165978de9dbcf9fa16c2acf74bf85a69dc44), [04ae07a](https://github.com/morluto/rea/commit/04ae07ab8c9ae1a6697b6ad15489bbde515da8b5), [dd0015a](https://github.com/morluto/rea/commit/dd0015aaf156c5701ef9598bcc6c885e6b8da803), [b231c28](https://github.com/morluto/rea/commit/b231c28cf64c2f799ada99343f8f21e0e6acb15e), [#4505](https://github.com/morluto/rea/issues/4505), [#5616](https://github.com/morluto/rea/issues/5616).
- **Provider diagnostics:** Give Ghidra restart recovery, stable Homebrew aliases, unsupported-host results for Linux-only integrations, correct IDA regex/readiness handling, and clear diagnostics when Windows refuses a verified native addon. [#5356](https://github.com/morluto/rea/pull/5356), [7c6142e](https://github.com/morluto/rea/commit/7c6142efdc19a42c148d0e8c96b69d18e63c7f87), [#5793](https://github.com/morluto/rea/pull/5793), [#4862](https://github.com/morluto/rea/pull/4862), [99ccfb6](https://github.com/morluto/rea/commit/99ccfb6e709ac4bc0afe176315280abde9ebcb9f), [#6832](https://github.com/morluto/rea/pull/6832), [#5730](https://github.com/morluto/rea/issues/5730), [#4860](https://github.com/morluto/rea/issues/4860), [#5723](https://github.com/morluto/rea/issues/5723). Thanks @coygeek, @dark13b, @gensokyo-dev, @jgt87, @jpmorris41488, @Lew1sWong.
- **MCP results:** Preserve protocol envelopes on oversized errors, enforce and advertise absolute Evidence paths, retain typed output-filter errors and selected JSON paths, and give client/provider timeout recovery. Compact advertisements now normalize reported enum and union shapes for strict gateway translation. [562dab0](https://github.com/morluto/rea/commit/562dab00c5cbf76a9b2695bad93792eccdc9a49d), [3cf6206](https://github.com/morluto/rea/commit/3cf6206614e65b5ebb6284a5709090dfb8f9aaee), [0e83d92](https://github.com/morluto/rea/commit/0e83d92c583ae36afe00b2c23dd2a0f1d025cbca), [4714c7e](https://github.com/morluto/rea/commit/4714c7eaa3d1d8b6a5df0aff586e844b9d9ab06d), [1bfa60a](https://github.com/morluto/rea/commit/1bfa60ac20e1a344bcf3132c08d15b6860b05c61), [#5608](https://github.com/morluto/rea/pull/5608), [#3806](https://github.com/morluto/rea/pull/3806), [#5598](https://github.com/morluto/rea/issues/5598), [#3800](https://github.com/morluto/rea/issues/3800). Thanks @gensokyo-dev, @Hahaa909, @wang1xiang.
- **Browser lifecycles:** Retain historical outcomes through cleanup, stop late navigation before session release, stop expired work, observe aborted fetch settlement, and mark later scenario actions skipped after failure or cancellation. [2e5171e](https://github.com/morluto/rea/commit/2e5171e391b83f8a467d0861fd452c1f78621fd8), [599fdac](https://github.com/morluto/rea/commit/599fdac029912f96c511f859660483405d2de26f), [b2fad36](https://github.com/morluto/rea/commit/b2fad362f2b37f81dc9a6f784f4521e4ca24d41e), [13a8640](https://github.com/morluto/rea/commit/13a864020414577006dce58d7fdaa877ddd0cb52), [#6023](https://github.com/morluto/rea/pull/6023), [#5943](https://github.com/morluto/rea/issues/5943). Thanks @coygeek, @kaoru0822-kitauji.
- **Browser comparisons:** Preserve frame identity, default origin scope, unknown body/shape coverage, correspondence and uncertainty, and independent retained-observation budgets. [#4040](https://github.com/morluto/rea/pull/4040), [#4413](https://github.com/morluto/rea/pull/4413), [#4476](https://github.com/morluto/rea/pull/4476), [a0c22ff](https://github.com/morluto/rea/commit/a0c22ff0125b8af7612db54d3708413e2bfecb25), [a0fcc60](https://github.com/morluto/rea/commit/a0fcc60d89eb0cc09b08181eb9f961acfd07d8e9), [#3878](https://github.com/morluto/rea/issues/3878), [#3910](https://github.com/morluto/rea/issues/3910), [#3941](https://github.com/morluto/rea/issues/3941), [#4385](https://github.com/morluto/rea/issues/4385), [#4477](https://github.com/morluto/rea/issues/4477). Thanks @Adolanium, @coygeek, @eco-null, @gensokyo-dev, @kaoru0822-kitauji, @XY-code1.
- **Electron observations:** Preserve listener callback ownership and count registrations by source callsite; block trash calls in the active hook, retain window handles through cancelled selection, and preserve cancellation during finalization. [dfe8378](https://github.com/morluto/rea/commit/dfe8378cf5a727e3d93d47a5986233175c6ef898), [5064007](https://github.com/morluto/rea/commit/506400748eb1d6c674f0cf51330073929c45f939), [816a9dc](https://github.com/morluto/rea/commit/816a9dc98886e6d824cd8ae88b68731b161d5dda), [1707d17](https://github.com/morluto/rea/commit/1707d171d750955bf6387e028e378183b360ffe3), [4e2a566](https://github.com/morluto/rea/commit/4e2a566d6c7e053ecbbee1b684311ed31ea163bf).
- **JavaScript source maps:** Keep unmapped and missing mappings unknown, preserve partial indexed-map facts and section paths, propagate preparation capability errors, and retain observed worker failures through cancellation. [a4dbb36](https://github.com/morluto/rea/commit/a4dbb36d8ea83e9f94dd289ac1a2d220d2c72c9b), [2b2e362](https://github.com/morluto/rea/commit/2b2e362a00fc5ba2f93fa6c62e2d8780072d1b4b), [498a667](https://github.com/morluto/rea/commit/498a667234b0140c1cfa43c819b35454ab11d14e), [#6149](https://github.com/morluto/rea/pull/6149), [#6214](https://github.com/morluto/rea/pull/6214), [f01dcca](https://github.com/morluto/rea/commit/f01dccab77f712fb4ac936d0a6e92a70d12f0789), [#6082](https://github.com/morluto/rea/issues/6082), [#6061](https://github.com/morluto/rea/issues/6061). Thanks @coygeek, @diyasharma05, @kaoru0822-kitauji.
- **JavaScript semantics:** Preserve overflowing timer uncertainty, unavailable-scope comparisons, directory-form URL/ESM path intent, and artifact confinement. Replaced globals lose built-in effects; erased imports no longer become runtime bindings. Treat a known truthy `fork()` `silent` option as piped stdio unless a string/array `stdio` override takes precedence; unresolved options stay dynamic. [#3456](https://github.com/morluto/rea/pull/3456), [8d198d6](https://github.com/morluto/rea/commit/8d198d6ec707402d75276f516e0090d71767b7e4), [#4646](https://github.com/morluto/rea/pull/4646), [#2700](https://github.com/morluto/rea/pull/2700), [#4565](https://github.com/morluto/rea/pull/4565), [#6994](https://github.com/morluto/rea/pull/6994), [#3400](https://github.com/morluto/rea/issues/3400), [#3882](https://github.com/morluto/rea/issues/3882), [#3367](https://github.com/morluto/rea/issues/3367), [#7083](https://github.com/morluto/rea/pull/7083), [#7084](https://github.com/morluto/rea/issues/7084). Thanks @Adolanium, @amyanger, @coygeek, @gensokyo-dev, @HawkForge, @rohitg00, @viniciusdsandrade.
- **JavaScript routes and imports:** Treat path data as route candidates, preserve route flag and lexical RPC identities, package-main trailing-slash selection, catch aliases, and factory parameter ownership. [#6360](https://github.com/morluto/rea/pull/6360), [38ecc02](https://github.com/morluto/rea/commit/38ecc0248e1993d25a2f7dd350024484cce797bb), [#4219](https://github.com/morluto/rea/pull/4219), [21097b1](https://github.com/morluto/rea/commit/21097b192b1ab94647e3446b1a2a4dd00e924d5e), [f774dc4](https://github.com/morluto/rea/commit/f774dc420b3befe2b8e68971b4116da2d62fc800), [#6349](https://github.com/morluto/rea/issues/6349), [#4189](https://github.com/morluto/rea/issues/4189). Thanks @coygeek, @hemv-857, @kaoru0822-kitauji, @Nikhi00718.
- **Managed/IL2CPP:** Preserve extension-only scope uncertainty and type discovery after member limits, nested names, ReadyToRun/declaration facts, selected type scope, valid shared-suffix strings, and truncated IL2CPP metadata prefixes. [#5674](https://github.com/morluto/rea/pull/5674), [#5312](https://github.com/morluto/rea/pull/5312), [87fa1b6](https://github.com/morluto/rea/commit/87fa1b613d2b6bda1722855a00f1390f27731b64), [bc2d494](https://github.com/morluto/rea/commit/bc2d494030efd1ccabaeaa0bd49226aa87c378a2), [1f0eb22](https://github.com/morluto/rea/commit/1f0eb228e50908ee2a2aaa72d329178ae1a2cdbd), [4c5fae5](https://github.com/morluto/rea/commit/4c5fae5179dd54c4a0d427cd2a115b9c887e3841), [b3aadaf](https://github.com/morluto/rea/commit/b3aadaf474e59b33e5ff465236008a67eccefb87), [#5467](https://github.com/morluto/rea/issues/5467), [#5146](https://github.com/morluto/rea/issues/5146). Thanks @coygeek, @eco-null, @elricfd, @umi008.
- **Format boundaries:** Preserve independent facts across malformed inputs in model, GPU, WASM, Rust, Android, and SQLite inspection; report optional metadata warnings without discarding core results. [eb41f35](https://github.com/morluto/rea/commit/eb41f358cf83d5626f09de68332f9718f9aa6482), [#5789](https://github.com/morluto/rea/pull/5789), [00a9e95](https://github.com/morluto/rea/commit/00a9e95de91a4bd4d2a41f8aee704a53fa575750), [fc63fe8](https://github.com/morluto/rea/commit/fc63fe8682654086261540dcf3c59afc36c7d58f), [39a91d4](https://github.com/morluto/rea/commit/39a91d4db96ca13224c25936330ea84fe78970e5), [6c6e2ad](https://github.com/morluto/rea/commit/6c6e2ad62b20f810b54f5a567e66fc55241514b2), [929a0a7](https://github.com/morluto/rea/commit/929a0a76c870d9991e015dde58308fd9df0280b8), [d137a4c](https://github.com/morluto/rea/commit/d137a4cdde0049de967f1d5cd429ed9e1f7d0a10), [ef475fc](https://github.com/morluto/rea/commit/ef475fcf8b0601c61052b7091568ba574546417d), [#5749](https://github.com/morluto/rea/issues/5749). Thanks @coygeek, @gensokyo-dev.
- **Capture decoding:** Preserve malformed protobuf trailer facts and wire/schema identity, HTTP/2 URI and UTF-8 failure facts, WebMCP property coverage, and offline capture availability without a browser provider. Report invalid record ordinals as selection errors with actual bounds where available, and trim surrounding SP/HTAB from gRPC-web trailer values while preserving internal whitespace. [9cf7a7c](https://github.com/morluto/rea/commit/9cf7a7cb0d3c72a1121fde280cb11bf911af8055), [b3c3b21](https://github.com/morluto/rea/commit/b3c3b21e43cab4e71b460d08829d8c6e4e9b9c21), [4cb1f1d](https://github.com/morluto/rea/commit/4cb1f1d820df4498ee464bcf72a5f47d40cf8dab), [51e5938](https://github.com/morluto/rea/commit/51e593877642ab42fb5bf25812f4a4110ee7ac26), [d23be5c](https://github.com/morluto/rea/commit/d23be5c0898c39c9f0c54e2e0b43ee746d92d316), [#6725](https://github.com/morluto/rea/pull/6725), [#3947](https://github.com/morluto/rea/issues/3947), [#7067](https://github.com/morluto/rea/pull/7067), [#7068](https://github.com/morluto/rea/issues/7068), [#7069](https://github.com/morluto/rea/pull/7069), [#7071](https://github.com/morluto/rea/issues/7071). Thanks @Adolanium, @coygeek, @Sriram-PR.
- **Evidence and errors:** Authenticate ledger Evidence before committing; preserve parent-schema diagnostics, canonical correction details and owned cleanup outcomes; identify the rejected function-Evidence side/kind. Fixed allowances now name their actual limits. [0407bbf](https://github.com/morluto/rea/commit/0407bbfbe65e8fa475e8bb0c78c30170e2c2f55e), [#6361](https://github.com/morluto/rea/pull/6361), [1629266](https://github.com/morluto/rea/commit/1629266e3c08c90fed6bba5383528af7986a12b7), [#6969](https://github.com/morluto/rea/pull/6969), [#5860](https://github.com/morluto/rea/pull/5860), [#6034](https://github.com/morluto/rea/issues/6034), [#5932](https://github.com/morluto/rea/issues/5932), [#5847](https://github.com/morluto/rea/issues/5847). Thanks @AyoubElleuch, @coygeek, @dhruvac29, @kaoru0822-kitauji.
- **ADB and firmware recovery:** Report an unparsed push checksum as unverified, normalize absent installer-package sentinels, describe extraction-relative Unblob links, and avoid recommending a budget increase at its fixed ceiling. [#6010](https://github.com/morluto/rea/pull/6010), [#4073](https://github.com/morluto/rea/pull/4073), [#4573](https://github.com/morluto/rea/pull/4573), [#6971](https://github.com/morluto/rea/pull/6971), [#5924](https://github.com/morluto/rea/issues/5924), [#4175](https://github.com/morluto/rea/issues/4175). Thanks @coygeek, @Ganesh2325, @kaoru0822-kitauji, @Rinkia, @XiaoBinGan, @yankikucuk.
- **CLI formatting:** Share Markdown report formatting, exclude inherited row values, and preserve arrays of empty objects as JSON. Render Markdown table headers and cells literally while keeping intraword underscores readable. [b784912](https://github.com/morluto/rea/commit/b784912811fd10ac9dc425696dc0e9af632afa34), [#6984](https://github.com/morluto/rea/pull/6984), [#7058](https://github.com/morluto/rea/pull/7058), [#7057](https://github.com/morluto/rea/issues/7057), [#7073](https://github.com/morluto/rea/pull/7073). Thanks @Plastic-time, @ShuaixinHuang, @viniciusdsandrade.
- **Format identity:** Preserve Python marshal scalar/decode limits, empty optional BEAM chunks, PE resource branch identities and certificate alignment, unsupported compressed ELF notes, and codec-framed GPU bundle extents. [0603223](https://github.com/morluto/rea/commit/06032237c6755e3dd5f9a8db732c89958b238cc7), [c46064c](https://github.com/morluto/rea/commit/c46064c3f6eaba121c5de7b17390bfdecdd132ef), [66671e8](https://github.com/morluto/rea/commit/66671e874888e98f7e2b27e68a7c6fb320bbb827), [03077d5](https://github.com/morluto/rea/commit/03077d54afc49613db4e38f70c506bc60f5223aa), [#6721](https://github.com/morluto/rea/pull/6721), [9fb8c11](https://github.com/morluto/rea/commit/9fb8c114a03ad60e57c11586967c6df845a3f97e), [f6a1e2c](https://github.com/morluto/rea/commit/f6a1e2c892dc23c7583c3d03af9edb01632ff5a5), [#6694](https://github.com/morluto/rea/issues/6694). Thanks @coygeek, @Deepak-Kambala.
- **WASM and APK metadata:** Preserve ambiguous WASM identity and partial facts, glue URL query/fragment text across line terminators, APK signer identity, and raw issuer identity when comparison rules are unknown. [af8f0ba](https://github.com/morluto/rea/commit/af8f0badf64fd3546392b6aa4625777904b2f051), [#6802](https://github.com/morluto/rea/pull/6802), [f783f0f](https://github.com/morluto/rea/commit/f783f0f1e82472cdddd4de2156d974f729e332a6), [115a78f](https://github.com/morluto/rea/commit/115a78fe241276c34917b496621954c5f29bfc15), [#6639](https://github.com/morluto/rea/issues/6639). Thanks @coygeek, @gensokyo-dev, @gongfpp.
- **Source identity:** Preserve Next route spelling, module grammar and loader request identity, directory package-main behavior, accessor effects, exact source targets, and resolver-discovery versus module-loading distinctions. [ddd7ac7](https://github.com/morluto/rea/commit/ddd7ac7b846b48191ead1f7dcd0e3bd735468fa8), [9974512](https://github.com/morluto/rea/commit/99745126009b15060d71c38809418858a07d5afa), [f2e5fd1](https://github.com/morluto/rea/commit/f2e5fd148a85fa737302dd194a33172c4561093e), [baf1678](https://github.com/morluto/rea/commit/baf16784af3592534687052431c145daef55cfd1), [6e324b2](https://github.com/morluto/rea/commit/6e324b224405f2befd953234f6dc210cf72ad50a), [75daf0c](https://github.com/morluto/rea/commit/75daf0c7df108f04ab41f34aec1a31d4a7ceb322), [f91b7f8](https://github.com/morluto/rea/commit/f91b7f8009c9af27a5e0148e3b07d20bfe3b7a2a).
- **Graphs and selected facts:** Preserve JavaScript graph omissions/version facts, Electron native-loader facts, web JSON identity/coverage, module-path selection, feature-trace qualifications, selected native paths, endpoint context, and mixed Swift demangler outcomes. [30db499](https://github.com/morluto/rea/commit/30db499db310f073d9bf3d3acfd77985e1cb7e2e), [5a7bddb](https://github.com/morluto/rea/commit/5a7bddb3aeb0345854e76b79d529aed1c40e2054), [79805f2](https://github.com/morluto/rea/commit/79805f244a1ec0850f3109929168a9c8ebb1b908), [f719dc4](https://github.com/morluto/rea/commit/f719dc46c51a55b3926eec9659d06e7dfb95132b), [7784fdb](https://github.com/morluto/rea/commit/7784fdbad78f8b384d513f0e7dae8e983cd48ef8), [76500f1](https://github.com/morluto/rea/commit/76500f12b54e07b1ea700a403b2593890d63b42a), [676d29f](https://github.com/morluto/rea/commit/676d29f9eef794cea668f5f09f0ff510c848a513), [edf7b3d](https://github.com/morluto/rea/commit/edf7b3d32c3248f221e27f7ce5a3d34e8590285b), [93de7bc](https://github.com/morluto/rea/commit/93de7bc7e324a761162a31625b838ba0d4e3824d).
- **Managed boundaries:** Honor native platform symbol lookup, consume resolved IL2CPP ELF declarations, and attribute malformed comparison inputs to the supplied argument. [64fa1db](https://github.com/morluto/rea/commit/64fa1db1309218283df19f3ee4dba61f5fbd2a67), [87745af](https://github.com/morluto/rea/commit/87745af52ab0ccd3a172c2534c516d41b0095035), [96ad94f](https://github.com/morluto/rea/commit/96ad94f69eda5d68c7335ff775b6927b1376fd2b).
- **Provider and worker recovery:** Preserve output loss after successful command exit, physical refusal diagnostics, bounded worker control framing, JavaScript worker-phase/budget attribution, admitted ambiguity choices, and selected EVM limiter availability. Scope objdump debug-link limitations to DWARF requests and keep process-capture preparation nonexecuting. [e639edf](https://github.com/morluto/rea/commit/e639edf6204c05285d96840478926592daf95876), [171b33b](https://github.com/morluto/rea/commit/171b33b2c4defc6cbea49545bedf4272b72d1ea2), [34563fc](https://github.com/morluto/rea/commit/34563fcc9bd92af30668820dcac5635e27181094), [3f9357b](https://github.com/morluto/rea/commit/3f9357b5990d166a38440f56f439452266650093), [cec8f1e](https://github.com/morluto/rea/commit/cec8f1e8f305d6332272457a6ee419d3c5fb6359), [a74d859](https://github.com/morluto/rea/commit/a74d859a19122fb8f1282b3632d320997b1caae0), [c284168](https://github.com/morluto/rea/commit/c284168c4b1cd214ce7f504d43da28528d82d603), [4fb936f](https://github.com/morluto/rea/commit/4fb936fe86bb7a07f94f819ae8510301f2ef69c9).
- **Setup and limits:** Retain unknown environment-budget facts, selected-skill/maintenance uncertainty, Hopper URL admission and bookmark verification, and listener failure recovery that matches the system cause. [a718037](https://github.com/morluto/rea/commit/a7180370332350bc0f998fbd643888221759570e), [d1fbebb](https://github.com/morluto/rea/commit/d1fbebb578538dc94fd5e376e6e1959218efa835), [60accad](https://github.com/morluto/rea/commit/60accad2de8f2862fe91901502fe26cc5390fb36), [fd7da77](https://github.com/morluto/rea/commit/fd7da77fc02dcfaf7d1bc9f2fb240d71cdd27c59), [0c440bf](https://github.com/morluto/rea/commit/0c440bfc07f3cb343e2e836d27cccaed539cfe9c).
- **State and Evidence:** Retain export tarball ownership and SQLite primary outcomes through cleanup retry, count emitted SQLite schema names across encodings, scope display refresh to matching snapshot bindings, and report retained-result validation failures consistently. Reconstruction uses one effective completion scope and kind-based labels when names are blank. [ab58edc](https://github.com/morluto/rea/commit/ab58edcdf50b4bd687900ee2dc362ecc5c3dce15), [1006714](https://github.com/morluto/rea/commit/100671427105791a5c5913a918b63a394f6bc83e), [a47788a](https://github.com/morluto/rea/commit/a47788a6a4abb067273f954275a9d0a269adf1e7), [8aa40e9](https://github.com/morluto/rea/commit/8aa40e9a61155cff8b6f3c138c601bf805b20653), [b8c1b45](https://github.com/morluto/rea/commit/b8c1b45316882beeab4a9de49381d94e131fab0f), [a35b933](https://github.com/morluto/rea/commit/a35b9335a0124674ee5e3abe8c075f1fff6b4fa6), [#6137](https://github.com/morluto/rea/pull/6137), [#6126](https://github.com/morluto/rea/issues/6126). Thanks @carlogiovannineri-rgb, @mohithhhh.
- **Capture coverage:** Name browser bundle budgets and deadline recovery, separate domain readiness from worker attribution, attribute rendered-frame omissions to their producing stage, share offline-capture host admission with discovery, preserve selected signal-file semantics, and retain short Flutter GNU build IDs. [#6036](https://github.com/morluto/rea/pull/6036), [ba78133](https://github.com/morluto/rea/commit/ba781330d6d28b22d15498e63c144fb40e086156), [5934387](https://github.com/morluto/rea/commit/5934387f4185129efe9dc8b7338d8bc3fa5b25bd), [ad629b2](https://github.com/morluto/rea/commit/ad629b2e31c3a9495b27179360b508fa021fe86a), [928ba76](https://github.com/morluto/rea/commit/928ba76882bfd216dde8ed79118789b2c3aad4f9), [678c4cf](https://github.com/morluto/rea/commit/678c4cfa68e2fee12f620075e12dc87d9b1195b0).
- **Additional boundary facts:** Retain parser format/query occurrence identity and independent malformed-input facts, unresolved nested Swift identifiers, and completed signing-capture coverage; describe crash documents without assuming a provider. [5f702dd](https://github.com/morluto/rea/commit/5f702dd6ccd9e8e505e224854b91678439c3b41f), [99d2d7a](https://github.com/morluto/rea/commit/99d2d7ad22bcbd780919760008aba6a88e7d1077), [90e433d](https://github.com/morluto/rea/commit/90e433df06b3f83417176b8db1b0a905f9df1d3a), [#5548](https://github.com/morluto/rea/pull/5548), [ee12d51](https://github.com/morluto/rea/commit/ee12d517a25d34853d40e31fc039c7d1576c60e1), [#5538](https://github.com/morluto/rea/issues/5538). Thanks @coygeek, @ShuaixinHuang.
- **Documentation:** Add the Romanian README. [#7026](https://github.com/morluto/rea/pull/7026). Thanks @An3treiu.
Full contribution record: [1740 commits and 497 merged PRs](https://github.com/morluto/rea/blob/rea-agents-6.8.0/docs/releases/6.8.0.contributions.json).
## [6.7.0](https://github.com/morluto/rea/compare/rea-agents-6.6.0...rea-agents-6.7.0) (2026-10-18)
### Highlights
@@ -132,8 +30,8 @@ Full contribution record: [4 commits and 1 merged PR](https://github.com/morluto
- **Kuna analysis:** Use a caller-installed Kuna for procedure listings, pseudocode, assembly, call relationships, exact-address xrefs and cached string searches. Project exports retain source spans and digests; repeated requests reuse completed work. Recorded real-provider coverage uses Kuna 1.833 on Linux; Windows, PE and universal Mach-O are unsupported. See [the provider guide](docs/kuna-provider.md). ([#3832](https://github.com/morluto/rea/pull/3832), [#3854](https://github.com/morluto/rea/pull/3854), [#4071](https://github.com/morluto/rea/pull/4071), [#4095](https://github.com/morluto/rea/pull/4095)). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
- **Retained JavaScript facts:** Query exact application paths, selected properties and filtered module references through `inspect_analysis_view`, with paging and complete row Evidence. These views reuse completed analysis and preserve source selection and unknown coverage. ([#4265](https://github.com/morluto/rea/pull/4265), [morluto/rea#4190](https://github.com/morluto/rea/issues/4190), [#4097](https://github.com/morluto/rea/pull/4097), [morluto/rea#4092](https://github.com/morluto/rea/issues/4092)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **APK locale tables:** Request several resource locales in one decode with repeatable CLI `--locales` or MCP `locales`. Each table reports its status and coverage; existing single-locale fields remain available. ([#4804](https://github.com/morluto/rea/pull/4804), [morluto/rea#4777](https://github.com/morluto/rea/issues/4777)). Thanks @abhay-2108, @N0zoM1z0.
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/providers/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/providers/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
- **JADX deadlines:** Set `REA_JADX_OPERATION_TIMEOUT_MS` to extend APK loading and upstream operation deadlines together. Doctor reports the effective value and setup forwards an explicit selection. ([#4466](https://github.com/morluto/rea/pull/4466), [morluto/rea#3918](https://github.com/morluto/rea/issues/3918)). Thanks @jaychou813, @SandstromPL.
- **Electron window options:** Resolve stable local constant `BrowserWindow` options while retaining source provenance. Relevant writes and escapes keep the result unknown. ([#4400](https://github.com/morluto/rea/pull/4400), [morluto/rea#4366](https://github.com/morluto/rea/issues/4366)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **Language editions:** Add the Greek website edition and Malayalam documentation, with Korean typography and translation corrections across the site. ([#3520](https://github.com/morluto/rea/pull/3520), [#4792](https://github.com/morluto/rea/pull/4792), [#3821](https://github.com/morluto/rea/pull/3821), [morluto/rea#3820](https://github.com/morluto/rea/issues/3820), [#3799](https://github.com/morluto/rea/pull/3799), [morluto/rea#3798](https://github.com/morluto/rea/issues/3798), [#4344](https://github.com/morluto/rea/pull/4344), [#3856](https://github.com/morluto/rea/pull/3856), [#4166](https://github.com/morluto/rea/pull/4166)). Thanks @gensokyo-dev, @hi2242, @kalayciburak, @kaoru0822-kitauji, @kraken-503, @kukinghan, @N0zoM1z0, @yeseong0412.
@@ -192,7 +90,7 @@ Full contribution record: [571 commits and 236 merged PRs](https://github.com/mo
## [6.5.0](https://github.com/morluto/rea/compare/rea-agents-6.4.0...rea-agents-6.5.0) (2026-10-13)
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/development/testing.md).
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/testing.md).
### Highlights
@@ -220,7 +118,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
- **HarmonyOS packages:** inventory HAP/HSP/App Pack structures without executing code. The recorded real workflow covers one pinned HAP through repository-built CLI/MCP; installed-package and real HSP/App Pack coverage remain unverified. ([#1447](https://github.com/morluto/rea/pull/1447)). Thanks @BaaRai.
- **Ghidra raw inputs and references:** compatible Linux/macOS Ghidra profiles accept caller-selected raw architecture/entry information and expose typed incoming references at exact addresses. Windows Ghidra P0 admits the raw NMOS 6502 profile described below; other raw targets and the incoming-reference facet remain outside Windows P0. ([#1868](https://github.com/morluto/rea/pull/1868), [issue morluto/rea#1836](https://github.com/morluto/rea/issues/1836), [#1715](https://github.com/morluto/rea/pull/1715), [issue morluto/rea#1245](https://github.com/morluto/rea/issues/1245)). Thanks @bugstan, @casesolved-co-uk, @mvanhorn, @N0zoM1z0.
- **Ghidra MIPS:** Add bounded MIPS32r2/o32 relocatable objects, soft-float integer-register storage, admitted MIPS16 shared ELF, and a narrowly admitted recordless legacy MIPS32 executable profile. The latter pins an endian-matched 32-bit language and records inferred o32 as an analysis assumption. Admission follows each documented ABI/flag/relocation boundary. ([#1949](https://github.com/morluto/rea/pull/1949), [issue morluto/rea#1937](https://github.com/morluto/rea/issues/1937), [#1950](https://github.com/morluto/rea/pull/1950), [issue morluto/rea#1938](https://github.com/morluto/rea/issues/1938), [#1991](https://github.com/morluto/rea/pull/1991), [issue morluto/rea#1851](https://github.com/morluto/rea/issues/1851), [#1898](https://github.com/morluto/rea/pull/1898), [issue morluto/rea#1620](https://github.com/morluto/rea/issues/1620)). Thanks @AurelioB, @BaaRai, @bqw371, @kaoru0822-kitauji, @N0zoM1z0.
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/providers/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
- **PSP static ELF profile:** the Ghidra profile is Linux x64, bounded little-endian ELF32 `ET_EXEC` with the documented PSP/ABI declarations and a compatible caller-installed Allegrex extension. PRX, shared/relocatable images, PBP/ISO/CSO and runtime emulation are outside scope. Historical extension experiments are separate from public-provider verification; real public Ghidra PSP coverage remains unverified. ([#1599](https://github.com/morluto/rea/pull/1599)). Thanks @jaelliot.
- **Ghidra analysis settings:** apply validated analyzer overrides before analysis while retaining the selected analysis profile. ([#1913](https://github.com/morluto/rea/pull/1913)). Thanks @akushonkamen, @kaoru0822-kitauji, @N0zoM1z0.
- **Attached native IDA MCP:** add an optional native bridge alongside existing legacy/headless profiles. Recorded native runs use Linux IDA Free 9.4 with a navigation correction and Windows IDA Pro 9.3 with a loopback-listener change; stock native switching, native macOS and Windows IDA Free remain unverified. Attached databases remain read-only and are never saved or closed. ([#1544](https://github.com/morluto/rea/pull/1544)). Thanks @imissapixel, @N0zoM1z0.
@@ -292,7 +190,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
- **Source maps:** original-source and mapping rows preserve a null source name when unavailable. Handle null explicitly instead of expecting an invented unknown-source string; new index fields remain optional when reading older Evidence. ([ee50ed1](https://github.com/morluto/rea/commit/ee50ed121e38cc6e1a2ff0dc8b789f15f71c5402)).
- **Streamed filtered JSON:** `--filter-output` now emits JSON `null` instead of `{}` when the selected value is null; consumers must distinguish a present null from a missing selection. Use `--full-output` when an object envelope is required. ([#2333](https://github.com/morluto/rea/pull/2333)). Thanks @Jackmeson1.
- **Call graphs:** `get_call_graph.normalized_result` is now `{ graph, limitations }`; read traversal-depth groups under `graph` and retain relationship-specific caveats. Contract/fixture validation does not establish new Ghidra or IDA engine coverage. ([5dfb9d0](https://github.com/morluto/rea/commit/5dfb9d050c2cc44a61b183dae765bef4d22a53b7)).
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/reference/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **Provider admission:** MIPS targets with unresolved GNU ABI attributes are refused. Supply supported ABI declarations. Saved Hopper databases on macOS are unavailable because the launcher cannot run the analysis bridge when opening a database; select an executable target instead. Database support requires a script-capable database launch route. ([d4bd65c](https://github.com/morluto/rea/commit/d4bd65c53c0254d0195cc7ab8da163dcadfd3b9b)).
- **Segment extents:** `binary_overview` segment length may be null when Hopper coordinates cannot establish a finite extent in one address space. Preserve unknown rather than converting it to zero or a saturated value. ([36db19c](https://github.com/morluto/rea/commit/36db19c59)).
@@ -372,7 +270,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
### ⚠ BREAKING CHANGES
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/guides/runtime/process-capture.md) and [browser scenario contract](docs/guides/web/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/process-capture.md) and [browser scenario contract](docs/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
* **browser:** JSON shape paths now contain typed property and array-element segments. WebMCP input_schema_shape is replaced by input_schema and input_schema_sha256, retaining declared examples and defaults.
* **process:** Filesystem digest omission entries now require system_code; budget and changed-file omissions use null, and unavailable-file omissions preserve the operating-system error code.
* **contracts:** Provider analysis outputs must explicitly supply metadata previously repaired with defaults; unsupported nullable facets must be reported as null.
@@ -540,7 +438,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
### ⚠ BREAKING CHANGES
Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
* **MCP / Evidence / snapshots / artifacts:** Success results are direct canonical Evidence; read `normalized_result` rather than the old `result`/`evidence` wrapper. Evidence requires nullable `analysis_profile`, session provider aliases are removed, snapshots require complete profile-bound query records, and artifact roles/permissions belong to occurrences. Recreate incompatible saved data; adding fields without recomputing their semantic identity is invalid. ([63bf914](https://github.com/morluto/rea/commit/63bf91405cf97b114a84f337fb1272d98a267999))
* **Process captures:** Legacy v3 conversion is removed. Explicit executable identity and `event_journal` are required; retain old captures and recapture when current comparisons are needed instead of inventing launch identity or event order. ([2a77469](https://github.com/morluto/rea/commit/2a7746998d47446c4db6d4f524e0f183d6803f3f))
@@ -2683,7 +2581,7 @@ Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before u
* enforce conventional pull request titles ([3a21f16](https://github.com/morluto/rea/commit/3a21f16c1832430765083636511a8c12dbccf2dd))
* verify installed package with Linux Hopper ([4f97f69](https://github.com/morluto/rea/commit/4f97f699e4d5471557d1ce0ff91e144065d06a61))
## [0.3.0](https://github.com/morluto/rea/compare/rea-0.2.1...rea-agents-0.3.0) (2026-07-12)
## [0.3.0](https://github.com/morluto/rea/compare/rea-agents-0.2.1...rea-agents-0.3.0) (2026-07-12)
### Features
@@ -2736,7 +2634,7 @@ Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before u
* redesign and localize README ([051b6db](https://github.com/morluto/rea/commit/051b6db65687fd14826b6462cd623feb2504f123))
* redesign and localize README ([910eda6](https://github.com/morluto/rea/commit/910eda6f323ba1151b0771fe5de6c60a91e3d46d))
## [0.2.0](https://github.com/morluto/rea/releases/tag/rea-0.2.0) (2026-07-12)
## [0.2.0](https://github.com/morluto/rea/compare/rea-0.1.0...rea-0.2.0) (2026-07-12)
### Features
+21 -68
View File
@@ -11,31 +11,20 @@ incorrect user instructions warrant standalone corrections. Collect other
cosmetic-only suggestions in an issue rather than separate PRs.
For selected architectural owners and dependency paths, use the
[architecture map](docs/architecture/architecture.mermaid). It is not a complete directory or
[architecture map](docs/architecture.mermaid). It is not a complete directory or
import inventory. For current provider wiring, start with
[`src/composition/binary.ts`](src/composition/binary.ts) for deep candidates and
the other `src/composition/` factories for tool services, including
[`gpu.ts`](src/composition/gpu.ts) and [`model.ts`](src/composition/model.ts).
Live Frida, Cutter, GDB and Rizin construction also uses shared composition
factories. For service ownership, availability and catalog declarations, follow
the [capability composition guide](docs/capability-composition.md).
Live Frida and Cutter adapters are constructed directly in
[`src/cli/fridaCommands.ts`](src/cli/fridaCommands.ts),
[`src/cli/cutterCommands.ts`](src/cli/cutterCommands.ts), and
[`src/server/createServer.ts`](src/server/createServer.ts).
Production factories construct fresh providers without starting engines or
acquiring targets; optional adapter failures must preserve successful peers. Run
`npm run verify:test-discovery` after adding or moving tests. Resolved import
boundaries are checked by `npm run verify:module-boundaries`, included in lint.
Before retiring an alias or facade, trace runtime registration, dynamic imports,
JavaScript verifiers, subprocess scripts, and packaged entrypoints as well as
TypeScript references. Tests may be the last consumers of a superseded internal
path; migrate their distinct assertions to the current production path before
removing it. Preserve validation, lazy loading, caller policy, and resource
acquisition and cleanup supplied by the old boundary. Published APIs, persisted
formats, and independently versioned bridges require an explicit compatibility
decision; an internal representation alone does not establish that obligation.
See [interpreting boundary facts once](docs/development/tool-design.md#interpret-boundary-facts-once)
for admission ownership and the distinction between repeated validation and
independent checks.
For analysis workflows and agent interfaces, follow the
[performance priorities](AGENTS.md#performance). Explain the selected output
scope, reuse/cache decisions, and relevant before/after measurements when these
@@ -44,11 +33,9 @@ Use the [analysis optimization skill](.agents/skills/rea-analysis-optimization/S
to locate provider and shared-workflow bottlenecks, preserve Evidence, and
compare equivalent real tasks before claiming an improvement.
When adding or changing an MCP tool, follow the [tool design guide](docs/development/tool-design.md) and preserve the canonical contracts and generated catalog.
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
When adding or changing an admission ceiling, budget, or deadline, follow
[analysis settings, limits, and recovery](docs/design/limits-and-recovery.md)
for ownership, operation snapshots, provider integration, and the steps for
adding a runtime setting.
[admission limits and recovery](docs/design/limits-and-recovery.md).
## Reporting issues
@@ -93,8 +80,7 @@ Before requesting maintainer review, the human owner must:
passing. Follow the [verification requirements](#development-feedback-and-pr-verification)
for required checks and real-provider coverage.
- Complete the **Author readiness** declaration in the PR template, explicitly
stating that the PR is ready for initial review, unless the PR author is
exempt from the declaration as listed below.
stating that the PR is ready for initial review.
Keep the PR description concise and current. Explain the problem, observable
outcome, approach, and useful evidence; link long logs or put them in optional
@@ -103,11 +89,7 @@ Human ownership and review remain required when CI or an automated reviewer
reports success. Readiness requests maintainer review; it does not grant scope
approval or permission to merge.
Draft status is optional. Draft PRs may request early feedback with open questions
and incomplete checks. Agents preserve the current draft status unless asked to
change it; missing readiness declarations or later updates do not require a PR
to become a draft.
Draft PRs may request early feedback with open questions and incomplete checks.
Leave the readiness declaration unchecked until the human owner has completed
it. The declaration records initial readiness; later pushes do not invalidate
it or require another human sign-off. Keep the PR description and affected
@@ -115,12 +97,9 @@ validation current as work changes. AI agents must leave human declarations to
the human owner and must not attest to a human review on their behalf.
The `PR readiness` check reports pending for drafts and requires a named human
owner and all three template declarations for non-drafts. PRs authored by
@morluto, @kaoru0822-kitauji, @N0zoM1z0, or @gensokyo-dev are exempt from
the declaration; their human ownership, review, and evidence responsibilities
above still apply. The check identifies authors by GitHub account ID. It checks
that the declaration is filled, not whether the code or evidence is correct.
Existing PRs can copy the **Author readiness** section from the current template.
owner and all three template declarations for non-drafts. It checks that the
declaration is filled, not whether the code or evidence is correct. Existing PRs
can copy the **Author readiness** section from the current template.
Open PRs must have distinct head commits because GitHub shares commit statuses.
Maintainers enable merge enforcement after the workflow is merged: run
@@ -234,9 +213,7 @@ guarantee.
Use the development toolchain pinned in `.nvmrc` and
`package.json#packageManager`. Run `nvm use` before installing dependencies;
installed-package runtime support is documented in
[installation](docs/getting-started/installation.md#start-setup).
Use Node.js and npm for source builds and verifier scripts; the installed
package's Bun runtime support does not extend to those development scripts.
[installation](docs/installation.md#start-setup).
```bash
npm ci
@@ -251,15 +228,6 @@ catalogs, documentation projections, or managed verification evidence. Turbo
caches deterministic builds and static checks across Git worktrees. After a package, lockfile, or managed-skill version change, run
`npm run metadata:generate` before building.
Dependabot proposes updates for the npm lockfiles, website Python requirements,
and GitHub Actions. Review dependency changes and run the checks relevant to
their consumers. For dependencies maintained outside Dependabot, including
`third_party/` and the HTTP/3 lab's pinned requirements, the maintainer should
review upstream security advisories when updating or using them. Preserve source
and version provenance, update license notices when needed, and report which
verification was actually repeated. The lab requirements describe a recorded
environment; a dependency update does not verify the revised environment.
Repository invariants live in [AGENTS.md](AGENTS.md). Keep provider-specific
protocols in adapters and shared CLI/MCP workflows in the application layer.
@@ -286,23 +254,16 @@ The general source suite (`npm run test:fast`, `npm run test:changed`, and
On a Windows checkout, some source tests fail because fixtures use POSIX paths,
`#!/bin/sh` tool stubs, or POSIX rename semantics. Compare failures against the
base revision, or run the suite on Linux or WSL, before attributing them to a
change. The embedded-artifact parent-swap tests exercise the selected-path
replacement when the rename succeeds; if Windows refuses it while the admitted
file is open, they instead verify the refusal and cleanup. See
[testing](docs/development/testing.md#developer-commands) for command scopes
change. See [testing](docs/testing.md#developer-commands) for command scopes
and lane selection.
`git submodule update --init --recursive` fails at the pinned Unblob submodule
on Windows and can leave later submodules unchecked out. Apply the
[Unblob checkout workaround](docs/guides/firmware/firmware-analysis.md#windows-submodule-checkout),
[Unblob checkout workaround](docs/firmware-analysis.md#windows-submodule-checkout),
then rerun the command.
## Documentation website
Use the [documentation directory](docs/README.md) and
[authoring guide](docs/development/documentation.md) for document ownership,
domain-based feature specifications and architectural decision records.
The VitePress site uses the Markdown files in `docs/`. Run `npm run docs:dev`
for live editing, `npm run docs:build` to check the production build and
links, and `npm run docs:preview` to preview that build at `/rea/`.
@@ -356,7 +317,7 @@ npm run check:fast
`test:local` runs source tests without building. `check:changed` combines cached
static checks with source tests affected since the merge base with `origin/main`.
Import-graph selection can miss runtime registration, bridges, and generated data;
select relevant boundary tests explicitly. See [testing](docs/development/testing.md#developer-commands)
select relevant boundary tests explicitly. See [testing](docs/testing.md#developer-commands)
for commands, test placement, pruning, and real-provider prerequisites.
Before requesting review, run relevant tests and `npm run check`; add
@@ -407,17 +368,9 @@ Real-provider execution is uncached.
### Temporary files and diagnostics
Tests use `createTestTempDirectory` from `tests/fixtures/temporaryDirectory.ts`
for exact-path, awaited cleanup, including failures and timeouts. It registers
deletion with Vitest's `onTestFinished`; do not also add its result to a roots
array or an `afterEach` removal. `createTestWorkspace` from
`tests/support/workspace/workspaceFixture.ts` leaves cleanup with its caller,
which must await `removeTestWorkspace`. Keep explicit resource teardown before
directory deletion, using the case's `finally` or `afterEach`; Vitest runs
`afterEach` before `onTestFinished` and awaits finish hooks in reverse
registration order.
for exact-path, awaited cleanup, including failures and timeouts.
`npm run verify:test-temp-hygiene` runs the suite under a fresh `TMPDIR` and
rejects remaining REA-owned paths. It cannot detect duplicate deletion owners or
prove resource-close ordering. Never glob-clean shared `/tmp/rea-*` content.
rejects remaining REA-owned paths. Never glob-clean shared `/tmp/rea-*` content.
`REA_LOG_LEVEL` selects `trace`, `debug`, `info`, `warn`, `error`, `fatal`, or
`silent`. Logs go to stderr: MCP defaults to `info`, and one-shot CLI logging
@@ -431,8 +384,8 @@ Run `npm run verify:hopper` on a supported macOS host or
`npm run verify:hopper:linux` on the supported Linux demo runner. These build
native fixtures and verify the installed engine through CLI/MCP; package or
simulated-provider checks do not establish engine behavior. See
[real-toolchain verification](docs/development/testing.md#end-to-end-integration-and-golden-evidence)
for prerequisites and scope, and [Hopper setup](docs/getting-started/installation.md#hopper)
[real-toolchain verification](docs/testing.md#end-to-end-integration-and-golden-evidence)
for prerequisites and scope, and [Hopper setup](docs/installation.md#hopper)
for supported hosts and unattended startup.
Use the default `build/conformance/manifest.json` unless explicitly verifying
@@ -441,7 +394,7 @@ Generated fixtures and manifests remain ignored.
## Maintainer release checklist
Use the [release guide](docs/development/releasing.md). Main pushes automatically refresh a
Use the [release guide](docs/releasing.md). Main pushes automatically refresh a
Release Please PR with the next version and changelog. Review its migration
notes, wait for the current CI checks to pass, and merge it into main to publish.
+70 -128
View File
@@ -1,7 +1,7 @@
<div align="center">
**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md)
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md) · [Română](docs/readme/README_ro.md)
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md)
# REA: Reverse Engineer Anything
@@ -10,7 +10,7 @@
**See a feature you like. Understand how it works, down to the binary level.**
[![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/reference/mcp-contracts.md#generated-catalog)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog)
[![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#what-you-can-analyze)
[![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything)
[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE)
@@ -20,9 +20,7 @@
**[Website](https://rea.tools/) · [Guides](https://rea.tools/guides/) · [Showcases](https://rea.tools/showcase/)**
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Join the REA community on Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [Build RL environments](#build-rl-environments) · [FAQ](#faq) · [Documentation](#documentation)
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [FAQ](#faq) · [Documentation](#documentation)
<code>npx rea-agents setup</code>
@@ -64,7 +62,7 @@ See a feature in an app that you want in your own product? Ask your agent to inv
REA connects your agent to tools for inspecting native binaries, JavaScript and Electron apps, .NET assemblies, and websites. You can also use the same tools from your terminal. Analysis runs locally, and results include the evidence and limitations behind each conclusion.
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, IDA or Kuna installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, or IDA installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
> **[Visit the REA website](https://rea.tools/)** for setup instructions, illustrated guides, and real case studies.
@@ -72,7 +70,7 @@ Setup registers REA with your agent and installs matching workflow instructions.
On Linux, Ghidra, JADX, firmware tool launches and static JavaScript/Electron
analysis require a procps-compatible `ps` on REA's `PATH` for process ownership
checks. Recovery steps are in `docs/getting-started/installation.md` under
checks. Recovery steps are in `docs/installation.md` under
"Linux owned-process prerequisite".
### Set up your agent
@@ -85,15 +83,12 @@ npx rea-agents setup
Choose your agents, review the proposed changes, and approve them. Setup adds
REA's MCP server and matching workflow instructions, with backups of existing
configuration. Restart your agent afterward. Setup records the selected runtime and REA
entry script directly. If acquired through a package runner, keep its cache entry
available. Run package acquisition from a trusted directory outside the target
project. Installation and recovery details are in the setup guide below.
configuration. Restart your agent afterward.
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
[other agents](docs/getting-started/installation.md#supported-agents). See
[installation and setup](docs/getting-started/installation.md) for provider configuration and
manual MCP registration and Claude Code/Codex plugin installation.
[other agents](docs/installation.md#supported-agents). See
[installation and setup](docs/installation.md) for provider configuration and
manual MCP registration.
MCP uses compact input schemas by default to reduce context overhead. Set
`REA_MCP_INPUT_SCHEMA_PROFILE=full` in your MCP registration when you need the
@@ -101,12 +96,6 @@ complete advertised field guidance and nested validation details. Both profiles
keep the full tool set and canonical server validation. See
"Compact input schema profile" in the MCP contracts guide.
Complete output schemas are advertised by default. If your client caps the
complete `tools/list` frame, set `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact` to
bound each advertised output schema, or `off` to omit output schemas entirely.
Both settings keep canonical server-side validation and complete structured
results. See "Output schemas" in the MCP contracts guide.
### Ask your agent
```text
@@ -135,7 +124,7 @@ rea --help
```
For native analysis, configure a provider first. See the
[CLI and Evidence guide](docs/getting-started/cli.md) for native commands, provider selection,
[CLI and Evidence guide](docs/cli.md) for native commands, provider selection,
snapshots and scripting.
### Update REA
@@ -191,46 +180,43 @@ CLI commands use the same workflows.
REA requires Node.js 22.x (>=22.23.4), 24.x (>=24.22.1), or 26.12.1+, plus npm.
Additional tools and host support depend on the target:
| Target | What REA returns | Requirements and guide |
| ----------------------- | ------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/guides/native/native-investigation.md) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/guides/native/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
| Windows shortcuts | Shell Link target, arguments, working dir, ExtraData and Tracker droid facts | Built-in static LNK inspection; [Windows shortcuts](docs/guides/native/windows-shortcut.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
| Binary hardening | PIE, NX, RELRO, CFG and related header flags for ELF, PE and Mach-O | Built-in static header inspection; [binary hardening](docs/binary-hardening.md) |
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/guides/evm/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/guides/native/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/guides/javascript/javascript-application-workflows.md) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/guides/web/browser-observation.md) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/guides/web/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/guides/managed/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/guides/android/android-analysis.md) |
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/guides/android/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/providers/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/guides/android/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/guides/android/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/guides/firmware/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/guides/data/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/guides/data/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/guides/signals/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/guides/javascript/javascript-artifact-reconstruction.md), [Apple applications](docs/guides/apple/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/guides/runtime/process-capture.md) |
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
| Target | What REA returns | Requirements and guide |
| ----------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
Static JavaScript and .NET inspection read the supplied files without running
the application. Runtime capture runs or interacts with the selected target
@@ -244,15 +230,15 @@ or update either provider.
<a id="choosing-a-deep-analysis-provider"></a>
Native formats and host support vary by provider. See
[Hopper and Ghidra setup](docs/getting-started/installation.md#hopper), the
[IDA guide](docs/providers/ida-provider.md) (including an optional native MCP bridge), the
[Hopper and Ghidra setup](docs/installation.md#hopper), the
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), the
[Kuna decompiler guide](docs/kuna-provider.md), and
[experimental Windows Ghidra support](docs/providers/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/providers/ghidra-dos.md).
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
For large binaries, raise the provider's startup deadline with
`REA_GHIDRA_STARTUP_TIMEOUT_MS` or `REA_HOPPER_STARTUP_TIMEOUT_MS`.
For provider selection, see the [CLI guide](docs/getting-started/cli.md#choose-a-provider).
Check [release availability](docs/getting-started/installation.md#released-package-and-main)
For provider selection, see the [CLI guide](docs/cli.md#choose-a-provider).
Check [release availability](docs/installation.md#released-package-and-main)
for features added since the latest npm release.
## Showcases
@@ -289,72 +275,28 @@ historical compiler output.
[Read the case study](https://rea.tools/showcase/th04/) ·
[Reconstruction repository](https://github.com/N0zoM1z0/th04)
### Build RL environments with REA
Record a web app session, turn its API into tools, and export an environment an AI agent can use. Run it with AgentEnv, then add tasks and reward checks for RL training or evaluation.
[Read the guide](https://rea.tools/guides/rl-environments/) ·
[Watch the video](https://x.com/reatools/status/2112574822584918075?s=20)
If you've used REA on something interesting, we'd love to see it. Share your
case in an [issue](https://github.com/morluto/rea/issues) or a
[pull request](https://github.com/morluto/rea/pulls), including the target,
your question, how REA helped, and what you found.
## Build RL environments
Give an AI agent a local environment to practice tasks such as finding an issue or creating a folder, without running the original app. Use it for reinforcement learning (RL) or to test how well an agent completes tasks.
REA turns recorded API requests and responses into tools the agent can call. The environment uses the recorded data and supports changes only when the recording confirms their effects.
With REA set up in your agent, replace the example URL and folder tasks below with your own:
```text
Use REA to build an RL environment from my test web app at
http://127.0.0.1:3000.
Cover listing, creating, renaming, and deleting folders. You may create and
remove test folders. Leave existing data unchanged.
Save it in a folder named my-env. Check which operations work and tell me
how to run it.
```
<details>
<summary>Use the command line instead</summary>
Use these commands if you already have a file containing the app’s recorded browser requests and responses. Running the result also requires AgentEnv and Docker.
```sh
rea environment-build ./session.har ./my-env --name my-env --json
agent-env run ./my-env
```
</details>
REA exports a runnable environment with a basic test to check that it starts and responds. Use it with your RL framework to define task goals, add reward checks, and run training.
[Build an environment](https://rea.tools/guides/rl-environments/) ·
[Environment reference](docs/agent-environments.md) ·
[Watch the demo on X](https://x.com/reatools/status/2112574822584918075?s=20)
## FAQ
<details>
<summary><strong>Which agents can use REA?</strong></summary>
Any agent that supports local MCP servers. Setup configures the
[supported agents](docs/getting-started/installation.md#supported-agents); other clients can use
[manual MCP registration](docs/getting-started/installation.md#mcp-registry).
[supported agents](docs/installation.md#supported-agents); other clients can use
[manual MCP registration](docs/installation.md#mcp-registry).
</details>
<details>
<summary><strong>Do I need a native analysis engine?</strong></summary>
<summary><strong>Do I need Hopper, Ghidra or IDA?</strong></summary>
Deep native analysis uses Hopper, Ghidra, IDA or Kuna, depending on the target
and operation. Static JavaScript and .NET inspection work without a native
analysis engine. Setup can install Hopper after approval; Ghidra, IDA and Kuna
use your existing installations. See [provider setup](docs/getting-started/installation.md#hopper)
for the options and their supported targets.
Deep native analysis uses one of them. Static JavaScript and .NET inspection
work without a native analysis engine. Setup can install Hopper after approval;
Ghidra and IDA use your existing installations. See [provider setup](docs/installation.md#hopper).
</details>
@@ -365,7 +307,7 @@ REA starts Hopper when an operation needs it. On macOS, a first-run dialog may
ask you to choose demo mode or activate your license. Complete that prompt in the
same user session before your first CLI/MCP operation or unattended job. Open
Hopper manually if needed. See
[Hopper startup and troubleshooting](docs/getting-started/installation.md#launcher-paths-and-troubleshooting).
[Hopper startup and troubleshooting](docs/installation.md#launcher-paths-and-troubleshooting).
</details>
@@ -374,7 +316,7 @@ Hopper manually if needed. See
The skill supplies investigation instructions for your agent. Use `npx rea-agents setup`
to register REA's MCP server and install the matching instructions, then restart
your agent. See [skill-only installation](docs/getting-started/installation.md#skill-only-installation).
your agent. See [skill-only installation](docs/installation.md#skill-only-installation).
</details>
@@ -423,18 +365,18 @@ with your REA version, target type, steps to reproduce and error output.
Start with the website's [worked guides](https://rea.tools/guides/).
For exact options, prerequisites and result contracts:
- [Installation and setup](docs/getting-started/installation.md): agent registration, provider configuration, updates and uninstall.
- [Readiness and troubleshooting](docs/getting-started/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/getting-started/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/reference/mcp-contracts.md) and [agent prompts](docs/getting-started/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/reference/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/development/roadmap.md): planned work and capability trackers.
- [Installation and setup](docs/installation.md): agent registration, provider configuration, updates and uninstall.
- [Readiness and troubleshooting](docs/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/mcp-contracts.md) and [agent prompts](docs/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/roadmap.md): planned work and capability trackers.
Report vulnerabilities through [SECURITY.md](SECURITY.md).
## Star history
🎉 **400,000 GitHub stars — thank you!**
🎉 **300,000 GitHub stars — thank you!**
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
@@ -463,8 +405,8 @@ report a bug or suggest a feature, or [send a pull request](https://github.com/m
to improve the code or docs.
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
[testing](docs/development/testing.md) for verification lanes, and the
[architecture map](docs/architecture/architecture.mermaid) for selected layers and dependency paths.
[testing](docs/testing.md) for verification lanes, and the
[architecture map](docs/architecture.mermaid) for selected layers and dependency paths.
## License
+28 -112
View File
@@ -4,124 +4,13 @@
Security fixes are provided for the latest published version of REA. Upgrade to the latest release before reporting a problem that may already have been resolved.
## Security boundary
REA is a local reverse-engineering and automation tool. CLI and MCP use the
same application workflows and run with the host process's permissions.
REA is not a general-purpose sandbox or a multi-tenant authorization service.
### Threat model
- **Trusted authority:** the operating-system account running REA, the caller
or agent host that selects operations, and configured analysis providers,
their executables and extensions, and service endpoints. An MCP client can
invoke the operations exposed to it; its host must enforce the user's tool
permissions and approvals.
- **Untrusted data:** target binaries, archives, source files, project contents,
captures, web pages and returned target content can be malicious. Selecting
an input for static inspection does not authorize commands embedded in that
input. Agents must treat target text and Evidence as data rather
than instructions granting additional authority.
- **Operation-specific effects:** static inspection parses target data and may
launch an analysis engine. Runtime, debugger, instrumentation, browser,
desktop and environment workflows can deliberately execute code, attach to
processes, make network requests, write files or interact with applications
and devices. Review the selected operation's effects and limitations; static
inspection does not authorize launching or interacting with the target
application. See
[runtime and network responsibilities](DISCLAIMER.md#runtime-network-and-device-operations).
Standard client registrations use a dedicated stdio process. Opt-in
[HTTP MCP](docs/reference/mcp-contracts.md#transports) uses plain HTTP without
TLS or client authentication and shares provider/active-target state across
protocol sessions. Its session ID is not a user-authentication or isolation
boundary. Keep this endpoint within a trusted local or isolated environment.
### Intended security properties
Static JavaScript and .NET inspection do not run the target application.
Unexpected execution of input code through a data-only analysis path is a
security concern. Analysis engines themselves are executable software trusted
with the host account's permissions; parsing an input in an external engine
does not isolate that engine from the host.
REA authenticates its managed Hopper and Ghidra bridges with per-session random
capability tokens transferred through private session files, rather than
process arguments or environment variables. POSIX uses current-user Unix
sockets. Windows Ghidra uses authenticated loopback TCP and a private NTFS
runtime whose DACL admits only the current user and SYSTEM. These controls
protect the managed session boundary, not every external provider or debugger
endpoint. See [Windows Ghidra transport and lifecycle](docs/providers/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
Provider ownership and mutation rules remain specific to each workflow:
- Ghidra's working projects are temporary and owned by REA. Opt-in
[checkpoints](docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program)
retain REA-owned Program state. A
[caller-selected existing project](docs/getting-started/installation.md#reuse-an-existing-ghidra-project)
is fingerprinted and copied into an owned runtime for read-only processing;
Ghidra does not open or save the original database through that workflow.
- [IDA analysis](docs/providers/ida-provider.md) is read-only. An attached GUI
database is never saved or closed by REA; owned headless sessions have a
separate lifecycle.
- Cleanup must stay within resources REA owns. Failed or unverified cleanup
can retain a workspace or session for retry; closing a session
does not imply that every investigation artifact has been erased.
`rea setup` shows planned configuration and installation changes and requires
approval before applying them. Ghidra, Java and IDA are user-managed. Approved
Linux Hopper installation authenticates the package's detached PGP signature
against the pinned publisher identity and checks the signed launcher against
the selected release's digest. The release feed's display version is separate
from that authenticated identity. See [installation and publisher verification](docs/getting-started/installation.md#hopper).
### Limits and caller responsibilities
- **Host isolation:** private runtimes and capability tokens do not protect
against malicious software already running as the same operating-system
user, or a compromised trusted client or provider. Use an appropriately
isolated account, VM or restricted environment when handling hostile targets.
- **Resource bounds:** timeouts, cancellation, streaming and size limits are
operation-specific. They are not a universal CPU, memory, disk or wall-time
limit on every target and third-party tool. Enforce hard host resource limits
externally when needed. Failures of a documented boundary or cancellation
control still warrant investigation.
- **Data and network exposure:** Evidence, caches, checkpoints, exports and
captures can retain private target data, paths, URLs and memory contents.
Local execution does not make these outputs safe to publish. Requested
network operations, agent hosts and provider features such as
[IDA Free's cloud decompiler](docs/providers/ida-provider.md) can communicate
outside the host. Review their data handling and secure any exposed endpoint.
- **Interpretation:** analysis, reconstruction and generated environments can
be incomplete or incorrect. Identity checks, signatures and successful
analysis do not establish that a target, provider or generated program is
safe to execute. Review the Evidence and proposed actions before relying on
them.
### Vulnerability scope
Please report suspected violations of these boundaries privately. Examples
include unintended code execution during data-only inspection, escaping a
documented extraction/output boundary, bypassing bridge authentication or
target/session identity checks, unauthorized effects on an unrelated resource,
and disclosure of transport credentials or explicitly marked sensitive values.
An attacker-triggered bypass of a documented resource or cancellation control
can also be security-relevant without code execution.
Incorrect analysis, unsupported formats, crashes, hangs and high resource use
without a security-boundary impact are generally ordinary bug reports. Effects
explicitly requested through an execution or interaction workflow are not by
themselves a vulnerability. A dependency advisory needs its actual REA exposure
assessed; a complete exploit or diagnosis is not required to report a concern.
If you are unsure whether a boundary is affected, use the private channel below.
## Reporting a vulnerability
Do not disclose vulnerabilities, proof-of-concept exploits, credentials,
private binaries, Hopper documents, or private Ghidra projects in a public
issue.
Use **Report a vulnerability** in the repository's **Security** tab as the preferred way to submit the report privately. If GitHub private reporting is unavailable to you, email [[email protected]](mailto:[email protected]). Do not open a public issue to request a private contact channel.
Use **Report a vulnerability** in the repository's **Security** tab to submit the report privately. Do not open a public issue to request a private contact channel.
Include the affected REA version, operating system/distribution, selected
provider and version, impact, reproduction conditions, and any suggested
@@ -132,3 +21,30 @@ unrelated local paths from logs or examples.
The maintainer will acknowledge the report, assess its scope, and coordinate
remediation and disclosure timing with the reporter. Please allow time for
provider-dependent behavior to be reproduced safely.
## Security boundary
REA authenticates its Hopper and Ghidra bridges with per-session random
capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses
authenticated loopback TCP with a private NTFS runtime whose DACL admits only
the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
Tokens are transferred through private session descriptors rather than process
arguments or environment variables.
This is not a sandbox and does not protect against malicious processes already
running as the same operating-system user. Opening an untrusted binary
delegates parsing and analysis to the selected local provider with that user's
permissions.
Ghidra sessions use a packaged Java `HeadlessScript`, an isolated temporary
project, and private home, cache, and runtime directories. REA imports only the
requested target, bounds startup and protocol messages, requests bounded CPU
and heap settings, and deletes the temporary project when the session closes.
These controls limit accidental persistence and resource use; they do not make
Ghidra a sandbox. REA does not open or modify user-owned Ghidra projects.
`GHIDRA_INSTALL_DIR` and optional `JAVA_HOME` identify user-managed
installations. Setup can add those values to detected client configuration only
after showing its plan and receiving approval. REA does not download, install,
upgrade, or modify Ghidra or Java.
On Linux, REA accepts release metadata only from Hopper's HTTPS endpoint, restricts package URLs to Hopper's public download origin, bounds the package size, and compares the downloaded bytes with Hopper's published checksum before requesting installation. The published SHA-1 value is a corruption check, not a modern package signature; HTTPS origin validation remains part of the trust boundary. Dependency installation is delegated without shell evaluation to `apt-get`, `dnf`, or `pacman`, directly as root or through `pkexec`. REA never invokes `sudo`.
+21 -22
View File
@@ -17,7 +17,6 @@ notices supplied with the software you actually obtain or redistribute.
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| npm dependencies | [Complete inventory](licenses/npm-dependencies.txt) of every entry in `package-lock.json`, including transitive, optional platform, development, and website dependencies. [Runtime notices](licenses/npm-runtime-notices.txt) preserve the copyright, license, and NOTICE documents from integrity-verified release archives, plus the MIT notice for Rolldown's generated runtime helpers. Development dependencies are inventoried but are not all distributed in REA's npm package. |
| pefile ordinal tables | Import-hash ordinal names are derived from [pefile 2024.8.26](https://github.com/erocarrera/pefile/tree/4b3b1e2e568a88d4f1897d694d684f23d9e270c4), copyright 2004–2024 Ero Carrera. Preserve the [MIT license](licenses/pefile-2024.8.26-LICENSE.txt). The Python package is not a runtime dependency. |
| UF2 family names | UF2 family IDs and short names are copied from [microsoft/uf2 `utils/uf2families.json`](https://github.com/microsoft/uf2/blob/cf3bd7f9c274b949528f8cd6a19eef2502a687d2/utils/uf2families.json), copyright Microsoft Corporation. Preserve the [MIT license](licenses/microsoft-uf2-MIT.txt). |
| NativeAOT parser | REA's derived parser uses the MIT-licensed [ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot/tree/effeb734fc570c32650f88b159608979dc7b423e) work of Washi and the NativeAOT layout definitions in [dotnet/runtime](https://github.com/dotnet/runtime/tree/a3b2d40328be0be3f8dd43f0e5cc925b8827b044). Preserve [Washi's MIT notice](licenses/nativeaot-parser-MIT.txt) and the [.NET Foundation MIT notice](licenses/dotnet-runtime-MIT.txt). |
| EVM analysis | `[email protected]`, MIT, copyright 2023 Maxim Andreev. The published npm archive omits its license document; REA retains the [original upstream license](third_party/evmole/LICENSE) and [provenance](third_party/evmole/README.md). It is also included in the runtime notices. |
| Windows native addon | The release addon is compiled with MinGW-w64 and statically links GCC runtime libraries. Preserve the [MinGW-w64 runtime notices](licenses/MinGW-w64-runtime.txt), [GPLv3 text](licenses/GPL-3.0.txt), and [GCC Runtime Library Exception 3.1](licenses/GCC-Runtime-Library-Exception-3.1.txt). The exception governs eligible compiled combinations; it does not relicense the runtime itself under MIT. The Node-API build headers are from Node 22.19.0; its [original license and component notices](licenses/Node-22.19.0-LICENSE.txt) are retained. Node itself is supplied by the user. Snapshot provenance and release-toolchain review requirements are in [licenses/README.md](licenses/README.md). |
@@ -54,27 +53,27 @@ REA's adapters do not grant, replace, or waive these licenses. The upstream
executables and their dependency graphs are supplied separately. Git submodules
pin inspection sources; they are not included in the npm package.
| Project | License and source | REA relationship |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/getting-started/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
| Project | License and source | REA relationship |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.833/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.833/NOTICE) | Bring your own `kuna`; REA verified 1.833 and never downloads, installs or bundles it. REA invokes it as a separate process. |
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
Ghidra's original notice credits software developed at the National Security
Agency. It explains that US-government-created portions are not subject to US
-10
View File
@@ -1,10 +0,0 @@
# Plugin icon
`rea-plugin.png` is a square adaptation of the left panel of REA's GitHub
social preview, retrieved on 2026-10-28:
https://repository-images.githubusercontent.com/1209966933/1a3fef9a-be4b-41f4-8830-a94c0e8288cc
The image was adapted with imagegen, preserving the cyan REA lettering,
pixel typography, orange rule, and taglines. Both plugin manifests reference
this shared asset; only the PNG is included in the npm package.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

-318
View File
@@ -1,318 +0,0 @@
"""Run REA inside an agentenv build computer for REA on the host: capture the app, draft its task and export it.
REA sends this file's source through the computer's ``computer-exec/v1`` route as
``python3 -I -c <source> <command> <JSON arguments>``. JSON that would exceed the bytes Linux
passes as one string travels as several arguments, which this file joins before parsing. Each command prints
one JSON line and exits. ``start``
leaves the capture running on its own: its stdin is a FIFO it holds open for reading and writing, so it never
sees the end of its input, and ``mark`` and ``stop`` write their requests there and wait for the capture's own
acknowledgement on its progress log. A small shell around the capture writes its exit status to a file, the
mark a later command reads. REA runs as the session user in the home, with no ``AGENTENV_*`` variable, so
neither REA nor the app it launches holds the computer's control token, and with ``APT_CONFIG`` naming the
package lists ``start`` downloads as that user, so the capture finds the package's sources.
"""
from __future__ import annotations
import hashlib
import json
import os
import re
import shutil
import subprocess
import sys
import time
POLL_S = 0.2
TAIL_BYTES = 4096
def _say(value: dict) -> int:
print(json.dumps(value), flush=True)
return 0
def _tail(path: str) -> str:
try:
with open(path, "rb") as handle:
handle.seek(max(0, os.path.getsize(path) - TAIL_BYTES))
return handle.read().decode("utf-8", "replace")
except OSError:
return ""
def _environment(work: str) -> dict:
environment = {name: value for name, value in os.environ.items() if not name.startswith("AGENTENV_")}
environment["APT_CONFIG"] = os.path.join(work, "apt", "apt.conf")
return environment
def _apt(work: str, timeout: float) -> dict:
"""Download the package lists into the work folder as the session user, until one update succeeds."""
root = os.path.join(work, "apt")
lists = os.path.join(root, "lists")
updated = os.path.join(root, "updated")
os.makedirs(os.path.join(lists, "partial"), exist_ok=True)
os.makedirs(os.path.join(root, "cache", "archives", "partial"), exist_ok=True)
with open(os.path.join(root, "apt.conf"), "w", encoding="utf-8") as conf:
conf.write(
f'Dir::State::Lists "{lists}";\n'
f'Dir::Cache "{root}/cache";\n'
'Debug::NoLocking "true";\n'
'APT::Update::Error-Mode "any";\n'
)
# apt keeps lists compressed as it fetched them, such as *_Packages.lz4 on Ubuntu.
count = lambda: len([name for name in os.listdir(lists) if re.search(r"_Packages(\.\w+)?$", name)]) # noqa: E731
# A marker left by an update that fetched nothing is not a success; download again.
if os.path.exists(updated) and count() > 0:
return {"lists": count(), "updated": False}
log = os.path.join(root, "update.log")
try:
with open(log, "wb") as output:
code = subprocess.run(["apt-get", "update"], env=_environment(work), stdin=subprocess.DEVNULL,
stdout=output, stderr=subprocess.STDOUT, timeout=timeout).returncode
except (OSError, subprocess.TimeoutExpired) as error:
return {"lists": count(), "updated": True, "error": f"{type(error).__name__}: {error}"}
lists_fetched = count()
if code == 0 and lists_fetched > 0:
# Only a whole update that fetched lists is kept, so a failed or empty one runs again.
open(updated, "w").close()
return {"lists": lists_fetched, "updated": True, "exit_code": code}
if code == 0:
return {"lists": lists_fetched, "updated": True, "exit_code": code,
"error": "apt-get update exited 0 without a package list"}
return {"lists": lists_fetched, "updated": True, "exit_code": code,
"error": f"apt-get update exited {code}: {_tail(log)[-1500:].strip()}"}
def _exit_code(capture: str):
try:
with open(os.path.join(capture, "exit"), encoding="utf-8") as handle:
return int(handle.read().strip() or "-1")
except (OSError, ValueError):
return None
def _ended(capture: str, code: int) -> dict:
"""A capture that has exited: its status, the end of its progress log and its output's size, which REA reads
whole for the Evidence or failure it ended with, whichever step was waiting on it."""
return {"exited": code, "progress": _tail(os.path.join(capture, "progress.log")),
"output_bytes": os.path.getsize(os.path.join(capture, "capture.out"))}
def _event_line(path: str, offset: int, kinds: set, name):
"""The first progress line after ``offset`` whose event is of one of ``kinds``, for ``name`` when given."""
try:
with open(path, "rb") as handle:
handle.seek(offset)
lines = handle.read().decode("utf-8", "replace").splitlines()
except OSError:
return None
for line in lines:
try:
event = json.loads(line)["rea_progress"]["event"]
except (ValueError, KeyError, TypeError):
continue
if isinstance(event, dict) and event.get("kind") in kinds and (name is None or event.get("name") == name):
return line
return None
def _wait(capture: str, offset: int, kinds: set, name, timeout: float) -> dict:
"""Wait for the capture's event, or for it to exit, or for time to run out."""
log = os.path.join(capture, "progress.log")
deadline = time.monotonic() + timeout
while True:
line = _event_line(log, offset, kinds, name)
if line is not None:
return {"line": line}
code = _exit_code(capture)
if code is not None:
return _ended(capture, code)
if time.monotonic() > deadline:
return {"timed_out": True, "progress": _tail(log)}
time.sleep(POLL_S)
def _within(path: str, folder: str) -> bool:
return path == folder or path.startswith(folder + os.sep)
def _state_paths(values: list, work: str):
"""Each state path as an absolute path strictly under the home and outside REA's work folder, both as named
and with its links resolved, or the refusal of the first that is not. The capture resolves the links again
and retains what they lead to, so a link out of the home is refused here. Each passes on as named, so the
capture keeps what its spelling means, such as a hidden settings folder."""
home = os.path.normpath(os.path.expanduser("~"))
real_home, real_work = os.path.realpath(home), os.path.realpath(work)
spellings = {home, real_home}
paths = []
for value in values:
relative = value[2:] if value.startswith("~/") else value
path = os.path.normpath(os.path.join(home, relative))
if ".." in relative.split("/") or not any(path.startswith(spelling + os.sep) for spelling in spellings):
return None, f"The state path {value} is not under the computer's home, {home}."
# Every link on the way is followed, a dangling one too; a path that does not exist yet resolves
# through its nearest existing folder.
resolved = os.path.realpath(path)
if not resolved.startswith(real_home + os.sep):
return None, f"The state path {value} leads through a link to {resolved}, which is not under the computer's home, {home}."
if _within(path, work) or _within(resolved, real_work):
return None, f"The state path {value} is in REA's work folder {work}, which every capture leaves out."
paths.append(path)
return paths, None
def start(arguments: dict) -> int:
work, capture = os.path.abspath(arguments["work"]), os.path.abspath(arguments["capture"])
rea = shutil.which("rea")
if rea is None:
return _say({"error": "REA is not installed in this computer; start the computer from a request, which installs it."})
state_paths, refusal = _state_paths(arguments.get("state_paths", []), work)
if refusal is not None:
return _say({"refused": refusal})
if os.path.exists(capture):
return _say({"error": f"the capture folder {capture} is already used"})
apt = _apt(work, arguments.get("apt_timeout_s", 600))
try:
os.makedirs(capture)
except FileExistsError:
return _say({"error": f"a stop claimed {capture} before the capture launched", "apt": apt})
control = os.path.join(capture, "control")
os.mkfifo(control, 0o600)
argv = [rea, "capture-desktop-application", "--desktop-entry", arguments["desktop_entry"],
"--output", os.path.join(capture, "capture"), "--exclude", work, "--json"]
for value in arguments.get("arguments", []):
argv += ["--argument", value]
for path in state_paths:
argv += ["--state-path", path]
if arguments.get("package_file"):
argv += ["--package-file", arguments["package_file"]]
# Reading and writing, so the capture never reads the end of its input between requests.
reader = os.open(control, os.O_RDWR)
try:
with open(os.path.join(capture, "capture.out"), "wb") as out, \
open(os.path.join(capture, "progress.log"), "wb") as progress:
process = subprocess.Popen(
["sh", "-c", 'done=$1; shift; "$@"; code=$?; printf "%s" "$code" > "$done.tmp" && mv "$done.tmp" "$done"',
"sh", os.path.join(capture, "exit"), *argv],
cwd=os.path.expanduser("~"), env=_environment(work), stdin=reader, stdout=out, stderr=progress,
start_new_session=True)
finally:
os.close(reader)
waited = _wait(capture, 0, {"window_open"}, None, arguments.get("timeout_s", 120))
return _say({"pid": process.pid, "apt": apt, **waited})
def _request(capture: str, line: str) -> bool:
"""Write one request to the running capture; False when nothing reads its FIFO any more."""
try:
fd = os.open(os.path.join(capture, "control"), os.O_WRONLY | os.O_NONBLOCK)
except OSError:
return False
try:
os.write(fd, (line + "\n").encode("utf-8"))
finally:
os.close(fd)
return True
def mark(arguments: dict) -> int:
"""Request a checkpoint and answer with the capture's acknowledgement line as it wrote it: the checkpoint's
counts and up to ten of the files that changed, home-relative, with a count of the rest."""
capture, name = os.path.abspath(arguments["capture"]), arguments["name"]
code = _exit_code(capture)
log = os.path.join(capture, "progress.log")
if code is not None:
return _say(_ended(capture, code))
# A checkpoint already taken, whose answer was lost, is answered from the capture's own log.
taken = _event_line(log, 0, {"checkpoint"}, name)
if taken is not None:
return _say({"line": taken})
offset = os.path.getsize(log)
if not _request(capture, f"checkpoint {name}"):
return _say({"error": "the capture is not reading its requests"})
return _say(_wait(capture, offset, {"checkpoint", "checkpoint_refused"}, name, arguments.get("timeout_s", 600)))
def stop(arguments: dict) -> int:
capture = os.path.abspath(arguments["capture"])
if _exit_code(capture) is None and not os.path.exists(os.path.join(capture, "control")):
# Never launched: claim the folder, so a start still on its way finds it used and launches nothing.
os.makedirs(capture, exist_ok=True)
return _say({"absent": True})
if _exit_code(capture) is None:
_request(capture, "stop")
deadline = time.monotonic() + arguments.get("timeout_s", 600)
while (code := _exit_code(capture)) is None and time.monotonic() < deadline:
time.sleep(POLL_S)
log = os.path.join(capture, "progress.log")
if code is None:
return _say({"timed_out": True, "progress": _tail(log)})
output = os.path.join(capture, "capture.out")
return _say({"exit_code": code, "progress": _tail(log), "output_bytes": os.path.getsize(output)})
def run(arguments: dict) -> int:
"""Run one REA command with its output in a file, after writing the files it reads."""
rea = shutil.which("rea")
if rea is None:
return _say({"error": "REA is not installed in this computer"})
os.makedirs(os.path.dirname(arguments["stdout"]), exist_ok=True)
for path, text in arguments.get("files", {}).items():
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as handle:
handle.write(text)
with open(arguments["stdout"], "wb") as out:
try:
done = subprocess.run([rea, *arguments["argv"]], cwd=os.path.expanduser("~"),
env=_environment(os.path.abspath(arguments["work"])), stdin=subprocess.DEVNULL, stdout=out,
stderr=subprocess.PIPE, timeout=arguments.get("timeout_s", 1800))
except subprocess.TimeoutExpired:
return _say({"timed_out": True})
return _say({"exit_code": done.returncode, "stdout_bytes": os.path.getsize(arguments["stdout"]),
"stderr": done.stderr[-TAIL_BYTES:].decode("utf-8", "replace")})
def listing(arguments: dict) -> int:
"""Every regular file under a folder, by its path relative to the home, with its size and SHA-256.
The list goes to the file ``output`` names, outside the folder, and the answer gives its size: a large
bundle's list outgrows the 1 MiB of stdout a command answers with, so REA reads it through ``data/get``.
"""
home = os.path.expanduser("~")
files, skipped = [], []
def fail_walk(error: OSError) -> None:
raise error
for directory, folders, names in os.walk(os.path.abspath(arguments["root"]), onerror=fail_walk):
folders.sort()
# os.walk does not follow directory symlinks; disclose them to the consumer.
for name in folders[:]:
path = os.path.join(directory, name)
if os.path.islink(path):
skipped.append(os.path.relpath(path, home))
folders.remove(name)
for name in sorted(names):
path = os.path.join(directory, name)
relative = os.path.relpath(path, home)
if os.path.islink(path) or not os.path.isfile(path):
skipped.append(relative)
continue
digest = hashlib.sha256()
with open(path, "rb") as handle:
for chunk in iter(lambda: handle.read(1 << 20), b""):
digest.update(chunk)
files.append({"path": relative, "size": os.path.getsize(path), "sha256": digest.hexdigest()})
output = os.path.abspath(arguments["output"])
os.makedirs(os.path.dirname(output), exist_ok=True)
with open(output, "w", encoding="utf-8") as handle:
json.dump({"files": files, "skipped": skipped}, handle)
return _say({"listing_bytes": os.path.getsize(output)})
COMMANDS = {"start": start, "mark": mark, "stop": stop, "run": run, "list": listing}
if __name__ == "__main__":
sys.exit(COMMANDS[sys.argv[1]](json.loads("".join(sys.argv[2:]))))
-83
View File
@@ -1,83 +0,0 @@
"""Keep one agentenv computer up for REA until stdin closes or SIGTERM arrives.
REA runs ``computer.py <bundle folder> [task]`` with the agentenv runner's own interpreter, so ``agent_env``
imports. It deploys the bundle's task, ``smoke`` by default, on this machine with
``run_bundle(keep=True, emulate=False)``, prints one JSON line naming the computer's address, MCP URL, control
token and sandbox id, or ``{"error": ...}``, and tears the computer down once its stdin closes or it gets SIGTERM
or SIGINT. Everything else it or agentenv prints, a build's output included, goes to stderr. It exits 0 once
the teardown left nothing up, 1 when it did, and 2 when no computer came up.
"""
from __future__ import annotations
import json
import os
import signal
import sys
import threading
def _protocol():
"""A private copy of stdout for the JSON line; fd 1 then points at stderr, as do children that inherit it."""
protocol = os.fdopen(os.dup(1), "w", encoding="utf-8")
os.dup2(2, 1)
sys.stdout = sys.stderr
return protocol
def _say(protocol, value: dict) -> None:
protocol.write(json.dumps(value) + "\n")
protocol.flush()
def _computer(result):
"""The run's deployed computer, or why there is none."""
run = result.runs[0] if result.runs else None
if run is None:
return None, "the bundle ran no task"
if run.error is not None or run.failed_steps:
return None, f"the task failed: {run.error or run.failed_steps}"
for env in run.context.deployed_envs:
if env.environment_url and env.mcp_url and getattr(env, "sandbox_id", None):
return env, None
return None, "the task deployed no computer"
def main(argv: list[str]) -> int:
protocol = _protocol()
bundle, task = argv[1], argv[2] if len(argv) > 2 else "smoke"
try:
from agent_env.bundle import run_bundle
except ImportError as error:
_say(protocol, {"error": f"agent_env does not import with {sys.executable}: {error}"})
return 2
try:
result = run_bundle(bundle, tasks=[task], keep=True, emulate=False, sandbox="local",
on_progress=lambda line: print(line, file=sys.stderr, flush=True))
except BaseException as error: # noqa: BLE001 - a signal's RunInterrupted too; run_bundle tore its runs down
_say(protocol, {"error": f"{type(error).__name__}: {error}"})
return 2
stop = threading.Event()
for signum in (signal.SIGTERM, signal.SIGINT):
signal.signal(signum, lambda *_: stop.set())
env, problem = _computer(result)
try:
if env is None:
_say(protocol, {"error": problem})
else:
_say(protocol, {"base_url": env.environment_url, "mcp_url": env.mcp_url,
"control_token": env.control_token, "sandbox_id": env.sandbox_id})
threading.Thread(target=lambda: (sys.stdin.buffer.read(), stop.set()), daemon=True).start()
while not stop.wait(0.5):
pass
finally:
torn_down = result.teardown()
still_up = [sandbox for run in torn_down.runs if run.torn_down is not None for sandbox in run.torn_down.still_up]
if still_up:
print(f"still up after the teardown: {still_up}", file=sys.stderr, flush=True)
return 1
return 0 if env is not None else 2
if __name__ == "__main__":
sys.exit(main(sys.argv))
-64
View File
@@ -1,64 +0,0 @@
"""Exercise the capture controller's filesystem inventory boundary."""
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
CONTROLLER = Path(__file__).with_name("capture.py")
class CaptureListingTest(unittest.TestCase):
def run_listing(self, root, output):
return subprocess.run(
[
sys.executable,
str(CONTROLLER),
"list",
json.dumps({"root": str(root), "output": str(output)}),
],
text=True,
capture_output=True,
check=False,
)
def test_directory_symlink_is_disclosed_without_following_it(self):
with tempfile.TemporaryDirectory() as parent:
parent = Path(parent)
root = parent / "root"
target = parent / "target"
output = parent / "listing.json"
root.mkdir()
target.mkdir()
(root / "present.txt").write_text("present")
(target / "outside.txt").write_text("outside")
try:
(root / "linked").symlink_to(target, target_is_directory=True)
except OSError as error:
self.skipTest(f"directory symlinks unavailable: {error}")
result = self.run_listing(root, output)
self.assertEqual(result.returncode, 0, result.stderr)
listing = json.loads(output.read_text())
self.assertEqual(
[item["path"] for item in listing["files"]],
[os.path.relpath(root / "present.txt", Path.home())],
)
self.assertEqual(
listing["skipped"], [os.path.relpath(root / "linked", Path.home())]
)
def test_missing_root_does_not_produce_an_empty_successful_inventory(self):
with tempfile.TemporaryDirectory() as parent:
parent = Path(parent)
output = parent / "listing.json"
result = self.run_listing(parent / "missing", output)
self.assertNotEqual(result.returncode, 0)
self.assertFalse(output.exists())
if __name__ == "__main__":
unittest.main()
+26 -521
View File
@@ -1,35 +1,20 @@
import com.atxx.jhmcp.*;
import com.google.gson.*;
import com.android.tools.smali.dexlib2.Opcodes;
import com.android.tools.smali.dexlib2.dexbacked.DexBackedDexFile;
import com.android.tools.smali.dexlib2.dexbacked.DexBackedClassDef;
import com.android.tools.smali.dexlib2.dexbacked.DexBackedMethod;
import io.modelcontextprotocol.kotlin.sdk.server.*;
import io.modelcontextprotocol.kotlin.sdk.types.*;
import jadx.api.*;
import jadx.api.plugins.input.data.annotations.EncodedValue;
import jadx.api.plugins.input.data.attributes.JadxAttrType;
import jadx.api.plugins.input.data.ICodeReader;
import jadx.api.plugins.input.data.IMethodRef;
import jadx.api.plugins.input.insns.InsnIndexType;
import jadx.api.plugins.input.insns.Opcode;
import jadx.core.codegen.TypeGen;
import jadx.core.dex.info.MethodInfo;
import jadx.core.dex.instructions.args.ArgType;
import jadx.core.dex.nodes.*;
import jadx.core.xmlgen.BinaryXMLParser;
import java.io.PrintStream;
import java.io.StringReader;
import java.io.InputStream;
import java.io.IOException;
import java.nio.file.Path;
import java.nio.charset.StandardCharsets;
import java.util.*;
import java.util.concurrent.CountDownLatch;
import java.util.stream.Collectors;
import java.util.zip.CRC32;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
import kotlin.Unit;
import kotlin.ResultKt;
import kotlin.coroutines.Continuation;
@@ -52,25 +37,16 @@ class ReaJadxBridge {
private Map<String, List<ClassSnapshot>> classes = Map.of();
private JadxSession manifestSession;
private ManifestSnapshot manifestSnapshot;
private ExternalInventory externalInventory;
private record MethodSnapshot(String name, String rawName, String shortId, List<String> bodyAbsenceMarkers,
Map<String, Object> summary) {}
private record ClassSnapshot(JavaClass node, List<MethodSnapshot> methods, Map<String, Object> summary,
Map<String, Object> summaryWithValues) {}
private record ClassSnapshot(JavaClass node, List<MethodSnapshot> methods, Map<String, Object> summary) {}
private record ManifestSnapshot(String text, Map<String, Object> summary) {}
private record ExternalMethod(MethodInfo info, Map<String, Object> summary) {}
private record ExternalInventory(JadxSession session, String className, String methodName,
List<ExternalMethod> methods) {}
private static final class InputFailure extends IllegalArgumentException {
final String field;
InputFailure(String field, String message) { super(message); this.field = field; }
}
private static final class ReferenceLimit extends IllegalArgumentException {
final String stage;
ReferenceLimit(String stage) { super("External reference representation limit"); this.stage = stage; }
}
private ReaJadxBridge(SessionHolder holder) { this.holder = holder; }
@@ -166,14 +142,6 @@ class ReaJadxBridge {
return value.getAsString();
}
private static boolean flag(JsonObject args, String name) {
JsonElement value = args.get(name);
if (value == null) return false;
if (!value.isJsonPrimitive() || !value.getAsJsonPrimitive().isBoolean())
throw new IllegalArgumentException(name + " must be a boolean");
return value.getAsBoolean();
}
private static int integer(JsonObject args, String name, int fallback) {
JsonElement value = args.get(name);
if (value == null) return fallback;
@@ -197,206 +165,6 @@ class ReaJadxBridge {
private void clearManifest() {
manifestSession = null;
manifestSnapshot = null;
externalInventory = null;
}
// One bounded DEX at a time, beside the already admitted analysis JVM. The
// response allowance accounts for worst-case JSON escaping before retention.
private static final int NATIVE_DEX_BYTES = 64 * 1024 * 1024;
private static final long NATIVE_TOTAL_DEX_BYTES = 256L * 1024 * 1024;
// This JSON is escaped again inside an 8 MiB MCP frame.
private static final long NATIVE_JSON_BYTES = 4L * 1024 * 1024;
/** The bundled decoder allocates its declared UTF-16 length before validating bytes. */
private static final class NativeDex extends DexBackedDexFile {
private OptionalIndexedSection<String> boundedStrings;
NativeDex(byte[] bytes) { super(Opcodes.getDefault(), bytes); }
@Override public OptionalIndexedSection<String> getStringSection() {
if (boundedStrings == null) {
var source = super.getStringSection();
boundedStrings = new OptionalIndexedSection<>() {
@Override public int size() { return source.size(); }
@Override public int getOffset(int index) { return source.getOffset(index); }
@Override public String getOptional(int index) { return index == -1 ? null : get(index); }
@Override public String get(int index) {
int offset = getBuffer().readSmallUint(source.getOffset(index));
int units = getDataBuffer().readerAt(offset).readSmallUleb128();
if (units > NATIVE_JSON_BYTES / 6
|| units > getDataBuffer().getBuf().length - offset)
throw new IllegalArgumentException("DEX string exceeds the native decoding allowance");
return source.get(index);
}
};
}
return boundedStrings;
}
}
private static void checkNativeInterrupted() {
if (Thread.currentThread().isInterrupted())
throw new java.util.concurrent.CancellationException("Native declaration inventory interrupted");
}
private static final class NativeInventory {
final List<Map<String, Object>> methods = new ArrayList<>();
final List<String> unknowns = new ArrayList<>();
long retainedBytes = 1024;
int dexEntries;
int examinedDexEntries;
int examinedClasses;
int examinedMethods;
boolean partial;
boolean diagnosticLimit;
void unknown(String reason) {
partial = true;
long bytes = 64L + reason.length() * 6L;
if (unknowns.size() < 16 && retainedBytes + bytes <= NATIVE_JSON_BYTES - 4096) {
unknowns.add(reason);
retainedBytes += bytes;
} else if (!diagnosticLimit) {
diagnosticLimit = true;
unknowns.add("Additional native-inventory limitations exceed the retained diagnostic allowance.");
}
}
boolean retain(Map<String, Object> method, long bytes) {
if (retainedBytes + bytes > NATIVE_JSON_BYTES - 4096) {
unknown("Native declarations exceed the 4 MiB retained JSON allowance; remaining declarations are unknown.");
return false;
}
retainedBytes += bytes;
methods.add(method);
return true;
}
Map<String, Object> result() {
return Map.of("scope", "apk-root-dex-declarations", "methods", List.copyOf(methods),
"coverage", Map.of("status", partial ? "partial" : "complete",
"dex_entries", dexEntries, "examined_dex_entries", examinedDexEntries,
"examined_classes", examinedClasses, "examined_methods", examinedMethods),
"unknowns", List.copyOf(unknowns));
}
}
/** Match Android's standard root multidex carriers, without a basename guess. */
private static boolean nativeDexEntry(String name) {
return name.equals("classes.dex") || name.matches("classes[0-9]+\\.dex");
}
private static byte[] readNativeDex(ZipFile archive, ZipEntry entry) throws IOException {
long size = entry.getSize();
if (size < 0 || size > NATIVE_DEX_BYTES)
throw new IllegalArgumentException("DEX exceeds the 64 MiB single-member read allowance");
byte[] bytes = new byte[(int) size];
CRC32 crc = new CRC32();
try (InputStream input = archive.getInputStream(entry)) {
int offset = 0;
while (offset < bytes.length) {
checkNativeInterrupted();
int read = input.read(bytes, offset, Math.min(65536, bytes.length - offset));
if (read < 0) throw new IOException("DEX is shorter than its ZIP declaration");
if (read == 0) continue;
crc.update(bytes, offset, read);
offset += read;
}
if (input.read() != -1) throw new IOException("DEX exceeds its ZIP declaration");
}
if (entry.getCrc() < 0 || entry.getCrc() != crc.getValue())
throw new IOException("DEX does not match its ZIP CRC32 declaration");
return bytes;
}
/** Raw DEX ownership precedes JADX's duplicate-class selection and aliases. */
static Map<String, Object> nativeDeclarations(Path apk) {
NativeInventory inventory = new NativeInventory();
try (ZipFile archive = new ZipFile(apk.toFile())) {
Map<String, Integer> names = new HashMap<>();
var directory = archive.entries();
while (directory.hasMoreElements()) {
checkNativeInterrupted();
ZipEntry entry = directory.nextElement();
if (!entry.isDirectory() && nativeDexEntry(entry.getName())) {
inventory.dexEntries++;
names.merge(entry.getName(), 1, Integer::sum);
}
}
long totalBytes = 0;
int entryIndex = -1;
directory = archive.entries();
while (directory.hasMoreElements()) {
checkNativeInterrupted();
ZipEntry entry = directory.nextElement();
entryIndex++;
String path = entry.getName();
if (entry.isDirectory() || !nativeDexEntry(path)) continue;
if (names.get(path) != 1) {
inventory.unknown("Duplicate root DEX entry name cannot be selected uniquely by the ZIP reader: " + path);
continue;
}
if (entry.getSize() < 0 || entry.getSize() > NATIVE_DEX_BYTES
|| entry.getSize() > NATIVE_TOTAL_DEX_BYTES - totalBytes) {
inventory.unknown("Root DEX member exceeds the 64 MiB member or 256 MiB total read allowance: " + path);
continue;
}
totalBytes += entry.getSize();
try {
DexBackedDexFile dex = new NativeDex(readNativeDex(archive, entry));
var classes = dex.getClassSection();
for (int classIndex = 0; classIndex < classes.size(); classIndex++) {
checkNativeInterrupted();
DexBackedClassDef cls = classes.get(classIndex);
String type = cls.getType();
if (!type.startsWith("L") || !type.endsWith(";"))
throw new IllegalArgumentException("Class definition has no reference type");
String className = type.substring(1, type.length() - 1).replace('/', '.');
// Passing false retains duplicate encoded method references too.
for (Iterable<? extends DexBackedMethod> group : List.of(
cls.getDirectMethods(false), cls.getVirtualMethods(false))) {
for (DexBackedMethod method : group) {
checkNativeInterrupted();
inventory.examinedMethods++;
if ((method.getAccessFlags() & 0x100) == 0) continue;
String methodName = method.getName();
String returnType = method.getReturnType();
long characters = 2L + returnType.length();
for (CharSequence parameter : method.getParameterTypes()) {
characters += parameter.length();
if (characters * 6L > NATIVE_JSON_BYTES) break;
}
long bytes = 512L + 6L * (className.length() + methodName.length()
+ path.length() + characters);
if (inventory.retainedBytes + bytes > NATIVE_JSON_BYTES - 4096) {
inventory.unknown("Native declarations exceed the 4 MiB retained JSON allowance; remaining declarations are unknown.");
return inventory.result();
}
StringBuilder descriptor = new StringBuilder((int) characters).append('(');
for (CharSequence parameter : method.getParameterTypes()) descriptor.append(parameter);
descriptor.append(')').append(returnType);
Map<String, Object> row = Map.of("source_index", inventory.methods.size(),
"class_name", className, "method_name", methodName, "descriptor", descriptor.toString(),
"dex_entry", path, "dex_entry_index", entryIndex, "class_index", classIndex,
"method_index", method.getMethodIndex());
if (!inventory.retain(row, bytes)) return inventory.result();
}
}
inventory.examinedClasses++;
}
inventory.examinedDexEntries++;
} catch (IOException | RuntimeException failure) {
if (failure instanceof java.util.concurrent.CancellationException) throw (RuntimeException) failure;
inventory.unknown("Root DEX native declarations could not be fully read from " + path + ": "
+ failure.getClass().getSimpleName()
+ (failure.getMessage() == null ? "" : " (" + failure.getMessage() + ")"));
}
}
} catch (IOException failure) {
// A failed archive close must fail the operation so the provider retires
// its owned JVM; do not cache a result with lost file-resource ownership.
throw new IllegalArgumentException("APK ZIP native declaration inventory could not finish its reader", failure);
}
return inventory.result();
}
/** Original archive identity, never a deobfuscated basename or name suffix. */
@@ -411,7 +179,10 @@ class ReaJadxBridge {
selected = resource;
}
if (selected == null) throw new IllegalArgumentException("Root AndroidManifest.xml not found");
String xml = manifestText(selected);
var content = selected.loadContent();
if (content == null || content.getDataType() != jadx.core.xmlgen.ResContainer.DataType.TEXT)
throw new IllegalArgumentException("Root AndroidManifest.xml did not decode as text");
String xml = content.getText().getCodeStr();
ManifestSnapshot snapshot = new ManifestSnapshot(xml, manifestSummary(xml));
manifestSnapshot = snapshot;
manifestSession = current;
@@ -419,23 +190,6 @@ class ReaJadxBridge {
return manifestSnapshot;
}
/** Decode with resource names but without JADX's class alias substitution. */
static String manifestText(ResourceFile resource) {
var decompiler = resource.getDecompiler();
// BinaryXMLParser resolves android:name against its root's class identities
// and replaces raw '$' and relative names with Java-style aliases. A fresh
// root owns no classes, while its resource-name table preserves reference
// rendering from the loaded target.
RootNode root = new RootNode(decompiler.getArgs());
root.getConstValues().setResourcesNames(decompiler.getRoot().getConstValues().getResourcesNames());
try {
return ResourcesLoader.decodeStream(resource,
(size, input) -> new BinaryXMLParser(root).parse(input).getCodeStr());
} catch (jadx.core.utils.exceptions.JadxException failure) {
throw new IllegalArgumentException("Root AndroidManifest.xml did not decode as text", failure);
}
}
/** One parsed XML owner supplies namespace-aware values alongside the exact decoded text. */
static Map<String, Object> manifestSummary(String xml) {
Element root;
@@ -479,7 +233,7 @@ class ReaJadxBridge {
return Collections.unmodifiableMap(summary);
}
private void indexClasses() {
private ClassSnapshot selected(JsonObject args) {
JadxSession current = session();
if (indexedSession != current) {
Map<String, List<ClassSnapshot>> index = new LinkedHashMap<>();
@@ -505,46 +259,21 @@ class ReaJadxBridge {
.map(Object::toString).collect(Collectors.joining(", "))
+ "): " + method.getReturnType().toString();
return new MethodSnapshot(name, rawName, shortId, List.copyOf(bodyAbsenceMarkers), Map.of(
"name", name, "raw_name", rawName, "signature", signature,
"name", name, "signature", signature,
"dex_descriptor", descriptor,
"is_constructor", method.isConstructor(), "def_pos", 0));
}).toList();
List<FieldNode> orderedFields = new ArrayList<>(node.getFields());
orderedFields.sort(Comparator.comparing(FieldNode::getAlias));
List<Map<String, Object>> fields = new ArrayList<>();
List<Map<String, Object>> valuedFields = new ArrayList<>();
boolean anyValue = false;
for (FieldNode field : orderedFields) {
// Raw DEX identity for mapping lookups; the display type can carry generics.
Map<String, Object> entry = Map.of("name", field.getAlias(),
"raw_name", field.getFieldInfo().getName(),
"type", field.getType().toString(),
"dex_type", TypeGen.signature(field.getFieldInfo().getType()), "def_pos", 0);
fields.add(entry);
Map<String, Object> value = dexStaticValue(field);
if (value == null) valuedFields.add(entry);
else {
anyValue = true;
Map<String, Object> valued = new LinkedHashMap<>(entry);
valued.put("dex_static_value", value);
valuedFields.add(Collections.unmodifiableMap(valued));
}
}
List<Map<String, Object>> fields = node.getFields().stream()
.sorted(Comparator.comparing(FieldNode::getAlias))
.map(field -> Map.<String, Object>of("name", field.getAlias(),
"type", field.getType().toString(), "def_pos", 0)).toList();
List<String> inner = node.getInnerClasses().stream().map(ClassNode::getFullName).toList();
List<String> innerRaw = node.getInnerClasses().stream().map(ClassNode::getRawName).toList();
Map<String, Object> summary = Map.of(
"full_name", cls.getFullName(), "raw_name", cls.getRawName(), "name", cls.getName(),
"method_count", methods.size(), "field_count", fields.size(),
"inner_class_count", inner.size(), "inner_classes", inner, "inner_class_raw_names", innerRaw,
"methods", methods.stream().map(MethodSnapshot::summary).toList(), "fields", List.copyOf(fields));
// DEX static values are opt-in, so method and reference lookups keep the member-only reply.
Map<String, Object> summaryWithValues = summary;
if (anyValue) {
Map<String, Object> copy = new LinkedHashMap<>(summary);
copy.put("fields", List.copyOf(valuedFields));
summaryWithValues = Collections.unmodifiableMap(copy);
}
ClassSnapshot snapshot = new ClassSnapshot(cls, methods, summary, summaryWithValues);
"inner_class_count", inner.size(), "inner_classes", inner,
"methods", methods.stream().map(MethodSnapshot::summary).toList(), "fields", fields);
ClassSnapshot snapshot = new ClassSnapshot(cls, methods, summary);
index.computeIfAbsent(cls.getFullName(), ignored -> new ArrayList<>()).add(snapshot);
// Keep the binary identity supplied by JADX; '$' can also be a literal
// name character, so replacing separators would invent lookup aliases.
@@ -555,10 +284,6 @@ class ReaJadxBridge {
classes = Collections.unmodifiableMap(index);
indexedSession = current;
}
}
private ClassSnapshot selected(JsonObject args) {
indexClasses();
String name = string(args, "class_name");
List<ClassSnapshot> candidates = classes.get(name);
if (candidates == null) throw new InputFailure("class_name", "Exact class not found: " + name
@@ -568,28 +293,6 @@ class ReaJadxBridge {
return candidates.get(0);
}
/** A static final field's class-definition static value; a static initializer can still assign it. */
private static Map<String, Object> dexStaticValue(FieldNode field) {
if (!field.getAccessFlags().isStatic() || !field.getAccessFlags().isFinal()) return null;
EncodedValue encoded = field.get(JadxAttrType.CONSTANT_VALUE);
if (encoded == null) return null;
Object value = encoded.getValue();
// JADX also attaches the shared null to static finals without a value, so null is never reported.
// Wide and floating values keep their Java text instead of a lossy JSON number.
return switch (encoded.getType()) {
case ENCODED_BOOLEAN -> Map.of("type", "boolean", "value", value);
case ENCODED_BYTE -> Map.of("type", "byte", "value", value);
case ENCODED_SHORT -> Map.of("type", "short", "value", value);
case ENCODED_CHAR -> Map.of("type", "char", "value", (int) (Character) value);
case ENCODED_INT -> Map.of("type", "int", "value", value);
case ENCODED_LONG -> Map.of("type", "long", "value", value.toString());
case ENCODED_FLOAT -> Map.of("type", "float", "value", value.toString());
case ENCODED_DOUBLE -> Map.of("type", "double", "value", value.toString());
case ENCODED_STRING -> Map.of("type", "string", "value", value);
default -> null;
};
}
private List<MethodSnapshot> methods(ClassSnapshot cls, JsonObject args) {
String name = string(args, "method_name");
List<MethodSnapshot> result = cls.methods().stream().filter(method -> method.name().equals(name)).toList();
@@ -658,18 +361,12 @@ class ReaJadxBridge {
// JSON-RPC/CallToolResult envelope and generated numeric request id, and
// account for JSON text being escaped again inside TextContent.
private static final int MAX_XREF_TEXT_BYTES = 8 * 1024 * 1024 - 1024;
// Leave room for the prototype reply, reference reply, retained raw calls,
// normalized Evidence and their text/structured MCP copies, not just one frame.
private static final int MAX_EXTERNAL_XREF_BYTES = MAX_XREF_TEXT_BYTES / 8;
private static int escapedJsonBytes(Object value) {
return JSON.toJson(JSON.toJson(value)).getBytes(StandardCharsets.UTF_8).length;
}
private static final class InvokeBudget {
long remaining;
InvokeBudget(long remaining) { this.remaining = Math.max(0, remaining); }
boolean fitsText(long characters) {
return remaining >= 256 && characters <= (remaining - 256) / 12;
}
boolean admit(Map<String, Object> site) {
// Check a conservative two-layer escape bound before serialization.
long bound = 256;
@@ -684,55 +381,26 @@ class ReaJadxBridge {
}
private record DirectInvokes(List<Map<String, Object>> sites, boolean exhaustive) {}
@FunctionalInterface private interface InvokeVisitor {
void accept(MethodInfo target, int offset, int index, Opcode opcode);
}
/** Decode raw invoke identity once at the shared bridge boundary, without generating source. */
private static void visitInvokes(MethodNode node, InvokeVisitor visitor) {
checkNativeInterrupted();
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
MethodNode node = source.getMethodNode();
ICodeReader code = node.getCodeReader();
if (code == null) return;
if (code == null) return null;
List<Map<String, Object>> sites = new ArrayList<>();
boolean[] exhaustive = {true};
code.visitInstructions(insn -> {
checkNativeInterrupted();
insn.decode();
if (insn.getIndexType() != InsnIndexType.METHOD_REF) return;
Opcode opcode = insn.getOpcode();
if (!opcode.name().startsWith("INVOKE_")) return;
IMethodRef ref = insn.getIndexAsMethod();
ref.load();
visitor.accept(MethodInfo.fromRef(node.root(), ref), insn.getOffset(), insn.getIndex(), opcode);
});
}
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
return directInvokes(source.getMethodNode(), target, budget);
}
private static void visitDirectInvokes(MethodNode node, MethodInfo target, InvokeVisitor visitor) {
visitInvokes(node, (referenced, offset, index, opcode) -> {
if (referenced.getRawFullId().equals(target.getRawFullId()))
visitor.accept(referenced, offset, index, opcode);
});
}
private static boolean referencesMethod(MethodNode node, MethodInfo target) {
boolean[] found = {false};
visitDirectInvokes(node, target, (referenced, offset, index, opcode) -> found[0] = true);
return found[0];
}
private static DirectInvokes directInvokes(MethodNode node, MethodInfo target, InvokeBudget budget) {
if (node.getCodeReader() == null) return null;
List<Map<String, Object>> sites = new ArrayList<>();
boolean[] exhaustive = {true};
visitDirectInvokes(node, target, (referenced, offset, index, opcode) -> {
if (!MethodInfo.fromRef(node.root(), ref).getRawFullId().equals(target.getRawFullId())) return;
Map<String, Object> site = Map.of(
"dex_entry", node.getInputFileName(),
"source_raw_method_id", node.getMethodInfo().getRawFullId(),
"dex_code_unit_offset", offset,
"dex_method_index", index,
"dex_code_unit_offset", insn.getOffset(),
"dex_method_index", insn.getIndex(),
"opcode", opcode.name());
if (budget.admit(site)) sites.add(site);
else exhaustive[0] = false;
@@ -740,143 +408,6 @@ class ReaJadxBridge {
return new DirectInvokes(sites, exhaustive[0]);
}
private List<ClassSnapshot> sourceClasses() {
indexClasses();
Set<ClassSnapshot> seen = Collections.newSetFromMap(new IdentityHashMap<>());
List<ClassSnapshot> result = new ArrayList<>();
for (List<ClassSnapshot> group : classes.values())
for (ClassSnapshot source : group) if (seen.add(source)) result.add(source);
result.sort(Comparator.comparing(source -> source.node().getRawName()));
return result;
}
private static long displayTypeCharacters(ArgType type) {
if (type.isArray())
return 2L * type.getArrayDimension() + displayTypeCharacters(type.getArrayRootElement());
if (type.isPrimitive()) return type.getPrimitiveType().getLongName().length();
if (type.isObject() && !type.isGeneric() && !type.isGenericType()) return type.getObject().length();
throw new IllegalArgumentException("External prototype contains a non-DEX type");
}
private static Map<String, Object> externalMethodSummary(MethodInfo info, InvokeBudget budget) {
String name = info.getName();
long signatureCharacters = name.length() + 4L + displayTypeCharacters(info.getReturnType())
+ 2L * Math.max(0, info.getArgsCount() - 1);
for (ArgType type : info.getArgumentsTypes()) signatureCharacters += displayTypeCharacters(type);
long descriptorCharacters = info.getShortId().length() - (long) name.length();
// Use parsed type facts and the existing short-id length before copying/joining strings.
if (!budget.fitsText(2L * name.length() + descriptorCharacters + signatureCharacters))
throw new ReferenceLimit("method_inventory");
String descriptor = info.getShortId().substring(name.length());
String signature = name + "(" + info.getArgumentsTypes().stream()
.map(Object::toString).collect(Collectors.joining(", "))
+ "): " + info.getReturnType();
Map<String, Object> summary = Map.of("name", name, "raw_name", name, "signature", signature,
"dex_descriptor", descriptor, "is_constructor", info.isConstructor(), "def_pos", 0);
if (!budget.admit(summary)) throw new ReferenceLimit("method_inventory");
return summary;
}
/** Only directly referenced prototypes, not invented definitions or an external classpath. */
private List<ExternalMethod> externalMethods(JsonObject args) {
String className = string(args, "class_name");
String methodName = string(args, "method_name");
JadxSession current = session();
if (externalInventory != null && externalInventory.session() == current
&& externalInventory.className().equals(className) && externalInventory.methodName().equals(methodName))
return externalInventory.methods();
// Retain only the latest bounded selection, never results from a replaced APK/session.
externalInventory = null;
SortedMap<String, ExternalMethod> methods = new TreeMap<>();
boolean[] observedClass = {false};
InvokeBudget budget = new InvokeBudget(MAX_EXTERNAL_XREF_BYTES
- 4096L - 12L * (className.length() + (long) methodName.length()));
for (ClassSnapshot source : sourceClasses()) {
for (MethodSnapshot selected : source.methods()) {
visitInvokes(method(source, selected).getMethodNode(), (info, offset, index, opcode) -> {
if (!info.getDeclClass().getRawName().equals(className)) return;
observedClass[0] = true;
if (!info.getName().equals(methodName) || methods.containsKey(info.getShortId())) return;
methods.put(info.getShortId(), new ExternalMethod(info, externalMethodSummary(info, budget)));
});
}
}
if (!observedClass[0]) throw new InputFailure("class_name",
"No APK definition or direct DEX invoke references the exact class: " + className);
if (methods.isEmpty()) throw new InputFailure("method_name",
"No direct DEX invoke references " + className + "." + methodName);
List<ExternalMethod> result = List.copyOf(methods.values());
externalInventory = new ExternalInventory(current, className, methodName, result);
return result;
}
private Map<String, Object> externalSummary(JsonObject args) {
List<ExternalMethod> methods = externalMethods(args);
return Map.of("class_name", string(args, "class_name"), "reference_target", "external",
"observed_method_count", methods.size(),
"prototypes", methods.stream().map(ExternalMethod::summary).toList());
}
private record ExternalCaller(MethodNode node, Map<String, Object> row) {}
/** Reserve the complete using-node inventory before optional instruction positions. */
private List<ExternalCaller> externalCallers(MethodInfo target, InvokeBudget budget) {
List<ExternalCaller> callers = new ArrayList<>();
for (ClassSnapshot source : sourceClasses()) {
String className = String.valueOf(source.summary().get("full_name"));
Map<String, Integer> overloads = new HashMap<>();
for (MethodSnapshot candidate : source.methods()) {
int overload = overloads.merge(candidate.name(), 1, Integer::sum) - 1;
MethodNode node = method(source, candidate).getMethodNode();
if (!referencesMethod(node, target)) continue;
if (classes.get(className).size() != 1)
throw new IllegalArgumentException("External caller has an ambiguous APK class identity: " + className);
Map<String, Object> row = new LinkedHashMap<>(Map.of(
"kind", "method", "name", candidate.name(), "full_name", className + "." + candidate.name(),
"containing_class", className, "top_class", source.node().getTopParentClass().getFullName(),
"def_pos", 0, "source_selector", Map.of("class_name", className,
"method_name", candidate.name(), "overload_index", overload),
"direct_invokes", List.of(), "direct_invokes_exhaustive", false));
if (!budget.admit(row)) throw new ReferenceLimit("caller_inventory");
callers.add(new ExternalCaller(node, row));
}
}
return List.copyOf(callers);
}
private Map<String, Object> externalReferences(JsonObject args) {
List<ExternalMethod> methods = externalMethods(args);
if (methods.size() != 1 && !args.has("overload_index"))
throw new InputFailure("overload_index", "Select one of " + methods.size() + " observed external prototypes");
int index = integer(args, "overload_index", 0);
if (index >= methods.size()) throw new InputFailure("overload_index", "External prototype index out of range: " + index);
ExternalMethod method = methods.get(index);
MethodInfo target = method.info();
Map<String, Object> selected = method.summary();
InvokeBudget budget = new InvokeBudget(MAX_EXTERNAL_XREF_BYTES
- 4096L - escapedJsonBytes(selected));
List<ExternalCaller> callers = externalCallers(target, budget);
List<Map<String, Object>> items = new ArrayList<>();
for (ExternalCaller caller : callers) {
DirectInvokes invokes = directInvokes(caller.node(), target, budget);
caller.row().put("direct_invokes", invokes.sites());
caller.row().put("direct_invokes_exhaustive", invokes.exhaustive());
items.add(caller.row());
}
Map<String, Object> result = new LinkedHashMap<>();
result.put("target", string(args, "class_name") + "." + target.getName());
result.put("reference_target", "external");
result.put("reference_scope", "direct_dex_invokes");
result.put("dex_descriptor", selected.get("dex_descriptor"));
result.put("reported_signature", selected.get("signature"));
result.put("overload_index", index);
result.put("total", items.size());
result.put("count", items.size());
result.put("resolve_line", false);
result.put("items", items);
return result;
}
private CallToolResult handle(String name, kotlinx.serialization.json.JsonObject input) {
JsonObject args = JsonParser.parseString(input.toString()).getAsJsonObject();
if (name.equals("rea_jvm_status")) return json(Map.of(
@@ -885,8 +416,6 @@ class ReaJadxBridge {
"available_processors", Runtime.getRuntime().availableProcessors(),
"java_version", System.getProperty("java.version"),
"metadata_scope", "parsed_members"));
if (name.equals("rea_native_declarations"))
return json(nativeDeclarations(Path.of(string(args, "path"))));
JadxSession current = session();
if (name.equals("get_app_info")) return json(manifest(current).summary());
if (name.equals("get_android_manifest"))
@@ -913,15 +442,8 @@ class ReaJadxBridge {
"scope", "top_level", "offset", offset, "limit", limit,
"items", items));
}
indexClasses();
if (flag(args, "reference_target") && !classes.containsKey(string(args, "class_name"))) {
if (name.equals("get_class_summary")) return json(externalSummary(args));
if (name.equals("get_xrefs_to_method")) return json(externalReferences(args));
throw new InputFailure("class_name", "External prototypes are only available to method reference queries");
}
ClassSnapshot cls = selected(args);
if (name.equals("get_class_summary"))
return json(flag(args, "dex_static_values") ? cls.summaryWithValues() : cls.summary());
if (name.equals("get_class_summary")) return json(cls.summary());
if (name.equals("get_method_body")) {
List<MethodSnapshot> overloads = methods(cls, args);
int index = integer(args, "overload_index", 0);
@@ -992,9 +514,6 @@ class ReaJadxBridge {
try { return bridge.handle(name, request.getArguments()); }
catch (InputFailure error) { return text(JSON.toJson(Map.of(
"kind", "invalid-input", "field", error.field, "message", error.getMessage())), true); }
catch (ReferenceLimit error) { return text(JSON.toJson(Map.of(
"kind", "reference-output-limit", "stage", error.stage,
"limit_bytes", MAX_EXTERNAL_XREF_BYTES)), true); }
catch (IllegalArgumentException error) { return text(error.getMessage(), true); }
});
}
@@ -1007,7 +526,7 @@ class ReaJadxBridge {
SessionHolder holder = new SessionHolder(defaults);
ServerCapabilities capabilities = new ServerCapabilities(
new ServerCapabilities.Tools(null), null, null, null, null, null, null, Map.of());
Server server = new Server(new Implementation("rea-jadx-bridge", "10", null, null, null),
Server server = new Server(new Implementation("rea-jadx-bridge", "6", null, null, null),
new ServerOptions(capabilities), "", ignored -> Unit.INSTANCE);
SessionToolsKt.registerSessionTools(server, holder);
ClassToolsKt.registerClassTools(server, holder);
@@ -1027,11 +546,6 @@ class ReaJadxBridge {
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of()), List.of(), null),
"Observed worker JVM and metadata scope", null, null, null, null, null, null);
server.addTool(diagnostics, (connection, request, continuation) -> bridge.handle("rea_jvm_status", request.getArguments()));
registerBridgeTool(server, bridge, "rea_native_declarations",
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of("path",
new kotlinx.serialization.json.JsonObject(Map.of("type",
kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))))), List.of("path"), null),
"Original ACC_NATIVE declarations from the admitted APK's standard root DEX members, without loading or decompiling the APK.");
for (String name : List.of("get_app_info", "get_android_manifest")) {
String description = name.equals("get_app_info")
? "Namespace-aware summary of the exact root AndroidManifest.xml resource."
@@ -1054,18 +568,9 @@ class ReaJadxBridge {
if (name.equals("list_classes")) properties.put("query",
new kotlinx.serialization.json.JsonObject(Map.of(
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))));
if (name.equals("get_class_summary")) properties.put("dex_static_values",
new kotlinx.serialization.json.JsonObject(Map.of(
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("boolean"))));
if (name.equals("get_class_summary") || name.equals("get_xrefs_to_method"))
properties.put("reference_target", new kotlinx.serialization.json.JsonObject(Map.of(
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("boolean"))));
if (name.equals("get_class_summary")) properties.put("method_name",
new kotlinx.serialization.json.JsonObject(Map.of(
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))));
String description = switch (name) {
case "list_classes" -> "Pages of case-sensitive matching top-level class names; offset and next_offset are inventory positions, limit bounds matching rows. Omitted or empty query inventories all names.";
case "get_class_summary" -> "Parsed member metadata for one exact class name; no source generation. Set dex_static_values to include encoded static final field values.";
case "get_class_summary" -> "Parsed member metadata for one exact class name; no source generation.";
case "get_method_body" -> "Source for an exact class and stable same-name overload index; static initializers use smali to preserve folded field initialization.";
default -> "Complete static using-node inventory for an exact class or selected method overload; one node may contain multiple invoke instructions, and source lines are unresolved.";
};
File diff suppressed because it is too large. Load diff
-193
View File
@@ -1,193 +0,0 @@
import ghidra.app.script.GhidraScript;
import ghidra.program.database.mem.FileBytes;
import ghidra.program.model.address.Address;
import ghidra.program.model.address.AddressSpace;
import ghidra.program.model.listing.Program;
import ghidra.program.model.mem.Memory;
import ghidra.program.model.mem.MemoryBlock;
import ghidra.program.model.mem.MemoryBlockSourceInfo;
import java.nio.file.Files;
import java.nio.file.LinkOption;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
/**
* Rebuild a GameCube/Wii DOL memory map from the validated load ranges REA's TypeScript
* admission wrote. BinaryLoader maps the whole file once at address 0; this
* script replaces that block with text, data, and BSS blocks at their load addresses.
*/
public final class ReaGhidraPrepareDol extends GhidraScript {
// Canonical v2 header, 18 section rows, 12 gap rows and entry, using uint32 hex.
private static final int MAX_PLAN_BYTES = 13 + 18 * 49 + 12 * 30 + 17;
private static final class Section {
final String kind;
final int index;
final long fileOffset;
final long address;
final long size;
Section(String kind, int index, long fileOffset, long address, long size) {
this.kind = kind;
this.index = index;
this.fileOffset = fileOffset;
this.address = address;
this.size = size;
}
}
private record BssGap(long address, long size) {}
@Override
public void run() throws Exception {
String[] arguments = getScriptArgs();
if (arguments.length != 1) {
throw new IllegalArgumentException(
"GameCube DOL preparation requires exactly one section table path"
);
}
if (!currentProgram.getExecutableFormat().equals("Raw Binary") ||
!currentProgram.getLanguageID().getIdAsString().startsWith("PowerPC:BE:32:")) {
throw new IllegalArgumentException(
"GameCube DOL preparation requires a BinaryLoader import in big-endian 32-bit PowerPC"
);
}
Path tablePath = Path.of(arguments[0]);
if (!Files.isRegularFile(tablePath, LinkOption.NOFOLLOW_LINKS)) {
throw new IllegalArgumentException("GameCube DOL section table is not a regular file");
}
if (Files.size(tablePath) > MAX_PLAN_BYTES) {
throw new IllegalArgumentException("GameCube DOL plan exceeds the canonical format's byte bound");
}
List<Section> sections = new ArrayList<>();
List<BssGap> bssGaps = new ArrayList<>();
int expectedSections = -1;
int expectedGaps = -1;
long entry = -1;
for (String line : Files.readAllLines(tablePath)) {
String[] fields = line.trim().split("\\s+");
switch (fields[0]) {
case "plan" -> {
requireFields(fields, 4, line);
if (expectedSections >= 0 || !fields[1].equals("2")) {
throw new IllegalArgumentException("Duplicate or unsupported GameCube DOL plan header");
}
expectedSections = Integer.parseInt(fields[2]);
expectedGaps = Integer.parseInt(fields[3]);
// A DOL has 18 slots, including at most 11 data slots and 12 BSS gaps.
if (expectedSections < 1 || expectedSections > 18 || expectedGaps < 0 || expectedGaps > 12) {
throw new IllegalArgumentException("GameCube DOL plan counts exceed the format's slot bounds");
}
}
case "section" -> {
requireFields(fields, 6, line);
if (expectedSections < 0 || sections.size() >= expectedSections) {
throw new IllegalArgumentException("GameCube DOL section exceeds its declared count");
}
sections.add(new Section(
fields[1],
Integer.parseInt(fields[2]),
Long.decode(fields[3]),
Long.decode(fields[4]),
Long.decode(fields[5])
));
}
case "bss-gap" -> {
requireFields(fields, 3, line);
if (expectedGaps < 0 || bssGaps.size() >= expectedGaps) {
throw new IllegalArgumentException("GameCube DOL BSS gap exceeds its declared count");
}
long address = Long.decode(fields[1]);
long size = Long.decode(fields[2]);
if (address < 0 || size <= 0) {
throw new IllegalArgumentException("GameCube DOL BSS gap is not a positive range");
}
bssGaps.add(new BssGap(address, size));
}
case "entry" -> {
requireFields(fields, 2, line);
entry = Long.decode(fields[1]);
}
default -> throw new IllegalArgumentException("Unknown GameCube DOL table line: " + line);
}
}
if (expectedSections != sections.size() || expectedGaps != bssGaps.size() || entry < 0) {
throw new IllegalArgumentException("GameCube DOL section table is incomplete");
}
Memory memory = currentProgram.getMemory();
List<FileBytes> files = memory.getAllFileBytes();
if (files.size() != 1) {
throw new IllegalStateException("GameCube DOL preparation requires exactly one imported file image");
}
FileBytes fileBytes = files.get(0);
AddressSpace space = currentProgram.getAddressFactory().getDefaultAddressSpace();
// Remove the whole-file block before creating section blocks, so no two blocks claim one address.
for (MemoryBlock block : memory.getBlocks()) {
for (MemoryBlockSourceInfo source : block.getSourceInfos()) {
if (source.getFileBytes().isPresent() && source.getFileBytes().get().equals(fileBytes)) {
memory.removeBlock(block, monitor);
break;
}
}
}
for (Section section : sections) {
if (section.fileOffset + section.size > fileBytes.getSize()) {
throw new IllegalArgumentException(
"GameCube DOL " + section.kind + " " + section.index + " exceeds the imported file"
);
}
boolean text = section.kind.equals("text");
if (!text && !section.kind.equals("data")) {
throw new IllegalArgumentException("Unknown GameCube DOL section kind: " + section.kind);
}
MemoryBlock block = memory.createInitializedBlock(
"." + section.kind + section.index,
space.getAddress(section.address),
fileBytes,
section.fileOffset,
section.size,
false
);
block.setRead(true);
block.setWrite(!text);
block.setExecute(text);
}
// Apply the domain owner's exact uninitialized ranges; do not reinterpret BSS here.
int gapIndex = 0;
for (BssGap gap : bssGaps) {
MemoryBlock block = memory.createUninitializedBlock(
".bss" + gapIndex++,
space.getAddress(gap.address()),
gap.size(),
false
);
block.setRead(true);
block.setWrite(true);
block.setExecute(false);
}
Address entryAddress = space.getAddress(entry);
MemoryBlock entryBlock = memory.getBlock(entryAddress);
if (entryBlock == null || !entryBlock.isExecute()) {
throw new IllegalArgumentException("GameCube DOL entry must lie in an executable section");
}
addEntryPoint(entryAddress);
if (!disassemble(entryAddress) || createFunction(entryAddress, "__start") == null) {
throw new IllegalStateException("GameCube DOL entry cannot be decoded as a function");
}
currentProgram.getOptions(Program.PROGRAM_INFO).setBoolean("REA raw binary prepared", true);
currentProgram.getOptions(Program.PROGRAM_INFO).setBoolean("REA GameCube DOL prepared", true);
}
private static void requireFields(String[] fields, int count, String line) {
if (fields.length != count) {
throw new IllegalArgumentException("Malformed GameCube DOL table line: " + line);
}
}
}
@@ -1,283 +0,0 @@
import ghidra.app.script.GhidraScript;
import ghidra.program.database.SpecExtension;
import ghidra.program.model.data.DataType;
import ghidra.program.model.data.DoubleDataType;
import ghidra.program.model.data.Float16DataType;
import ghidra.program.model.data.FloatDataType;
import ghidra.program.model.data.FunctionDefinition;
import ghidra.program.model.data.IntegerDataType;
import ghidra.program.model.data.StructureDataType;
import ghidra.program.model.lang.CompilerSpec;
import ghidra.program.model.lang.Language;
import ghidra.program.model.lang.PrototypeModel;
import ghidra.program.model.lang.Register;
import ghidra.program.model.listing.Function;
import ghidra.program.model.listing.Parameter;
import ghidra.program.model.listing.Program;
import ghidra.program.model.listing.VariableStorage;
import ghidra.program.model.pcode.Varnode;
import ghidra.program.model.pcode.XmlEncode;
import java.io.StringReader;
import java.io.StringWriter;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.regex.Pattern;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.InputSource;
/**
* Apply the committed RISC-V calling-convention policy after analysis. Ghidra's generic RV32
* compiler specification models ilp32d. Each policy derives a program-specific prototype from the
* program's own default prototype, as its {@link Policy} row and docs/providers/ghidra-riscv.md state. The
* script installs the prototype as a specification extension and assigns it, with dynamic
* storage, to every non-thunk function, external function and function definition.
*/
public final class ReaGhidraRiscvCallingConvention extends GhidraScript {
static final String REPORT_KEY = "REA RISC-V calling convention";
/** RISC-V floating-point registers by ABI name: ft0-ft11, fs0-fs11 and fa0-fa7. */
private static final Pattern FLOAT_REGISTER = Pattern.compile("f[ast]\\d+");
/**
* What a policy changes in the default ilp32d prototype. Every policy also passes values of 9
* bytes or more, such as {@code long double}, by reference.
*
* @param floatEntryBytes the largest value a floating-point register may pass or return; 0
* removes those entries and every rule that assigns floating-point storage
* @param preservesFloatRegisters whether fs0-fs11 stay preserved across calls
*/
private record Policy(
String id, String prototype, int floatEntryBytes, boolean preservesFloatRegisters) {}
private static final List<Policy> POLICIES =
List.of(new Policy("ilp32-prototype-v1", "__rea_ilp32", 0, false));
@Override
public void run() throws Exception {
String[] arguments = getScriptArgs();
Policy policy =
POLICIES.stream()
.filter(candidate -> arguments.length == 1 && candidate.id().equals(arguments[0]))
.findFirst()
.orElseThrow(
() ->
new IllegalArgumentException(
"REA RISC-V calling convention requires one of the policies "
+ POLICIES.stream().map(Policy::id).toList()));
Language language = currentProgram.getLanguage();
if (!language.getProcessor().toString().equals("RISCV")
|| currentProgram.getDefaultPointerSize() != 4)
throw new IllegalStateException(
"REA " + policy.id() + " requires a 32-bit RISC-V program, not "
+ language.getLanguageID());
CompilerSpec spec = currentProgram.getCompilerSpec();
new SpecExtension(currentProgram)
.addReplaceCompilerSpecExtension(
derive(spec.getDefaultCallingConvention(), spec, language, policy), monitor);
PrototypeModel model =
currentProgram.getCompilerSpec().getCallingConvention(policy.prototype());
if (model == null)
throw new IllegalStateException("REA " + policy.prototype() + " was not installed");
selfCheck(model, policy, language);
int functions = 0;
List<Function> callables = new ArrayList<>();
currentProgram.getFunctionManager().getFunctions(true).forEach(callables::add);
currentProgram.getFunctionManager().getExternalFunctions().forEach(callables::add);
for (Function function : callables) {
// A thunk's signature is its target's; the target is in this list.
if (function.isThunk()) continue;
if (function.hasCustomVariableStorage()) function.setCustomVariableStorage(false);
function.setCallingConvention(policy.prototype());
if (policy.floatEntryBytes() == 0) {
List<VariableStorage> storage = new ArrayList<>();
storage.add(function.getReturn().getVariableStorage());
for (Parameter parameter : function.getParameters())
storage.add(parameter.getVariableStorage());
// An untyped function has no assigned storage yet; the decompiler applies the
// prototype when it recovers one.
requireIntegerStorage(
storage.toArray(VariableStorage[]::new), function.getName(true), true);
}
functions += 1;
}
int definitions = 0;
Iterator<DataType> types = currentProgram.getDataTypeManager().getAllDataTypes();
while (types.hasNext()) {
if (!(types.next() instanceof FunctionDefinition definition)) continue;
definition.setCallingConvention(policy.prototype());
definitions += 1;
}
// Written last: any failure above leaves no report, and REA refuses the session.
currentProgram.getOptions(Program.PROGRAM_INFO).setString(
REPORT_KEY,
"{\"policy\":\"" + policy.id() + "\",\"functions\":" + functions
+ ",\"definitions\":" + definitions + "}");
println(
"REA RISC-V " + policy.prototype() + ": functions=" + functions
+ " definitions=" + definitions);
}
/** The default ilp32d prototype changed as the policy states. */
private static String derive(
PrototypeModel base, CompilerSpec spec, Language language, Policy policy) throws Exception {
XmlEncode encoder = new XmlEncode();
base.encode(encoder, spec.getPcodeInjectLibrary());
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
Document document =
factory.newDocumentBuilder().parse(new InputSource(new StringReader(encoder.toString())));
Element root = document.getDocumentElement();
if (!root.getTagName().equals("prototype"))
throw new IllegalStateException("Unexpected default prototype encoding " + root.getTagName());
root.setAttribute("name", policy.prototype());
List<Element> floatEntries = new ArrayList<>();
NodeList entries = root.getElementsByTagName("pentry");
for (int index = 0; index < entries.getLength(); index += 1) {
Element entry = (Element) entries.item(index);
if (entry.getAttribute("storage").equals("float")) floatEntries.add(entry);
}
if (floatEntries.isEmpty())
throw new IllegalStateException("Default prototype has no floating-point entries");
if (policy.floatEntryBytes() == 0) {
List<Node> removed = new ArrayList<>(floatEntries);
NodeList rules = root.getElementsByTagName("rule");
for (int index = 0; index < rules.getLength(); index += 1) {
Element rule = (Element) rules.item(index);
if (usesFloatStorage(rule)) removed.add(rule);
}
if (removed.size() == floatEntries.size())
throw new IllegalStateException("Default prototype has no floating-point rules to remove");
for (Node node : removed) node.getParentNode().removeChild(node);
} else
for (Element entry : floatEntries)
entry.setAttribute("maxsize", Integer.toString(policy.floatEntryBytes()));
// Values over 2 x XLEN go by reference. Returns need no rule: a value that the return
// registers cannot hold already becomes a caller buffer passed in a0.
NodeList inputs = root.getElementsByTagName("input");
if (inputs.getLength() != 1)
throw new IllegalStateException("Default prototype has no single input list");
Element input = (Element) inputs.item(0);
Element last = null;
for (Node child = input.getFirstChild(); child != null; child = child.getNextSibling())
if (child instanceof Element element && element.getTagName().equals("rule")) last = element;
if (last == null || !joinsAnyValue(last))
throw new IllegalStateException("Default prototype's input list does not end with a join");
Element byReference = document.createElement("rule");
Element datatype = document.createElement("datatype");
datatype.setAttribute("name", "any");
datatype.setAttribute("minsize", "9");
byReference.appendChild(datatype);
byReference.appendChild(document.createElement("convert_to_ptr"));
input.insertBefore(byReference, last);
if (!policy.preservesFloatRegisters()) {
NodeList killedLists = root.getElementsByTagName("killedbycall");
Element killed = killedLists.getLength() == 0 ? null : (Element) killedLists.item(0);
if (killed == null) {
killed = document.createElement("killedbycall");
root.appendChild(killed);
}
List<Element> moved = new ArrayList<>();
NodeList preservedLists = root.getElementsByTagName("unaffected");
for (int list = 0; list < preservedLists.getLength(); list += 1) {
NodeList preserved = preservedLists.item(list).getChildNodes();
for (int index = 0; index < preserved.getLength(); index += 1) {
if (preserved.item(index) instanceof Element varnode
&& isFloatRegister(varnode, language))
moved.add(varnode);
}
}
for (Element varnode : moved) {
varnode.getParentNode().removeChild(varnode);
killed.appendChild(varnode);
}
}
StringWriter xml = new StringWriter();
var transformer = TransformerFactory.newInstance().newTransformer();
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
transformer.transform(new DOMSource(document), new StreamResult(xml));
return xml.toString();
}
/** The installed prototype places each class of value as its policy states. */
private void selfCheck(PrototypeModel model, Policy policy, Language language) {
StructureDataType pair = new StructureDataType("rea_float_pair", 0);
pair.add(FloatDataType.dataType);
pair.add(FloatDataType.dataType);
VariableStorage[] storage =
model.getStorageLocations(
currentProgram,
new DataType[] {
DoubleDataType.dataType, FloatDataType.dataType, DoubleDataType.dataType,
IntegerDataType.dataType, pair, Float16DataType.dataType
},
false);
VariableStorage quad = storage[storage.length - 1];
if (quad.size() != currentProgram.getDefaultPointerSize() || quad.getRegister() == null)
throw new IllegalStateException(
"REA " + policy.prototype() + " does not pass a 16-byte value by reference: " + quad);
if (policy.floatEntryBytes() == 0)
requireIntegerStorage(storage, policy.prototype() + " self-check", false);
if (policy.preservesFloatRegisters()) return;
for (Varnode preserved : model.getUnaffectedList()) {
Register register = language.getRegister(preserved.getAddress(), preserved.getSize());
if (register != null && FLOAT_REGISTER.matcher(register.getName()).matches())
throw new IllegalStateException(
"REA " + policy.prototype() + " still preserves " + register.getName());
}
}
private static boolean joinsAnyValue(Element rule) {
List<Element> children = new ArrayList<>();
for (Node child = rule.getFirstChild(); child != null; child = child.getNextSibling())
if (child instanceof Element element) children.add(element);
return children.size() == 2
&& children.get(0).getTagName().equals("datatype")
&& children.get(0).getAttribute("name").equals("any")
&& children.get(1).getTagName().equals("join");
}
/** Whether a rule's action, at any depth, assigns floating-point storage. */
private static boolean usesFloatStorage(Element element) {
NodeList children = element.getChildNodes();
for (int index = 0; index < children.getLength(); index += 1) {
if (!(children.item(index) instanceof Element child)) continue;
if (child.getAttribute("storage").equals("float") || usesFloatStorage(child)) return true;
}
return false;
}
private static boolean isFloatRegister(Element varnode, Language language) {
if (!varnode.getTagName().equals("varnode")
|| !varnode.getAttribute("space").equals("register"))
return false;
Register register =
language.getRegister(
language.getAddressFactory().getRegisterSpace()
.getAddress(Long.decode(varnode.getAttribute("offset"))),
Integer.parseInt(varnode.getAttribute("size")));
return register != null && FLOAT_REGISTER.matcher(register.getName()).matches();
}
private static void requireIntegerStorage(
VariableStorage[] storage, String owner, boolean allowUnassigned) {
for (VariableStorage item : storage) {
if (item.isBadStorage() || (!allowUnassigned && item.isUnassignedStorage()))
throw new IllegalStateException("REA RISC-V storage unresolved for " + owner);
List<Register> registers = item.getRegisters();
if (registers == null) continue; // stack-only storage
for (Register register : registers) {
if (FLOAT_REGISTER.matcher(register.getName()).matches())
throw new IllegalStateException(
"REA RISC-V storage still uses " + register.getName() + " for " + owner);
}
}
}
}
+6 -163
View File
@@ -22,9 +22,6 @@ HOPPER_SYMBOL_NAME_UTF16_UNITS = 1024
# that does not fit is null, not an older descriptor path.
# 536870887 is Node's 536870888-character string ceiling minus the framing LF.
# The ceiling is the encoded payload; JSON expansion is measured before it.
# Encoded JSON allowance, in bytes, shared by one instruction list, one
# native-caller list, and one caller-scan page. This release has no runtime override.
CALLER_SCAN_ALLOWANCE_BYTES = 512 * 1024
_BRIDGE_MAX_REQUEST_BYTES_SETTING = "REA_BRIDGE_MAX_REQUEST_BYTES"
DEFAULT_BRIDGE_MAX_REQUEST_BYTES = 16 * 1024 * 1024
MINIMUM_BRIDGE_MAX_REQUEST_BYTES = 7464
@@ -235,9 +232,6 @@ EXHAUSTIVE_ANALYSIS_METHODS = frozenset(
"procedure_assembly",
"procedure_callees",
"procedure_callers",
"rea_caller_scan_page",
"rea_caller_scan_target",
"rea_caller_base",
"procedure_info",
"procedure_pseudo_code",
"procedure_references",
@@ -491,30 +485,19 @@ def _instruction_addresses(procedure):
return result
def _iter_native_calls(procedure, direction):
def _native_calls(procedure, direction):
references = procedure.getAllCallees() if direction == "outgoing" else procedure.getAllCallers()
kinds = {0: "none", 1: "unknown", 2: "direct", 3: "objective_c"}
calls = {}
unresolved = []
for reference in references:
source, target, kind = reference.fromAddress(), reference.toAddress(), reference.type()
if source in BAD_ADDRESSES:
yield None, None, None
continue
if target in BAD_ADDRESSES:
yield source, None, {"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind}
unresolved.append({"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind})
continue
yield source, target, {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
def _native_calls(procedure, direction):
calls = {}
unresolved = []
for source, target, detail in _iter_native_calls(procedure, direction):
if detail is None:
continue
if target is None:
unresolved.append(detail)
else:
calls[(source, target)] = detail
calls[(source, target)] = {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
return calls, sorted(unresolved, key=lambda item: int(item["address"], 16))
@@ -564,136 +547,6 @@ def _procedure_map(document):
return result
def _caller_target_data(procedure):
addresses = []
seen = set()
size = 0
for block in procedure.basicBlockIterator():
for address in _block_instruction_addresses(procedure, block):
if address in seen:
continue
encoded = _hex(address)
size += len(encoded) + 4
if size > CALLER_SCAN_ALLOWANCE_BYTES:
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": False}
addresses.append(encoded)
seen.add(address)
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": True}
def _caller_scan_target(document, params):
return _caller_target_data(_procedure(document, params.get("procedure")))
def _caller_base(document, params):
procedure = _procedure(document, params.get("procedure"))
callers = []
size = 0
complete = True
for item in procedure.getAllCallerProcedures():
address = _hex(item.getEntryPoint())
size += len(address) + 4
if size > CALLER_SCAN_ALLOWANCE_BYTES:
complete = False
break
callers.append(address)
return {
**_caller_target_data(procedure),
"callers": sorted(callers, key=lambda value: int(value, 16)),
"callers_complete": complete,
}
def _caller_scan_page(document, params):
"""Observe forward relationships in one bounded, stateless bridge request."""
start = params.get("start")
if isinstance(start, bool) or not isinstance(start, int) or start < 0:
raise InvalidRequestError("Caller scan start must be a non-negative integer")
segments = document.getSegmentsList()
counts = [segment.getProcedureCount() for segment in segments]
total = sum(counts)
end = min(start + 64, total)
if start > total:
raise InvalidRequestError("Caller scan start exceeds the procedure inventory")
items = []
item_bytes = 0
page_limit = CALLER_SCAN_ALLOWANCE_BYTES
oversized = False
identities = {}
containing = {}
def identity(procedure):
entry = procedure.getEntryPoint()
if entry not in identities:
identities[entry] = _procedure_identity(procedure)
return identities[entry]
def owner(address):
if address not in containing:
procedure, _ = _containing_procedure(document, address)
containing[address] = identity(procedure) if procedure is not None else None
return containing[address]
segment_start = 0
for segment, count in zip(segments, counts):
begin, stop = max(start - segment_start, 0), min(end - segment_start, count)
segment_start += count
for index in range(begin, max(begin, stop)):
candidate = segment.getProcedureAtIndex(index)
item = {"caller": identity(candidate), "callees": [], "references": []}
size = len(json.dumps(item, ensure_ascii=True).encode("utf-8")) + 1
remaining = page_limit - item_bytes
too_large = size > remaining
if not too_large:
for callee in candidate.getAllCalleeProcedures():
address = _hex(callee.getEntryPoint())
size += len(address) + 4
if size > remaining:
too_large = True
break
item["callees"].append(address)
if not too_large:
callee_entries = set(item["callees"])
for source, destination, call in _iter_native_calls(candidate, "outgoing"):
# Account for every native record, even when its target cannot
# be associated with an observed procedure.
size += 32
if size > remaining:
too_large = True
break
if destination is None:
continue
source_procedure = item["caller"] if _hex(source) == item["caller"]["address"] else owner(source)
target_address = _hex(destination)
if target_address in callee_entries:
target_entry = target_address
else:
target_procedure = owner(destination)
if target_procedure is None:
continue
target_entry = target_procedure["address"]
edge = {
"source_address": _hex(source), "target_address": _hex(destination),
"source_procedure": "caller" if source_procedure is not None and source_procedure["address"] == item["caller"]["address"] else source_procedure,
"target_procedure": target_entry,
"call": call,
}
size += len(json.dumps(edge, ensure_ascii=True).encode("utf-8")) + 1
if size > remaining:
too_large = True
break
item["references"].append(edge)
if too_large:
oversized = not items
break
items.append(item)
item_bytes += size
else:
continue
break
return {"start": start, "next": start + len(items), "total": total, "items": items, "oversized": oversized}
def _string_record(segment, address, display):
"""Read one native typed object, retaining its shortened display separately."""
length = segment.getObjectLength(address)
@@ -1034,15 +887,6 @@ def _dispatch(method, params):
if method in EXHAUSTIVE_ANALYSIS_METHODS:
_api().require_analysis_complete(document, method)
if method == "rea_caller_scan_target":
return _caller_scan_target(document, params)
if method == "rea_caller_base":
return _caller_base(document, params)
if method == "rea_caller_scan_page":
return _caller_scan_page(document, params)
if method == "analyze_function":
return _analyze_function(document, params)
if method == "read_function_instructions":
@@ -1399,8 +1243,7 @@ def _serve_connection(connection):
break
except (ConnectionError, OSError):
# A lost client cannot confirm document closure or revoke this run.
# Keep the listener available for the owner to confirm cleanup.
should_stop = False
pass
finally:
try:
file.close()
-13
View File
@@ -24,16 +24,3 @@ func captureAccessibilityValue(_ source: Any?) -> AccessibilityValue {
}
return AccessibilityValue(value: NSNull(), gap: "AXValue type \(type(of: source)) is not a supported JSON scalar")
}
/** Keep an unavailable action list distinct from a successful empty list. */
func captureActionNames(status: Int32, success: Int32, names: [String]?) -> (names: [String], error: Int32?) {
guard status == success else { return ([], status) }
return (names ?? [], nil)
}
/** AX absence is distinct from a failed attribute read. */
func captureAttributeValue<Value>(status: Int32, success: Int32, absent: [Int32], value: Value?) -> (value: Value?, error: Int32?) {
if status == success { return (value, nil) }
if absent.contains(status) { return (nil, nil) }
return (nil, status)
}
+21 -49
View File
@@ -18,10 +18,9 @@ struct Request: Decodable {
struct Action: Decodable { var kind: String; var path: [Int]?; var direction: String?; var text: String? }
struct BoundaryFailure: Error { var code: String; var message: String }
func fail(_ code: String, _ message: String) throws -> Never { throw BoundaryFailure(code: code, message: message) }
func attribute(_ element: AXUIElement, _ key: String) -> (value: CFTypeRef?, error: Int32?) {
func attribute(_ element: AXUIElement, _ key: String) -> CFTypeRef? {
var value: CFTypeRef?
let status = AXUIElementCopyAttributeValue(element, key as CFString, &value)
return captureAttributeValue(status: status.rawValue, success: AXError.success.rawValue, absent: [AXError.noValue.rawValue, AXError.attributeUnsupported.rawValue], value: value)
return AXUIElementCopyAttributeValue(element, key as CFString, &value) == .success ? value : nil
}
func childCount(_ element: AXUIElement) -> ChildCount {
var count = 0
@@ -35,15 +34,13 @@ func children(_ element: AXUIElement, count: Int) -> ChildBatch<AXUIElement> {
return value as? [AXUIElement]
}
}
func windowBounds(_ element: AXUIElement) -> (bounds: CGRect?, error: Int32?) {
let position = attribute(element, kAXPositionAttribute)
let size = attribute(element, kAXSizeAttribute)
if let error = position.error ?? size.error { return (nil, error) }
guard let p = position.value, let s = size.value,
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return (nil, nil) }
var point = CGPoint.zero; var dimensions = CGSize.zero
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &dimensions) else { return (nil, nil) }
return (CGRect(origin: point, size: dimensions), nil)
func text(_ element: AXUIElement, _ key: String) -> Any { (attribute(element, key) as? String) ?? NSNull() as Any }
func windowBounds(_ element: AXUIElement) -> CGRect? {
guard let p = attribute(element, kAXPositionAttribute), let s = attribute(element, kAXSizeAttribute),
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return nil }
var point = CGPoint.zero; var size = CGSize.zero
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &size) else { return nil }
return CGRect(origin: point, size: size)
}
func observe(_ request: Request) async throws -> [String: Any] {
guard let app = NSRunningApplication(processIdentifier: request.pid) else {
@@ -68,10 +65,7 @@ func observe(_ request: Request) async throws -> [String: Any] {
guard SHA256.hash(data: executableBytes).map({ String(format: "%02x", $0) }).joined() == request.sha256 else {
try fail("target-mismatch", "Running executable bytes differ from the active target digest")
}
// .optionIncludingWindow omits a selected window while it is off screen, which
// is not absence: select from every window, and let capture report what it can.
guard let windows = (CGWindowListCopyWindowInfo([.optionAll], kCGNullWindowID) as? [[String: Any]])?
.filter({ ($0[kCGWindowNumber as String] as? NSNumber)?.uint32Value == request.window_id }),
guard let windows = CGWindowListCopyWindowInfo([.optionIncludingWindow], request.window_id) as? [[String: Any]],
windows.count == 1, let window = windows.first,
(window[kCGWindowOwnerPID as String] as? NSNumber)?.int32Value == request.pid,
(window[kCGWindowLayer as String] as? NSNumber)?.intValue == 0,
@@ -87,22 +81,11 @@ func observe(_ request: Request) async throws -> [String: Any] {
guard AXIsProcessTrusted() else { try fail("accessibility-denied", "Grant Accessibility to the REA host in System Settings > Privacy & Security, then retry") }
let application = AXUIElementCreateApplication(request.pid)
AXUIElementSetMessagingTimeout(application, 2)
let windowsAttribute = attribute(application, kAXWindowsAttribute)
if let error = windowsAttribute.error {
try fail("accessibility-unavailable", "Cannot read accessibility windows: AXError \(error)")
guard let axWindows = attribute(application, kAXWindowsAttribute) as? [AXUIElement] else {
try fail("accessibility-unavailable", "Selected application does not expose accessibility windows")
}
guard let axWindows = windowsAttribute.value as? [AXUIElement] else {
try fail("accessibility-unavailable", "Accessibility windows are unavailable for the selected application")
}
let candidates = axWindows.map { (element: $0, observation: windowBounds($0)) }
if let error = candidates.compactMap({ $0.observation.error }).first {
try fail("accessibility-unavailable", "Cannot read accessibility window geometry: AXError \(error)")
}
guard candidates.allSatisfy({ $0.observation.bounds != nil }) else {
try fail("accessibility-unavailable", "Accessibility window geometry is unavailable; the selected window cannot be identified")
}
let matches = candidates.filter { $0.observation.bounds == bounds }.map { $0.element }
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(candidates.map { String(describing: $0.observation.bounds) })") }
let matches = axWindows.filter { windowBounds($0) == bounds }
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(axWindows.map { String(describing: windowBounds($0)) })") }
selected = matches[0]
}
if let action = request.action, let root = selected {
@@ -113,16 +96,13 @@ func observe(_ request: Request) async throws -> [String: Any] {
guard let available = count.value else {
try fail("element-count-unavailable", "Cannot validate selected accessibility path because child count failed with AXError \(count.error ?? -1)")
}
guard index >= 0 && index < available else { try fail("element-missing", "Selected accessibility path no longer exists") }
let childStatus = AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild)
guard childStatus == .success else { try fail("accessibility-unavailable", "Cannot read selected accessibility child: AXError \(childStatus.rawValue)") }
guard let item = (selectedChild as? [AXUIElement])?.first else { try fail("accessibility-unavailable", "Selected accessibility child was not returned") }
guard index >= 0 && index < available,
AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild) == .success,
let item = (selectedChild as? [AXUIElement])?.first else { try fail("element-missing", "Selected accessibility path no longer exists") }
element = item
}
var owner: pid_t = 0
let ownerStatus = AXUIElementGetPid(element, &owner)
guard ownerStatus == .success else { try fail("accessibility-unavailable", "Cannot identify accessibility element owner: AXError \(ownerStatus.rawValue)") }
guard owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
guard AXUIElementGetPid(element, &owner) == .success && owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
let outcome: AXError
switch action.kind {
case "click": outcome = AXUIElementPerformAction(element, kAXPressAction as CFString)
@@ -139,18 +119,10 @@ func observe(_ request: Request) async throws -> [String: Any] {
if nodes.count >= request.max_nodes { truncated = true; break }
let childCountResult = childCount(element)
var actions: CFArray?
let actionStatus = AXUIElementCopyActionNames(element, &actions)
let observedActions = captureActionNames(status: actionStatus.rawValue, success: AXError.success.rawValue, names: actions as? [String])
if let error = observedActions.error { gaps.append("AX actions unavailable at path \(path): AXError \(error)"); truncated = true }
let role = attribute(element, kAXRoleAttribute)
let title = attribute(element, kAXTitleAttribute)
let value = attribute(element, kAXValueAttribute)
for (name, error) in [("role", role.error), ("title", title.error), ("value", value.error)] {
if let error { gaps.append("AX \(name) unavailable at path \(path): AXError \(error)"); truncated = true }
}
let observedValue = captureAccessibilityValue(value.value)
AXUIElementCopyActionNames(element, &actions)
let observedValue = captureAccessibilityValue(attribute(element, kAXValueAttribute))
if let gap = observedValue.gap { gaps.append("\(gap) at path \(path)"); truncated = true }
nodes.append(["path": path, "role": (role.value as? String) ?? NSNull() as Any, "title": (title.value as? String) ?? NSNull() as Any, "value": observedValue.value, "actions": observedActions.names, "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
nodes.append(["path": path, "role": text(element, kAXRoleAttribute), "title": text(element, kAXTitleAttribute), "value": observedValue.value, "actions": actions as? [String] ?? [], "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
guard let totalChildren = childCountResult.value else {
truncated = true
gaps.append("AX child count unavailable at path \(path): AXError \(childCountResult.error ?? -1)")
+15 -28
View File
@@ -26,6 +26,7 @@ ARGUMENT_REGISTERS = {
"x86_64h": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
}
MAX_SELECTOR_BYTES = 1024
MAX_REPORTED_LOCATIONS = 64
MAX_OTHER_STOPS = 256
MAX_TRACE_BYTES = 8 * 1024 * 1024
MAX_TRACE_FRAMES = 65_536
@@ -243,20 +244,6 @@ def _hex(value):
return "0x%x" % value
def _source_location(address):
"""Preserve available line-entry facets without acquiring source files."""
entry = address.GetLineEntry()
if not entry.IsValid():
return None
file_spec = entry.GetFileSpec()
file_path = str(file_spec) if file_spec.IsValid() else None
line = entry.GetLine() or None
column = entry.GetColumn() or None
if not file_path and line is None and column is None:
return None
return {"file_path": file_path or None, "line": line, "column": column}
def _location(address, target):
"""Load and file address, module and symbol of one code address."""
module = address.GetModule()
@@ -270,7 +257,6 @@ def _location(address, target):
"module": module.GetFileSpec().GetFilename() if module.IsValid() else None,
"module_path": str(module.GetFileSpec()) if module.IsValid() else None,
"symbol": symbol.GetName() if symbol.IsValid() else None,
"source_location": _source_location(address),
}
return location
@@ -519,19 +505,20 @@ def _breakpoint_report(target, created, config):
if not _wanted(config["breakpoints"][index], location["symbol"] or ""):
continue
count += 1
remaining_bytes = (
MAX_BREAKPOINT_LOCATION_BYTES - retained_location_bytes
)
if not _fits_trace_budget(location, remaining_bytes):
locations_truncated = True
continue
location_bytes = len(json.dumps(location).encode("utf-8"))
addition = location_bytes + (1 if locations else 0)
if addition > remaining_bytes:
locations_truncated = True
continue
locations.append(location)
retained_location_bytes += addition
if len(locations) < MAX_REPORTED_LOCATIONS:
remaining_bytes = (
MAX_BREAKPOINT_LOCATION_BYTES - retained_location_bytes
)
if not _fits_trace_budget(location, remaining_bytes):
locations_truncated = True
continue
location_bytes = len(json.dumps(location).encode("utf-8"))
addition = location_bytes + (1 if locations else 0)
if addition > remaining_bytes:
locations_truncated = True
continue
locations.append(location)
retained_location_bytes += addition
report.append({"index": index, "location_count": count, "locations": locations})
return report, locations_truncated
+13 -116
View File
@@ -25,45 +25,6 @@ spec.loader.exec_module(tracer)
class NativeTraceBudgetTests(unittest.TestCase):
def test_source_coordinates_preserve_partial_metadata_without_reading_source(self):
def address(valid=True, file_valid=True, line=23, column=7):
class SourceFileSpec:
def IsValid(self):
return file_valid
def __str__(self):
return "/missing/é space/fixture.pas"
file_spec = SourceFileSpec()
entry = types.SimpleNamespace(
IsValid=lambda: valid,
GetFileSpec=lambda: file_spec,
GetLine=lambda: line,
GetColumn=lambda: column,
)
return types.SimpleNamespace(GetLineEntry=lambda: entry)
self.assertEqual(
tracer._source_location(address()),
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": 7},
)
self.assertEqual(
tracer._source_location(address(column=0)),
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": None},
)
self.assertEqual(
tracer._source_location(address(file_valid=False)),
{"file_path": None, "line": 23, "column": 7},
)
self.assertEqual(
tracer._source_location(address(line=0)),
{"file_path": "/missing/é space/fixture.pas", "line": None, "column": 7},
)
self.assertIsNone(tracer._source_location(address(valid=False)))
self.assertIsNone(
tracer._source_location(address(file_valid=False, line=0, column=0))
)
def test_invalid_launch_process_reports_explicit_no_creation(self):
class InvalidProcess:
def IsValid(self):
@@ -429,9 +390,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
return FileSpec()
class Address:
def GetLineEntry(self):
return types.SimpleNamespace(IsValid=lambda: False)
def GetModule(self):
return Module()
@@ -472,34 +430,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
self.assertEqual(report[0]["locations"], [])
self.assertFalse(locations_truncated)
def test_breakpoint_report_keeps_all_locations_under_aggregate_budget(self):
class Location:
def __init__(self, index):
self.index = index
def GetAddress(self):
return self.index
class Breakpoint:
def GetNumLocations(self):
return 65
def GetLocationAtIndex(self, index):
return Location(index)
with patch.object(
tracer,
"_location",
side_effect=lambda address, _target: {"address": address, "symbol": None},
), patch.object(tracer, "_wanted", return_value=True):
report, locations_truncated = tracer._breakpoint_report(
object(), [[Breakpoint()]], {"breakpoints": [{}]}
)
self.assertEqual(report[0]["location_count"], 65)
self.assertEqual(len(report[0]["locations"]), 65)
self.assertFalse(locations_truncated)
def test_breakpoint_report_bounds_aggregate_location_json(self):
tracer.lldb.LLDB_INVALID_ADDRESS = -1
symbol_name = (
@@ -533,24 +463,7 @@ class NativeTraceBudgetTests(unittest.TestCase):
def GetFileSpec(self):
return FileSpec()
class SourceFileSpec:
def IsValid(self):
return True
def __str__(self):
return source_path
class Address:
def GetLineEntry(self):
if source_path is None:
return types.SimpleNamespace(IsValid=lambda: False)
return types.SimpleNamespace(
IsValid=lambda: True,
GetFileSpec=SourceFileSpec,
GetLine=lambda: 15,
GetColumn=lambda: 3,
)
def GetModule(self):
return Module()
@@ -577,35 +490,19 @@ class NativeTraceBudgetTests(unittest.TestCase):
breakpoint_count = 1000
created = [[Breakpoint()] for _ in range(breakpoint_count)]
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count}
for source_path in (None, "/tmp/" + "源🚀" * 3000 + "/calls.pas"):
with self.subTest(source_path_present=source_path is not None):
report, locations_truncated = tracer._breakpoint_report(
object(), created, config
)
retained_bytes = sum(
len(json.dumps(location).encode("utf-8"))
+ (1 if index > 0 else 0)
for row in report
for index, location in enumerate(row["locations"])
)
self.assertTrue(locations_truncated)
self.assertEqual(
sum(row["location_count"] for row in report), 64_000
)
self.assertLess(
sum(len(row["locations"]) for row in report), 64_000
)
self.assertLessEqual(
retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES
)
if source_path is not None:
self.assertTrue(
any(
location["source_location"]["file_path"] == source_path
for row in report
for location in row["locations"]
)
)
report, locations_truncated = tracer._breakpoint_report(
object(), created, config
)
retained_bytes = sum(
len(json.dumps(location).encode("utf-8"))
+ (1 if index > 0 else 0)
for row in report
for index, location in enumerate(row["locations"])
)
self.assertTrue(locations_truncated)
self.assertEqual(sum(row["location_count"] for row in report), 64_000)
self.assertLess(sum(len(row["locations"]) for row in report), 64_000)
self.assertLessEqual(retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES)
if __name__ == "__main__":
+5 -7
View File
@@ -78,9 +78,14 @@ public final class NativeAotExtension {
new FrozenObjectAnnotator(program, manager).analyze(located.directory(), scan.getPointerLocations(), monitor, log);
}
monitor.checkCancelled();
report.addProperty("method_tables", manager.getMethodTableCount());
JsonObject coverage = auditFrozenObjects(program, scan.getPointerLocations(), monitor);
report.add("coverage", coverage);
if (coverage.get("frozen_objects_annotated").getAsInt() != coverage.get("frozen_object_candidates").getAsInt())
diagnostics.add("Some frozen-object candidates lack committed instance annotations; inspect coverage counts and Ghidra diagnostics.");
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
report.add("reason", JsonNull.INSTANCE);
report.add("derived_memory", derivedRange.deepCopy());
JsonArray tables = new JsonArray();
for (var mt : manager.getMethodTables()) {
monitor.checkCancelled();
@@ -89,13 +94,6 @@ public final class NativeAotExtension {
identity.addProperty("type", mt.getMTType().getPathName());
tables.add(identity);
}
// Publish recovery facts only after every fallible recovery and audit step.
// A failed transaction is rolled back, so it cannot report committed metadata.
report.addProperty("method_tables", manager.getMethodTableCount());
report.add("coverage", coverage);
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
report.add("reason", JsonNull.INSTANCE);
report.add("derived_memory", derivedRange.deepCopy());
report.add("types", tables);
commit = true;
return finish(log);
+34 -72
View File
@@ -15,8 +15,6 @@ struct ProcessIdentityObservation: Encodable {
let effective_uid: UInt32?
let lifecycle: String?
let relation: ProcessRelationObservation?
/** Display-only kernel accounting name (p_comm, at most MAXCOMLEN bytes). */
let name: String?
let reason: String?
}
@@ -33,7 +31,6 @@ struct ProcessMetadata {
let parentPid: Int32
let processGroupId: Int32
let pointerSize: Int
let name: String
}
struct ProcessSnapshot {
@@ -95,8 +92,7 @@ func readProcessSnapshot(
lifecycle: process.kp_proc.p_stat == SZOMB ? "zombie" : "non-zombie",
parentPid: process.kp_eproc.e_ppid,
processGroupId: process.kp_eproc.e_pgid,
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4,
name: processName(process)
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4
)
)
},
@@ -107,13 +103,6 @@ func readProcessSnapshot(
return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(ENOMEM)")
}
/** Decode the NUL-terminated accounting name; invalid UTF-8 is replaced for display. */
private func processName(_ process: kinfo_proc) -> String {
withUnsafeBytes(of: process.kp_proc.p_comm) { bytes in
String(decoding: bytes.prefix { $0 != 0 }, as: UTF8.self)
}
}
func readProcessIdentities(pids: [Int32]) -> [ProcessIdentityObservation] {
readProcessIdentities(pids: pids, snapshot: readProcessSnapshot())
}
@@ -141,7 +130,6 @@ func readProcessIdentities(
effective_uid: nil,
lifecycle: nil,
relation: nil,
name: nil,
reason: "not-in-snapshot"
)
}
@@ -156,7 +144,6 @@ func readProcessIdentities(
parent_pid: metadata.parentPid,
process_group_id: metadata.processGroupId
),
name: metadata.name,
reason: nil
)
}
@@ -170,25 +157,12 @@ private func unavailableIdentity(pid: Int32, reason: String) -> ProcessIdentityO
effective_uid: nil,
lifecycle: nil,
relation: nil,
name: nil,
reason: reason
)
}
enum ProcargsMalformedStage: String, CaseIterable {
case header
case argumentCount = "argument_count"
case executablePath = "executable_path"
case pointerSize = "pointer_size"
case pathPadding = "path_padding"
case arguments
case terminator
case trailingEntries = "trailing_entries"
}
enum RunTokenReadError: Error {
case invalidBuffer
case invalidBufferAt(ProcargsMalformedStage)
case duplicateToken
case invalidTokenEncoding
case environmentUnavailable
@@ -228,22 +202,6 @@ func processRunTokenObservation(
parse: Result<ParsedRunToken, RunTokenReadError>,
codeSigningFlags: () -> UInt32?
) -> RunTokenObservation {
// Native inspection restrictions apply even when argv is incomplete or
// overwritten, before parsing reaches the environment. Keep this fact at
// the native boundary; cleanup still decides whether the process is related.
func withheldEnvironment() -> RunTokenObservation? {
guard let flags = codeSigningFlags() else { return nil }
if (flags & csPlatformBinary) != 0 {
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
}
if (flags & csRestrict) != 0 {
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
}
return nil
}
func unavailableEnvironment(_ reason: String) -> RunTokenObservation {
withheldEnvironment() ?? unavailable(pid: pid, reason: reason)
}
switch parse {
case .success(let parsed):
if let runId = parsed.runId {
@@ -254,25 +212,38 @@ func processRunTokenObservation(
if parsed.environmentEntryCount > 0 {
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
}
if let withheld = withheldEnvironment() { return withheld }
let flags = codeSigningFlags()
if let flags, (flags & csPlatformBinary) != 0 {
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
}
if let flags, (flags & csRestrict) != 0 {
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
}
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
case .failure(let failure):
switch failure {
case .environmentUnavailable:
return unavailableEnvironment("environment_unavailable")
let flags = codeSigningFlags()
if let flags, (flags & csPlatformBinary) != 0 {
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
}
if let flags, (flags & csRestrict) != 0 {
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
}
return unavailable(pid: pid, reason: "environment_unavailable")
case .ambiguousEnvironmentBoundary:
return unavailableEnvironment("ambiguous_environment_boundary")
// These failures already establish that token bytes were available.
if let flags = codeSigningFlags(), (flags & csRestrict) != 0 {
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
}
return unavailable(pid: pid, reason: "ambiguous_environment_boundary")
case .duplicateToken:
return unavailable(pid: pid, reason: "duplicate_run_token")
case .invalidTokenEncoding:
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
case .appleVectorUnavailable:
return unavailableEnvironment("apple_vector_unavailable")
return unavailable(pid: pid, reason: "apple_vector_unavailable")
case .invalidBuffer:
return unavailableEnvironment("malformed_procargs")
case .invalidBufferAt(let stage):
return unavailableEnvironment("malformed_procargs_\(stage.rawValue)")
return unavailable(pid: pid, reason: "malformed_procargs")
}
}
}
@@ -331,30 +302,27 @@ func parseProcessEnvironment(
in bytes: [UInt8],
pointerSize: Int = MemoryLayout<UnsafeRawPointer>.size
) throws -> ParsedRunToken {
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBufferAt(.header) }
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBuffer }
let argc = bytes.withUnsafeBytes { $0.loadUnaligned(as: Int32.self).littleEndian }
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBufferAt(.argumentCount) }
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBuffer }
var offset = MemoryLayout<Int32>.size
guard let executablePath = readString(in: bytes, offset: &offset) else {
throw RunTokenReadError.invalidBufferAt(.executablePath)
throw RunTokenReadError.invalidBuffer
}
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBufferAt(.pointerSize) }
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBuffer }
let pathRecordSize = Array("executable_path=".utf8).count + executablePath.count + 1
let padding = (pointerSize - pathRecordSize % pointerSize) % pointerSize
guard offset + padding <= bytes.count,
bytes[offset..<(offset + padding)].allSatisfy({ $0 == 0 }) else {
throw RunTokenReadError.invalidBufferAt(.pathPadding)
throw RunTokenReadError.invalidBuffer
}
offset += padding
let tokenPrefix = Array("REA_PROCESS_RUN_ID=".utf8)
var tokenInArgument = false
for _ in 0..<argc {
guard let argument = readString(in: bytes, offset: &offset, allowEmpty: true) else {
throw RunTokenReadError.invalidBufferAt(.arguments)
guard readString(in: bytes, offset: &offset, allowEmpty: true) != nil else {
throw RunTokenReadError.invalidBuffer
}
tokenInArgument = tokenInArgument || argument.starts(with: tokenPrefix)
}
guard bytes.last == 0 else { throw RunTokenReadError.invalidBufferAt(.terminator) }
guard bytes.last == 0 else { throw RunTokenReadError.invalidBuffer }
if offset == bytes.count {
throw RunTokenReadError.environmentUnavailable
}
@@ -372,7 +340,7 @@ func parseProcessEnvironment(
continue
}
guard let entry = readString(in: bytes, offset: &offset) else {
throw RunTokenReadError.invalidBufferAt(.trailingEntries)
throw RunTokenReadError.invalidBuffer
}
trailingEntries.append(entry)
}
@@ -393,28 +361,22 @@ func parseProcessEnvironment(
throw RunTokenReadError.appleVectorUnavailable
}
if trailingEntries[appleVectorStart...].contains(where: {
$0.starts(with: tokenPrefix)
$0.starts(with: Array("REA_PROCESS_RUN_ID=".utf8))
}) {
throw RunTokenReadError.ambiguousEnvironmentBoundary
}
let environmentEntries = trailingEntries[..<appleVectorStart]
var found: String?
for entry in environmentEntries {
if entry.starts(with: tokenPrefix) {
if entry.starts(with: Array("REA_PROCESS_RUN_ID=".utf8)) {
guard found == nil else { throw RunTokenReadError.duplicateToken }
let value = entry.dropFirst(tokenPrefix.count)
let value = entry.dropFirst("REA_PROCESS_RUN_ID=".utf8.count)
guard let decoded = String(bytes: value, encoding: .utf8) else {
throw RunTokenReadError.invalidTokenEncoding
}
found = decoded
}
}
// A changed process title can overwrite the original argument boundaries,
// causing a real environment entry to be consumed as an argument. A real
// argument can have this form too, so it proves neither ownership nor absence.
if tokenInArgument {
throw RunTokenReadError.ambiguousEnvironmentBoundary
}
return ParsedRunToken(runId: found, environmentEntryCount: environmentEntries.count)
}
+1 -1
View File
@@ -109,7 +109,7 @@ class CoreContextTests(unittest.TestCase):
self.assertFalse(result["ok"])
self.assertEqual(result["reason"], "output-limit")
self.assertNotIn("exceeds64", result["message"])
self.assertIn("64 MiB", result["message"])
self.assertEqual(result["message"], "Complete debugger context exceeds the 64 MiB reply budget.")
self.assertEqual(reads, [])
self.assertEqual(queries, ["maps"])
+10 -18
View File
@@ -15,11 +15,6 @@ import sys
PROFILE = "[email protected];[email protected];[email protected]"
PACKAGES = {"pwntools": "4.15.0", "pyelftools": "0.33", "unicorn": "2.1.2"}
OUTPUT_BYTES = 64 * 1024 * 1024
# (e_machine, ELF class bits, byte order) -> architecture pwntools must also report.
PROFILES = {(62, 64, "little"): "amd64", (183, 64, "little"): "aarch64", (3, 32, "little"): "i386",
(40, 32, "little"): "arm", (8, 32, "little"): "mips", (8, 32, "big"): "mips"}
PROFILE_SCOPE = ("Layout profile requires ELF64 x86-64, ELF64 AArch64 little-endian, ELF32 i386, "
"ELF32 ARM little-endian or ELF32 MIPS little- or big-endian objects.")
class LayoutFailure(Exception):
@@ -121,16 +116,13 @@ def inspect_elf(path, cache):
raise LayoutFailure("format", "Selected object does not contain an ELF header.")
if len(content) < 16:
raise LayoutFailure("format", "Selected ELF identification is truncated.")
if content[4] not in (1, 2) or content[5] not in (1, 2):
raise LayoutFailure("unsupported-target", "Selected ELF declares an unknown class or byte order.")
bits, header_size = (32, 52) if content[4] == 1 else (64, 64)
order = "little" if content[5] == 1 else "big"
if len(content) < header_size:
raise LayoutFailure("format", f"Selected ELF{bits} header is truncated.")
profile = (int.from_bytes(content[18:20], order), bits, order)
if profile not in PROFILES:
raise LayoutFailure("unsupported-target", PROFILE_SCOPE)
if int.from_bytes(content[16:18], order) not in (1, 2, 3):
if content[4] != 2 or content[5] != 1:
raise LayoutFailure("unsupported-target", "Initial layout profile requires ELF64 little-endian objects.")
if len(content) < 64:
raise LayoutFailure("format", "Selected ELF64 header is truncated.")
if int.from_bytes(content[18:20], "little") != 62:
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF objects.")
if int.from_bytes(content[16:18], "little") not in (1, 2, 3):
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
from pwnlib.context import context
with context.local(cache_dir=str(cache), log_level="warning"):
@@ -193,8 +185,8 @@ def inspect_elf(path, cache):
if type is None or section.header.sh_type == type)
try:
image = LayoutELF(str(path), checksec=False)
if (image.arch, image.bits, image.endian) != (PROFILES[profile], bits, order):
raise LayoutFailure("unsupported-target", PROFILE_SCOPE)
if image.arch != "amd64" or image.bits != 64 or image.endian != "little":
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF64 little-endian objects.")
if image.header.e_type not in ("ET_EXEC", "ET_DYN", "ET_REL"):
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
length = len(content)
@@ -216,7 +208,7 @@ def inspect_elf(path, cache):
})
if h.sh_type in ("SHT_SYMTAB", "SHT_DYNSYM"):
if h.sh_entsize < image.structs.Elf_Sym.sizeof():
raise LayoutFailure("format", f"ELF symbol entry is smaller than the decoded Elf{bits}_Sym structure.")
raise LayoutFailure("format", "ELF symbol entry is smaller than the decoded Elf64_Sym structure.")
if h.sh_link >= len(all_sections) or all_sections[h.sh_link].header.sh_type != "SHT_STRTAB":
raise LayoutFailure("format", f"ELF symbol table {index} links to invalid string table {h.sh_link}.")
strings = all_sections[h.sh_link]
+84 -88
View File
@@ -1,18 +1,4 @@
import { defineConfig } from "vitepress";
import { dirname, relative, resolve, sep } from "node:path";
import { fileURLToPath } from "node:url";
import {
documentationRewrites,
documentationSections,
} from "./documentation.js";
const documentationRoot = fileURLToPath(new URL("..", import.meta.url));
const rewrites: Record<string, string> = {
...documentationRewrites,
"README.md": "documentation.md",
"specs/README.md": "specs/index.md",
"adr/README.md": "adr/index.md",
};
export default defineConfig({
title: "REA",
@@ -20,92 +6,102 @@ export default defineConfig({
"Reverse engineer binaries, applications, and runtime behavior with one CLI and MCP server.",
lang: "en-US",
base: "/rea/",
rewrites,
markdown: {
config(md) {
// Repository links resolve from their physical source. Render them from
// the stable public route instead, including links across categories.
md.core.ruler.after("inline", "documentation-source-links", (state) => {
const physicalPath =
state.env.realPath ??
(state.env.relativePath
? resolve(documentationRoot, state.env.relativePath)
: null);
if (physicalPath === null) return;
const renderedPath =
state.env.path ??
resolve(
documentationRoot,
rewrites[state.env.relativePath] ?? state.env.relativePath,
);
for (const block of state.tokens) {
for (const token of block.children ?? []) {
const attribute =
token.type === "link_open"
? "href"
: token.type === "image"
? "src"
: null;
if (attribute === null) continue;
const destination = token.attrGet(attribute);
if (
!destination ||
/^(?:[a-z][a-z\d+.-]*:|[/#])/iu.test(destination)
)
continue;
const separator = destination.search(/[?#]/u);
const path =
separator === -1 ? destination : destination.slice(0, separator);
const suffix = separator === -1 ? "" : destination.slice(separator);
const source = relative(
documentationRoot,
resolve(dirname(physicalPath), path),
)
.split(sep)
.join("/");
const rendered = relative(
dirname(renderedPath),
resolve(
documentationRoot,
source.startsWith("public/")
? source.slice("public/".length)
: (rewrites[source] ?? source),
),
)
.split(sep)
.join("/");
token.attrSet(attribute, rendered + suffix);
}
}
});
},
},
srcExclude: ["readme/**"],
themeConfig: {
nav: [
{ text: "Get started", link: "/installation" },
{ text: "Documentation", link: "/documentation" },
{ text: "Reference", link: "/mcp-contracts" },
{ text: "Roadmap", link: "/roadmap" },
],
sidebar: [
{
text: "Documentation directory",
text: "Getting started",
items: [
{ text: "All guides", link: "/documentation" },
{ text: "Authoring conventions", link: "/development/documentation" },
{ text: "Feature specifications", link: "/specs/" },
{ text: "Architecture decisions", link: "/adr/" },
{ text: "Installation and setup", link: "/installation" },
{ text: "Client configuration", link: "/client-configuration" },
{ text: "CLI and Evidence", link: "/cli" },
{ text: "Agent prompts", link: "/mcp-prompts" },
{ text: "Roadmap", link: "/roadmap" },
],
},
{
text: "Investigation guides",
items: [
{ text: "Native binaries", link: "/native-investigation" },
{ text: "Go build metadata", link: "/go-binaries" },
{ text: "IL2CPP metadata", link: "/il2cpp-metadata" },
{ text: "Model artifacts", link: "/model-artifacts" },
{ text: "Rust dependencies", link: "/rust-binaries" },
{ text: "PE certificates", link: "/pe-certificates" },
{ text: "PE imports", link: "/pe-imports" },
{ text: "ELF notes", link: "/elf-notes" },
{ text: "Apple IPS crash reports", link: "/apple-ips" },
{ text: "CPython pyc files", link: "/python-pyc" },
{ text: "Erlang BEAM", link: "/erlang-beam" },
{
text: "JavaScript and Electron",
link: "/javascript-artifact-reconstruction",
},
{
text: "JavaScript workflows",
link: "/javascript-application-workflows",
},
{ text: "Browser observation", link: "/browser-observation" },
{
text: "Web network captures",
link: "/web-network-captures",
},
{ text: "Agent environments", link: "/agent-environments" },
{ text: "Electron observation", link: "/electron-observation" },
{ text: "Electron fuses", link: "/electron-fuses" },
{ text: "Process capture", link: "/process-capture" },
{ text: ".NET assemblies", link: "/managed-code-analysis" },
{
text: "Python packaged applications",
link: "/python-packaged-applications",
},
{ text: "Android applications", link: "/android-analysis" },
{ text: "Apple applications", link: "/apple-application-analysis" },
{ text: "Desktop installations", link: "/desktop-applications" },
{ text: "Firmware", link: "/firmware-analysis" },
{ text: "SQLite databases", link: "/sqlite-databases" },
{ text: "Packet captures", link: "/packet-captures" },
{ text: "Signal recordings", link: "/signal-recordings" },
],
},
{
text: "Provider guides",
collapsed: true,
items: [
{ text: "IDA", link: "/ida-provider" },
{ text: "Kuna", link: "/kuna-provider" },
{ text: "Ghidra on Windows", link: "/windows-ghidra-p0" },
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
{ text: "Ghidra raw binary", link: "/raw-binary" },
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
{ text: "Ghidra PowerPC", link: "/ghidra-ppc" },
{ text: "Ghidra semantics", link: "/ghidra-provider" },
],
},
{
text: "Reference and development",
collapsed: true,
items: [
{ text: "MCP contracts", link: "/mcp-contracts" },
{ text: "Tool design", link: "/tool-design" },
{
text: "Admission limits and recovery",
link: "/design/limits-and-recovery",
},
{ text: "Testing", link: "/testing" },
{ text: "Client verification", link: "/client-verification" },
{
text: "Reconstruction readiness",
link: "/reconstruction-readiness",
},
{ text: "Releasing", link: "/releasing" },
],
},
...documentationSections.map((section) => ({
text: section.text,
collapsed: section.text !== "Getting started",
items: section.items.map((item) => ({
text: item.text,
link: item.route,
})),
})),
],
search: { provider: "local" },
socialLinks: [
-664
View File
@@ -1,664 +0,0 @@
// Authored guide ownership: physical sources and stable public routes.
export const documentationSections = [
{
text: "Getting started",
items: [
{
text: "Installation and setup",
source: "getting-started/installation.md",
route: "/installation",
},
{
text: "Client configuration and troubleshooting",
source: "getting-started/client-configuration.md",
route: "/client-configuration",
},
{
text: "CLI and Evidence",
source: "getting-started/cli.md",
route: "/cli",
},
{
text: "Guided MCP prompts and completion",
source: "getting-started/mcp-prompts.md",
route: "/mcp-prompts",
},
],
},
{
text: "Native binaries",
items: [
{
text: "Native binary investigation",
source: "guides/native/native-investigation.md",
route: "/native-investigation",
},
{
text: "Go binary build metadata",
source: "guides/native/go-binaries.md",
route: "/go-binaries",
},
{
text: "Offline binary diagnostics",
source: "guides/native/binary-diagnostics.md",
route: "/binary-diagnostics",
},
{
text: "Recorded crash evidence",
source: "guides/native/recorded-crashes.md",
route: "/recorded-crashes",
},
{
text: "Windows Shell Link shortcuts",
source: "guides/native/windows-shortcut.md",
route: "/windows-shortcut",
},
],
},
{
text: "JavaScript and Electron",
items: [
{
text: "JavaScript Application Graph",
source: "guides/javascript/javascript-application-graph.md",
route: "/javascript-application-graph",
},
{
text: "Cross-layer JavaScript application workflows",
source: "guides/javascript/javascript-application-workflows.md",
route: "/javascript-application-workflows",
},
{
text: "JavaScript artifact reconstruction",
source: "guides/javascript/javascript-artifact-reconstruction.md",
route: "/javascript-artifact-reconstruction",
},
{
text: "JavaScript source recovery",
source: "guides/javascript/javascript-recovery.md",
route: "/javascript-recovery",
},
{
text: "Passive Node and Electron runtime observation",
source: "guides/javascript/javascript-runtime-observation.md",
route: "/javascript-runtime-observation",
},
{
text: "JavaScript static/runtime reconciliation",
source: "guides/javascript/javascript-runtime-reconciliation.md",
route: "/javascript-runtime-reconciliation",
},
{
text: "Electron file-page observation",
source: "guides/javascript/electron-observation.md",
route: "/electron-observation",
},
{
text: "V8 code cache inspection",
source: "guides/javascript/v8-code-cache.md",
route: "/v8-code-cache",
},
],
},
{
text: "Browser and captured websites",
items: [
{
text: "Website observation with CDP",
source: "guides/web/browser-observation.md",
route: "/browser-observation",
},
{
text: "Browser scenario contract",
source: "guides/web/browser-scenario-contract.md",
route: "/browser-scenario-contract",
},
{
text: "Website runtime source attribution",
source: "guides/web/web-runtime.md",
route: "/web-runtime",
},
{
text: "Historical web network captures",
source: "guides/web/web-network-captures.md",
route: "/web-network-captures",
},
{
text: "HTTP/3 capture lab",
source: "http3-capture-lab.md",
route: "/http3-capture-lab",
},
{
text: "Captured website source locations",
source: "guides/web/web-source-location.md",
route: "/web-source-location",
},
{
text: "Captured website module relationships",
source: "guides/web/website-module-trace.md",
route: "/website-module-trace",
},
{
text: "Captured website scripts and static JavaScript analysis",
source: "guides/web/website-script-export.md",
route: "/website-script-export",
},
],
},
{
text: "Android applications",
items: [
{
text: "Static Android APK analysis",
source: "guides/android/android-analysis.md",
route: "/android-analysis",
},
{
text: "Apktool resource decoding",
source: "guides/android/apktool-resource-analysis.md",
route: "/apktool-resource-analysis",
},
{
text: "ADB device inspection and acquisition",
source: "guides/android/adb-device-analysis.md",
route: "/adb-device-analysis",
},
{
text: "Flutter build identification",
source: "guides/android/flutter-analysis.md",
route: "/flutter-analysis",
},
],
},
{
text: "Apple applications",
items: [
{
text: "Apple Application Inventory Projection",
source: "guides/apple/apple-application-analysis.md",
route: "/apple-application-analysis",
},
],
},
{
text: ".NET assemblies",
items: [
{
text: "Managed-code analysis",
source: "guides/managed/managed-code-analysis.md",
route: "/managed-code-analysis",
},
],
},
{
text: "Firmware",
items: [
{
text: "Firmware analysis",
source: "guides/firmware/firmware-analysis.md",
route: "/firmware-analysis",
},
],
},
{
text: "HarmonyOS packages",
items: [
{
text: "Static HarmonyOS package inventory",
source: "guides/harmony/harmony-analysis.md",
route: "/harmony-analysis",
},
],
},
{
text: "WebAssembly",
items: [
{
text: "Offline WebAssembly artifacts",
source: "guides/wasm/wasm-artifacts.md",
route: "/wasm-artifacts",
},
],
},
{
text: "EVM contracts",
items: [
{
text: "Offline EVM bytecode",
source: "guides/evm/evm-bytecode.md",
route: "/evm-bytecode",
},
],
},
{
text: "Databases and recorded packets",
items: [
{
text: "SQLite database snapshots",
source: "guides/data/sqlite-databases.md",
route: "/sqlite-databases",
},
{
text: "Recorded packet capture inspection",
source: "guides/data/packet-captures.md",
route: "/packet-captures",
},
{
text: "Packet capture summaries",
source: "guides/data/pcap-analysis.md",
route: "/pcap-analysis",
},
],
},
{
text: "Process capture and instrumentation",
items: [
{
text: "Process capture",
source: "guides/runtime/process-capture.md",
route: "/process-capture",
},
{
text: "Frida instrumentation",
source: "guides/runtime/frida-instrumentation.md",
route: "/frida-instrumentation",
},
],
},
{
text: "Provider setup and formats",
items: [
{
text: "Ghidra provider semantics",
source: "providers/ghidra-provider.md",
route: "/ghidra-provider",
},
{
text: "DOS MZ and COM analysis with Ghidra",
source: "providers/ghidra-dos.md",
route: "/ghidra-dos",
},
{
text: "Optional NativeAOT metadata recovery",
source: "providers/ghidra-nativeaot.md",
route: "/ghidra-nativeaot",
},
{
text: "MIPS ELF analysis through Ghidra",
source: "providers/ghidra-mips.md",
route: "/ghidra-mips",
},
{
text: "PSP static ELF analysis through Ghidra",
source: "providers/ghidra-psp.md",
route: "/ghidra-psp",
},
{
text: "RISC-V ELF analysis through Ghidra",
source: "providers/ghidra-riscv.md",
route: "/ghidra-riscv",
},
{
text: "Windows Ghidra P0",
source: "providers/windows-ghidra-p0.md",
route: "/windows-ghidra-p0",
},
{
text: "Headerless raw-binary analysis with Ghidra",
source: "providers/raw-binary.md",
route: "/raw-binary",
},
{
text: "IDA provider",
source: "providers/ida-provider.md",
route: "/ida-provider",
},
{
text: "JEB instrumentation and analysis",
source: "providers/jeb-analysis.md",
route: "/jeb-analysis",
},
],
},
{
text: "Contracts and evidence",
items: [
{
text: "MCP runtime contracts",
source: "reference/mcp-contracts.md",
route: "/mcp-contracts",
},
{
text: "Reconstruction obligation ledgers",
source: "reference/reconstruction-obligation-ledgers.md",
route: "/reconstruction-obligation-ledgers",
},
],
},
{
text: "Development and verification",
items: [
{
text: "Settings and recovery",
source: "design/limits-and-recovery.md",
route: "/design/limits-and-recovery",
},
{
text: "MCP tool design",
source: "development/tool-design.md",
route: "/tool-design",
},
{
text: "Testing REA",
source: "development/testing.md",
route: "/testing",
},
{
text: "Client and agent verification",
source: "development/client-verification.md",
route: "/client-verification",
},
{
text: "Reconstruction readiness conformance",
source: "development/reconstruction-readiness.md",
route: "/reconstruction-readiness",
},
{
text: "Releasing REA",
source: "development/releasing.md",
route: "/releasing",
},
{
text: "REA roadmap",
source: "development/roadmap.md",
route: "/roadmap",
},
],
},
{
text: "Release migrations",
items: [
{
text: "Upgrading from REA 6.1 to 6.2",
source: "migrations/migration-6.2.md",
route: "/migration-6.2",
},
],
},
{
text: "Signal recordings",
items: [
{
text: "Recorded signal inspection",
source: "guides/signals/signal-recordings.md",
route: "/signal-recordings",
},
],
},
{
text: "Additional guides",
items: [
{
text: "Offline API contract inventory",
source: "api-contracts.md",
route: "/api-contracts",
},
{
text: "Bun standalone executables",
source: "bun-standalone.md",
route: "/bun-standalone",
},
{
text: "Capability composition and ownership",
source: "capability-composition.md",
route: "/capability-composition",
},
{
text: ".NET single-file bundles",
source: "dotnet-bundles.md",
route: "/dotnet-bundles",
},
{
text: "ESP flash images",
source: "esp-flash-image.md",
route: "/esp-flash-image",
},
{
text: "U-Boot FIT images",
source: "fit-images.md",
route: "/fit-images",
},
{
text: "Measure MCP catalog costs in Codex",
source: "mcp-catalog-evaluation.md",
route: "/mcp-catalog-evaluation",
},
{
text: "Offline Playwright traces",
source: "playwright-traces.md",
route: "/playwright-traces",
},
{
text: "Testing lanes and provider verification",
source: "testing-lanes.md",
route: "/testing-lanes",
},
{
text: "UF2 and DfuSe images",
source: "uf2-dfuse.md",
route: "/uf2-dfuse",
},
{
text: "WeChat mini-program wxapkg archives",
source: "wxapkg.md",
route: "/wxapkg",
},
{
text: "Binary hardening",
source: "binary-hardening.md",
route: "/binary-hardening",
},
{
text: "Desktop application capture",
source: "desktop-application-capture.md",
route: "/desktop-application-capture",
},
{
text: "Desktop application export",
source: "desktop-application-export.md",
route: "/desktop-application-export",
},
{
text: "Desktop application state",
source: "desktop-application-state.md",
route: "/desktop-application-state",
},
{
text: "Desktop application tasks",
source: "desktop-application-tasks.md",
route: "/desktop-application-tasks",
},
{
text: "Desktop build computer",
source: "desktop-build-computer.md",
route: "/desktop-build-computer",
},
{
text: "Desktop build witness",
source: "desktop-build-witness.md",
route: "/desktop-build-witness",
},
{
text: "iNES containers",
source: "ines-containers.md",
route: "/ines-containers",
},
{
text: "Build RL environments with REA",
source: "agent-environments-quickstart.md",
route: "/agent-environments-quickstart",
},
{
text: "Agent environments from recorded captures",
source: "agent-environments.md",
route: "/agent-environments",
},
{
text: "APK signature schemes",
source: "apk-signature.md",
route: "/apk-signature",
},
{
text: "Apple IPS crash reports",
source: "apple-ips.md",
route: "/apple-ips",
},
{
text: "Static analysis in GitHub Actions",
source: "ci-analysis.md",
route: "/ci-analysis",
},
{
text: "Desktop installation directories",
source: "desktop-applications.md",
route: "/desktop-applications",
},
{
text: "Flattened devicetree inspection",
source: "devicetree.md",
route: "/devicetree",
},
{
text: "Electron fuse state",
source: "electron-fuses.md",
route: "/electron-fuses",
},
{ text: "ELF note tables", source: "elf-notes.md", route: "/elf-notes" },
{
text: "Erlang BEAM files",
source: "erlang-beam.md",
route: "/erlang-beam",
},
{ text: "ESP images", source: "esp-image.md", route: "/esp-image" },
{
text: "GameCube DOL analysis through Ghidra",
source: "ghidra-dol.md",
route: "/ghidra-dol",
},
{
text: "PowerPC ELF analysis through Ghidra",
source: "ghidra-ppc.md",
route: "/ghidra-ppc",
},
{
text: "Godot PCK archives",
source: "godot-pck.md",
route: "/godot-pck",
},
{
text: "Static CUDA and HIP code objects",
source: "gpu-code-objects.md",
route: "/gpu-code-objects",
},
{
text: "Intel HEX and S-record images",
source: "hex-records.md",
route: "/hex-records",
},
{
text: "IL2CPP metadata",
source: "il2cpp-metadata.md",
route: "/il2cpp-metadata",
},
{
text: "Static JavaScript route inventory",
source: "javascript-route-inventory.md",
route: "/javascript-route-inventory",
},
{
text: "Linux kernel modules",
source: "kernel-modules.md",
route: "/kernel-modules",
},
{
text: "Kuna provider",
source: "kuna-provider.md",
route: "/kuna-provider",
},
{
text: "Model file declarations",
source: "model-artifacts.md",
route: "/model-artifacts",
},
{
text: "Match native functions across builds",
source: "native-function-matching.md",
route: "/native-function-matching",
},
{
text: "Node.js single executable applications",
source: "node-sea.md",
route: "/node-sea",
},
{
text: "PE Authenticode certificate tables",
source: "pe-certificates.md",
route: "/pe-certificates",
},
{
text: "PE import and delay-load tables",
source: "pe-imports.md",
route: "/pe-imports",
},
{
text: "PE TLS callbacks and load configuration",
source: "pe-load-config.md",
route: "/pe-load-config",
},
{
text: "Python packaged applications",
source: "python-packaged-applications.md",
route: "/python-packaged-applications",
},
{
text: "CPython pyc inspection",
source: "python-pyc.md",
route: "/python-pyc",
},
{
text: "Reconstruction coverage",
source: "reconstruction-coverage.md",
route: "/reconstruction-coverage",
},
{
text: "Reverse-engineering tool integrations",
source: "reverse-engineering-tools.md",
route: "/reverse-engineering-tools",
},
{
text: "Rust binary dependency metadata",
source: "rust-binaries.md",
route: "/rust-binaries",
},
{
text: "U-Boot legacy uImage containers",
source: "uimage.md",
route: "/uimage",
},
{
text: "UnityFS AssetBundle archives",
source: "unityfs.md",
route: "/unityfs",
},
],
},
];
export const documentationRewrites = Object.fromEntries(
documentationSections.flatMap((section) =>
section.items.flatMap((item) => {
const target = `${item.route.slice(1)}.md`;
return item.source === target ? [] : [[item.source, target]];
}),
),
);
-197
View File
@@ -1,197 +0,0 @@
# Documentation directory
REA documentation is organized by reader task and document owner. This directory
links every authored guide; the [site](https://rea.tools/) also serves separate
product pages. The VitePress documentation source is this `docs/` tree.
## Getting started
- [Installation and setup](getting-started/installation.md)
- [Client configuration and troubleshooting](getting-started/client-configuration.md)
- [CLI and Evidence](getting-started/cli.md)
- [Guided MCP prompts and completion](getting-started/mcp-prompts.md)
## Native binaries
- [Native binary investigation](guides/native/native-investigation.md)
- [Go binary build metadata](guides/native/go-binaries.md)
- [Offline binary diagnostics](guides/native/binary-diagnostics.md)
- [Recorded crash evidence](guides/native/recorded-crashes.md)
- [Windows Shell Link shortcuts](guides/native/windows-shortcut.md)
## JavaScript and Electron
- [JavaScript Application Graph](guides/javascript/javascript-application-graph.md)
- [Cross-layer JavaScript application workflows](guides/javascript/javascript-application-workflows.md)
- [JavaScript artifact reconstruction](guides/javascript/javascript-artifact-reconstruction.md)
- [JavaScript source recovery](guides/javascript/javascript-recovery.md)
- [Passive Node and Electron runtime observation](guides/javascript/javascript-runtime-observation.md)
- [JavaScript static/runtime reconciliation](guides/javascript/javascript-runtime-reconciliation.md)
- [Electron file-page observation](guides/javascript/electron-observation.md)
- [V8 code cache inspection](guides/javascript/v8-code-cache.md)
## Browser and captured websites
- [Website observation with CDP](guides/web/browser-observation.md)
- [Browser scenario contract](guides/web/browser-scenario-contract.md)
- [Website runtime source attribution](guides/web/web-runtime.md)
- [Historical web network captures](guides/web/web-network-captures.md)
- [HTTP/3 capture lab](http3-capture-lab.md)
- [Captured website source locations](guides/web/web-source-location.md)
- [Captured website module relationships](guides/web/website-module-trace.md)
- [Captured website scripts and static JavaScript analysis](guides/web/website-script-export.md)
## Android applications
- [Static Android APK analysis](guides/android/android-analysis.md)
- [Apktool resource decoding](guides/android/apktool-resource-analysis.md)
- [ADB device inspection and acquisition](guides/android/adb-device-analysis.md)
- [Flutter build identification](guides/android/flutter-analysis.md)
## Apple applications
- [Apple Application Inventory Projection](guides/apple/apple-application-analysis.md)
## .NET assemblies
- [Managed-code analysis](guides/managed/managed-code-analysis.md)
## Firmware
- [Firmware analysis](guides/firmware/firmware-analysis.md)
## HarmonyOS packages
- [Static HarmonyOS package inventory](guides/harmony/harmony-analysis.md)
## WebAssembly
- [Offline WebAssembly artifacts](guides/wasm/wasm-artifacts.md)
## EVM contracts
- [Offline EVM bytecode](guides/evm/evm-bytecode.md)
## Databases and recorded packets
- [SQLite database snapshots](guides/data/sqlite-databases.md)
- [Recorded packet capture inspection](guides/data/packet-captures.md)
- [Packet capture summaries](guides/data/pcap-analysis.md)
## Process capture and instrumentation
- [Process capture](guides/runtime/process-capture.md)
- [Frida instrumentation](guides/runtime/frida-instrumentation.md)
## Signal recordings
- [Recorded signal inspection](guides/signals/signal-recordings.md)
## Provider setup and formats
- [Ghidra provider semantics](providers/ghidra-provider.md)
- [DOS MZ and COM analysis with Ghidra](providers/ghidra-dos.md)
- [Optional NativeAOT metadata recovery](providers/ghidra-nativeaot.md)
- [MIPS ELF analysis through Ghidra](providers/ghidra-mips.md)
- [PSP static ELF analysis through Ghidra](providers/ghidra-psp.md)
- [RISC-V ELF analysis through Ghidra](providers/ghidra-riscv.md)
- [Windows Ghidra P0](providers/windows-ghidra-p0.md)
- [Headerless raw-binary analysis with Ghidra](providers/raw-binary.md)
- [IDA provider](providers/ida-provider.md)
- [JEB instrumentation and analysis](providers/jeb-analysis.md)
## Contracts and evidence
- [MCP runtime contracts](reference/mcp-contracts.md)
- [Reconstruction obligation ledgers](reference/reconstruction-obligation-ledgers.md)
## Development and verification
- [Settings and recovery](design/limits-and-recovery.md)
- [MCP tool design](development/tool-design.md)
- [Testing REA](development/testing.md)
- [Client and agent verification](development/client-verification.md)
- [Reconstruction readiness conformance](development/reconstruction-readiness.md)
- [Releasing REA](development/releasing.md)
- [REA roadmap](development/roadmap.md)
## Release migrations
- [Upgrading from REA 6.1 to 6.2](migrations/migration-6.2.md)
## Architecture and authoring
- [Architecture map](https://github.com/morluto/rea/blob/main/docs/architecture/architecture.mermaid)
- [Documentation ownership and contribution model](development/documentation.md)
- [Feature specifications](specs/README.md)
- [Architectural decision records](adr/README.md)
## Machine-owned documentation
| Path | Owner and update command |
| ----------------------------------------- | ----------------------------------------------------------------------------------------- |
| `public/product-catalog.json` | Ignored contract projection; `npm run docs:generate` |
| `verification/managed-conformance-*.json` | Ignored managed conformance projections; `npm run evidence:generate` |
| `error-contract.schema.json` | Tracked schema from contracts; `npm run docs:error-schema` |
| `user-visible-errors.csv` | Authored error classification inventory; preserve its schema |
| `releases/<version>.contributions.json` | Durable release contribution record; follow the [release guide](development/releasing.md) |
| `assets/` | Shared authored images referenced by guides |
Do not edit generated catalogs or evidence to change product facts. Change their
canonical contracts or producer and regenerate. Historical release records retain
the exact commit messages and references from the release they describe.
## Additional guides
- [Offline API contract inventory](api-contracts.md)
- [Bun standalone executables](bun-standalone.md)
- [Capability composition and ownership](capability-composition.md)
- [.NET single-file bundles](dotnet-bundles.md)
- [ESP flash images](esp-flash-image.md)
- [U-Boot FIT images](fit-images.md)
- [Measure MCP catalog costs in Codex](mcp-catalog-evaluation.md)
- [Offline Playwright traces](playwright-traces.md)
- [Testing lanes and provider verification](testing-lanes.md)
- [UF2 and DfuSe images](uf2-dfuse.md)
- [WeChat mini-program wxapkg archives](wxapkg.md)
- [Binary hardening](binary-hardening.md)
- [Desktop application capture](desktop-application-capture.md)
- [Desktop application export](desktop-application-export.md)
- [Desktop application state](desktop-application-state.md)
- [Desktop application tasks](desktop-application-tasks.md)
- [Desktop build computer](desktop-build-computer.md)
- [Desktop build witness](desktop-build-witness.md)
- [iNES containers](ines-containers.md)
- [Build RL environments with REA](agent-environments-quickstart.md)
- [Agent environments from recorded captures](agent-environments.md)
- [APK signature schemes](apk-signature.md)
- [Apple IPS crash reports](apple-ips.md)
- [Static analysis in GitHub Actions](ci-analysis.md)
- [Desktop installation directories](desktop-applications.md)
- [Flattened devicetree inspection](devicetree.md)
- [Electron fuse state](electron-fuses.md)
- [ELF note tables](elf-notes.md)
- [Erlang BEAM files](erlang-beam.md)
- [ESP images](esp-image.md)
- [GameCube DOL analysis through Ghidra](ghidra-dol.md)
- [PowerPC ELF analysis through Ghidra](ghidra-ppc.md)
- [Godot PCK archives](godot-pck.md)
- [Static CUDA and HIP code objects](gpu-code-objects.md)
- [Intel HEX and S-record images](hex-records.md)
- [IL2CPP metadata](il2cpp-metadata.md)
- [Static JavaScript route inventory](javascript-route-inventory.md)
- [Linux kernel modules](kernel-modules.md)
- [Kuna provider](kuna-provider.md)
- [Model file declarations](model-artifacts.md)
- [Match native functions across builds](native-function-matching.md)
- [Node.js single executable applications](node-sea.md)
- [PE Authenticode certificate tables](pe-certificates.md)
- [PE import and delay-load tables](pe-imports.md)
- [PE TLS callbacks and load configuration](pe-load-config.md)
- [Python packaged applications](python-packaged-applications.md)
- [CPython pyc inspection](python-pyc.md)
- [Reconstruction coverage](reconstruction-coverage.md)
- [Reverse-engineering tool integrations](reverse-engineering-tools.md)
- [Rust binary dependency metadata](rust-binaries.md)
- [U-Boot legacy uImage containers](uimage.md)
- [UnityFS AssetBundle archives](unityfs.md)
@@ -25,21 +25,7 @@ REA never installs the Android SDK, an emulator, or a system image.
Readiness is `rea inspect-adb-client`: it runs `adb version` and reports the
resolved binary, its version, and where it was selected from. A missing
binary reports `capability_unavailable` with the `REA_ADB_PATH` remediation
rather than attempting an installation. A path that cannot resolve to a file
(`ENOENT`, `ENOTDIR`, `ELOOP` or `ENAMETOOLONG`) is reported as missing. A
selected path the host refuses to execute, such as a directory or a file
without execute permission (`EACCES`) or one the host does not permit
(`EPERM`), and any other spawn error report `capability_unavailable` naming
that path. Exhausted host capacity (`EAGAIN`, `EMFILE`, `ENFILE` or `ENOMEM`)
is not a configuration failure. `inspect_adb_client`, and any device
operation whose adb cannot start at its version probe or at a later adb
command, report a retryable `capacity` resource constraint rather than a
capability refusal or an adb protocol failure; the `binary_session` inventory
reports the provider as `runtime_unavailable`. Both say the host could not
start adb and ask for a retry after freeing processes or descriptors, without
blaming `REA_ADB_PATH`. When adb never started, the refusal diagnostics record
the spawn error as `filesystem_error` with `path_source`, never as an
`exit_code`.
rather than attempting an installation.
The `binary_session` inventory uses the same bounded client-version probe; it
does not query devices, start the adb server, or establish that a requested
serial is reachable. Device state is checked by each device operation.
@@ -117,10 +103,6 @@ existing analysis families directly, for example
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
Screen capture validates the PNG signature, IHDR framing, dimensions, and IHDR
checksum before publishing. It does not validate encoding fields or decode and
validate image contents; that limitation is retained in Evidence.
When `push_adb_file` receives an existing directory, adb places the local file
inside it under the local basename; Evidence reports that effective file path
while retaining the caller's directory path in the input parameters. The
@@ -188,7 +170,7 @@ and during shutdown.
property value remains available in `properties`.
- **Device paths are not host paths.** `base_apk_device_path` values live on
the device filesystem. Only pulls and captures produce host-local paths,
with byte size and SHA-256 digest on success. A pulled file's host name is
each paired with byte size and SHA-256 digest. A pulled file's host name is
the device path's final POSIX segment. REA preserves the original device
path and filename; a missing final segment or `.`/`..` is refused before
output creation. Windows hosts follow [Windows filename rules](https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file),
@@ -217,18 +199,12 @@ and during shutdown.
`uninstall_adb_package` declare `mutatesTarget` and never run implicitly:
pushes check for existing device files before transfer and refuse them
unless `overwrite` is set. A failed or unexpected existence probe stops
transfer even with `overwrite`. A completed `test -e` exit of 1 establishes
absence only with empty stdout and either empty stderr or a recognized,
completed adb-server startup notice sequence. Startup notices are host
observations, not remote-path diagnostics; successful push receipts retain
them in `raw_result.device_path_probes`. Incomplete startup, extra diagnostics,
truncation and cancellation do not establish absence. The check is not atomic, so a different
transfer even with `overwrite`; only a clean `test -e` exit of 1 with no
output is interpreted as absence. The check is not atomic, so a different
process can create the path before `adb push` writes it. When available, the device's
`sha256sum` verifies the transfer. A missing command, output without a
recognizable digest, or host capacity that prevents the digest check from
starting reports `device_digest_source: "unavailable"` with an
unverified-transfer limitation rather than a retryable error; installs
preserve the
`sha256sum` verifies the transfer. A missing command or output without a
recognizable digest reports `device_digest_source: "unavailable"` with an
unverified-transfer limitation; installs preserve the
device's `Failure [...]` reason verbatim.
- **Producer identity stays attached.** Every result embeds the producing
adb client (`binary_path`, `path_source`, `version`, `installed_path`).
@@ -249,10 +225,6 @@ and during shutdown.
- Pull and capture outputs are additive: the destination must not already
exist. Re-running a completed pull refuses the existing directory instead
of overwriting evidence.
- If a single-file pull fails and REA observes a regular file at the destination,
the error includes its `local_path` with `transfer_status: "unconfirmed"`
and `digest_status: "unknown"`. The file remains in place; the receipt does
not establish that the transfer completed.
- Cancellation during a package pull preserves caller-selected output. The
error includes partial observations identifying completed artifacts and the
interrupted local path; remaining APKs were not attempted.
@@ -281,18 +253,18 @@ beyond already-connected devices (`adb connect`) are not exposed yet.
## Real verification
`npm run verify:adb` exercises observation and transfer operations against a live device. It
`npm run verify:adb` exercises every operation against a live device. It
requires `adb` (or `REA_ADB_PATH`) and any attached emulator or phone;
`--serial` selects one when several are attached, `--pull PACKAGE` pulls a
real package and verifies the returned digests against the files on disk.
The optional `--install-apk` lifecycle is unavailable until the selected APK
identity and device cleanup ownership can be established safely.
real package and verifies the returned digests against the files on disk,
and `--install-apk PATH` drives a full install → resolve → uninstall
lifecycle with a lane-owned probe.
Record: verified against adb 34.0.5 on Linux against an Android 14
(API 34) x86_64 emulator — all operations above, a 224-package inventory,
287 services, 92 features, a real two-APK split set pulled with byte-exact
digests, a verified push/pull roundtrip, a 1.3 MB screen capture, and the
complete package lifecycle; see `docs/development/testing.md` for the lane summary.
complete package lifecycle; see `docs/testing.md` for the lane summary.
Intent integer extras use signed decimal text. The ADB argument encoder removes
redundant leading zeros for Android's radix-sensitive `--ei` parser while retained
-9
View File
@@ -1,9 +0,0 @@
# Architectural decision records
Store one actual decision per `docs/adr/<domain>/<decision-slug>.md`, without a
global sequence number. Use meaningful names and the Michael Nygard sections:
Status, Context, Decision and Consequences. Link each decision from its owning
feature specification.
See [the authoring guide](../development/documentation.md#feature-lifecycle) for
the distinction between specifications, roadmaps and isolated decisions.
-206
View File
@@ -1,206 +0,0 @@
# Build RL environments with REA
REA turns a recorded web app session into a runnable environment for training
and testing AI agents. An agent can practice tasks such as finding an issue,
creating a folder, or editing a record without running the original app.
REA builds the tools the agent calls, the local data those tools use, and a
test that checks the environment starts and responds. It supports changes
when the recording confirms their effects. You can then use the environment
with your RL framework to set task goals, check rewards, and run training.
## Before you start
Install [REA and its agent integration](getting-started/installation.md). To record an app,
you also need a running test instance and a Chrome-family browser. Choose an
instance where the agent is allowed to create, edit, and delete data. REA
records the app you provide; you start it and prepare its test data.
To run the exported environment, install the
[AgentEnv framework](https://github.com/scaleapi/agentenv-framework#install)
and start Docker. REA's [compatibility check](development/testing.md#exported-environment-bundles)
uses AgentEnv version `0.9.1431` with Python 3.12.
There are two ways to build an environment: ask your agent to record the app,
or start from a recording you already have.
## 1. Record and build with your agent
Give your agent the app URL, the operations you need, your browser's executable
path, and an output directory. For an app with folders, adapt this prompt:
```text
Use REA to build an AgentEnv environment from my test web app at
http://127.0.0.1:3000. Cover listing, creating, renaming, and deleting folders.
You may create and remove test folders. Leave the existing data unchanged.
Sign in as admin using the password in APP_PASSWORD. Use the browser at
/absolute/path/to/chromium. Save the environment to
/absolute/path/to/my-env, which does not exist yet.
Record the session, check which operations the environment supports, and
export it. Report any unsupported operations and what is missing.
```
Replace the URL, account, paths, and folder operations with your own. Set
`APP_PASSWORD` in the environment that starts REA's MCP server so REA can use
it without putting the password in the prompt.
Your agent inspects the app, records a session, checks the result, and exports
a folder containing the runnable environment. It reports which operations
work and which need more evidence. The app is used during recording; the
exported environment runs from the saved data.
After your agent exports the folder, skip to
[run the environment](#run-the-environment). The next section is an
alternative for an existing recording.
## 2. Or build from a saved recording
A HAR file records a browser's network requests and responses. Export one
with API response bodies from your browser's network panel, or use a
recording you already have. To include create, edit, and delete operations,
record those actions and the reads that show their results.
Build the environment into a new or empty directory:
```sh
rea environment-build ./session.har ./my-env --name my-env --json
```
You can also pass capture Evidence JSON saved by REA's
`capture-browser-scenario` or `inspect-web-network-capture` commands. Here,
Evidence means REA's saved recording with its source information. The build
accepts HAR files and those JSON recordings. It reads local data and writes
the environment folder without contacting the app.
Add `--dry-run` to review the build report without writing the folder. For an
unreleased REA checkout, use `--package-tarball` with its packed package so
the exported environment runs the same implementation.
The build report tells you what the environment supports:
| Report field | What to check |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `state` | Whether the environment can answer from local state or uses recorded-response replay. |
| `coverage.reads` | Which read operations use state and which replay recorded answers. |
| `coverage.writes` | Which create, edit, and delete operations are supported. Each withheld operation includes a reason and a recording hint. |
| `warnings` | Recorded content or local paths to review before sharing the folder. |
A successful build means the folder was created. Check the report to confirm
it supports the operations your tasks need. `coverage: null` means the build
could not prepare state for this environment and it will use replay.
<a id="run-the-environment"></a>
## 3. Run the environment
Whether your agent exported the folder or you built it from a HAR, run it
with AgentEnv:
```sh
agent-env run ./my-env
```
Use the folder path your agent reported if it differs. The first run builds
the container image, so it needs access to the image and package sources.
REA includes a basic test, called the smoke task. It checks the expected tool
count and, when a suitable read tool exists, calls it and checks its response
status. Passing this test shows the environment starts and responds in the
tested setup. It does not yet score an agent on an application task.
The folder contains the recording, a tool manifest describing the available
operations, a Dockerfile, environment configuration, and the test. The
[export reference](agent-environments.md#export-a-bundle) describes each file.
## 4. Add tasks and reward checks
Start with one task the exported tools support. Give it a clear goal and a
checkable result. For example, if folder creation is supported, ask your
agent:
```text
Use this exported environment to add an AgentEnv task that asks an agent
to create a folder named Practice. Check the saved environment state to
confirm the folder exists with that name, and use that check to score the
result. Reset the environment before each run so attempts start with the
same data. Use the AgentEnv task and verifier formats.
```
This asks your agent to help build the task and its grading check around the
environment REA generated. A retrieval task might check an answer against a
recorded value. A task that changes data should check the resulting state.
When state is available, the environment supports reset, load, and read-back
operations for repeatable runs.
Use the [AgentEnv documentation](https://www.agentenvframework.com/docs) for
task configuration, agent connections, scoring, and execution. Connect the
resulting tasks to your chosen RL training workflow.
## Improve an operation that is missing
For REA to support a change, the recording needs to show what happened before
and after it. When recording against your test app:
1. Read the list and an existing item before making changes. Repeat the
initial reads to show which values change independently.
2. Create a test item, read it back, and read the list before the next change.
Repeat with a second item and different text.
3. Edit each test item, then read the item and list again. Leave more than a
second between creation and the first edit to show time-dependent changes.
4. Delete the test items, then read them back and read the list to confirm
their removal.
Also record filters, searches, sorting, and paging if your tasks need them.
The agent should check that every recording step completed before using it.
The [recording workflow](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/environments.md)
has the detailed browser instructions, including timeouts and handling IDs
assigned by the app.
Use the report's hints to decide what to record next:
| Reason | Next step |
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| `unconfirmed` | Read the item and its list after the change. |
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
| `no-observation` | Read back a created item. For an edit or delete, also read it before changing it. |
| `contradicted` or `side-effect` | Inspect the reported disagreement. The operation may need behavior REA cannot model yet. |
See the [full reason table](agent-environments.md#build-a-bundle-in-one-step)
for the other cases. Rebuild from the new recording and check the report
again. A new recording replaces the build input; combining separate recording
sessions is not supported.
## Compare with the original app
For a stronger check, record a separate sequence of calls against the real
app, starting from the same test data as the original recording. This is a
probe. Restore that starting data before each probe sequence, then compare
the saved probe recordings with the environment:
```sh
rea environment-compare ./my-env/envs/my-env/manifest.json \
--evidence ./my-env/envs/my-env/evidence.json \
--probe ./probe-create-edit-delete.json --probe ./probe-reads.json --json
```
Each probe file must be capture Evidence JSON from REA. The comparison runs
offline and reports differences in responses, unsupported calls, and missing
observations. It accounts for supported differences in IDs, times, and
modeled generated values. Agreement applies to the calls those probes check.
See the [comparison reference](agent-environments.md#measure-fidelity-against-the-application)
for the exact rules.
## Before sharing the environment
The exported folder includes retained response bodies and local source paths.
Standard authentication headers are excluded, but custom headers, query
values, and body tokens can remain. Review the content you plan to share.
Credential-name warnings point to values worth reviewing; they do not remove
those values or establish that the recording contains no credentials.
For direct MCP serving, replay behavior, state controls, and resource limits,
see the [environment reference](agent-environments.md). The
[implementation index](specs/agent-environments/README.md) links the contributing
PRs and their verification records.
+43 -371
View File
@@ -1,27 +1,16 @@
# Agent environments from recorded captures
For a first environment, start with the
[quickstart](agent-environments-quickstart.md): record an app or use a saved
session, build and run the environment, and add tasks and reward checks.
The [implementation index](specs/agent-environments/README.md) links
the contributing PRs and their verification records. This page is the detailed
command and behavior reference.
Use `rea environment-build` to turn one recorded capture into an agent
environment bundle in one offline step: a folder `agent-env run` builds and
runs, whose MCP server answers from what the capture observed. See
[Build a bundle in one step](#build-a-bundle-in-one-step).
The steps it composes are separate commands too. `rea environment-serve` serves
an environment manifest as an agent environment: an MCP server whose tools
answer from what one retained web network capture observed, plus the
environment card the agentenv framework reads to discover it. By default the
server derives the [state the capture observed](#the-state-a-capture-observed)
at startup and answers read tools from it, and the write tools whose effect the
capture confirmed by changing it; `--replay-only` replays the capture's records
Use `rea environment-serve` to serve an environment manifest as an agent
environment: an MCP server whose tools answer from what one retained web
network capture observed, plus the environment card the agentenv framework
reads to discover it. By default the server derives the
[state the capture observed](#the-state-a-capture-observed) at startup and
answers read tools from it, and the write tools whose effect the capture
confirmed by changing it; `--replay-only` replays the capture's records
instead. Nothing is fetched from the recorded application; every answer comes
from retained Evidence. `rea environment-export` writes the same pair as a
bundle folder; see [Export a bundle](#export-a-bundle).
bundle folder `agent-env run` builds and runs; see
[Export a bundle](#export-a-bundle).
```sh
rea inspect-web-network-capture /absolute/session.har har --format json > capture.json
@@ -30,7 +19,7 @@ rea environment-serve manifest.json --evidence capture.json --port 8765
```
The first command inspects the capture; the second projects the
[environment manifest](guides/web/web-network-captures.md#environment-manifest) view from
[environment manifest](web-network-captures.md#environment-manifest) view from
it, where `view.json` wraps the capture Evidence as the inline source:
```json
@@ -47,195 +36,11 @@ The third command serves that manifest from the state it derives from the
capture; see [Serve from the state](#serve-from-the-state). Both served files
are complete REA Evidence JSON as the commands write it with `--format json`;
the `evidence` member of `view.json` is the whole of `capture.json`.
A `view.json` can also name the capture file as
`{"kind": "evidence-file", "path": "capture.json"}`, which projects it as
retained Evidence instead of charging the whole capture as inline input.
## Build a bundle in one step
```sh
rea environment-build grafana.har ./grafana --json
agent-env run ./grafana
```
`rea environment-build <capture> [directory]` takes a `.har` file, or capture
Evidence JSON from `inspect-web-network-capture` (HAR) or
`capture-browser-scenario`, and writes the bundle
[Export a bundle](#export-a-bundle) describes. It inspects a HAR as
`inspect-web-network-capture` does (Evidence input skips this), projects the
environment manifest from that Evidence as retained Evidence, authenticates
the pair, prepares the state the bundle's server derives at startup, and
writes the bundle with the export. For the same capture path, every file it
writes is byte-identical to what `inspect-web-network-capture`,
`inspect-analysis-view` and `environment-export` write, Evidence ids included,
when `view.json` lists the same rules in the views' order and the export is
given the build's name. It reads one file and writes one folder: it fetches
nothing and launches no browser (HAR inspection runs REA's own decoder
process), so record a browser scenario with `capture-browser-scenario` first
and pass its Evidence. Other capture formats are refused, as the manifest view
refuses them, and a HAR document saved as `.json` is refused with a hint to
name it `.har`.
- `--template-rule <rule>`, repeated, replaces the default opt-in rules
`after-uid-literal`, `opaque-token`, `sibling-variance` and
`prefix-variance`, the rules the published capture-built environments use;
`--format-rules-only` applies none, the views' own default. Rules apply in
the views' fixed order, the report lists them, and a repeated rule is
refused as the views refuse it.
- `--name` names the env folder and env id. Without it the name is the
manifest's, unless the capture's origin is a loopback name or an IP literal:
the capture file then names it (`grafana.har` and `grafana.capture.json`
become `grafana`) and a `name-from-file` warning says so. A file name that
yields no portable label (kebab-case, at most 63 characters for a DNS service
label, not a Windows device name or a label interpreted as an IP address) is
refused; pass `--name`.
- `--package-tarball` works as for the export, and `--capture-timeout-ms`
as for `inspect-web-network-capture`.
- `--dry-run` runs every stage and prints the report with `status: "planned"`
and the files a build would write, but writes nothing and does not check the
directory, which may then be omitted. It reads and hashes a
`--package-tarball` as a build does before copying it, so it refuses the
tarballs a build refuses, including one whose contents cannot be read.
The directory must be absent or empty, as for the export; the build never
replaces a folder. A refusal before the write leaves nothing behind. A capture
Evidence file, an inspected HAR's Evidence and the projected manifest must each
fit the live heap headroom for parsing and sealing, checked before the file is
read or the Evidence serialized; one that does not is refused as
`resource_constraint`. Each
refusal names its stage: `environment-build` for the build's own checks,
`inspect_web_network_capture` and `inspect_analysis_view` for inspection and
projection, `environment-export` for the bundle. SIGINT cancels the build
(`cancelled`); before the bundle stage nothing is written, and a cancellation
during the write names the bundle it wrote. Stage progress is logged to stderr
at info level; the report is the only output on stdout:
```json
{
"status": "built",
"directory": "/work/grafana",
"name": "grafana",
"capture": {
"evidence_id": "ev_…",
"operation": "inspect_web_network_capture",
"records": 105,
"input": "har"
},
"manifest": {
"evidence_id": "ev_…",
"tools": 48,
"excluded_groups": 0,
"template_rules": [
"after-uid-literal",
"opaque-token",
"sibling-variance",
"prefix-variance"
]
},
"state": { "served": true, "evidence_id": "ev_…" },
"coverage": {
"reads": {
"state": 36,
"recorded": 6,
"recorded_reasons": { "query-varies": 3, "…": 1 }
},
"writes": {
"served": [{ "tool": "delete_folders_by_uid", "effect": "delete" }],
"withheld": [
{
"tool": "post_folders",
"effect": "create",
"reason": "unconfirmed",
"hint": "No later read showed the effect. Read the entity and its list after the write.",
"exemplar_ordinal": 94,
"contradicted_by": null
}
]
}
},
"tool_count": 37,
"package_source": { "kind": "registry", "specifier": "rea-agents@…" },
"files": [{ "path": "README.md", "bytes": 4210 }],
"warnings": [{ "code": "name-from-file", "message": "…" }],
"next": ["agent-env run /work/grafana"]
}
```
`coverage` is the state's read views and write plans summarized, and `null`
when the state is unavailable and the bundle serves replay. The
[environment coverage](guides/web/web-network-captures.md#environment-coverage) view
reports the same summary for any capture without building, so an agent can
check it after each recording. The view summarizes the state as derived; it
does not apply the serve-time admission checks, so for an environment whose
served catalog exceeds the discovery budget, or whose state exceeds what
`data/add` loads, it still reports the state while the build reports `coverage:
null` and the bundle serves replay. Each withheld write carries a hint about
what to record next; hints never change what is served:
| Reason | Hint |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `no-state-target` | The write could not be linked to a modelled target. Check the path template and recorded views; record missing reads. Some route forms may not be supported. |
| `no-call` | No recorded call of this tool was modelled, for example because every call failed. Record a successful call. |
| `body-unusable` | The retained request or response cannot be modelled. Record a successful call with a complete supported request and reproducible response. |
| `no-observation` | Nothing showed the entity or document this write changed. Read a created entity back by id right after the write; for an update or delete the state must already hold the entity, so read it before the write or record the create that made it. |
| `unexplained-change` | The write changed fields it did not send, in a way no rule explains. A second, different call of the same tool can show the pattern. |
| `contradicted` | A later read disagreed with the modelled effect. Check the read at contradicted_by. |
| `side-effect` | The write changed something besides its own entity, which the environment does not model. |
| `unconfirmed` | No later read showed the effect. Read the entity and its list after the write. |
| `incomplete-entities` | The write is confirmed, but the views that show its entity cannot render it in full. Read the written entity through every view that lists it. |
| `unknown-clock` | The learned rules generate timestamps, but the capture establishes no global served clock start. Record usable start times and completed durations for every request. |
The bundle ships `evidence.json`, the whole capture Evidence, response bodies
included. Inspection excludes the values of the standard authentication headers
(`Authorization`, `Cookie` and `Set-Cookie` among them) and URL userinfo; a
custom key header such as `X-Api-Key`, a token in the query string and tokens
an application returns in bodies remain. The environment views refuse captures
inspected with `--sensitive-value`, so scrub a capture before building it. The
report's `credential-named-members` warning counts recorded header names, query
parameter names and JSON or form body members whose names look like
credentials (whole name segments such as `token`, `secret`, `password`,
`api_key`, `authorization` or `session`) and hold a value, and lists the first
20 by record and pointer, never by value. A body is read as a form or as JSON by
its recorded media type, or by its shape when it has none or one naming
neither. A browser scenario is
scanned by event (`event 4 response /token` names
`normalized_result.events.items[4]`): every retained request and response
content event, including a response whose request began before the browser was
attached and a transaction no record represents, and each transaction URL's
query once. Bodies larger than 4 MiB of UTF-8 text are not
scanned; `credential-scan-incomplete` counts them and lists up to 20 record or
event pointers, including base64 bodies whose decoded scan was cut. An empty name
warning does not establish that these bodies were checked. A browser scenario's
saved values (`save` and `save_response`) named like credentials but not marked
`secret: true` are recorded unredacted; `credential-named-saved-values` lists up
to 20 of them by parameter pointer. Names do not establish sensitivity,
so the build never refuses on them. Capture
Evidence also records the capture file's absolute path (`subject.local_path`,
`parameters.capture_path`, `locations`, `artifact.path`), and the bundle
carries it: build from a neutral folder when that path should not ship.
Over MCP, `export_environment_bundle` builds the same bundle and returns the
same report. Its `source` is capture Evidence the session retained
(`{"kind": "retained-evidence", "evidence_id"}`), such as a
`capture_browser_scenario` or `inspect_web_network_capture` result, or a
capture Evidence file (`{"kind": "evidence-file", "path"}`); inspect a HAR with
`inspect_web_network_capture` first. Retained Evidence is written into the
bundle as `--format json` writes it, so the bundle is the one
`environment-build` writes from that file. `output_directory` must be an
absolute path that does not exist yet, whose parent does; the tool creates it
and refuses an existing directory, empty or not. `name`, `template_rules`,
`package_tarball` and `dry_run` work as `--name`, `--template-rule`,
`--package-tarball` and `--dry-run` do, and retained Evidence has no file to
name the environment after, so a loopback or IP origin needs `name`. Unlike
`--dry-run`, `dry_run` still checks `output_directory`. Destination refusals
name `export_environment_bundle`; the others name their stage as
`environment-build`'s do.
## From a browser scenario
The capture can also come from REA itself: a
[browser scenario](guides/web/browser-scenario-contract.md) records a scripted session
[browser scenario](browser-scenario-contract.md) records a scripted session
in a browser REA launches, and its Evidence is a manifest parent like a HAR
inspection. Select the `network` event family and response bodies. A replayed
reply carries up to 256 KiB of a recorded body, but the scenario keeps at most
@@ -320,51 +125,6 @@ the replayed body is the bytes the browser exposes for each response, with any
declared secret replaced by its redaction marker (a body that redaction had to
empty replays no body).
To record writes, sign in through the UI and then send the application's API
calls with [request steps](guides/web/browser-scenario-contract.md#request-steps): each
sends one request to the start origin through the page's own `fetch`, with the
session's cookies, and a header can carry the CSRF cookie or stored token the
application's own requests send. A write is served only when the capture shows
its effect, so read each collection before writing, read every written entity
back by id and in its list right after the write, and leave more than a second
between a create and its first edit. A request step's path is fixed when the
scenario is written and cannot use a value an earlier response returned, so an
id must be known before the run: one the caller chooses, such as a `uid` in the
create body, or one an earlier scenario created. When the server assigns ids,
read the list after each create (the create's response or the list shows the
new entity), and record updates and deletes in a second scenario that reads
the first one's objects by id before editing them, or make them through the
UI. The environment coverage view names each
write it withholds and what to record next. A scenario with write requests
changes the application's data: record against an instance whose data may
change.
```json
{
"step_id": "create-folder-1",
"action": "request",
"method": "POST",
"path": "/api/folders",
"headers": [
{
"name": "X-CSRFToken",
"value": { "source": "cookie", "name": "csrftoken" }
}
],
"json": { "title": "Scratch folder 1" },
"expected_status": [200, 201]
}
```
Each record takes its start and duration from the browser's timing (the time
the scenario observed the request where the browser reported no start), so the
state orders and overlaps
records as it does a HAR's and learns a write clock: an update the application
stamps with a modification time can be served. A scenario captured before
these times were recorded has none; its records keep request order and the
state learns no clock, so such updates stay `unexplained-change` until it is
recorded again.
## What a manifest is
An environment manifest is a projection of one application's recorded request
@@ -406,8 +166,7 @@ returns that record's recorded response. Selection compares the supplied path
parameters against the record's path segments at the parameter indexes, then
the supplied query values against the record's query string, and keeps the
record with the most matches; ties keep manifest order. A query argument accepts
one string, a number or boolean sent as its JSON text, or an array of strings for
repeated values; the array is matched
one string or an array of strings for repeated values; the array is matched
against every occurrence in captured order. The raw recorded query remains the
replay evidence. The result reports
where the answer came from:
@@ -481,7 +240,7 @@ captured text without changing the tool names or schema fields clients use.
Replay answers each call from one recording, so the answers reflect whichever
moment of the capture recorded them. The
[environment state](guides/web/web-network-captures.md#environment-state) view extracts
[environment state](web-network-captures.md#environment-state) view extracts
instead the data the capture's reads observed before the first related write:
entity types with an identity field, entities with their recorded values,
documents for reads that are not entities, and a render plan per read tool,
@@ -503,14 +262,14 @@ see [Serve from the state](#serve-from-the-state).
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the
environment, then to `127.0.0.1` and an ephemeral port, exactly as
[`rea mcp --http`](getting-started/cli.md#serve-mcp-over-streamable-http) resolves them. The
[`rea mcp --http`](cli.md#serve-mcp-over-streamable-http) resolves them. The
bound MCP URL is written to stderr as one JSON log line
(`"msg":"MCP transport listening"` with `url`); `REA_LOG_LEVEL` overrides that
log level. `ENVIRONMENT_NAME` renames the served environment; otherwise the
manifest's `name` is used, and a set but blank value is refused as
`configuration_invalid` before binding rather than silently falling back. The
listener serves one MCP client at a time with the same session, `Host` header
and shutdown contract as [transports](reference/mcp-contracts.md#transports) describes.
and shutdown contract as [transports](mcp-contracts.md#transports) describes.
It runs until SIGINT or SIGTERM, then closes the listener, prints one
`{"status":"stopped", …}` document whose `state` member says whether it served
from the state, and exits `130` or `143`.
@@ -585,7 +344,7 @@ names the method and path template, then what the tool returns, one level deep:
`returns a JSON object with fields …`, `returns a JSON array of objects with
fields …` or `returns a JSON object whose <member> lists objects with fields …`,
where a list is an array of objects found by the
[environment state](guides/web/web-network-captures.md#environment-state)'s list rule (the
[environment state](web-network-captures.md#environment-state)'s list rule (the
root array, or the one array of objects an envelope carries). Field and member
names are written as JSON strings, so a captured name cannot read as more of
the description; a member name too long to show whole is a quoted prefix marked
@@ -614,9 +373,8 @@ shape and carries no annotations.
Path parameters are required strings whose
description names the segment index, the templating rule and the observed
values (sampled within the list budget above; the manifest keeps them all);
query parameter names accept an optional string, number or boolean (a number or
boolean is sent as its JSON text) or array of strings (for repeated values in
captured order); write methods accept an
query parameter names accept an optional string or array of strings (for
repeated values in captured order); write methods accept an
optional `body` object described by the recorded top-level key types. Each
input keeps its own argument: a query name equal to a path parameter's name is
advertised as `query_<name>`, a query name longer than 64 characters as
@@ -641,7 +399,7 @@ rea environment-serve manifest.json --evidence capture.json
```
Before it binds, the server derives the
[environment state](guides/web/web-network-captures.md#environment-state) the capture
[environment state](web-network-captures.md#environment-state) the capture
observed: it runs the `environment-state` view on the capture in-process with
the manifest's `template_rules` in the same order, exactly as
`inspect-analysis-view` projects it from the same capture, so the state
@@ -675,7 +433,7 @@ Each read tool answers according to its view in the state:
| View | Answer |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`, whose `per_response_members` (such as a request id), like a list's or a document's, keep their recorded values. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`, except that a call matching one of the view's `projected_query_keys`, or on a view with `projected_record_ordinals` asking for some fields only with a value other than `*` no checked line gave or one a replayed read gave, replays its recording. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error, unless a read showed the application leaves that field out; a body is never invented. |
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`, except that a call matching one of the view's `projected_query_keys`, or on a view with `projected_record_ordinals` asking for some fields only with a value other than `*` no checked line gave or one a replayed read gave, replays its recording. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error, unless a read showed the application leaves that field out; a body is never invented. |
| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"` or a line `all_record_ordinals` names, or the recorded members still held. A call with no query argument answers from such a line of the collection its path arguments name that only pages its list, with `replay.exact_match` false. On a line the view's `query` answers, and for a call whose every query argument has a role there (an argument-less call aside), the answer is the query evaluated over the state: filtered, searched, sorted and paged, with counters at the number listed and `query.totals` at the number matched, and `state.membership: "query"`; such a call answers on a query or whole line of its collection with `replay.exact_match` false. A value an `inert` parameter never recorded, a parameter without a role, a call matching one of the view's `projected_query_keys`, or a field selection the checked lines do not show (as for `item`), replays its recording. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
| `recorded` | The recording, as replay answers it. |
@@ -693,21 +451,6 @@ An item or list view that always spells a clock field otherwise than the
entity's stored value names that spelling in `clock_formats`, and a value it
never recorded, such as a created entity's clock, is rendered in it: a list that
writes `+00:00` where item reads write `Z` lists a new entity the same way.
A view that names a field in `sorted_arrays` lists that string array sorted, as
a read-back showed the application does, while the state keeps it in the order
it was written, so a write's own answer echoes it. If a later counted read
contradicts that ordering, the view stays recorded and its earlier reads no
longer confirm served writes. A view that names a field in `nested_fields`
always showed that field's objects with the same keys, in any order, and fewer
than an entity stores, such
as a list showing a parent without the parent's own nested type; a stored
object it never recorded, such as a created entity's parent, is rendered with
those keys. Members a view names in
`per_response_members`, such as a request id each response mints, keep the
value its recording carried. A list's `counters` and `query.totals` are
top-level members or members one level down in a paging object
(`/pageInfo/totalRows`); either is set to the number listed or matched, and
members beside it such as `isLastPage` stay as recorded.
A list or document call that names no checked line, including a line recorded
after a write or with other query values, answers from the recording. Every
@@ -723,8 +466,6 @@ and `POST` tools the capture shows only reading (a lookup answering as a `GET`
did, an RPC tunnel naming read operations, a session renewal), stay served and
answer from their recordings with the reason `query-body`; they are not write
tools.
`HEAD`, `OPTIONS` and `TRACE` tools also replay their captured responses; their
state views report `method-replay` and coverage counts them as recorded reads.
A line establishes whole-collection membership only from the beginning:
`offset`, `start`, `_start` and `page` must be zero when present. Positive or
@@ -733,64 +474,35 @@ query model may describe supported paging; an empty later page cannot start
returning earlier entities merely because it was empty in the capture.
`POST`, `PUT`, `PATCH` and `DELETE` tools are served only when the state's
[write plan](guides/web/web-network-captures.md#environment-state) for them is served:
[write plan](web-network-captures.md#environment-state) for them is served:
the capture's later reads confirmed the effect the plan models. Other write
tools are not served, so no call claims a change it does not make; each one's
plan states why in its `reason`. A served write changes the live state, which
every MCP session and `data/get` then see:
| Effect | Call |
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, nested clock, increment, slug, stamp, derived, composed, reference, unique and renewed changes the plan models. |
| `update` (action) | A `POST`, `PUT` or `PATCH` to an item path plus an action segment (`/{type}/{id}/archive`) sets, on the entity before the segment, each scalar the recorded calls showed the action set (`set`), or flips each boolean they showed it flipping both ways (`toggle`), with any sent members; it is served only when a recorded call found what it sets already set, or flipped back what it toggles. |
| `delete` | Removes the entity the path names. |
| bulk `update` or `delete` | A `DELETE`, `PUT` or `PATCH` at a collection, or a `POST` to it plus `delete`, `remove` or `destroy`, applies its effect to every entity the body names, as recorded: an array of ids or of objects carrying the id field, at the root or as the body's one array member (`{"IDs": […], "Read": true}`). |
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
An update's unsent changes follow the plan's rules. A `held-clock` is a clock
the client sends back as it read it and the application stamps anew (Ghost's
`updated_at`): the call must send the entity's value and the field takes the
write's clock. A `nested-clock` sets the clocks inside an object the entity
holds, in that object's recorded format. A `stamp` writes the acting user
the entity already names under its `created…by` counterpart into a
`modified…by` or `updated…by` field. A `derived` field takes a value showing
the one sent member's text that the entity stores nowhere (a field Ghost
derives from `html`), only with that member's recorded value. An action's
`set` rule sets a scalar to the value the recorded calls left it at, and its
`toggle` rule flips a boolean, refusing a call on an entity that holds none
there (Miniflux's `PUT /v1/entries/{id}/bookmark` stars and unstars).
| Effect | Call |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, increment, slug, unique and renewed changes the plan models. |
| `delete` | Removes the entity the path names. |
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
Its answer has the status and content type of the plan's recorded response,
with the written entity or its identity rendered in, no body, or the recorded
body. A call whose body is not an object, sends a member the capture did not
show stored, sends a fixed member with another value or a member of another
JSON type, or lacks a required member is a tool error naming the accepted
members; so is a call sending query parameters other than the exemplar's (they
may change what the body does), a body not wrapped in the exemplar's envelope
(Ghost's `{"posts": [{…}]}`), a bulk call naming an id the state does not hold,
a `held-clock` member sent with another value than the entity holds, a call
sending a `derived` rule's member to an entity that does not hold the value the
recorded call found, a nested clock on an entity holding no such object, and a
rendered answer missing a field the recorded response carried, and an update or delete of an entity the state does not hold answers
members; so is a rendered answer missing a field the recorded response
carried, and an update or delete of an entity the state does not hold answers
the recorded 400, 404 or 410 of the tool or of an item view of the type, if
any. A served delete of a type whose item view has a `deleted_flag` (the
member, `deleted`, `isDeleted` or `is_deleted`, its reads set `true` on a
record the application soft-deleted, as EspoCRM does) keeps the entity until
`data/reset` or `data/add`, within the size `data/add` loads, so a read of it by
id answers its last fields with that flag `true`. A refused call leaves the
state unchanged. Values a write generates are
any. A refused call leaves the state unchanged. Values a write generates are
deterministic: the clock starts at `write_clock` and advances one second per
served write, sequences take one past both the largest held value and the
largest they generated, and UUIDs and tokens derive from the state Evidence
id, tool, field and write count, skipping held values. If `write_clock` is null,
plans needing `clock`, `held-clock`, or `nested-clock` retain their learned rules
but are withheld with `unknown-clock`; other confirmed plans can still be served.
`data/reset` and
id, tool, field and write count, skipping held values. `data/reset` and
`data/add` restart the clock, counter and sequences, so the same calls after a
reset give the same state. Answers carry `state: {"source": "state", "view":
"create" | "update" | "delete", …}` or `"document-write"`; a bulk answer adds
`ids`, every entity it wrote in body order, with `id` naming the first.
"create" | "update" | "delete", …}` or `"document-write"`.
The data plane serves three JSON-RPC methods:
@@ -890,9 +602,8 @@ names a `document` view once, at its initial document's `path`. The `version`,
and the `state_evidence_id` and `types` that `data/get` reports, must match
the served state, so a `data/get` result loads back unchanged. A number without
an exact native value, or whose serialized spelling would not be read as exact
(an integer from 2^53 below 1e21, such as `1e20`, in any spelling), is refused
rather than altered. Caller data preserves all own JSON members, including
`__proto__`; unknown members of the state document's structure are refused.
(an integer from 2^53 below 1e21, such as `1e20`, in any spelling), or a
`__proto__` member is refused rather than altered.
An invalid document is `-32602` naming the first problem by JSON pointer, as in
`/rea_state/entities/3/collection` (also in `error.data.path`); a problem
inside caller data (`fields`, `value`, `types`) is located at that member, so
@@ -946,25 +657,6 @@ after an update and referential rules on delete are not modelled. An exported
bundle carries no state file: its server derives the state at startup, as the
export does to predict what it serves.
Known limits of served writes:
- A recorded view answers from its recordings after a served write: Mailpit's
`get_messages`, recorded as `list-differs`, still lists a message a served
delete removed.
- Served n8n workflow updates accept any `versionId`, while served Ghost post
updates require `updated_at` to match the entity (a `held-clock`).
- A slug derived from a title can repeat across served creates with the same
title, where the application would make it unique.
- A `composed` string copies sent text as sent: an application that escapes or
renders it (HTML entities, Markdown) differs for text that needs it, as
BookStack's `description_html` would for a description holding `&`.
- A served create's sequence id is the state's next value, not the gaps an
application's own sequence would leave.
- Nested clocks are served in the spelling of the write's own response, which
can differ from what a later read of the application shows.
- A served write to an entity that a replayed (unserved) create made answers
as for a missing entity, since the state never held it.
## Export a bundle
`rea environment-export` writes the same pair as a bundle folder the agentenv
@@ -1001,8 +693,7 @@ serves replay. The complete tool catalog the bundle serves must also fit the
server's advertised catalog budget before any files are written. Privately parsed JSON is sealed once so authentication
and replay share immutable response trees rather than copying the capture repeatedly.
`--name` replaces the manifest's portable kebab-case name as the env folder and env id
(at most 63 characters for its DNS service label; Windows device names and
labels interpreted as IP addresses are refused);
(Windows device names such as `con` and `lpt1` are refused);
`--package-tarball <path>` copies a packed `rea-agents` tarball into the env
folder and makes the Dockerfile install it instead of the registry package and
remove it in the same layer, which is how a checkout rather than a release is
@@ -1020,12 +711,7 @@ Windows inherits ACLs and REA does not establish owner-only access there. Choose
an already private Windows destination when captured content needs it.
The Dockerfile sets `MCP_HOST=0.0.0.0` and `MCP_PORT=18765`, the port the
framework maps for an MCP server env, and `MCP_ALLOWED_HOSTS` naming the loopback
addresses and the environment name, because the container binds outside loopback
and the server refuses to serve on such a bind without an operator-declared
allowlist: the framework's published port is reached over loopback and its
gateway proxies to the env container by the compose service name (the
environment name). The framework's own deploy supplies
framework maps for an MCP server env; the framework's own deploy supplies
`MCP_HOST`, `MCP_PORT` and `ENVIRONMENT_NAME`, so the served card carries the
env's name. It installs Debian's `python3` because the framework's local
sandbox probes a container's readiness with `python3` inside it, which the slim
@@ -1049,9 +735,8 @@ unknown recorded status stays unknown; it is not borrowed from another
record). A bundle with no such tool yields a verifier that checks the count
only and says so. `tasks/smoke.json` deploys the env on the local
sandbox and runs that verifier against the env's MCP endpoint; the first run
also pulls the framework's published infrastructure images from `ghcr.io`,
building one only if its pull fails, and the run tears everything down when it
ends.
also builds the framework's own infrastructure, and the run tears everything
down when it ends.
The README written into the bundle records the capture artifact's SHA-256, the
capture and manifest Evidence ids, the manifest view digest, the served tool
@@ -1099,11 +784,9 @@ Numeric arrays retain that containing field's clock context through nested
arrays; an object's own fields establish their own context. Numeric values at
the body root, in root arrays, or under measurement fields remain literal.
Generated values normalize only where this confirmed, served write's entity
response rules identify generation. A UUID identity rule must still compare two
UUIDs, and a `unique` rule two UUIDs or two generated integers (10 to 20
digits) of one JSON kind and length; a `renewed` rule applies only at its
recorded pointers,
Generated strings normalize only where this confirmed, served write's entity
response rules identify generation. A `unique` rule or UUID identity rule must
still compare two UUIDs; a `renewed` rule applies only at its recorded pointers,
with `*` matching actual array items. Token/sequence strings retain the existing
format check: equal length of at least eight, the same character classes, and
a digit. Numeric sequences continue to compare literally unless an ID mapping
@@ -1115,12 +798,6 @@ Admitted generated values must also preserve aliases and distinct identities
within that response. UUID identity uses the generator's case-insensitive
matching; token and sequence strings retain their literal identity. This
correspondence ends with the answer and is not inferred across calls.
A `composed` response field must spell the value its parts build from that
response's own fields. Its parts then compare by their rendered text or their
source fields' clock and ID rules. A coherent archive URL can therefore differ
only in normalized clock digits without reporting a value difference; an
incoherent URL still differs even when both sides repeat the same stale text.
Normalized compositions count under `generated`; source fields are counted once.
The report counts each kind of normalised
value; the generation rules remain model-derived facts, rather than new
observations of the application.
@@ -1150,12 +827,7 @@ global conflict involving an equal pair is not used to normalize either pair; th
A step agrees when the statuses and the normalised bodies are equal. The report gives agreement overall, per probe and
per tool, `agreement_ignoring_order` for steps that differ only in item order, and each step's differences, at most 20,
with a JSON pointer and both values cut short. Each step reports `differences_total` before that cap,
`differences_omitted`, and `differences_truncated` (true when entries were omitted). Agreement and categories use the
complete comparison. Omitted entries are not retained, so `--full-output` cannot recover them; inspect the original
capture and probe Evidence to investigate beyond the returned pointers.
The difference categories are:
with a JSON pointer and both values cut short:
| Category | Meaning |
| --------------- | ------------------------------------------------------------------------------------------------------------ |
@@ -1187,7 +859,7 @@ read, not that the write had no other effect.
## Compatibility verification
`npm run verify:agentenv:environment` is the hosted `real-agentenv` lane
described in [testing](development/testing.md#exported-environment-bundles). It projects
described in [testing](testing.md#exported-environment-bundles). It projects
the Grafana-shaped source-owned capture through the built CLI, packs the
checkout, exports a bundle with `--package-tarball`, runs it with the pinned
`agentenv-framework` release on the runner's Docker and requires the smoke task
@@ -6,12 +6,12 @@ kept as an unmodified, commit-pinned Git submodule; see
application workflow and return artifact-bound Evidence inline.
This family is present on repository main and in npm 4.1.0. Check the
[released package boundary](../../getting-started/installation.md#released-package-and-main) and the
[released package boundary](installation.md#released-package-and-main) and the
connected server's tool list before selecting it.
On Linux, JADX launches an REA-owned process and requires a procps-compatible
`ps` on REA's `PATH`; see the shared
[Linux process prerequisite](../../getting-started/installation.md#linux-owned-process-prerequisite).
[Linux process prerequisite](installation.md#linux-owned-process-prerequisite).
## Optional package delivery
@@ -30,118 +30,10 @@ code units; follow `coverage.next_offset`. The page preserves `source_status` an
`reported_total_bytes`: exhausting retained text does not make a partial decoded
manifest complete. Complete source Evidence remains exportable under the existing
session lifetime. Servers without retention reject summary before starting JADX.
One-shot CLI output remains complete by default. Select `--detail summary` with
`--evidence_output /local/capture.json` to retain complete Evidence before returning
a linked view. Older servers require the existing complete request after schema discovery. APK authenticity and runtime behavior remain
One-shot CLI output remains complete. Older servers require the existing complete
request after schema discovery. APK authenticity and runtime behavior remain
unverified.
## Manifest component projection
`inspect_android_package` also returns a static inventory of declared
activities, aliases, services, receivers and providers from the decoded
manifest. The binary XML decoder preserves component class spelling, including
`$` in nested classes, without applying JADX class aliases. `declared_name`
preserves the decoded source attribute; `name` resolves a
relative name only when the manifest package is known and the resolved copy
fits the UTF-8 byte budget. An unresolved package or exhausted name budget
retains the declared name and marks the inventory partial. Empty component
names and names with forbidden DEX characters retain their raw spelling and
are not treated as identities. Valid DEX spaces, including boundary spaces,
are preserved when resolving component names and matching alias targets.
Whitespace in the manifest package is rejected by its separate package grammar.
Component permissions inherit the application
permission when no component permission is declared, except an activity alias
with no permission: Android documents that case as requiring no permission
through the alias. `permission_source` distinguishes a local declaration,
inherited value, no requirement (including an explicit empty override), and a
value omitted because its resource/manifest-placeholder token is unresolved
or the total permission-byte budget was reached. Resource and placeholder
references are not expanded at this parser boundary. Repeated inherited
values share one inventory-wide byte budget.
Provider `android:readPermission` and `android:writePermission` overrides are
not projected as separate fields; their presence marks the inventory partial
and is called out in `limitations`. Relative component names are not resolved
against a manifest package with surrounding whitespace.
Class-name segments follow DEX `SimpleName` characters, including a leading
digit or hyphen, rather than Java source identifier rules. Relative-name
expansion does not prove class existence, DEX-version compatibility or runtime
loadability. SDK defaults accept JADX's decimal and hexadecimal integer
spellings; the original SDK declaration strings remain unchanged.
`exported` is the declared literal `true` or `false`, otherwise null.
`exported_default` is the documented default only where it can be established;
it remains null for unknown provider target SDKs, nonliteral declarations, or
components with filters targeting API 31 or later that omit the now-required
explicit declaration. A limitation records the latter validation error.
As in Android's package parser, an activity, alias or receiver filter without
an `<action>` is discarded and does not count toward either rule; a service
keeps such a filter. `intent_filter_count` still counts every declared filter.
Provider authorities and all inventory dimensions are bounded. `status:
"partial"` and `limitations` identify omitted declarations or values; a
capacity limitation appears only when an actual collection limit was reached.
The projection does not merge manifests, expand resource references, or
observe runtime enabled/export state.
`enabled` and `grant_uri_permissions` report only literal component-level
`android:enabled` and provider-level `android:grantUriPermissions` declarations.
They are `null` when absent or not a literal boolean; missing attributes are not
filled from application defaults. A nonliteral declaration gets a limitation.
`process` preserves the exact component-level `android:process` spelling (including
empty or symbolic values), or `null` when absent or omitted under the inventory-wide
1 MiB copy budget. Neither field resolves resources, merges manifest values or
establishes the effective runtime process or enabled state. The exact boolean
attributes are retained in `enabled_declaration` and
`grant_uri_permissions_declaration`, including resource references.
`intent_filters` preserves each filter's source ordinal, exact priority spelling,
actions, categories and individual data declarations (scheme, host, port, path,
path prefix/pattern/advanced pattern/suffix and MIME type). These are declarations,
not computed Android matching rules. `meta_data` lists names; values remain in the
complete decoded manifest. Component, filter, data and metadata `location` values
use one-based lines/columns and UTF-16 offsets into that manifest. A component's
`declaration_index` preserves its original declaration ordinal, including gaps.
`declaration_coverage` distinguishes complete, partial and unknown observations.
Historical Evidence omits newly introduced fields and defaults coverage to
`unknown`; omission never establishes absence. Current records contain the fields,
with null for absent or unresolved scalar declarations. The inventory retains at
most 32 filters and 64 nested action/category/data/metadata records per component,
and shares a 1 MiB structured-value copy budget across components. Omitted values
or unsupported filter constructs mark coverage partial and include a limitation;
complete retained manifest text remains accessible.
For a small answer, select `detail: "summary"` with an explicit view:
```json
{
"kind": "android",
"facet": "components",
"actions": ["android.provider.Telephony.SMS_RECEIVED"],
"fields": ["name", "permission", "enabled", "process", "intent_filters"],
"offset": 0,
"limit": 16
}
```
The same JSON works with CLI `--view` or retained `inspect_analysis_view`.
`kinds`, `names` and `actions` use exact matching; names accept either the recorded
declared or resolved name, actions match any requested action, and different
filters combine with AND. `exported` matches only explicit declarations, keeping
`exported_default` separate. Empty selectors match nothing. Every selected row
retains its parent-array `source_index` and `declaration_coverage`; source coverage,
parent identity and limitations remain alongside the page. Follow
`coverage.next_offset`. The 1 MiB projection budget can shorten a page; a single
oversized row requires fewer selected fields. Exhausting a partial source does
not establish complete manifest coverage. `facet: "component-summary"` applies
the same filters and counts kinds, declaration coverage and explicit/default
exported values separately. Both adapters use the same retained projection;
repeated pages do not reload JADX.
The service `foreground_service_type` field preserves the declared attribute,
including an empty value or surrounding spaces; `null` means the attribute is
absent or the component is not a service. This field does not validate service
type tokens or establish runtime eligibility.
## Supply tools
Use a **full JDK 17 or newer** (including its compiler module) and
@@ -255,50 +147,6 @@ queries accept the same `overload_index`; the bridge resolves the selected
method node and the result reports its index and displayed signature. These
indices are tied to the current artifact and engine, not stable across builds.
Method reference queries also accept an external class, such as
`android.app.NotificationManager --method-name notify`, when the APK directly
invokes that exact DEX class and method name. If multiple prototypes are observed,
the input error lists their exact DEX descriptors and local overload indices;
repeat with an explicit `--overload-index`. These are the prototypes observed in
the APK, not the complete method inventory of an Android framework or library.
Such results report `target_definition: "external"` and
`reference_scope: "direct_dex_invokes"`. Each using method has `source_selector`
with `class_name`, `method_name` and `overload_index`; pass those fields with the
same APK path to `inspect_android_method` to inspect the caller. Repeated direct
invokes in one method remain one using node with multiple instruction offsets.
This scans raw invoke instructions in JADX's admitted APK methods without
generating their source. It does not resolve reflection, JNI, method handles,
inherited method lookup or possible runtime dispatch. Class-only queries and
class/method inspection still require an APK definition; an external target has
no invented body. Unknown class/method names remain input errors.
External prototype and caller metadata use a bridge-frame-derived representation
allowance that reserves space for retained raw calls and CLI/MCP copies. Exhausting
it returns a typed `resource_constraint`, rather than a truncated caller list
reported as complete. Direct instruction offsets have their separate explicit
`direct_invokes_exhaustive` coverage. Older retained Evidence may omit the new
definition, scope and caller-selector fields; omission does not establish them.
The bridge checks prototype expansion before copying descriptors or joining
displayed signatures. It reuses one bounded prototype inventory for the exact
class/method selection in the current APK session; replacing the session or
selection invalidates that retained result.
`inspect_android_class` also reports `dex_static_value` for each `static final`
field whose class definition encodes a value in its DEX static values, so
compile-time constants are readable even when the class has no `<clinit>`.
Each value is typed: `boolean`, `byte`, `short`, `char` (UTF-16 code unit) and
`int` values are JSON booleans or numbers, while `long`, `float` and `double`
values are strings holding Java's text for the value, so 64-bit values stay
exact. The field reports `null` when no value is encoded, the encoded value is
null or of another kind, or the field is not `static final`. The value is read from parsed
metadata without generating source, and a static initializer can still assign
the field; use `inspect_android_method` on `<clinit>` for that evidence.
Only `inspect_android_class` asks the bridge for these values.
`inspect_android_method` and `trace_android_references` read the member-only
class summary, so large encoded strings cannot push their member lookup past
the 8 MiB protocol frame limit.
Smali fallbacks select one complete block using the original method name and DEX
descriptor, before applying the source byte limit. This preserves the selected
overload even when Java code generation renames a method. If the selected Java
@@ -320,162 +168,6 @@ instructions into field declarations, leaving an empty or incomplete method
body. The selected method's disassembly preserves those instructions and retains
the same source byte limit and explicit truncation reporting.
## Apply an R8/ProGuard mapping
Release builds shrunk by R8 or ProGuard rename classes and members. Select the
build's `mapping.txt` with `mapping_path` on `inspect_android_class` or
`inspect_android_method`, or `--mapping-path` on the matching CLI command:
```sh
rea inspect-android-class /path/Example.apk a.b --mapping-path /path/mapping.txt \
--detail summary --evidence_output /tmp/class-evidence.json
```
For agent workflows, select `detail: "summary"` on class and method inspection.
It preserves the complete normalized member inventory or selected method body,
including mapping annotations, while retaining duplicate raw provider calls in
the complete parent Evidence. The reply links that parent's identity and digest;
export it with `export_evidence_bundle` or read its summary again through
`inspect_analysis_view`. CLI summaries require `--evidence_output` and return
the portable bundle path. Omitted or `complete` detail preserves complete inline
Evidence. Search and reference operations retain their existing contracts.
Symbol summary views admit at most **1 MiB**, including view context. An admitted
method's source can already consume that amount; a larger combined summary
fails explicitly instead of silently truncating facts. MCP refusal includes
the retained parent ID. Select complete detail, or inspect one method rather
than a whole class, when the requested summary does not fit.
The mapping path must be absolute. Selectors still take the observed names, such as
`a.b`, and observed rows are unchanged. The result adds one `mapping` block with
the file's path, SHA-256 and size, the header values the file reports
(`compiler`, `compiler_version`, `min_api`, `map_version`, `pg_map_id`,
`pg_map_hash`), valid record counts and up to 16 malformed line samples. Its
`symbols` list has one entry per observed class, inner class, method and field;
`inspect_android_method` lists the class and the selected method. `coverage`
counts each resolution status. With summary delivery, these unchanged facts are
under the view's `summary` field; the view separately reports source coverage.
Lookups use the bridge's raw DEX names, not JADX display aliases such as a
renamed reserved word. Each symbol resolves as:
- `mapped`: exactly one record matches. A class needs one record for its raw
name (`verified_by: "class"`). A method must also match its DEX prototype
(`"descriptor"`) and a field its DEX type (`"type"`). R8 residual signature
metadata is used when present; otherwise original types are translated
through the file's class records.
- `ambiguous`: `multiple-records`, `descriptor-mismatch`, `type-mismatch`,
`ambiguous-class` or `residual-signature-conflict`, with `candidate_count` and
up to 8 candidates. Conflicting residual metadata never selects a winner. A name match
whose prototype or type does not verify is never reported as mapped.
- `unmapped`: no valid record matched the raw identity.
- `unknown` with reason `projection-budget`: this claim did not fit the combined
reply budget. The observed raw symbol remains present; `coverage.unknown`
counts these omissions. Inspect that specific class or method separately,
or consult the selected local mapping file for complete source records.
A line continues an inline frame stack when it repeats the previous line's
minified range and obfuscated name and gives a single call-site line, such as
`:7`. Only the outermost original of a stack becomes a candidate. Original
line numbers and source files are not projected. Malformed records are counted
and skipped, so an `unmapped` symbol might be described by one of them. A file
that is missing, not a regular file, not UTF-8, larger than the byte limit or
without any class record, such as `usage.txt` or `seeds.txt`, is refused before
JADX is used. Requests read their mappings one at a time, in the same queue as
the JADX work and under the same operation deadline. A mapping failure therefore never starts or retires the JADX
session, and a different mapping reuses the loaded APK.
## Link native declarations to JNI candidates
When advertised, `inspect_android_jni_bindings` links original `ACC_NATIVE`
declarations from standard root `classes*.dex` members to bundled ELF libraries.
The supplied JAR's DEX reader preserves duplicate declarations before decompiler
deduplication. This operation does not load the APK into the decompiler.
MCP defaults to retained summary delivery when `detail` is omitted; select
`complete` explicitly to receive the full report inline.
```json
{
"path": "/absolute/Example.apk",
"class_name": "org.conscrypt.NativeCrypto",
"method_names": ["EVP_PKEY_new_RSA"],
"abis": ["arm64-v8a"],
"detail": "summary"
}
```
Class/method names and optional `descriptors` select raw DEX identities.
`library_paths` selects exact logical APK paths such as
`lib/arm64-v8a/libconscrypt_jni.so`; each library's `source_path` preserves its
original ZIP spelling. Omitted filters include all scoped facts; `[]` matches
none. Class selection never narrows the global inventory used for table-class
attribution.
Independent `declarations`, `libraries`, `candidates` and `bindings` collections
have stable IDs and source ordinals. Candidates retain APK/library digests,
ABI, function addresses and file offsets. Table candidates also retain section,
table and entry identity plus all three name/signature/function pointer slots
and their relocation sources. JNI exports follow the
[UTF-16 escaping and short-before-long rules](https://docs.oracle.com/en/java/javase/21/docs/specs/jni/design.html#resolving-native-method-names);
failed escaping does not acquire an invented export match. Tables require
NUL-terminated [Modified UTF-8 strings](https://docs.oracle.com/en/java/javase/21/docs/specs/jni/types.html#modified-utf-8-strings),
valid descriptors and executable load mappings. REL, RELA and RELR retain
separate provenance; unsupported packed relocations and unresolved pointer
spans stay explicit unknowns.
A static `JNINativeMethod` entry names no class. Attribute it uniquely only
when exactly one declaration in the complete recorded native inventory has
that `(name, descriptor)` pair. Multiple classes remain ambiguous after a
class filter; a partial inventory cannot establish unique attribution.
Each selected declaration has a row per selected/observed ABI. `matched` means
candidate presence was observed; `ambiguous` means multiple observed targets
or class associations remain. Binding `coverage` is separate: a matched
partial row does not establish complete uniqueness, and an empty partial scan
is `unknown`, never `unmatched`. ELF/ABI label mismatches preserve facts without
asserting a binding. No result establishes library loading, runtime registration
or method invocation.
Summary returns the first 64 selected binding rows, their declaration/candidate
closure and every selected library's identity, coverage and failure reason;
counts and page coverage describe the full selected binding set. Selected
declaration IDs have separate preview coverage, so native declarations remain
visible even when the APK has no bundled ABI libraries. Complete
Evidence stays retained. Read exact fields or further pages without reopening
files or rerunning analysis:
```json
{
"source": { "kind": "retained-evidence", "evidence_id": "ev_<digest>" },
"view": {
"kind": "android-jni",
"collection": "bindings",
"offset": 64,
"limit": 64,
"fields": ["declaration_id", "abi", "status", "coverage", "candidate_ids"]
}
}
```
Collections also include `declarations`, `candidates` and `libraries`, with
exact filters and batch lookup by declaration/candidate/library IDs. `row_index`
is the original parent collection index; DEX and
ZIP source ordinals stay separate. Complete detail returns the full report.
CLI uses the same selection and delivery contracts. Its common one-shot
default remains complete; summary delivery requires an explicit Evidence path:
```sh
rea inspect-android-jni-bindings /absolute/Example.apk \
--query '{"class_name":"org.conscrypt.NativeCrypto","method_names":["EVP_PKEY_new_RSA"],"abis":["arm64-v8a"]}' \
--detail summary --evidence_output /absolute/fresh-jni-evidence.json
```
Optional `--view` accepts the same JSON page and requires summary delivery.
Evidence bundles never overwrite files. Complete declarations and one library
selection are cached within the immutable APK session. Changing ABI/library
selection replaces the scan; class/method selection reuses it. Fresh analysis
still validates current caller APK bytes. Idle/close releases the caches;
retained Evidence remains historical and independently readable.
## Interpret the result
- Package inspection retains decoded manifest XML, summary declarations,
@@ -512,11 +204,6 @@ retained Evidence remains historical and independently readable.
`dex_descriptor`, for example `(I)V`. Display names can be JADX aliases;
the descriptor alone is not a complete raw DEX method ID. Overload indices
are tied to this artifact and engine, not stable identities across builds.
- A `mapping` block appears only when the caller selected `mapping_path`. Its
names are claims from that file, not observations of the APK. REA does not
check that the mapping belongs to this build; `pg_map_id` is reported, not
compared. Evidence keeps the selected path in its parameters and the file's
SHA-256 in the result.
- Method inspection reports Java or smali, fallback markers and complete/partial
returned text. Decompiled source is a derived representation. A complete text
result does not establish complete semantics or a byte-identical reconstruction.
@@ -606,19 +293,6 @@ target/configuration changes, failure, timeout, cancellation, MCP disconnect or
server shutdown. One-shot CLI commands always join cleanup before returning.
REA never writes to the original APK or launches target code.
JNI scans reuse the analyzer process without loading the APK into the
decompiler. They admit at most 64 MiB per DEX member and 256 MiB of DEX bytes,
with a 4 MiB native-declaration JSON allowance before MCP string escaping.
Native libraries are processed sequentially: one 64 MiB buffer, 256 MiB total
selected read work, and a 16 MiB parsed-library-facts allowance. ZIP inventory
and library counts also have explicit resource bounds. Unexamined occurrences
remain unknown. A complete relationship report has a separate 16 MiB serialized
allowance; summaries/pages allow 1 MiB including their provenance envelope.
These are resource admissions, not claims about maximum valid Android formats.
Select fewer ABI/library occurrences or smaller retained pages when needed.
Completed parsed library facts survive failed reader finalization in typed
partial observations; the exact reader owner remains available for cleanup retry.
On Windows, stdin writes preserve stream backpressure and run outside the Node
event loop. Cancellation terminates the retained job, including descendants,
before workspace cleanup. On Windows, a detached REA helper owns the private
@@ -679,22 +353,16 @@ export REA_JADX_OPERATION_TIMEOUT_MS=600000
The value must be a decimal millisecond count from 1 to 2147483647; anything
else is refused before Java starts. REA reads it from the environment captured
at process start. Changing that environment requires a new REA process. On a
running MCP server, set `jadx.operation_timeout_ms` through
`update_analysis_settings` to change the default for newly accepted operations;
queued or running operations retain their accepted budget. The same deadline
bounds each upstream call. Timeout recovery reports the effective milliseconds
and range, with `details.limit` identifying the runtime setting. An invalid
provider-local launch value can also be repaired by a runtime override; reset
exposes that invalid launch value again. See the
[settings design](../../design/limits-and-recovery.md#invalid-launch-configuration).
A passing JADX configuration check in `rea doctor` reports its launch-derived
`operation_timeout_ms`; it does not inspect another running server's overrides. Explicit
at process start. A change applies on the next process start; an already
running server keeps the deadline it started with. The same deadline bounds
each upstream call. A timeout names `REA_JADX_OPERATION_TIMEOUT_MS`, the
effective milliseconds, and that range. A passing JADX configuration check in
`rea doctor` reports the effective `operation_timeout_ms`. Explicit
cancellation still ends the operation immediately, and partial observations
and owned-process cleanup are unchanged. Raising the deadline does not extend
the bridge's 30-second startup handshake or the upstream **90-second**
decompilation limit. MCP clients enforce their own tool-call timeouts, and
setup's derived [client tool timeout](../../getting-started/client-configuration.md) does not include
setup's derived [client tool timeout](client-configuration.md) does not include
this setting, so also raise the client timeout when selecting a longer
deadline.
@@ -704,27 +372,6 @@ complete, case-sensitive search contract. A protocol frame above **8 MiB**, or c
**32 MiB per operation**, fails with a diagnostic instead of returning an incomplete inventory
as complete. These are memory-safety boundaries, not undocumented result caps.
A selected R8/ProGuard mapping is re-read and SHA-256 checked on every call, up
to **64 MiB**. The serialized provider keeps one successful parsed index, keyed
by byte length and digest with fixed parser options. Identical bytes reuse that
index; changed bytes are reparsed, missing files still fail, and each response
preserves the currently selected path. The index is released on provider idle
retirement or close. Omitting `mapping_path` never applies the cached mapping.
During replacement, the old index can coexist with the new parse until it
succeeds; these transient allocations count toward process memory.
Before retaining records, the parser admits an estimated **128 MiB** index
representation and **8 MiB** of expanded mapped claims per residual class,
including shared header facts. The index allowance accounts for strings and
containers; it is not a measured V8 heap cap. Parameter and metadata expansion
is admitted before splitting or JSON decoding. These fixed resource bounds
have no runtime override; refusal reports `resource_constraint`, the source
line and mapping SHA-256. Ambiguous lookups count all candidates while keeping
only the first eight in stable file order. A separate **8 MiB** combined claim
budget applies to the actual query set, including outer and inner classes,
before cloning the report. Claims that do not fit become explicit `unknown`
entries; other admitted claims and all observed identities remain available.
## Real verification
Download the explicit public fixtures once; this installs no tools:
@@ -768,7 +415,3 @@ containing node's unknown `dex_offset`. `direct_invokes_exhaustive: false`
means instruction rows were bounded by the local bridge frame budget; `null`
means the producer did not establish instruction coverage. Using-node totals
and coverage do not imply complete instruction evidence.
`direct_invokes_coverage` reports counts of using nodes with complete, partial,
or unknown positions, plus the number of retained instruction sites. This
aggregate makes position truncation visible without scanning every caller;
older retained Evidence may omit it, which means the aggregate was unexamined.
-178
View File
@@ -1,178 +0,0 @@
# Offline API contract inventory
`inspect_api_contract` / `rea inspect-api-contract` supports local OpenAPI JSON
and GraphQL SDL (`format: "graphql-sdl"`). Neither profile contacts endpoints.
## OpenAPI JSON
The default profile reads one local OpenAPI
3.0, 3.1 or 3.2 JSON document. It inventories inline operation slots under
`paths`; the facts describe declarations, not implemented, reachable or
authorized runtime behavior. YAML, AsyncAPI, callbacks/webhooks, schema graphs,
contract diffs and traffic correlation are outside this version.
The OpenAPI importer never resolves references, opens sibling documents, evaluates
examples or contacts declared servers. OpenAPI 3.2 `query` and
`additionalOperations` follow the [Path Item contract](https://spec.openapis.org/oas/v3.2.1.html#path-item-object),
including case-sensitive additional HTTP method spelling.
## Select a useful answer
MCP defaults to a summary retaining the complete inventory. An optional view
returns selected facts immediately, taking precedence over `detail`:
```json
{
"path": "/artifacts/openapi.json",
"view": {
"kind": "api-contract",
"collection": "operations",
"methods": ["GET"],
"limit": 20
}
}
```
`operations` supports exact `operation_ids`, `operation_keys`, `methods`,
`paths`, `declared_ids` and `states` filters. Filters combine by intersection;
empty arrays select nothing. `fields` chooses columns; identity, source pointer,
state and unknown reason remain. The other collections are
`path-item-references` (optional `reference_ids`) and `diagnostics` (optional
`codes`), also with fields and offset/limit. The default page contains 64 rows.
Query a retained result with `inspect_analysis_view`, using the same view and
`source: {"kind":"retained-evidence","evidence_id":"..."}`. Copy the complete
parent ID from `normalized_result.parent_evidence_id`, not the summary's own ID.
These pages use immutable retained facts without rereading the specification.
Complete Evidence remains available through `get_evidence_bundle`.
CLI defaults to complete Evidence. For a selected answer, save that complete
Evidence in an absent bundle path:
```sh
rea inspect-api-contract ./openapi.json --detail summary \
--view '{"kind":"api-contract","collection":"operations","methods":["GET"],"limit":20}' \
--evidence_output ./contract.evidence.json --json
```
Later `rea inspect-analysis-view` can read that bundle through an exact
`evidence-file` source selection, even after the original specification is
unavailable. Publication never overwrites an existing file. CLI preserves its
original path spelling in `caller_input_path`; analysis locations and MCP inputs
use absolute filesystem paths.
## Identity, provenance and coverage
Each operation has an artifact-scoped `operation_id`, an exact method/path
`operation_key`, an optional `declared_id` from `operationId`, and its original
JSON Pointer. Keys can correlate exact declarations in a caller-selected pair
of revisions; they do not establish equivalence across unrelated APIs, renamed
path templates or runtime handlers. Declared IDs are not assumed unique.
The artifact includes the verified byte length, SHA-256 and selected path.
Malformed siblings retain useful declarations plus diagnostics. A Path Item
`$ref`, including a local fragment, remains unresolved; adjacent inline slots
are unknown because referenced fields may overlap. Missing or malformed paths
are explicit, and missing required OpenAPI 3.0 paths does not become a complete
empty inventory. `source_coverage` is limited to inline paths and records that
reference resolution and full schema validation were not performed. Page
exhaustion describes the selected rows and never upgrades source coverage.
Structured transport userinfo credentials are excluded from reference URIs;
`credential_redacted` reports that change, and an ambiguous credential span leaves
the displayed reference unknown. Ordinary reference query/fragment text and
names are retained. Original bytes remain in the caller's artifact; complete
inventory Evidence does not embed the entire source JSON. Duplicate JSON members
use the shared parser's last-value-wins behavior, recorded in source coverage.
If decoding or later validation fails after a verified read, the typed error
retains a compact `file-snapshot-hash` observation with the exact path, digest
and byte length. Syntax diagnostics do not quote source JSON. Cancellation
does not replace a parser failure that has already completed.
## Resource bounds and verification
The file-byte allowance and conservative expanded-facts allowance are each
32 MiB resource policies, not OpenAPI format limits. `max_file_bytes` can lower
the input allowance. Native JSON parsing additionally checks available heap
against the shared representation-expansion estimate before decoding. The
shared JSON nesting limit is 1,000, including uninterpreted schema regions.
Capacity refusals are typed; reply paging does not increase parser allowances.
The committed [Petstore fixture](https://github.com/morluto/rea/blob/b7c8caedd4166e6032c49024cdc6fb1154f1a2df/tests/fixtures/openapi/README.md) is an
original document from the published application, replayed offline through
public CLI/MCP tests. These check advertised schemas, complete/selected parity,
retained paging, partial references, zero requests to a loopback reference
server and unchanged source bytes. They do not establish runtime endpoint
behavior or full OpenAPI conformance.
## GraphQL SDL and operation documents
Select `format: "graphql-sdl"` for SDL. Optionally supply `operations_path` to
one local document containing operations and fragments. Both MCP paths are
absolute; CLI accepts relative spellings:
```sh
rea inspect-api-contract ./schema.graphql --contractFormat graphql-sdl \
--operationsPath ./queries.graphql --detail summary \
--view '{"kind":"graphql-contract","collection":"references","limit":20}' \
--evidence_output ./graphql.evidence.json --json
```
Collections are `types`, `fields`, `definitions`, `references`, `fragment-spreads`
and `diagnostics`. Each supports `fields`, `offset` and `limit` (default 64).
Exact batch filters include type/field/definition/reference IDs, type names,
`field_keys` such as `Query.viewer`, owning type names, binding/spread states
and diagnostic codes. Filters intersect; an empty array selects nothing.
Identity, source and applicable uncertainty fields always remain. Retained pages
and bundle-based queries use the same `graphql-contract` views without rereading
SDL or rebuilding its schema model.
Types and fields retain individual declaration occurrences, including extensions,
list/non-null wrappers, implemented interfaces and union member names. Operation
and fragment definitions own field references with aliases and parent-reference
IDs. Each reference links to a supplied SDL field occurrence, remains `model-only`
for a library-provided field such as `__typename`, or records an exact unknown
reason. Rejected schema construction keeps declarations but cannot supply trusted
bindings. Missing or duplicate fragment names remain explicit. Fragments are
visited once in their declared type scope and linked by spread edges; cycles
are never recursively expanded.
The versioned `graphql-sdl-field-bindings-v1` profile records the GraphQL library
version. It performs syntax parsing and schema construction for field resolution;
it does not establish whole-schema/operation validity, spread applicability,
reachability, authorization or endpoint behavior. Arguments, defaults,
descriptions, directive behavior, enum value inventory, introspection JSON,
federation composition and version comparison remain outside this profile.
Every fact has a document role, SHA-256, UTF-16 start/end offsets and parser token
line/column. Offsets address decoded UTF-8 text, including an optional BOM, rather
than byte positions. The two files are independently verified snapshots, not an
atomic pair. Fact IDs identify source occurrences; `field_key` separately supports
exact name-based joins and does not assert equivalence across revisions. CLI
retains its auxiliary path spelling in `caller_input_paths.operations_path`,
outside semantic identity, as well as its primary `caller_input_path`.
Malformed operation text retains the SDL inventory with partial coverage and a
source-located diagnostic. A primary parser refusal or later resource/cancellation
failure retains known source identities and available interpreted Evidence;
syntax diagnostics do not quote source text. Complete Evidence contains facts,
not original document bytes. An exhausted page never upgrades source coverage.
Each file has the same configurable maximum of 32 MiB. Conservatively charged
facts share a 32 MiB allowance; text/AST allocation is admitted against available
heap. The parser admits at most 262,144 lexical tokens and 65,536 comments per
document; subsequent iterative AST admission limits traversal depth to 128.
These are REA resource policies, not GraphQL language limits. Very deep input can
be refused by the parser before AST admission. Selected replies reduce transport
and retained queries reuse immutable facts; they do not avoid the initial parse.
The committed SWAPI SDL is an unchanged public application fixture with provenance
in `tests/fixtures/graphql/README.md`. It supplies 53 types and 243 fields;
verification does not execute its API.
Large complete inventories can exceed the default MCP response budget even when
selected pages fit. Use `export_evidence_bundle` with a caller-selected local
path for complete retained Evidence, or CLI complete JSON/bundle publication.
A transport refusal does not erase a completed retained analysis. Narrow pages
and source coverage remain available on the same continuing connection.
+14 -32
View File
@@ -179,31 +179,26 @@ share the application workflow, projection, retention and publication owners.
- `schemes_agree_on_signer` (observed): every present block scheme presents
the same _set_ of signer certificates. An extra signer in one scheme is
disagreement; a signer without a certificate makes it `null` (unknown).
A single present block scheme with known signers agrees with itself;
v1 is compared separately below.
Build-tools 33+ rotate through v3.1 by default, so a rotated app usually
reads `false` here and `true` below.
- `signer_change_declared_by_rotation` (declared): the signers differ, but
each appears in a lineage the APK declares. Read the two together: block
schemes with different signers covered by a declared lineage show `false`
/ `true`; differing signers without such a lineage show `false` / `false`.
A lineage alone does not establish a change between block schemes. It is
`null` when a signer's identity is unknown, or when the signers differ and
a declared lineage could not be decoded.
each appears in a lineage the APK declares. Read the two together: a
rotated APK shows `false` / `true`; signers that differ with no declared
lineage containing them show `false` / `false`. It is `null` when a
signer's identity is unknown, or when the signers differ and a declared
lineage could not be decoded.
- `v1_signers_match_block_signers` (observed): the v1 signer certificates
equal what the lowest block scheme that has signers (v2, else v3, else
v3.1) expects v1 to carry. That is its signer set, except that a v3/v3.1
signer declaring a rotation lineage expects the lineage's first
(original) certificate, as apksig requires. `false` is notable: the JAR
signature names a different key than the block signature. When v1 uses
the lineage's original key and v3 its newer key, this comparison reads
`true`. If v3 is the only present block scheme, agreement is `true` and
declared inter-block signer change is `false` even though rotation is
present. If v2 also uses the original key, agreement is `false` and the
declared inter-block change is `true`. `null` when
v1 or a populated block scheme is absent, any signer's identity is unknown (a
signature names a different key than the block signature. A key rotation
keeps v1 (and v2) on the original key, so it reads `true` while
`schemes_agree_on_signer` reads `false`, with or without v2. `null` when
v1 or a block scheme is absent, any signer's identity is unknown (a
`decode_error`, a `null` certificate or a `subjectKeyIdentifier`
`SignerInfo`), or the lineage's original certificate cannot be
identified. When identities or a lineage origin are unknown, limitations
explain why; absence is shown in the scheme records.
identified; a limitation then says why.
- `strongest_scheme`, `rotation_present`, `source_stamp_present`,
`self_consistent`.
@@ -244,26 +239,13 @@ serialized size of the report together with its limitations (which repeat
archive names) is then checked against an 8 MiB output budget (the exact check
is authoritative); an over-budget report is refused, not truncated.
Signing blocks admit at most 4096 ID/value pairs; at most 4096 signing-relevant
META-INF entries are retained. These fixed counts, signer/lineage counts and the
report byte budget return nonretryable `resource_constraint` errors naming the
effective allowance. This release has no runtime override for them. Waiting for
an active scan cannot raise a fixed count, and complete CLI JSON cannot raise the
report budget. Occupied scan-queue capacity remains retryable after a scan finishes.
The whole-file reader allocates the selected snapshot and bounded read scratch;
total process memory exceeds the selected file's size. It also computes the complete
The whole-file reader retains read chunks and a concatenated buffer; total
process memory exceeds the selected file's size. It also computes the complete
file SHA-256, so inspection is not header-only. Content digests are recomputed
at most once per SHA-256/SHA-512 algorithm during one inspection. Input,
inflation, decoded-record and serialized-output bounds keep work finite;
resource refusals are distinct from malformed format errors.
If the stable read completes but closing its descriptor fails or is cancelled,
the error retains `details.partial_observation` with the selected path, complete
file SHA-256 and byte count. These facts describe the acquired file; they do not
establish any decoded signing facts. Failed cleanup remains visible until the
provider releases the descriptor on a cleanup retry.
A file that is not a ZIP is refused as an unsupported target. A damaged ZIP
(local headers but no end-of-central-directory), a central directory that
cannot be walked or whose declared record count does not account for all of
@@ -100,13 +100,12 @@ still suppresses all string projection, including in this mode.
- **Metadata comes from apktool.yml as apktool printed it.** Version codes
keep their own spelling (`'7'` versus `7`); `package_name` is read from
the decoded manifest, not inferred.
The `version: 2.9.0`-or-later header that Apktool 2.9+ and 3 emit selects
their single-line string dialect: literal hashes, single quotes, empty
strings, and the text `null` stay part of the value; producer backslashes
and Unicode escapes decode once. Older/tagged documents retain YAML scalar
semantics; double-quoted values also retain YAML string semantics.
Apktool 3's emitted `version: 3.x.y` header selects its single-line string
dialect: literal hashes, single quotes, empty strings, and the text `null`
stay part of the value; producer backslashes and Unicode escapes decode
once. Older/tagged documents retain YAML scalar semantics; double-quoted
values also retain YAML string semantics.
This distinction follows the released
[2.9.0 metadata writer](https://github.com/iBotPeaches/Apktool/blob/b64e257c9b689eb9630ad0241f2fe4f4a749955b/brut.apktool/apktool-lib/src/main/java/brut/androlib/apk/VersionInfo.java#L38-L42),
[3.0.3 metadata writer](https://github.com/iBotPeaches/Apktool/blob/18b5e99cb56ff9451e8aa55b065dcf5bbd616975/brut.apktool/apktool-lib/src/main/java/brut/androlib/meta/VersionInfo.java#L49-L55)
and [string escape owner](https://github.com/iBotPeaches/Apktool/blob/18b5e99cb56ff9451e8aa55b065dcf5bbd616975/brut.j.yaml/src/main/java/brut/yaml/YamlStringEscapeUtils.java#L55-L116).
- **The manifest is the decoded text**, bounded at 512 KiB and reported as
@@ -174,7 +173,7 @@ OpenJDK 25 against a signed aapt2-built probe APK carrying two resource
locales — launcher version, on-disk digest agreement, apktool.yml metadata
(1.2.3, SDK 24–34), manifest package agreement, default and `de` string
tables projected. Parser goldens in the test suite come from the same
decode. See `docs/development/testing.md` for the lane summary.
decode. See `docs/testing.md` for the lane summary.
Multi-locale and table-only projection were also verified through real CLI and
stdio MCP workflows on Linux with Apktool 3.0.3/JDK 21 against NewPipe 0.29.1,
@@ -101,8 +101,8 @@ limitation reports how many inventoried entries were left out. IPA projection
still lists every archive component, including `SwiftSupport/`.
`platforms` lists `macos` for application roots with `Contents/`. It lists `ios`
for shallow application roots that have an `Info.plist` at their root. An
application root with neither, such as an empty inventory, has no platform.
for shallow application roots that have an `Info.plist`. An application root
with neither, such as an empty inventory, has no platform.
## macOS bundle anatomy
@@ -149,11 +149,6 @@ directories rather than `Versions/Current`. A framework's `info_plist_path` is
because `Versions/Current` decides which one applies) or when its single
version directory has no `Resources/Info.plist`.
A `shallow` bundle's `info_plist_path` is its root `Info.plist`. Without one,
it is `Resources/Info.plist`: a flattened macOS framework keeps its binary,
`_CodeSignature/`, and `Resources/` directly in the `.framework`, like a single
`Versions/<version>/` directory without the `Versions/` wrapper.
AppleDouble sidecar files (`._*` and `__MACOSX/`) are inventory facts, but they
describe neighbouring files. They are excluded from roots, bundle roles, and the
anatomy components.
+6 -87
View File
@@ -29,12 +29,7 @@ body is parsed only when that string is `"309"`. Other types, including `"288"`
stackshots, return no crash result.
`process`, `exception`, `termination`, `faulting_thread`, `threads`, and
`images` reproduce documented body fields when present, including each image's
`name` and each frame's `symbolLocation` (as `symbol_location`, the byte offset
from the start of `recorded_symbol`). `termination` keeps the terminating
process as `by_proc` and `by_pid` and its `flags`; `process.parent_pid` keeps
`parentPid` next to `parent`; and `metadata` keeps `build_version` and
`platform`. Integers keep their
`images` reproduce documented body fields when present. Integers keep their
decimal literals and a hexadecimal form, including arm64e registers above
2^53. Frame roles follow the recorded frame index: row zero of the resolved
faulting thread is `faulting_instruction`, row zero of other threads is
@@ -43,22 +38,9 @@ address uses that row's own image base and offset. The recorded `inline` flag
does not supply a missing offset or authorize borrowing another row's address.
When `faultingThread` is missing or does not resolve to a recorded thread
object, row-zero roles are `unknown`; the recorded lookup remains available.
A translated process records Rosetta's registers in a nested
`threadState.rosetta` object; each one is reported as `rosetta.<name>` and
counts toward the register budget. A [public ReportCrash `.ips` log](https://github.com/Mortennn/Dozer/issues/189)
shows `rosetta.tmp2`, `tmp1`, and `tmp0` beside ordinary x86 registers. The
test fixture quotes that register excerpt inside an authored report body; it
does not generate a new Rosetta crash during the test.
Observed JSON strings retain `""`; `null` means the field was missing or JSON
`null`. This includes image paths, frame symbols, process text, and the older
string `osVersion` form. Unlisted JSON keys and generic register names also
retain empty spellings. An empty image architecture remains recorded as `""`
while `uuid_unknown` stays true, since that spelling does not establish an
architecture. An empty UUID is malformed, so its exact spelling appears in
`coverage.type_mismatches` and its canonical UUID remains unknown.
`process.os_version` keeps the producer's dictionary (`train`, `build`,
`release_type`, `is_embedded`) or the older string form. Image UUIDs are uppercase 8-4-4-4-12; an all-zero UUID
or a missing or empty `arch` is `uuid_unknown`. Image `source` letters keep their
or a missing `arch` is `uuid_unknown`. Image `source` letters keep their
documented meaning (`P` process, `S` shared cache, `C` shared cache library,
`K` kernel, `U` kernel cache, `T` kernel text exec, `A` absolute). Keys the
documentation does not list remain in `unlisted_keys` or `unlisted_body_keys`.
@@ -85,49 +67,9 @@ canonical hexadecimal form), `EXC_GUARD` subtypes,
pointer-authentication suffixes, and dyld missing-library strings. A report
that matches none of those rules is `unclassified`.
## Joining frames to the crashed Mach-O or dSYM
Pass `join_path` (`--join-path` in the CLI) to select one local Mach-O
executable or dSYM. UUID bytes in the report and the artifact's `LC_UUID` load
commands are compared in hexadecimal, so an uppercase `dwarfdump --uuid` value
and a lowercase report `uuid` match.
```bash
rea inspect-apple-ips /absolute/path/to/report.ips \
--join-path /absolute/path/to/App.app/Contents/MacOS/App --json
```
Shared-cache image sources `S` and `C` remain unjoined: UUID agreement does not
establish that a standalone image has the cached layout.
For a FAT container every slice is examined. An image joins only when exactly
one slice has that UUID **and** an architecture that agrees with the report's
`arch`; `arm64` and `arm64e` stay distinct, and an all-zero UUID never joins. A
UUID match with a different architecture, more than one compatible slice, or a
slice without a readable `__TEXT` segment is reported as a rejected image with
its reason and stays unjoined rather than being guessed.
An unknown UUID in another slice prevents a unique join unless its architecture
rules out a match. Missing or unrecognized architecture and unsupported CPU
subtypes remain unknown; display-name fallbacks do not establish agreement.
The parsed slice facts remain available even when no match can be derived.
`join.status` is `selected` with the artifact path, SHA-256, byte length, every
slice's offset, arch, CPU type and subtype, UUID, and `__TEXT` vmaddr, plus
`image_matches`, per-frame results, and join-specific limitations. A joined
frame reports `static_address`, the slice `__TEXT` vmaddr plus the frame's
recorded `imageOffset`, in decimal and hexadecimal. An unjoined frame reports
a reason instead; an unknown image UUID, an absent `imageIndex` row, and a
frame without `imageOffset` all stay unjoined rather than treated as aligned.
An artifact that is not a Mach-O image or FAT container reports
`join.status: "unreadable"` with a typed reason and leaves every report fact
intact. Symbol names printed inside the report stay recorded claims. Function
or symbol lookup at a joined static address, and matching dSYM symbol tables,
are not performed by this step.
Symbol names printed inside the report are recorded claims and are not
verified. Linux ELF cores use the separate
[recorded crash](guides/native/recorded-crashes.md) workflow.
Opened Mach-O and dSYM UUID joins remain unobserved. Symbol names printed
inside the report are recorded claims and are not verified. Linux ELF cores
use the separate [recorded crash](recorded-crashes.md) workflow.
## Limits and failures
@@ -160,27 +102,4 @@ success or truncated integer facts.
rea inspect-apple-ips /absolute/path/to/report.ips --max-integer-text-bytes 16777216 --json
```
`max_join_bytes` (`--max-join-bytes`) independently bounds the join artifact
read, defaulting to 64 MiB. This workflow does not execute a target, launch
atos, or search for dSYMs.
## Join admission and failures
The join uses the shared Mach-O container, slice, command-table and segment
readers. Slice extents and `sizeofcmds` bound every identity observation; duplicate
UUIDs or malformed commands cannot select the first plausible value. Unreadable
join results retain the successfully read artifact's path, SHA-256 and byte count.
The integer-text allowance spans the report and join, including repeated integer
facts and derived static addresses. Reservation precedes derived address
expansion. If a selected join read, interpretation or allowance fails, the error
retains the completed IPS report. Join read limits and recovery use
`max_join_bytes`, even when the report and join paths are identical. Cancellation
retains the caller's reason independently of those completed facts.
When a failure follows a completed join-file read, `partial_observation` retains
both the decoded report and a `join_artifact` receipt with the independently
observed path, SHA-256 and byte count. Frame rejection reasons reference
`join.limitations`; source-sized architecture diagnostics are retained once per
image rather than copied into every frame. Universal-image admission includes
the shared reader's supported hidden ARM64 row and rejects overlapping slices.
This workflow does not execute a target, launch atos, or search for dSYMs.
@@ -8,7 +8,7 @@ graph TD
end
subgraph "Production Composition"
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>typed capability bindings: construction, availability and retryable lifetimes<br/>pure capability declarations feed provider catalogs"]
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>static WASM/EVM descriptors: named contracts, factories, retryable lifetimes"]
end
subgraph "Adapters"
@@ -21,15 +21,14 @@ graph TD
subgraph "Application Layer"
APP["src/application/<br/>Shared CLI/MCP workflows<br/>enhanced analysis and investigations"]
SESSION["SessionProviderRouter + BinarySession<br/>one immutable deep binding per target<br/>disjoint auxiliary routing"]
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback<br/>ProviderInspectionScope: prerequisite observation lifetime"]
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback"]
RECORDS["src/application/investigation/<br/>composed InvestigationRecords + narrow ports<br/>EvidenceLedger and Unknown ownership"]
LIFETIMES["CapabilityScope<br/>host-owned service close capabilities<br/>released peers removed; failed cleanup retained for retry"]
SNAPSHOT["BinarySessionRecords + AnalysisSnapshotCache<br/>binary target/profile snapshots and session file ownership<br/>record coordination and post-commit notifications"]
SNAPSHOT["BinarySessionRecords + AnalysisSnapshotCache<br/>binary target/profile snapshots<br/>compatibility facade and post-commit notifications"]
JS_RECON["src/application/javascript/ + src/artifacts/javascript/<br/>JavaScript artifact reconstruction<br/>directory/ASAR reads<br/>AST-only bundle projection"]
end
subgraph "Provider Layers (selected adapters)"
PROVIDER_OWNERS["src/providers/<br/>shared deferred client owners<br/>exact cleanup retry and attributed failures"]
PROVIDER_OWNERS["src/providers/<br/>shared deferred client and inspection owners<br/>exact cleanup retry and attributed failures"]
PWNTOOLS["src/native/pwntools/ + src/native/pwndbg/<br/>BYO offline ELF/core adapters<br/>owned snapshots/Python and optional core-only GDB<br/>unchanged upstream parsers, ABI structures and mapping API"]
WASM["src/wasm/<br/>caller-supplied WABT; immutable artifact inspection<br/>owned process supervision and retryable workspace cleanup"]
APKSIG["src/apkSignature/<br/>in-process APK Signing Block + v1 reader<br/>bounded single read, no device or execution"]
@@ -76,7 +75,7 @@ graph TD
end
subgraph "Domain Layer"
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>format declarations, provider selection, Evidence, comparisons, application graphs"]
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>evidence, comparisons, application graphs"]
end
subgraph "Configuration"
@@ -100,7 +99,6 @@ graph TD
MAIN --> COMPOSITION
CLI --> COMPOSITION
SERVER --> COMPOSITION
COMPOSITION --> LIFETIMES
COMPOSITION --> SESSION
APP --> FIRMWARE
FIRMWARE --> PROCESS
@@ -151,7 +149,8 @@ graph TD
OBJDUMP --> DOMAIN
COMPOSITION --> RIZIN
RIZIN --> DOMAIN
COMPOSITION --> GDB
CLI --> GDB
SERVER --> GDB
GDB --> PROCESS
GDB --> DOMAIN
HOPPER --> HOPPER_BRIDGE
@@ -207,7 +206,8 @@ graph TD
NATIVE_READERS --> DOMAIN
COMPOSITION --> FORMAT_READERS
FORMAT_READERS --> DOMAIN
COMPOSITION --> RUNTIME
SERVER --> RUNTIME
CLI --> RUNTIME
RUNTIME --> DOMAIN
PROCESS --> FILESYSTEM
AUX --> FILESYSTEM
@@ -8,14 +8,9 @@ ordinary binary inspection use the same modular capability.
## Bring your own engine
Real verification covers **Linux x64** hosts and **EXEC/DYN/REL** objects in
these profiles: ELF64 x86-64, ELF64 little-endian AArch64, ELF32 i386, ELF32
little-endian ARM and ELF32 little- or big-endian MIPS. Other machines, classes
and byte orders, including big-endian ARM, return `unsupported_target`. Other
hosts, including Linux arm64, return `capability_unavailable` naming the
detected host before any target is read. Provide an absolute
Python executable whose environment already contains unchanged pwntools 4.15.0,
pyelftools 0.33 and Unicorn 2.1.2:
Initial real verification covers **Linux x64, ELF64 x86-64 little-endian
EXEC/DYN/REL**. Provide an absolute Python executable whose environment already
contains unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2:
```sh
REA_PWNTOOLS_PYTHON=/absolute/isolated-env/bin/python \
@@ -105,8 +100,6 @@ one section or symbol, or a stable page without re-running the decoder. CLI
one-shot invocations pass portable inline Evidence JSON; an MCP session uses
the retained `evidence_id`. Page `limit` is required and bounded by the
measured MCP stdio budget documented on the tool contract.
An empty retained section or symbol collection has no valid item index. Its page
view can still return an empty page.
Malformed authenticated layout input remains rejected with the owning schema's
safe field diagnostics under `source.normalized_result`, including the field
path and type or value reason. Evidence authentication and decoder validation
@@ -125,7 +118,7 @@ reply budget applies to the decoder record; the CLI/MCP Evidence envelope adds
copies and encoding overhead. The
pinned MCP SDK defaults to a 10 MiB receive buffer. Large complete results need
a caller-configured `StdioClientTransport({ maxBufferSize: ... })` and a matching
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](../../reference/mcp-contracts.md#tool-results).
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](mcp-contracts.md#tool-results).
These settings affect delivery, not decoder coverage or limits. The large
extended-index verification uses 256 MiB and an explicit five-minute request
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to at most 3 GiB virtual address space,
@@ -272,12 +265,6 @@ regular files only and rejects symbolic links.
## Byte entropy triage
Entropy admission preserves the caller's spelling in `caller_input_path`. On
Windows, the selected regular file's case-only filesystem spelling is used
consistently in committed paths through both CLI and MCP. Symlink, junction and
short-name aliases are not replaced with a different path. General path
normalization remains lexical and does not probe source-discovered paths.
`inspect_byte_entropy` / `inspect-byte-entropy` profiles any local regular file
without an analysis engine, execution, extraction or network access.
@@ -432,10 +419,8 @@ rea inspect-native-exports ./libexample.so --detail summary --evidence_output ./
without an export directory is a complete empty inventory.
- ELF rows list defined `.dynsym` symbols with global, weak or GNU-unique
binding and default or protected visibility. Each row keeps `value`
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`,
`binding` and `visibility` (`"default"` or `"protected"`; `"unknown"` in older
historical Evidence where visibility was not recorded). ELF string tables do
not declare a text encoding. Names that
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`
and `binding`. ELF string tables do not declare a text encoding. Names that
contain non-ASCII bytes are decoded as UTF-8 when those bytes form valid
UTF-8; `name_utf8_decodable` records whether the non-ASCII name bytes are
valid UTF-8 and `name_bytes_base64` retains those exact bytes (`null` for
-62
View File
@@ -1,62 +0,0 @@
# Binary hardening
Use `inspect_binary_hardening` to read compile-time and link-time hardening from
an explicit local ELF, PE, or Mach-O file. No disassembler, pwntools install, or
target execution is required. Universal Mach-O files return one image per slice.
```sh
rea inspect-binary-hardening /absolute/path/to/program --json
rea inspect-binary-hardening /absolute/path/to/program --max-file-bytes 1048576 --json
```
MCP accepts `{ "path": "/absolute/path/to/program" }`. CLI paths may be relative
to the current directory; MCP paths must be absolute.
Each image has one row per property: `property`, `state`, `confidence`, `basis`,
`location`, `note`, and `next_probe`. States are `enabled`, `disabled`,
`partial`, `unknown`, and `not_applicable`. `confidence` is `observed` for a
single header or import, and `derived` when the state combines several fields,
such as ELF PIE or full RELRO. An image's `architecture` is its CPU family, or
null when REA has no family for it. `machine` and `slice.architecture` keep the
exact identity, such as `arm64e`.
ELF rows cover PIE (`ET_DYN` with `PT_INTERP` or `DF_1_PIE`), NX (`PT_GNU_STACK`;
a missing header is `unknown` and the architecture default is only noted), RELRO
(`PT_GNU_RELRO` plus `BIND_NOW`), stack protector and FORTIFY undefined dynamic
imports, CET and AArch64 BTI/PAC GNU property notes, `DT_RPATH`, `DT_RUNPATH`, and
whether a `SHT_SYMTAB` section is present. Only an undefined dynamic import
counts. Imports come from the table `DT_SYMTAB` declares. A `SHT_DYNSYM` section
only supplies the symbol count when `DT_HASH` and `DT_GNU_HASH` leave it open,
and only if it names the same symbol and string tables. Without that count the
import rows are `unknown` and give the reason. A missing `__stack_chk_fail` or fortify import is `unknown`. That
absence does not establish that the protection was left out of a static or
stripped build. A present import is a sign that some code uses the protection,
not proof that all code does. NX, RELRO, `DT_RPATH`, and `DT_RUNPATH` are
`not_applicable` when `e_type` is neither `ET_EXEC` nor `ET_DYN`, and `unknown`
for a linked image with no readable program headers.
PE rows read `DllCharacteristics` for dynamic base, high-entropy VA, NX
compatibility, force integrity, no SEH, and guard CF. `cfg_instrumented` is the
load-config `GuardFlags` linker claim when `Size` covers that field. An
unreadable load configuration, including a TLS failure in that parser, leaves
only `cfg_instrumented` `unknown`. Authenticode and debug rows report certificate
and debug directory presence, not signature validity or debug content. A
directory whose address and size disagree is `unknown`.
Mach-O rows read `MH_PIE`, `MH_ALLOW_STACK_EXECUTION`, `MH_NO_HEAP_EXECUTION`,
an `__RESTRICT` segment, `LC_CODE_SIGNATURE` presence, encryption from the
existing encryption reader, the arm64e subtype, and a stack-protector import.
`hardened_runtime` is `unknown` with `next_probe` `inspect_signature`.
CodeDirectory flags stay with that tool.
The report states presence of build-time features. It does not state
exploitability. `inspect_binary_layout` is unchanged.
The default read budget is 64 MiB (`max_file_bytes`, ceiling 512 MiB). A changed
file is refused. A malformed header, a Mach-O segment or encryption command that fails admission, or a program, section,
or segment payload declared outside the file returns no partial report, except that one bad slice in a universal Mach-O leaves the other slices. A symbol table or
symbol name over its budget leaves the import rows `unknown` and keeps the
header rows. A GNU property note over its budget, or a compressed note section
that may hold one, leaves the CET and BTI/PAC rows `unknown` and keeps the header
rows. A dynamic-table budget still fails the ELF report. The image is never
executed.
@@ -31,7 +31,7 @@ authority are validated independently before selection and connection.
For a compact route query, select `detail: "summary"` and a
`view: {kind: "javascript-routes", collection: "routes", limit: 20}`. The
shared [route inventory](../../javascript-route-inventory.md) preserves source IDs,
shared [route inventory](javascript-route-inventory.md) preserves source IDs,
precise locations, partial coverage, and the complete parent Evidence for
subsequent filtered pages. Static candidates never establish complete application
coverage or reachability.
@@ -52,11 +52,7 @@ observations may omit the coverage field; treat omission as `unknown` rather
than inferring completeness from the array.
Passive DOM comparison checks complete node inventories and complete frame
parent/child structure independently. Node value lengths are comparable only
when both nonempty snapshots declare the same `node_value_length_unit`.
Missing or different units leave length comparison unknown; independent node
or frame structural differences can still prove change. Empty inventories have
no value lengths to compare. Frame IDs and sibling enumeration order
parent/child structure independently. Frame IDs and sibling enumeration order
are capture-local and do not establish a change; duplicate IDs, missing parents,
and cyclic ancestry remain unknown. A complete frame difference can still prove
change when DOM nodes are incomplete, and vice versa. Partial DOM or metadata
@@ -98,14 +94,6 @@ content. Actual JSON properties with those names remain content. Reference-only
objects, unsupported values, and opaque cookie partition identities remain
incomplete. Previously exported fingerprints are not recomputed; recapture both
sides when comparing captures produced across this decoder change.
IndexedDB schema fingerprints with `projection_version: 2` include the observed
database version and object-store definitions. A missing, malformed, or imprecise
version leaves the schema fingerprint incomplete while retaining observed store
facts. Historical fingerprints without this marker covered object stores only:
matching historical hashes cannot establish unchanged database versions, and a
historical hash is not compared with a version-bearing hash. Recapture both
sides for a complete IndexedDB version comparison; other proven storage changes
remain reportable.
Script comparison groups observations by reported URL and compares unordered
content samples. Producer inventory keys include optional metadata, so a changed
@@ -147,7 +135,7 @@ Caller-owned JSON/base64 strings, input parsing and validation, transport
buffers, garbage-collector behavior, and other runtime memory are outside this
estimate; the budget does not promise a bound on total process memory.
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](../javascript/electron-observation.md).
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](electron-observation.md).
## Provider authorities
@@ -177,7 +165,7 @@ See [the reproduction and Chromium ownership limitation](https://github.com/morl
Existing static application Evidence and passive web/Electron captures can be
combined later through
[JavaScript static/runtime reconciliation](../javascript/javascript-runtime-reconciliation.md).
[JavaScript static/runtime reconciliation](javascript-runtime-reconciliation.md).
## Start a browser
@@ -239,21 +227,10 @@ file for you.
`observe_web_session`, `discover_webmcp_tools`, and `capture_web_screenshot`
accept a browser socket. Website runtime attribution, Electron inspection,
and scenario attach mode still require HTTP discovery.
- An admitted browser socket is retained by the active browser provider across
calls to the same normalized browser WebSocket URL, and concurrent cold-start
calls share one handshake so MCP sessions do not re-open the browser socket on
every step. Scheme case, uncompressed IPv6 loopback spellings, and leading
port zeros normalize to one reuse key, while `127.0.0.1`, `[::1]`, and
distinct `/devtools/browser/<id>` paths remain separate sockets. Each target
operation detaches its flattened target session when finished: unconfirmed
target attach or detach cleanup closes the shared browser socket for session
containment, whereas a caller's own discovery deadline or cancellation
releases only its lease when transport state remains clean. Provider shutdown
closes the retained browser socket. One-shot CLI invocations close their
provider on exit, so Chrome may still ask you to approve each separate CLI
connection. Discovery (`Browser.getVersion`, `Target.getTargets`) defaults to
a 10-second deadline and can be adjusted per call with `discovery_timeout_ms`
(`--discovery-timeout-ms` on the CLI).
- Each call opens one WebSocket and uses it for discovery
(`Browser.getVersion`, `Target.getTargets`) and for the selected page.
Discovery must finish within the same 10-second deadline as HTTP discovery.
Chrome may ask you to approve each connection.
- The browser identifier controls access to the socket. Evidence records the
endpoint as `ws://127.0.0.1:9222/devtools/browser/[REDACTED]`.
- `browser.revision` is the socket's `Browser.getVersion` value, such as
@@ -415,19 +392,7 @@ contextual evidence with `timestamp: null`, accompanied by a limitation stating
that it does not establish activity in the selected window. Main-frame scope
enforcement still applies to contextual navigation events. Network redirect
events retain their paired prior-hop response context when the timestamped
next-request event falls inside the window. A redirect row's optional
`redirect_response` contains the prior response `url`, CDP-reported `status`,
and `scope`. The URL has transport credentials removed; URL and status are
withheld unless its origin is approved. Missing or malformed response URLs have
null scope and add `invalid_protocol_value` timeline coverage; a valid URL with an
unsupported protocol has `unsupported` scope. Outside-policy and unsupported
responses mark the timeline as policy-filtered with `disallowed_origin` or
`unsupported_url`. Missing status remains null rather than zero. A supplied
non-numeric status is also null and records `invalid_protocol_value` coverage;
the other valid response facts remain available. Historical rows may omit this
field; omission does not establish that no prior response existed. Response
headers and bodies are discarded. The row timestamp belongs to the next request,
not an exact prior response time. The pre-arm timeline buffer is bounded
next-request event falls inside the window. The pre-arm timeline buffer is bounded
by original CDP message bytes. If clock sampling completes after that budget is
exhausted, the operation returns `payload_limit` instead of a partial window.
Cancellation and clock-request failures retain their own outcomes. Session
@@ -442,8 +407,7 @@ allowance; exhaustion fails the observation explicitly instead of guessing scope
Response metadata parses repeated CDP header field lines independently. The
Link projection retains their declarations in order. `csp.directives` lists the
directives of every CSP policy in order, across field lines and comma-separated
policies, without recording policy boundaries, so a directive name can repeat
across policies. As in CSP3, a repeated name within one policy is ignored.
policies, without recording policy boundaries, so a directive name can repeat.
Single-valued policy fields combine their lines as one comma-separated value,
as RFC 9110 specifies. Each link keeps its target in `href` and
its relationship context in `context_url` with a separate `context_scope`;
@@ -462,21 +426,10 @@ complete Structured Field Item, including parameters such as `report-to`.
These are declarations, not an assessment of the document's effective policy;
absent, malformed or non-token items project as `null`.
Console entries, WebSocket frame summaries, and selected request/response JSON
shape outputs share one 8 MiB retained-observation allowance per capture. Shape
inference admits expanded path bytes before allocating those paths, including
escaped and Unicode property names. Replacing or removing a request's shapes
refunds their retained charges. A refused body shape preserves network metadata
and any captured peer shape with `resource_budget_exhausted` coverage; a refused
WebSocket frame is counted as dropped rather than reclassified as non-JSON text.
The allowance has no runtime override. Protocol messages and JSON input parsing
remain subject to their separate transport admission; this output allowance does
not bound the whole process heap.
REA retains local observation data and removes only URL userinfo credentials, explicitly declared scenario secrets, and values from structured credential/storage fields:
- URL observations preserve the complete local URL, including query values, fragments, duplicate parameters, and their order. Only username/password userinfo is removed; malformed original URL strings remain available with `origin: null`.
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values. `dom.node_value_length_unit` is `utf16-code-units` for web observations and applies to every node's `node_value_length`; it counts JavaScript string code units, not Unicode code points or UTF-8 bytes. Electron observations declare `utf8-bytes` instead. Historical captures without the unit leave it unknown; reading them does not infer a unit or convert their stored lengths.
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values.
- Supported DOM URL attributes are projected separately as resolved destinations. Empty `action` on HTML or XHTML forms and empty `formaction` on HTML or XHTML submit controls resolve to the document URL; other relative URL attributes resolve against the captured document base.
- Accessibility structure and reported roles are retained by default, while names and descriptions require `include_accessibility_text: true`. The browser protocol may omit an individual role; that role remains unknown for comparison. Other observed accessibility differences can still establish a change.
- Network observations retain method, status, MIME type, size, type, initiator stack location, and complete URLs with userinfo credentials removed. Redirects reported by CDP are attached as ordered `redirects` on the one request ID, with the prior request URL, CDP response URL, method/type, response status/MIME/encoded length, request timestamp, and the next `requestWillBeSent` event timestamp. CDP delivers `redirectResponse` with that next request event, so the latter is an event boundary rather than an exact response timestamp. Redirect response headers and bodies are discarded; hops outside the selected origins are discarded with the request chain. Headers are discarded after an allowlisted projection of length/encoding, structured CSP/Link/policy fields, and untrusted agent hints. Cookies and authorization headers are never retained.
@@ -511,7 +464,7 @@ REA retains local observation data and removes only URL userinfo credentials, ex
connection/frame inventories; matching direction/opcode sequences alone do
not prove alignment after drops. Redirect resource types use only shared known metadata:
an omitted type leaves equality unknown rather than proving a modification.
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. Source locations are CDP's zero-based lines and UTF-16 columns. A negative column, which Chrome reports on the first line of some inline scripts, is kept as reported and does not locate a column in the source URL; an invalid line or a non-integer column is `null` (WebMCP `registration_source` is `null`) and is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
- WebSocket observations retain direction, opcode, and payload byte length. A missing or malformed CDP opcode is returned as `null`, with opcode-dependent payload byte length and shape also `null`, and marks `websocket_frames` incomplete. Missing or wrong-typed payload data is not treated as empty; its byte length is `null`, and a text payload shape is unknown. Numeric opcode `0` and an observed empty payload remain distinct from missing data. Opcode is nullable in the public inspection schema, so clients must handle unknown protocol values. When shape capture is selected, text messages are classified as text or complete value-free JSON shape; binary bytes, hashes, prefixes, and raw payloads are never retained.
- Storage observations always redact values. Key names, IndexedDB names, and cache names require `include_storage_keys`; stable content fingerprints require the additional `include_storage_fingerprints` selection. Cookie fingerprints cover cookies applicable to the current main-frame URL, including its path, rather than every cookie path on that origin. Other storage remains scoped to the origin. Cache fingerprints hash all decoded response bytes, including bodies above 64 KiB, subject to the [64 MiB per-message CDP limit](#request-boundary). Missing or undecodable cache bodies and partial IndexedDB remote objects make fingerprint coverage incomplete, so identical observations remain `unknown`.
- Unavailable or malformed storage inventories mark `storage_keys` unavailable, even without fingerprints. Comparisons suppress inferred key additions and removals while preserving modifications proven by complete fingerprints on both sides. Previously exported captures are not retroactively corrected; recapture them to obtain these coverage markers.
@@ -609,13 +562,13 @@ an unknown size, comparison reports unknown rather than treating the observation
as a resource addition or removal; complete inventory coverage still establishes
when an entire group is absent.
CDP discovery defaults to a 10-second response deadline, configurable per call
on passive browser tools with `discovery_timeout_ms` (default 10000). HTTP
discovery documents use an 8 MiB budget. Browser-socket discovery
(`Browser.getVersion`, `Target.getTargets`) uses the 64 MiB protocol message
budget. Neither byte budget has a runtime override. `list_browser_targets`
accepts `discovery_timeout_ms` and no `command_timeout_ms`, because it performs
only discovery without a post-discovery target command. Other CDP tools use
CDP discovery has a 10-second response deadline. HTTP discovery documents
use an 8 MiB budget. Browser-socket discovery (`Browser.getVersion`,
`Target.getTargets`) uses the 64 MiB protocol message budget. Neither budget
nor the discovery deadline has a runtime override. `list_browser_targets` accepts no
`command_timeout_ms`: HTTP discovery and browser-socket discovery
(`Browser.getVersion`, `Target.getTargets`) stay on that fixed deadline, and
the socket closes without a later command. Other CDP tools use
`command_timeout_ms` (default 20000) for each WebSocket handshake and command
response, separate from the caller-selected observation window, discovery, and
the 5000 ms cleanup grace. A browser-level socket's handshake and the following
@@ -637,22 +590,6 @@ Network and console coverage starts only after REA attaches and enables the rele
Source maps are not fetched by inspection. Bundle analysis fetches them only when `fetch_source_maps` is selected, with exact-origin redirect checks and no credentials, cookies, or referrer. Each fetch operation uses `source_map_fetch_timeout_ms` (default 30000) and a 64 MiB aggregate streamed-response budget across redirects. The byte budget has no runtime override. A fetch overrun is an item limitation, not a thrown analysis timeout. A deadline or byte-limit failure remains explicitly unavailable and makes bundle completeness partial. Source-map decoding also bounds raw resolved identities to 64 MiB and aggregate inline evidence to 32 MiB. Failed maps preserve their failure context when it fits; if another context cannot fit, collection stops with the full requested count, the returned processed count, and an explicit unknown-results limitation. Retained mappings and original sources are returned inline. See [source-location limits](web-source-location.md) for the structural decoder bounds. Original source artifacts with `.ts`, `.tsx`, `.mts`, or `.cts` suffixes carry `text/typescript`; other source names retain `text/javascript`.
A fully read response that fails UTF-8, JSON, mapping, or structural validation
retains `artifact: null`, empty original-source, module-edge, and mapping
arrays, and an explicit failure limitation. Invalid content is `invalid`;
decoder or structural capacity refusal is `fetch_failed`. Its optional
`response_identity` records the complete body's SHA-256 digest, byte count,
and `fetch-response-body-bytes` representation without retaining malformed
text or charging that text to the 32 MiB output budget. These are bytes
delivered by Fetch after HTTP content decoding and before UTF-8 decoding, not
HTTP wire bytes. The identity is absent when the body was not fully read, and
older observations may omit it. A response whose text cannot fit the existing
output budget is likewise `fetch_failed` with text omitted and completed-body
identity; JSON parsing and source-map decoding are not attempted. Successful and partial maps keep their
full text artifacts. Their original text, including an initial BOM or XSSI
line, remains distinct from the parsing view that removes that framing.
Collection reads the body once and does not raise its response or output limits.
When fetching is selected, source-map `excluded` facts preserve the capture
ledger's reasons and counts for declarations rejected before download. No
disallowed URL details are retained or fetched. `requested` and `processed`
@@ -660,17 +597,6 @@ count the admitted fetch requests; exclusions are counted separately. Excluded
declarations make the collection unavailable, or partial when maps were retained.
An empty collection with no declarations remains `not_requested`.
If the caller cancels while later source maps are being fetched, the failure's
partial Evidence retains the captured page and every completed map item. Its
`normalized_result.kind` is `web-bundle-acquisition` and
`analysis_status` is `not_performed`: no script-analysis result is claimed.
`source_maps.requested` remains the admitted request count,
`source_maps.processed` counts completed items, and any unprocessed requests
have an explicit unknown-results limitation. Capture exclusions remain in the
same collection. The cancellation error keeps the caller's exact reason;
an already observed response resource or cleanup failure keeps its own error
category and cleanup owner.
Source-map items retain the script's `declared_url` separately from the observed
`effective_url`. Successful maps use the latter as their source-resolution base.
Failed fetches retain the last response URL observed within the approved origins;
@@ -9,14 +9,9 @@ liveness deadlines. `INPUT_JSON` may be inline JSON or a JSON file path.
The minimal request contains only browser launch/connect selection, `start_url`,
and at least one `actions` entry. Launch `headless` defaults to `true` and can
be set to `false` where the host supports a visible browser. Launched Chromium
keeps its sandbox; on hosts that cannot provide one, such as root in a
container, set `REA_BROWSER_NO_SANDBOX=true` for the REA process. Termux on
Android always launches without it. Navigations follow
the destinations named by the scenario and the page's ordinary browser requests,
and [request steps](#request-steps) send only to the start URL's origin;
there is no separate origin allowlist. Request steps use a temporary dispatch
gate for cancellation; ordinary page requests retain their browser behavior. Environment
be set to `false` where the host supports a visible browser. Navigations follow
the destinations named by the scenario and the page's ordinary browser requests;
there is no separate origin allowlist or request-interception layer. Environment
settings (including service workers blocked by default), empty storage, and a
final sanitized URL capture are supplied by default. Set
`environment.service_workers` to `allow` when the application requires them.
@@ -54,30 +49,6 @@ covers endpoint discovery only. The following WebSocket handshake and
`Target.attachToTarget` each use `command_timeout_ms`. `capture.limits` controls retained event and network-content
budgets; it does not change those liveness deadlines.
## Validate a scenario before running it
`validate-browser-scenario INPUT_JSON` checks one scenario against the same
`browserScenarioSchema` refinement that `capture_browser_scenario` admits, and
stops there. `INPUT_JSON` may be inline JSON, a JSON file path, or `-` for
standard input.
```bash
rea validate-browser-scenario ./browser-scenario.json --json
```
A valid scenario reports `{"valid":true,"actions":<count>,"secrets":<count>}`
and exits 0. An invalid one exits 1 and reports the same field paths capture
would report, so a correction made here is the correction capture needs:
duplicate or reserved step IDs, duplicate secret identities and
environment-variable names, undeclared secret references, request-step
violations, and unknown fields.
Validation performs no browser discovery, launch, attachment, or navigation and
constructs no provider. It does not read secret environment variables, so a
scenario whose declared variables are unset still validates; the missing-value
failure belongs to capture. A schema-valid scenario does not prove that the
executable or target is available on the host.
## Minimal launch example
Save this as `browser-scenario.json`, replacing the executable path with an
@@ -200,12 +171,6 @@ distinction: it closes and deletes provider-owned profiles, but only disconnects
from an external CDP browser. Real-browser verification checks both outcomes and
confirms that an attached external browser remains alive.
CDP connections leave Playwright's default-context overrides disabled. The
selected page receives the declared scenario environment and temporary focus
emulation, so actions can complete while that page is in the background.
Its environment owner clears these overrides before disconnecting; other pages
keep their media settings. Connecting does not create a bootstrap page.
That cleanup requires REA to remain running. Abrupt termination, including
Windows process-tree termination or POSIX `SIGKILL`, can leave the launch
profile and Playwright artifacts behind. A profile can retain cookies, history,
@@ -239,9 +204,7 @@ language environments.
HTTP(S) URLs may include ordinary query values and fragments directly. Use
ordered structured query entries when a value needs to reference a declared
environment-backed secret. Form, storage, and cookie values are either literal
strings or declared secret references; request step headers can also read a
cookie or local storage value from the page, and request steps and `goto`
destinations can send [saved values](#saved-values). Raw URL userinfo credentials are rejected.
strings or declared secret references. Raw URL userinfo credentials are rejected.
Captured credential-header
values are not retained, and declared secret values are redacted automatically.
A secret may be declared solely to
@@ -352,263 +315,6 @@ external browser without closing it. The request itself defines the operation;
there is no additional REA grant step. The browser and host still enforce their
own access and process rules.
## Request steps
A `request` step sends one HTTP request from the page with the page's own
`fetch`, so the browser attaches the session's cookies and the request is
recorded in the step's network events like any request the application makes.
Use it to record an application's API, including writes, when driving the UI
for each call would be slower or less exact than calling the endpoint the UI
calls. It is the only action that changes application state without a user
gesture, so run scenarios with write requests only against an instance whose
data you may change. This scenario creates one item:
```json
{
"browser": {
"mode": "launch",
"executable_path": "/absolute/path/to/chromium"
},
"start_url": { "url": "http://127.0.0.1:3000" },
"actions": [
{
"step_id": "create-item",
"action": "request",
"method": "POST",
"path": "/api/items",
"query": [
{ "name": "notify", "value": { "source": "literal", "value": "false" } }
],
"headers": [
{
"name": "X-CSRFToken",
"value": { "source": "cookie", "name": "csrftoken" }
},
{
"name": "Authorization",
"value": {
"source": "local_storage",
"key": "auth",
"json_pointer": "/token"
},
"prefix": "Bearer "
}
],
"json": { "title": "Scratch item 1" },
"expected_status": [201],
"timeout_ms": 10000
}
]
}
```
- `method` is `GET`, `HEAD`, `POST`, `PUT`, `PATCH` or `DELETE`. `json` is the
body, sent with `Content-Type: application/json` unless a header sets one; a
`GET` or `HEAD` step cannot carry it.
- `path` starts with one `/` and must resolve to the start URL's origin, which
validation checks before the browser starts. When the step runs, the page must
be on that origin, checked again in the page as the request is sent; add a
`goto` step first if an earlier action left it. A query in `path` is sent as
written (a fragment is not sent), and `query` entries are appended after it
in order and may reference declared secrets.
- Request steps follow no redirects: a redirect answer fails the step, and
nothing is sent to the location it names, though a write the redirect
answered may already have been applied. `expected_status` cannot name a 3xx
status. Manual redirects can be opaque to the page and may not produce a
response event; their status and `Location` can therefore be unavailable even
when network events and header values were selected. Such captures report
incomplete event coverage.
- Header values are literals, declared secrets, a cookie the browser holds for
the request URL (`{"source": "cookie", "name"}`, HttpOnly cookies included,
whose value the page's scripts can then see while the request runs), or a
local storage entry on the start origin (`{"source": "local_storage",
"key"}`, with an optional `json_pointer` to a string or number in a stored
JSON value). `prefix` is sent before the value. A value read from the page is
redacted like a declared secret, as `[REDACTED:page:<step_id>.<header>]`,
from records made after the step reads it; a response recorded earlier, such
as the login response that returned a token, keeps the value. A value read
again keeps its first marker, and a value shorter than 8 characters is not
redacted, since redacting it would rewrite ids, ports and hosts throughout
the capture. A value is redacted as stored and as `fetch` sends it, without
surrounding spaces and tabs. A missing cookie, key or pointer fails the step.
Reading a page value raises the estimated cost of redacting every later
body, so the bodies kept within `max_network_body_allocation_bytes` (1 MiB
by default) get smaller: after a 32-character `X-CSRFToken` value, a body over
about 47 KiB is kept as `unavailable`. Raise `capture.limits` when the
application's responses are larger.
- A header can send Basic credentials, `{"source": "basic_credentials", "parts":
[username, password]}`: the base64 of `username:password`, after a `prefix`
such as `"Basic "` or with none, as some APIs take encoded credentials in
their own header. Each part is a literal, a declared secret, a saved value, a
cookie or a local storage entry, so a header a frontend builds from a login
token can be sent too; the password is never a literal other than empty, since
Evidence records literals as written, and a username with a colon fails the
step, since Basic authentication ends the username at its first colon. The
encoded value is redacted however short, as
`[REDACTED:page:<step_id>.<header>]`, and parts read from the page are
redacted as page values are.
- Validation refuses a header name given twice in a step, header names a
page's `fetch` cannot set, such as `Cookie`, `Origin`, `Host` or `Sec-*`, and
`User-Agent`, which Chrome drops; it refuses line breaks and other ASCII
control characters in literal values and prefixes, and the step fails before sending
when a secret or page value holds them.
- The step completes when the whole response has been read. With
`expected_status`, any other status fails the step and the later actions are
recorded as `cancelled`; without it, any response completes the step. A
network error fails the step. `timeout_ms` covers reading the header values
and the request: a step whose time runs out, or that is cancelled, before
its request is sent sends nothing, but a request already sent can still reach
the application after the step fails.
An absolute deadline also covers browser evaluation queued behind page work.
A temporary route checks cancellation again at network dispatch and is removed
when the request evaluation settles. A page controlled by a service worker
is refused because its fetch can bypass that dispatch gate; use the default
blocked service-worker policy for request steps.
The request runs in the page's main world, so an application that wraps
`fetch` sees it, and it goes through the same network observation as other
page requests: select network content (bodies or header values) to record it.
Captures with request steps state in `limitations` that the steps sent their
declared requests through the page's fetch. A scenario without request steps
is unchanged.
## Saved values
A step can keep a string or number from a JSON response, by name, and a later
step can send it, so a scenario can read back, edit and delete an object whose
id the application assigns:
```json
{
"browser": {
"mode": "launch",
"executable_path": "/absolute/path/to/chromium"
},
"start_url": { "url": "http://127.0.0.1:3000/items" },
"actions": [
{
"step_id": "save-item-1",
"action": "click",
"timeout_ms": 10000,
"locator": { "kind": "role", "role": "button", "name": "Save" },
"save_response": {
"method": "POST",
"path": "/api/items",
"save": [{ "name": "item_1", "json_pointer": "/id" }]
}
},
{
"step_id": "open-item-1",
"action": "goto",
"timeout_ms": 10000,
"destination": { "url": "http://127.0.0.1:3000/items/{saved:item_1}" },
"wait_until": "load"
},
{
"step_id": "copy-item-1",
"action": "request",
"timeout_ms": 10000,
"method": "POST",
"path": "/api/items/{saved:item_1}/copies",
"json": {
"source_id": "{saved:item_1}",
"title": "Copy of {saved:item_1}"
},
"expected_status": [201]
}
]
}
```
- A `request` step saves with `save: [{name, json_pointer, secret}]`, from its
own response. A `click` or `press` step saves with `save_response: {method,
path, save}`, from a newly observed matching response while the step waits.
The recorder watches before the action and excludes requests already running,
such as an earlier autosave. Method/path matching does not prove that the UI
caused the request: use a path that distinguishes the intended operation from
background polling. Multiple observed matches before binding fail as ambiguous;
requests starting after this observation window cannot be ruled out. It matches the method and a path on the
start origin as the browser spells it, so a path may name `My List` or
`My%20List` (`{*}` matches one segment, and the query is ignored), and fails
the step unless a 2xx JSON response arrives before its `timeout_ms`; one still
being read when the step times out or is cancelled saves nothing. A pointer
that names no string or number fails the step, so later steps never send a
missing value, as does a response over 4,000,000 characters: a request step
stops reading and aborts its fetch there. A UI step reads a body only when
its decoded size is bounded at 12,000,000 bytes or less: its received size,
or the declared Content-Length of one served from cache, times 1032 for each
gzip or deflate coding, the most DEFLATE expands. Other codings and unknown
or zero sizes are refused before materializing the body; decoded text over
the character limit also fails. The UI action may already have applied.
- `{saved:name}` sends a saved value in a request step's `path`, including its
query, and in a `goto` destination's `url`, percent-encoded; a destination's
structured `query` values are sent as declared. A saved value that makes a
path segment `.` or `..`, alone or with the text beside it (including `%2e`),
fails the step, since the URL would drop it. In a request step's `json` body,
a string that is only `{saved:name}` becomes the saved value with the JSON
type it was saved with, so a saved number id is sent as a number, and
`{saved:name}` inside a longer string is filled as text. A header value or
Basic credentials part sends one as `{"source": "saved", "name"}`. Other
braces, such as `{id}`, are sent as written, and so is a percent-encoded
`%7Bsaved:name%7D` anywhere but a `goto` URL, whose parsed form spells the
braces that way.
- Validation refuses a saved value that no earlier step saves. A later save of
the same name replaces the value. An integer past 2^53, which JSON parsing
would round to another number, and a number that is not finite, such as
`1e400`, negative zero, or a value that underflows to zero, such as `1e-400`, fail the step
that saves it.
- A scenario keeps at most 4,000,000 characters of saved values at once,
counting distinct retained redaction spellings, including trimmed variants,
of values saved as secret. Replacing a binding releases its live value
charge, while secret spellings remain protected for the scenario lifetime.
A save past the budget fails before changing the binding.
- A value saved with `secret: true` is redacted however short, as a declared
secret is, as `[REDACTED:page:<step_id>.<name>]`. The selected source
response is protected before retaining its body or header values, including
when only headers were requested. If that body cannot be read safely, its
body and header values remain unavailable. Before publishing the capture,
the selected source transaction's earlier URL, request headers and request
body are also protected using the admitted values. Unresolved source values
withhold that transaction's affected metadata/content with an explicit
limitation. Before publication, retained events are rechecked
with the final marked-response values, including immediate follow-up requests
and their response bodies, console/page errors, WebSocket text and event URL/name fields. Learned redaction facts are independent of source-identity readiness. Unknown source values withhold only the selected
source transaction. Other saved values, such as ids, stay visible, since
environments model them. The environment build warns about saved values
named like credentials that are not marked secret
(`credential-named-saved-values`).
- Page-derived cookie/local-storage values and constructed Basic credentials
remain redacted only from observations made after they were read/built.
They do not retroactively rewrite earlier facts. The final event projection
borrows only declared and response-marked patterns; promotion of an already
held page value to an explicit response secret preserves its first marker.
Projection state is session-local, references admitted pattern/byte owners,
and is invalidated on admission or response classification changes.
- Screenshots are withheld as `missing` from the start when the scenario plans
any response save marked `secret: true`, page-derived or Basic authentication
header values, or whenever nonempty sensitive values
are already known. Raw pixels cannot be safely text-redacted, so the provider
does not request pixels in those cases. This also protects earlier steps and
partial failures without retaining an image to scrub later. Other requested
artifacts remain available; affected screenshot/aggregate completeness is
partial and is not eligible for equality.
- Saved URLs and expanded request JSON are admitted before building their expanded strings or
serializing. Its allowance is the smaller of Node's string limit and
one sixteenth of current V8 available heap bytes, allowing for the expanded
tree, serialization, Playwright transport and browser-side copies. Size
accounting includes JSON escaping and repeated references; each distinct
saved value's size is measured once per request. An oversized request fails
before browser dispatch; the complete request envelope, including repeated
header values, is checked as well. URL encoding uses a conservative
nine-characters-per-UTF-16-unit allowance before encoding a saved value.
Reduce the payload/repetitions, or raise the parent
Node heap and restart; this is a conservative allocation estimate, not a
measurement or guarantee of browser/process-family memory.
- Evidence records a request step's empty `save` list as absent, so a scenario
that saves nothing records the same parameters and `scenario_sha256` as
before saved values existed, and REA versions without saved values read its
captures.
## Network evidence
`capture.events: ["network"]` retains request, response, completion, failure,
@@ -649,18 +355,6 @@ and unfinished-request metadata. Select content independently:
All three options default to `false`. Any selected network content also enables
network metadata observation; listing `network` in `events` is then optional.
Each observed request object receives a capture-scoped `transaction_id`.
Each `request` event carries `observed_at`, the wall-clock time REA observed the
request start. Playwright reports the browser's own start only once a request
ends, so each `request-finished` or `request-failed` event carries `started_at`,
the browser's wall-clock start, and `elapsed_ms`, its time from that start to
the end of the response, when the browser reports them. The environment views
time scenario records from these, as HAR entries are timed: the browser's start
where it reported one, else `observed_at`, which trails it by up to tens of
milliseconds while REA reads other responses. A duration is retained only from a
unique terminal event that reports it; missing or ambiguous durations stay
omitted, while an explicitly reported zero remains zero. Captures made before these fields existed lack them and
are read in record order. REA versions before them refuse captures that carry
them. Scenario comparisons ignore all three (`ignore_network_request_timing`).
Requests sharing a URL remain distinct. Redirects link to the preceding
transaction only when it was observed; a null predecessor does not establish
that no earlier hop existed. Request events require both transaction fields;
@@ -758,8 +452,8 @@ and compared as an unordered collection after caller rules apply, because
concurrent responses arrive in any order and can land in a later step; a
redirect retains the complete predecessor ancestry, including each hop's own
request, response outcome, and selected network content. Each transaction's
capture-local IDs, sequence/index fields, content source-sequence anchor and
request times (`observed_at`, `started_at`, `elapsed_ms`) are removed. Caller rules normalize retained strings before hashing. The
capture-local IDs, sequence/index fields, and content source-sequence anchor
are removed. Caller rules normalize retained strings before hashing. The
`preserve_redirect_transaction_ancestry` built-in rule fingerprints each hop
as canonical SHA-256 of `{ events, redirected_from }`: `events` is the sorted
list of its normalized event SHA-256 digests, and `redirected_from` is its
@@ -780,10 +474,7 @@ scope/flags, value fingerprints, and duplicate entries even when redaction or
literal rules make names collide. Captured storage arrays retain their original
order. The `compare_storage_entries_as_unordered_collection` built-in rule
participates in the normalization commitment; new storage comparison digests
therefore differ from earlier order-sensitive comparisons. The
`ignore_network_request_timing` built-in rule likewise joins the commitment, so
normalization digests differ from comparisons made before it, though captures
with and without request times compare alike.
therefore differ from earlier order-sensitive comparisons.
Each aligned step reports changed, unchanged, or unknown. Changed and unknown
action, screenshot, DOM, accessibility, URL, history, storage, and event
@@ -806,58 +497,3 @@ rules and preserve the same missing fact.
Network normalization also commits the removal of capture-local transaction IDs,
attribution to the requesting step, and unordered transaction comparison. These
built-in rule identities participate in the normalization digest.
## Summary delivery and retained views
`capture_browser_scenario` accepts `detail: "summary"` to retain the complete
capture in the MCP session and return its summary. The default remains
`complete`. Delivery detail does not change scenario parameters or identity.
The summary reports each step's status, error, URL and captured/missing/truncated
section names. It omits `not_requested` sections and repeated `before_url` values.
The accessibility tree comes from the first unsuccessful step, or the final
step on success; when that step captured no tree, its nearest earlier captured
tree is identified explicitly. Event and network counts cover retained records,
with capture omissions reported separately.
For the one-shot CLI, save the complete Evidence before selecting a summary:
```bash
rea capture-browser-scenario scenario.json --detail summary --evidence_output /absolute/path/scenario-evidence.json --json
```
Use the returned `normalized_result.parent_evidence_id` with
`inspect_analysis_view` source `{kind: "retained-evidence", evidence_id: "..."}`.
For a CLI bundle, use an `evidence-file` source with its absolute path and the
parent Evidence ID. Views read the recorded capture; they never replay actions.
- `{kind: "summary"}` returns the same overview.
- `{kind: "browser-scenario", collection: "steps", step_indices: [2], artifacts: ["accessibility"], offset: 0, limit: 1}` returns selected recorded steps and artifacts.
- `{kind: "browser-scenario", collection: "events", event_kinds: ["response"], transaction_ids: ["request-2"], offset: 0, limit: 20}` pages selected events with original item indices.
Omitted filters select all; empty arrays select none. Pagination counts matching
retained rows. Exhausting a page does not establish capture completeness or
current application state.
Transaction pages use `{kind: "browser-scenario", collection: "transactions", limit: 20}`.
Their stable identity is the parent Evidence ID plus `transaction_id`; each row
keeps original event indices. Metadata events are returned by default. Select
`event_kinds: ["network-content"]` for recorded bodies and header values, or
`event_kinds: []` for identity and indices alone. A `step_indices` filter selects
a whole transaction when any retained event belongs to a selected step; this
is receipt attribution, not proof that the action caused the request.
For a large retained tree, request `{kind: "browser-scenario", collection: "text",
step_index: 10, artifact: "accessibility", offset: 0, limit: 2048}`. `dom` is also
supported. Offsets and limits count UTF-16 code units. The full artifact digest
and byte size remain attached to each page. Missing, truncated and unrequested
text stay explicit; source truncation counts are preserved without treating an
exhausted page as a complete capture.
Network summary counts include every retained transaction identity, including
those whose request event was dropped. Missing request events, conflicting
metadata and observed request failures have separate counts. `transactions` is
`null` when network observation was unrequested or its legacy selection is
unknown and there are no retained transactions; an observed empty inventory is
`0`. Full capture omissions remain available and each page carries source
completeness, dropped-event counts and omitted-transaction accounting.
-62
View File
@@ -1,62 +0,0 @@
# Bun standalone executables
`inspect-artifact` inventories embedded Bun module graphs in Mach-O `__BUN,__bun`,
PE `.bun`, ELF `.bun`, and appended trailers. `extract-artifact` writes their stored
module contents through the existing safe output tree. Neither operation executes
the executable or decompiles bytecode.
```sh
rea inspect-artifact ./hello --json
rea extract-artifact ./hello --json
```
In MCP, open the executable with `open_binary`, then call `inspect_artifact`.
Use `detail: "summary"` for large graphs and `inspect_analysis_view` to page retained
Evidence. The root metadata's `entry_point_logical_path` is the reader-owned
location of its entry module. Use it as an inventory `logical_paths` filter,
then read selected node fields with the returned content identity. This also
works for names mapped to indexed extraction paths. The complete Evidence stays
retained; no source-name normalization is needed in the caller.
The root reports the carrier, layout, module count, entry point, raw flags and
compile exec-argv. Each module retains its source name, loader and encoding values,
entry-point status, sizes and SHA-256. Sourcemaps, bytecode and module-info regions
are separate children. Source names such as `/$bunfs/root/main.js` remain in
metadata; extraction uses a relative path. Unsafe or non-UTF-8 names receive an
indexed path and retain their source spelling or bytes. Virtual names deeper
than 256 logical path parts also use an indexed path with an explicit limitation,
preventing unbounded path-tree work while retaining the source fact. Duplicate names remain
separate occurrences; ordinary extraction collision rules apply.
The recognized record shapes are the 36-byte Bun 1.3.3 layout and the 52-byte
Bun 1.3.9 layout. These labels identify structural layouts, not a detected compiler
version. Bun does not store an explicit revision identifier: unknown table sizes
and ambiguous known layouts are unsupported. A future layout that is byte-for-byte
compatible cannot be distinguished. Invalid pointers and truncated claimed sections
are malformed. An absent trailer without a named section does not establish a Bun
graph. Unknown enum values remain raw, with a null name.
Sourcemaps are stored Bun regions, not necessarily standalone JSON source maps.
Extracted contents retain their original encoding. Use JavaScript analysis only on
compatible source files; bytecode is retained as evidence, not decoded.
Module parsing reads exact metadata ranges rather than loading stored asset
bodies or whole pages around small pointers. A cache retains at most 256 KiB
and 1,024 ranges for one reader preparation. Cumulative uncached metadata reads
are limited to 256 MiB, checked before allocation or I/O. The module table,
decoded strings and combined logical path tree have separate admission checks;
at most 250,000 distinct path prefixes are admitted before inventory construction.
Stored bodies are range-streamed for hashing and extraction. Header/suffix observations are reused
only for the same admitted file state, with selected-path checks before publication.
See [limits and recovery](design/limits-and-recovery.md) for admission ceilings.
Synthetic tests cover the carriers and malformed declarations. Run
`node scripts/verify-bun-standalone.mjs` after `npm run build:cached` for local Bun
verification, and add `--docker` to compile a Linux fixture with `oven/bun:1.4.0`.
Bun and Docker are fixture producers only and are not installed by REA.
Layout references: [Bun 1.3.3 serializer](https://github.com/oven-sh/bun/blob/bun-v1.3.3/src/StandaloneModuleGraph.zig)
and [Bun 1.3.9 serializer](https://github.com/oven-sh/bun/blob/bun-v1.3.9/src/StandaloneModuleGraph.zig).
The current 52-byte record order can also be checked in the
[Bun 1.4.4 Rust serializer](https://github.com/oven-sh/bun/blob/bun-v1.4.4/src/standalone_graph/StandaloneModuleGraph.rs).
-107
View File
@@ -1,107 +0,0 @@
# Capability composition and ownership
REA keeps domain contracts and shared workflows separate from provider adapters
and CLI/MCP translation. A capability can have several provider identities; a
provider can contribute operations to several capabilities. Those associations
are source facts, while host availability is an observation.
## Source declarations and runtime bindings
`src/composition/capabilityDeclarations.ts` contains static provider identities,
named contract-group references and precise operation subsets. Complete groups
refer to `TOOL_CONTRACT_GROUPS` in the existing contract owner; family definitions
and provider catalogs read the same arrays, so a new group operation is included
without updating a copied provider list. Metadata imports only the group ID type
and pure metadata leaves; it must not load factories, schemas, engines or
generated catalogs. Catalog generation resolves groups and subsets against the
canonical contracts and rejects unknown groups/operations and conflicting
identities. New provider provenance must come from the implementation
that actually produces the Evidence, including format-specific capture adapters
and derived workflows.
`src/composition/capabilityServices.ts` binds typed service factories to those
capability keys. Each runtime entry owns construction, cleanup and any available
host/configuration facts or live probes. A service entry requires a close
capability; entries adapting managers retain their existing `closeAll` operation.
The host maps entries without erasing their individual service/port types. There
is no generic JSON operation dispatcher.
These two declarations have different loading needs. Static facts can be read
without loading a schema or provider graph. The full runtime composition is
materialized only when the MCP runtime needs it. One-shot CLI adapters import the
selected factory rather than constructing a complete capability host. CLI and MCP
reuse the same provider/service factories and application workflows, while
keeping their respective argument translation and lifetime.
## Resource lifetime
`CapabilityScope` registers a module's actual close capability when its instance
is composed. There is no independent server provider-close list to update.
Concurrent shutdown calls join the same cleanup attempt. A successful peer is
removed from pending cleanup; failed peers remain owned and are retried on the
next close. Synchronous throws and rejections without a reason remain failures,
and cannot prevent other peers from attempting cleanup. Shutdown seals further
ownership registration.
The host distinguishes these lifetimes:
- Host-owned service instances are released by that host.
- Caller-owned optional observations remain borrowed by SDK server instances;
their process-level owner closes them, including discarded discovery probes.
- CLI invocations use their existing finalizers to join cleanup and preserve the
original observation, cancellation and cleanup errors.
- Binary engines, bundle readers and investigation records retain their existing
owners. The `BinarySession` transition/drain barrier still coordinates target
changes, reset and disposal. A module refactor must not create independent
barriers that allow late work to repopulate reset records.
An application workflow may borrow a shared resource scope, such as byte entropy
using the host's artifact scope. Its wrapper does not acquire a second ownership
claim over the borrowed scope.
## Availability and registration
Module bindings provide host support, selected configuration and precisely typed
live availability probes through `CapabilityAvailabilityFacts`. Constructors do
not run these probes or launch engines. Session discovery invokes the existing
provider-owned probes only for advertised families; live failures remain fresh
observations. Caller policy, selected environments and injected providers retain
their existing precedence.
MCP registration remains in the MCP adapter. It consumes composed services,
record ports, admission and logging; it must not add fallback constructors that
can escape the host owner. Named contracts bind handlers and schemas. The complete
tool-family registrar map is checked by TypeScript, and catalog order never
selects a handler or schema.
## Extending a capability
1. Add or reuse the domain contract and application workflow, with explicit
Evidence provenance, coverage, unknowns and caller choices.
2. Put concrete construction in `src/composition/` and keep its default factory
lazy. Reference adapter-owned pure identity metadata from both the provider
and its static capability declaration.
3. Add the runtime binding if a host-owned service is needed. Bind cleanup and
availability/probes there, preserving borrowed versus owned resources.
4. Add the CLI/MCP argument adapters and their named registrations. A one-shot
command uses the shared selected factory and existing finalization helpers.
5. Run source-boundary, discovery, SDK-schema and relevant public workflow checks.
Exercise partial cleanup, retry, cancellation and caller-owned peers at the
applicable production seam; use actual applications/providers for routing and
complete Evidence verification. Recording ports do not prove real providers.
The constructor boundary guard covers the centralized Frida, Cutter, GDB and
Rizin constructors without rejecting legitimate type-only injected interfaces.
Other settled pure/provider boundaries continue checking type imports too. The
shared parser records module-edge erasure once. Whole-declaration type imports
are erased; inline type bindings retain their runtime module edge under
`verbatimModuleSyntax`. Static dynamic imports remain runtime edges too.
Tests belong with their semantic owner and must be
found exactly once; provider roots need the corresponding adapter discovery lane.
This is an internal composition contract. External plugin distribution, automatic
provider installation and a universal provider interface are separate concerns.
The [initial verification report](verification/capability-composition-2026-11-01.md)
records the executed public workflows, cleanup boundaries, resource measurements
and remaining provider qualification gaps for this composition increment.
+2 -2
View File
@@ -65,14 +65,14 @@ cancellation can still prevent upload.
that analysis was complete. A successful operation can retain partial analysis,
unknowns, and limitations. Exit zero is not a security verdict, proof of complete
coverage, or a policy gate for those findings. Inspect the saved Evidence and
coverage before making such a decision. See [output and exit status](getting-started/cli.md#output-and-exit-status).
coverage before making such a decision. See [output and exit status](cli.md#output-and-exit-status).
A nonzero exit can represent invalid input, unavailable tooling, an unsupported
host/capability, or another operation failure. Keep the original JSON error (when
emitted), stderr, and exit code, and use their category/message to diagnose it.
An early startup failure may leave `analysis.json` empty; do not parse that as
successful analysis. Select a supported operation or explicitly provision its
[documented prerequisites](getting-started/installation.md); do not silently switch providers,
[documented prerequisites](installation.md); do not silently switch providers,
execute the artifact, or ignore the failure to obtain a green job.
To exercise the failure path, temporarily set `ARTIFACT_PATH` to
+110 -558
View File
@@ -4,7 +4,7 @@ Use the CLI for a direct inspection or a script. It uses the same application
workflows and evidence contracts as REA's MCP server. Each CLI invocation is a
separate process; save results when you want to use them in a later command.
For a pinned, noninteractive workflow with retained reports, see
[static analysis in GitHub Actions](../ci-analysis.md).
[static analysis in GitHub Actions](ci-analysis.md).
## Run a command
@@ -19,7 +19,7 @@ Use an extracted application directory or an ASAR as the target. Static
JavaScript analysis returns the application graph and Evidence directly.
Generic `rea analyze PATH` selects this workflow for directories and `.asar`
files when neither `--provider` nor `--snapshot` is supplied. See
[JavaScript artifact reconstruction](../guides/javascript/javascript-artifact-reconstruction.md)
[JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
for results, integrity checks and coverage.
The complete application Evidence can be hundreds of megabytes for a real
@@ -39,17 +39,6 @@ pass an absolute path. REA reads and authenticates the file, then projects it
like retained Evidence, so a large file is bounded by the file reader rather
than charged as inline input.
An argument that accepts inline JSON or a JSON file path also accepts `-`, which
reads the request from standard input, so a generated request needs no
temporary file:
```bash
jq -nc '{source: {kind: "evidence-file", path: "app-evidence.json"}, view: {kind: "summary"}}' | rea inspect-analysis-view -
```
Standard input is read once as a single UTF-8 JSON document without a
byte-order mark, and it is limited to 8 MiB. Put a larger request in a file.
Local file inspection commands reject empty paths and NUL bytes before resolving
relative inputs, auxiliary files or output destinations. An empty string never
selects the working directory; use an explicit `.` when selecting that directory
@@ -59,18 +48,9 @@ For example, a 334 MB Obsidian application Evidence file yields a summary view
of about 10 KB. Replace the view with
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
module page. See
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md) for
[JavaScript application workflows](javascript-application-workflows.md) for
views, feature traces and comparisons that take the same saved Evidence.
Add `--timings` to a command to write one JSON timing record to stderr. For
example, `rea inspect-elf-notes program --json --timings > result.json` keeps
stdout as a parseable result and writes
`{"rea_timings":{"version":1,"command_ms":42}}` to stderr. `command_ms`
measures CLI command dispatch and result delivery; it is not provider-only or
analysis-only time. The flag is off by default, and the record omits stages
that REA did not measure. Root version reporting does not dispatch a command
and has no command timing record.
## JSON Lines record output
Use `--format jsonl` to consume row-oriented results one record per line:
@@ -93,14 +73,9 @@ arbitrary nested graph array.
A top-level array emits its elements in source order. An object must have one
clear `rows`, `items`, `records`, or `occurrences` array, directly or under
`page`. Evidence uses the same boundaries inside `normalized_result`.
Any other selection is refused before a record is written, including an
arbitrary report, a scalar, a nested array elsewhere, and a columnar object such
as the `inspect-sqlite-database --table` `normalized_result.rows`
(`{columns, values}`). The refusal is the caller error `invalid_request`
(`invalid_input`) and names the selection and its observed shape; the analysis
itself succeeded, so use `--format json` for the complete document. If several
collections exist, select one with `--filter-output`, for example
`normalized_result.rows` for an analysis view.
An arbitrary report or a scalar selection is refused with an actionable error;
use `--format json` for a complete document. If several collections exist,
select one with `--filter-output`, for example `normalized_result.rows`.
Object collections finish with a control record:
@@ -176,15 +151,6 @@ into the current shell:
eval "$(rea completions bash)"
```
For zsh, initialize completion before loading the hook. If your shell already
runs `compinit`, place the hook after that existing initialization:
```zsh
autoload -Uz compinit
compinit
eval "$(rea completions zsh)"
```
The Bash hook supports macOS system Bash 3.2. It uses optional completion
features when the shell provides them and falls back to portable registration.
Completion includes standard global options and the active command's options;
@@ -226,7 +192,7 @@ saving the complete original Evidence in a new bundle. Summary requires that
explicit destination; existing files are preserved. Later
`inspect-analysis-view` calls can page the exact parent from the saved bundle
without reopening the assembly. See
[CLI summary and saved member rows](../guides/managed/managed-code-analysis.md#cli-summary-and-saved-member-rows)
[CLI summary and saved member rows](managed-code-analysis.md#cli-summary-and-saved-member-rows)
for the complete command sequence. Omitted or `--detail complete` retains the
full inline response.
@@ -234,8 +200,8 @@ Choose the workflow from the artifact's contents rather than its `.exe` or
`.dll` extension. The managed parser requires PE/CLI metadata; it does not
unpack .NET single-file hosts or decode IL2CPP. For the image, type and method
names in a Unity `global-metadata.dat`, use
[IL2CPP metadata](../il2cpp-metadata.md). NativeAOT method bodies require native
analysis. See the [managed-code guide](../guides/managed/managed-code-analysis.md)
[IL2CPP metadata](il2cpp-metadata.md). NativeAOT method bodies require native
analysis. See the [managed-code guide](managed-code-analysis.md)
for supported deployment forms, selection semantics, and coverage limits.
## SQLite snapshots
@@ -243,7 +209,7 @@ for supported deployment forms, selection semantics, and coverage limits.
For a local database snapshot, use `rea inspect-sqlite-database PATH --json`.
Add `--table NAME --row-limit 20` to read a bounded sample from an ordinary
table. This command does not require a native analysis provider. See
[SQLite database snapshots](../guides/data/sqlite-databases.md) for database/WAL identity,
[SQLite database snapshots](sqlite-databases.md) for database/WAL identity,
source-file handling and row-value semantics.
## Native analysis
@@ -256,7 +222,7 @@ rea inspect-go-binary /absolute/path/to/program --json
```
The shared CLI/MCP workflow reports compiler version, modules, replacements,
build settings, original bytes and SHA-256. See [Go binaries](../guides/native/go-binaries.md)
build settings, original bytes and SHA-256. See [Go binaries](go-binaries.md)
for supported containers and absent or malformed metadata.
Inspect a Godot PCK without running the game:
@@ -267,113 +233,9 @@ rea extract-artifact game.pck --json
```
Formats 2, 3 and 4 retain header, directory and integrity facts. Compact delivery
keeps complete Evidence in the requested bundle. See [Godot PCK](../godot-pck.md)
keeps complete Evidence in the requested bundle. See [Godot PCK](godot-pck.md)
for embedded packs, batch queries and unknown encrypted coverage.
Inspect a UnityFS AssetBundle without running Unity:
```sh
rea inspect-artifact content.bundle --detail summary --evidence_output evidence.json --json
rea extract-artifact content.bundle --json
```
Format versions 6 through 8 retain header, block and directory facts with
none/LZ4/LZ4HC block decompression. See [UnityFS](../unityfs.md) for encryption
and LZMA coverage boundaries.
Inspect a WeChat mini-program package without WeChat or an unpacker:
```sh
rea inspect-artifact app.wxapkg --detail summary --evidence_output evidence.json --json
rea extract-artifact app.wxapkg --json
```
Plaintext packages retain header, table and member facts with verbatim member
bytes. `V1MMWX` packages are detected as encrypted coverage and refused on
extraction. See [wxapkg](../wxapkg.md) for layout and coverage boundaries.
Inspect an ESP32-family app or bootloader image without esptool:
```sh
rea inspect-artifact firmware.bin --json
rea extract-artifact firmware.bin --json
```
The inventory lists each segment with its ESP-IDF load classification, the
checksum and SHA-256 results, the descriptor and any Secure Boot v2 signature
blocks. See [ESP images](../esp-image.md) for recognition, coverage and refusals.
Inspect an ESP flash dump, or one read from a flash address other than 0:
```sh
rea inspect-artifact flash.bin --json
rea inspect-artifact dump.bin --target-format esp-flash-image --flash-offset 0x1000 --json
rea extract-artifact flash.bin --json
```
The inventory lists the bootloader, the partition table and each partition,
with ESP image facts for app partitions and the OTA records. See
[ESP flash images](../esp-flash-image.md) for table location, inputs and limits.
Inspect a Node.js SEA carrier without running it:
```sh
rea inspect-artifact ./application --json
rea extract-artifact ./application --json
```
See [Node.js SEA](../node-sea.md) for supported layouts, script and asset paths,
fuse observations, and the real-fixture verification lane.
Inventory and extract Intel HEX or S-record regions:
```sh
rea inspect-artifact ./firmware.hex --json
rea extract-artifact ./firmware.hex --json
```
See [Intel HEX and S-record images](../hex-records.md) for recognition, region
names, metadata and limits.
Inspect and extract stored U-Boot legacy uImage payloads and components:
```sh
rea inspect-artifact ./firmware.bin --detail summary --evidence_output firmware.json --json
rea extract-artifact ./firmware.bin --json
```
See [U-Boot uImage](../uimage.md) for verified CRCs, physical component ranges,
unknown names, resource limits and retained views. Compressed bytes stay stored.
`inspect-artifact` and `extract-artifact` also read ZIP members compressed with
Zstandard (standard method 93). They retain the shared ZIP CRC, declared-size,
overlap and archive expansion checks. The native streaming decoder keeps the
existing 128 MiB Zstandard window cap; frames requiring a larger window are
reported as a resource limit (`reason: limit`, `boundary: memory`), rather than
as archive corruption. The cap cannot be raised at runtime: recompress with a
window of at most 128 MiB, or decompress externally before retrying.
Deprecated method 20 and other unregistered ZIP codecs remain unsupported.
Inventory and extract UF2 or DfuSe regions:
```sh
rea inspect-artifact ./firmware.uf2 --json
rea extract-artifact ./firmware.dfu --json
```
See [UF2 and DfuSe images](../uf2-dfuse.md) for recognition, families, numbering,
the DfuSe suffix CRC and limits.
Inventory and extract U-Boot FIT images:
```sh
rea inspect-artifact ./boot.itb --json
rea extract-artifact ./boot.itb --json
```
See [U-Boot FIT images](../fit-images.md) for recognition, data sources, hashes
and limits.
Inspect local model declarations without loading weights:
```bash
@@ -385,7 +247,7 @@ Default hashing reads only headers and leaves full-file identity unknown.
`--hash full` streams the complete file for SHA-256. Both return typed metadata,
original tensor declarations and derived totals. MCP summaries retain complete
Evidence for filtered metadata, tensor and type-total pages; CLI compact delivery
uses `--detail summary --evidence_output FILE`. See [model files](../model-artifacts.md).
uses `--detail summary --evidence_output FILE`. See [model files](model-artifacts.md).
Inspect Unity IL2CPP metadata names without Unity or a disassembler:
@@ -402,7 +264,7 @@ method instances and `--ghidra-seeds PATH` then writes Ghidra function seeds. It
with retained Evidence; query indexed rows with filters, batch indices and
selected fields. The CLI defaults to complete Evidence;
`--detail summary --evidence_output FILE` exports the full result to a new file
and returns a compact overview. See [IL2CPP metadata](../il2cpp-metadata.md).
and returns a compact overview. See [IL2CPP metadata](il2cpp-metadata.md).
Inspect an embedded cargo-auditable Rust dependency list without installing
Rust or configuring a disassembler:
@@ -414,7 +276,7 @@ rea inspect-rust-binary /absolute/path/to/program --json
The same workflow reports package names, versions, sources, the root package,
dependency edges, and the section's file range. A missing `.dep-v0` section is
null metadata, not proof that the binary has no dependencies. See
[Rust binaries](../rust-binaries.md).
[Rust binaries](rust-binaries.md).
Inspect a PE Authenticode certificate table without executing the image or
verifying signatures:
@@ -426,7 +288,7 @@ rea inspect-pe-certificates /absolute/path/to/application.exe --json
The workflow reports `WIN_CERTIFICATE` entries and, for PKCS#7 signedData,
embedded X.509 serial, issuer, subject, and validity fields. A missing security
directory is observed absence of that table. See
[PE Authenticode certificate tables](../pe-certificates.md).
[PE Authenticode certificate tables](pe-certificates.md).
Inspect PE import and delay-load tables without executing the image:
@@ -437,20 +299,7 @@ rea inspect-pe-imports /absolute/path/to/application.exe --json
The workflow reports DLL names, named and ordinal functions, delay-load
descriptors, and a pefile-compatible import hash of the standard import table.
A missing import directory is observed absence of that table. See
[PE import and delay-load tables](../pe-imports.md).
Inspect a Windows Shell Link (`.lnk`) without resolving or executing the
target:
```bash
rea inspect-windows-shortcut /absolute/path/to/shortcut.lnk --json
```
The workflow reports header flags and FILETIMEs, IDList items, LinkInfo volume
and network names, StringData arguments and icon location, ExtraData blocks
including Tracker droid facts labelled derived, and derived observations such
as argument length and a minimized ShowCommand. See
[Windows Shell Link shortcuts](../guides/native/windows-shortcut.md).
[PE import and delay-load tables](pe-imports.md).
Inspect ELF PT_NOTE and SHT_NOTE tables without executing the image:
@@ -460,7 +309,7 @@ rea inspect-elf-notes /absolute/path/to/program --json
The workflow reports note owners, types, descriptor hashes, and GNU build-id
or ABI-tag fields when those descriptors match. A missing note table is
observed absence of those declarations. See [ELF note tables](../elf-notes.md).
observed absence of those declarations. See [ELF note tables](elf-notes.md).
Inspect an Apple `.ips` crash report without executing a target:
@@ -471,26 +320,9 @@ rea inspect-apple-ips /absolute/path/to/report.ips --json
The workflow reports first-line metadata, process/exception/termination facts,
threads, usedImages, lossless integer registers, and documented crash classes
when `bug_type` is `"309"`. Other DiagnosticReports types return no crash
result. See [Apple IPS crash reports](../apple-ips.md).
Inspect Linux kernel module metadata without executing the module or requiring
`modinfo`:
result. See [Apple IPS crash reports](apple-ips.md).
```bash
rea inspect-kernel-module /absolute/path/to/module.ko --json
```
The workflow reports ordered `.modinfo` keys (duplicate `alias` and `parm`
entries preserved), `__versions` symbol CRCs, presence of
`.gnu.linkonce.this_module` and `__ksymtab` sections, observed `.init.text`
and `.exit.text` sizes, and an appended signature trailer. `struct module` is
not decoded and the signature is not validated. Other ELF types are not
applicable. gzip, xz, and zstd containers are unsupported and name
`external_decompression` as the next probe: decompress with an external
utility, then inspect the decompressed file. See
[Linux kernel modules](../kernel-modules.md).
Inspect a CPython 3.7–3.13 `.pyc` header and nested marshal code objects (3.14 and
3.15 headers are qualified but their marshal layout is not pinned)
Inspect a CPython 3.7–3.13 `.pyc` header and nested marshal code objects
without executing bytecode or installing Python:
```bash
@@ -499,7 +331,7 @@ rea inspect-python-pyc /absolute/path/to/module.pyc --json
The workflow reports the pinned CPython version for the magic, PEP 552 hash or
mtime/size fields, nested function names, and bytecode SHA-256. Bytecode is
not decompiled. See [CPython pyc inspection](../python-pyc.md).
not decompiled. See [CPython pyc inspection](python-pyc.md).
Inspect an Erlang BEAM file without executing bytecode:
@@ -509,19 +341,10 @@ rea inspect-erlang-beam /absolute/path/to/app.beam --json
The workflow reports IFF chunk names, atom tables, exports, imports, lambdas,
and the Code chunk header. Missing optional tables stay null. See
[Erlang BEAM files](../erlang-beam.md).
Complete `rea decompile --json` Evidence can be saved and inspected through
`inspect-analysis-view` with an inline source and
`view: {kind: "native", facet: "pseudocode", offset: 0, limit: 20000}`.
The projection reads that saved observation without starting a provider;
offsets/limits count UTF-16 code units. A summary view preserves the original
selector and distinguishes null from empty text. The MCP tool additionally
supports `detail: "summary"` on its first call to retain the complete source
without returning the body inline; see [native investigation](../guides/native/native-investigation.md).
[Erlang BEAM files](erlang-beam.md).
Configure [Hopper or Ghidra](installation.md#hopper), the
[IDA adapter](../providers/ida-provider.md), or [Kuna](../kuna-provider.md) before analyzing a
[IDA adapter](ida-provider.md), or [Kuna](kuna-provider.md) before analyzing a
native target. Substitute
your target, search text and function name or address in these examples:
@@ -549,11 +372,11 @@ MCP `analyze_function` defaults to the same summary and retains the complete
dossier in the session for `inspect_analysis_view`.
Run `rea --help` or a command's `--help` for arguments and output options.
The [generated catalog](../reference/mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
The [generated catalog](mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
tools. Provider-specific target support is described in the
[native guide](../guides/native/native-investigation.md), [DOS guide](../providers/ghidra-dos.md),
[raw binary guide](../providers/raw-binary.md), [GameCube DOL guide](../ghidra-dol.md), and
[Windows Ghidra guide](../providers/windows-ghidra-p0.md).
[native guide](native-investigation.md), [DOS guide](ghidra-dos.md),
[raw binary guide](raw-binary.md), and
[Windows Ghidra guide](windows-ghidra-p0.md).
## Choose a provider
@@ -589,46 +412,20 @@ The selected provider remains bound to the session until an explicit switch
or close. Provider failures are returned with their original reason. For an
`ambiguous` selection error, choose from `details.candidate_ids`; for
`provider_unavailable`, follow `message`, `remediation.action`, and the original
reasons in `details.rejections`. Each rejection names its `origin`: `availability`
entries describe local readiness failures, including missing configuration or an
invalid existing registration; they do not establish target support. `target`
entries record a refused kind, format, architecture, or role. An availability-only
message describes the recorded rejections without claiming that every provider
is unconfigured or that target support is known.
For Hopper, Ghidra, IDA, JADX, and Kuna, run
reasons in `details.rejections`. For Hopper, Ghidra, IDA, JADX, and Kuna, run
`rea doctor --provider ID --json` to diagnose the selected engine. Other adapters,
such as `pwntools-elf`, report their configuration requirements directly and are
not accepted by `doctor --provider`. See
[task readiness](installation.md#check-readiness-for-your-task) and
[provider selection](../reference/mcp-contracts.md#identity-and-discovery).
[provider selection](mcp-contracts.md#identity-and-discovery).
The session reports work still in progress through `analysis_activity`.
A client timeout can end its wait while the provider continues analyzing.
`cleanup_incomplete` identifies resources whose shutdown or removal could not
be verified. See [MCP contracts](../reference/mcp-contracts.md) for session lifecycle and
[Ghidra first-query deadlines](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and
[Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
for import, client timeouts and recovery.
A one-shot command attempts provider-session and runtime cleanup before reporting
a failure. If cleanup remains incomplete, inspect the reported resources and
follow the error's remediation. [Same-session cleanup retry](../reference/mcp-contracts.md#retained-cleanup-retry),
reuse and `update_analysis_settings` apply to a live MCP connection.
For one-shot commands, `details.limit` reports `scope: "rea-invocation"` and
`applies_at: "next_invocation"`; set the named `environment` (or the per-call
`parameter`) for a new invocation and rerun. For example, a Ghidra decompile
deadline names `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS`.
`rea debug-with-gdb` and `rea debug-with-rizin` hold their debugger only for the
command's own process. If cleanup cannot confirm the debugger stopped, they print
`cleanup_incomplete` and exit nonzero without stopping it. `remediation.action`
names the process, as does `retained_debugger_pids` in `details.diagnostics`, or
in `details.diagnostics.cleanup_error.details.diagnostics` when the command also
failed or was interrupted. Exiting closes the debugger's command and output pipes.
REA does not establish what the debugger or its target does next, so check that
process. An MCP server that shuts down likewise exits with status 1 instead of
waiting on a debugger it could not stop. It names the process on stderr and as
`retained_debugger_pids` in its error log.
If one-shot analysis fails and closing its session also fails, the original
error details remain inline and `details.cleanup_error` describes the later
close failure. Each diagnostic retains its own reported resources; the later
@@ -642,12 +439,7 @@ an exact result when the target bytes, operation, parameters, provider and
settings match. Mutations and cursor-dependent calls are excluded from the
cache. Snapshot files are local and use owner-only permissions.
A binary CLI replay reports the current invocation's selected path while leaving
the saved Evidence and snapshot query unchanged. See
[caller-selected paths](../reference/mcp-contracts.md#caller-selected-paths)
for primary and auxiliary path fields and their relation to Evidence identity.
Importing an Evidence bundle refreshes retained request and subject display metadata.
Importing an Evidence bundle refreshes retained subject display names and paths.
It preserves cached analysis results, including observations for other targets.
Snapshot bindings that use a replaced record retain its updated display path.
@@ -655,14 +447,6 @@ Dispatching a provider metadata write invalidates the active session's snapshot
even if its reply fails: the provider may already have applied the write.
Close and reopen the target before importing or saving another snapshot.
Saved snapshots include only cache entries whose subject identifies the selected
target. Unbound observations and observations of child artifacts remain available
in the open session and its Evidence bundle; they are excluded from the target's
saved replay entries without changing their subjects.
The snapshot also retains available Evidence linked by selected records, including
transitive sources with an unbound or different subject. Those sources are not
replay entries for the selected target. Unrelated records remain excluded.
Snapshots retain eligible target-scoped question histories in full, including
mutation Evidence recorded while another target was active. Observations and
related questions must still belong to the saved target. If those dependencies
@@ -676,11 +460,6 @@ rea analyze /absolute/path/to/program --provider ghidra --snapshot /absolute/pat
```
An exact CLI cache hit is normally read before starting a provider process.
`decompile --provider ghidra --snapshot ...` also retains its single selected
Program's pseudocode result, even without an explicit document selector. Old
snapshots that contain only the Evidence need one new query to add that replay
entry. This does not retain every body from `decompile-all` or make a persistent
session's implicit document replayable.
On Windows, Ghidra first admits the selected source through its native snapshot
owner, so replay can start that provider.
In MCP, `open_binary` accepts `snapshot_path` to import a snapshot atomically
@@ -689,30 +468,6 @@ cached result. `close_binary` accepts `snapshot_path` and optional
`overwrite: true` to save before releasing provider resources. A failed save
leaves the session open so the caller can resolve the output failure.
When a validated snapshot does not match the selected binding, the error's
`details.binding_mismatch` reports confirmed target identities, provider
identities, or profile digests that differ. `snapshot` is the saved value and
`selected` is the requested value. Multiple differences can be reported. A
`profile_comparison: "not_compared"` means the profile was not available at
that boundary; it does not establish a match. A profileless snapshot can report
`workflow_scope: "incompatible"` even when its target and profile match.
Choose a matching target, provider, and profile, or analyze without the snapshot
and save a new one for the intended binding.
For a profile mismatch, the saved snapshot contains
`binding.analysis_profile.parameters`. If `binary_session` reports an active
profile whose digest matches `binding_mismatch.profile_digest.selected`, its
`analysis_provider_binding.analysis_profile.parameters` can be compared with
those saved parameters. A rejected switch leaves the previous session active,
so its profile may be different from the attempted selection; check the digest
before using it for this comparison. A configuration difference does not by
itself establish why analysis results differ.
If no target is open and you intend to inspect the selected binding, open that
target without the snapshot, then check its profile digest in `binary_session`
before comparing parameters. This may start the selected provider; the saved
snapshot remains a separate file.
If a selected snapshot's file descriptor cannot close, `open_binary` reports
`cleanup_incomplete`. `binary_session` reports `cleanup_pending` even when no
target opened. Retry `close_binary` to release the retained descriptor before
@@ -733,10 +488,6 @@ results and Evidence. These are cached observations, not a saved provider
database: only eligible exact queries can reuse a result, and new or live
queries can still require provider startup. The CLI uses the same snapshot
format and preserves these records when loading and updating it.
When an MCP session replays a provider observation, its delivered Evidence identifies
the original observation through `evidence_links` and reports that the
provider was not rerun. The original and its available source records remain
in the investigation bundle.
## Import, export and compare Evidence
@@ -751,26 +502,10 @@ rea compare /absolute/path/to/evidence/left.json /absolute/path/to/evidence/righ
rea compare left.json right.json --record_pairs '[{"left_evidence_id":"ev_<left-sha256>","right_evidence_id":"ev_<right-sha256>"}]' --json
```
Migration: replace `rea compare-bundles` with `rea compare`; the former command
has been removed. The MCP tool remains `compare_bundles`.
`rea evidence-import` performs process-local bundle authentication and
validation, reporting record and Unknown-revision counts. `unknowns_added`
counts revisions, not distinct Unknown IDs. It does not retain records in a
separate or background MCP server process. To retain records across calls in an
active MCP session, use the MCP tool `import_evidence_bundle` in that session.
`rea compare` compares records by Evidence identity and explicit pairs. It does
not read artifact contents: the inventories of two builds are unresolved
records, or one opaque change when you pair them. Use `compare-artifacts`,
described at the end of this section, to compare their contents.
`--record_pairs` accepts an inline JSON array or the path to a local JSON
file containing that array, matching MCP `compare_bundles.record_pairs`. For a
file, use `rea compare left.json right.json --record_pairs pairs.json --json`.
Pair IDs must exist on their named sides and be one-to-one. Without pairs,
one-sided membership remains unknown; REA does not infer correspondence from
position or display names.
`--record_pairs` supplies an explicit JSON array of observation pairs, matching
MCP `compare_bundles.record_pairs`. Pair IDs must exist on their named sides and
be one-to-one. Without pairs, one-sided membership remains unknown; REA does
not infer correspondence from position or display names.
Exports preserve an existing destination unless `--overwrite` is explicit.
@@ -781,66 +516,14 @@ protection still applies. A genuine pathname refusal uses `resource_constraint`
with `details.reason: path-limit` and `boundary: filesystem-path`.
For application traces and version comparisons, see
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md).
MCP [retained Evidence references](../reference/mcp-contracts.md#retained-application-evidence-inputs)
belong to one connection. Separate CLI calls can consume full saved records or
resolve exact retained IDs from an explicitly selected local bundle:
```bash
rea trace-application-feature request.json --evidence-bundle /absolute/path/evidence.json --json
```
Here `request.json` uses the same logical Evidence input as MCP, for example
`"application": {"kind":"retained-evidence","evidence_id":"ev_<64 lowercase hex characters>"}`.
Inline records and references may be mixed in the request's declared source
fields, including arrays and nested inputs. The bundle is read and authenticated
once for that command; no target is reopened, no producer is rerun, and no
record is selected by file order. Source identity, provenance, coverage and
partial or unknown facts agree with the complete inline form. When starting
from summary or inspection Evidence, select its
`normalized_result.parent_evidence_id` for a workflow requiring complete source
facts. Omit `--evidence-bundle` when all sources are inline.
The JSON workflow commands supporting this option are:
- `trace-application-feature`, `trace-javascript-semantics`,
`compare-application-versions`, `compare-javascript-export-shapes`,
`compare-source-to-bundle`, and `reconcile-javascript-runtime`.
- `import-managed-reconstruction`, `verify-managed-native-boundaries`, and
`project-managed-application-graph`.
- `project-android-application-graph`, `project-apple-application-graph`, and
`project-harmony-application-graph`.
- `compare-wasm-interfaces` and `inspect-analysis-view`.
For `inspect-analysis-view`, the request's source reference selects a parent
from the bundle for every supported view kind. Bundles exported from MCP work
for these commands. Its inline source form cannot also select a bundle.
A missing ID is refused with `details.reason: "missing"`;
supplying a bundle does not make incompatible Evidence valid for a workflow.
`compare-managed-members` keeps its two assembly paths and an optional
repeatable `--type` selection applied to both sides, and
`compare-process-captures` keeps its two capture Evidence JSON paths. The bundle
option above selects an input; producer options that write a new bundle, such
as `inspect-managed-members --evidence-bundle`, retain their existing meaning.
File selectors remain available only on contracts that explicitly support them.
Compare two artifact inventories with `compare-artifacts`. It takes the same
`left` and `right` input as MCP `compare_artifacts`: complete Evidence, or an
`evidence-file` source with an optional `evidence_id` that selects one record
from a bundle. Relative paths resolve from the working directory:
```bash
rea inspect-artifact old.zip --json > old.json
rea inspect-artifact new.zip --json > new.json
rea compare-artifacts '{"left":{"kind":"evidence-file","path":"old.json"},"right":{"kind":"evidence-file","path":"new.json"}}' --json
```
A one-shot `compare-artifacts` command keeps no session Evidence, so it refuses a
`retained-evidence` source with `details.reason: "missing"`. Select the record
from a saved bundle instead.
For an artifact summary selected from a bundle, include its complete parent
inventory in the same bundle. The command resolves that parent from the
authenticated file records; a summary-only file cannot supply missing facts.
[JavaScript application workflows](javascript-application-workflows.md).
MCP [retained Evidence references](mcp-contracts.md#retained-application-evidence-inputs)
belong to one connection. Separate CLI calls can consume full saved records.
`inspect-analysis-view REQUEST.json --evidence-bundle BUNDLE.json` resolves
the request's `retained-evidence` source ID from that authenticated bundle,
including bundles exported from an MCP session. The same view JSON works for
every supported parent kind; no target is reopened and no record is selected
by file order. Omit `--evidence-bundle` when the request supplies inline Evidence.
## Import historical source
@@ -856,7 +539,6 @@ Import an older source tree to compare with the current artifact:
```bash
rea import-reference-source /absolute/path/to/source
rea import-reference-source /absolute/path/to/source --detail summary --outline-depth 2 --json
rea import-reference-source /absolute/path/to/source --exclude-paths '["vendor/","*.generated","!keep.generated"]'
```
The default (`--detail full`) returns the complete `HistoricalSourceGraph`.
@@ -990,23 +672,19 @@ for percent signs, query marks and fragments.
Historical-source import requires safe no-follow file opens on Linux or
macOS. Native Windows returns `unsupported_host`; use Linux REA inside WSL
or another supported host. See
[source-to-bundle comparison](../guides/javascript/javascript-application-workflows.md#historical-source-to-bundle-comparison)
[source-to-bundle comparison](javascript-application-workflows.md#historical-source-to-bundle-comparison)
for mapping identities, inferences and unknowns.
## Capture runtime behavior
Choose the guide for the target and intended interaction:
- [Browser observation](../guides/web/browser-observation.md): inspect a selected page through CDP.
- [Browser scenarios](../guides/web/browser-scenario-contract.md): run declared interactions and capture their results.
- [Electron observation](../guides/javascript/electron-observation.md): inspect an Electron renderer or capture an application scenario.
- [Electron fuses](../electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
- [Node/Electron Inspector](../guides/javascript/javascript-runtime-observation.md): record script locations and execution contexts.
- [Process capture](../guides/runtime/process-capture.md): run an executable and compare terminal, exit and filesystem observations.
- [Desktop application capture](../desktop-application-capture.md): launch a Linux desktop application and record its windows and home-directory state at checkpoints while it is used.
- [Desktop application state](../desktop-application-state.md): read a capture folder into the application's state files, typed views of each version and kinds of change.
- [Desktop application export](../desktop-application-export.md): write a capture folder as an agentenv computer app bundle with its class, skill and seed.
- [Desktop application tasks](../desktop-application-tasks.md): describe how a capture's files changed, then validate an agent-written task against the capture and export it with its verifier.
- [Browser observation](browser-observation.md): inspect a selected page through CDP.
- [Browser scenarios](browser-scenario-contract.md): run declared interactions and capture their results.
- [Electron observation](electron-observation.md): inspect an Electron renderer or capture an application scenario.
- [Electron fuses](electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
- [Node/Electron Inspector](javascript-runtime-observation.md): record script locations and execution contexts.
- [Process capture](process-capture.md): run an executable and compare terminal, exit and filesystem observations.
Runtime requests name the target and actions. Launched targets run with your
user permissions; consult the chosen guide for host requirements and effects.
@@ -1022,30 +700,14 @@ rea mcp --http --port 8765
```
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the environment,
then to `127.0.0.1` and an ephemeral port. A bind outside loopback requires
`MCP_ALLOWED_HOSTS` or `--allowed-hosts`, a comma-separated list of client host
names or IP addresses without URLs or ports.
Repeat `--allowed-hosts` to append names, or put multiple names in one
comma-separated value. All occurrences are validated together and deduplicated
after hostname normalization; the explicit list replaces `MCP_ALLOWED_HOSTS`.
For example:
```sh
rea mcp --http --host 0.0.0.0 --allowed-hosts 127.0.0.1,rea.lan
```
Requests with a different `Host` or browser `Origin` receive `403`; a missing
`Origin` is accepted. The announced client URL uses the bound hostname when
admitted, otherwise the first declared hostname, with the actual bound port.
It is written to stderr as one JSON log line (`"msg":"MCP transport listening"`) whatever `REA_LOG_LEVEL`
then to `127.0.0.1` and an ephemeral port. The bound URL is written to stderr as
one JSON log line (`"msg":"MCP transport listening"`) whatever `REA_LOG_LEVEL`
is set to; nothing is written to stdout. `rea mcp --http --help` lists the accepted options. The listener serves
one client at a time: a session ends with the client's `DELETE`, and a client
that connects while another session is open takes it over, so a client that
exited without ending its session needs no restart. The listener runs until
SIGINT or SIGTERM and does not stop when stdin closes. See
[transports](../reference/mcp-contracts.md#transports) for the session, `Host` header and
[transports](mcp-contracts.md#transports) for the session, `Host` header and
shutdown contract. Client registrations written by setup and the MCP registry
entry keep stdio.
@@ -1070,54 +732,20 @@ answers every method with JSON-RPC `-32601`. The stopped document's `state`
member is `{"served": true, "evidence_id": …}`,
`{"served": false, "reason": "replay-only"}` or
`{"served": false, "reason": "unavailable", "code": …, "message": …}`; see
[serve from the state](../agent-environments.md#serve-from-the-state).
[serve from the state](agent-environments.md#serve-from-the-state).
```sh
rea environment-serve manifest.json --evidence capture.json --port 8765
```
`--host` and `--port` resolve like `rea mcp --http`: flags, then `MCP_HOST` and
`MCP_PORT`, then `127.0.0.1` and an ephemeral port. A non-loopback bind requires
`MCP_ALLOWED_HOSTS` with the same hostname rules and Host/Origin gates as above.
`environment-serve` uses this environment setting; `--allowed-hosts` belongs to
`rea mcp --http`. For example:
```sh
MCP_ALLOWED_HOSTS=127.0.0.1,rea.lan rea environment-serve manifest.json --evidence capture.json --host 0.0.0.0
```
`ENVIRONMENT_NAME` renames
`MCP_PORT`, then `127.0.0.1` and an ephemeral port. `ENVIRONMENT_NAME` renames
the served environment; set but blank, it is refused as `configuration_invalid`
like a blank `MCP_HOST`. The admitted client URL is written to stderr as one JSON log line
like a blank `MCP_HOST`. The bound URL is written to stderr as one JSON log line
(`"msg":"MCP transport listening"`). See
[agent environments](../agent-environments.md) for what the state and replay
[agent environments](agent-environments.md) for what the state and replay
return and do not establish.
## Build an environment bundle
`rea environment-build <capture> [dir] [--name <name>] [--template-rule <rule> ...]
[--format-rules-only] [--package-tarball <path>] [--capture-timeout-ms <ms>] [--dry-run]` builds the bundle
`environment-export` writes from one recorded capture: a `.har` file, which it
inspects as `inspect-web-network-capture` does, or capture Evidence JSON from
`inspect-web-network-capture` (HAR) or `capture-browser-scenario`. It projects
the environment manifest from the in-memory Evidence with the opt-in rules
`after-uid-literal`, `opaque-token`, `sibling-variance` and `prefix-variance`
unless `--template-rule` or `--format-rules-only` selects others, prepares the
state, and writes the bundle; every file is byte-identical to the three-step
path's when that path lists the same rules in the views' order and exports
under the build's name. A loopback or IP-literal capture is named after its file unless `--name`
is given. It prints one JSON report: the capture and manifest identities, the
state, a `coverage` summary of which tools answer from the state, which replay
and why each write is withheld with a hint, the files, warnings and the next
command. `--dry-run` writes nothing and may omit the directory.
```sh
rea environment-build grafana.har ./grafana --json
```
See [agent environments](../agent-environments.md#build-a-bundle-in-one-step)
for the report, the hints and what the bundle ships.
## Export an environment bundle
`rea environment-export <manifest.json> --evidence <capture.json> <dir>
@@ -1148,26 +776,9 @@ name Windows reserves (`con`, `nul`, `com1` and the like) is refused. The
command prints one JSON document listing the written files with their sizes,
the pinned package source, the served `tool_count`, and `state` in the shape
of the serve command's stopped document. See
[agent environments](../agent-environments.md#export-a-bundle) for the folder
[agent environments](agent-environments.md#export-a-bundle) for the folder
layout and what the smoke task checks.
`rea environment-export` prints the written files and exits `0` on success.
Input-pair refusals, an occupied destination, a missing package tarball, and
invalid labels fail before writing output. Host permission failures identify
whether reading or writing was denied and retain the affected path.
A failure during writing can leave a partial folder. Inspect it before removing
it and retrying; the next attempt refuses a non-empty destination. When one
file caused the failure, the diagnostic names that file. A folder-sync failure
can report only its cause.
If `details.partial_observation.kind` is `environment-bundle-publication`, the
bundle was already published. Its `directory` and `files` record the committed
output, even when a later identity check or cleanup failed. Inspect that bundle
and resolve the reported failure before using or replacing it; the receipt is
not an instruction to delete published output. Cleanup resources remain a
separate obligation.
## Compare an environment with its application
`rea environment-compare <manifest.json> --evidence <capture.json> --probe <probe.json> [--probe <probe.json> ...]`
@@ -1178,7 +789,7 @@ answer is compared with the application's after ids are mapped and clocks and ge
makes no network call. It prints the report and exits `0`, or prints a structured `invalid_request` error and exits `1`
for the same pair refusals as `environment-serve`, a probe that is not capture Evidence, or a browser probe captured
without response bodies, or without the request bodies its writes need. See
[agent environments](../agent-environments.md#measure-fidelity-against-the-application) for the report.
[agent environments](agent-environments.md#measure-fidelity-against-the-application) for the report.
```sh
rea environment-compare manifest.json --evidence capture.json --probe probe.json --json
@@ -1235,10 +846,6 @@ such as `UNSUPPORTED_OUTPUT_COMBINATION` for `--json` with `--token-limit`. Use
Commands reject surplus positional arguments before execution, including commands
that accept no positionals. Use `rea <command> --help` for the declared argument
shape; a final variadic argument continues to accept multiple values.
`rea mcp add` accepts options only and rejects unknown options before reading or
backing up client settings or starting registration. Its `--agent` and
`--command`/`-c` options accept spaced and equals forms. Values must be nonempty; a following flag does not satisfy a spaced option's
missing value.
`rea mcp doctor --json` checks production MCP startup and reports child logs in
`diagnostics`. It drains stderr while retaining at most its first 64 KiB of raw
@@ -1256,58 +863,17 @@ Malformed format values and token windows fail with `INVALID_OUTPUT_VALUE` in th
selected structured format. Token limits and offsets must be nonnegative safe
integers; zero selects an empty window. These checks run before command execution. `--json` is indented on a terminal and compact when piped or
redirected, which keeps large results smaller for agents and scripts; pipe
through `jq .` for indented files. A formatting refusal exits nonzero even when
the underlying operation succeeded.
through `jq .` for indented files. Output selection and formatting do not change
operation status.
`--format md` escapes backslashes, pipes, and inline Markdown syntax in table
headers and cells so source text is displayed literally, and renders cell line
breaks as `<br>`. Underscores between letters or digits, as in `source_index`,
cannot form emphasis and stay unescaped. These are presentation changes; retained
`--format md` escapes backslashes and pipes in table headers and cells, and
renders cell line breaks as `<br>`. These are presentation changes; retained
Evidence and JSON output keep the original values. Nested objects and arrays
use JSON inside their owning Markdown cells, retaining the field names and
values instead of replacing them with an object label. Table widths are measured
iteratively, so an admitted row page is not limited by the JavaScript function
argument count.
Markdown refuses selected NUL characters and unpaired UTF-16 surrogates because
renderers or UTF-8 output would substitute them. TOON refuses selected unpaired
surrogates for the same UTF-8 reason. Use JSON or YAML to retain those values.
An authored replacement character and valid supplementary characters are not
treated as malformed.
`--format csv` and `--format tsv` export one selected row collection. Select an
exact array path with `--filter-output` and provide comma-separated direct
scalar columns with `--columns`; column order is the output column order:
For an artifact inventory request that selects the `occurrences` collection
and includes `logical_path` and `artifact_id` in its view fields:
```bash
rea inspect-analysis-view request.json --format csv \
--filter-output normalized_result.items --columns logical_path,artifact_id
```
The first line is a REA metadata record,
`# rea-delimited-v1 <JSON>`. It carries the selected path, view request, limitations and unknowns,
coverage, source coverage and completeness, command, and duration. A metadata
value of `null` means that the selected row collection did not provide that
fact; it does not mean complete coverage. The following line is the header.
CSV records use CRLF line endings; TSV records use LF. Cells containing the
delimiter, quotes, or line breaks are quoted and embedded quotes are doubled.
Values are emitted as UTF-8 without Unicode normalization. Missing and `null`
cells are empty; booleans and finite numbers use their ordinary text forms.
String cells and headers that begin with spreadsheet formula syntax are refused
with a structured error; unpaired surrogates in selected cells or headers are
also refused. JSON retains those values unchanged.
Delimited output does not flatten nested Evidence or nested field values.
It returns a structured error and a nonzero status if the filter does not
select exactly one row array, a row is not an object, or a selected cell is
nested or formula-leading. JSON remains the complete canonical output; use it
to retain fields outside the selected columns or preserve exact formula-leading
text. Token counting and token windows are not
available for CSV/TSV because slicing would corrupt records or metadata.
`--filter-output` selects fields from successful command data. Classified failures
retain their complete typed diagnostic projection even when the selector matches
a field such as `code` or `message`, or names a missing field. `--full-output`
@@ -1326,10 +892,10 @@ Cutter when available. This is separate from `--format json`, which selects the
CLI result representation; the global `--json` flag selects that same CLI
representation.
Cutter no longer accepts command-local `--json=true` or `--json=false`.
Replace these with `--json-output=true` or `--json-output=false`, respectively.
Combine `--json-output` with bare `--json` when both Cutter command output and
the CLI result should use JSON; either argument order is accepted.
Cutter still accepts deprecated `--json=true` and `--json=false` compatibility
values. When both forms are present, `--json-output` takes precedence in either
argument order. Bare `--json` retains its global meaning of selecting JSON CLI
output formatting.
| Status | Meaning |
| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -1355,21 +921,48 @@ server resources. It preserves a primary listener diagnostic, or a
tool count and Evidence identifiers. The successful `stopped` document is
returned only after cleanup is confirmed. The same in-process server owner
retains failed stages for retry; the CLI exposes no separate retry command.
`rea environment-build` is one-shot too: it prints its report and exits `0`,
or prints the refusing stage's structured error and exits `1`, writing nothing
when the refusal comes before the bundle is written. SIGINT cancels it with a
`cancelled` error; a cancellation before the bundle stage writes nothing.
A `cancelled` result does not by itself determine the exit status. Commands
that handle SIGINT or SIGTERM assign `130` or `143` after their operation and
cleanup settle; their structured result can be printed before that status is
assigned. Read the process status and typed result separately.
`rea environment-export` is one-shot: it prints the written files and exits
`0`, or prints a structured `invalid_request` error and exits `1` for the same
pair refusals, a non-empty or file-occupied directory, a missing package
tarball or a name that is not a kebab-case label or is a Windows device name,
writing nothing in each case. A file occupying the destination is reported as
requiring a directory; identity-change diagnostics are reserved for admitted
paths that fail later identity checks. Host permission failures identify whether
input reading or bundle writing was denied and retain the reported path.
A write that fails after the first file (a permission, disk-space or
pathname-length failure on a later path) propagates as the host's failure,
naming its cause and, when it failed on one bundle file or the package
tarball, that file. A failure while syncing the finished folder names only its
cause. Either way the partial folder stays in place for inspection; remove it
before retrying, since the next attempt refuses the non-empty directory.
Commands with `--evidence_output` save the complete Evidence bundle before
delivering the selected view. For committed-file receipts, retained partial
Evidence and missing source records, see
[Evidence publication and source recovery](../reference/mcp-contracts.md#evidence-publication-and-source-recovery).
Other outputs have their own receipt shapes; for example,
[environment export](#export-an-environment-bundle) reports a published directory.
A `cancelled` result does not by itself determine the exit status; it depends on
the command's signal handling. Commands routed through the shared
`withCommandCancellation` handler in the CLI set status `130` for SIGINT or
`143` for SIGTERM after their operation settles. Current source registrations
use that handler for process capture and comparison (`capture-process`,
`compare-process-captures`), JavaScript application analysis (`analyze` when it
selects the JavaScript lane, and `analyze-javascript-application`), selected
browser, web and Electron commands (including `analyze-web-bundle`,
`observe-web-execution`, `capture-browser-scenario` and
`capture-electron-scenario`), and the platform and provider command families
for Android/ADB, Apktool/JEB/Flutter, firmware, PCAP and signal analysis,
WASM/EVM, Go/Rust and binary diagnostics, Frida instrumentation, the Cutter
command, objdump, Rizin and GDB, and the agent environment server
(`environment-serve`). The handler also wraps `doctor`,
`inspect-sqlite-database`, `recover-javascript-sources`,
`inspect-il2cpp-metadata`, and `inspect-desktop-application`. In these paths, a returned structured
`cancelled` result can be printed before the wrapper assigns the
signal-derived status. This describes the current source routing; it is not a
claim that every command or provider runtime has been independently exercised.
Commands with `--evidence_output` select their delivered view, then publish the
complete Evidence bundle atomically. Cancellation before publication leaves no
new file. A publication that has committed can win a later cancellation; the
`cancelled` result then keeps the written path and byte count under
`details.diagnostics.evidence_bundle`. If staging cleanup stays incomplete
after commit, the `cleanup_incomplete` error reports the same file as
`details.published_path` and `details.bytes`.
When the reader of piped output closes early, as `rea ... | head` does, the
CLI stops writing, prints no error and exits `141` (`128 + 13`, the status of a
@@ -1382,11 +975,7 @@ status with the analysis failure status, so it exits `1`. Setup has its own cont
structured result and process status as separate signals when scripting
cancellation.
`setup --dry-run` returns `planned` and exits `0`. For `setup` and
`environment-build`, explicit `--dry-run=true` and `--dry-run=false` are
accepted; empty or other values fail with `PARSE_ERROR` before any changes.
Every supplied value is checked even when a later occurrence overrides it.
A cancelled setup also exits
`setup --dry-run` returns `planned` and exits `0`; a cancelled setup also exits
`0`. Setup returns `1` for `needs_confirmation` or `needs_human`.
`doctor` returns `1` when required checks in its selected readiness scope fail;
unavailable optional providers remain informational for unrelated tasks.
@@ -1408,40 +997,3 @@ Counting retains the existing bounded formatted-string allocation after capacity
admission; omit the flag to stream large JSON documents without materializing
the complete formatted text. Exhausted token windows display an empty range at
the total token count and have no next offset.
### `inspect-devicetree`
Inspect an explicit local FDT v16/v17 blob without execution or an external tool.
Returns header, reservations, node/property tree, derived standard values and
selected phandle references. See [devicetree inspection](../devicetree.md) for
limits and retained-view pagination.
```bash
rea inspect-devicetree /artifacts/board.dtb --json
rea inspect-devicetree /artifacts/board.dtb --detail summary --evidence_output /artifacts/board-evidence.json --json
```
### PE TLS and load configuration
```sh
rea inspect-pe-load-config /absolute/path/to/application.exe --json
```
Reports TLS callbacks and size-covered load-config fields without execution.
Use `--max-callbacks` and `--max-file-bytes` to set read budgets. See
[PE TLS and load configuration](../pe-load-config.md).
### Binary hardening
```sh
rea inspect-binary-hardening /absolute/path/to/program --json
```
Reports ELF, PE, and Mach-O hardening rows without execution. A missing import
is `unknown`, not `disabled`. Use `--max-file-bytes` to set the read budget. See
[binary hardening](../binary-hardening.md).
Bun standalone executables use the existing `inspect-artifact` and
`extract-artifact` commands to inventory and materialize embedded source,
sourcemaps and bytecode without running the executable. See
[Bun standalone executables](../bun-standalone.md) for layout coverage and limits.
@@ -6,85 +6,6 @@ for the reviewed setup transaction and [client IDs](installation.md#supported-ag
The versioned verification notes below describe their stated runs; live service
acceptance is verified only where explicitly recorded.
## Launchers and PATH
A client launched from a desktop app, IDE or service can inherit a different
`PATH` from your terminal. An absolute path to `rea-agents` does not resolve its
interpreter: the npm launcher's `#!/usr/bin/env node` still searches `PATH` for
Node. Running that command successfully in a terminal does not establish that
the client can launch it.
For JavaScript installations, setup registers the selected runtime
executable and REA script by absolute path. On POSIX, it prefers executable
`node` or `bun` and REA aliases on setup's `PATH` only when they resolve to the
same selected files. This preserves manager-owned aliases across version updates;
otherwise setup uses the selected paths directly. This also applies when setup
runs through a package runner: retain the selected cache entry and runtime while
the registration uses them. Clearing either makes startup fail without resolving
another package from the client's project. Restore the same installation before
repairing it; an unidentified registration remains untouched. See
[installation and recovery](installation.md#package-runners-and-global-installation).
Setup forwards supported, explicitly selected REA settings into client
configuration, but does not copy unrelated ambient environment values. To refresh an older registration, preview
`rea setup --client <client-id> --skill=false --dry-run`, then run the same command
without `--dry-run` and approve the displayed changes. Setup preserves unrelated
settings, backs up changed configuration and leaves an aligned registration
unchanged. Restart the client after applying the update.
If a runtime manager removes the registered executable and no surviving alias
was available, doctor reports its identity as unknown and setup retains the
registration. REA does not infer that an arbitrary missing executable was Node
from its script argument. Back up the client's configuration, verify the absolute
path and supported version of the runtime you intend to use, then update only
the runtime command, preserving the REA script arguments and other settings.
Rerun setup to review the registration and restart the client.
`rea mcp doctor --json` tests the production MCP adapter using the runtime
executable and environment of the terminal running doctor. It does not launch
the client's configured command or reproduce the client's environment. If doctor
passes but the client disconnects, inspect the client's startup error and registered
launcher; see [verification scope](../development/client-verification.md#launch-environment-verification).
`rea doctor` has the same limit for provider checks, and reports a provider setting
the registration's file does not carry as a
[non-blocking unknown](installation.md#check-readiness-for-your-task).
## Setting launch variables in PowerShell
Shell examples that prefix a command with `REA_MCP_INPUT_SCHEMA_PROFILE=compact`
use POSIX shell syntax. In PowerShell, set the process environment before
launching REA:
```powershell
$env:REA_MCP_INPUT_SCHEMA_PROFILE = "compact"
rea mcp doctor --json
```
The assignment applies to the current PowerShell process and commands it launches.
Restore the previous value when finished, or use
`Remove-Item Env:REA_MCP_INPUT_SCHEMA_PROFILE` if it was previously unset. It does
not update an already running MCP server or desktop client.
To select this profile for an agent, set `REA_MCP_INPUT_SCHEMA_PROFILE` in that
client's existing REA registration `env` object (`environment` for OpenCode) and
restart the client, preserving its other settings. A successful terminal doctor run does not verify the client's
launch environment. See [launch-environment verification](../development/client-verification.md#launch-environment-verification)
and the client-specific instructions below. REA setup forwards supported selections,
including `REA_MCP_INPUT_SCHEMA_PROFILE`, from its environment into the registration;
it does not copy unrelated environment values. Restoring the shell's previous
value does not undo a selection already saved by setup.
## Runtime settings and client deadlines
`update_analysis_settings` changes curated defaults inside the running REA
server. It does not edit the client registration, extend the client's tool-call
deadline, or recompute timeout values previously derived by `rea setup`. When
raising a REA startup or operation deadline, separately ensure the client allows
enough time for the full request. Use `inspect_analysis_settings` for the running
server's defaults; `rea doctor` checks launch configuration and cannot observe
another process's runtime overrides. See the
[settings design](../design/limits-and-recovery.md) for ownership and lifetime.
## Client tool names and discovery
REA advertises canonical names such as `binary_session`. The MCP host decides
@@ -102,97 +23,11 @@ does not repair a client's naming behavior or infer permission drift from an
unfamiliar prefix. The unavailable fork reported in
[#5512](https://github.com/morluto/rea/issues/5512) has not been reverified.
<a id="hotcold-catalogs"></a>
## Hot/cold catalogs
For a client that supports MCP catalog refresh, set
`REA_MCP_TOOL_CATALOG_PROFILE=hot-cold` in REA's server environment and restart
the connection. REA then advertises a small initial catalog and loads other
named tools through `discover_tools`. Setup preserves this explicit setting.
See the [discovery contract](../reference/mcp-contracts.md#hotcold-catalog-discovery) for
pagination, identity and family-scope semantics.
Analysis tools, including `inspect_analysis_view`, load on demand. For example,
load `inspect_api_contract` and supply its `view` to answer an initial OpenAPI
or GraphQL question directly. When later questions need retained facts, load
`inspect_analysis_view` and use the complete `parent_evidence_id`; source files
need not be reopened. If both tools are needed from the start, request both
names in one `discover_tools` batch to avoid another catalog refresh. The same
pattern applies to selected Playwright trace and endpoint-comparison results.
This defers unused schema delivery and client validator preparation; it does
not reduce parsing work, alter Evidence or shorten requested facts. Clients
without catalog refresh should keep the default `full` profile.
Codex CLI 0.165.0 requires its experimental `mcp_tool_list_changed` feature for
this workflow. Merge these entries into the existing `~/.codex/config.toml`
without replacing other feature or REA settings:
```toml
[features]
mcp_tool_list_changed = true
[mcp_servers.rea.env]
REA_MCP_TOOL_CATALOG_PROFILE = "hot-cold"
```
The server setting alone does not enable client refresh. If a successful
discovery call is followed by a missing tool, check the client's refresh feature
and tool allow/deny lists. Clients without refresh should use the default
`REA_MCP_TOOL_CATALOG_PROFILE=full`; reconnect after changing it. The
[FastMCP HTTP trial](#fastmcp-and-unsloth-sse-catalog-limits) below covers
explicit refreshes; other clients' automatic refresh remains unverified.
## FastMCP and Unsloth SSE catalog limits
If an HTTP client reports `Server-sent event exceeded the 1048576 byte limit`
during `tools/list`, merge these settings into the **REA server process's**
environment, then reconnect:
```text
REA_MCP_INPUT_SCHEMA_PROFILE=compact
REA_MCP_OUTPUT_SCHEMA_PROFILE=compact
REA_MCP_TOOL_CATALOG_PROFILE=full
```
This advertises all tools without requiring catalog refresh. Compact output
schemas widen client-side validation; REA still validates complete results
against the canonical contracts. See [output schemas](../reference/mcp-contracts.md#output-schemas).
On main `3e0599a94`, a real FastMCP 4.1.6 / Python MCP 2.3.0 HTTP client with
the default 1 MiB SSE event limit rejected the 4,108,975-byte full catalog.
The settings above delivered all 240 tools in 611,305 bytes and a
`binary_session` call succeeded. These are measured SSE response sizes for
that catalog, not a size guarantee for future catalogs or other profiles.
For a client that refreshes tools, combine `hot-cold` with the compact schema
settings to reduce initial context further: the same trial delivered six
initial tools in 33,903 bytes, then explicitly refreshed through 34 loaded
tools. Hot/cold alone later exceeded the SSE limit during refresh. Follow
the [refresh requirements](#hotcold-catalogs).
The Unsloth report [#9093](https://github.com/morluto/rea/issues/9093) fails at
catalog delivery; the Unsloth application itself was not exercised in this
trial. Schema compaction does not bound analysis results. Continue to use
summary/detail selection and tool-specific paging for larger results; client
receive limits and full Evidence access still apply.
## Fiddler Everywhere alongside REA
Connect Fiddler Everywhere's MCP server separately for HTTP traffic debugging,
preserving your REA client entry. See the
[setup instructions and public skills](../guides/web/web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
## VS Code
`rea setup --client vscode` configures the default VS Code user profile's
`mcp.json`. Setup honors `VSCODE_PORTABLE` and `VSCODE_APPDATA` when they are
present in its environment. It cannot infer a running window's named profile
or `--user-data-dir`; configure that profile separately when needed.
`rea setup --client copilot_cli` instead configures Copilot's user
`mcp-config.json`. VS Code can forward eligible profile servers to Copilot
sessions; see the [VS Code MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration).
[setup instructions and public skills](web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
## GitHub Copilot CLI
@@ -219,7 +54,7 @@ tool inventory on Linux using a loopback model fixture. After setup, add
settings, then restart the client. Compact advertisements can reduce nested
schema constraints; REA still validates complete canonical inputs. Full-profile
`gpt-5.4` metadata is also verified locally. Live model API acceptance and
Windows remain unverified; see [native client verification](../development/testing.md).
Windows remain unverified; see [native client verification](testing.md).
## Claude Desktop
@@ -230,7 +65,7 @@ in the file the installed app reads:
- Without Claude Desktop's MSIX package storage, setup uses
`%APPDATA%\Claude\claude_desktop_config.json`.
- The legacy MSIX storage layout (package family `Claude_pzs8sxrjxfjjc`, installed under
- The MSIX build (package family `Claude_pzs8sxrjxfjjc`, installed under
`C:\Program Files\WindowsApps\Claude_<version>_x64__pzs8sxrjxfjjc`)
virtualizes AppData writes. Once it has saved its configuration, it reads
the package-private copy
@@ -239,11 +74,6 @@ in the file the installed app reads:
private copy when it exists. Before the package has saved one, they use the
`%APPDATA%` file it still reads; with neither file, setup creates the
private copy.
- If `%LOCALAPPDATA%\Claude-Data\claude_pzs8sxrjxfjjc` exists, setup, doctor
and uninstall report an unresolved location. Migrated MSIX installations
have been observed reading `claude_desktop_config.json` in this directory.
Quit Claude Desktop and verify its active configuration before registering
manually. See [#8968](https://github.com/morluto/rea/issues/8968).
- If `%LOCALAPPDATA%\Packages` holds a `Claude_<publisher ID>` package from
another publisher, an `AnthropicPBC.Claude_<publisher ID>` package, or cannot
be read, REA cannot establish which file Claude
@@ -329,26 +159,15 @@ and Claude Code reports that autocompact is thrashing. To avoid this:
`mcpServers.rea` entry in `~/.claude.json` (`$CLAUDE_CONFIG_DIR/.claude.json`
when that variable is set), then restart Claude Code. Setup keeps this
override.
- To remove entire tool families from the upfront catalog, run
`rea setup --client claude_code --tool-families <ids>` with the families the
workflow needs (see [the advertising scope contract](../reference/mcp-contracts.md#tool-family-advertising-scope)).
Out-of-scope tools are not advertised and their calls fail with the SDK's
disabled-tool error, so widen the selection when a task needs another
family. Setup writes the scope into the entry's `env` as
`REA_MCP_TOOL_FAMILIES`; rerun setup with a different selection to refresh
it, then restart Claude Code.
The sizes above come from REA's own `tools/list`. A live Claude Code session
against DeepSeek's API is unverified.
For a client whose stdio reader rejects the complete catalog frame, set
`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "compact"` to bound every advertised output
schema, or `"off"` to omit them, in REA's `env` object, then restart the client.
Both keep canonical result validation and complete structured results; `compact`
keeps a widened root shape and `off` advertises nothing. On catalogs large
enough that full input schemas alone approach the frame limit, also keep the
input profile at its `compact` default. See [MCP output schema
profiles](../reference/mcp-contracts.md#output-schemas).
For a client whose stdio reader rejects the complete catalog frame, add
`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "off"` to REA's `env` object and restart the
client. This omits only advertised output schemas; REA still validates complete
structured results against its canonical contracts. See [MCP output schema
profiles](mcp-contracts.md#output-schemas).
## Kimi Code
@@ -380,21 +199,6 @@ registration but does not contribute to the derived tool timeout; raise
`toolTimeoutMs` to cover a longer selected JADX deadline.
This headroom is a practical default, not a guarantee for every analysis
duration. Setup preserves positive timeout values already in the `rea` entry.
Setup also forwards explicitly selected input/output schema profiles,
`REA_MCP_MAX_RESPONSE_BYTES`, `REA_MCP_MAX_INPUT_ELEMENTS`, the Windows ChangeTime settle settings `REA_WINDOWS_CHANGE_TIME_DRIFT_MS`, `REA_WINDOWS_CHANGE_TIME_SETTLE_POLL_MS`, and `REA_WINDOWS_CHANGE_TIME_SETTLE_MAX_POLLS`, provider paths and supported Ghidra, JADX and Kuna
settings. It also forwards explicit executable and resource selections for ADB,
apktool, packet capture, GDB/Rizin/objdump, Cutter, WABT, Wakaru, firmware and
signal tools, browser inspection, and pwntools/pwndbg. This includes their
supported prlimit overrides, Darwin Swift compiler/SDK/module-cache selections,
and compile timeout, logging level and explicitly configured reference-secret
patterns. A provider being unavailable does not discard its selected paths;
doctor reports its readiness separately. Explicit selections override discovered
defaults, including when setup runs again with a changed selection. Unrelated
environment variables, process ownership tokens and one-shot target paths are not
copied. For an explicitly configured startup target, see the
[initial-target settings](../reference/mcp-contracts.md#initial-target-at-startup). Existing
client environment overrides remain intact unless setup supplies the same key.
You can edit `startupTimeoutMs` and `toolTimeoutMs` there; doctor keeps positive
caller-selected values aligned and reports a non-blocking advisory when a tool
timeout is shorter than the known provider allowance.
@@ -415,22 +219,6 @@ Kimi Code also discovers personal skills in `~/.agents/skills` and project-local
and verifies the matching skill files through the normal shared-skill flow.
Start a new Kimi Code session to pick up MCP and skill changes.
## LM Studio
For [LM Studio](https://lmstudio.ai/docs/app/mcp), setup writes `mcpServers.rea`
into `mcp.json` in the LM Studio home directory. Setup finds that directory as
LM Studio does: the path stored in `~/.lmstudio-home-pointer`, otherwise
`~/.cache/lm-studio` when it exists, otherwise `~/.lmstudio`. Setup only reads
the pointer file. LM Studio parses `mcp.json` as strict JSON, so setup rejects
a file with comments, trailing commas, or a byte order mark without a backup
or a write. The entry carries `command`, `args`, and `env`, and setup keeps
every other server. Local models often have small context windows, so setup
selects `REA_MCP_INPUT_SCHEMA_PROFILE=compact` unless the stored entry or the
setup environment selects a profile explicitly. Doctor reports an entry with no
explicit profile as stale, as for Kimi Code. LM Studio 0.4.26 on macOS loaded
this registration, and a local model called an REA tool through it. Linux and
Windows remain unverified.
## Pi
For [Pi](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/mcp.md),
@@ -468,15 +256,6 @@ Setup and doctor follow Qwen Code's tilde and working-directory-relative home
overrides. Qwen Code also discovers personal skills in `~/.agents/skills`,
where setup installs the bundled workflow.
If an existing direct-script registration fails with `env: node: No such file or
directory`, refresh it through setup as described under [Launchers and PATH](#launchers-and-path).
For an immediate Qwen-specific workaround, when Node is installed in
`/opt/homebrew/bin`, add `"PATH": "/opt/homebrew/bin:${PATH}"` to that server's
existing `env` object and restart Qwen. Qwen expands `${PATH}` from its parent
environment; other clients can use different substitution syntax. Qwen's stdio
launcher inherits the parent `PATH` and applies server overrides; a reduced
parent environment does not establish that Qwen removed Homebrew from it.
## Antigravity
Antigravity 2.0 discovers personal skills in `~/.gemini/config/skills`.
@@ -504,7 +283,7 @@ standalone CLI check.
A model API's `400 INVALID_ARGUMENT` alone does not identify a rejected schema,
tool-count limit or skill defect. Preserve the response's parameter path and
diagnostic, the actual running REA version, Gemini CLI version and selected
model. For schema rejection, review the [input presentation profiles](../reference/mcp-contracts.md#compact-input-schema-profile)
model. For schema rejection, review the [input presentation profiles](mcp-contracts.md#compact-input-schema-profile)
and reconnect after changing REA's registration environment. Compact is the
current default; an explicitly stored `full` choice is preserved. The generic
error in [#4620](https://github.com/morluto/rea/issues/4620) does not establish
@@ -585,7 +364,7 @@ Responses-compatible Kimi gateway as “infinite recursion”; expanding those
references succeeded. A separate full-profile input failed the provider's
schema-size limit. These are distinct reported failures, not proof that the
canonical schema is recursive. Current REA inputs are already self-contained;
see [schema advertisement](../reference/mcp-contracts.md#schema-advertisement).
see [schema advertisement](mcp-contracts.md#schema-advertisement).
For this failure, inspect the active MCP launch command and refresh the REA
installation that command actually runs. A configured package selector or the
@@ -628,51 +407,17 @@ comments and unrelated settings. It resolves that file from `HERMES_HOME`, else
from the platform default Hermes itself uses — `%LOCALAPPDATA%\hermes` on
Windows, `~/.hermes` elsewhere — including any `HERMES_DATA_DIR_SUFFIX`. Hermes
connects an entry unless `enabled` is false, so setup states `enabled: true`.
Aliases whose anchor is defined in the `rea` entry retain their original resolved
values when setup refreshes or uninstall removes that entry. Setup does not
redirect another server's alias to REA's new launcher or policy.
This also covers an anchor on the removed registration key. References to
definitions outside the entry keep sharing the original value, including when
another definition later reuses the same anchor name.
Hermes personal skills install under the active Hermes home's `skills` directory.
Setup, doctor and uninstall share REA's sticky-profile resolver.
A custom `HERMES_HOME` outside the platform default home
reads its own `active_profile`. An override physically inside the default home
uses that default home's sticky selection and profile root together, including
when the final override directory does not yet exist.
For example, with `HERMES_HOME=~/.hermes/sub` and `~/.hermes/active_profile` set
to `coder`, setup selects `~/.hermes/profiles/coder/config.yaml` and that
profile's skills directory. It does not read `~/.hermes/sub/active_profile`.
Without an outer named selection, this override uses `~/.hermes/sub/config.yaml`.
Use a home outside the platform default tree for an independent sticky selection.
This describes REA's current routing, not full Hermes CLI equivalence.
At Hermes commit `748c79a1`, the
[sticky lookup](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_cli/main.py#L612-L632)
also reads the outer selection, but
[`resolve_profile_env`](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_cli/profiles.py#L2431-L2461)
resolves it beneath the configured `HERMES_HOME`. For nested overrides, confirm
the profile Hermes actually uses and point `HERMES_HOME` directly at that
`profiles/<name>` directory before setup. Compatibility is tracked in
[#3939](https://github.com/morluto/rea/issues/3939).
REA resolves existing ancestor directories before checking physical containment,
including symlinks and junctions, even when the override directory does not exist.
An override beneath a symlinked default home therefore uses the default home's
sticky selection. If `~/.hermes/link` points outside the default home,
`HERMES_HOME=~/.hermes/link/new` instead uses that override root, whether or not
`new` exists. Previewed destinations retain the configured path spelling.
An unreadable path or dangling alias leaves the home unresolved; repair that path
before configuring Hermes. Inspect setup's previewed destinations before applying
changes.
An explicit `HERMES_HOME` pointing to `profiles/<name>` selects that named
profile directly, bypassing sticky selection. An invalid, missing or deleted
sticky profile is reported instead of writing into the default home; repair the
profile or run `hermes profile use default` before retrying. It also remains an
unresolved skill selection, so setup cannot report a successful empty installation.
Setup, doctor and uninstall follow the root's `active_profile` selection, as
plain Hermes CLI commands do. An explicit `HERMES_HOME` pointing to
`profiles/<name>` selects that named profile directly. An invalid, missing or
deleted sticky profile is reported instead of writing into the default home;
it also remains an unresolved skill selection, so setup cannot report a
successful empty installation. Overrides nested beneath the native Hermes home
use that native home's sticky selection and profile root together, matching
[Hermes's root rules](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_constants.py#L83).
Repair the profile or run `hermes profile use default` before retrying.
REA follows `HERMES_HOME` whitespace trimming, environment-variable expansion,
and home expansion before resolving that profile. Setup previews the resolved
configuration and skill destinations; doctor and uninstall use those same paths.
@@ -689,12 +434,10 @@ installs its skill copy there.
## Factory Droid
For [Factory Droid](https://docs.factory.com/harness/mcp), setup writes
`mcpServers.rea` into the user-level `~/.factory/mcp.json` (the
`FACTORY_HOME_OVERRIDE` override replaces the home that contains `.factory`).
Entries carry `command`, `args`, and `env`. Droid discovers personal skills
from `~/.factory/skills` ([skills](https://docs.factory.com/harness/skills),
following `FACTORY_HOME_OVERRIDE` when set), so setup installs its skill copy
there.
`mcpServers.rea` into the user-level `~/.factory/mcp.json`. Entries carry
`command`, `args`, and `env`. Droid discovers personal skills from
`~/.factory/skills` ([skills](https://docs.factory.com/harness/skills)), so
setup installs its skill copy there.
## Trae
@@ -711,18 +454,6 @@ The user-level MCP paths follow the editor's VS Code layout. Trae's official
documentation describes the raw JSON editor without specifying those paths;
loading this registration in Trae remains unverified.
## CodeBuddy
For CodeBuddy, setup writes `mcpServers.rea` into the global
`~/.codebuddy/mcp.json` (a standard `mcpServers` JSON file at the user home
root on every platform). Entries carry `command`, `args`, and `env`; stdio is
the default transport. CodeBuddy discovers personal skills from
`~/.codebuddy/skills`, so setup installs its skill copy there.
The global MCP path is a plain JSON file; setup preserves every other server
and top-level key when it upserts `rea`. Loading this registration, the
bundled skill, and real REA tool calls through CodeBuddy are verified.
## WorkBuddy
For [WorkBuddy](https://www.workbuddy.cn/docs/workbuddy/From-Beginner-to-Expert-Guide/Function-Description/MCP-Guide), setup writes `mcpServers.rea` into the
@@ -754,43 +485,14 @@ the fallback file untouched. Existing REA environment overrides are preserved
when refreshing its command. A malformed or unreadable fallback file fails setup instead of
being silently suppressed; repair or remove it and rerun. Uninstall removes an
REA-owned `rea` entry from both the native table and the fallback, backing up
each changed file. If removing the last native server would activate a retained
fallback server, uninstall refuses before changing either file; review the
fallback configuration before retrying. Other fallback servers, settings and a
foreign `rea` entry are kept. ZCode disables a
each changed file, so an emptied native table cannot load a leftover fallback
registration; other fallback servers, settings and a foreign `rea` entry are
kept. ZCode disables a
server with `enable: false` on its entry; doctor reports such a registration
as stale, and rerunning setup removes the flag to repair it. Native configuration
uses strict JSON; setup refuses comments, trailing commas and a BOM rather than
writing a registration ZCode cannot load.
## FreeBuff
For FreeBuff, `rea setup --client freebuff` writes `mcpServers.rea` into the
user-scope `~/.agents/mcp.json` shared-agent file. Entries carry `command`,
`args`, and `env`; stdio is the default transport. The personal skill location
is `~/.agents/skills`, so setup installs its skill copy there. Because the same
file doubles as ZCode's user-scope fallback (see above), a `freebuff`
registration is also in the file ZCode uses until ZCode defines its own native
server. Setup and doctor detect FreeBuff from its own profile directory
(`~/.config/freebuff-desktop`), not from the shared `.agents` folder that REA's
skill installs create for every client.
FreeBuff MCP loading and skill discovery have not been verified with an
installed native FreeBuff client/version; filesystem tests cover REA's editing,
discovery, and ownership behavior only. Automatic applicability requires the
FreeBuff profile directory even when the shared MCP file exists.
When uninstall cannot establish FreeBuff or another shared-storage reader from
product discovery, a malformed or unreadable shared MCP file is retained
unchanged and does not stop other REA removals, including `--purge-data`. When
FreeBuff or ZCode product applicability is established or unresolved, the same
malformed shared file stops uninstall before removals. See
[Diagnose, update, and remove](installation.md#diagnose-update-and-remove).
FreeBuff setup support requires `[email protected]` or later (including the
current npm `latest` release). Older versions reject
`rea setup --client freebuff` as an unsupported client. See the
[installation guide](installation.md#supported-agents) for client availability.
## DeepSeek Harness
DeepSeek Harness registers MCP clients through a Cordis YAML patch sequence
@@ -834,110 +536,13 @@ in `~/.agents/skills` (`$DSH_AGENTS_HOME/skills` when configured); Harness
discovers that directory, and its native `skill` tool can load
`reverse-engineer-anything`.
REA's optional [native client verification](../development/testing.md#end-to-end-integration-and-golden-evidence)
REA's optional [native client verification](testing.md#end-to-end-integration-and-golden-evidence)
checks headless chat, skill loading, all tool input schemas and actual analysis
using the installed Harness and a local model fixture. This establishes the
Harness integration, with live DeepSeek API compatibility remaining unverified.
See the [Harness MCP guide](https://github.com/deepseek-ai/deepseek-harness/blob/639ed015397290b3745d163aafe02ffee4aa3f84/packages/mcp/mcp-client/README.md)
for its native configuration contract.
<a id="llamacpp"></a>
## llama.cpp
Use REA from llama.cpp's local Web UI through `llama-server`'s native stdio
MCP support. REA supplies the analysis tools; llama.cpp runs your chosen model.
It is not a replacement for a native analysis provider such as Ghidra or Hopper.
Install REA in a persistent location so the configuration below can use a stable
entrypoint. For a project-local installation, run this in your chosen directory:
```bash
npm install --save-exact rea-agents
```
Or install it globally:
```bash
npm install --global rea-agents
```
See [package runners and global installation](installation.md#package-runners-and-global-installation)
for runtime requirements and installation options. Running `npx rea-agents setup`
alone does not create either installation; do not use its temporary package-cache path.
Use a llama.cpp build whose `llama-server --help` lists
`--mcp-servers-config`; older builds without that option cannot use this route.
The configuration below uses an already installed REA package, so tool discovery
does not depend on downloading an npm package during server startup.
Save a JSON file, for example `rea-mcp.json`:
```json
{
"mcpServers": {
"rea": {
"command": "/absolute/path/to/node",
"args": ["/absolute/path/to/rea-agents/scripts/rea.mjs", "mcp"],
"timeout_ms": 450000
}
}
}
```
Replace both paths with your installation's actual paths. For a project-local npm
installation, run `npm root` in that project; the script is under
`$(npm root)/rea-agents/scripts/rea.mjs`. For a global installation, use
`$(npm root -g)/rea-agents/scripts/rea.mjs` instead. Write the resulting absolute
paths as JSON strings; llama-server does not expand shell expressions there.
On Windows, use forward slashes, such as `"C:/Program Files/nodejs/node.exe"`,
or double each backslash, as in `"C:\\Program Files\\nodejs\\node.exe"`.
Apply this to both `command` and the script path in `args`. Single backslashes
can make the JSON invalid or change the path through escape sequences.
`node -p "JSON.stringify(process.execPath)"` prints your current Node.js executable
as a quoted, escaped JSON string that can be used as the `command` value.
Use the version supported by REA. Preserve any other
servers if you are adding REA to an existing configuration. `timeout_ms` is
llama.cpp's per-tool-call timeout, including cold engine startup. The example
allows REA's default 330-second Ghidra startup budget plus 120 seconds for the
operation. Increase it if you configure a longer provider allowance or need more
operation time; see [provider deadlines](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
Start the server with your own tool-capable GGUF model:
```bash
llama-server -m /absolute/path/to/model.gguf \
--host 127.0.0.1 --port 8080 --parallel 1 --ctx-size 16384 \
--mcp-servers-config /absolute/path/to/rea-mcp.json
```
Open `http://127.0.0.1:8080`. REA tools appear with a `rea_` prefix, such as
`rea_analyze_javascript_application`. Select only the tools needed for the
conversation in the Web UI's tool settings: discovery of REA's complete catalog
does not mean that its schemas and results fit the selected model's context.
Choose a context size your model and machine support. For a first check, enable
`rea_analyze_javascript_application` and ask it to inspect a small JavaScript
directory you own. This task needs no native analysis engine. Inspect the
returned Evidence and limitations rather than relying only on the model's summary.
If the model prints a proposed call as ordinary text, REA has not been called.
Check the selected model and its chat template against llama.cpp's function-calling
guidance. A working MCP connection does not establish reliable tool selection by
every model.
`--tools all` is not needed; it enables llama.cpp's separate built-in tools.
`--ui-mcp-proxy` is also not needed for this server-side stdio connection.
REA's setup wizard does not register llama.cpp automatically; pass the JSON
file to llama-server explicitly and restart it after changes.
The MCP child runs with llama-server's permissions. Loopback binding does not
sandbox it or make every REA operation read-only. Enable only trusted commands,
keep the server local, and select targets and operations you are authorized to use.
See llama.cpp's [MCP server configuration](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md#mcp-servers)
and [function-calling guidance](https://github.com/ggml-org/llama.cpp/blob/master/docs/function-calling.md).
The [verification record](../development/client-verification.md#llamacpp) distinguishes connection
checks from model behavior and lists the tested versions.
## Other client settings
For OpenCode, setup writes the V1 `mcp.rea` entry, which OpenCode V1 and V2
@@ -953,10 +558,8 @@ become active again. Malformed applicable files block setup before changes.
Grok Build loads `[mcp_servers.rea]` from `$GROK_HOME/config.toml`, or from
`~/.grok/config.toml` when `GROK_HOME` is unset or empty. Setup edits that server
table, `[mcp_servers.rea.env]`, and a root `disabled_mcp_servers` entry that
names `rea`. It defaults `startup_timeout_sec` to 30 seconds, preserves an
existing positive timeout, and leaves every other name in that list.
[Grok supports longer startup timeouts](https://docs.x.ai/build/features/mcp-servers)
for cold package-runner launches. The shared skill installed under `~/.agents/skills` is already
names `rea`. It sets `startup_timeout_sec = 30` and leaves every other name
in that list. The shared skill installed under `~/.agents/skills` is already
on Grok Build's skill path.
Grok Bot (`grok_bot`) is detected from `~/.grokbot`, or from `SAND_DATA_ROOT`
@@ -986,6 +589,10 @@ can override an inherited disable. Lower files and other server entries remain
unchanged; a legacy server named `servers` that conflicts with the native table
requires manual repair before setup can write.
When Hermes aliases refer to an anchored REA entry, refreshing or removing REA
retains the aliases' original resolved values. Setup does not redirect another
server's alias to REA's new launcher or policy.
With no client selected, the interactive skill choice targets the shared
`~/.agents/skills` copy. Selecting a client targets its personal destination
when supported. Use `rea setup --skill-client shared --skill` for the shared
@@ -1,26 +1,11 @@
# Client and agent verification
FreeBuff MCP loading and personal skill discovery are currently unverified with
an installed native client/version. Its temporary-filesystem coverage verifies
REA's configuration and applicability boundaries only; it does not establish
that FreeBuff consumes the written registration or skill.
Use these optional lanes to verify an installed client or evaluate an agent's
REA workflow. [Testing REA](testing.md) provides the normal development commands
and provider lanes. Each record below states its host, client version, model
fixture, and remaining coverage; a native-client run with a loopback model does
not establish live model-service acceptance.
## Launch environment verification
`rea mcp doctor --json` starts the production MCP adapter with the current runtime
executable and environment. A healthy result establishes that adapter's
initialization, discovery and target-free request flow; it does not verify a
client's configured launcher or its inherited environment. Test the generated
registration through the actual client, including a reduced `PATH` when checking
desktop or service startup. Record the client version and launch environment
separately from terminal doctor results. See [launcher troubleshooting](../getting-started/client-configuration.md#launchers-and-path).
## Published Pi code-mode renderer limit
A separate probe of the published `@earendil-works/pi-codemode` 1.1.0 and 1.2.0
@@ -33,7 +18,7 @@ shared form would leave later fields unknown instead. The existing
its character limit by advertising reduced nested validation structure, while
REA continues to validate calls against the complete canonical input schema.
Compact is the default for the server's advertised inputs; an explicit `full`
choice is preserved. See the [compact input schema profile](../reference/mcp-contracts.md#compact-input-schema-profile)
choice is preserved. See the [compact input schema profile](mcp-contracts.md#compact-input-schema-profile)
for details of its reduced forms.
Custom renderer hosts can raise `inputMaxChars` through `renderToolSignature` or
@@ -43,30 +28,6 @@ does not establish broader Pi compatibility. See the pinned upstream
[`declarations.ts`](https://github.com/earendil-works/pi/blob/b45e0ebcc5ae2fc42a6f2f23104d10e656e6bedb/packages/codemode/src/declarations.ts)
and [renderer usage notes](https://github.com/earendil-works/pi/blob/b45e0ebcc5ae2fc42a6f2f23104d10e656e6bedb/packages/codemode/README.md#declarations-for-the-model).
<a id="llamacpp"></a>
## llama.cpp
The [manual stdio connection](../getting-started/client-configuration.md#llamacpp) was checked on
macOS arm64 with Node.js 24.18.0, llama-server b11715 (`9d1878a43`), and the
published `[email protected]` package. `GET /tools` exposed 207 REA tools without
enabling llama.cpp's built-in tools. With the source checkout at `9e372c6`, all
207 forwarded names, descriptions and input schemas matched a direct MCP client.
Calls through `POST /tools` returned session metadata and JavaScript analysis
Evidence for a fixture at a Unicode path. A missing target returned REA's input
error, and a subsequent session call still succeeded. The owned llama-server
process exited successfully after the checks. No native analysis engine or
Windows host was exercised.
A local Qwen3-1.7B Q8_0 run selected the JavaScript analysis tool and received its
Evidence, but its follow-up answer reached the output-token limit. Other small
Qwen3 attempts returned proposed calls as ordinary JSON text rather than tool
calls. These runs do not establish dependable model-driven investigations.
A request with the full catalog and task history exceeded a 16,384-token context;
select tools for the task instead of assuming that discovery implies model fit.
The checks exercised HTTP and stdio, not browser interaction with the Web UI.
## End-to-end, integration and golden evidence
The optional `verify:qwen-client`, `verify:pi-client`, and `verify:hermes-client`
@@ -87,14 +48,6 @@ skill setting; Hermes discovers it through the selected `HERMES_HOME`.
Set `REA_VERIFY_HERMES_STICKY_PROFILE=1` to exercise native Hermes selection of
a named sticky profile with `HERMES_HOME` still pointing to its root.
For Qwen on POSIX, set `REA_VERIFY_QWEN_REDUCED_PATH=1` and select its JavaScript
entry point with `REA_VERIFY_QWEN_COMMAND` to test a restricted client environment.
Setup retains the caller's `PATH`; the native Qwen `chat` or `call` run then uses
an isolated `PATH` containing only Bash for the shell-tool fixture. The lane
checks that `node`, `npm` and `npx` cannot be found and uses the generated REA
registration unchanged. Its receipt records `reducedPath`, `clientPath` and
`registrationCommand`; the option alone does not establish a passing native run.
The Qwen lane also supports native Windows (verified with Qwen Code 0.25.1 and
Node 24.19.0 for both `chat` and `call`). With Node 24.18.0, the same `call`
journey reached full Evidence readback but the client aborted during exit with
@@ -486,7 +439,7 @@ existing static JavaScript analysis through CLI and stdio MCP, including
manifest readback, competing query variants, and resolved relative imports.
It uses an installed browser and accepts an optional installed REA entrypoint.
The complete `verify:browser` lane also exercises passive script export through
both public adapters. See [website script export](../guides/web/website-script-export.md).
both public adapters. See [website script export](website-script-export.md).
`verify:browser:modules` compares CLI and stdio MCP traces against an independent
real Chromium module-loading fixture: import-map scopes, package prefixes,
@@ -494,7 +447,7 @@ null/backtracking rejection, query/fragment identities, repeated module instance
lazy and computed unknowns, exact source/map identities and no implicit refetch.
Set `REA_BROWSER_EXECUTABLE`; the optional script argument selects an installed
package entrypoint. The complete `verify:browser` lane and existing conditional
Chrome CI include this verifier. See [module relationships](../guides/web/website-module-trace.md).
Chrome CI include this verifier. See [module relationships](website-module-trace.md).
## Agent evaluation and conformance records
+42 -350
View File
@@ -1,145 +1,45 @@
# Analysis settings, limits, and recovery
# Admission limits and recovery
This guide explains the runtime settings design for maintainers and developers:
who owns each value, when an update takes effect, how providers enforce it, and
how to extend the system. The public tool contract is in
[MCP contracts](../reference/mcp-contracts.md#editable-analysis-settings); recorded runtime
checks and their limitations are in the
[verification report](../verification/runtime-analysis-settings.md).
## Problem and design boundary
A valid workload can exceed a conservative default. An oversized ASAR header
may prevent analysis of the entire archive; a slow Ghidra function may exceed
its decompile deadline. Requiring a new REA process to change either budget
interrupts the investigation and can discard reusable provider state.
REA exposes a curated set of analysis defaults through MCP so callers can
adjust a reported limit and retry the same workload in the running runtime.
Each setting has a typed value, resource bounds, an enforcement owner, and a
defined application point. The design applies across archive admission,
provider startup, operation deadlines, and worker memory requests.
The setter changes in-memory defaults. It does not edit arbitrary environment
variables, configuration files, MCP registrations, or provider installations.
Those mechanisms have different owners and lifetimes. It also cannot change a
client deadline, extend accepted work, release retained resources, or remove a
fixed representation limit. Exposing a setting requires a working enforcement
path, not just an entry in the tool schema.
Further ceilings that should become settings are tracked in
How to add a REA-owned admission ceiling, budget, or deadline, and what a caller
does when one is hit. This is a design rule, not an inventory. Ceilings that
should become settings are tracked in
[#6176](https://github.com/morluto/rea/issues/6176). Naming honesty for limit
errors is [#5910](https://github.com/morluto/rea/issues/5910).
## Ownership and data flow
| Owner | Responsibility |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Domain definitions](https://github.com/morluto/rea/blob/main/src/domain/analysisSettingDefinitions.ts) and [types](https://github.com/morluto/rea/blob/main/src/domain/analysisSettings.ts) | Exact setting IDs, value types, shared bounds/defaults, units, provider/operation associations, application points, and structured limit metadata. Numeric constants stay with their domain owner. |
| [Launch parser](https://github.com/morluto/rea/blob/main/src/config/analysisSettings.ts) | Parse selected launch values once, preserving an invalid value as a typed configuration failure rather than substituting a default. |
| [AnalysisSettings](https://github.com/morluto/rea/blob/main/src/application/AnalysisSettings.ts) | Own one runtime's launch baseline, immutable override revisions, atomic updates, inspection, and accepted-operation snapshots. It does not own provider lifecycle or cleanup. |
| [Composition](https://github.com/morluto/rea/blob/main/src/main/eagerRuntime.ts) and [typed configuration views](https://github.com/morluto/rea/blob/main/src/config/runtimeAnalysisSettings.ts) | Inject the same settings reader into providers and workflows; translate overrides into the configuration each consumer understands. |
| Provider or resource owner | Resolve the effective budget, enforce it at the applicable boundary, and attach the actual value and known source to failures. |
| [MCP registration boundary](https://github.com/morluto/rea/blob/main/src/server/EvidenceMcpServerCore.ts) | Enter the operation's settings scope before invoking its handler. |
| [Settings contracts](https://github.com/morluto/rea/blob/main/src/contracts/analysisSettingsContracts.ts) and [tool registration](https://github.com/morluto/rea/blob/main/src/server/registerAnalysisSettingsTools.ts) | Validate named requests and deliver inspection/update results without requiring an open target or healthy analysis provider. |
The normal path is: composition creates the store, an MCP handler captures its
current revision, and providers read that snapshot when resolving budgets.
If an owner rejects work, it reports the effective limit through the shared
error projection. A subsequent settings update publishes a new revision for
later handlers. It does not reach into the failed operation or retry it.
### Accepted-operation snapshots
`AnalysisSettings.run()` installs the current immutable snapshot in
`AsyncLocalStorage`. Its `read` function returns that snapshot inside the
operation and the latest revision outside it. Providers depend on the reader
contract rather than on the mutable store or an ambient environment lookup.
Acceptance here is entry into the registered handler's settings scope, not
arrival of bytes at the MCP transport. Work queued or awaited within that scope
keeps its snapshot. Deferred server initialization must still dispatch through
the canonical scoped handler.
| Event | Revision used |
| --------------------------------------------------------- | ----------------------------------- |
| Operation A enters its handler, then waits for a provider | Current revision 0 |
| A settings update publishes revision 1 | Future handlers can read revision 1 |
| Operation A resumes and starts its worker | Its accepted revision 0 |
| Operation B enters its handler | Revision 1 |
This gives each operation stable budget semantics even if another connection
updates settings while it waits. `next_worker` and `next_startup` describe where
an accepted value is applied; they do not cause workers or startups belonging
to an older operation to adopt a newer revision.
Inspection deliberately reads the store's current defaults, rather than the
calling operation's snapshot. It reports `active_values.status: "not_observed"`:
an inspection result cannot establish the budget or remaining time of an
already-running worker.
### Composition and embedding
The MCP runtime creates one store and supplies its reader to provider factories.
`createServer` uses a supplied store or creates one, and wires its default
providers to that store. An embedder supplying a custom binary session or other
provider must also inject the same reader into that provider's composition and
pass the store to `createServer`. Merely exposing the settings tools cannot
make a custom provider consume updates.
Keep default factories, explicitly supplied providers, transport composition,
and deferred handler dispatch covered separately where their wiring differs.
Resolve settings at the operation boundary; copying a runtime value into a
provider constructor would freeze it at launch. For entrypoints outside the
MCP wrapper, an embedder needing the same snapshot guarantee must establish
the operation scope with `AnalysisSettings.run()`.
## Name a default, then a setting when operators need headroom
Put the number in a named constant with a unit. Do not leave it as a magic
number at the call site.
When a workload may need more room, add a setting whose unset value is that
constant. Pure numeric contracts live in `src/domain/analysisResourceLimits.ts`
or their existing domain owner. `analysisSettingDefinitions.ts` binds the
curated runtime IDs to those same bounds, defaults and application points.
Environment parsers and recovery text stay under `src/config/`; register launch
schemas on `environmentSchema` so startup and doctor agree. Add launch settings
to `SELECTED_REGISTRATION_SETTINGS` when setup should forward them. Unset keeps
today's behavior. An invalid launch value remains a configuration failure,
never a silent fallback; its validation boundary determines whether MCP can
start (see [invalid launch configuration](#invalid-launch-configuration)). An
invalid runtime write is `invalid_request` and changes nothing.
constant. One module under
[`src/config/`](https://github.com/morluto/rea/blob/main/src/config) owns the
default, any floor, the maximum, the schema, the parser, and the recovery
text. Register that schema on `environmentSchema` in
[`environment.ts`](https://github.com/morluto/rea/blob/main/src/config/environment.ts).
`parseConfig` calls `parseEnvironment`. Doctor uses that same parser through
`nativeAnalysisConfigurationError`, so startup and `rea doctor` accept the
same values. Add the name to `SELECTED_REGISTRATION_SETTINGS` so `rea setup`
copies it into the client registration. Unset keeps today's behavior. An
invalid value is a `ConfigurationError` and projects as `configuration_invalid`.
Do not fall back to the default.
For example, `regex.matching_deadline_ms` and its launch seed
`REA_REGEX_MATCHING_DEADLINE_MS` share a 5000 ms default, 1 ms floor and
2147483647 ms maximum. Node schedules a longer `setTimeout` delay as 1 ms.
Changing the runtime setting affects the next accepted search; changing the
launch environment affects the next REA process.
[`REA_REGEX_MATCHING_DEADLINE_MS`](https://github.com/morluto/rea/blob/main/src/config/regexMatchingDeadline.ts)
([#6267](https://github.com/morluto/rea/pull/6267)) is the reference. Default
5000 ms, floor 1 ms, maximum 2147483647 ms, because Node schedules a longer
`setTimeout` delay as 1 ms. Setup copies the setting. A hit says:
> The effective REA_REGEX_MATCHING_DEADLINE_MS value is 5000 ms (source default; default 5000 ms, floor 1 ms, maximum 2147483647 ms). Set REA_REGEX_MATCHING_DEADLINE_MS in the environment that starts REA to a positive decimal integer from 1 to 2147483647, then restart REA. The worker old-generation heap does not raise this deadline.
[`REA_MCP_MAX_RESPONSE_BYTES`](https://github.com/morluto/rea/blob/main/src/config/mcpResponseBudget.ts)
is the byte-budget shape: a decimal safe integer with a 10 MiB floor
(`MINIMUM_RESPONSE_BYTES`), the same shared schema, and no silent fallback. A
change needs a REA restart and a matching client receive buffer. It does not
raise REA's reserved string limit or Node's string ceiling. Like the regex
deadline, it is in `SELECTED_REGISTRATION_SETTINGS`, so setup forwards an
explicit selection into the client registration.
raise REA's reserved string limit or Node's string ceiling. The setup
allowlist does not copy this setting. A new headroom setting should, following
the regex deadline.
Windows ChangeTime admission waits out a fresh timestamp before recording a
baseline. `REA_WINDOWS_CHANGE_TIME_DRIFT_MS` (default 2000 ms, floor 1 ms,
maximum 60000 ms), `REA_WINDOWS_CHANGE_TIME_SETTLE_POLL_MS` (default 25 ms,
floor 1 ms, maximum 1000 ms), and `REA_WINDOWS_CHANGE_TIME_SETTLE_MAX_POLLS`
(default 40, floor 1, maximum 1000) are launch settings. Unset keeps those
defaults. An invalid value is a configuration error: it names the setting, the
rejected value, source `setting`, the default, the floor, and the maximum, and
says to set a positive decimal integer in the environment that starts REA and
then restart REA. There is no runtime override. Setup forwards an explicit
selection. When the poll budget runs out, admission keeps the last observation
instead of failing the readable file. Raising a setting only changes how long
that wait can last. It does not change Windows ChangeTime itself.
A per-call budget can have a runtime default without having an environment
setting. JavaScript `max_heap_mb`
A per-call budget is not an environment setting. JavaScript `max_heap_mb`
(CLI `--max-heap-mb`) is an integer from 128 to 16384 MiB, default 1024. It
applies on the next analysis, which starts a new worker, and does not restart
REA. It does not raise the parent result budget, transfer ceilings, or fixed
@@ -151,144 +51,8 @@ does: its recovery says to set `NODE_OPTIONS` (for example
`--max-old-space-size=<MiB>`) on the process that starts REA, then restart
that process. `max_heap_mb` does not raise it.
## Runtime settings through MCP
`inspect_analysis_settings` lists the curated settings, their current and launch
values, source, bounds, application point and relevant operation names. Filters
by exact setting IDs, provider or operation intersect. Active worker values are explicitly `not_observed`; a configured default is
not a measurement of active work. Inspection also separates known server limits from unknown client deadlines and receive buffers.
`update_analysis_settings` accepts an atomic `changes` array. Each entry names
an exact `setting` and either `action: "set"` with a schema-validated `value`, or
`action: "reset"`. Reset restores the launch value. `expected_revision` is an
optional compare-and-set guard; conflicts and duplicate keys change nothing.
An unchanged write does not advance the revision.
The revision tracks override state, not only effective numeric values. Setting
an explicit override equal to the launch value still changes its source to
`runtime` and advances the revision. Reset removes that override, even if the
effective number stays the same. Validation, revision checks, and duplicate-key
checks all happen before publishing the replacement snapshot; a rejected batch
cannot partially apply. Clients coordinating writes can supply
`expected_revision`; without it, valid updates apply to the current state.
| Runtime setting | Application point |
| ------------------------------------------------------------- | ---------------------------------------------------------------------- |
| `asar.max_header_bytes` | Next archive operation, including full JavaScript application analysis |
| `ghidra.startup_timeout_ms`, `hopper.startup_timeout_ms` | Next provider startup attempt |
| `ghidra.decompile_timeout_seconds` | Next operation on the existing Ghidra Program; `null` means unbounded |
| `jadx.operation_timeout_ms` | Next JADX operation |
| `regex.matching_deadline_ms`, `regex.worker_heap_request_mib` | Next search / new regex worker |
| `capture.decoder_old_generation_mib` | Next owned capture decoder |
| `javascript.analysis_timeout_ms`, `javascript.max_heap_mb` | Next source analysis / new JavaScript worker |
| `wasm.command_timeout_ms` | Next WABT operation |
| `native.matching_representation_bytes` | Next `match_native_functions` request |
### Precedence and lifetime
Where a setting supports an explicit per-call argument, precedence is:
1. The caller's explicit argument.
2. The accepted operation's runtime override.
3. Valid launch configuration, where that setting has an environment seed.
4. The built-in default when launch configuration is absent.
An invalid launch value does not fall through to the built-in default. Preserve
omitted caller arguments in Evidence: resolving an internal default must not
rewrite the request as though the caller supplied it. `null` is the explicit
unbounded value for `ghidra.decompile_timeout_seconds`; it is not a substitute
for an unknown or omitted value elsewhere.
A write does not extend active work, relaunch a provider, clear retained
cleanup, or invalidate successful Evidence.
Overrides belong to one REA runtime, survive target close/open, and disappear
when REA exits. Connections sharing that runtime share its overrides. No files,
ambient environment, registrations or upstream software are edited. Overrides
are runtime-wide, not scoped to a target or an individual connection.
### Failure and retry
An ASAR admission error supplies `details.limit` with the exact setting,
effective value, units, bounds, scope, application point, restart requirement
and, when known, required bytes and value source. The agent can immediately call:
```json
{
"changes": [
{ "setting": "asar.max_header_bytes", "action": "set", "value": 33554432 }
]
}
```
Then retry the same full-archive request once work and cleanup have settled.
No preliminary inspection or source subset is required. The setter does not
retry automatically. If an explicit per-call value caused the failure, correct
that argument; raising a runtime default cannot override it.
The resource owner constructs `details.limit` from the registry plus the budget
it actually enforced. Metadata includes `agent_editable: true`,
`scope: "rea-runtime"`, and `rea_restart_required: false`. Optional
`required_value`, `source`, and `parameter` fields are supplied only where known
or applicable. Do not infer a source because a value happens to equal a default.
Error copying, attribution, redaction, and delivery must preserve these facts.
Human-readable remediation explains the same failure; clients should not need
to parse that sentence to identify an editable setting.
`scope` and `applies_at` describe a continuing caller, such as an MCP
connection. A one-shot invocation, such as a CLI command, has released its
provider session and exited its runtime before its caller reads the failure,
so its adapter projects with an explicit `"invocation"` caller lifetime.
`details.limit` then keeps every other field and reports
`scope: "rea-invocation"`, `applies_at: "next_invocation"` and the launch
`environment` seed when one exists. `agent_editable: true` still holds, because
the caller chooses that environment value or per-call `parameter` for the new
invocation; `rea_restart_required: false` means no running REA process needs a
restart. A producer keeps guidance that holds only while its caller retains the
session, such as "the Ghidra session remains usable" or a same-session runtime
update, in `continuingCallerGuidance`; the shared presentation adds it only for
a continuing caller. The producer's own message and remediation must stay true
after the session is released. A composed error records each embedded child
failure through `childFailureDiagnostics`, which snapshots it at composition
for both lifetimes when they differ, so cleanup history keeps its state and
the final projection selects the caller's form. Failures retained inside
Evidence keep the continuing form, so CLI and MCP Evidence stay identical.
Cache policy is part of enforcement. Successful reusable analysis can survive
a budget update. Transient Ghidra decompile failures remain uncached so a raised
deadline can recover on the same Program. Admission checks still apply to reused
data: lowering an ASAR limit must reject a cached oversized header. Neither
raising a limit nor clearing a transient analysis failure authorizes skipping
cleanup or changing the identity of the provider session.
### Invalid launch configuration
Launch validation has two relevant boundaries. Settings validated by global
configuration parsing can prevent REA from starting; an MCP setter cannot
repair a server that is not running. Correct the launch environment or
registration, use `rea doctor` for its configuration checks, and start REA again.
Provider-local validation can leave MCP available. For example, an invalid
`REA_JADX_OPERATION_TIMEOUT_MS` is inspectable with `source: "invalid"`, a
`configuration_error`, and omitted `value` and `launch_value`. A valid runtime
override repairs the effective JADX timeout without editing that launch seed.
Inspection then reports the runtime value while retaining the launch error.
Reset exposes the invalid launch state again; it does not repair or replace it
with the built-in default. Provider availability and execution must resolve the
same effective setting so readiness changes along with repair and reset.
`REA_NATIVE_MATCHING_REPRESENTATION_BYTES` is validated the same way, by
`match_native_functions` alone.
## Keep a fixed ceiling honest
Inspection's separate
[constraints](https://github.com/morluto/rea/blob/main/src/server/analysisSettingsConstraints.ts)
describe operator-owned, external, and fixed controls. REA can report its
configured response budget and parent heap limit; client deadlines and receive
buffers remain explicitly unknown. These are not entries accepted by the
runtime setter. Raising a worker heap request cannot raise the parent heap,
and raising a provider deadline cannot make the client wait longer.
If the ceiling must stay fixed, say so in the failure. Name the limit and the
effective value, say this release has no runtime override, and name what does
not raise it. Do not invent a setting.
@@ -304,24 +68,20 @@ The string the caller sees includes:
- the limit or setting name
- the effective value and unit
- the source when known (`default`, `setting`, `runtime`, or `request`)
- the source (`default` or `setting`)
- the default, the maximum, and the floor when the setting has one
- the next action, including where to set it ("in the environment that starts REA")
- whether a restart is required
- what the change does not raise
Distinguish an invalid configuration from a valid budget exhausted by work.
Preserve the configuration failure and its setting identity; apply the
[launch validation boundary](#invalid-launch-configuration) when describing
whether live repair is possible.
Invalid settings never take this path. They are `configuration_invalid`.
Doctor reports `runtime-configuration:<setting>` and says to correct it in the
shell or the MCP registration, then rerun.
## What agents do
Use `details.limit` for a direct runtime update when it names an editable
setting with `scope: "rea-runtime"`. For `scope: "rea-invocation"`, set its
`environment` or `parameter` for a new invocation instead.
Follow `remediation.action` for per-call, fixed and external controls.
Inspect settings only when bounds or application details are missing. Restart only when it says the value is
Follow `remediation.action`. Change a request field or a `REA_*` setting only
when that text recommends the change. Restart only when it says the value is
read at startup. A sentence that a control does not raise the limit is not a
request to change it.
@@ -343,90 +103,22 @@ request to change it.
- `resource_constraint` for capacity, when `retryable` is true: wait for the
active admission to finish, then retry within the reported limits. Do not
restart REA to free the slot.
- `configuration_invalid`: when MCP remains available and the failure belongs
to a curated provider-local setting, inspect and set a valid runtime override.
Otherwise correct the named setting in the environment that starts REA, then
start REA again. Run `rea doctor` for the
- `configuration_invalid`: correct the named setting in the environment that
starts REA, then start REA again. Run `rea doctor` for the
`runtime-configuration:` check. Re-run `rea setup` for that client when the
setting is on the registration allowlist, then reconnect so the new process
sees it.
- `invalid_request`: follow the specific recovery and reported issue. Inspect
the schema or CLI help when the input violates the contract or the response
asks for `inspect_input_contract`. A valid selector can instead require a
different choice, such as selecting a reported address; an inventory miss
does not establish that the target lacks the item. Do not repeat the
unchanged call when the response says it cannot succeed unchanged.
- `invalid_request`: correct the reported fields from the current schema or
CLI help. Do not repeat the unchanged call.
A restart is required to adopt a changed launch environment or registration.
The curated runtime settings above require no REA restart. It is not the first step for a transient
A restart is required for an environment setting and for a client still
running the previous registration. It is not the first step for a transient
MCP drop, a capacity wait, or retained cleanup.
## Adding a runtime setting
## Tests
1. Identify the enforcing owner and the real resource constraint. Decide whether
callers need a per-call argument, a runtime default, a launch-only control,
or a fixed limit. A registry entry must not advertise authority REA lacks.
2. Define the value type and named numeric bounds with the domain owner. Add the
ID and metadata to `analysisSettings.ts` and `analysisSettingDefinitions.ts`.
Reuse those bounds in runtime validation, launch parsing, and error metadata.
Choose the application point and provider/operation associations from actual
consumers; a shared control need not apply to every provider.
3. If there is a launch seed, add its parser and appropriate configuration/doctor
validation. Decide whether an invalid value blocks the runtime or only the
provider. Add it to setup's `SELECTED_REGISTRATION_SETTINGS` when it should
be forwarded; runtime editability does not automatically propagate it.
4. Thread the shared reader through every applicable composition path. Resolve
the effective value within the accepted operation, honor explicit arguments,
and preserve their omission in Evidence. Keep availability checks consistent
with execution. A worker heap request must not be described as a measured
process memory limit.
5. Enforce the value at its resource boundary and construct structured limit
facts there. Preserve those facts through shared error projection and copies.
Check cache reuse, lowered admission, transient failures, and retained cleanup
against the same ownership rules.
6. Update the public contracts and affected guides, then regenerate catalogs
and packaged skill metadata using the repository commands. Keep externally
owned and fixed controls in the separate inspection constraints, not in the
editable registry. Validate both store semantics and a public recovery path.
## Tests and verification
Cover the unset default, accepted overrides, rejected floor/maximum values,
and invalid syntax through the owning parser and configuration boundary.
Lock the unset default, one accepted override, rejected floor, maximum, and
syntax, and the recovery sentence through `parseEnvironment` and `parseConfig`.
[`regexMatchingDeadline.test.ts`](https://github.com/morluto/rea/blob/main/src/config/regexMatchingDeadline.test.ts)
is one example. Assert error codes, setting IDs, values, units, provenance,
application points, and resulting behavior. Check that adapters deliver the
canonical recovery guidance, with independent assertions for its underlying
facts. Avoid freezing prose or deriving all expected values from the same
registry used by the implementation; either can hide the wrong failure.
See [test design](../development/testing.md#test-design-and-pruning) for the assertion policy.
Runtime checks cover atomic validation, stale writes, reset, runtime isolation,
accepted-operation snapshots, and explicit argument precedence. Public MCP
acceptance must reject an oversized ASAR, raise admission, and successfully
retry the same complete archive on the same connection. Lowering admission must
also refuse previously cached headers. The real Ghidra timeout verifier checks
recovery after a live update without replacing its session. Also cover invalid
provider-local launch configuration, repair/reset availability, default and
custom composition paths, and preservation of structured limits through error
copies. Gates used to overlap work and updates need bounded acknowledgements
and release/join in `finally`.
The [verification report](../verification/runtime-analysis-settings.md) separates
public MCP checks, simulated provider coverage, real Ghidra execution, and
measured task costs. It does not establish real-provider behavior for every
editable setting or active worker telemetry.
## Bun standalone module graphs
`MAX_BUN_MODULE_COUNT` admits at most 100,000 module records.
`MAX_BUN_RETAINED_BYTES` admits at most 256 MiB of serialized graph payload and
separately bounds retained entry metadata. Physical metadata read work is also
limited to 256 MiB per preparation, including repeated uncached ranges. A
256 KiB / 1,024-entry exact-range cache avoids full-page amplification. The
combined logical path tree admits at most 250,000 distinct prefixes; this is
checked before adding nodes, with shared prefixes counted once. The shared
artifact expansion budget also bounds the sum of referenced member lengths. Refusals name the boundary.
There is no runtime override; additional host memory or `NODE_OPTIONS` does not
raise these ceilings. Use a smaller producer fixture or a separately reviewed
reader for larger graphs. No partial extraction is claimed on admission failure.
is the pattern. Project the failure and assert `configuration_invalid` for a
bad value and the recovery wording for a hit.
-410
View File
@@ -1,410 +0,0 @@
# Desktop application capture
`rea capture-desktop-application` launches an installed Linux desktop
application, watches it while a person or a driver uses it, and records what
it is, which windows it opened and how the files under the home directory
changed. The result is the starting point for turning the application into an
agent environment: the inventory says how to install and launch it, the windows
say how to recognise it, and the checkpoints hold the state it read and wrote.
The application runs with the current user's permissions, in the current
desktop session, against the user's real files. This is not a sandbox. The
capture folder keeps copies of real files from the home directory, so treat it
as private data and keep it local.
## Host support
Capture runs on Linux with an X11 display: `DISPLAY` must be set, and `xprop`
(from `x11-utils`) must be installed, or `REA_XPROP_COMMAND` must name it.
The window manager must support the EWMH client list (`_NET_CLIENT_LIST`,
listed in `_NET_SUPPORTED`), as xfwm4, Mutter, KWin and Openbox do; without it
the capture is refused, because no window could be seen. A supported list that
is unset, as xfwm4 leaves it while no window is open, means no windows. Applications that run under XWayland are visible;
native Wayland windows are not. Other hosts are refused before anything is
launched.
## Capture an application
Name the application by its desktop entry id, the entry's absolute path, or an
executable:
```sh
rea capture-desktop-application --desktop-entry org.xfce.mousepad --output ./mousepad-capture
rea capture-desktop-application --executable gnucash --argument /home/ada/books.gnucash --output ./gnucash-capture
```
Desktop entries are looked up in `$XDG_DATA_HOME/applications` and then each
`$XDG_DATA_DIRS` directory; an entry installed as a symbolic link, as flatpak
and Nix do, is followed to its file. The entry's `Exec` line is expanded with no files;
`--argument` values are appended in its place, and the application runs in the
entry's `Path` or the home directory. When the line starts with
`env NAME=value`, the program is found on the `PATH` those assignments set,
with an empty or relative entry read from the folder the application runs
in, and their XDG folders name the state directories. Otherwise the program is
looked up on the host's `PATH` in the order the launch searches it, and one
found only through a relative or empty entry is refused, since the exported
computer would not have that entry. An `--executable` runs from
the current directory, as it would from a shell, and a relative path to it is
resolved there.
Desktop-entry string values retain literal trailing spaces. For example,
`Path=/tmp/workspace ` and `Path=/tmp/workspace\s` name the same directory
ending in a space. Use the escaped spelling to make such a suffix visible.
Before writing anything the command refuses to start if a window of the
application is already open, or if the window list cannot be read. A launch
beside a running instance can hand off to it, and that instance's unsaved
state would be missing from the baseline. It then records the starting state,
launches the application in its own process group with a run token in its
environment, and reads commands from stdin, one per line. The application gets
REA's environment without the `AGENTENV_*` variables an agent environment sets,
such as its control token: an app the exported computer launches never has
them, and an app captured inside a computer must not hold a credential to it.
| Line | Effect |
| ------------------------------------------- | ----------------------------------------------------------- |
| empty, `checkpoint`, or `checkpoint <name>` | Record the current state as a checkpoint |
| any other text except `stop` | Record a checkpoint named by that text |
| `stop`, or the end of stdin | Record a final checkpoint, close the application and finish |
An unnamed checkpoint is named `checkpoint-<ms>` after its time in the
session. A name that repeats an earlier checkpoint, or is one of the capture's
own names (`baseline`, `stop`, `time-limit`, `exit`), is refused with a
progress line on stderr; no checkpoint is taken and the session continues.
Name the checkpoints a later step reads by what they mark. The
[export](desktop-application-export.md) seeds the bundle from the baseline
unless `--seed-checkpoint` names another, and a
[task](desktop-application-tasks.md) is checked against the last checkpoint
unless `--end-checkpoint` names another. So one capture can build a seed and
then the change: launch the app, create and save the document, send
`checkpoint seed`, make the edit and save it, send `checkpoint end`, then
`stop`.
The session also finishes when the launched process has exited and the
application's processes and windows are all gone in two consecutive samples,
or when `--max-duration-seconds` passes. Before deciding the processes are
gone, the capture looks for any process still carrying the run token, so a
worker whose launcher exited before it was sampled keeps the session open. A
sample whose window list could not be read is not taken as an empty one; the
limitations say how many there were. One queued line is handled between
samples, and reading stdin pauses while 16 lines wait.
Progress lines on stderr are JSON, `{"rea_progress": {"phase":
"desktop_capture", "message": ...}}`. Those a driver waits on also carry an
`event`: `{"kind": "window_open"}` once the window opens; `{"kind":
"checkpoint", "name", "trigger", "at_ms", "entries", "changes",
"retained_files", "retained_bytes", "changed_files", "more_changed_files"}`
once a checkpoint has been taken, the baseline, `stop` and `exit` included,
with the paths walked, the paths changed since the checkpoint before, the
files retained, and up to ten of the files (not folders) that changed,
home-relative: those under a `--state-path` or absolute `--argument` first,
then those the application held open, then those in its state directories,
then the rest, and last caches (a folder whose name ends in `cache`, such as
`~/.cache`) and WebKit or Chromium storage, each by path, with a count of the
rest. An application can rewrite dozens of cache files on a save, and KDE's
holds its service cache open, so the saved document would otherwise fall past
the ten;
and `{"kind": "checkpoint_refused", "name", "reason"}` for a requested name it
refused. A driver that sends `checkpoint <name>` waits for the event naming it
rather than for a fixed time.
When the session ends the capture sends SIGTERM to the launched process, its
process group and every process that carries the run token, waits up to five
seconds for them to exit, and then removes every process that still carries
the token, whether or not it stayed in the process group. If that cleanup
cannot be verified after a second attempt, the capture is still written, and
the command then fails, naming the run token and process group that may
remain. Interrupting the command (Ctrl-C) ends the session the same way
without a further checkpoint; the capture is still written with end reason
`cancelled`, and the command exits with status 130. Interrupted before the
application is launched, the command writes no capture and removes the
content it had retained.
If the capture fails for any other reason before `capture.json` is written,
the application is closed the same way and the content already retained is
removed from the capture folder. The error names any process or file that
could not be removed.
A person types these lines in the terminal. An agent can drive the
application with a tool such as `xdotool` and write the lines to a FIFO that
it passes as stdin. Because the end of stdin stops the session, the driver
must keep the FIFO open for writing between checkpoints, for example with
`exec 3>capture.fifo` in a shell, rather than open and close it for each
line.
| Option | Default | Meaning |
| ------------------------ | ------- | ----------------------------------------------------------------------------------------- |
| `--state-path` | none | File or directory whose starting content is retained too (repeatable) |
| `--exclude` | none | Path left out of every checkpoint walk (repeatable); the output folder is always left out |
| `--max-duration-seconds` | 3600 | Stop the session after this long |
| `--max-entries` | 250,000 | Paths recorded per checkpoint walk, its roots included |
| `--max-file-bytes` | 64 MiB | Largest file whose content is retained |
| `--max-content-bytes` | 1 GiB | Total retained content |
| `--package-file` | none | The `.deb` file the application was installed from; its facts are recorded, not the file |
`--state-path`, `--exclude` and `--output` refuse an empty value, which
would otherwise name the current directory.
## The capture folder
The output folder must be new, or empty and owned by the current user; the
command makes it private (mode 0700). It holds:
- `capture.json`: the Evidence record, also printed on stdout.
- `content/<sha256>`: the bytes of each retained file, stored once per
distinct content.
The Evidence operation is `capture_desktop_application` and its predicate type
is `rea.desktop-application-capture`. Its `host` names the platform,
architecture, display and home directory, and the host's name (`hostname`)
and the user's (`user`), which captured state can hold, as in a mail
template's `ada@e81232be457d`; the export removes them from the bundle. A
capture written before they were recorded has neither, and one is left out
when the system cannot say, as for a user with no passwd entry. The rest of
the result has four parts.
**Application.** The desktop entry (id, path, name, `Exec`, icon,
`StartupWMClass`, categories, MIME types), the launch command and working
directory, the executable as found on `PATH` and after resolving links, and the
package that installed it. A Debian package records its name, version,
architecture and the apt sources that list the installed version, with the
origin (`o=`) apt reports for each source in `source_origins`, such as `Ubuntu`
for the Ubuntu archive and its mirrors or `LP-PPA-<name>` for a PPA; an empty
source list means the version came from a local `.deb` or the package lists
are missing, which `package_lists_present` tells apart. apt's lists folder,
and the trusted keys below, are where `apt-config` resolves
`Dir::State::Lists`, `Dir::Etc::TrustedParts` and `Dir::Etc::Trusted` under
`Dir`, so an `APT_CONFIG` file that moves them is followed: on a computer
without lists, an ordinary user can run `apt-get update` with an `APT_CONFIG`
naming writable `Dir::State::Lists` and `Dir::Cache` folders and capture with
the same `APT_CONFIG`. Without `apt-config`, or when it fails, they are
`/var/lib/apt/lists`, `/etc/apt/trusted.gpg.d` and `/etc/apt/trusted.gpg`.
The Debian package is the one `dpkg-query -S` names as the owner of the
desktop entry the capture launched, or else of the executable, since the
entry's package is what installs the application: LibreOffice Writer's entry
belongs to `libreoffice-writer`, while the `/usr/bin/libreoffice` it runs
belongs to `libreoffice-common`, which does not bring Writer with it. When the
two differ, `executable_package` names the executable's package, which the
export installs beside the entry's, since the entry's package need not depend
on it (`vim-common` owns Vim's entry but does not depend on `vim`).
`executable_package_reason` says why the computer could not install it by
name, by the rule for library packages below: neither the Ubuntu 24.04 archive
nor a vendor repository the export declares offers it, or `apt-cache policy`
did not say. When a package owns the entry but none owns its program, such as
a `~/.local/bin/<name>` found first on `PATH`, the entry's package is recorded
without `executable_package`, and `executable_package_reason` says no package
owns the program, so the export refuses rather than install a package that
does not bring it. A program link that dpkg does not own counts as owned when
dpkg owns the file it resolves to, as an alternatives link does. An
`--executable` launch, or an entry no package owns, such as
one in `~/.local/share/applications`, records the executable's package. When
`dpkg-query` does not finish or exits with an error for the executable or the
entry, the package is `unknown` and the limitations say which path, since a
failed lookup is not proof that no package owns it. The sources, vendor
repositories and `--package-file` match below are the recorded package's; the
library walk still reads the executable.
Each source whose origin is not `Ubuntu` is also recorded in
`apt_repositories`. `apt-get indextargets` names the sources file entry apt
read it from and its release, so flat repositories match too, whether their
suite is `./` (Typora's `deb https://downloads.typora.io/linux ./`) or `/`
(pkgs.k8s.io's). The record holds that entry (`sources_file`, `uri`, `suite`,
`components`, none for a flat repository, and `architectures`, the entry's
`arch=` or `Architectures:` restriction, empty when it has none), `release`,
the release's Suite (apt's `a=`) or else its Codename, and the key that signs
it. A user name or password in an entry's URI (`https://user:password@host/…`)
is never recorded: REA removes it from every URI it reads, in the sources file
and in `apt-get indextargets` and `apt-cache policy` output alike, so entries
match whether or not apt prints it, and sets `uri_credentials_removed` on a repository whose URI held
one (a capture written before this was recorded omits the field). The key is
found by verification, not by name: the candidates are the
entry's `signed-by` keyrings, or else every `.gpg` and `.asc` key in apt's trusted
parts folder and its trusted keyring, and each is checked with
`gpgv` against the release's `InRelease` (or `Release` with `Release.gpg`)
beside its index in apt's lists folder. A deb822 entry whose `Signed-By` field
holds the armored key itself, as `add-apt-repository ppa:<owner>/<name>`
writes it on Ubuntu 24.04, has that key as its only candidate, as apt trusts
no other for it: REA reads the field's continuation lines as deb822 writes
them, a `.` line standing for an empty one, from the key block's `BEGIN` line
to its last `END` line, and records the key's path as the entry's sources
file. A key counts when gpgv reports a valid signature by it and no bad,
expired, revoked or unverifiable signature by it, whatever gpgv's exit status:
a release also signed by a key the host lacks makes gpgv fail. Each candidate
is copied, an armored key decoded to a binary keyring, into a private
temporary folder that is removed afterwards; a folder REA cannot remove
fails the capture with the folder as its cleanup resource. The key that
verifies is recorded in `signing_key` with its path,
SHA-256, format (`binary` or `armored`) and primary fingerprint, and its bytes
are retained under `content/`. A key file holding one key is retained as it
is; from one holding several, such as a legacy `/etc/apt/trusted.gpg`, only
the transferable key with the verifying fingerprint is retained, in binary, so
the bundle trusts no other key. When no candidate verifies, `gpgv` is
missing, apt holds no signed release, no sources entry matches, the key the
entry's `Signed-By` holds does not verify the release, or the key is larger
than `--max-file-bytes` or the content budget, `signing_key` is null and
`signing_key_reason` says why. Ubuntu Pro's ESM repositories (origins such as
`UbuntuESMApps`, host `esm.ubuntu.com`) are not checked: apt reaches them only
with the host's Ubuntu Pro credentials, which a computer does not have.
Unless the installed version comes from the Ubuntu 24.04 archive, so for a
vendor repository's package and for a local install that no apt source lists,
`library_packages` lists the packages that own the shared libraries the
executable loads at start; the walk runs beside the repository check, and not
at all for an archive package, whose export does not use it. REA reads
DT_NEEDED from the resolved executable and from each library it finds, and
looks each soname up as ld.so does: in the RPATH of the needing image and of
each image up its chain of loaders to the executable when the needing image
has no RUNPATH, then in its RUNPATH (with `$ORIGIN` expanded and paths
normalized), and then in the `ldconfig -p` cache entries for the executable's
machine. It asks `dpkg-query -S` for each library's package. Libraries in the
executable's own folder belong to the application: they are walked but not
listed, and neither is the application's own package. One batched
`apt-cache policy` then keeps only the packages the Ubuntu 24.04 archive or a
recorded vendor repository with a verified key offers, in any component its
sources entry lists, since the export declares them all; the rest, such as
packages from a PPA or a local `.deb`, are listed in
`uninstallable_library_packages`. The same query checks `executable_package`.
Each recorded repository's `supplies` lists the kept packages, and the
executable's package, that the archive does not offer and its package files
list, so the export can leave out those only a repository it does not declare
supplies, such as one whose key the content budget later dropped. Sonames
found nowhere are listed in
`missing_libraries`, and libraries no package owns in `unpackaged_libraries`.
`library_packages_reason` says why the lists are incomplete: the executable is
not an ELF image (a launcher script, for example), `ldconfig`, `dpkg-query` or
`apt-cache` is missing, `dpkg-query -S` could not tell which package owns a
library (it exits 2 rather than 1, which only says no package owns the paths
it did not name), a library could not be read or names more than 512
libraries, or the walk stopped after 512 libraries. A failed owner query does
not list the libraries it covered in `unpackaged_libraries`, since a lookup
that could not finish is not proof that no package owns them. Completed batches
retain their package and unowned-library facts. Paths containing package-query
pattern characters stay unknown unless another candidate identifies an owner.
A sources file, key or library whose handle REA cannot close fails the capture
with the file as its cleanup resource. When the repository check and the
library walk both fail, the capture reports both failures and every folder or
file either could not release.
An application installed from a `.deb` file you downloaded, such as VariCAD
from varicad.com, has no apt source a computer could install it from. Name
that file with `--package-file` so the export can have the computer download
it instead. REA reads the file's control fields with
`dpkg-deb --field FILE Package Version Architecture Depends Pre-Depends`, and
when its Package, Version and Architecture are the installed package's, it
reads the file once more for its SHA-256 and size and records `package_file`:
the file's `name` without its folder, `sha256`, `size`, `package`, `version`,
`architecture`, and the `depends` and `pre_depends` entries as the file
declares them. Otherwise `package_file_reason` says why, for example that
`dpkg-deb` is not installed, that the file holds another version than the
installed one, or that the file changed between the two reads (its device,
inode, size and times are compared), since its fields and its hash could then
describe different files. The capture never copies the file: it is the vendor's, and only
these facts are kept.
Snaps, flatpaks and
AppImages are recognised from their paths, and flatpaks also from a
`flatpak run` launch, whose application id is the first operand after
flatpak's options and their values. Electron applications are marked by
the fuse sentinel in their binary or a `resources/app.asar` beside it. State
directories are the XDG config, data, state and cache folders and the
dot-folder named after the package, the executable, the desktop entry's full
id and its last part, and the entry's name, with whether each existed at the
start. A folder in those places whose name is one of these in another letter
case joins them, such as GIMP's `~/.config/GIMP` for the `gimp` package; only
the whole name is compared, so `~/.config/GIMPshop` does not. Dotfiles in the
home named after the application (`.<name>rc`, or
`.<name>` followed by `-`, `.` or `_`, for a name of three or more characters)
that exist at the start join them, since an application may only read such a
file during the capture. So does KDE's settings file in the config folder,
`<name>rc` such as Tellico's `~/.config/tellicorc`, for each such name of three
or more characters, whether or not it exists yet. A `flatpak run` launch is named after its application
id rather than the `flatpak` launcher, and adds `~/.var/app/<id>`.
**Session.** When it started, how long it ran and why it ended; the launched
process's exit status; every process seen in the application's tree; every
client window with its class pair, titles, owner pid and type, marked when it
belongs to the application. Titles and class names decode xprop's quoted
delimiters, newline and tab escapes, and three-digit octal byte escapes;
unsupported escape spellings leave that string unknown rather than inventing
text. The session also records when the first application window appeared; the
most common class pair of the application's windows, for matching its window
later; files the application's processes held open under the walked roots; the last
8 KiB of the launched process's stderr, where a failing app's last words are,
with `stderr_truncated` when earlier output was dropped; and whether cleanup was verified. Each
process records whether the capture started it: the launched process, a
process carrying the capture's run token, or a descendant of one, even when
its environment cannot be read, as for a zombie or a sandboxed renderer.
Processes are told apart by their start time, so a pid reused by another
process is not counted as the application's.
A window belongs to the application when its owner pid is one of the
application's live processes or its class matches the desktop entry's
`StartupWMClass`, the executable's name, the entry's full id or its last part,
or a flatpak's application id. A flatpak's windows report pids from its
sandbox, so they are matched by class only.
Ownership is checked again at every sample, because a window can appear before
the process that owns it is found. A matching window's pid joins the
application's processes, so a launch handed to a process the capture did not
start, such as a D-Bus activated service, is still followed. That process does
not carry the run token, so the capture never signals it and leaves it
running.
**State.** The walked roots (the home directory plus any content root
outside it), the content roots (state directories, `--state-path` values and
absolute `--argument` values, except `/` or a directory that contains the home
directory, which are named in the limitations instead; a selected path that is
not a directory brings the `-wal` and `-journal` paths beside it, so a database
selected on its own is captured with its logs), the state directories
as walked (their physical spelling, which differs from the inventory's when one
is a symlink), the selected paths (each `--state-path` and content-root
`--argument` as named, absolute, beside the physical path walked, so a
symlinked settings folder keeps what its name means), the paths every walk left out (the output folder and
`--exclude` values), and the checkpoints in order: `baseline` before
launch, each requested checkpoint, `stop` or `time-limit`, and `exit` after the
application has gone. Each checkpoint records how many paths its walk saw,
whether it was truncated, how many paths it could not read or directories on
another device it did not enter (`unchecked_paths`), which paths were created, modified, deleted or
changed type since the previous checkpoint, and which files it retained. A
path counts as created or deleted only where the walk listed its nearest
existing directory completely; differences under a directory that could not be
listed, or that a truncated walk did not reach, are counted as unconfirmed
instead. A walk does not enter a directory on another filesystem than its
root, such as an NFS, sshfs or other FUSE mount under the home directory; the
limitations list each such mount point. The
baseline retains every file under the content roots; later checkpoints retain
every changed file. A change records the file's type, size, mode,
modification time and, when known, the SHA-256 of its content before and
after, plus whether it lies in a state directory and whether the application
held it open. A file is retained only if it is still the version the walk
recorded and does not change while it is read. A file larger than the
remaining content budget is not read, unless stored content of the same size
could make it a duplicate. Files that could not be
retained are listed with the reason: too large, over the content budget,
changed since the walk, changed during the read, or unreadable.
The baseline also records the paths and types of non-file entries it observed,
including symlinks, without following them. This is required to distinguish an
unreadable SQLite sidecar from an absent one. Captures made before this field
was added must be made again before state inspection, task drafting or export.
Directories the application held files open in, but whose starting content was
not retained, are listed so a second capture can pass them with
`--state-path`, or `state_paths` on a build computer's `start_build_capture`,
and a limitation names them, spelled from `~/` under the home.
**Limitations.** What the capture could not establish, such as a truncated
walk or a missing window.
## What the capture does not establish
Changes are found by comparing walks, using each path's size, modification
time, inode and mode, so a change undone between two checkpoints is not seen
and a rewrite that keeps all four is missed. A change is not attributed to the
process that made it: other desktop services write under the home directory
too, such as the dconf database, the window manager's settings and GTK's
recently used list. The markers for state directories and open files are
evidence for attribution, not proof.
Clicks and keystrokes are not recorded. Window titles are sampled every half
second, so short-lived titles can be missed.
-464
View File
@@ -1,464 +0,0 @@
# Desktop application export
`rea export-desktop-application <capture-folder> <directory>` turns a folder
written by [desktop application capture](desktop-application-capture.md) into an
agentenv bundle holding one computer app environment: the real application,
installed from the distribution archive, its vendor's apt repository or the
`.deb` file it was installed from on the `ubuntu-24.04` computer base, launched
at boot, seeded with the user's state and serving that state through agentenv's
data operations.
```sh
rea export-desktop-application ./mousepad-capture ./mousepad-bundle
agent-env run ./mousepad-bundle --dry-run
```
The export reads the capture and its [state view](desktop-application-state.md);
it does not run the application. `--name` sets the environment name; otherwise
it comes from the desktop entry id or the program, with an `app-` prefix when
that name would not be a portable bundle name (for example all digits). The
output directory must be new or empty. `--task` adds a task an agent wrote for
the capture, once REA has validated it against the capture; see
[desktop application tasks](desktop-application-tasks.md). `--package-url`
says where the computer downloads the package of an application installed from
a `.deb` file; see [Applications from a .deb file](#applications-from-a-deb-file).
`--seed-checkpoint` names the checkpoint whose state seeds the bundle, the
baseline by default, and `--end-checkpoint` the one a task must pass on, the
capture's last by default; see [The seed checkpoint](#the-seed-checkpoint).
`--synthetic-seed` says the seed was made for the task, as a build agent makes
it in a [build computer](desktop-build-witness.md), rather than taken from the
user's own data; see [A seed made for the task](#a-seed-made-for-the-task).
The command returns Evidence with operation `export_desktop_application` and
predicate type `rea.desktop-application-export`, listing the environment name,
the seed and end `checkpoints`, owned paths, seed files, the seeds it
rewrote to remove the capture host's name and user (`scrubbed_files`), the
gzip seeds it compressed again only to clear their headers
(`recompressed_files`), the bundle files that still hold them
(`unscrubbed_files`), and written files, the last four by their path in the
bundle. A failed export, cancellation
included, removes the bundle it had not yet published, so no partial copy of the
user's data remains. If the bundle was published but its output handles could
not be closed, the command reports a cleanup error that carries the export
Evidence; the bundle itself is complete.
## The bundle
| Path | Contents |
| -------------------------------- | -------------------------------------------------------------------------------------------------- |
| `envs/<name>/env.toml` | `type = "computer_app"`, the module and class, `computer = "ubuntu-24.04"`, `driver = "reference"` |
| `envs/<name>/<file>.py` | The `ComputerAppEnvironment` class |
| `envs/<name>/SKILL.md` | What the app is, its window class and where its files live |
| `envs/<name>/seed/` | The application's files at the seed checkpoint, by home-relative path |
| `envs/<name>/apt-keys/` | The signing key of each vendor apt repository the app installs from |
| `envs/<name>/install-package.sh` | With `--package-url`: the run script that installs the downloaded package with dpkg |
| `tasks/smoke.json` | A task that deploys the app |
| `tasks/<task>.json` | With `--task`: the agent's task, its verifier in `artifacts/` and the eval in `evals/` |
| `README.md` | What was exported, from which capture, and what to keep in mind |
The class file is `<name>_app.py` with hyphens as underscores, for example
`mousepad_app.py`: agentenv imports it under its stem, so a plain `<name>.py`
could shadow a standard library module such as `calendar`. A class name that
would collide with a name the module imports or defines, such as `Path`, takes
an `App` suffix.
The class declares:
- `Packages(apt=[...])` with the package the capture found. A vendor package
is named `<package>/<release>`, its repository's release (the Suite apt
reports as `a=`, or else its Codename), so apt takes it from that repository
even when the Ubuntu archive offers a higher version or epoch, as noble's
transitional `firefox` does over packages.mozilla.org's; it is named alone
when the release names neither, or when the capture was written before
releases were recorded. Its library packages follow, so a package that
leaves out dependencies on the libraries it loads, as Typora's Electron
package does, still starts; library packages the capture found neither the
Ubuntu 24.04 archive nor a declared repository offers are left out, with a
limitation, and so are those only repositories the export leaves out supply
(a repository's `supplies`), such as one whose key the capture could not
retain. An archive package's declared dependencies are trusted as they
are. When the desktop entry's package is not the one that owns the program
it runs, as `vim-common`'s entry runs `vim`'s program, the program's package
(`executable_package`) follows the app's by name, since the entry's package
need not depend on it; a capture that found no archive or declared
repository offering it records why, and the export refuses it with that
reason, as it does when only repositories it leaves out supply it, or when
no package owns the program at all (`executable_package_reason` without
`executable_package`). The package must come from the Ubuntu 24.04 archive
or a mirror of it,
or from a vendor apt repository whose signing key the capture verified and
retained. A source counts as the archive when apt reports its origin as
`Ubuntu` for a noble suite; a capture made before origins were recorded
counts only Canonical's own archive hosts.
- For a vendor repository, `sources=[AptSource(apt=..., suite=...,
components=[...], key="apt-keys/<name>.gpg")]`, with the retained key copied
to that path (`.asc` when it is armored, which agentenv tells by its
content). A PPA added with `add-apt-repository`, whose sources entry holds its
key in `Signed-By`, exports that key as `apt-keys/<name>.asc`. apt takes one
key per URI and suite, so sources that share them, such as one repository
listed for two architectures, are declared once, with their components
merged; one with a different key is left out. A repository
whose key the capture did not verify, an Ubuntu Pro (ESM) repository, one
whose sources entry's URI held a user name or password (which the capture
did not record and the export never writes; the reason names neither them
nor the URI), one not reached over http or https, or one whose `arch=`
restriction leaves out the capture host's architecture is refused with its
reason (a capture
written before restrictions were recorded counts as unrestricted); when
another usable repository lists the same version it is left out instead,
with a limitation. The export also refuses snaps, flatpaks, AppImages, and a
package no recorded repository lists unless it was installed from a `.deb`
file the capture recorded and `--package-url` is given, since the computer
could not install them by name, and a capture written before credentials
were removed whose repository URI still holds them, since it counted that
repository's libraries as installable; capture it again.
- `@computer_app(supports=...)` with both `amd64` and `arm64` for an archive
package, which the archive serves for both, and only the capture host's
architecture for a vendor package, since its repository may serve no other.
A vendor package captured on any other architecture is refused. A package
installed from its `.deb` file supports only that file's architecture.
- A launch at boot with the captured command, ready when a window of the
captured class appears and one of them is listed in the taskbar and is not
typed as a splash: a splash screen or a dialog of the class skips the
taskbar or says it is a splash, and an app whose splash shows first would
otherwise count as ready before its own window opens. Arguments that are the captured home or under it, and
`<option>=<path>` values under it, are respelled under the computer user's
`$HOME`. agentenv starts the app in its own folder, where an open or save
dialog would show the framework's files, so the command first changes to the
captured working directory (`"$HOME"` when that was the home, as for a
desktop entry without `Path`), which a relative argument or program path
such as `./app` also needs, and still starts if that folder is missing. The
export refuses a capture that saw no application window.
- The seed's document, opened at launch, so the app shows it rather than its
start screen: when the seed holds exactly one file the state view gives the
`document` role, and the desktop entry's `Exec` takes files (a `%f`, `%F`,
`%u` or `%U` field code), its path under `$HOME` is appended to the command.
A command that already names a document, absolute or from its working
directory, as one captured with `--argument <file>` does, is kept as it is;
a settings path under the home does not count. A seed with no document launches
as captured. With several, no desktop entry or an `Exec` that takes no file,
the app launches as captured and the README and the result's limitations say
why. A reset
relaunches the app, so it opens the document again.
- For an Electron application (the capture found its fuse sentinel or a
`resources/app.asar`), `--no-sandbox` right after the program unless the
command already has it: Chromium's sandbox needs user namespaces the
computer's container does not grant, and Electron exits without them. The
README and the result's limitations say the sandbox is off.
- `paths`: the state directories holding the application's state files and the
folders holding its documents, nested folders collapsed. A folder beside a
state directory whose name differs from it only in letter case is owned
whole in the files' spelling, so a `~/.config/GIMP` profile GIMP created
during the capture beside the candidate `~/.config/gimp` is owned as
`.config/GIMP`, not as the folder of its first file. The home itself and
the shared `.config`, `.local`, `.local/share`, `.local/state` and `.cache`
folders are never owned whole: a file directly in one is owned as itself.
Files the capture recorded under a symlinked state directory's physical
spelling are mapped back to the innermost such directory's home spelling,
unless that spelling is outside the captured home, as when the home itself is
a symbolic link. When a selected link names an application state directory,
the bundle seeds the application's state path and restores the selected
alias at boot. Selections nested under that alias are seeded under the same
state path. An existing incompatible alias stops setup without replacing it. When two files map to one home path, the export uses the one present
before the capture, else the first by path, and a limitation names the other.
- Data hooks, because declaring any hook switches off agentenv's default data
plane: the class seeds the home once per boot and keeps it as the initial
state with `keep_initial`, `data/add` and `data/reset` call agentenv's `add`
and `reset`, and `data/get` returns the files under `paths` plus a data part
with a typed JSON view of each, read in a worker thread. An application with
no observed state declares no paths and no hooks.
A file under the owned paths whose bytes at the seed the capture did not
retain is not seeded: one in the state view without retained content at that
checkpoint, or one the capture omitted (for example too large or past the
content budget). The result lists them in `unseeded_files` and the README
names them. When the application held files open in folders whose starting
content the capture did not retain, a limitation names those folders, spelled
from `~/` under the home: a file there is seeded only when the capture
retained it, as it does one written during the capture, and the limitation
names the seed files there or says there are none. The desktop Trash,
`~/.local/share/Trash`, is never the application's, so a file the user moved
there is neither owned nor seeded, even when a window title names it; a
limitation lists the files the Trash held at the seed.
Files that changed during the capture outside the owned paths are neither
owned nor seeded, so the app starts without what it saved there: MuseScore
keeps its settings in `~/.config/MuseScore`, which its name rules miss, and
showed its first-run wizard on every boot. A limitation and the README name
those folders, leaving out desktop services' files, each collapsed to its
folder in `~/.config`, `~/.local/share`, `~/.local/state` or `~/.local`, the
hidden folder of the home it lies in, or else its own folder, up to eight and
a count of the rest. Capture again naming them with `--state-path`, or
`state_paths` on a build computer, to own them.
Lock files the app holds while it runs are left out of the seed: a copied one
makes the app find its profile or document in use, and a LibreOffice Calc
bundle whose seed held its profile's `.lock` showed no window at boot. They
are `lockfile`, `lock`, `.lock`, `.parentlock`, `parent.lock` and
`SingletonLock` anywhere, LibreOffice's document lock `.~lock.<name>#`, and
`*.lock` or `*.lck` directly in one of the application's state directories.
Elsewhere such a file may be a project's own, such as `Cargo.lock` or
`yarn.lock`, which pin dependencies, and is seeded, even under a folder whose
name starts with a dot. A file the state view makes a document, or one named
itself by `--state-path` or an absolute `--argument`, is never left out,
whatever its name. A limitation and the README list them, up to
eight and a count, after "Left out of the seed". They stay under the owned
paths, so `data/get` returns the lock the running app holds, and `data/reset`
restores the seed without them. Other seed files the app keeps for its own
running are seeded as captured and named, collapsed to the folder that holds
them: WebKit or Chromium storage (such as `hsts-storage.sqlite`, `storage/`,
`mediakeys/`, `Local Storage/`, `IndexedDB/`, `Cookies`), which may hold the
app's own data, and caches (a folder whose name ends in `cache`); a stale one
can change how the app starts.
The application's dconf keys at the seed, if any, are written once when the
app first starts, also for an application that owns no files, and the class adds
`dconf-cli` to its apt packages for that. The server starts before the
desktop's D-Bus session, so the class does not use `dconf load`: it dumps any
existing user database through a `file-db:` profile, then runs `dconf compile`
on that dump and the captured keys, in that name order so the captured keys
win, and replaces `~/.config/dconf/user`. If that fails, the server prints a
one-line warning and the app starts without the captured keys. `data/reset` does
not restore them. No file of the bundle holds them, so `data/get` does not
return them and no task check reads them; a draft whose only change between
its checkpoints is to the application's dconf keys says so with
`uncheckable_changes`, and one whose dconf database changed but could not be
read says so with `unreadable_changes`.
## The seed checkpoint
By default the seed is the state before the capture, its `baseline`, and a
task's end state is the capture's last checkpoint. `--seed-checkpoint` and
`--end-checkpoint` name others, so one capture can make a seed and then a
change: the user creates and saves a document and takes `checkpoint seed`,
then edits and saves it and takes `checkpoint end`. Exported with
`--seed-checkpoint seed --end-checkpoint end`, the bundle seeds the document
as it was saved at `seed`, owned paths, dconf keys and unseeded files
included, and a task must fail on that seed and pass on the state at `end`.
A file's state at a checkpoint is its last version recorded at or before it.
The export refuses a name the capture does not hold exactly once, and an end
checkpoint taken before the seed checkpoint; `inspect-desktop-application-state`
lists the names in `capture.checkpoints`.
## A seed made for the task
By default the seed is the user's real data: the README and the result's
limitations say to keep the bundle local and not publish it. With
`--synthetic-seed` they say instead that the seed was made for the task, and
to read `envs/<name>/seed/` before publishing the bundle; the export cannot
tell the two apart, so the flag is the caller's word. `export_build_environment`
passes it, since the building agent makes the seed in the build computer. The
Evidence records it as `synthetic_seed`.
## Host name and user
A seed can hold the capture host's name and the user's: Geany saved its mail
template as `pref_template_mail=ada@e81232be457d`, the user `ada` on a
container named `e81232be457d`, and many files hold the home path
`/home/ada`. The capture records `host.hostname` and `host.user`, and the
export removes them from what the bundle ships:
- In seeded text files (text, JSON, key files and XML in the state view), in
UTF-16 files that start with a byte-order mark (keeping the mark, byte order
and a following U+FEFF content character), and in the dconf keys,
`user@host` becomes `user@localhost`, the
host name standing alone becomes `localhost`, and the captured home becomes
`/home/user` wherever it is followed by `/`, the end of the text, or a
character that cannot continue a file name. The host name counts only as a
whole name: not inside a longer word, before a `-`, or as a label of a
domain such as `e81232be457d.local`, so a host named `ubuntu` leaves
`ports.ubuntu.com/ubuntu-ports` alone. A host named `localhost` is left
alone, but its `user@localhost` still takes the computer's user. The user
name alone is never replaced, since a short one such as `ada` or `user` is
also ordinary text. A rewritten seed is written with its own SHA-256, and
`scrubbed_files` and the README list it.
- JSON and XML keep their syntax. In JSON only string values change, each
read as its decoded value, so `\u0065` escapes cannot hide the host, and is
written back as JSON only when it changed; numbers and `true`, `false` and
`null` stay as they are even for a host named `true`. Member names are never
rewritten, since `{"ubuntu": 1, "localhost": 2}` on a host named `ubuntu`
would get two `localhost` keys and lose a setting; a file whose names hold
the host is listed in `unscrubbed_files` instead. In XML only character
data, attribute values, comments and CDATA change, character references
decoded the same way; element and attribute names and processing
instructions stay. A UTF-16 file is scrubbed by the format its text would
have in UTF-8, so JSON or XML in UTF-16 follows these rules too.
- In the dconf keys only GVariant strings change, each read as its decoded
value and written back in its own quotes only when it changed. Group names,
keys, dictionary keys, bytestrings and everything outside quotes stay, so a
host named `true` leaves `wrap=true` alone. A value whose string does not
parse, or that holds an object path or signature, ships as captured.
- `SEED_HOME` in the class is `/home/user`, the home the seed spells, which
the class respells to the computer's own home when it seeds the home at
boot, as it did the captured home before. The replacements read as the
computer's own: its user is `user`, with its home in `/home/user`, and
`localhost` is the name every host has for itself.
- A gzip-compressed seed, such as a Gnumeric workbook, is decompressed, up to
256 MiB, scrubbed as the format the state view read its document as, and
compressed again whether or not its document changed, so no file name,
comment or time its header held ships: with no file name, comment or extra
field, modification time 0 and the operating system unknown, so the same
document gives the same bytes on every host. It is deflated at the fastest
level, or stored when deflating would make it expand more than the 100 times
a view reads, so a seed that read still reads. It is listed in
`scrubbed_files` when its document changed, and otherwise in
`recompressed_files`, which the README counts rather than lists, so the
example and template files an app copies on first run do not bury the
document scrubbed; one already in that form ships as captured. The class copies it at
boot as it is, so the home it holds stays `/home/user`. One that does not
decompress ships as captured, its bytes searched, and one the state view did
not read is only searched. One whose document needs no edit but a member of
which has an extra field, where BGZF (`.vcf.gz`, `.bam`) and dictzip
(`.dict.dz`) files index their blocks, also ships as captured, its bytes
searched, so its blocks stay where its index says; one whose document
changed is still compressed again as one member, losing them, as the
export's limitation says.
- Binary files, SQLite databases among them, and text files over 256 MiB are
copied as captured. Then every file the bundle ships, the class module with
its dconf keys and the README included, is searched by a looser rule than
the replacement's: the host name and `user@host` anywhere, as in
`e81232be457d.local` or a seed named `e81232be457d-notes.txt`, and the
captured home as a whole path, in its path and its content, JSON and dconf
strings and XML references decoded and the replacements themselves set
aside. Each one found is listed in `unscrubbed_files`, and in the README
unless it is the README, to check before sharing the bundle. Bytes that are not text are
searched as they are, so a binary file that holds the host in another
encoding is not listed.
- A task check whose values name `user@host` or the host name anywhere, by
that rule, is refused, as one naming the captured home is: the bundle does
not hold them. Its field names, kind and id are not values, so a host named
`file` or `kind` refuses no check. A task's checks must fail on the seed as
the bundle ships it, scrubbed, so a check the scrub itself satisfies, such
as a mail setting the user changed from `ada@e81232be457d` to
`user@localhost`, does not count as failing there.
A capture written before the host's name and user were recorded has only its
home removed, with a limitation saying so; capture again to remove the rest.
## Applications from a .deb file
A closed-source application you installed from a `.deb` file you downloaded,
such as VariCAD from varicad.com, is not in any apt repository the computer can
use. Capture it with `--package-file` naming that file (see
[desktop application capture](desktop-application-capture.md)), then pass
`--package-url` to the export with an `http://` or `https://` URL the computer
can download that same file from:
```sh
rea capture-desktop-application --executable varicad --package-file ~/Downloads/varicad2026-en_2.03_amd64.deb --output ./varicad-capture
rea export-desktop-application ./varicad-capture ./varicad-bundle --package-url https://files.example.com/varicad2026-en_2.03_amd64.deb
```
The class then declares, beside `Packages`:
- `downloads=[Download(url=..., sha256=..., to="vendor/<file>", arch=...)]`
with the captured file's SHA-256. agentenv downloads the URL once, when the
bundle is put, and refuses the file unless its SHA-256 matches, so the
computer installs exactly the captured version. The bundle does not contain
the package.
- `Packages(apt=[...])` with the first alternative of each of the file's
Depends and Pre-Depends entries, without version constraints or `:any`
qualifiers, then the library packages the capture found, since a vendor's
binary may load libraries its Depends leaves out.
- `run=[RunScript(script="install-package.sh")]`, a script the export writes
into the environment folder. agentenv runs it as root from the app's folder
once the apt packages are installed and the download is in place, with the
network off: it runs `dpkg -i` on the download and then removes it.
The bundle supports only the file's architecture. Vendor sites often refuse
automated downloads, as varicad.com does, so the URL may need to point at a
copy you keep, such as a private mirror; you must be allowed to use that copy,
and the package's licence, such as a 30-day trial's terms, is yours to check.
The README and the result's limitations say so.
The export refuses a package no recorded repository lists when `--package-url`
is missing, saying to capture with `--package-file` if the capture lacks it. It
also refuses `--package-url` when the capture recorded no package file, giving
the capture's `package_file_reason` when there is one; when the archive or a
declared vendor repository installs the package already; when the URL is not
written as `http://` or `https://` or holds a user name or password; and when
the file is built for an architecture other than `amd64`, `arm64` or `all`.
## Grading once the app has saved
A task's verifier, `artifacts/<task>-check/verify.py`, grades the app's files
only once they have settled, because an app can save on its own timer after the
last change. Gnote saves a note 4 s after the last key, so a read taken as
soon as the agent stops finds the text on screen but not yet in the file. The
verifier reads the app's `data/get` every 0.5 s. It compares each file's bytes
between reads by their SHA-256, as `data/get` returns them, and grades once
every file has held for 3 s. The 3 s count from when a read returned the
current bytes, not when it began, so a slow read that brings a change late
does not count its own run as quiet. 15 s after the first read began, the
bound, the verifier grades the last completed read: no read starts at or
after the bound, and a read still running then is cut. Each result records
whether the files settled in `settled`, and whether the bound cut a read in
`read_cut`.
The window has a cost and a limit:
- An app whose files do not change during the check, such as one that saved
at once, is graded 3 s after the first read returns.
- A save that lands more than 3 s after the first read returns is still
missed. An app that saves 4 s after the last key is caught only when the
agent stops at least a second after that key.
- Files the app rewrites more often than every 3 s, such as a log, never
settle, so every check waits until the bound.
- The bound counts the reads too, so a slow `data/get` ends the check at
15 s. Only the first read, which every grade needs, runs to its own
timeout.
The bound and window are `SETTLE_BOUND_S` and `SETTLE_WINDOW_S` in the
verifier. `check_build_task` waits the same way before it grades each
witness stage; see [witnessing](desktop-build-witness.md#witness).
## Typed views
The views follow the [state view](desktop-application-state.md)'s formats and
shapes: SQLite tables (at most 64 ordinary tables, `tables_truncated` past
that, and per table `columns`, `primary_key`, at most 500 `rows` and
`truncated`, with blobs, unsafe integers and text that is not UTF-8 tagged as
REA tags them), JSON, XML as `{root, truncated}` with REA's node, depth and
whitespace rules and its `doctype` when the document declares one, key file
groups, text as
`{lines, line_count, truncated}` split on `\n` and `\r\n`, and zip archives
as their entries, read with Python's `zipfile` from the file in place, with
REA's entry, size, compression ratio and archive limits and its wording for an
entry it does not read. The module reads the central directory itself, as REA
does, and shows `zipfile` only the records it lists, reading each entry in
bounded chunks no further than a byte past its declared size. A gzip file is
the view of the document inside it, with `"compression": "gzip"` beside its
format, decompressed with Python's `zlib` within the same entry size and ratio
limits, its members joined as REA joins them. A JSON file
holding a number JavaScript cannot read exactly, such as an integer above 2^53,
or whose retained value nests more than 500 levels deep, gets an `error` instead of a view, as in
REA. Duplicate keys keep their last value before the depth check. Text that is not one JSON
document, such as JSON Lines, stays text. Format detection follows REA's
rules. Files larger than 64 MiB get an `error` instead of a view
and are not read. XML is read with Python's `expat` twice: with namespace
processing, which refuses what REA's parser refuses, such as an undeclared
prefix, then without it, so element and attribute names keep the prefixes the
document spells and namespace declarations stay attributes, as in REA's view;
text splits into nodes where REA's splits it, at CDATA sections, comments and
processing instructions. Both passes read a DOCTYPE as REA does: with
parameter entity parsing off, so the DTD it names is never read, and handlers
that refuse an internal subset, an external entity and an entity expat would
skip; the first pass also checks each start tag for a reference to an
undeclared entity, which expat drops from an attribute value without a word
when a DOCTYPE names a DTD. Known differences remain:
- dconf databases are reported as binary, since REA reads them with `dconf`;
inside an archive, one is `dconf` with an error, as in REA.
- Zip entry names: a name in an Info-ZIP Unicode Path extra field is used only
where Python's `zipfile` reads it (3.12 and later), and an entry whose local
header names it differently from the central directory is damaged rather
than read.
- SQLite databases are read in place, read-only, rather than from a staged copy;
a write lock the running app holds is waited out for a quarter of a second
and then gives the file an `error`, and so does a table or column name that
is not UTF-8.
Unless it was [made for the task](#a-seed-made-for-the-task), the seed is the
user's real data. Keep the bundle local and do not publish it.
The class was checked against agentenv-framework 0.9.1431 with
`python -m agentenv_protocol.computer._declare` and `agent-env run --dry-run`.
-262
View File
@@ -1,262 +0,0 @@
# Desktop application state
`rea inspect-desktop-application-state <capture-folder>` reads a folder written
by [desktop application capture](desktop-application-capture.md) and describes
the application's state: which observed files are the application's, what each
retained version contains as typed JSON, and how each file changed between
checkpoints. It reads only the capture folder; the application is not run
again.
```sh
rea inspect-desktop-application-state ./mousepad-capture --format json
```
The result is Evidence with operation `inspect_desktop_application_state` and
predicate type `rea.desktop-application-state`. Its confidence is `inferred`,
because roles come from heuristics; the views themselves are read from the
retained bytes, which are checked against their SHA-256 before use.
## Paths and roles
The view covers every file retained at the baseline and every file that
changed later. Each path records the evidence the capture holds about it:
| Reason | Meaning |
| ------------------------ | ------------------------------------------------------------------------- |
| `state_directory` | It lies in one of the application's state directories, in either spelling |
| `opened_by_application` | An application process held it open during a sample |
| `named_in_window_title` | An application window title spelled its name or `~/` path on its own |
| `selected_path` | It lies under a `--state-path` or absolute `--argument` of the capture |
| `dconf_application_keys` | It is a dconf database holding groups whose path names the application |
| `desktop_service` | It lies where desktop services and toolkits keep shared files, or caches |
| `desktop_trash` | It lies in the desktop Trash, `~/.local/share/Trash` |
A folder beside a state directory whose name differs from it only in letter
case counts as that state directory, so a profile the application first
creates during the capture, such as GIMP's `~/.config/GIMP` beside the
candidate `~/.config/gimp`, is its state too.
The role follows from the reasons, in this order: `shared_desktop` for a file
in the desktop Trash, which the user deleted, whatever else names it;
`application_state` for application keys in dconf, or a state directory outside shared desktop
locations; `document` for a file named in a window title or selected by the
caller, even if the application holds it open, except that a selected file
under a hidden folder of the home, such as `~/.config/MuseScore`, a dotfile
there, or a lock an application holds, such as LibreOffice's `.~lock.<name>#`
beside a document, is `application_state`, judged by the path as the caller
named it, so a file reached through `~/.config/MuseScore` stays the application's
when that folder is a link to `~/settings/MuseScore`; the innermost selected
path holding a file decides, so a score selected itself inside the link's
target keeps its own name and is a document (a capture written before it
recorded selected paths is judged by their physical spelling);
`application_state` for another
open file outside shared locations; `shared_desktop` for other files in shared
locations; `unrelated` otherwise. A window title names a file only when no
other file name character adjoins the name, so `day.txt` does not match
`today.txt - Notes`.
Application names, which select dconf groups, come from the package, the
executable, the desktop entry's id and `StartupWMClass`, and the window match.
The entry's display name is left out, since a generic name such as `Settings`
would claim other applications' groups.
## Versions and views
Each path lists its versions: the baseline, then its state after each
checkpoint where it, or a database's `-wal` or `-journal` file, changed. A
checkpoint that leaves an absent file absent adds no version. A file's state
at any checkpoint is its last version recorded at or before it, which is how
the [export](desktop-application-export.md#the-seed-checkpoint) and the
[task draft](desktop-application-tasks.md) read it at the seed and end
checkpoints they are given; `capture.checkpoints` lists the names in order.
Views hold the captured bytes as they are, the capture host's name and user
included; the export removes those from the bundle. A version records
whether the file was present, the SHA-256 of its retained bytes, and a view:
| Format | Detected by | View |
| --------- | ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| `sqlite` | The `SQLite format 3` header | Every ordinary table's columns, primary key and up to 500 rows |
| `dconf` | The `GVariant` header | Groups whose path names the application, plus the names of the other groups |
| `zip` | A zip local file header, `PK\x03\x04`, whatever the name | Its file entries in archive order, each with its own format and view |
| `json` | A `.json` name or a leading `{` or `[` that parses | The parsed value, from files up to 4 MiB nested up to 500 levels |
| `xml` | An XML name or declaration that parses | An element tree (`name`, `attributes`, `children`); text kept exactly, DTDs never read |
| `keyfile` | Only `[group]`, `key=value` and comment lines, with at least one group | Groups of `key` → string value, from files up to 4 MiB |
| `text` | Other UTF-8 text | Lines, up to 5,000 |
| `binary` | NUL bytes or invalid UTF-8 | None |
A gzip file is read as the document inside it; see
[compressed documents](#compressed-documents).
A JSON value holding a number JavaScript would round, such as an integer above
2^53, is not read rather than shown rounded.
An XML view names elements and attributes as the document spells them, prefix
included: a saved Inkscape drawing's root is `svg`, in its default namespace,
with `sodipodi:docname` and the declarations `xmlns` and `xmlns:sodipodi`
among its attributes, and a Writer paragraph is `text:p`. An element's
`children` hold its elements and text in order, text split where a CDATA
section, comment or processing instruction falls. Whitespace-only text between
child elements is formatting, and dropped, in an element that holds no other
text, unless `xml:space="preserve"` applies; in mixed content, an element with
text of its own beside its children such as a paragraph split into runs, it is
kept, so `<p>Meet on <s>Sunday</s> <s>evening</s></p>` keeps the space between
the runs. Whitespace here is XML's: space, tab, carriage return and line feed;
a no-break space is text. The view keeps up to 5,000 nodes, comments and
processing instructions counted, nested up to 64 levels, and sets `truncated`
when it leaves some out.
A document that begins with a DOCTYPE naming its DTD, as AbiWord's `.abw`
does, reads as the same element tree, and the view records the declaration
beside `root`, its public identifier normalized as XML matches it:
```json
{
"doctype": {
"name": "abiword",
"public_id": "-//ABISOURCE//DTD AWML 1.0 Strict//EN",
"system_id": "http://www.abisource.com/awml.dtd"
},
"root": { "name": "abiword", "attributes": {}, "children": [] },
"truncated": false
}
```
An identifier the DOCTYPE leaves out is `null`, and a document without one
has no `doctype`. The DTD is never fetched or read, so it adds no default
attributes and declares no entities. A DOCTYPE with an internal subset
(`<!DOCTYPE a [ … ]>`), which could declare entities, is not read as XML, nor is
a document that refers to any entity but XML's five (`&amp;`, `&lt;`, `&gt;`,
`&apos;`, `&quot;`); such a file is text.
Exported XML views use REA's same formatting-whitespace classification, so
retained XML text and the task checks that address it stay aligned.
SQLite databases are read through REA's SQLite worker from a private copy,
with any retained `-wal` or `-journal` file staged beside it; the worker
accepts an inactive rollback journal and refuses a hot one. A version is not
read when its `-wal` or `-journal` file existed but was not retained, or when
the capture did not record whether either existed, since the database alone
could misstate its committed rows. The capture records them under its content
roots, which include both beside a database selected on its own, when its
baseline walk listed every directory, read every path, entered no other
device, did not stop early and recorded what it left out, and wherever they change; an excluded log, and any elsewhere, is
unknown, since an unchanged file and a missing one look alike. After an
incomplete checkpoint, a database or sidecar without an explicit
later change remains unconfirmed even if a later walk completes. REA does not
combine that older file with a newly recorded member of the SQLite set. The
historical versions remain listed; the affected SQLite view has a `view_error`.
A baseline non-file at a sidecar path, or a non-directory ancestor, cannot
establish an absent file. Captures without the required baseline non-file
inventory must be made again before inspection.
A table
that cannot be read keeps its `error` in place of its rows, and changes to that
database are marked incomplete. dconf
databases are read with `dconf dump /` against a private file profile, so the
live database and the dconf service are not touched; this needs `dconf-cli`, or
`REA_DCONF_COMMAND`.
A version has a `view_error` when its bytes were not retained, are over the 64
MiB read limit, are missing or no longer match their digest, exceed a format's
limit above, or its reader failed.
## Archives
Many applications save a document as a zip archive of XML and other files:
FreeCAD's `.FCStd` and `.FCBak`, OpenDocument (`.odt`, `.ods`, whose
`content.xml` is the document), Office Open XML, EPUB and JAR. Such a file is
`zip`, and its view lists its file entries in archive order, leaving out
directories: an entry whose name ends in `/`, or whose attributes mark a Unix
directory or, from an MS-DOS host, the directory attribute:
```json
{
"entries": [
{
"name": "Document.xml",
"size": 11923,
"sha256": "…",
"format": "xml",
"view": {
"root": { "name": "Document", "attributes": {}, "children": [] },
"truncated": false
},
"view_error": null
}
],
"entry_count": 15,
"truncated": false
}
```
Each entry's bytes are read from the archive's verified bytes in memory,
checked against its CRC-32 and declared size, and given a `format` and `view`
by the rules and limits above, with the entry's name in place of a file name.
Nothing is extracted to disk. An entry has a `view_error`, with `sha256`,
`format` and `view` null, when it is encrypted, compressed other than stored or
deflated, larger than 16 MiB, more than 100 times its compressed size, past the
64 MiB read from one archive, or its data is damaged. A SQLite or dconf
database, or an archive, inside an archive keeps its format and `sha256` and
has a `view_error` instead of a view: it is not read. The view lists at most
1,000 entries; `entry_count` counts them all, and `truncated` is true when some
are not listed. REA reads the central directory itself, a record at a time,
through the Zip64 end record whenever a Zip64 locator precedes the end record,
as Python's `zipfile` does, and hands the zip reader only the records it lists,
so an archive of hundreds of thousands of entries costs no more to read
than one of a thousand. An archive whose central directory cannot be read has a
`view_error` of its own.
## Compressed documents
Gnumeric's `.gnumeric`, Dia's `.dia`, Xournal++'s `.xopp`, GnuCash's default
book and Inkscape's `.svgz` are XML compressed with gzip. A file that starts
with gzip's magic bytes, `1f 8b 08`, whatever its name, is read as the
document inside it: its members joined, then given a format and view by the
rules above under the file's own name, so a pointer or check into it is
written exactly as for the same document uncompressed, and the task draft is
the same. The version records the compression beside the format:
```json
{
"format": "xml",
"compression": "gzip",
"view": {
"root": { "name": "gnm:Workbook", "attributes": {}, "children": [] },
"truncated": false
},
"view_error": null
}
```
The document is read only when it is at most 16 MiB and at most 100 times the
file's size, an archive entry's limits, so decompressing stops there; past
them, or when the stream is damaged or cut short, the version has `format`
null and a `view_error`, and a check into it is undecided. A SQLite or dconf
database inside keeps its format and has a `view_error`, an archive inside is
read as an archive, and another gzip stream inside is binary. A gzip file
inside an archive reads as binary. Other compression, such as xz, Zstandard
or bzip2, is not read: such a file is binary.
## Kinds of change
Between consecutive versions a change is `created`, `deleted`, or a
format-specific kind with what was added, removed and changed (up to 100 of
each, with counts):
- `lines` for text: lines added and removed, as a multiset.
- `values` for JSON and XML: JSON pointers to values, containers included, so
an array that becomes an object with the same members is changed. A pointer
longer than 4096 characters is counted but not listed.
- `values` for zip archives too, with array members named by
[selectors](desktop-application-tasks.md#pointers-and-selectors) rather than
positions wherever a selector names one member in both versions: entries by
name, as `/entries/[name=Document.xml]`, and XML elements by tag and an
attribute, as `/view/root/children/[name=Object][@name=Hole]`, so an object
inserted before another does not shift it. A comparison is incomplete when
either view leaves out entries, or an entry's view is missing or truncated.
- `keys` for key files and dconf: `group/key`, with `~` and `/` in the key
written `~0` and `~1` as in a JSON pointer, so `[a/b]` `c` and `[a]` `b/c`
stay apart.
- `rows` for SQLite: `table:` and the row's primary key, or the whole row when
the table has none.
- `content` for binary files, and `unreadable` when either version has no
view.
-503
View File
@@ -1,503 +0,0 @@
# Desktop application tasks
A [desktop application export](desktop-application-export.md) can carry one
task: a prompt for a computer-use agent and checks that grade what the agent
left in the application's files. An agent, not REA, writes the prompt and the
checks from the change the capture recorded. REA validates the task against
the capture and keeps it only when its checks fail on the seed and all pass on
the captured end state. REA runs no model and no code the agent wrote: the
checks are declarative, and the exported verifier evaluates them the same way
REA does.
```sh
rea describe-desktop-application-task ./mousepad-capture --format json > draft.json
# edit the draft's task into task.json: a name, the user's request, the checks
rea export-desktop-application ./mousepad-capture ./mousepad-bundle --task task.json
agent-env run ./mousepad-bundle --task add-milk --agent claude-code
```
## Describe the change
`rea describe-desktop-application-task <capture-folder>` reads the capture and
its [state view](desktop-application-state.md) and returns Evidence with
operation `describe_desktop_application_task` and predicate type
`rea.desktop-application-task-draft`:
- `application`: the environment name the export would use, the title and the
window class.
- `owned_paths`: the paths the export owns, the only ones whose files the
app's `data/get` serves and checks can read.
- `checkpoints`: the two checkpoints compared, `baseline` for the seed and
`final` for the end. They are the capture's first and last unless
`--seed-checkpoint` and `--end-checkpoint` name others, as for the
[export](desktop-application-export.md#the-seed-checkpoint), which takes
the same options; a name the capture does not hold, or an end before the
seed, is refused.
- `files`: each file under them whose state differs between those
checkpoints, with its home path, role, its versions at both (`baseline` and
`final`, the last recorded at or before each) and the net change between them, from the state view's own diff (`lines`, `values`,
`keys` or `rows` with what was added, removed and changed, or `created`,
`deleted`, `format`, `content`, `unreadable`; `content` also when the
views hold only part of a file and match while the file changed). A
database is compared by its rows, since its hash leaves out the `-wal`
file. Text changes list each line
once and add `line_counts`, the lines found at both ends whose number of
occurrences changed. An XML file's change names its members by the
[selectors](#pointers-and-selectors) checks use, as an archive's already
does, and by position only where nothing names a member or where only the
members' order changed. Views appear only for the files the suggestions
cover, with arrays past 200 items and strings past 2,000 characters cut
(`view_cut`).
- `values`, in each of those files: the values its change added, changed and
removed, each with the `pointer` a check names it by, its `kind` (`added`,
`changed` or `removed`), and `before` and `after`: the scalar, an element as
compact XML with the text the XML reader kept, spaces between runs included,
or a JSON value as compact JSON, cut at 600 characters, and null at the end
that lacks it. A file created or deleted lists what its one end holds,
`added` or `removed`: an archive's entries, an XML file's root element, a
JSON file's members or a key file's keys, each whole once. A value only
position reaches has `note: "by_position"`, and its pointer cannot go into a
check; noise an app writes as it saves has `view_state`, `renumbered`,
`volatile`, `timestamp` or `upkeep` (see `task`). `refers_to` names up to
three other elements a value equals the `key` or `id` of, with the
`attribute` holding it, as HomeBank's `category="44"` names
`<cat key="44" name="Groceries"/>`; where several elements share that key,
the one whose tag shares the attribute's stem, as `payee` and `<pay>`. What a check can
name comes first, then what only position reaches, then noise. A file lists
at most 20 values and 8 KB, the draft 24 KB, and `values_cut` says when a
limit left some out. Values cover XML, archives, JSON and key files; a text
or SQLite file's change and suggestions already carry its lines and rows.
- `task`: a draft with the name `change`, a placeholder prompt the export
refuses, and suggested checks derived from the net change of the files with
role `document`, or of the app's own state files when no document changed.
A text file gains `lines_present` for its new lines and `lines_absent` for
lines it lost (when either is at most ten), and `line_count` for each line
holding a letter whose count changed; JSON and key files gain
`value_equals` and `value_absent` by pointer; SQLite tables gain rows added,
changed and removed, by primary key, and their new row count. An XML file,
and the JSON, key file and XML views of a zip archive's entries, gain checks
by [selectors](#pointers-and-selectors) and never through an element by
position: in XML, first [`text_equals`](#whole-text) for each element whose
text changed, then elements added or removed whole, then values changed,
added and removed, an entry or a JSON value added whole through its
members, and an empty array or object as it is.
- A selector names a member by its tag, then by its identifiers (`name`,
`id`, `xid`, `xml:id`), then by one other attribute, a name-like key such
as Scribus's `ANNAME` or `inkscape:label`, a worded value or an integer of
up to five digits first, then by two such attributes together, as
Gnumeric's cells by `[@Col=3][@Row=2]`, and last by a coordinate, a long
number or a token, such as `XPOS` or `ItemID`, or an attribute naming
view state, such as which layer is `selected`. A pair counts every
member holding both its values, whichever attributes that member ranks
first. An element with identifiers is named by those alone. The selector
must name the member alone at both ends, with the same values. An
element whose one identifier changed, at its place among its tag's
members, with every other attribute and its children alike, is that
element renamed, as a MyPaint layer renamed from `Untitled` to `Sun`: it
is named by its first other attribute naming it alone at both ends, as
`[@src=data/layer-00.png]`, so the rename reads as its `name` changed,
and the old name gets a check that it names nothing, unless its change
is noise, as an id the app renumbered.
An identifier the app numbers by position names nothing: where the
members of a tag hold it as 1..N (or 0..N-1) in document order at both
ends, members were inserted or removed, and every member another
attribute names alone at both ends kept its number or moved it the way
the count did, one at least, as Planner's task `id` after a task is
inserted, members are paired by their other attributes, so an insertion
reads as one task added. A value two members swap, or an id appended at
the end, as Tellico's next `<entry id="4">`, leaves the ids identity.
- The text check names the paragraph rather than a run inside it: it goes up
from an element sitting in its parent's text, and from one named by its
tag or position alone to the nearest ancestor named by an attribute, as
AbiWord's `p` by its `xid`; where no attribute names one, to the nearest
ancestor within three levels its tag names with no step by position, as
MuseScore's `VBox` around untitled `<Text>` elements; and where only
position reaches the element, to the ancestor above the outermost step by
position, by an attribute of its own that names it alone where one does,
as an ODS sheet's `table:table[@table:name=Q1 Sales]` above its rows. The
text nodes and runs inside it get no other check, but an element holding
no text that is added or removed there, such as an image or a line break,
keeps its own, since the text check never reads it. A decimal that is an
element's only text, such as a spreadsheet cell's, is checked by its
number instead. Free text that gains a line, such as a shopping list,
uses `text_equals` on the whole text: the current XML checks cannot
compare an unstructured line as a whole item. This can reject a correct
edit that puts the item elsewhere or changes spacing. A record added
or removed whole, an element holding only elements, its fields, such as
Tellico's `<entry>` or a record of repeated `<field>` elements, never
makes the list holding records of its kind a check, whose text would hold
the records the edit left alone, except a record removed whole that only
position names, whose absence only the list's text shows. Text made of
runs is no record: an element whose tag names a paragraph or a piece of
text (`p`, Writer's `text:p` and `text:h`, `h1` to `h6`, `li`, `para`,
SVG's `text` and `tspan`, a `span` or an `a`), or whose runs split a
sentence at a space, as `Meet ` and a linked `Monday`. Nor does an
element holding a date the app stamps in parts take a text check.
- An element added to a list, where another element of its tag is at either
end, gets a `value_equals` for each of its attributes, best ranked first,
at most six, leaving out its selector's attributes, identifiers, times,
volatile values and view state, as HomeBank's new `<ope>` gets its
`amount`, `category` and `payee`. An element whose other attributes, two
or more, a like sibling holds every one of, as a new Planner task copies
its neighbour's defaults, and any other element added whole, get a check
that it exists. A record added to a list that a selector names, as
Tellico's `<entry id="4">`, gets a `text_equals` for each field, at most
five, each by its selector among the record's fields, as `[name=title]`,
leaving out a field repeating the record's identifier, a date the app
stamps in parts, and blank text or times. A record only position names,
such as Osmo's `<record>` of fields, also gets a `text_contains` for each
new field of three or more characters, at most five, at the nearest
ancestor a check can name, each absent from that ancestor's text at the
seed.
- A decimal written as text, such as `146.83`, is suggested as a number
with `tolerance: 0.001`, unless the seed's value is that close too.
- Noise leaves the suggestions when the file's change holds anything
else, a change only position reaches included: view state, which a tag
or attribute on the way names (the cursor, a selection, scrolling, zoom,
the viewport, windows, splitters, Inkscape's `namedview`, view and
configuration settings, the printer, a save's history, edit time,
editing duration and cycles, last-saved time, the generator and
document statistics), or the value of a name-like attribute or `key` on
the way, when it is one token of several words, as LibreOffice's
`config:config-item[@config:name=CursorPositionX]` inside
`[@config:name=ooo:view-settings]`; an attribute ending in `ID` or `Id`,
or an `rsid`, that the app renumbered, with everything else in its
element, as Gnumeric empties a formula cell whose shared expression it
renumbers, and an identifier the app numbers by position; volatile
values, UUIDs and epoch times, and durations and counts under a
document's metadata; times the app stamps, also as a date written in
parts whose tag, or one it sits in, names when the app made, changed or
saved something, as Tellico's `<cdate>` of `<year>`, `<month>` and
`<day>` (`cdate`, `mdate`, `created`, `modified`, `saved`, `updated`),
which no check ever reads, while a date a user enters in parts, as a
birthday, is an edit like any other; and `upkeep`, the
`loext:tab-stop-distance` attributes LibreOffice fills in `styles.xml`
while another archive entry holds an edit. However much noise
comes first, it never takes a check's place. When the change holds
nothing else, the checks stay and a note flags them: up to three in a
document, and as many as fit in the app's own state, where a view
setting may be the task.
- Whitespace an app lays out between elements is not checked, and values
that look like a time the app stamps, in ISO 8601, as C's `asctime`
writes it (`Thu Nov 5 09:20:31 2026`) or as a time of day to the
second (`05:16:21.675302540`), are left out; noise never makes its
ancestor's whole text a check.
- `guards`: checks an author may add to show the edit left the rest alone,
kept out of `task.checks`, with ids `guard-1` and on: for each suggested
`value_equals` or `text_equals` on a member of a list, a field on up to
two like siblings, nearest first, at its seed value, which the end holds
too: the check's own field where like siblings hold different values in
it, and otherwise one that varies among them, as Planner's `work` rather
than the `type="normal"` every task shares; never through a sibling only
position or a renumbered id pairs. A text file gets a `lines_present` of the
unchanged list items beside its first new line with the same marker, as
Zim's other `[ ]` action items, or else its nearest lines with words in
them. At most four a file. They pass on the seed, which the export allows,
since one check failing there is enough.
- `notes`: why `files` is empty, or why a file's suggestions are thin. Each
note has a `code` and a `message`. For a draft with no files: `no_change`
when no file under the home changed between the two checkpoints and nor
did the application's dconf keys; `unowned_changes` when files changed but
none under the owned paths, with up to five of them in `files`, each with
its home path and role (desktop services' files last), and how many more
in `more`; `uncheckable_changes` when the application's keys in a dconf
database changed, with the database in `files`: no check reads dconf, and
the export only writes the keys as they were at the seed checkpoint, so
make the task's change one the app saves in a file; `unreadable_changes`
when a dconf database changed but REA could not read it, with the read
failure in `error` (install `dconf-cli` or set `REA_DCONF_COMMAND` when it
is missing), since whose keys changed is then unknown; and `no_document`
when no file has the document role, as when no window title shows the
file's name. A capture again with `--state-path` naming the file, or
`state_paths` on a build computer's `start_build_capture`, makes it the
app's. For a drafted file, named in `file`: `by_position` when
changes reach their element only by position, the tags no attribute names
alone in `elements`, or when only the elements' order changed;
`view_state_only` when every change is noise, the checks kept in
`pointers`; `long_text` when an element's text changed past the 2,000
characters a text check compares, the elements in `pointers`; `cut_view`
when the views hold only part of the file; and `unreadable_view` when no
check can read it, as a document REA reads as binary.
The suggestions are a starting point. Keep the ones that say what the user
did; drop the ones that only follow from how the app saves its file. Read
`values` for what changed and its pointers, and `refers_to` for what a number
stands for, before writing the prompt and any check of your own. A LibreCAD
capture that added four holes to a drawing rewrote it from DXF R12 to DXF 2007,
so most of its lines changed; the draft suggests counts of `SECTION`, `ENDSEC`
and `CIRCLE`, and only `CIRCLE` going from 5 to 9 is the user's change.
## The task file
```json
{
"name": "add-milk",
"prompt": "Add buy milk to my to-do list in Mousepad and save it.",
"checks": [
{
"id": "milk",
"kind": "lines_present",
"file": "Documents/todo.txt",
"lines": ["buy milk"]
}
]
}
```
`name` is a portable bundle name other than `smoke` (lowercase letters, digits
and hyphens); `prompt` is what the user asks for; `checks` holds 1 to 32
checks. Each check has a `kind`, a `file` given as a path under the home such as
`Documents/todo.txt`, and an optional `id` (`check-1`, `check-2` and so on by
position otherwise). A check reads the typed view `data/get` returns for that
file, in the shapes of the state view.
| Kind | Fields | Passes when |
| --------------- | ------------------------------------------------------------ | ------------------------------------------------------------------------ |
| `file_present` | | The file exists |
| `file_absent` | | The file does not exist |
| `lines_present` | `lines` | Every listed line is a whole line of the text view |
| `lines_absent` | `lines` | None of the listed lines is |
| `line_count` | `exact` or `contains`; `equals`, `at_least` or `at_most` | The number of lines equal to `exact`, or holding `contains`, compares so |
| `value_equals` | `pointer`, `value`; optional `tolerance` | The pointer names a value equal to `value`, or within `tolerance` of it |
| `value_absent` | `pointer` | The pointer names no value |
| `text_equals` | `pointer`, `value`; optional `normalize_whitespace` | The XML element's [whole text](#whole-text) equals `value` |
| `text_contains` | `pointer`, `value`; optional `normalize_whitespace` | The XML element's whole text holds `value` |
| `row_present` | `table`, `where` | A row of the SQLite table has every column in `where` equal |
| `row_absent` | `table`, `where` | No row does |
| `row_count` | `table`; optional `where`; `equals`, `at_least` or `at_most` | The number of rows, or of rows matching `where`, compares so |
The rules both evaluators follow:
- An absent file reads as empty: no lines, values or rows. So `lines_absent`,
`value_absent` and `row_absent` pass on it and a count is 0.
- A file that has no view (binary, over the read limit, or unreadable) fails
every check but `file_present` and `file_absent`. Line checks need a text
view, row checks a SQLite view and text checks an XML view, in a file or an
archive entry; another format fails.
- Equality is JSON equality: numbers by value, never a boolean for a number,
objects by their keys whatever their order, arrays in order.
- Pointers follow RFC 6901 (`~1` for `/`, `~0` for `~`) and must start with
`/`; on an array a segment may be a selector, below. Key files read as
`{"groups": {"GROUP": {"KEY": "VALUE"}}}`, so a setting is
`/groups/GROUP/KEY`; XML as `{"root": {"name", "attributes", "children"}}`;
SQLite as `{"tables": {"TABLE": {"columns", "rows"}}}`; a zip archive as
`{"entries": [{"name", "format", "view", …}]}`; a gzip file, such as a
Gnumeric `.gnumeric`, as the document inside it, so its pointers are those
of the same document uncompressed.
- With `tolerance`, a non-negative number, `value` must be a number, and the
value found passes when it is a number, or a string that is a decimal
numeral such as `"5.0000000000000000"`, at most `tolerance` from `value`.
`tolerance: 0` compares numerically, so `"1.2500"` equals `1.25`. Anything
else fails as not a number.
- `contains` matches a line that holds it anywhere, case and all. Checks take
no regular expressions: a pattern run on what the agent wrote could stall
the verifier, and JavaScript and Python read some patterns differently.
- A check decided by what a view leaves out (`lines_absent`, `value_absent`,
`row_absent`, the counts, `value_equals` of a whole object or array, and
the text checks of an element) fails when that view is truncated; a check
satisfied by what the view holds is decided on it. An element's text is not:
the reader may have dropped text from between the runs it kept, so
`text_contains` fails on a truncated view even where the kept runs hold its
value. A text node or attribute value is whole, and decided.
The export refuses, naming the check, a file outside the owned paths, a file
spelled with a leading slash, `.` or `..`, a pointer into a file that is text
at the seed, as the bundle ships it, or at the end checkpoint, the empty
pointer (the whole file), a value that names the captured home, or the
capture host's name or `user@host` anywhere (which the
[export removes from the seed](desktop-application-export.md#host-name-and-user)), a
`tolerance` with a value that is not a number, an empty `text_contains`
value, or one of only whitespace with `normalize_whitespace`, and a value that
looks like a date and time, in ISO 8601 or `asctime`'s form. Other volatile
content, such as a
counter the app bumps on every save, is for the author to leave out.
## Pointers and selectors
An app that rewrites a document moves things: FreeCAD 1.0 records more
properties than 0.21, so a hole's radius sits at another position among its
siblings, and a plate's entry moves once a hole is added before it. A pointer
segment applied to an array may therefore name a member by what it holds
rather than by its position:
- `[KEY=VALUE]` names the member whose string field `KEY` is `VALUE`, such as
a zip entry by `[name=Document.xml]` or an XML element by its tag,
`[name=Object]`.
- `[@KEY=VALUE]` names the member whose `attributes` hold `KEY="VALUE"`, as
the XML view keeps an element's attributes.
- Conditions can follow one another, `[name=Object][@name=Hole]`, and all must
hold. They compare strings only, the selector is escaped like any pointer
segment (`~1` for `/`), and a value cannot hold `][`.
A selector must name exactly one member. When several match, or none does
before the last segment, the check fails and says which selector matched what.
When the last segment's selector matches none, the pointer names no value:
`value_absent` passes, which is how a check says an element was removed, and
`value_equals` fails. On an object, a bracketed segment is an ordinary key.
A FreeCAD task that checks the user cut a 10 mm hole through the plate, in
FreeCAD 0.21 and 1.0 alike:
```json
{
"name": "cut-hole",
"prompt": "Cut a 10 mm hole through the middle of the plate and save it.",
"checks": [
{
"id": "cut",
"kind": "value_equals",
"file": "Documents/plate.FCStd",
"pointer": "/entries/[name=Document.xml]/view/root/children/[name=Objects]/children/[name=Object][@type=Part::Cut]/attributes/type",
"value": "Part::Cut"
},
{
"id": "radius",
"kind": "value_equals",
"file": "Documents/plate.FCStd",
"pointer": "/entries/[name=Document.xml]/view/root/children/[name=ObjectData]/children/[name=Object][@name=Hole]/children/[name=Properties]/children/[name=Property][@name=Radius]/children/[name=Float]/attributes/value",
"value": 5,
"tolerance": 0.01
}
]
}
```
Before the edit the archive has no `Part::Cut` and no `Hole`, so both checks
fail on the seed; after it, `Radius` is `"5.0000000000000000"` and both pass.
A check decided by what a view leaves out depends, in an archive, on the entry
its pointer goes into: it fails when that entry's view is missing or
truncated, or when the entry is not listed and the listing is truncated. A
selector must see every member to know it names one, so a check whose pointer
searches an array the view may hold only part of fails as undecided: any array
of a truncated XML view, of a truncated entry's view, or the listing when it
is cut. A position is decided on what the view holds. Like a file, an entry
that has no view (binary, not read, or a database or archive inside the
archive) fails every check into its `view`; its `null` there is no value, while
an entry holding the JSON document `null` reads as `null`. Checks on its `name`,
`size`, `sha256` and `format` still read.
XML that uses namespaces is named as the document spells it, in both
evaluators: an element or attribute by its prefixed name, such as `text:p`
or `inkscape:label`, an element in a default namespace by its bare name, such
as `svg`, and namespace declarations as the attributes `xmlns` and
`xmlns:PREFIX`. Text is a string among an element's `children`. These checks
hold a poster's headline in an Inkscape SVG and a date in a LibreOffice Writer
document:
```json
[
{
"id": "headline",
"kind": "value_equals",
"file": "Pictures/poster.svg",
"pointer": "/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0",
"value": "Summer Bake Sale"
},
{
"id": "date",
"kind": "value_equals",
"file": "Documents/flyer.odt",
"pointer": "/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Subtitle]/children/0",
"value": "Monday 24 November"
}
]
```
Inkscape writes a line of text as a `tspan` inside its `text` element, and
Writer keeps a paragraph's style in `text:style-name`. A prefix is the one the
document uses, so a check names the prefixes the app writes.
## Whole text
A text node is only the stretch of text between two pieces of markup, and
apps split a paragraph into runs as they save it. AbiWord wraps text typed
into a reopened document in `c` elements, so changing a time to `3:30` saves
`<p>The team meeting starts at <c>3</c>:<c>30</c> in room 4.</p>`; Writer,
which records revision IDs by default, writes text typed in one session as
`text:span` runs. A `value_equals` on the paragraph's `children/0` then reads
`"The team meeting starts at "` and fails. `text_equals` and `text_contains`
read the element's whole text instead:
```json
{
"id": "time",
"kind": "text_equals",
"file": "Documents/meeting.abw",
"pointer": "/root/children/[name=section]/children/[name=p][@xid=3]",
"value": "The team meeting starts at 3:30 in room 4."
}
```
- The pointer names an element of an XML view, in a file or in an archive
entry's `view`, or a text node or attribute value in one, whose text is its
value. Another format, a pointer into an archive outside an XML entry's
view, or any other value fails, an element's `name` and the `doctype`'s
fields included: they are strings of the view, not text of the document.
- An element's text is XPath's `string()` of what the view keeps: every text
node in it and in its descendants, CDATA included, in document order, with
nothing between them. The view drops whitespace-only text between child
elements only in an element that holds no other text, outside
`xml:space="preserve"`, so the space between two runs of a paragraph is
part of its text. Elements that stand for characters, such as Writer's
`text:s`, `text:tab` and `text:line-break`, add none.
- `text_equals` passes when the text equals `value` exactly, case and all;
`text_contains` when it holds `value`, as a time within a sentence, so the
check holds whatever the rest of the sentence says.
- `normalize_whitespace: true` collapses each run of XML whitespace (space,
tab, carriage return, line feed) in both the text and `value` to one space
and trims both ends before comparing, as XPath's `normalize-space()` does,
for an app that reflows text or a value written across lines. A no-break
space is not whitespace there.
## Validation and export
`rea export-desktop-application <capture> <directory> --task <task.json>`
evaluates the checks on the capture's state view: on each file as it was at
the seed checkpoint, which is the seed, as the bundle ships it with the
capture host's name and user removed, and at the end checkpoint, the
captured end state. Those are the capture's first and last checkpoints unless
`--seed-checkpoint` and `--end-checkpoint` name others.
It refuses a task whose every check passes on the seed, and a check that fails
on the end state, with that check's reason. Only a failure on what the seed's
view shows counts: a task whose checks fail on the seed only because a view
could not be read or may have left out what they look for, such as a line past
the 5,000 a text view keeps, is refused, naming each failure. A capture yields
one task, so the option is taken once. The result's `task` names the task, its
number of checks and those that fail on what the seed shows. A refused task
writes no bundle.
The bundle then holds, besides the app and `tasks/smoke.json`:
| Path | Contents |
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `tasks/<name>.json` | Deploy the app, deploy the agent and its skills, record the screen, prompt the agent, run the verifier, stop recording |
| `artifacts/<name>-check/verify.py` | Reads the app's `data/get` through the computer and evaluates the checks, one result per check |
| `evals/<env>.toml` | The task, so a run of the bundle runs it; the smoke task runs with `--task smoke` |
The verifier finds the app among the computer's child environments, calls its
`data/get` under the run's grant of the computer's control token, and reads
the `state` data part. It reads again until the app's files settle, as
[grading once the app has saved](desktop-application-export.md#grading-once-the-app-has-saved)
describes, and each result's `settled` says whether they did, and `read_cut`
whether the bound cut a read still running. It raises when the app cannot be reached, so the run
fails rather than scoring the agent. The README lists the task, its prompt, its
checks and the recipe below.
## Solve rate
REA does not run agents. To measure how often an agent solves the task, run it
k times, three by default, and count the runs whose summary line says
`passed`:
```sh
for run in 1 2 3; do
agent-env run ./mousepad-bundle --task add-milk --agent claude-code --model MODEL
done
```
The agent defaults to Claude Code and the model is the caller's choice (Claude
Sonnet 5.5 is a reasonable reference). A run reported as `scored below 1`
failed the checks and counts as a miss; one reported as `failed` did not finish,
as when the computer or the app could not be reached, and is run again rather
than counted. Each run's instance keeps the verifier's results per check under
`verifications`.
+6 -17
View File
@@ -53,7 +53,7 @@ Every collection supports `fields`, `offset` and `limit`; its stable row ID is
always included. Selection changes delivery, not source inspection. Pages keep
source coverage and parent Evidence links. Page exhaustion does not establish
absence in a partial source. Complete Evidence remains accessible through
`get_evidence_bundle` and Evidence bundle export.
`get_evidence` and Evidence bundle export.
The summary's `probe_issues` counts files with reported issues and their retained
issue entries. It previews at most two such files, including stable `file_id`,
@@ -110,15 +110,6 @@ not establish full metadata validity or correspondence to a native binary;
version 24 inspection still applies only to supported header layouts. See
[IL2CPP metadata](il2cpp-metadata.md).
In a managed directory with a Unity Mono witness, `Assembly-CSharp.dll` and
`Assembly-CSharp-firstpass.dll` get an `inspect_managed_members` probe listed
first; project-named asmdef assemblies keep only the ordinary managed probes.
The same probe is offered for those names in `Managed/` of a
`<Product>_BackUpThisFolder_ButDontShipItWithYourGame` folder (or when that
folder is the root); whether its IL matches the shipped IL2CPP code is unknown.
Point the scan at the game folder: a root that is the `<Product>_Data` or
`Managed` folder itself has no Mono witness path and gets no member probe.
Candidates are derived or inferred from these facts and do not prove loading or
execution. BepInEx, MelonLoader, Easy Anti-Cheat and BattlEye labels identify
path conventions only; runtime presence stays unknown. No bypass is provided.
@@ -144,13 +135,11 @@ observations, not an atomic directory snapshot. Unread bytes remain unknown.
## Resource bounds and reuse
Defaults are 20,000 examined entries and 8 MiB of total probe reads. Callers may
select up to 100,000 entries and 64 MiB. A separate 32 MiB representation
budget bounds retained rows, paths and facts before growth. Each retained path
is charged at the larger of its UTF-16 heap size and its escaped JSON size, so
typical macOS bundle paths fit about 20,000 rows, and control-character paths
fewer. Limits and failed probes produce partial coverage. Traversal and reads
are serial. Each reader admits one inspection at a time; overlapping requests
receive a retryable capacity refusal without cancelling the admitted caller.
select up to 100,000 entries and 64 MiB. A separate 32 MiB conservative
representation budget bounds retained paths and facts before growth. Limits
and failed probes produce partial coverage. Traversal and reads are serial.
Each reader admits one inspection at a time; overlapping requests receive a
retryable capacity refusal without cancelling the admitted caller.
Unity reference decoding stops after a witness in each `Data/Managed` directory;
other assemblies' references remain unexamined, while their headers are still
-207
View File
@@ -1,207 +0,0 @@
# Desktop build computer
A build computer is a local agentenv computer that an agent drives while it
builds a desktop app environment. Claude Code cannot attach a new MCP server
in the middle of a session, so REA keeps the computer up and forwards the
agent's computer-use actions to it through three MCP tools:
| Tool | What it does |
| ----------------------- | ------------------------------------------------------------------------------------------ |
| `start_build_computer` | Starts a computer from a request or a bundle folder; returns its id and a first screenshot |
| `act_on_build_computer` | Sends one action to the computer's driver and returns the driver's answer, images included |
| `close_build_computer` | Tears the computer down, by agentenv's teardown or by removing its sandbox's containers |
The tools are MCP-only, like the debugger sessions: the computer lives as long
as the REA MCP server that started it, or until it is closed. Capturing the
app, exporting its bundle and witnessing the task on such a computer are in
[Building and witnessing an environment](desktop-build-witness.md), and REA's
skill for doing all of it from a request is
[desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md).
## Requirements
- Docker, and an agentenv-framework install whose `agent-env` program is named
by the absolute `REA_AGENT_ENV_COMMAND`. Bundles target agentenv
0.9.1431. REA runs its helper, `bridge/agentenv/computer.py`, with that
runner's Python: the interpreter on the program's `#!` line, or the
`python3` beside it in its virtual environment.
- `REA_DOCKER_COMMAND`, the absolute path of the Docker CLI, lets REA remove a
computer whose agentenv teardown was not confirmed. Without it, REA names the
command to run instead.
- For a request, `REA_BUILD_COMPUTER_REA_ARCHIVE`: the absolute path of a tar
archive of a runnable REA, with its production dependencies, whose one top
folder holds `scripts/rea.mjs`. The computer runs REA from it. From a
checkout, `npm run pack:build-computer-runtime -- /path/to/rea-prod.tar.xz`
writes one: `npm pack`, its package folder as
`rea-prod` with the lockfile, `npm ci --omit=dev --omit=optional`, then
`rea-prod` as `.tar.xz`.
Set these where the MCP server starts; a shell export does not reach a server
that is already running.
## Start a computer
From a request, REA renders a bundle in a private temporary folder: a computer
app on the `ubuntu-24.04` base with the `reference` driver that installs the
app, opens it on demand with `gtk-launch <desktop entry>` from the computer
user's home, so its open and save dialogs start there rather than in the
framework's folder for the app, and keeps no seed and no data hooks. The app is not open when the computer starts:
`start_build_capture` opens it under REA's capture. When no capture is under
way, open it from the desktop's Applications menu, where agentenv lists it
under Other by the environment's name, such as `mousepad`; that entry launches
it through the app's server.
When the app is Electron, set `electron` to true. The install rewrites that
desktop entry's command with the same policy an export uses: `--no-sandbox`
immediately after the program, unless the command already has it. The
computer's container does not grant the user namespaces Chromium's sandbox
needs, and Electron exits without the flag. With the rewritten command, the
Applications menu entry can open the app.
```json
{
"source": {
"kind": "request",
"desktop_entry": "org.xfce.mousepad",
"window_class": "Mousepad",
"install": { "kind": "archive", "package": "mousepad" }
}
}
```
`install` takes one of:
| `kind` | Installs |
| ---------------- | ---------------------------------------------------------------------------------------------------------- |
| `archive` | `package` from the Ubuntu 24.04 archive, on amd64 and arm64 |
| `apt_repository` | `package` from the repository at `uri` with `suite` and `components`, trusting the key file `key_file` |
| `package_file` | The `.deb` at `url`, checked against `sha256`, for one `architecture`, after its apt `dependencies` are in |
Build steps run with the network off, so a `.deb`'s dependencies must be
listed. The bundle adds REA in a build-only layer: Node.js
24.22.1, downloaded per architecture and checked against its release checksum,
and REA from `REA_BUILD_COMPUTER_REA_ARCHIVE`, both unpacked into `/opt/rea` by
a root build step, with `node` and `rea` on `PATH`. An exported bundle carries
none of it.
From a bundle folder, such as one `export-desktop-application` wrote, REA
starts that bundle's `tasks/smoke.json` as it is:
```json
{ "source": { "kind": "bundle", "path": "/home/ada/mousepad-bundle" } }
```
REA runs its helper with `PYTHONDONTWRITEBYTECODE=1`, which agentenv's own
import of the bundle's app class inherits, so a start leaves no `__pycache__`
in the folder and its files stay as they were shared.
REA runs the deploy with `run_bundle(keep=True, emulate=False)` on the local
sandbox provider, so only the host's own architecture is built. The first start
of a bundle builds its image and can take minutes; later starts reuse it.
agentenv makes a start wait while another agentenv run on this host writes its
bundle's ids, so starts at the same time wait for each other, several minutes
when a few run together; the progress line then says it is waiting for another
agent-env run to finish writing this bundle's ids.
While it waits, REA sends an MCP progress notification every five seconds with
agentenv's latest line, so a client that resets its request timeout on
progress keeps waiting. REA then waits until the computer's readiness checks
pass and opens one MCP session to its driver. Readiness can pass before the
desktop has drawn, so REA takes a whole-screen PNG screenshot every 500 ms
while it is one colour throughout. The retries share 10 s, each given only
what is left of it, and when that runs out REA returns the latest frame;
progress notifications continue meanwhile. Given `screenshot` with a
`region`, `format` or `quality`, REA then takes one with them, a `region`
cropping that frame. The result holds
`computer_id`, agentenv's `sandbox_id`, the environment name or bundle folder,
and the screenshot's frame as structured content beside its image.
A failed start returns the last 16 KiB of the helper's stderr, where a failing
build prints its error, and its exit status; a cancelled one stops the helper,
which tears down whatever agentenv brought up. When that teardown is not
confirmed, the error's `computer_id` names what REA keeps for
`close_build_computer` to retry. Closing the server cancels starts still in
progress and waits for them before closing every computer.
## Act
`act_on_build_computer` takes `computer_id`, an `action` and the driver's own
arguments for `computer_<action>`, which REA passes through:
| `action` | Arguments |
| ------------ | --------------------------------------------------------------- |
| `screenshot` | `region`, `format` (`png` or `jpeg`), `quality` |
| `click` | `x`, `y`, `button`, `count` (2 double-clicks), `modifiers` |
| `drag` | `path` of `[x, y]` points, `button`, `duration_ms`, `modifiers` |
| `scroll` | `direction`, `x`, `y`, `amount`, `modifiers` |
| `type` | `text`, where `\n` presses Enter and `\t` Tab |
| `key` | `keys`, one chord pressed in order, and `repeat` |
| `wait` | `ms` |
| `reset` | `screenshot` |
`reset` is not a driver tool: REA calls the app's `data/reset`, which restores
the bundle's seed and relaunches the app, then waits until the computer's
readiness checks pass again, sending an MCP progress notification every five
seconds while it waits. With `screenshot: true` it then takes a screenshot; if
that fails, the reset still counts as applied and its result has no `frame`
but a `screenshot_error` with the computer's message. A computer started from
a request has no data operations, so it refuses a reset.
Pointer actions also take `frame_id`, `space` and `norm_scale`, and every
action but `screenshot` takes `screenshot: true` for an image afterwards. That
image is taken at once, so after an action that opens a menu or a window, ask
for it with a `wait` of a few hundred milliseconds instead.
Points are pixels of the latest frame, so take a screenshot first; with
`space: "norm"` they are on a `0..norm_scale` grid over it. Keys are W3C names
or one character; REA translates model spellings such as `ctrl+s`, `Return`,
`esc` and `cmd` and leaves other names for the driver to judge. An argument the
action does not take is refused before anything is sent.
The result carries the driver's content unchanged, its text and images, after
REA's own JSON: `action_index` and the driver's structured result. A refusal,
such as a stale frame, is an error carrying the computer's message. A response
larger than the MCP budget is refused with how to ask for a smaller image: a
`region`, or `jpeg` with a lower `quality`. That error keeps REA's own result,
`computer_id` included, as `details.result`, since the action or start has
happened.
Actions, the capture steps and the task checks on one computer run one at a
time, in the order REA receives them; one that has to wait for the call
before it sends an MCP progress notification every five seconds meanwhile.
Different computers run side by side.
Every action is logged on the computer in order with the arguments sent, its
start and end and whether it was applied, refused or failed. The log keeps the
latest 256 actions within 256 KiB of JSON, dropping the oldest first. Each
string argument, such as `type`'s `text`, is kept to its first 1 KiB and each
list, such as `drag`'s `path`, to its first 64 items; an entry names the
arguments it cut in `truncated_arguments`, and the driver gets them whole. The
log counts every action, so it says how many it dropped, and keeps the index of
the last applied action of each kind for rules over the whole session. The log
and the computer's recording hooks (`computer-recording/v1` start and end under
REA's own request id, and the segments fetched through the driver's `data/get`
one at a time, each read within its manifest size and checked against it) are
what the witness records; they are not tools of their own. A recording larger
than a quarter of the V8 heap is refused before any segment is read.
## Close
`close_build_computer` ends REA's MCP session and closes the helper's stdin,
on which the helper runs agentenv's teardown. When agentenv confirms nothing is
left, the result says `teardown: "agentenv"`. Otherwise REA removes the
containers of that exact sandbox: agentenv runs a computer as
`agent-<sandbox id>` without a label and labels its other containers
`agentenv.sandbox=<sandbox id>`, so REA lists both with `docker ps -aq
--filter`, removes them with `docker rm -f`, and says
`teardown: "container_label"`. When neither confirms it, the error names what
is left and the computer stays listed, so closing it again retries. A close
cancels the call in progress on the computer, refuses the calls waiting for
it, and tears down once that call has ended. Closing the
MCP server closes every computer the same way, and a helper whose REA server
exits sees its stdin close and tears down.
The control token that guards the computer's framework-only routes stays in
REA: no result, error or log carries it. Code running in the computer as its
user can still read it, which is acceptable for a local build.
See the [specification](specs/desktop-app-environments/build-computer/spec.md)
for scope and verification.
-280
View File
@@ -1,280 +0,0 @@
# Building and witnessing an environment
An agent such as Claude Code builds a desktop app environment from a request
on a [build computer](desktop-build-computer.md): it makes the seed through
the app, makes the change the task asks for, exports the bundle with the task
it writes, and then witnesses the task in a fresh computer of that bundle.
REA runs its own capture, task draft and export inside the build computer and
grades the witness with the same checks the exported verifier runs.
| Tool | What it does |
| -------------------------- | ------------------------------------------------------------------------------------------- |
| `start_build_capture` | Downloads the package lists as the computer's user and launches the app under REA's capture |
| `mark_build_checkpoint` | Takes a named checkpoint and returns the capture's counts and the files that changed |
| `stop_build_capture` | Stops the capture and returns its checkpoints and the task draft between `seed` and `end` |
| `export_build_environment` | Exports the bundle with the agent's task inside the computer and copies it to a folder here |
| `check_build_task` | Grades a task against the app's live state; with a `stage`, files the check in the witness |
Like the computer tools, these are MCP-only and act on a `computer_id` from
`start_build_computer`.
REA's skill walks an agent from a request such as "build me an Inkscape
environment where the agent changes a poster's headline" to a witnessed
bundle with these tools: the questions it asks first, when it stops for the
person, the install, the seed, the task, the witness and its final report.
See [desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md).
## Build
1. `start_build_computer` from a request. REA is installed in the computer.
2. `start_build_capture`. REA first checks the installed runtime's export
schema for synthetic-seed support. If the archive is too old, repack a
current REA, update `REA_BUILD_COMPUTER_REA_ARCHIVE`, restart the REA MCP
server and start a new build computer. REA then writes an apt configuration under
`~/.rea-build/apt` and runs `apt-get update` as the computer's user, so the
capture finds the package's sources (`package_lists` counts the lists,
and `package_lists_error` gives apt's exit status and last lines when the
download failed; a later start runs the update again until one succeeds). It then starts `rea capture-desktop-application
--desktop-entry <entry>`
in the computer, leaving `~/.rea-build` out of every checkpoint, and returns
once the app's window is open, with a screenshot. For an app installed
from a `.deb` file, the build layer keeps that file at
`/opt/rea/package.deb` and the capture records it with `--package-file`. Do not open the app from
the menu first: the capture refuses an app whose window is already open.
When the capture ends before the window opens, REA reads its output. An app
that exits at launch, such as one missing a runtime, is reported with its
program, exit code, run time and the last lines of its stderr, at most 8
lines and 800 characters of the last 8 KiB the capture keeps, and
`details.diagnostics` keeps the capture's record of the session. A capture
that failed, such as one refusing to start, gives its own message and
remediation, with the whole failure in `details.diagnostics.command_failure`
and, when its cleanup is incomplete, the run tokens and paths it left in
`details.resources`. A checkpoint or stop that finds the capture ended
reads its output the same way.
`state_paths` names files or folders in the computer's home that hold the
app's state, relative to the home, after `~/` or absolute under it, at
most 16; each becomes a `--state-path` of the capture, so it is retained
from the start and its files count as the app's even before they exist.
REA counts a file as a document only when an app window's title shows its
name or `~/` path, so name the file the app will save when its title shows
something else, as MuseScore's shows a score's title: a file outside
hidden folders, other than a lock such as LibreOffice's `.~lock.<name>#`,
becomes a document, which the bundle owns, seeds and opens
at launch, and one under a hidden folder such as `~/.config/MuseScore` the
app's own state, also when that folder is a link elsewhere in the home. A
path with `..`, or naming the home itself, is refused before the computer
is asked; the computer refuses an absolute path outside its home, one in
`~/.rea-build`, and one a link leads out of the home or into
`~/.rea-build`, resolving a path that does not exist yet through its
nearest existing folder, and starts nothing.
3. Make the seed through the app with `act_on_build_computer`, saving it
where the user keeps such files, such as `~/Documents`, then
`mark_build_checkpoint` `seed`.
4. Make the change the task will ask for, save it, and
`mark_build_checkpoint` `end`.
5. `stop_build_capture`. The app is closed and the capture finishes. The
result lists the checkpoints with their counts, the package and the
sources the capture found, its limitations and the draft: each file under
the paths the export owns that changed between `seed` and `end`, with its
net change and, for XML files, archives, JSON and key files, its `values`,
the values the change added, changed and removed with what each was and
became, at the pointer a check names it by; suggested checks with a
placeholder prompt; and `guards`, optional checks that the rest kept its
seed values. The draft's
[task section](desktop-application-tasks.md#describe-the-change) describes
each. `notes`, each with a `code` and a `message`, say why a draft has no
files: `no_change` when no file under the home changed between them and
nor did the app's dconf keys; `unowned_changes` when files changed but
none under the owned paths, listing up to five in `files` with their
roles, desktop files last, and counting the rest in `more`;
`uncheckable_changes` when the app's keys in dconf were read and changed,
listing the database in `files`, since no check reads dconf (a version
the capture kept no copy of, as it keeps none of the database at the
start, counts as holding none of the app's keys);
`unreadable_changes` when a dconf database changed but REA could not read
it, such as without `dconf-cli`, with the read `error`, so whose keys
changed is unknown; and `no_document` when no file has the document role.
`unowned_changes` and `no_document` point at `state_paths`. They also say,
for a drafted `file`, why its suggestions are thin: `by_position`,
`view_state_only`, `long_text`, `cut_view` or `unreadable_view`.
6. Write the task: its name, the prompt in the user's own words, and checks
that hold after the change and not on the seed, from the suggestions,
`values` and any `guards` worth keeping.
7. `export_build_environment` with the task and a new or empty `directory`.
The export runs inside the computer with `--seed-checkpoint seed
--end-checkpoint end`, so it seeds what was saved at `seed`, scrubs the
computer's host name and user from it, and refuses a task whose checks
all pass on the seed. It also passes `--synthetic-seed`, since the agent
made the seed for the task, so the README does not call it the user's real
data. When the seed holds exactly one document and the app's desktop entry
takes files, the bundle's app opens that document at launch. A file moved
to the desktop Trash is not seeded, and the export's limitations name what
the Trash held at `seed`. The limitations and the README also name the
folders where files changed during the capture that the bundle neither
owns nor seeds, such as settings that keep a first-run wizard away, up to
eight collapsed to their top folder, such as `~/.config/MuseScore`, and a
count of the rest; capture again with them in `state_paths` to own them.
Lock files the app holds while it runs, such as LibreOffice's profile
`.lock` and its document lock `.~lock.<name>#`, are left out of the seed,
since a copied one can keep the app from starting, and the limitations
and the README list them after "Left out of the seed"; a project's
`Cargo.lock` or `yarn.lock`, a document or a path in `state_paths` is
seeded. They name seeded
WebKit or Chromium storage and caches too, which the app keeps for its
own running. When the application held files open in a folder whose
starting content the capture did not retain, a limitation names the
folder and the seed files in it. For a
`.deb` app that no apt source the capture found offers, the export takes
the request's URL with `--package-url`; one apt offers installs from apt. REA lists the bundle's files with their sizes
and SHA-256 in the computer, into a file beside the bundle that it reads
through `data/get`, since a large bundle's list outgrows a command's
output; it then brings the files back through the driver's
`data/get` in answers of at most 8 MiB, and checks and writes each answer
into the folder before asking for the next, so one answer at a time is
held; a failure leaves no partial folder.
8. `close_build_computer`.
A checkpoint name is lowercase letters, digits, dot, dash and underscore; the
capture's own `baseline`, `stop`, `time-limit` and `exit` are refused, and so
is a name already taken. The acknowledgement comes from the capture after it
has walked the home and retained what changed: `entries` walked, `changes`
since the checkpoint before, `retained_files` and `retained_bytes`, and up to
ten `changed_files`, home-relative, with `more_changed_files` counting the
rest. Folders are left out; files under `state_paths` come first, then
those the app held open, then those in its state directories, then the rest,
and caches and WebKit or Chromium storage last, each by path, so a save's
document leads the list. When the computer's REA is older than these lists,
as when `REA_BUILD_COMPUTER_REA_ARCHIVE` still names an earlier build, the
checkpoint is still taken and both are `null`: which files changed is
unknown. Pack the current REA into the archive to get them back.
Each step can be asked again after its answer was lost:
- A start whose answer is lost, or whose window never opened, stops the
capture it may have left running. Until the computer confirms that stop,
REA keeps the capture and the next `start_build_capture` retries the stop
before starting another.
- A checkpoint the capture took whose acknowledgement was lost comes back
from the capture's own log when the same name is marked again.
- A stop whose summary or draft was not read finishes when stop is asked
again, and a finished stop repeats its result. The draft and the export
trust the checkpoints the capture's own Evidence records.
- A step that saved something, such as an export's bundle or the witness's
record, returns it even when its call is cancelled or the computer closes
as the save finishes. `start_build_capture` saves nothing, so a close or
cancellation during its screenshot of the opened app cancels it.
The capture, the draft and the export run as the computer's user in its home,
without any `AGENTENV_*` variable, so neither REA nor the app holds the
computer's control token. REA sends each command through agentenv's
`computer-exec/v1` route with that token, which stays in REA, and reads larger
outputs, such as the draft, the export's Evidence and the bundle's file
list, from files under `~/.rea-build` through `data/get`, since a command's
output is capped at 1 MiB. In a command's Evidence or failure read this way,
as in its own output, REA replaces the token in every string, after checking
the Evidence's digest; the bundle's files arrive as they are.
Calls on one computer run one at a time, in the order REA receives them:
an action, a capture step, a check or a close waits until the call before
it on that computer has answered, and sends progress notifications while
it waits. So a check reads the app with no action half done, and a witness
stage is admitted, read and filed before anything else on that computer
runs. Calls on different computers run side by side. Closing a computer
cancels the call in progress on it and refuses the calls still waiting.
## Witness
Start a fresh computer from the exported folder with `start_build_computer`
and `{"kind": "bundle", "path": ...}`; agentenv boots the app with the seed,
on the seed's document when the export opens one. When the computer never
gets ready and the bundle's seed holds lock files, the start's error names
them: delete them from the bundle, or export again, which leaves them out.
A bundle records no state directories, so the error names only files by a
profile lock's name and LibreOffice's document locks.
Then, with the task exactly as exported:
| `stage` | Before it | Its checks must |
| ------------- | ----------------------------------------------------------------- | ---------------------------- |
| `seed` | nothing; give `directory`, a new or empty folder for the record | fail at least once |
| `after_edit` | the task done through the app from its prompt alone | all pass |
| `near_miss` | a change that is almost right, such as a misspelled line | fail at least once |
| `after_reset` | `act_on_build_computer` with `action: "reset"` as the last change | each come out as on the seed |
`check_build_task` reads the app's `data/get`, takes the views it returns
beside the files and evaluates each check with REA's evaluator, the one the
export validates tasks with and the exported verifier mirrors, one result per
check. Like the verifier, it grades only once the app's files settle, since an
app such as Gnote saves a few seconds after the last key. It reads `data/get`
every 0.5 s and grades once every file's bytes have held for 3 s, counted
from the read that returned them. 15 s after the first read began, it grades
the last completed read: no read starts then, and one still running is cut.
It sends progress notifications while it waits. The result's `settled` says
whether the files settled, and `read_cut` whether the bound cut a read; each
stage in the record keeps both. Every check therefore costs at least 3 s,
and a save that lands more than 3 s after the first read returns is still
missed; see
[grading once the app has saved](desktop-application-export.md#grading-once-the-app-has-saved).
Without `stage` it only grades. The `seed` stage begins a recording
of the display under REA's own request id, one per computer; a seed check
that fails after it, such as on a failed read of the app's state or a lost
reply to the recording's start, can be asked again, and asks for the
recording under the same id, which the computer answers with the recording
it already began. The stages come in order. The edit and the near miss each
need an applied click, drag, scroll, type or key since the stage before, with
no reset after the last of them, since a reset puts the seed back rather than
changing the app; `after_reset` needs the reset as the last change, and a
reset that leaves any check unlike on the seed fails the witness, its
stage's `remediation` naming those checks. On a computer started from a
bundle folder, the seed refuses a task whose `tasks/<name>.json` or verifier
in the bundle differs from what the export writes for it. A reset
relaunches the app, and REA waits until the computer is ready again before
answering, sending progress notifications while it waits.
Only a check that fails on what the app's view shows counts as a failure. A
check that fails because its file or table could not be read, such as an
SQLite file locked past the reader's wait, or because what it looks for may
lie in what its view cut, is `undecided` in its result: it shows nothing
about the task. That covers a line `lines_present` misses in a text view cut
at 5,000 lines, a row `row_present` misses in a table whose view keeps only
its first rows, a `value_equals` or text check pointer that names nothing
in a truncated view or past a cut zip listing, the absence checks and counts
on a cut view, and `text_equals` or `text_contains` of an element on one,
even where the element's kept text holds the value, since the cut may have
dropped text from between its runs. Any other check that passes on what the
view kept stays decided, and so does a value found there that differs from
the one expected.
The result's `passed` and description are the exported verifier's, which has
no undecided results. A stage that only undecided checks would settle, such
as a near miss whose one failing check could not read its file, files nothing
and is refused; check it again once the files read whole.
A stage whose checks do not show what it must ends the witness, and so does
`after_reset`. REA then ends the recording, fetches its segments through
`data/get`, each checked against the recording's manifest, and writes into
`directory`:
- `recording/NNNN.mp4`, the recording's segments;
- `witness.json`, REA Evidence (`rea.desktop-build-witness`) holding the
verdict, the stage that failed if one did, the task's name and SHA-256,
each stage's results with how many logged actions came before it and
whether the app's files had settled (`settled`), the
computer's [action log](desktop-build-computer.md#act) as it keeps it, each
action with its arguments, timing and outcome, with `actions_dropped`
counting the earliest actions it no longer keeps, and the recording's
segments with their sizes and SHA-256. The record's limitations say when
the log dropped actions or cut arguments, which each entry names in
`truncated_arguments`, and which stages graded files that had not settled.
The stages' rules read the log's count of actions and the last applied
action of each kind, which it keeps for the whole session, so they hold
however many entries it dropped. The result returns the same record. When writing it fails, such as for a
folder something else wrote into, the witness keeps its stages: check the
stage that ended it again, with the same task, and REA writes the record
without checking again. Once it is saved, the same check returns the saved
record again, so a record saved as its call was cancelled is not lost. A
computer is witnessed once; witness again from a fresh computer.
See the [specification](specs/desktop-app-environments/build-witness/spec.md)
for scope and verification.
-82
View File
@@ -1,82 +0,0 @@
# Documentation ownership and authoring
Use the [documentation directory](../README.md) to find existing material before
adding another guide. Preserve one canonical owner for each contract or workflow;
link to that owner instead of copying its claims into several guides.
## Choose a destination
| Document | Destination |
| -------------------------------------------------------- | -------------------------------------- |
| Installation, client setup and first CLI/MCP steps | `docs/getting-started/` |
| Analyst workflows and evidence interpretation | `docs/guides/<domain>/` |
| Analysis-engine configuration and format prerequisites | `docs/providers/` |
| Shared runtime contracts and evidence semantics | `docs/reference/` |
| Development, testing, release and authoring instructions | `docs/development/` |
| Composition diagrams | `docs/architecture/` |
| Compatibility changes for a released version | `docs/migrations/` |
| One feature specification, plan and tasks | `docs/specs/<domain>/<feature-slug>/` |
| One architectural decision | `docs/adr/<domain>/<decision-slug>.md` |
Use meaningful, lowercase domain and feature slugs. Global numeric prefixes are
unnecessary: different domains can add features independently, and branches do
not compete for the next sequence number. Check the existing domain directory
and linked issues before creating a feature with the same slug.
Guides describe existing behavior and limitations. Specs define one proposed
feature or domain change; an existing guide does not become a specification by
moving it into `specs/`.
## Feature lifecycle
Create the specification before implementation. Each feature directory contains:
- `spec.md`: problem, desired behavior, requirements, acceptance and non-goals.
- `plan.md`: the single implementation plan, contracts and architectural choices.
- `tasks.md`: incremental, testable steps linked to requirements and verification.
Split independent domains or separately reviewable features before writing their
specs. A broad `ROADMAP.md`, when useful, only lists child specifications, order
and dependencies; requirements and implementation tasks belong to each child.
Cross-repository contracts have one producer-owned specification. Consumers link
to that contract instead of maintaining competing copies.
Record an isolated architectural decision separately in
`docs/adr/<domain>/<decision-slug>.md`, with **Status**, **Context**,
**Decision** and **Consequences**. Link the ADR from affected specifications.
Do not create decision records for routine file moves or invent past decisions.
Specs do not replace the contribution-scope decision required for a new provider
or expanded execution authority. Follow
[contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
## Links and public routes
Use relative links to physical Markdown files so GitHub readers can follow them.
Link repository files outside `docs/` with their full GitHub URLs. Preserve
existing heading anchors when moving a guide.
[`documentation.ts`](https://github.com/morluto/rea/blob/main/docs/.vitepress/documentation.ts)
owns categorized site navigation and the existing guides' stable public routes.
Update it when adding or moving a guide. VitePress rewrites categorized sources
to the existing public paths; its Markdown hook resolves repository links from
the physical source and renders them against those public routes. Keep basenames
and route assignments unique. Edit links use the physical source file.
For a move, search for both the basename and its old full path. Update README
translations, authored skills, verification instructions, documentation-fact
readers and release configuration when they consume that path. Preserve
historical release records and references bound to earlier commits or tags.
## Generated ownership and checks
The [documentation directory](../README.md#machine-owned-documentation) identifies
the machine-owned outputs and their producers. Their existing paths are retained
because scripts, package consumers and release history depend on them. Change
canonical sources and regenerate; do not fix a generated projection by hand.
Run `npm run docs:check` for documentation facts and generated consistency, and
`npm run verify:docs:routes` to build the complete production site and check local
links, anchors and physical edit targets.
When a physical path moves, also run the tests that read it. Follow
[development checks](testing.md#developer-commands) before requesting review.
File diff suppressed because it is too large. Load diff
-105
View File
@@ -1,105 +0,0 @@
# Flattened devicetree inspection
`inspect_devicetree` / `rea inspect-devicetree` reads a local DTB without an
engine, target execution or external tools. It supports FDT versions 16 and 17;
other versions return `unsupported_target`. Corrupt recognized blobs return
`invalid_request`, and exhausted budgets return `resource_constraint` without
partial success.
```bash
rea inspect-devicetree /artifacts/board.dtb --json
rea inspect-devicetree /artifacts/board.dtb --detail summary --evidence_output /artifacts/board-evidence.json --json
```
The complete Evidence includes the artifact SHA-256, validated header fields,
64-bit memory reservations as exact hexadecimal strings, and a flat node tree
with explicit parent paths. Each property retains its file offset, byte length,
and up to 256 raw bytes as hex. `truncated` qualifies the preview only; parsing
coverage includes the entire declared tree. Version 16 has no declared structure
size, so `header.size_dt_struct` is null.
Derived fields carry `confidence: derived`: standard ASCII string lists, scalar
phandles and cell counts, and `reg` tuples using the parent's `#address-cells`
and `#size-cells` (defaults 2 and 1). Zero-cell components are null (omitted), distinct from an encoded zero. Cell
counts above four, invalid counts and incomplete tuples remain undecoded with a
limitation. Addresses are bus-local; no `ranges` translation is attempted.
Invalid or ambiguous strings remain raw, rather than being guessed.
The summary is labeled derived and includes root compatible/model, chosen
bootargs/stdout-path and aliases. An undecodable alias retains its name with a
null path; it is not silently treated as absent. Single-cell `interrupt-parent`, `remote-endpoint` and `msi-parent`
references resolve through phandle declarations. Missing or duplicate targets
have null target paths. A node whose `phandle` and `linux,phandle` disagree
adds a limitation, and references to either value stay unresolved. A present
`phandle` that is not one cell also adds a limitation; `linux,phandle` cannot
resolve that node when the canonical declaration is malformed. Binding-specific
reference arrays and overlays are not resolved; an empty reference list does not
prove absence of hardware links.
MCP `detail: summary` retains complete Evidence in the session. Follow its
`parent_evidence_id` with `inspect_analysis_view`:
```json
{
"source": { "kind": "retained-evidence", "evidence_id": "ev:<digest>" },
"view": {
"kind": "devicetree",
"collection": "properties",
"path": "/serial@1000",
"offset": 0,
"limit": 32
}
}
```
Collections are `nodes` (identity and property counts), `properties` (requires an
exact node path), `reservations`, and `references`. Pages use retained parsed
facts and do not reopen the file. Each page carries fixed-size context
(artifact, header, coverage and counts) rather than the summary strings, so a
large `bootargs` value does not repeat on every page. Source indices are scoped to the selected
collection and parent Evidence; property indices are scoped to their node.
| Input budget | Default | Ceiling |
| ---------------- | ------: | ------: |
| `max_file_bytes` | 4 MiB | 16 MiB |
| `max_nodes` | 8,192 | 65,536 |
| `max_properties` | 32,768 | 262,144 |
| `max_depth` | 64 | 256 |
Decoded names, paths, values and row overhead share a fixed 32 MiB representation
allowance, charged before expansion. It has no runtime override.
The reservation count also uses `max_properties`. CLI flags use kebab case,
for example `--max-nodes 16384`. View pages contain at most 256 rows.
## Verification
Synthetic tests cover v16/v17, malformed headers/blocks/names/tokens, resource
limits, derived values, and CLI/MCP delivery including retained pages after
removing the input. The optional real lane requires **dtc 1.8.1**:
```bash
npm run build:cached
node scripts/verify/firmware/devicetree.mjs
```
It compiles a DTS with reserved-memory, chosen, aliases and phandles, compares
REA's node paths against `dtc -I dtb -O dts`, and checks the derived report.
`REA_DTC_PATH` selects the verification executable; it is not runtime setup.
The dtc 1.8.1 lane ran on macOS arm64 with Node 26.5.0. Its seven node paths
matched the decompiled tree. The report included `model: "REA DTB verification"`,
`bootargs: "console=ttyS0"`, alias `serial0 -> /serial@1000`, reservation
`0x80000000 + 0x1000`, and a resolved reference from `/consumer` to
`/serial@1000`. The deliberately minimal remote-endpoint fixture produces dtc
graph-binding warnings; this lane verifies FDT structure and selected property
derivations, not hardware-binding validity.
A repeat run of that complete compile/decompile/CLI verification took 2.13 s
elapsed, 1.52 s user CPU and 0.27 s system CPU under `/usr/bin/time -l`. This is a
small fixture with warm filesystem caches, not a cold-start or large-board
benchmark. Peak process-family memory and whole-workflow token usage were not
measured; the sandbox denied the memory-reporting system query. No pre-feature
equivalent REA workflow exists for comparison. The pinned Node 24.22.1/npm
11.16.0 toolchain was unavailable; dependency installation used the local cache
with Node 26.5.0/npm 12.2.0 instead.
Loaded 100 of 5675 files, more files were not shown because too many files have changed in this diff. Show more