mirror of
https://github.com/morluto/rea.git
synced 2026-11-08 08:11:31 -10:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 morluto
|
800a3f24b7 |
No files matched your search
@@ -10,8 +10,8 @@ equivalent completed investigations. A useful outcome can also be a measured
|
||||
decision to leave an implementation alone.
|
||||
|
||||
Follow the checkout's [performance priorities](../../../AGENTS.md#performance),
|
||||
[tool design](../../../docs/development/tool-design.md), and
|
||||
[testing guide](../../../docs/development/testing.md). This skill supplies an optimization
|
||||
[tool design](../../../docs/tool-design.md), and
|
||||
[testing guide](../../../docs/testing.md). This skill supplies an optimization
|
||||
workflow, not additional permission to execute targets, publish, or merge.
|
||||
Use the current contracts, provider prerequisites, and user authorization.
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ description: Prepare or rewrite REA release changelogs and GitHub release notes
|
||||
# REA changelog update
|
||||
|
||||
Use this for release-note preparation and contributor attribution. Read
|
||||
[docs/development/releasing.md](../../../docs/development/releasing.md) from the repository root for
|
||||
[docs/releasing.md](../../../docs/releasing.md) from the repository root for
|
||||
publication authority and the existing checkpoint checks. This skill prepares
|
||||
notes; it does not authorize merging, tagging, or publishing.
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ Establish which checks still catch real regressions and which costs can be
|
||||
removed without losing that protection. The outcome may be a verified fix,
|
||||
an evidence-backed recommendation, or an explicit decision to stop.
|
||||
|
||||
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/development/testing.md),
|
||||
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/testing.md),
|
||||
and [contribution rules](../../../CONTRIBUTING.md). Use their current commands
|
||||
and pinned toolchain, rather than commands remembered from an earlier audit.
|
||||
Preserve the requested scope: investigation alone does not authorize edits;
|
||||
|
||||
@@ -1,36 +1,20 @@
|
||||
---
|
||||
name: rea-tool-design
|
||||
description: Audit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
|
||||
description: Design or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
|
||||
---
|
||||
|
||||
# REA tool design
|
||||
|
||||
Use the [tool-design guide](../../../docs/development/tool-design.md) for tool boundaries,
|
||||
Use the [tool-design guide](../../../docs/tool-design.md) for tool boundaries,
|
||||
contract semantics, provider ownership, and discoverability. Inspect the current
|
||||
contract and nearest existing tool relevant to the analyst question; source,
|
||||
documentation, and live catalogs may describe different versions.
|
||||
|
||||
For provider-backed work, exercise the analyst's task through REA and compare
|
||||
relevant native provider and independent artifact observations where available.
|
||||
Check applicable partial results, recovery and session boundaries; report what
|
||||
was not exercised. For failure guidance, follow [design recovery around the
|
||||
failure](../../../docs/development/tool-design.md#design-recovery-around-the-failure).
|
||||
Use [workflow verification](../../../docs/development/testing.md#provider-backed-workflow-verification)
|
||||
to select evidence for the proposed contract instead of seeking provider API
|
||||
parity.
|
||||
|
||||
For an audit, follow [audit existing tools](../../../docs/development/tool-design.md#audit-existing-tools):
|
||||
scope representative tasks, exercise the live workflow, and distinguish
|
||||
reproduced bugs, supported limitations, design opportunities and unverified
|
||||
claims. Check relevant rendered MCP prompts, guided workflows and packaged skill
|
||||
advice against the live contracts and exercise their suggested handoffs. Report
|
||||
working alternatives and evidence limits as well as findings.
|
||||
|
||||
For a design request, return the proposed boundary, input/result shape, Evidence
|
||||
semantics, nearest alternative, and verification approach. Implement when
|
||||
requested, preserving the user's scope and existing authorization.
|
||||
|
||||
For implementation or verification, consult [testing](../../../docs/development/testing.md)
|
||||
For implementation or verification, consult [testing](../../../docs/testing.md)
|
||||
for the affected consumer and provider lanes, and
|
||||
[contributing](../../../CONTRIBUTING.md) for generated-file ownership. Read
|
||||
provider/workflow guides only as the task requires.
|
||||
@@ -2,7 +2,7 @@
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "61"
|
||||
version: "53"
|
||||
---
|
||||
|
||||
# REA
|
||||
@@ -94,9 +94,6 @@ explicit path or endpoint and do not need it.
|
||||
than extra calls and schemas. Runtime options are metadata, not files.
|
||||
Compact delivery still inventories and hashes members. See the
|
||||
[Python packaged applications guide](https://github.com/morluto/rea/blob/main/docs/python-packaged-applications.md).
|
||||
- Bun `--compile` executable: `open_binary`, then `inspect_artifact`; the Bun
|
||||
module graph is the application layer. Extract stored modules and analyze the
|
||||
JavaScript. See [Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
|
||||
`open_bundle_resources`, then the selected resource tool. This does not
|
||||
require `CFBundleExecutable` or a deep provider. Resource sessions have no
|
||||
@@ -108,9 +105,7 @@ explicit path or endpoint and do not need it.
|
||||
- Firmware image: `inspect_firmware_regions` when advertised. Use
|
||||
`extract_firmware` when extraction is requested, with a caller-selected new
|
||||
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
|
||||
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md).
|
||||
- ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact`
|
||||
without a provider; read [ESP images](references/firmware-and-raw-images.md#esp-images).
|
||||
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
|
||||
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
|
||||
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
|
||||
optional metadata recovery and its supported host/layout boundary.
|
||||
@@ -121,15 +116,6 @@ explicit path or endpoint and do not need it.
|
||||
cleanup effects, and do not require a preliminary static session unless their
|
||||
own contract says so.
|
||||
- User-owned browser page already open: `list_browser_targets`.
|
||||
- Passive traffic capture of a SPA: the default scope is the page's own origin,
|
||||
and data APIs usually live on another origin. Discover API origins from
|
||||
absolute URLs in loaded bundles or script text via JavaScript analysis tools
|
||||
(another passive capture without those origins still scopes to the page
|
||||
origin, and `completeness.excluded` counts lack URLs). Pass both the page
|
||||
origin and those API origins in `allowed_origins` — the list replaces the
|
||||
default scope — and read `completeness.excluded` before reporting that an
|
||||
API is absent. For long captures with body shapes, use
|
||||
`export_evidence_bundle` instead of expecting an inline result.
|
||||
- Browser action the agent must perform (click, type, navigate):
|
||||
`capture_browser_scenario` with declared actions. See
|
||||
[runtime observation](references/runtime-observation.md) for connect/launch
|
||||
@@ -137,9 +123,6 @@ explicit path or endpoint and do not need it.
|
||||
- Agent environment (AgentEnv bundle) of a web application: read
|
||||
[environments](references/environments.md) before recording; it records with
|
||||
`capture_browser_scenario` and writes the bundle with `export_environment_bundle`.
|
||||
- Task for a desktop application's exported environment, from a
|
||||
`capture-desktop-application` folder: read
|
||||
[desktop application tasks](references/environments.md#tasks-for-a-desktop-application-environment).
|
||||
- Fiddler Everywhere traffic or a native `.saz` capture: read the
|
||||
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
|
||||
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
|
||||
@@ -150,7 +133,7 @@ explicit path or endpoint and do not need it.
|
||||
old-generation heap is `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` (default 192 MiB,
|
||||
floor 8, maximum 16384), set where REA starts. Host memory and `NODE_OPTIONS` do not
|
||||
raise it. See
|
||||
[historical captures](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md) for the exact upstream
|
||||
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
|
||||
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
|
||||
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
|
||||
or establish network loss. Prefer detail summary, then kind qlog trace/stream
|
||||
@@ -162,10 +145,8 @@ explicit path or endpoint and do not need it.
|
||||
and field names/types before choosing fields for `inspect_packet_capture`.
|
||||
Query/page the relevant records; registrations do not establish observed traffic.
|
||||
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and
|
||||
TShark 4.x on an absolute `PATH` entry or via the optional absolute
|
||||
`REA_TSHARK_COMMAND` override; see
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/guides/data/packet-captures.md).
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
|
||||
Packet limits count input packets before filtering. Output defaults to 50 matching
|
||||
frames; use `match_offset` / `match_limit` for later pages, or explicit null
|
||||
limit for complete matching output. Preserve the source digest across pages.
|
||||
@@ -173,7 +154,7 @@ explicit path or endpoint and do not need it.
|
||||
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
|
||||
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
|
||||
and capinfos, with Linux prlimit or Windows Job Objects; see
|
||||
[packet summaries](https://github.com/morluto/rea/blob/main/docs/guides/data/pcap-analysis.md).
|
||||
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
|
||||
Select an exact DNS question for address pages; page exhaustion does not prove
|
||||
network completeness. This reads recordings and does not acquire traffic.
|
||||
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
|
||||
@@ -181,7 +162,7 @@ explicit path or endpoint and do not need it.
|
||||
relevant details before choosing settings; this does not scan hardware.
|
||||
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
|
||||
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
|
||||
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/guides/signals/signal-recordings.md).
|
||||
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
|
||||
Output defaults to 50 literal lines; select annotation classes and page with
|
||||
`line_offset` / `line_limit`, or explicit null for all remaining output.
|
||||
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
|
||||
@@ -190,7 +171,7 @@ explicit path or endpoint and do not need it.
|
||||
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
|
||||
Select JavaScript glue and candidate paths explicitly and preserve URL/path
|
||||
ambiguity. No implicit fetch or target execution; WAT is not original source.
|
||||
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/guides/wasm/wasm-artifacts.md).
|
||||
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/wasm-artifacts.md).
|
||||
- Explicit local Android APK (or ZIP/AAB) signing: `inspect_apk_signature`.
|
||||
Prefer its summary or selected signer/digest/lineage page; retain complete
|
||||
Evidence and use `inspect_analysis_view` kind `apk-signature` for follow-ups.
|
||||
@@ -209,13 +190,13 @@ explicit path or endpoint and do not need it.
|
||||
Evidence stays retained under its original identity. Use the returned
|
||||
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
|
||||
Candidate coverage does not establish interface completeness. See the
|
||||
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/guides/evm/evm-bytecode.md).
|
||||
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
|
||||
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
|
||||
with the explicit local file. This built-in target-free inspection requires no
|
||||
Go installation or disassembler. Preserve raw module bytes and source ranges;
|
||||
absent metadata does not prove a binary is not Go, and unparsed text records
|
||||
remain explicit. See the
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/go-binaries.md).
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
|
||||
- Embedded CUDA/HIP code objects: `inspect_gpu_code_objects`; inventory targets
|
||||
first, then decode selected payloads and query retained GPU Evidence.
|
||||
- Bundled GGUF or safetensors model declarations: `inspect_model_artifact`
|
||||
@@ -276,15 +257,6 @@ explicit path or endpoint and do not need it.
|
||||
linker declarations, not proof of enforcement. A missing directory is observed
|
||||
absence of that declaration. See the
|
||||
[PE load configuration guide](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
|
||||
- ELF, PE, and Mach-O hardening: `inspect_binary_hardening` with the explicit
|
||||
local file. This built-in target-free inspection requires no pwntools or other
|
||||
host tool. One image per Mach-O slice. Each property is `enabled`, `disabled`,
|
||||
`partial`, `unknown`, or `not_applicable`, with the header field or import
|
||||
behind it. A missing stack-protector or fortify import is `unknown`, not
|
||||
proof the protection was omitted. Authenticode and Mach-O code signatures
|
||||
report presence, not validity. Hardened runtime stays unknown and names
|
||||
`inspect_signature`. The image is not executed. See the
|
||||
[binary hardening guide](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
|
||||
- ELF note tables: `inspect_elf_notes` with the explicit local ELF32/ELF64
|
||||
file. This built-in target-free inspection requires no disassembler. Preserve
|
||||
PT_NOTE and SHT_NOTE ranges, owners, types, descriptor SHA-256, and GNU
|
||||
@@ -332,17 +304,17 @@ explicit path or endpoint and do not need it.
|
||||
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
|
||||
report truncation and have unspecified order. Use a quiescent snapshot rather
|
||||
than assuming a live database/WAL pair was copied atomically. See the
|
||||
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
|
||||
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
- Explicit Linux ELF file for offline layout, symbols, relocations or static
|
||||
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
|
||||
caller-supplied pwntools without opening a disassembler database. Preserve its
|
||||
raw locations and inference/coverage limits; see the
|
||||
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
- Supplied Linux x86-64 ELF core: `inspect_recorded_crash`. Read every recorded
|
||||
thread, raw note and source-bound register without opening a live target.
|
||||
Historical PIDs do not select live processes. Request `include_debugger_context`
|
||||
only when core-only mapping candidates help; see the
|
||||
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md).
|
||||
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md).
|
||||
- PE resource type/name/language trees and static icon group references:
|
||||
`inspect_pe_resources` with an explicit path. This portable operation requires
|
||||
no active target or engine; opaque payloads retain file ranges and SHA-256.
|
||||
@@ -358,9 +330,9 @@ The legacy attached profile binds the already-open GUI input by SHA-256 and
|
||||
leaves its database open. The modern headless profile creates and closes an
|
||||
owned private database without saving. Installation and registration reuse
|
||||
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
|
||||
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md).
|
||||
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
|
||||
For migration to the separate native IDA MCP plugin, follow the guide's
|
||||
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md#migrate-a-legacy-attached-registration).
|
||||
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/ida-provider.md#migrate-a-legacy-attached-registration).
|
||||
Replace the upstream registration explicitly; modern mrexodia tools are not the
|
||||
native profile. Native attachment preserves the database but does not provide
|
||||
direct callers; check session availability and the guide's verification limits.
|
||||
@@ -398,18 +370,13 @@ unless the group overrides them. The grouped name union is the complete tool
|
||||
inventory; use `tools/list` for canonical tool schemas. The default summary keeps
|
||||
each provider candidate's availability and target-support status, code and
|
||||
reason; `omitted_fields` lists what it leaves out. Request `{"detail": "complete"}`
|
||||
only when you need an unavailable candidate's remediation, provider
|
||||
diagnostics, candidate capability sets, or tool surface, effect and annotation
|
||||
flags; in those closed flag vocabularies, listed
|
||||
only when you need provider diagnostics, candidate capability sets, or tool
|
||||
surface, effect and annotation flags; in those closed flag vocabularies, listed
|
||||
flags are true and omitted known flags are false. Provider candidate facts
|
||||
describe possible providers, while top-level session capabilities describe the
|
||||
current route and `analysis_provider_binding` names the selected provider. Do
|
||||
not treat those scopes as interchangeable.
|
||||
|
||||
For editable Ghidra analysis that must survive a later agent process, select
|
||||
the [owned Program checkpoint workflow](references/native-and-artifacts.md#continue-editable-ghidra-analysis-across-processes)
|
||||
before making metadata edits. It is separate from Evidence/Unknown persistence.
|
||||
|
||||
For a Ghidra startup timeout on a large native target, follow
|
||||
[deadline recovery](references/native-and-artifacts.md#ghidra-startup-deadline-recovery)
|
||||
before abandoning deep analysis or repeating the same cold import. The default
|
||||
@@ -427,18 +394,9 @@ call `analyze_javascript_application` or `inspect_binary_layout` with
|
||||
a flag: its default reply is the summary, and the complete dossier stays
|
||||
retained under `parent_evidence_id` for facet paging. Pass that ID as a
|
||||
`retained-evidence` reference to `compare_functions`, or inside
|
||||
`match_native_functions` `left_observations`/`right_observations` or
|
||||
`build_call_path.functions`; request `"detail": "complete"` only when the
|
||||
dossier must travel inline.
|
||||
Use the same exact-reference form in other advertised logical Evidence inputs,
|
||||
including optional sources, arrays and nested sources across domains. Each
|
||||
source can independently be inline or retained; preserve its source identity,
|
||||
coverage, partial facts and unknowns. A summary or inspection page has its own
|
||||
Evidence ID: use `normalized_result.parent_evidence_id` when the consumer needs
|
||||
the complete source. For portable CLI follow-ups, commands advertising
|
||||
`--evidence-bundle` resolve references from that selected exported bundle.
|
||||
See [Evidence composition](references/evidence-workflows.md) for scope and
|
||||
recovery; file selectors are supported only where explicitly advertised.
|
||||
`match_native_functions` `left_observations`/`right_observations`; request
|
||||
`"detail": "complete"` only when the dossier must travel inline, as
|
||||
`build_call_path` requires.
|
||||
Page module identities and select the returned `node_id` before requesting an
|
||||
item; a single file path can represent several graph roles. Do not repeat an
|
||||
identical analysis or rejected call without a changed input or resolved blocker.
|
||||
@@ -457,10 +415,7 @@ output also streams without a single MCP frame.
|
||||
When the retained parent ID is unknown, use `get_evidence_bundle` with
|
||||
`detail: "summary"` and exact filters for operation, target SHA-256,
|
||||
analysis-profile digest or canonical procedure address. Select its returned
|
||||
Evidence ID for the next view instead of requesting the complete bundle. When
|
||||
that record has no supported view (for example an annotation), read it with
|
||||
`detail: "records"` and an `evidence_id` filter; `missing_evidence_links`
|
||||
names linked sources this session does not retain.
|
||||
Evidence ID for the next view instead of requesting the complete bundle.
|
||||
`retention: "complete-record"` means the whole record is stored; native
|
||||
value-flow truncation and unknown metadata remain separate. For an exported
|
||||
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
|
||||
@@ -523,14 +478,8 @@ as complete while required questions remain open.
|
||||
|
||||
## Read only the relevant guide
|
||||
|
||||
- Native binaries and managed assemblies:
|
||||
- Native binaries, managed assemblies, archives, and extraction:
|
||||
[references/native-and-artifacts.md](references/native-and-artifacts.md)
|
||||
- Firmware, headerless images, ESP images, and address-bearing containers:
|
||||
[references/firmware-and-raw-images.md](references/firmware-and-raw-images.md)
|
||||
- Archive inventory, extraction, Godot PCK, UnityFS, wxapkg, and Bun:
|
||||
[references/packages-and-archives.md](references/packages-and-archives.md)
|
||||
- Offline format headers, metadata, signatures, and database snapshots:
|
||||
[references/format-inspectors.md](references/format-inspectors.md)
|
||||
- ASARs, extracted JavaScript, feature tracing, and version comparison:
|
||||
[references/javascript-applications.md](references/javascript-applications.md)
|
||||
- Android APK declarations, classes, methods, and static references:
|
||||
@@ -543,16 +492,10 @@ as complete while required questions remain open.
|
||||
- Agent environments (AgentEnv bundles) of a web application, recorded from
|
||||
a live instance or a HAR:
|
||||
[references/environments.md](references/environments.md)
|
||||
- Tasks for a desktop application's exported environment:
|
||||
[references/environments.md](references/environments.md#tasks-for-a-desktop-application-environment)
|
||||
|
||||
## Finish the task
|
||||
|
||||
Explain findings in plain language and tie them to returned evidence. When the
|
||||
user asks to build something, use normal coding tools and separate observed
|
||||
behavior from design choices. Close an opened native session with
|
||||
`close_binary` when its engine is no longer needed. Evidence references and Unknown
|
||||
history remain in memory on the same MCP connection. End the investigation with
|
||||
`reset_investigation`; supply `save_path` to checkpoint the complete bundle before
|
||||
clearing. Import the saved bundle to restore original references and Unknown
|
||||
history in a new process. Retention is bounded and is not automatic persistence.
|
||||
`close_binary` when the investigation is complete.
|
||||
@@ -22,7 +22,7 @@ Preserve slot/relocation provenance and source coverage: matched means observed
|
||||
candidate presence; partial scans do not establish uniqueness or absence.
|
||||
This reads DEX/ELF bytes without loading the APK into the decompiler or executing
|
||||
target code. Runtime registration and loading remain unknown. See the
|
||||
[JNI guide](https://github.com/morluto/rea/blob/main/docs/guides/android/android-analysis.md#link-native-declarations-to-jni-candidates).
|
||||
[JNI guide](https://github.com/morluto/rea/blob/main/docs/android-analysis.md#link-native-declarations-to-jni-candidates).
|
||||
|
||||
For an R8/ProGuard release build with its `mapping.txt`, pass the absolute
|
||||
`mapping_path` to `inspect_android_class` or `inspect_android_method`. Prefer
|
||||
|
||||
@@ -11,7 +11,7 @@ main and an older release expose the same arguments. Tool-name prefixes belong
|
||||
to the client: find the canonical REA operation in its actual tool listing and
|
||||
server association before concluding that it is absent. A shared prefix alone
|
||||
does not establish which server an approval wildcard covers. See
|
||||
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/getting-started/client-configuration.md#client-tool-names-and-discovery).
|
||||
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/client-configuration.md#client-tool-names-and-discovery).
|
||||
|
||||
## Choose the remedy from the observed failure
|
||||
|
||||
|
||||
@@ -8,9 +8,7 @@ confirmed. The workflow records the application once with
|
||||
again if a hint names a gap, then writes the bundle with
|
||||
`export_environment_bundle`. For a HAR the user already has, skip to
|
||||
[export](#export-and-hand-off) through `inspect_web_network_capture`, or use
|
||||
`rea environment-build <capture.har> <dir>`. For a task in a desktop
|
||||
application's exported environment, read
|
||||
[its section](#tasks-for-a-desktop-application-environment).
|
||||
`rea environment-build <capture.har> <dir>`.
|
||||
|
||||
## Before recording
|
||||
|
||||
@@ -126,11 +124,9 @@ shape, with the application's own locators, paths and headers:
|
||||
}
|
||||
```
|
||||
|
||||
Give each step that accepts `timeout_ms` a deadline. Without one, a step waits
|
||||
until it completes or the scenario is cancelled, so a locator that never
|
||||
matches hangs the capture and the client with it. The `wait_for_timeout` pause
|
||||
instead uses `duration_ms` and does not accept `timeout_ms`. Match `wait_for`
|
||||
locators to exactly one element.
|
||||
Give every step a `timeout_ms`. A step without one waits until the scenario
|
||||
is cancelled, so a locator that never matches hangs the capture, and the
|
||||
client with it. Match `wait_for` locators to exactly one element.
|
||||
|
||||
1. Sign in through the UI: `fill` the user and password fields (the password
|
||||
as a secret reference), submit, then `wait_for` an element only a signed-in
|
||||
@@ -157,17 +153,13 @@ locators to exactly one element.
|
||||
with varied free text. After each write, and before the next one, read the
|
||||
object back by id and then its list: create 1, read 1, list, create 2,
|
||||
read 2, list; never several writes before their reads.
|
||||
4. Leave more than a second between two creates and between a create and its
|
||||
first edit (`wait_for_timeout` with `duration_ms: 1100`), so lists sorted
|
||||
by creation time show them in order.
|
||||
4. Leave more than a second between a create and its first edit
|
||||
(`wait_for_timeout` with `duration_ms: 1100`).
|
||||
5. Update both objects and read each back, changing the fields a person
|
||||
would change (a name, an email address, a status, a due date) rather than
|
||||
only a description: a served update accepts only the members a recorded
|
||||
update stored. Run an action that turns something on or off (star,
|
||||
bookmark, archive) twice in a row on one object, reading it back after
|
||||
each: the second call shows whether it sets or flips the field, and the
|
||||
environment serves the action only once a call has shown it. Delete both
|
||||
and read each back (expect 404, or the object marked deleted) and the list.
|
||||
update stored. Delete both and read each back (expect 404, or the object
|
||||
marked deleted) and the list.
|
||||
6. Vary what the two creates send, however you write them: the first every
|
||||
field the application takes, the second only what a person must give (its
|
||||
title or name), with the other fields left out or empty. A served create
|
||||
@@ -243,17 +235,17 @@ Re-record a withheld write at most twice, and only for a recording gap. Stop at
|
||||
once when the reason is application behaviour, and report the write as
|
||||
withheld with its hint:
|
||||
|
||||
| Reason | What to do |
|
||||
| --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
|
||||
| `no-observation` | Re-record: read the entity back by id after the write; read it first for an update or delete. |
|
||||
| `unconfirmed` | Re-record: read the entity and its list after the write. |
|
||||
| `incomplete-entities` | Re-record: read the entity through every view that lists it. |
|
||||
| `unexplained-change` | Re-record once with a second, different call of the same write. |
|
||||
| `no-call` | Re-record once with a write that succeeds. |
|
||||
| `side-effect` | Stop: the application changes other entities. |
|
||||
| `contradicted` | Stop: a later read disagrees with the modelled effect. |
|
||||
| `body-unusable` | Re-record a UI write once with `request` steps (its body was not exposed); else stop. |
|
||||
| Reason | What to do |
|
||||
| --------------------- | --------------------------------------------------------------------------------------------- |
|
||||
| `no-state-target` | Re-record: read the collection or item the write changes. |
|
||||
| `no-observation` | Re-record: read the entity back by id after the write; read it first for an update or delete. |
|
||||
| `unconfirmed` | Re-record: read the entity and its list after the write. |
|
||||
| `incomplete-entities` | Re-record: read the entity through every view that lists it. |
|
||||
| `unexplained-change` | Re-record once with a second, different call of the same write. |
|
||||
| `no-call` | Re-record once with a write that succeeds. |
|
||||
| `side-effect` | Stop: the application changes other entities. |
|
||||
| `contradicted` | Stop: a later read disagrees with the modelled effect. |
|
||||
| `body-unusable` | Re-record a UI write once with `request` steps (its body was not exposed); else stop. |
|
||||
|
||||
Use new scratch names on each attempt, and delete what an earlier attempt
|
||||
created before recording again (through the UI, or a `request` step with the
|
||||
@@ -287,90 +279,3 @@ See
|
||||
[agent environments](https://github.com/morluto/rea/blob/main/docs/agent-environments.md)
|
||||
for what the state, replay and the bundle establish, and
|
||||
[evidence workflows](evidence-workflows.md) for the manifest and state views.
|
||||
|
||||
## Tasks for a desktop application environment
|
||||
|
||||
Use this when the user has a folder from `rea capture-desktop-application` and
|
||||
wants a task for the environment `rea export-desktop-application` makes from
|
||||
it: a prompt for a computer-use agent and checks that grade what the agent
|
||||
left in the app's files. You write the prompt and the checks; REA validates
|
||||
the task against the capture and exports it. These commands are CLI only.
|
||||
|
||||
### Describe the change
|
||||
|
||||
Run `rea describe-desktop-application-task CAPTURE --format json`. Read:
|
||||
|
||||
- `owned_paths`: the only paths whose files the app serves, so the only files
|
||||
a check can read.
|
||||
- `files`: each file there whose state differs between the capture's first
|
||||
and last version, with its `role`, the `baseline` and `final` versions and
|
||||
the net `change`. Text changes list each added and removed line once, and
|
||||
`line_counts` gives lines found at both ends whose count changed.
|
||||
- `task`: a draft whose `checks` are suggestions from the net change of the
|
||||
documents the user worked on, or of the app's own state when no document
|
||||
changed.
|
||||
|
||||
### Write the prompt
|
||||
|
||||
Say what the user wants done, in their words and the app's terms: "Add buy
|
||||
milk to my to-do list and save it". Never say how the file stores it: no
|
||||
paths the user would not name, line formats, XML, keys or SQL, and never the
|
||||
checks. Replace the draft's placeholder; the export refuses it.
|
||||
|
||||
### Pick the checks
|
||||
|
||||
Checks must express only the observed change:
|
||||
|
||||
- Keep a suggestion only if it says what the user did. Drop those that follow
|
||||
from how the app saves: a format rewrite, renumbered lines, counters. A
|
||||
LibreCAD drawing gaining four holes was rewritten from DXF R12 to 2007, and
|
||||
the draft suggests counts of `SECTION`, `ENDSEC` and `CIRCLE`; only
|
||||
`CIRCLE` equals 9 is the user's change.
|
||||
- Never check unchanged content, whole files, or values the app stamps:
|
||||
times, ids it assigns, window sizes. REA refuses ISO 8601 and `asctime`
|
||||
date-times (`Thu Nov 5 09:20:31 2026`); the
|
||||
rest is your judgment.
|
||||
- Prefer a few robust checks: whole lines over positions, rows by meaningful
|
||||
columns over generated ids, `at_least` when the user may add more.
|
||||
|
||||
Kinds: `file_present` and `file_absent`; `lines_present` and `lines_absent`
|
||||
(`lines`); `line_count` (`exact` or `contains`, and `equals`, `at_least` or
|
||||
`at_most`); `value_equals` and `value_absent` (a JSON `pointer` into the view,
|
||||
such as `/groups/GROUP/KEY` for a key file); `text_equals` and
|
||||
`text_contains` (an XML element's whole text at a `pointer`, optional
|
||||
`normalize_whitespace`); `row_present` and `row_absent` (`table`, `where` by
|
||||
column) and `row_count`. Each names its `file` as a path under the home. A
|
||||
document saved as a zip archive, such as FreeCAD's `.FCStd`, reads as its
|
||||
entries: point into an entry's view, and name array members by selector
|
||||
rather than position, as
|
||||
`/entries/[name=Document.xml]/view/root/children/[name=Object][@name=Hole]`,
|
||||
so the check survives an app version that writes them in another order. Give
|
||||
`value_equals` a `tolerance` to compare a number the app writes as text, such
|
||||
as `"5.0000000000000000"`, with a number. Check what the user wrote in a
|
||||
document with `text_equals` on its paragraph element, not `value_equals` on a
|
||||
text node: apps such as AbiWord and Writer split a paragraph into runs as they
|
||||
save, so its first text node holds only part of it; use `text_contains` for a
|
||||
value within a sentence, such as a time. The
|
||||
[desktop application tasks](https://github.com/morluto/rea/blob/main/docs/desktop-application-tasks.md)
|
||||
guide has the exact rules.
|
||||
|
||||
### Export the task
|
||||
|
||||
Write the task as JSON with a `name`, the `prompt` and the `checks`, then run
|
||||
`rea export-desktop-application CAPTURE DIRECTORY --task TASK.json`. REA keeps
|
||||
the task only when its checks fail on the seed and all pass on the captured
|
||||
end state. A refusal names the check: one that fails on the end state does not
|
||||
match what the user left; "every check already passes on the seed" means no
|
||||
check captures the change.
|
||||
|
||||
When the export refuses an application that no apt repository installs, ask
|
||||
the user for the `.deb` file they installed it from and capture with
|
||||
`--package-file`, then ask where the computer may download that file and pass
|
||||
it as `--package-url`. Never fetch a vendor's download page yourself: such
|
||||
sites often refuse automated downloads, and the copy and its licence are the
|
||||
user's to choose.
|
||||
|
||||
Tell the user the task's name, prompt and checks, and how to measure a solve
|
||||
rate: run `agent-env run DIRECTORY --task NAME --agent claude-code` with
|
||||
`--model MODEL` three times and count the runs reported `passed`. Do not run
|
||||
agents unless asked. The bundle's seed is the user's real data: keep it local.
|
||||
@@ -11,44 +11,6 @@ returned inline. When complete `inspect_binary_layout` or
|
||||
`inspect_analysis_view` for a summary, one section/symbol/module, or a stable
|
||||
page instead of repeating the producer.
|
||||
|
||||
For any logical Evidence source that advertises retained references, pass
|
||||
`{kind: "retained-evidence", evidence_id}` instead of copying the complete
|
||||
record back into a call. The same contract covers individual, optional, array
|
||||
and nested sources across native, process, JavaScript, managed, mobile and WASM
|
||||
workflows. Sources can independently mix inline records and exact references.
|
||||
Resolution preserves the source's identity, provenance, coverage, partial facts
|
||||
and unknowns; it neither reruns analysis nor makes incomplete facts complete.
|
||||
Keep domain role and identity requirements: a native dossier cannot substitute
|
||||
for a managed member observation, for example.
|
||||
|
||||
When starting from a summary or inspection page, use its
|
||||
`normalized_result.parent_evidence_id` to reference the complete source.
|
||||
The view's own `evidence_id` names
|
||||
only that view. `compare_artifacts` explicitly supports artifact summary inputs
|
||||
and resolves their parents; do not assume every consumer does. When a workflow
|
||||
follows source links, keep those linked records available too.
|
||||
|
||||
MCP references belong to the current connection's ledger. If the ID is known,
|
||||
pass it directly; no discovery call or provider setup is needed. If it is missing,
|
||||
import the exported bundle or supply complete inline Evidence. Do not retry an
|
||||
unchanged reference or rerun a producer merely to move an existing record.
|
||||
Export before closing the session when the source must survive it. For CLI JSON
|
||||
workflows that advertise `--evidence-bundle`, the same reference JSON resolves
|
||||
against one explicitly selected authenticated bundle:
|
||||
|
||||
```bash
|
||||
rea trace-application-feature request.json --evidence-bundle /absolute/path/evidence.json --json
|
||||
```
|
||||
|
||||
This includes JavaScript traces/comparisons and runtime reconciliation, managed
|
||||
reconstruction/native verification/graph projection, Android/Apple/Harmony graph
|
||||
projection, WASM interface comparison and retained analysis views. The managed
|
||||
member comparison CLI still takes two assembly paths; process comparison takes
|
||||
two capture Evidence JSON paths. Check command help for the installed version.
|
||||
An input bundle grants no implicit file lookup: `evidence-file` and historical
|
||||
graph file selectors are accepted only by contracts advertising them. See the
|
||||
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/getting-started/cli.md#import-export-and-compare-evidence).
|
||||
|
||||
Completed `inspect_artifact` inventory also supports a summary and
|
||||
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
|
||||
Omit `logical_paths` to page all available occurrences; `[]` selects none.
|
||||
@@ -73,14 +35,6 @@ Parent Evidence ID plus source index preserves same-address
|
||||
alternatives. Follow `coverage.next_offset` only as far as the question needs;
|
||||
exhausting retained rows does not establish complete program discovery.
|
||||
|
||||
`batch_decompile` returns every body unless you select `detail: "summary"` when
|
||||
the connected tool advertises it. Use its parent Evidence ID with
|
||||
`view: {kind: "batch-decompile", collection: "items"}` for body-free rows and
|
||||
exact `where.indices`, `addresses`, `procedure_addresses` or `statuses`. Add
|
||||
`include_pseudocode: true` for several chosen bodies, or use
|
||||
`collection: "pseudocode"` with one row's `index` to page a large body. Batch
|
||||
limitations aggregate every item, so do not attribute them to one successful row.
|
||||
|
||||
Use `record_unknown` to track unresolved questions in the user's investigation
|
||||
and name the authority or environment still required. Supply supporting and
|
||||
contradicting evidence IDs. It updates the session registry directly.
|
||||
@@ -92,22 +46,18 @@ Use comparisons with complete, compatible page sets when claiming equivalence
|
||||
or absence:
|
||||
|
||||
- `compare_artifacts` compares inventory pages by occurrence path, content,
|
||||
metadata, and graph relations. Name each side as
|
||||
`{kind: "retained-evidence", evidence_id}` (an `inspect_artifact` result,
|
||||
its nested inventory or its summary) or
|
||||
`{kind: "evidence-file", path, evidence_id?}` instead of resending Evidence.
|
||||
metadata, and graph relations.
|
||||
- `compare_functions` compares explicit function Evidence, not fuzzy
|
||||
whole-binary matches. It rejects an `analyze_function` summary reply; pass
|
||||
the summary's `parent_evidence_id` as a `retained-evidence` reference or
|
||||
the complete dossier.
|
||||
- `compare_bundles` compares canonical bundle membership and unknown history.
|
||||
- `find_changed_behavior` combines existing comparisons; static differences
|
||||
remain candidates, not causal proof. Its `comparisons` accept exact references.
|
||||
remain candidates, not causal proof.
|
||||
- `build_call_path` needs explicit function dossiers from one artifact/provider;
|
||||
pass complete parent IDs in `functions`. An incomplete frontier makes absence
|
||||
unknown.
|
||||
an incomplete frontier makes absence unknown.
|
||||
- `correlate_static_and_runtime` uses explicit mappings; matching patterns are
|
||||
hypotheses, not causality. Both comparison arrays accept exact references.
|
||||
hypotheses, not causality.
|
||||
- `verify_reconstruction` evaluates a finite typed specification. A pass covers
|
||||
only declared comparable claims, not global equivalence.
|
||||
|
||||
|
||||
@@ -1,76 +0,0 @@
|
||||
# Firmware and raw images
|
||||
|
||||
## Firmware regions and extraction
|
||||
|
||||
Use `inspect_firmware_regions` when advertised. Use `extract_firmware` when
|
||||
extraction is requested, with a caller-selected new absolute output directory.
|
||||
These tools use caller-supplied Binwalk/Unblob on Linux; see the
|
||||
[firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md).
|
||||
|
||||
## Headerless raw images and firmware containers
|
||||
|
||||
For other headerless images (embedded firmware without ELF/Mach-O/PE/MZ
|
||||
headers), open with `format: "raw-binary"`, `architecture` (`x86`, `x86_64`,
|
||||
`arm`, `arm64`, `6502`, or `m68k`), and `entry_point` (`0x` hex). File offset 0 is mapped at
|
||||
that address. Hopper and IDA refuse this interpretation; Ghidra uses
|
||||
`BinaryLoader` with a generic language: big-endian for `m68k`, little-endian
|
||||
otherwise. Big-endian ARM, Thumb-only, and Cortex-M variants are unsupported.
|
||||
`6502` selects NMOS 6502; the entry and complete file mapping must fit within
|
||||
`0x0000..0xffff`. 65C02, 65816, cartridge containers, bank switching, and mapper
|
||||
semantics are unsupported. `m68k` requires an even entry and decodes the 68040
|
||||
superset of the 68000 family; Amiga/Atari containers and vector-table entry
|
||||
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/providers/raw-binary.md).
|
||||
|
||||
Intel HEX, S-record, UF2 and DfuSe files are containers, not raw images:
|
||||
`inspect_artifact` lists address regions (UF2 per familyID, DfuSe per element
|
||||
with its alternate setting), and `extract_artifact` writes each region. Open an
|
||||
extracted region as `raw-binary` with its start address as `entry_point`. UF2
|
||||
numbering gaps leave coverage unknown; a DfuSe suffix CRC mismatch follows
|
||||
`integrity_policy`. See [UF2 and DfuSe](https://github.com/morluto/rea/blob/main/docs/uf2-dfuse.md).
|
||||
|
||||
## ESP images
|
||||
|
||||
For an ESP32-family app or bootloader image, `open_binary` then
|
||||
`inspect_artifact` with `detail: "summary"`. `root_format_metadata` holds the
|
||||
observed `header` and `descriptor` (its `kind` is the app or bootloader role)
|
||||
and the derived `chip`, `checksum`, `appended_sha256` and
|
||||
`undescribed_trailing` ranges. Segment rows (`segment-NN.bin`) carry
|
||||
`load_addr`, `classification` (`mapped`, `loaded`, `not-loaded` or `unknown`,
|
||||
per ESP-IDF `should_map`/`should_load`; RAM ranges are not checked) and
|
||||
`source_location`; select them with `inspect_analysis_view` and
|
||||
`logical_paths`. Signature blocks are `unverified`; `public_key_sha256`
|
||||
matches eFuse key digests.
|
||||
|
||||
Unknown `source_coverage` means truncation or undescribed trailing bytes, such
|
||||
as the flash segments of a `--ram-only-header` image; do not infer absence
|
||||
from it. Erased flash is described. Integrity mismatches are facts, not
|
||||
refusals. ESP8266 images are refused, and flash-encrypted images are not
|
||||
recognized. Segments are not disassembled, and RISC-V or Xtensa segments
|
||||
cannot be opened as raw-binary. See
|
||||
[ESP images](https://github.com/morluto/rea/blob/main/docs/esp-image.md).
|
||||
|
||||
## ESP flash images
|
||||
|
||||
A whole-flash dump opens as `esp-flash-image`. `root_format_metadata` holds
|
||||
`partition_table` (its `method`: `default`, `caller` or a derived
|
||||
`scan`), `bootloader` and `apps` ESP image facts, the `otadata` records,
|
||||
`partitions_outside_dump` and `boot_selection`. Every table has a
|
||||
`boot_selection`: its `basis` names the bootloader's branch (`no-otadata`
|
||||
without otadata, `none` when the bootloader rejects the table, `unknown`
|
||||
when a record is missing), and it lists its `assumptions`;
|
||||
`running_partition` is always `unknown`. Each image names its entry
|
||||
`path`. With `detail: "summary"` the root kind is `esp-flash-image-summary`
|
||||
and each image is a compact `summary` (chip, entry point, segment count,
|
||||
classified load ranges, signature block count, descriptor identity,
|
||||
integrity); complete detail keeps every segment. Partition rows
|
||||
(`partitions/<label>.bin`) carry the declared `type`, `subtype`, names,
|
||||
`flash_address`, `size` and flags; `encrypted` is a storage flag, not
|
||||
proof of ciphertext. `written_offset`/`written_end` bound the bytes that
|
||||
are not erased: an app slot whose facts say `erased` can still hold an older
|
||||
image, and a limitation says so. For a dump read from another flash address, call
|
||||
`open_binary` with `format: "esp-flash-image"` and `flash_offset`; an
|
||||
`esp-image` whose `flash_dump` is set names that offset in a next probe; a
|
||||
relocated table can be named with `partition_table_offset`. No table found
|
||||
means unknown, often flash encryption. Extraction writes `nvs_keys` as
|
||||
ordinary data. See
|
||||
[ESP flash images](https://github.com/morluto/rea/blob/main/docs/esp-flash-image.md).
|
||||
@@ -1,303 +0,0 @@
|
||||
# Offline format and metadata inspection
|
||||
|
||||
Read the section matching the supplied artifact; other sections are optional.
|
||||
|
||||
- [Go build metadata](#go-build-metadata)
|
||||
- [Model files](#model-files)
|
||||
- [IL2CPP metadata](#il2cpp-metadata)
|
||||
- [Rust dependency metadata](#rust-dependency-metadata)
|
||||
- [PE Authenticode certificate tables](#pe-authenticode-certificate-tables)
|
||||
- [PE import and delay-load tables](#pe-import-and-delay-load-tables)
|
||||
- [PE TLS callbacks and load configuration](#pe-tls-callbacks-and-load-configuration)
|
||||
- [Binary hardening](#binary-hardening)
|
||||
- [ELF note tables](#elf-note-tables)
|
||||
- [Apple IPS crash reports](#apple-ips-crash-reports)
|
||||
- [Linux kernel modules](#linux-kernel-modules)
|
||||
- [CPython pyc files](#cpython-pyc-files)
|
||||
- [Erlang BEAM files](#erlang-beam-files)
|
||||
- [SQLite snapshots](#sqlite-snapshots)
|
||||
- [Recorded Linux crashes](#recorded-linux-crashes)
|
||||
- [Offline ELF layout](#offline-elf-layout)
|
||||
- [Offline WASM](#offline-wasm)
|
||||
- [Offline APK signatures](#offline-apk-signatures)
|
||||
- [Offline PE resource inventory](#offline-pe-resource-inventory)
|
||||
- [Offline PE and ELF export tables](#offline-pe-and-elf-export-tables)
|
||||
|
||||
## Go build metadata
|
||||
|
||||
Use `inspect_go_binary` for an explicit local ELF, PE or Mach-O (universal
|
||||
files need `architecture`) when the
|
||||
question concerns the embedded compiler version, module/dependency versions,
|
||||
local replacements or build settings. No `open_binary`, Go installation or
|
||||
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
|
||||
Preserve artifact SHA-256, original file ranges, framed module bytes and every
|
||||
unparsed module-text line. A missing record is an unknown, not proof of a
|
||||
non-Go binary; observed metadata does not authenticate its claims. Function,
|
||||
source and type recovery remain a separate native-analysis question.
|
||||
|
||||
## Model files
|
||||
|
||||
Use `inspect_model_artifact` (`inspect-model-artifact PATH`) for local GGUF v2/v3
|
||||
and safetensors declarations without a model runtime. Header-only mode does not
|
||||
read weights or establish full-file identity. `hash: "full"` explicitly streams
|
||||
all bytes. Retained `model-artifact` views page `metadata`, `tensors` and derived
|
||||
`types`, with exact batch filters and selected fields. Arrays retain typed counts
|
||||
and eight-element previews; omitted values, unknown types and runtime behavior
|
||||
stay explicit. See [model files](https://github.com/morluto/rea/blob/main/docs/model-artifacts.md).
|
||||
|
||||
## IL2CPP metadata
|
||||
|
||||
Use `inspect_il2cpp_metadata` for an explicit local Unity `global-metadata.dat`
|
||||
(versions 24, 27, 29, 31 and 39) when the question concerns assembly images,
|
||||
names, member owners, string literals, namespaces or index ranges. No `open_binary`, Unity installation or target
|
||||
execution is required. The CLI equivalent is `inspect-il2cpp-metadata PATH`.
|
||||
MCP defaults to counts with complete retained Evidence. Query `sections`,
|
||||
`images`, `types`, `methods`, `fields`, `properties`, `events`, `parameters` or
|
||||
`string_literals` through `inspect_analysis_view`; use exact names,
|
||||
batch indices and selected fields. Row indices are scoped by artifact SHA-256
|
||||
and collection; page offsets count filtered matches. Parameters support batch
|
||||
`method_indices` to read one method's parameters directly. Historical reports
|
||||
without member collections retain unknown counts rather than empty inventories. Preserve
|
||||
artifact SHA-256 and index widths. Unrecognized magic may mean encryption and
|
||||
is unsupported, not decrypted. Linking names to `GameAssembly` or
|
||||
`UnityFramework` code addresses remains a separate native-analysis question.
|
||||
See [IL2CPP metadata](https://github.com/morluto/rea/blob/main/docs/il2cpp-metadata.md).
|
||||
|
||||
## Rust dependency metadata
|
||||
|
||||
Use `inspect_rust_binary` for an explicit local ELF, PE, or Mach-O (universal
|
||||
files need `architecture`) when
|
||||
the question concerns the cargo-auditable crate list embedded in `.dep-v0`
|
||||
(Mach-O: `__DATA,.dep-v0`). No `open_binary`, Rust installation, or target
|
||||
execution is required. The CLI equivalent is `inspect-rust-binary PATH`.
|
||||
Preserve artifact SHA-256, package order, dependency indices, and the section
|
||||
file range. A missing section is null metadata, not proof the binary has no
|
||||
dependencies. An omitted package kind stays null. A rustc commit hash is
|
||||
inferred from a read-only `/rustc/<40-hex>/` path and does not by itself
|
||||
identify the compiler. See [Rust binaries](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
|
||||
|
||||
## PE Authenticode certificate tables
|
||||
|
||||
Use `inspect_pe_certificates` for an explicit local PE32/PE32+ image when the
|
||||
question concerns the Authenticode certificate table, WIN_CERTIFICATE types, or
|
||||
embedded PKCS#7 signer and X.509 identity fields. No `open_binary`, Windows SDK,
|
||||
or target execution is required. The CLI equivalent is
|
||||
`inspect-pe-certificates PATH`. Preserve artifact SHA-256, table file ranges,
|
||||
payload hashes, original validity-time spellings, and
|
||||
`subject_key_identifier_hex` when the signer uses that form. A missing
|
||||
`IMAGE_DIRECTORY_ENTRY_SECURITY` directory is observed absence of that table,
|
||||
not proof the image is unsigned in every catalog or overlay sense. Signatures,
|
||||
chains, timestamps, and Authenticode digest coverage remain unobserved. See
|
||||
[PE Authenticode certificate tables](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
|
||||
## PE import and delay-load tables
|
||||
|
||||
Use `inspect_pe_imports` for an explicit local PE32/PE32+ image when the
|
||||
question concerns imported DLL and function names, ordinals, delay-load
|
||||
descriptors, or the pefile import hash. No `open_binary`, Windows SDK, or
|
||||
target execution is required. The CLI equivalent is `inspect-pe-imports PATH`.
|
||||
Preserve artifact SHA-256, directory file ranges, original DLL spellings, and
|
||||
ILT/IAT RVAs. A missing `IMAGE_DIRECTORY_ENTRY_IMPORT` directory is observed
|
||||
absence of that table, not proof the image never imported APIs through another
|
||||
table or overlay. Bound snapshots, loaded IAT contents, and export-forward
|
||||
resolution remain unobserved. See
|
||||
[PE import and delay-load tables](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
|
||||
|
||||
## PE TLS callbacks and load configuration
|
||||
|
||||
Use `inspect_pe_load_config` for an explicit local PE32/PE32+ image when the
|
||||
question concerns code that runs before the entry point through TLS callbacks,
|
||||
or the security cookie, SafeSEH, CFG, EH-continuation and XFG declarations in
|
||||
the load configuration. No `open_binary`, Windows SDK, or target execution is
|
||||
required. The CLI equivalent is `inspect-pe-load-config PATH`. Preserve artifact
|
||||
SHA-256, directory and callback-array file ranges, and callback VAs as
|
||||
hexadecimal strings. Fields outside the declared `Size` are `null`. Guard-flag
|
||||
mitigations carry `evidence_kind: "inferred"`; enforcement remains unobserved.
|
||||
See
|
||||
[PE TLS callbacks and load configuration](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
|
||||
|
||||
## Binary hardening
|
||||
|
||||
Use `inspect_binary_hardening` for an explicit local ELF, PE, or Mach-O file
|
||||
when the question concerns PIE or ASLR, NX, RELRO, stack-protector or FORTIFY
|
||||
signs, control-flow flags, or stripped state. No `open_binary`, pwntools, or
|
||||
target execution is required. The CLI equivalent is
|
||||
`inspect-binary-hardening PATH`. Universal Mach-O files return one image per
|
||||
slice. A missing import is `unknown`, not proof the protection was omitted.
|
||||
Authenticode and Mach-O code-signature rows report presence, not validity.
|
||||
`hardened_runtime` stays unknown and names `inspect_signature`. See
|
||||
[binary hardening](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
|
||||
|
||||
## ELF note tables
|
||||
|
||||
Use `inspect_elf_notes` for an explicit local ELF32/ELF64 image when the
|
||||
question concerns PT_NOTE/SHT_NOTE owners, GNU build-id, or GNU ABI-tag
|
||||
fields. No `open_binary` or target execution is required. The CLI equivalent
|
||||
is `inspect-elf-notes PATH`. Preserve artifact SHA-256, table file ranges,
|
||||
descriptor hashes, and original owner spellings. A missing table is observed
|
||||
absence of those declarations, not proof the image never carried notes in
|
||||
another container. See
|
||||
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
|
||||
## Apple IPS crash reports
|
||||
|
||||
Use `inspect_apple_ips` for an explicit local Apple `.ips` file when the
|
||||
question concerns crash metadata, exception and termination facts, threads,
|
||||
usedImages, lossless registers, or documented crash classes. No `open_binary`,
|
||||
Xcode, atos, or target execution is required. The CLI equivalent is
|
||||
`inspect-apple-ips PATH`. Preserve artifact SHA-256, `bug_type` as a string,
|
||||
uppercase image UUIDs, and integer literals that exceed IEEE-754 range. Only
|
||||
`bug_type` `"309"` yields a crash body; other DiagnosticReports types return
|
||||
no crash result. Opened Mach-O and dSYM UUID joins remain unobserved. See
|
||||
[Apple IPS crash reports](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
|
||||
|
||||
## Linux kernel modules
|
||||
|
||||
Use `inspect_kernel_module` for an explicit local ELF32/ELF64 relocatable
|
||||
object when the question concerns `.modinfo` license, vermagic, alias,
|
||||
depends, or parm strings, the `__versions` symbol CRCs, or an appended module
|
||||
signature trailer. No `open_binary`, Linux host, or `modinfo` program is
|
||||
required. The CLI equivalent is `inspect-kernel-module PATH`. Preserve
|
||||
artifact SHA-256, key order, duplicate keys, CRC hex, and section file ranges.
|
||||
A missing section is observed absence when section names are known, not proof
|
||||
the module never carried that metadata. `.gnu.linkonce.this_module` is reported
|
||||
as present or absent; `struct module` stays unknown because its layout changes
|
||||
with the kernel version. `__ksymtab` records are not decoded. The PKCS#7 range
|
||||
and its SHA-256 are observed; certificate, signer, and digest coverage remain
|
||||
unvalidated. An absent trailer is not proof the module is unsigned in another
|
||||
container. gzip, xz, and zstd whole-file containers are unsupported and name
|
||||
`external_decompression` as the next probe: use an external codec utility to
|
||||
decompress into a separate file, then call `inspect_kernel_module` on that file. See
|
||||
[Linux kernel modules](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
|
||||
|
||||
## CPython pyc files
|
||||
|
||||
Use `inspect_python_pyc` for an explicit local CPython 3.7–3.13 `.pyc` when the
|
||||
question concerns the header magic, PEP 552 source hash or timestamp, nested
|
||||
code-object names, or bytecode identity. No `open_binary`, Python installation,
|
||||
or target execution is required. The CLI equivalent is `inspect-python-pyc PATH`.
|
||||
Preserve artifact SHA-256, the 16-byte header fields, nested code names, and
|
||||
bytecode SHA-256. Bytecode is not decompiled. A decoded header does not prove
|
||||
the matching `.py` is present. See [CPython pyc inspection](https://github.com/morluto/rea/blob/main/docs/python-pyc.md).
|
||||
|
||||
## Erlang BEAM files
|
||||
|
||||
Use `inspect_erlang_beam` for an explicit local `FOR1`/`BEAM` file when the
|
||||
question concerns IFF chunks, atom names, exports, imports, lambdas, or the
|
||||
Code header. No `open_binary`, Erlang installation, or bytecode execution is
|
||||
required. The CLI equivalent is `inspect-erlang-beam PATH`. Preserve artifact
|
||||
SHA-256, chunk file ranges, 1-based atom indices, and original atom spellings.
|
||||
Missing optional tables stay null. For a few facts, use `detail: "summary"` with
|
||||
a `beam` view selecting a collection, exact names/indices and fields; read later
|
||||
pages from `parent_evidence_id` or the complete Evidence bundle. Same-name
|
||||
exports retain every arity and original atom index. CLI summary delivery requires
|
||||
`--evidence_output`. Null table counts remain distinct from decoded empty tables. See
|
||||
[Erlang BEAM files](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
|
||||
|
||||
## SQLite snapshots
|
||||
|
||||
Use `inspect_sqlite_database` with an explicit local database snapshot for its
|
||||
schema and optional rows from one exact ordinary table. It requires Node.js
|
||||
24.x >=24.15 or 26+ with the native `DatabaseSync.limits` API. It also consumes the
|
||||
existing sibling `-wal`; retain both original digests when citing its result.
|
||||
Default output is a compact inventory; a selected table expands only its own
|
||||
schema and samples 20 rows. Select `columns` to avoid irrelevant or large cells,
|
||||
and request `schema_scope: "database"` only for complete schema details.
|
||||
Choose `row_limit` when a finite sample is useful and keep truncation and
|
||||
unspecified row order explicit. Virtual tables, shadow tables and views are
|
||||
schema facts, not row-read targets. REA inspects private copies and leaves
|
||||
the source files untouched; obtain a coherent quiescent database/WAL snapshot
|
||||
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
|
||||
|
||||
## Recorded Linux crashes
|
||||
|
||||
Use `inspect_recorded_crash` with an explicit Linux x86-64 ELF core path to
|
||||
inspect historical thread registers, signals, raw notes and original file
|
||||
ranges. It is independent of the active disassembler target. Source-bound
|
||||
register values and note bytes are observations; missing notes, signal-thread
|
||||
association and current executable/library identity remain unknown. PIDs are
|
||||
historical metadata and never authorize live attach or control.
|
||||
|
||||
The optional `include_debugger_context` facet adds mapping candidates through
|
||||
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
|
||||
establish current file identity; zero reported flags leave permissions unknown.
|
||||
Requested unavailable context fails with setup guidance. See
|
||||
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md)
|
||||
for exact upstream profiles, bounds and verification coverage.
|
||||
|
||||
## Offline ELF layout
|
||||
|
||||
Use `inspect_binary_layout` with an explicit local ELF path for file-backed
|
||||
sections, symbols, relocations, linkage names, and static mitigation inferences
|
||||
without opening a disassembler target. When that complete record is already
|
||||
retained, call `inspect_analysis_view` for a summary, mitigations or linkage
|
||||
facet, one section or symbol, or a stable page. Do not re-run the decoder to
|
||||
read a single object.
|
||||
|
||||
## Offline WASM
|
||||
|
||||
Use `inspect_wasm_artifact` with caller-supplied WABT for selected local bytes.
|
||||
For counts, validation and WAT identity, select `detail: "summary"` on the first
|
||||
MCP call. The complete original Evidence stays retained; use
|
||||
`normalized_result.parent_evidence_id` for exact exports or text pages. Omitted
|
||||
or complete detail keeps the existing complete result.
|
||||
For a completed result, use `inspect_analysis_view` with its reported retained
|
||||
reference for a summary or `view: {kind: "wasm", facet: "exports", offset: 0, limit: 32}`.
|
||||
Facets also include sections, imports, candidates, glue, WAT, headers, details
|
||||
and tool profile. Text offsets count UTF-16 code units; follow `next_offset`.
|
||||
The complete source remains retained after an oversized MCP reply. No target
|
||||
execution or network acquisition is implied, and WAT is decoded representation.
|
||||
|
||||
For exact core-module import/export differences, use `compare_wasm_interfaces`
|
||||
with two complete inspection Evidence records (or current-session retained
|
||||
references) and explicit scope. Preserve unknown types/duplicate names; static
|
||||
interface equality is not runtime compatibility. CLI accepts complete inline
|
||||
Evidence in `rea compare-wasm-interfaces input.json --json`, or exact references
|
||||
with `--evidence-bundle /absolute/path/evidence.json`. Use the complete parent
|
||||
ID when the inspection returned a summary.
|
||||
|
||||
## Offline APK signatures
|
||||
|
||||
Use `inspect_apk_signature` for a selected local APK/ZIP/AAB. Default MCP summaries
|
||||
retain complete Evidence. Select `view.collection: signers/digests/lineage`,
|
||||
with scheme or ID batches, fields and paging; use `inspect_analysis_view` kind
|
||||
`apk-signature` for retained reads after deletion. IDs retain artifact/scheme
|
||||
scope and signer links; `fields: []` keeps identity links. Empty pages preserve
|
||||
partial decoding reasons and unknown authenticity. CLI compact delivery needs
|
||||
`--evidence_output` to preserve the complete parent.
|
||||
It reads the APK
|
||||
Signing Block (v2/v3/v3.1) and v1 `META-INF` signing without executing anything,
|
||||
and reports each signer's certificate facts (for v1, the certificate the PKCS#7
|
||||
block's SignerInfo names), the declared v3 rotation lineage,
|
||||
the declared source-stamp certificate and, for SHA-256/SHA-512 digests, whether
|
||||
the declared content digest matches the current bytes. That match is not a
|
||||
signature verification: anyone who changes the bytes can recompute the digest.
|
||||
`posture` summarises v1-only signing (which cannot tell a v1-only APK from one
|
||||
whose v2/v3 block was removed), observed signer agreement across schemes,
|
||||
whether the v1 signer equals the block-scheme signer (`false` is notable),
|
||||
whether every differing signer appears in a declared (unverified) rotation
|
||||
lineage, and the strongest scheme.
|
||||
It never claims authenticity: no signature or lineage is cryptographically
|
||||
verified.
|
||||
|
||||
## Offline PE resource inventory
|
||||
|
||||
Use `inspect_pe_resources` with an absolute `path`, or
|
||||
`rea inspect-pe-resources ./selected.exe --json`, for portable PE32/PE32+
|
||||
resource inspection without opening a native target. Preserve returned Evidence:
|
||||
type/name/language IDs and UTF-16 names, original directory/data/payload ranges,
|
||||
code pages, payload hashes and static icon-group references. Same-language icon
|
||||
resolution is explicit; missing references and size mismatch remain visible.
|
||||
Other payloads are opaque, and this operation does not decode images or simulate
|
||||
Windows resource loading. Consult the budgets and supported tree profile in
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
|
||||
## Offline PE and ELF export tables
|
||||
|
||||
Use `inspect_native_exports` with an absolute `path`, or
|
||||
`rea inspect-native-exports ./libexample.so --json`, to read a PE export
|
||||
directory or ELF dynamic symbol table on any host without opening a native
|
||||
target. Ordinal-only, forwarded and hidden-version exports stay distinct;
|
||||
partial or unsupported coverage means the inventory is incomplete, not that
|
||||
exports are absent. Use `inspect_macho` for Mach-O. See
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
@@ -6,7 +6,7 @@ For captured or bundled JavaScript that needs readable source recovery, use
|
||||
`additional_paths` when advertised, then analyze the returned `analysis_input`
|
||||
directory together. Each input retains its own snapshot, digest and byte-range
|
||||
provenance; inputs and recovered code are never executed. Recovery requires a
|
||||
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/guides/javascript/javascript-recovery.md).
|
||||
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
|
||||
|
||||
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
|
||||
extracted tree. When the question concerns known source paths and the connected
|
||||
@@ -169,10 +169,7 @@ leaving it, and `both` when the question needs both. Trace traversal includes
|
||||
containment relationships, so even a narrow seed can expand many nodes; a
|
||||
bidirectional trace can reach unrelated sibling modules. A module item or page
|
||||
is preferable when its observations already answer the question. Do not treat
|
||||
all reached nodes as participants in the feature. For a broad trace, pass
|
||||
`view: {"kind": "feature-trace", "collection": "paths"}` (or `summary`) to
|
||||
retain the complete trace and read terminal paths first; page `nodes` or `edges`
|
||||
with exact `where.node_ids` through `inspect_analysis_view` as needed.
|
||||
all reached nodes as participants in the feature.
|
||||
|
||||
Reuse retained parent Evidence when the server advertises it, or supply complete
|
||||
Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest
|
||||
@@ -239,19 +236,10 @@ This notation is
|
||||
a template: replace it with the actual returned ID. Versions before 4.1.0 accept only full
|
||||
inline Evidence. Use the exact ID
|
||||
returned by the producer on the same MCP connection. Resolution does not run
|
||||
analysis or select a provider; findings remain inline. For summary or inspection
|
||||
replies, select the complete `normalized_result.parent_evidence_id`. Native
|
||||
observation arrays and runtime reconciliation's `static_layers[].analysis` and
|
||||
`runtime_observations[]` also accept exact references where advertised. Each
|
||||
source can independently be inline or retained, preserving identity, coverage
|
||||
and unknowns. CLI workflows advertising `--evidence-bundle` resolve these inputs
|
||||
from the explicitly selected bundle. `close_binary` releases the engine while
|
||||
retaining references and complete Unknown history on the same MCP connection.
|
||||
`reset_investigation` explicitly clears them; optional `save_path` checkpoints the
|
||||
complete bundle before clearing. Import that bundle in another connection to
|
||||
restore its original references. Retention is bounded, in-memory and never silently
|
||||
evicted; `binary_session.investigation` reports the budget. A missing reference
|
||||
includes its ID and recovery guidance.
|
||||
analysis or select a provider; findings remain inline. Native observation
|
||||
arrays still take full Evidence. `close_binary` clears retained references;
|
||||
export a bundle before closing, or supply portable inline Evidence in another
|
||||
connection. A missing reference includes its ID and recovery guidance.
|
||||
|
||||
For exact file/native metadata or processing gaps, use
|
||||
`view: {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"], limit: 32}`.
|
||||
|
||||
@@ -1,28 +1,8 @@
|
||||
# Native and managed analysis
|
||||
# Native, managed, and packaged artifacts
|
||||
|
||||
Read the section matching the supplied artifact; other sections are optional.
|
||||
|
||||
- [Other artifact guides](#other-artifact-guides)
|
||||
- [Apple app resources](#apple-app-resources)
|
||||
- [Desktop installation directories](#desktop-installation-directories)
|
||||
- [Native targets](#native-targets)
|
||||
- [Continue editable Ghidra analysis across processes](#continue-editable-ghidra-analysis-across-processes)
|
||||
- [Reuse caller-selected Ghidra analysis](#reuse-caller-selected-ghidra-analysis)
|
||||
- [Ghidra startup deadline recovery](#ghidra-startup-deadline-recovery)
|
||||
- [Bridge request ceiling](#bridge-request-ceiling)
|
||||
- [Selected native facts and call graphs](#selected-native-facts-and-call-graphs)
|
||||
- [Static memory and original file bytes](#static-memory-and-original-file-bytes)
|
||||
- [DOS MZ with Ghidra](#dos-mz-with-ghidra)
|
||||
- [Managed PE/CLI](#managed-pecli)
|
||||
- [DOS COM loader interpretation](#dos-com-loader-interpretation)
|
||||
- [Function annotations in Ghidra](#function-annotations-in-ghidra)
|
||||
|
||||
For release comparisons on one MCP connection, retain `list_procedures` Evidence
|
||||
from each target and reuse its exact IDs after target switches or `close_binary`.
|
||||
For a handoff or new process, export the complete bundle and import it there,
|
||||
including its source closure. `reset_investigation` explicitly clears retained
|
||||
records; optional `save_path` checkpoints them before clearing.
|
||||
`match_native_functions` pairs unique exact names and, when supplied,
|
||||
For release comparisons, export `list_procedures` Evidence from each target before
|
||||
closing it. Close clears session records: import both bundles after collecting the
|
||||
two targets, or supply portable inline Evidence. `match_native_functions` pairs unique exact names and, when supplied,
|
||||
complete observed body bytes. Reuse optional same-context `list_names`,
|
||||
`procedure_info`, `read_bytes` and existing `analyze_function` dossiers as
|
||||
`left_observations`/`right_observations`; an MCP `analyze_function` summary
|
||||
@@ -33,13 +13,6 @@ Select columns, batch IDs and page individual facts; aliases remain unresolved.
|
||||
Unpaired observations do not prove additions/removals. Existing matched dossier
|
||||
pairs include inputs for `compare_functions`; otherwise analyze just chosen pairs.
|
||||
|
||||
Aggregate completed comparisons with `find_changed_behavior`, selecting each
|
||||
comparison's exact `retained-evidence` ID instead of resending full records.
|
||||
Inline comparison Evidence remains accepted. References belong to the current
|
||||
server: after handoff, import the original bundle including its source closure.
|
||||
Do not substitute dossier IDs or summary views. Static differences remain
|
||||
behavior candidates, with runtime behavior unknown unless separately observed.
|
||||
|
||||
For a matching representation refusal, inspect `details.limit` and raise
|
||||
`native.matching_representation_bytes` with `update_analysis_settings` before
|
||||
retrying the unchanged MCP request. CLI launch seed:
|
||||
@@ -79,6 +52,243 @@ runtime presence. For compact CLI output, use
|
||||
`inspect-desktop-application PATH --detail summary --evidence_output BUNDLE`.
|
||||
See [desktop installations](https://github.com/morluto/rea/blob/main/docs/desktop-applications.md).
|
||||
|
||||
## Go build metadata
|
||||
|
||||
Use `inspect_go_binary` for an explicit local ELF, PE or Mach-O (universal
|
||||
files need `architecture`) when the
|
||||
question concerns the embedded compiler version, module/dependency versions,
|
||||
local replacements or build settings. No `open_binary`, Go installation or
|
||||
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
|
||||
Preserve artifact SHA-256, original file ranges, framed module bytes and every
|
||||
unparsed module-text line. A missing record is an unknown, not proof of a
|
||||
non-Go binary; observed metadata does not authenticate its claims. Function,
|
||||
source and type recovery remain a separate native-analysis question.
|
||||
|
||||
## Model files
|
||||
|
||||
Use `inspect_model_artifact` (`inspect-model-artifact PATH`) for local GGUF v2/v3
|
||||
and safetensors declarations without a model runtime. Header-only mode does not
|
||||
read weights or establish full-file identity. `hash: "full"` explicitly streams
|
||||
all bytes. Retained `model-artifact` views page `metadata`, `tensors` and derived
|
||||
`types`, with exact batch filters and selected fields. Arrays retain typed counts
|
||||
and eight-element previews; omitted values, unknown types and runtime behavior
|
||||
stay explicit. See [model files](https://github.com/morluto/rea/blob/main/docs/model-artifacts.md).
|
||||
|
||||
## IL2CPP metadata
|
||||
|
||||
Use `inspect_il2cpp_metadata` for an explicit local Unity `global-metadata.dat`
|
||||
(versions 24, 27, 29, 31 and 39) when the question concerns assembly images,
|
||||
names, member owners, string literals, namespaces or index ranges. No `open_binary`, Unity installation or target
|
||||
execution is required. The CLI equivalent is `inspect-il2cpp-metadata PATH`.
|
||||
MCP defaults to counts with complete retained Evidence. Query `sections`,
|
||||
`images`, `types`, `methods`, `fields`, `properties`, `events`, `parameters` or
|
||||
`string_literals` through `inspect_analysis_view`; use exact names,
|
||||
batch indices and selected fields. Row indices are scoped by artifact SHA-256
|
||||
and collection; page offsets count filtered matches. Parameters support batch
|
||||
`method_indices` to read one method's parameters directly. Historical reports
|
||||
without member collections retain unknown counts rather than empty inventories. Preserve
|
||||
artifact SHA-256 and index widths. Unrecognized magic may mean encryption and
|
||||
is unsupported, not decrypted. Linking names to `GameAssembly` or
|
||||
`UnityFramework` code addresses remains a separate native-analysis question.
|
||||
See [IL2CPP metadata](https://github.com/morluto/rea/blob/main/docs/il2cpp-metadata.md).
|
||||
|
||||
## Rust dependency metadata
|
||||
|
||||
Use `inspect_rust_binary` for an explicit local ELF, PE, or Mach-O (universal
|
||||
files need `architecture`) when
|
||||
the question concerns the cargo-auditable crate list embedded in `.dep-v0`
|
||||
(Mach-O: `__DATA,.dep-v0`). No `open_binary`, Rust installation, or target
|
||||
execution is required. The CLI equivalent is `inspect-rust-binary PATH`.
|
||||
Preserve artifact SHA-256, package order, dependency indices, and the section
|
||||
file range. A missing section is null metadata, not proof the binary has no
|
||||
dependencies. An omitted package kind stays null. A rustc commit hash is
|
||||
inferred from a read-only `/rustc/<40-hex>/` path and does not by itself
|
||||
identify the compiler. See [Rust binaries](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
|
||||
|
||||
## PE Authenticode certificate tables
|
||||
|
||||
Use `inspect_pe_certificates` for an explicit local PE32/PE32+ image when the
|
||||
question concerns the Authenticode certificate table, WIN_CERTIFICATE types, or
|
||||
embedded PKCS#7 signer and X.509 identity fields. No `open_binary`, Windows SDK,
|
||||
or target execution is required. The CLI equivalent is
|
||||
`inspect-pe-certificates PATH`. Preserve artifact SHA-256, table file ranges,
|
||||
payload hashes, original validity-time spellings, and
|
||||
`subject_key_identifier_hex` when the signer uses that form. A missing
|
||||
`IMAGE_DIRECTORY_ENTRY_SECURITY` directory is observed absence of that table,
|
||||
not proof the image is unsigned in every catalog or overlay sense. Signatures,
|
||||
chains, timestamps, and Authenticode digest coverage remain unobserved. See
|
||||
[PE Authenticode certificate tables](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
|
||||
## PE import and delay-load tables
|
||||
|
||||
Use `inspect_pe_imports` for an explicit local PE32/PE32+ image when the
|
||||
question concerns imported DLL and function names, ordinals, delay-load
|
||||
descriptors, or the pefile import hash. No `open_binary`, Windows SDK, or
|
||||
target execution is required. The CLI equivalent is `inspect-pe-imports PATH`.
|
||||
Preserve artifact SHA-256, directory file ranges, original DLL spellings, and
|
||||
ILT/IAT RVAs. A missing `IMAGE_DIRECTORY_ENTRY_IMPORT` directory is observed
|
||||
absence of that table, not proof the image never imported APIs through another
|
||||
table or overlay. Bound snapshots, loaded IAT contents, and export-forward
|
||||
resolution remain unobserved. See
|
||||
[PE import and delay-load tables](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
|
||||
|
||||
## PE TLS callbacks and load configuration
|
||||
|
||||
Use `inspect_pe_load_config` for an explicit local PE32/PE32+ image when the
|
||||
question concerns code that runs before the entry point through TLS callbacks,
|
||||
or the security cookie, SafeSEH, CFG, EH-continuation and XFG declarations in
|
||||
the load configuration. No `open_binary`, Windows SDK, or target execution is
|
||||
required. The CLI equivalent is `inspect-pe-load-config PATH`. Preserve artifact
|
||||
SHA-256, directory and callback-array file ranges, and callback VAs as
|
||||
hexadecimal strings. Fields outside the declared `Size` are `null`. Guard-flag
|
||||
mitigations carry `evidence_kind: "inferred"`; enforcement remains unobserved.
|
||||
See
|
||||
[PE TLS callbacks and load configuration](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
|
||||
|
||||
## ELF note tables
|
||||
|
||||
Use `inspect_elf_notes` for an explicit local ELF32/ELF64 image when the
|
||||
question concerns PT_NOTE/SHT_NOTE owners, GNU build-id, or GNU ABI-tag
|
||||
fields. No `open_binary` or target execution is required. The CLI equivalent
|
||||
is `inspect-elf-notes PATH`. Preserve artifact SHA-256, table file ranges,
|
||||
descriptor hashes, and original owner spellings. A missing table is observed
|
||||
absence of those declarations, not proof the image never carried notes in
|
||||
another container. See
|
||||
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
|
||||
## Apple IPS crash reports
|
||||
|
||||
Use `inspect_apple_ips` for an explicit local Apple `.ips` file when the
|
||||
question concerns crash metadata, exception and termination facts, threads,
|
||||
usedImages, lossless registers, or documented crash classes. No `open_binary`,
|
||||
Xcode, atos, or target execution is required. The CLI equivalent is
|
||||
`inspect-apple-ips PATH`. Preserve artifact SHA-256, `bug_type` as a string,
|
||||
uppercase image UUIDs, and integer literals that exceed IEEE-754 range. Only
|
||||
`bug_type` `"309"` yields a crash body; other DiagnosticReports types return
|
||||
no crash result. Opened Mach-O and dSYM UUID joins remain unobserved. See
|
||||
[Apple IPS crash reports](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
|
||||
|
||||
## Linux kernel modules
|
||||
|
||||
Use `inspect_kernel_module` for an explicit local ELF32/ELF64 relocatable
|
||||
object when the question concerns `.modinfo` license, vermagic, alias,
|
||||
depends, or parm strings, the `__versions` symbol CRCs, or an appended module
|
||||
signature trailer. No `open_binary`, Linux host, or `modinfo` program is
|
||||
required. The CLI equivalent is `inspect-kernel-module PATH`. Preserve
|
||||
artifact SHA-256, key order, duplicate keys, CRC hex, and section file ranges.
|
||||
A missing section is observed absence when section names are known, not proof
|
||||
the module never carried that metadata. `.gnu.linkonce.this_module` is reported
|
||||
as present or absent; `struct module` stays unknown because its layout changes
|
||||
with the kernel version. `__ksymtab` records are not decoded. The PKCS#7 range
|
||||
and its SHA-256 are observed; certificate, signer, and digest coverage remain
|
||||
unvalidated. An absent trailer is not proof the module is unsigned in another
|
||||
container. gzip, xz, and zstd whole-file containers are unsupported and name
|
||||
`external_decompression` as the next probe: use an external codec utility to
|
||||
decompress into a separate file, then call `inspect_kernel_module` on that file. See
|
||||
[Linux kernel modules](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
|
||||
|
||||
## CPython pyc files
|
||||
|
||||
Use `inspect_python_pyc` for an explicit local CPython 3.7–3.13 `.pyc` when the
|
||||
question concerns the header magic, PEP 552 source hash or timestamp, nested
|
||||
code-object names, or bytecode identity. No `open_binary`, Python installation,
|
||||
or target execution is required. The CLI equivalent is `inspect-python-pyc PATH`.
|
||||
Preserve artifact SHA-256, the 16-byte header fields, nested code names, and
|
||||
bytecode SHA-256. Bytecode is not decompiled. A decoded header does not prove
|
||||
the matching `.py` is present. See [CPython pyc inspection](https://github.com/morluto/rea/blob/main/docs/python-pyc.md).
|
||||
|
||||
## Erlang BEAM files
|
||||
|
||||
Use `inspect_erlang_beam` for an explicit local `FOR1`/`BEAM` file when the
|
||||
question concerns IFF chunks, atom names, exports, imports, lambdas, or the
|
||||
Code header. No `open_binary`, Erlang installation, or bytecode execution is
|
||||
required. The CLI equivalent is `inspect-erlang-beam PATH`. Preserve artifact
|
||||
SHA-256, chunk file ranges, 1-based atom indices, and original atom spellings.
|
||||
Missing optional tables stay null. For a few facts, use `detail: "summary"` with
|
||||
a `beam` view selecting a collection, exact names/indices and fields; read later
|
||||
pages from `parent_evidence_id` or the complete Evidence bundle. Same-name
|
||||
exports retain every arity and original atom index. CLI summary delivery requires
|
||||
`--evidence_output`. Null table counts remain distinct from decoded empty tables. See
|
||||
[Erlang BEAM files](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
|
||||
|
||||
## SQLite snapshots
|
||||
|
||||
Use `inspect_sqlite_database` with an explicit local database snapshot for its
|
||||
schema and optional rows from one exact ordinary table. It requires Node.js
|
||||
24.x >=24.15 or 26+ with the native `DatabaseSync.limits` API. It also consumes the
|
||||
existing sibling `-wal`; retain both original digests when citing its result.
|
||||
Default output is a compact inventory; a selected table expands only its own
|
||||
schema and samples 20 rows. Select `columns` to avoid irrelevant or large cells,
|
||||
and request `schema_scope: "database"` only for complete schema details.
|
||||
Choose `row_limit` when a finite sample is useful and keep truncation and
|
||||
unspecified row order explicit. Virtual tables, shadow tables and views are
|
||||
schema facts, not row-read targets. REA inspects private copies and leaves
|
||||
the source files untouched; obtain a coherent quiescent database/WAL snapshot
|
||||
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
|
||||
## Recorded Linux crashes
|
||||
|
||||
Use `inspect_recorded_crash` with an explicit Linux x86-64 ELF core path to
|
||||
inspect historical thread registers, signals, raw notes and original file
|
||||
ranges. It is independent of the active disassembler target. Source-bound
|
||||
register values and note bytes are observations; missing notes, signal-thread
|
||||
association and current executable/library identity remain unknown. PIDs are
|
||||
historical metadata and never authorize live attach or control.
|
||||
|
||||
The optional `include_debugger_context` facet adds mapping candidates through
|
||||
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
|
||||
establish current file identity; zero reported flags leave permissions unknown.
|
||||
Requested unavailable context fails with setup guidance. See
|
||||
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md)
|
||||
for exact upstream profiles, bounds and verification coverage.
|
||||
|
||||
## Offline ELF layout
|
||||
|
||||
Use `inspect_binary_layout` with an explicit local ELF path for file-backed
|
||||
sections, symbols, relocations, linkage names, and static mitigation inferences
|
||||
without opening a disassembler target. When that complete record is already
|
||||
retained, call `inspect_analysis_view` for a summary, mitigations or linkage
|
||||
facet, one section or symbol, or a stable page. Do not re-run the decoder to
|
||||
read a single object.
|
||||
|
||||
## Offline WASM
|
||||
|
||||
Use `inspect_wasm_artifact` with caller-supplied WABT for selected local bytes.
|
||||
For counts, validation and WAT identity, select `detail: "summary"` on the first
|
||||
MCP call. The complete original Evidence stays retained; use
|
||||
`normalized_result.parent_evidence_id` for exact exports or text pages. Omitted
|
||||
or complete detail keeps the existing complete result.
|
||||
For a completed result, use `inspect_analysis_view` with its reported retained
|
||||
reference for a summary or `view: {kind: "wasm", facet: "exports", offset: 0, limit: 32}`.
|
||||
Facets also include sections, imports, candidates, glue, WAT, headers, details
|
||||
and tool profile. Text offsets count UTF-16 code units; follow `next_offset`.
|
||||
The complete source remains retained after an oversized MCP reply. No target
|
||||
execution or network acquisition is implied, and WAT is decoded representation.
|
||||
|
||||
## Offline APK signatures
|
||||
|
||||
Use `inspect_apk_signature` for a selected local APK/ZIP/AAB. Default MCP summaries
|
||||
retain complete Evidence. Select `view.collection: signers/digests/lineage`,
|
||||
with scheme or ID batches, fields and paging; use `inspect_analysis_view` kind
|
||||
`apk-signature` for retained reads after deletion. IDs retain artifact/scheme
|
||||
scope and signer links; `fields: []` keeps identity links. Empty pages preserve
|
||||
partial decoding reasons and unknown authenticity. CLI compact delivery needs
|
||||
`--evidence_output` to preserve the complete parent.
|
||||
It reads the APK
|
||||
Signing Block (v2/v3/v3.1) and v1 `META-INF` signing without executing anything,
|
||||
and reports each signer's certificate facts (for v1, the certificate the PKCS#7
|
||||
block's SignerInfo names), the declared v3 rotation lineage,
|
||||
the declared source-stamp certificate and, for SHA-256/SHA-512 digests, whether
|
||||
the declared content digest matches the current bytes. That match is not a
|
||||
signature verification: anyone who changes the bytes can recompute the digest.
|
||||
`posture` summarises v1-only signing (which cannot tell a v1-only APK from one
|
||||
whose v2/v3 block was removed), observed signer agreement across schemes,
|
||||
whether the v1 signer equals the block-scheme signer (`false` is notable),
|
||||
whether every differing signer appears in a declared (unverified) rotation
|
||||
lineage, and the strongest scheme.
|
||||
It never claims authenticity: no signature or lineage is cryptographically
|
||||
verified.
|
||||
|
||||
## Native targets
|
||||
|
||||
After `open_binary`, use focused search, procedure, or function tools directly.
|
||||
@@ -88,18 +298,6 @@ cross-references. Addresses and recovered pseudocode are analysis observations,
|
||||
not original source. Provider unavailability and unsupported metadata remain
|
||||
unknown rather than false.
|
||||
|
||||
For large pseudocode, request `procedure_pseudo_code` with `detail: "summary"`
|
||||
on the first MCP call. It retains the complete observation and returns the
|
||||
original selection, text availability and length without the body. Use
|
||||
`normalized_result.parent_evidence_id` with `inspect_analysis_view` and
|
||||
`view: {kind: "native", facet: "pseudocode", offset: 0, limit: 20000}`;
|
||||
follow `coverage.next_offset` for later pages. Choose a smaller limit if one
|
||||
page exceeds the client or transport budget. Offsets and limits count UTF-16
|
||||
code units, not lines or tokens. Pages reuse the same observation without
|
||||
repeating decompilation. Null and empty text remain distinct. Omitted or
|
||||
`complete` detail preserves full inline delivery. Summary delivery requires a
|
||||
retaining session and does not reduce the work or memory of decompilation.
|
||||
|
||||
For a completed `analyze_function` dossier, use `inspect_analysis_view` with
|
||||
`view: {kind: "native", facet: "value_flow_operations", offset: 0, limit: 32}`
|
||||
and the exact retained Evidence ID to read selected facts without reanalysis.
|
||||
@@ -138,9 +336,8 @@ the complete tool inventory grouped by current status. Expand each group's
|
||||
overrides them; availability, reasons, remediation, and modes remain inline.
|
||||
Provider candidates keep availability and target-support status, code, and
|
||||
reason. `omitted_fields` names what the summary leaves out; `{"detail":
|
||||
"complete"}` adds each unavailable candidate's `remediation`, provider
|
||||
diagnostics, candidate capability sets, and tool surface, `effects`, and
|
||||
`annotations`. Those arrays name true flags from fixed
|
||||
"complete"}` adds provider diagnostics, candidate capability sets, and tool
|
||||
surface, `effects`, and `annotations`. Those arrays name true flags from fixed
|
||||
vocabularies, so an omitted known flag is false. `tools/list` remains the
|
||||
canonical source for full tool schemas when target or provider state changes.
|
||||
|
||||
@@ -150,28 +347,6 @@ operation's full ordered list. Candidate capabilities, returned with complete
|
||||
detail, describe possible providers. Top-level session capabilities describe the current route, and
|
||||
`analysis_provider_binding` identifies the selected provider.
|
||||
|
||||
## Continue editable Ghidra analysis across processes
|
||||
|
||||
For a task spanning agent sessions, check `binary_session` after selecting
|
||||
Ghidra: the current provider's `capabilities[].shared_limitations` states whether
|
||||
Program checkpoints are enabled or the session is ephemeral. Successful native
|
||||
queries distinguish initial analysis from an actually restored packed Program;
|
||||
do not infer that restoration happened merely because checkpoint mode is enabled.
|
||||
|
||||
In ephemeral mode, choose a private absolute directory and set
|
||||
`REA_GHIDRA_CHECKPOINT_DIR` in the environment starting REA. Restart the MCP server
|
||||
after changing that launch configuration; no current tool call or runtime
|
||||
analysis-setting update enables it. Stock ELF/PE/Mach-O imports on Linux/macOS
|
||||
are supported; see the [checkpoint setup and limitations](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program).
|
||||
|
||||
With the setting enabled, normal opens restore compatible saved Programs
|
||||
automatically. Initial completed analysis is saved, and `close_binary` saves
|
||||
later metadata edits; no separate save/restore tool is needed. Repair a reported
|
||||
save failure and retry close. Cancellation or process failure can lose edits
|
||||
after the last successful save; unfinished initial analysis cannot resume.
|
||||
Keep the same checkpoint configuration for later processes. Evidence/Unknown
|
||||
history and immutable REA query snapshots have separate persistence workflows.
|
||||
|
||||
## Reuse caller-selected Ghidra analysis
|
||||
|
||||
When the caller supplies an expanded project, configure `REA_GHIDRA_PROJECT_PATH`
|
||||
@@ -183,7 +358,7 @@ import overrides and repair scripts are unavailable in this mode. Treat original
|
||||
analysis completeness/settings as unknown and prior edits as caller-project state.
|
||||
The program's recorded import digest must match the target; changed project content
|
||||
requires reopening for a fresh profile. See the
|
||||
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#reuse-an-existing-ghidra-project).
|
||||
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/installation.md#reuse-an-existing-ghidra-project).
|
||||
|
||||
## Ghidra startup deadline recovery
|
||||
|
||||
@@ -195,7 +370,7 @@ When the error names `ghidra.startup_timeout_ms`, call
|
||||
`update_analysis_settings` with `action: "set"` and a larger value within the
|
||||
reported bounds. Resolve retained cleanup before the next startup attempt.
|
||||
No REA restart is needed; an already accepted startup keeps its original
|
||||
budget. Follow the error's session-specific restart guidance, and preserve useful Evidence before closing
|
||||
budget. A new attempt repeats import. Preserve useful Evidence before closing
|
||||
and reopening the target when remediation requires that lifecycle step.
|
||||
|
||||
For a per-function timeout, update `ghidra.decompile_timeout_seconds` and retry
|
||||
@@ -217,7 +392,7 @@ and memory controls in place. Disable specific expensive analyzers only when
|
||||
their missing coverage is acceptable and retain the resulting limitations.
|
||||
Exported C and embedded resources remain useful independent evidence, but a
|
||||
timeout does not establish complete analysis or absence of code. See the
|
||||
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
|
||||
## Bridge request ceiling
|
||||
|
||||
@@ -341,11 +516,6 @@ names), coverage and per-collection counts. Page rows with
|
||||
`view: {kind: "managed-members", collection: "call_edges", offset: 0, limit: 64}`
|
||||
over `types`, `exported_types`, `fields`, `methods`, `member_refs`, `call_edges`,
|
||||
`field_accesses` or `user_strings`.
|
||||
Use exact `names` and `declaring_types` filters before paging supported
|
||||
collections. For a compact method lookup, request `fields: ["name",
|
||||
"declaring_type", "signature"]`; omit `body` to avoid anchors. Identity tokens,
|
||||
original `source_index`, module/MVID and source coverage remain. Check
|
||||
`unresolved_filter_rows` before interpreting an empty match as absence.
|
||||
|
||||
### .NET NativeAOT
|
||||
|
||||
@@ -368,69 +538,134 @@ broader metadata recovery on Linux; it requires an explicitly configured
|
||||
NativeAOT JAR. These are separate routes. The built-in parser also runs on
|
||||
Windows Ghidra P0; read frozen literals from its report, because
|
||||
`search_strings` does not include them. See the
|
||||
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-nativeaot.md)
|
||||
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/ghidra-nativeaot.md)
|
||||
for setup, supported layouts, and host restrictions. Preserve binary identity
|
||||
and distinguish derived metadata from Ghidra database observations. Recovered
|
||||
types and native pseudocode are not original C# source. Windows Ghidra P0 has
|
||||
no database mutation authority.
|
||||
|
||||
For headerless firmware and address-bearing containers, read
|
||||
[raw images](firmware-and-raw-images.md#headerless-raw-images-and-firmware-containers).
|
||||
## Packages and extraction
|
||||
|
||||
## DOS COM loader interpretation
|
||||
Use `inspect_artifact` for application bundles, archives, ZIP/APK/IPA/MSIX/AppX,
|
||||
ASAR, or DMG inputs when the artifact graph and findings help answer the
|
||||
question. It returns the complete artifact graph inline. Cite graph manifest
|
||||
IDs when using them.
|
||||
|
||||
For an IPA or a macOS `.app`, ZIP, or DMG, pass the inventory Evidence to
|
||||
`project_apple_application_graph`. It reports application roots and nested
|
||||
bundles by path convention: app extensions, XPC services, frameworks, login
|
||||
items, system and driver extensions, and plug-ins. It also lists privileged
|
||||
helpers, launchd plists, symlinks, and each bundle's `info_plist_path` and
|
||||
executable candidates. Roles are path conventions, not parsed plists; read the
|
||||
listed plists with `inspect_plist`.
|
||||
|
||||
After `open_binary` on a `.app` or Mach-O, use `trace_dylib_resolution` to see
|
||||
which file each `@rpath`, `@loader_path` and `@executable_path` load reaches for
|
||||
every executable in the bundle. It also lists missing or weak loads and earlier
|
||||
`@rpath` candidates that are absent. Narrow large bundles with `roots` or
|
||||
`architecture`. System paths stay undetermined because the dyld shared cache
|
||||
provides them.
|
||||
|
||||
Use `extract_artifact` when materialized files from a supported format are needed.
|
||||
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md).
|
||||
It materializes all regular files into a fresh temporary directory chosen by
|
||||
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
|
||||
The result reports the materialized directory; callers do not choose the
|
||||
destination. No separate permission grant is needed.
|
||||
|
||||
Native DMG traversal automatically uses a read-only mount on supported macOS
|
||||
hosts, with an owned temporary mount directory and cleanup. It needs no approval
|
||||
flag. Unsupported hosts or mount failures remain explicit limitations; do not
|
||||
claim child inventory when only root identity is available. DMG extraction is
|
||||
unavailable through `extract_artifact`. To access DMG files on macOS, create a
|
||||
read-only mount with your own tools, select files from that mount for analysis,
|
||||
and detach the mount you created when finished. REA's inspection mount is
|
||||
temporary, not a retained export of the image's contents.
|
||||
|
||||
For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
|
||||
`--target-format dos-com`). Inspect the load image before trusting function
|
||||
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
|
||||
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-dos.md).
|
||||
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
|
||||
|
||||
## Function annotations in Ghidra
|
||||
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
|
||||
|
||||
For other headerless images (embedded firmware without ELF/Mach-O/PE/MZ
|
||||
headers), open with `format: "raw-binary"`, `architecture` (`x86`, `x86_64`,
|
||||
`arm`, `arm64`, `6502`, or `m68k`), and `entry_point` (`0x` hex). File offset 0 is mapped at
|
||||
that address. Hopper and IDA refuse this interpretation; Ghidra uses
|
||||
`BinaryLoader` with a generic language: big-endian for `m68k`, little-endian
|
||||
otherwise. Big-endian ARM, Thumb-only, and Cortex-M variants are unsupported.
|
||||
`6502` selects NMOS 6502; the entry and complete file mapping must fit within
|
||||
`0x0000..0xffff`. 65C02, 65816, cartridge containers, bank switching, and mapper
|
||||
semantics are unsupported. `m68k` requires an even entry and decodes the 68040
|
||||
superset of the 68000 family; Amiga/Atari containers and vector-table entry
|
||||
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
|
||||
|
||||
### Function annotations in Ghidra
|
||||
|
||||
Fresh-import sessions admit annotations; existing-project reuse is read-only.
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits. Normal close saves them when owned Program checkpoint mode is enabled; ephemeral mode discards them. CLI `annotate-native-function` follows the same setting when closing after its updated result. Select the [checkpoint workflow](#continue-editable-ghidra-analysis-across-processes) before edits that must survive a later process. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier. Ghidra rejects batch addresses that request the same name or a name another selected address holds; `raw_result.failures` gives each failure's reason.
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier.
|
||||
|
||||
A provider switch or different target can retire the active Ghidra database and discard
|
||||
its edits, even if the new provider fails on a later call. Same-target,
|
||||
same-profile reopen keeps the active database. Retained annotation Evidence
|
||||
records what happened; it does not restore the edits.
|
||||
## Offline PE resource inventory
|
||||
|
||||
For `compare_functions`, obtain `analyze_function` Evidence before and after the
|
||||
edit while the MCP session is open; annotation Evidence is not a comparison
|
||||
input. Export the needed Evidence before close if comparison will happen later.
|
||||
Authenticated same-artifact coordinates preserve renamed entries as
|
||||
`matched / same-entry`, with the canonical `entry_address` inline; name and
|
||||
exact-text changes remain separate dimensions. This requires compatible
|
||||
architecture, provider-owned mapping and selected-image commitments. Equal
|
||||
numeric addresses alone cannot establish correspondence; inspect retained
|
||||
source Evidence and coverage when the match remains ambiguous.
|
||||
Use `inspect_pe_resources` with an absolute `path`, or
|
||||
`rea inspect-pe-resources ./selected.exe --json`, for portable PE32/PE32+
|
||||
resource inspection without opening a native target. Preserve returned Evidence:
|
||||
type/name/language IDs and UTF-16 names, original directory/data/payload ranges,
|
||||
code pages, payload hashes and static icon-group references. Same-language icon
|
||||
resolution is explicit; missing references and size mismatch remain visible.
|
||||
Other payloads are opaque, and this operation does not decode images or simulate
|
||||
Windows resource loading. Consult the budgets and supported tree profile in
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
|
||||
## Other artifact guides
|
||||
## Offline PE and ELF export tables
|
||||
|
||||
Read the owning page for offline formats, containers or firmware. These explicit
|
||||
anchors preserve older links without creating duplicate searchable sections.
|
||||
Use `inspect_native_exports` with an absolute `path`, or
|
||||
`rea inspect-native-exports ./libexample.so --json`, to read a PE export
|
||||
directory or ELF dynamic symbol table on any host without opening a native
|
||||
target. Ordinal-only, forwarded and hidden-version exports stay distinct;
|
||||
partial or unsupported coverage means the inventory is incomplete, not that
|
||||
exports are absent. Use `inspect_macho` for Mach-O. See
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
|
||||
- <a id="go-build-metadata"></a>[Go build metadata](format-inspectors.md#go-build-metadata)
|
||||
- <a id="model-files"></a>[Model files](format-inspectors.md#model-files)
|
||||
- <a id="il2cpp-metadata"></a>[IL2CPP metadata](format-inspectors.md#il2cpp-metadata)
|
||||
- <a id="rust-dependency-metadata"></a>[Rust dependency metadata](format-inspectors.md#rust-dependency-metadata)
|
||||
- <a id="pe-authenticode-certificate-tables"></a>[PE Authenticode certificate tables](format-inspectors.md#pe-authenticode-certificate-tables)
|
||||
- <a id="pe-import-and-delay-load-tables"></a>[PE import and delay-load tables](format-inspectors.md#pe-import-and-delay-load-tables)
|
||||
- <a id="pe-tls-callbacks-and-load-configuration"></a>[PE TLS callbacks and load configuration](format-inspectors.md#pe-tls-callbacks-and-load-configuration)
|
||||
- <a id="binary-hardening"></a>[Binary hardening](format-inspectors.md#binary-hardening)
|
||||
- <a id="elf-note-tables"></a>[ELF note tables](format-inspectors.md#elf-note-tables)
|
||||
- <a id="apple-ips-crash-reports"></a>[Apple IPS crash reports](format-inspectors.md#apple-ips-crash-reports)
|
||||
- <a id="linux-kernel-modules"></a>[Linux kernel modules](format-inspectors.md#linux-kernel-modules)
|
||||
- <a id="cpython-pyc-files"></a>[CPython pyc files](format-inspectors.md#cpython-pyc-files)
|
||||
- <a id="erlang-beam-files"></a>[Erlang BEAM files](format-inspectors.md#erlang-beam-files)
|
||||
- <a id="sqlite-snapshots"></a>[SQLite snapshots](format-inspectors.md#sqlite-snapshots)
|
||||
- <a id="recorded-linux-crashes"></a>[Recorded Linux crashes](format-inspectors.md#recorded-linux-crashes)
|
||||
- <a id="offline-elf-layout"></a>[Offline ELF layout](format-inspectors.md#offline-elf-layout)
|
||||
- <a id="offline-wasm"></a>[Offline WASM](format-inspectors.md#offline-wasm)
|
||||
- <a id="offline-apk-signatures"></a>[Offline APK signatures](format-inspectors.md#offline-apk-signatures)
|
||||
- <a id="packages-and-extraction"></a>[Packages and extraction](packages-and-archives.md#packages-and-extraction)
|
||||
- <a id="offline-pe-resource-inventory"></a>[Offline PE resource inventory](format-inspectors.md#offline-pe-resource-inventory)
|
||||
- <a id="offline-pe-and-elf-export-tables"></a>[Offline PE and ELF export tables](format-inspectors.md#offline-pe-and-elf-export-tables)
|
||||
- <a id="godot-pck"></a>[Godot PCK](packages-and-archives.md#godot-pck)
|
||||
- <a id="esp-images"></a>[ESP images](firmware-and-raw-images.md#esp-images)
|
||||
- <a id="esp-flash-images"></a>[ESP flash images](firmware-and-raw-images.md#esp-flash-images)
|
||||
- <a id="bun-standalone-executables"></a>[Bun standalone executables](packages-and-archives.md#bun-standalone-executables)
|
||||
For exact core-module import/export differences, use `compare_wasm_interfaces`
|
||||
with two complete inspection Evidence records (or current-session retained
|
||||
references) and explicit scope. Preserve unknown types/duplicate names; static
|
||||
interface equality is not runtime compatibility. CLI accepts complete inline
|
||||
Evidence in `rea compare-wasm-interfaces input.json --json`.
|
||||
|
||||
## Godot PCK
|
||||
|
||||
For a standalone PCK, `open_binary` then `inspect_artifact` with
|
||||
`detail: "summary"` retains the complete static inventory. Use the returned
|
||||
`parent_evidence_id` with `inspect_analysis_view` inventory occurrences filtered
|
||||
by exact `logical_paths` or `occurrence_ids`, and join `artifact_id` to nodes
|
||||
filtered by `artifact_ids`. Select only needed fields; identity links remain.
|
||||
Prefer `inspect-artifact --detail summary --evidence_output PATH` for CLI tasks;
|
||||
the same CLI discovers embedded packs via their EOF trailer without opening a
|
||||
native analysis provider. Normal native opening remains unchanged.
|
||||
|
||||
Distinguish declared MD5 from observed SHA-256 and graph/page completeness from
|
||||
`source_coverage`. Encrypted directories and sparse bundles have unknown entry
|
||||
coverage; encrypted members have unknown content identity. An empty filtered
|
||||
page cannot prove absence in unknown coverage. Retained views are historical,
|
||||
not fresh filesystem checks. No decryption, Godot resource decoding, GDScript
|
||||
decompilation or target execution is provided. See
|
||||
[Godot PCK](https://github.com/morluto/rea/blob/main/docs/godot-pck.md).
|
||||
|
||||
UnityFS AssetBundle containers (`.bundle`, `.ab`, YooAsset packages and other
|
||||
byte-identified files) list `CAB-*` SerializedFiles and `.resS` members with
|
||||
header and node facts; none/LZ4/LZ4HC member blocks stream on demand. Bundles
|
||||
declaring asset-bundle encryption keep header facts with unknown directory
|
||||
coverage; LZMA member bytes retain named occurrences with unavailable hashes.
|
||||
Virtual member offsets are not physical carrier locations. Format 6–8 readers
|
||||
cache one decoded block and support retained filtered/paged views. No
|
||||
decryption or SerializedFile object decoding is provided. See
|
||||
[UnityFS](https://github.com/morluto/rea/blob/main/docs/unityfs.md).
|
||||
|
||||
U-Boot legacy uImages use the same inspection/extraction workflows. Prefer a
|
||||
retained summary and selected inventory views for header facts or component
|
||||
IDs. Header addresses/CPU/OS are declarations; payloads remain as stored.
|
||||
Multi-file components retain their aligned physical ranges; the full payload
|
||||
is also materialized. Unknown name text retains its original 32 bytes. See
|
||||
[U-Boot uImage](https://github.com/morluto/rea/blob/main/docs/uimage.md).
|
||||
@@ -1,117 +0,0 @@
|
||||
# Packages and archives
|
||||
|
||||
Read the section matching the supplied artifact; other sections are optional.
|
||||
|
||||
- [Packages and extraction](#packages-and-extraction)
|
||||
- [Godot PCK](#godot-pck)
|
||||
- [Bun standalone executables](#bun-standalone-executables)
|
||||
|
||||
## Packages and extraction
|
||||
|
||||
Use `inspect_artifact` for application bundles, archives, ZIP/APK/IPA/MSIX/AppX,
|
||||
ASAR, or DMG inputs when the artifact graph and findings help answer the
|
||||
question. It returns the complete artifact graph inline. Cite graph manifest
|
||||
IDs when using them.
|
||||
|
||||
For an IPA or a macOS `.app`, ZIP, or DMG, pass the inventory Evidence to
|
||||
`project_apple_application_graph`. It reports application roots and nested
|
||||
bundles by path convention: app extensions, XPC services, frameworks, login
|
||||
items, system and driver extensions, and plug-ins. It also lists privileged
|
||||
helpers, launchd plists, symlinks, and each bundle's `info_plist_path` and
|
||||
executable candidates. Roles are path conventions, not parsed plists; read the
|
||||
listed plists with `inspect_plist`.
|
||||
|
||||
After `open_binary` on a `.app` or Mach-O, use `trace_dylib_resolution` to see
|
||||
which file each `@rpath`, `@loader_path` and `@executable_path` load reaches for
|
||||
every executable in the bundle. It also lists missing or weak loads and earlier
|
||||
`@rpath` candidates that are absent. Narrow large bundles with `roots` or
|
||||
`architecture`. System paths stay undetermined because the dyld shared cache
|
||||
provides them.
|
||||
|
||||
Use `extract_artifact` when materialized files from a supported format are needed.
|
||||
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md).
|
||||
It materializes all regular files into a fresh temporary directory chosen by
|
||||
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
|
||||
The result reports the materialized directory; callers do not choose the
|
||||
destination. No separate permission grant is needed.
|
||||
|
||||
For classic iNES cartridges, `open_binary` recognizes the byte signature without
|
||||
an executable architecture. `inspect_artifact` retains header facts and each
|
||||
PRG/CHR file unit's offset, length and SHA-256; its nested inventory's
|
||||
`ines.reconstruction` verifies the actual ordered region bytes against the
|
||||
whole-file digest. Extraction includes the original header, optional trainer
|
||||
and uninterpreted trailing bytes. These file units are not mapper/CPU bank
|
||||
maps. NES 2.0, UNIF, dirty legacy headers and truncated payloads are explicitly
|
||||
refused. No emulator or 6502 analysis is performed. See the
|
||||
[iNES guide](https://github.com/morluto/rea/blob/main/docs/ines-containers.md).
|
||||
|
||||
Native DMG traversal automatically uses a read-only mount on supported macOS
|
||||
hosts, with an owned temporary mount directory and cleanup. It needs no approval
|
||||
flag. Unsupported hosts or mount failures remain explicit limitations; do not
|
||||
claim child inventory when only root identity is available. DMG extraction is
|
||||
unavailable through `extract_artifact`. To access DMG files on macOS, create a
|
||||
read-only mount with your own tools, select files from that mount for analysis,
|
||||
and detach the mount you created when finished. REA's inspection mount is
|
||||
temporary, not a retained export of the image's contents.
|
||||
|
||||
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
|
||||
|
||||
## Godot PCK
|
||||
|
||||
For a standalone PCK, `open_binary` then `inspect_artifact` with
|
||||
`detail: "summary"` retains the complete static inventory. Use the returned
|
||||
`parent_evidence_id` with `inspect_analysis_view` inventory occurrences filtered
|
||||
by exact `logical_paths` or `occurrence_ids`, and join `artifact_id` to nodes
|
||||
filtered by `artifact_ids`. Select only needed fields; identity links remain.
|
||||
Prefer `inspect-artifact --detail summary --evidence_output PATH` for CLI tasks;
|
||||
the same CLI discovers embedded packs via their EOF trailer without opening a
|
||||
native analysis provider. Normal native opening remains unchanged.
|
||||
|
||||
Distinguish declared MD5 from observed SHA-256 and graph/page completeness from
|
||||
`source_coverage`. Encrypted directories and sparse bundles have unknown entry
|
||||
coverage; encrypted members have unknown content identity. An empty filtered
|
||||
page cannot prove absence in unknown coverage. Retained views are historical,
|
||||
not fresh filesystem checks. No decryption, Godot resource decoding, GDScript
|
||||
decompilation or target execution is provided. See
|
||||
[Godot PCK](https://github.com/morluto/rea/blob/main/docs/godot-pck.md).
|
||||
|
||||
UnityFS AssetBundle containers (`.bundle`, `.ab`, YooAsset packages and other
|
||||
byte-identified files) list `CAB-*` SerializedFiles and `.resS` members with
|
||||
header and node facts; none/LZ4/LZ4HC member blocks stream on demand. Bundles
|
||||
declaring asset-bundle encryption keep header facts with unknown directory
|
||||
coverage; LZMA member bytes retain named occurrences with unavailable hashes.
|
||||
Virtual member offsets are not physical carrier locations. Format 6–8 readers
|
||||
cache one decoded block and support retained filtered/paged views. No
|
||||
decryption or SerializedFile object decoding is provided. See
|
||||
[UnityFS](https://github.com/morluto/rea/blob/main/docs/unityfs.md).
|
||||
|
||||
WeChat mini-program packages (`.wxapkg` files with `0xBE`/`0xED` marks) list
|
||||
members with declared names and absolute carrier offsets; member bytes are
|
||||
stored verbatim. `V1MMWX` PC-client packages report `encrypted` directory
|
||||
status with unknown coverage and are refused on extraction; no decryption is
|
||||
attempted. See
|
||||
[wxapkg](https://github.com/morluto/rea/blob/main/docs/wxapkg.md).
|
||||
|
||||
U-Boot legacy uImages use the same inspection/extraction workflows. Prefer a
|
||||
retained summary and selected inventory views for header facts or component
|
||||
IDs. Header addresses/CPU/OS are declarations; payloads remain as stored.
|
||||
Multi-file components retain their aligned physical ranges; the full payload
|
||||
is also materialized. Unknown name text retains its original 32 bytes. See
|
||||
[U-Boot uImage](https://github.com/morluto/rea/blob/main/docs/uimage.md).
|
||||
|
||||
## Bun standalone executables
|
||||
|
||||
A `bun build --compile` executable stores its module graph in Mach-O
|
||||
`__BUN,__bun`, a PE or ELF `.bun` section, or an appended trailer.
|
||||
`open_binary` then `inspect_artifact` reports root format `bun-standalone` with
|
||||
the carrier, structural layout, module count, entry point, raw flags and
|
||||
compile exec-argv; each module keeps its `/$bunfs/...` source name, loader,
|
||||
encoding, sizes and SHA-256, with sourcemap, bytecode and module-info regions as
|
||||
separate children. Use summary root metadata `entry_point_logical_path` as the
|
||||
exact inventory `logical_paths` selector, then read the selected content node by
|
||||
its artifact ID; the reader owns virtual-name normalization and indexed fallback. `extract-artifact` writes stored contents through the safe
|
||||
output tree; route extracted JavaScript to `analyze_javascript_application`.
|
||||
Layout labels describe record shapes, not a detected compiler version; unknown
|
||||
or ambiguous layouts are unsupported. Bytecode is retained, not decoded, and the
|
||||
executable is never run. See
|
||||
[Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
@@ -41,10 +41,10 @@ behavior unknown. Record truncation, unmatched breakpoints, changed target
|
||||
identity and cleanup uncertainty as limitations, rather than treating no retained
|
||||
event as proof that the behavior never occurs.
|
||||
|
||||
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/guides/runtime/frida-instrumentation.md),
|
||||
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/frida-instrumentation.md),
|
||||
[GDB/Rizin/Cutter](https://github.com/morluto/rea/blob/main/docs/reverse-engineering-tools.md),
|
||||
[process capture](https://github.com/morluto/rea/blob/main/docs/guides/runtime/process-capture.md),
|
||||
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/guides/native/native-investigation.md#native-desktop-observation).
|
||||
[process capture](https://github.com/morluto/rea/blob/main/docs/process-capture.md),
|
||||
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/native-investigation.md#native-desktop-observation).
|
||||
|
||||
## Browser and Electron
|
||||
|
||||
@@ -130,5 +130,5 @@ Inspect native `.saz` snapshots with `inspect_web_network_capture` and
|
||||
traffic analysis, connect Everywhere's separate MCP server and use its
|
||||
advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or
|
||||
`AgentCalls`). Preserve existing sessions and filters when inspecting a
|
||||
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md)
|
||||
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md)
|
||||
for the workflow and Telerik's public setup and traffic-debugging skills.
|
||||
@@ -1,12 +0,0 @@
|
||||
# Review changes to security-sensitive automation and dependency inputs.
|
||||
/.github/CODEOWNERS @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/dependabot.yml @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/workflows/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/actions/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/verify-tools/skills/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/verify-tools/skills/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/fixtures/http3-lab-requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/website/requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/third_party/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
@@ -27,7 +27,7 @@ body:
|
||||
attributes:
|
||||
label: Documentation location
|
||||
description: Link the page, section, command, API, skill, or inline comment.
|
||||
placeholder: README.md#quick-start or docs/reference/mcp-contracts.md
|
||||
placeholder: README.md#quick-start or docs/mcp-contracts.md
|
||||
validations:
|
||||
required: true
|
||||
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
name: REA production code
|
||||
|
||||
# Match the existing default-setup scan's local-input threat model.
|
||||
threat-models: [local]
|
||||
|
||||
# Retain root traversal, including future production roots. Hidden directories
|
||||
# need their own root; use a directory so Python can share this configuration.
|
||||
paths:
|
||||
- .
|
||||
- docs/.vitepress
|
||||
|
||||
# Keep the default query suite and all production roots, including bridge/,
|
||||
# install.sh, runtime scripts, release controllers, and GitHub Actions.
|
||||
# Exclude test inputs at extraction time so they cannot taint production callers.
|
||||
paths-ignore:
|
||||
- tests
|
||||
- scripts/fixtures
|
||||
- "**/*.test.*"
|
||||
- "**/*.fixture.*"
|
||||
- "**/test_*.py"
|
||||
- "**/*_test.py"
|
||||
@@ -1,33 +0,0 @@
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: npm
|
||||
directories:
|
||||
- /
|
||||
- /scripts/verify-tools/skills
|
||||
schedule:
|
||||
interval: weekly
|
||||
cooldown:
|
||||
default-days: 7
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
npm-minor-and-patch:
|
||||
update-types:
|
||||
- minor
|
||||
- patch
|
||||
|
||||
- package-ecosystem: pip
|
||||
directory: /website
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
|
||||
- package-ecosystem: github-actions
|
||||
directory: /
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
actions-minor-and-patch:
|
||||
update-types:
|
||||
- minor
|
||||
- patch
|
||||
@@ -71,8 +71,8 @@ source, link, or rendered-document checks. -->
|
||||
- User-visible CLI/MCP output (if applicable):
|
||||
- Remaining proof gaps:
|
||||
|
||||
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#boundary-regressions)
|
||||
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#lifecycle-regressions)
|
||||
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/testing.md#boundary-regressions)
|
||||
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/testing.md#lifecycle-regressions)
|
||||
covered by the evidence above, and explain relevant gaps.
|
||||
|
||||
For evidence-bearing changes:
|
||||
@@ -99,15 +99,14 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
|
||||
its new behavior and migration, and why compatibility/deprecation cannot
|
||||
preserve it. Every release increments minor, including breaking changes;
|
||||
keep breaking markers and migration notes so callers can assess compatibility.
|
||||
See docs/development/releasing.md. -->
|
||||
See docs/releasing.md. -->
|
||||
- Real Hopper/Ghidra, browser, or OS coverage:
|
||||
- Package or release metadata impact:
|
||||
- Security, privacy, process, or containment review:
|
||||
|
||||
## Author readiness
|
||||
|
||||
<!-- The human owner completes these declarations before requesting review,
|
||||
unless the PR author is exempt as listed in CONTRIBUTING.md.
|
||||
<!-- The human owner completes these declarations before requesting review.
|
||||
The PR author is the owner by default. A different owner must explicitly accept
|
||||
ownership on this PR.
|
||||
AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA.
|
||||
|
||||
@@ -61,9 +61,6 @@ jobs:
|
||||
qualification_run_id: ${{ steps.qualification.outputs.run_id }}
|
||||
code: ${{ steps.plan.outputs.code }}
|
||||
reverse_tools: ${{ steps.plan.outputs.reverse_tools }}
|
||||
python: ${{ steps.plan.outputs.python }}
|
||||
packet_capture: ${{ steps.plan.outputs.packet_capture }}
|
||||
signal_recording: ${{ steps.plan.outputs.signal_recording }}
|
||||
sqlite: ${{ steps.plan.outputs.sqlite }}
|
||||
runtime: ${{ steps.plan.outputs.runtime }}
|
||||
package: ${{ steps.plan.outputs.package }}
|
||||
@@ -147,9 +144,6 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Validate source architecture before dependency installation
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
run: node scripts/verify/architecture-guards.mjs
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
@@ -270,27 +264,6 @@ jobs:
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
python:
|
||||
name: Python producer tests
|
||||
needs: changes
|
||||
if: needs.changes.outputs.python == 'true' && needs.changes.outputs.tier == 'fast'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
# Analysis-efficiency checks import compiled process modules.
|
||||
- name: Reuse portable build outputs across runs
|
||||
uses: ./.github/actions/portable-build-cache
|
||||
- run: npm run test:prepare
|
||||
- run: npm run test:python
|
||||
|
||||
test-source:
|
||||
name: Source tests
|
||||
needs: changes
|
||||
@@ -418,9 +391,7 @@ jobs:
|
||||
needs.changes.outputs.web_network == 'true' ||
|
||||
needs.changes.outputs.web_runtime == 'true' ||
|
||||
needs.changes.outputs.web_source_map == 'true' ||
|
||||
needs.changes.outputs.agentenv == 'true' ||
|
||||
needs.changes.outputs.packet_capture == 'true' ||
|
||||
needs.changes.outputs.signal_recording == 'true'
|
||||
needs.changes.outputs.agentenv == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
@@ -779,12 +750,6 @@ jobs:
|
||||
npm run verify:legacy-simulator
|
||||
npm run verify:apple-dispatch
|
||||
npm run verify:interface-builder
|
||||
- name: Verify portable iNES CLI, MCP and source boundaries
|
||||
run: >-
|
||||
npm run test:focused --
|
||||
tests/acceptance/inesContainer.test.ts
|
||||
tests/boundary/filesystem/inesArtifact.test.ts
|
||||
tests/boundary/filesystem/inesCancellation.test.ts
|
||||
- name: Run native Apple and process boundary regressions
|
||||
run: >-
|
||||
npm run test:focused --
|
||||
@@ -805,13 +770,9 @@ jobs:
|
||||
tests/boundary/cli/nativeDemangleCli.test.ts
|
||||
tests/boundary/cli/nativeDispatchPropertyAttributes.test.ts
|
||||
tests/boundary/mcp/nativeDemangleMcp.test.ts
|
||||
tests/boundary/mcp/nativeMachoSummaryMcp.test.ts
|
||||
tests/boundary/providers/native/nativeSignatureTargetBinding.test.ts
|
||||
tests/boundary/providers/native/nativeUiOffscreenWindow.test.ts
|
||||
tests/boundary/process/nativeUiVerifierLane.test.ts
|
||||
tests/boundary/cli/nativeSignatureCli.test.ts
|
||||
tests/boundary/cli/nativeMachoArm64_32Cli.test.ts
|
||||
tests/boundary/cli/nativeMachoEncryptionCli.test.ts
|
||||
tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts
|
||||
src/process/capture/FilesystemSnapshot.test.ts
|
||||
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
|
||||
@@ -1016,8 +977,8 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- run: npm run docs:check
|
||||
- name: Verify published routes, source links and local anchors
|
||||
run: npm run verify:docs:routes
|
||||
- name: Render documentation and validate page links
|
||||
run: npm run docs:build
|
||||
- name: Generation leaves authored files unchanged
|
||||
run: git diff --exit-code
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
@@ -1130,26 +1091,6 @@ jobs:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
packet_capture:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.packet_capture == 'true'
|
||||
secrets:
|
||||
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
|
||||
uses: ./.github/workflows/real-packet-capture.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
signal_recording:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.signal_recording == 'true'
|
||||
secrets:
|
||||
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
|
||||
uses: ./.github/workflows/real-signal-recording.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
nativeaot:
|
||||
needs: [changes, static]
|
||||
if: needs.changes.outputs.nativeaot == 'true'
|
||||
@@ -1176,9 +1117,6 @@ jobs:
|
||||
static-deep,
|
||||
build,
|
||||
reverse-tools-quality,
|
||||
python,
|
||||
packet_capture,
|
||||
signal_recording,
|
||||
test-shard,
|
||||
test-source,
|
||||
sqlite-runtime,
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
name: CodeQL production
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
schedule:
|
||||
- cron: "37 5 * * 1"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: codeql-production-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
# A repository administrator must disable default setup before enabling
|
||||
# this workflow; default setup rejects advanced CodeQL result uploads.
|
||||
if: vars.REA_CODEQL_ADVANCED == 'true'
|
||||
name: Analyze production (${{ matrix.language }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write
|
||||
strategy:
|
||||
fail-fast: false
|
||||
max-parallel: 1
|
||||
matrix:
|
||||
language: [javascript-typescript, python, actions]
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Initialize production analysis
|
||||
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
build-mode: none
|
||||
config-file: .github/codeql/production.yml
|
||||
threads: 2
|
||||
ram: 4096
|
||||
- name: Analyze production code
|
||||
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
|
||||
with:
|
||||
category: /language:${{ matrix.language }}
|
||||
output: ${{ runner.temp }}/codeql-production-results
|
||||
threads: 2
|
||||
ram: 4096
|
||||
- name: Retain complete analysis evidence
|
||||
if: always()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: codeql-production-${{ matrix.language }}
|
||||
path: ${{ runner.temp }}/codeql-production-results
|
||||
if-no-files-found: error
|
||||
retention-days: 7
|
||||
@@ -62,36 +62,3 @@ jobs:
|
||||
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
|
||||
cat "${probe_root}/output.log"
|
||||
grep -F 'REA_ARM_CONTEXT_OK arm=0 thumb=1 unavailable=null (injected provider value)' "${probe_root}/output.log"
|
||||
# Each probe imports source-owned bytes at 0x1000, away from the Cortex-M
|
||||
# vector symbols that ARMCortex.pspec defines at 0, and decodes them from
|
||||
# the language's default context, so M-profile Thumb state must hold at
|
||||
# the first byte. The bytes are never executed.
|
||||
- name: Decode M-profile ARM and Xtensa bytes in their default context
|
||||
shell: bash
|
||||
run: |
|
||||
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
|
||||
export JAVA_HOME="${!java_home_variable}"
|
||||
probe_root="${RUNNER_TEMP}/rea-decode"
|
||||
mkdir -p "${probe_root}"
|
||||
# mrs r0, msp; bx lr
|
||||
printf '\357\363\010\200\160\107' > "${probe_root}/cortex.bin"
|
||||
# mrs r0, msplim (ARMv8-M only); bx lr
|
||||
printf '\357\363\012\200\160\107' > "${probe_root}/v8m.bin"
|
||||
# entry a1, 32; retw.n
|
||||
printf '\066\101\000\035\360' > "${probe_root}/xtensa.bin"
|
||||
decode() {
|
||||
local name="$1" language="$2"
|
||||
shift 2
|
||||
mkdir -p "${probe_root}/${name}-project"
|
||||
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/${name}-project" Decode \
|
||||
-import "${probe_root}/${name}.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
|
||||
-processor "${language}" -noanalysis -readOnly -deleteProject \
|
||||
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra" \
|
||||
-postScript ReaDecodeProbe.java "${language}" "$@" \
|
||||
> "${probe_root}/${name}.log" 2>&1 || { cat "${probe_root}/${name}.log"; exit 1; }
|
||||
cat "${probe_root}/${name}.log"
|
||||
grep -F "REA_DECODE_OK ${language} $*" "${probe_root}/${name}.log"
|
||||
}
|
||||
decode cortex ARM:LE:32:Cortex mrs/4/msp bx/2/lr
|
||||
decode v8m ARM:LE:32:v8-m mrs/4/msplim bx/2/lr
|
||||
decode xtensa Xtensa:LE:32:default entry/3/a1 retw.n/2
|
||||
@@ -27,4 +27,4 @@ jobs:
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- run: npm run verify:docs:routes
|
||||
- run: npm run docs:build
|
||||
@@ -54,11 +54,10 @@ jobs:
|
||||
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
# The pinned framework builds its gateway and service-db images only when pulling them from
|
||||
# ghcr.io fails, from public ECR base images; public ECR caps anonymous pulls by data per source
|
||||
# IP and hosted runners share IPs, so such a build can fail here for hours. mirror.gcr.io serves
|
||||
# the same Docker official images under the same digests, so each base image those Dockerfiles
|
||||
# name is pulled there and named as its Dockerfile names it, as the framework's own CI does.
|
||||
# Public ECR caps anonymous pulls by data per source IP and hosted runners share IPs, so the
|
||||
# framework's image builds can fail here for hours. mirror.gcr.io serves the same Docker
|
||||
# official images under the same digests, so each base image the pinned framework builds from
|
||||
# is pulled there and named as its Dockerfile names it, as the framework's own CI does.
|
||||
# Digest-pinned names are set in containerd, whose images Docker reads only on the containerd
|
||||
# image store; switching stores restarts dockerd, so it happens before anything uses Docker.
|
||||
- name: Use the containerd image store
|
||||
@@ -73,16 +72,16 @@ jobs:
|
||||
run: |
|
||||
runner_root="${RUNNER_TEMP}/rea-agentenv-runner"
|
||||
python -m venv "${runner_root}"
|
||||
"${runner_root}/bin/python" -m pip install --require-virtualenv --no-cache-dir 'agentenv-framework==0.9.1431'
|
||||
"${runner_root}/bin/python" -m pip install --require-virtualenv --no-cache-dir 'agentenv-framework==0.9.1334'
|
||||
"${runner_root}/bin/agent-env" --version
|
||||
docker version --format '{{.Server.Version}}'
|
||||
echo "REA_AGENT_ENV_COMMAND=${runner_root}/bin/agent-env" >> "${GITHUB_ENV}"
|
||||
echo "REA_DOCKER_COMMAND=$(command -v docker)" >> "${GITHUB_ENV}"
|
||||
- name: Pull the gateway and service-db public ECR base images from mirror.gcr.io
|
||||
- name: Pull the pinned framework's public ECR base images from mirror.gcr.io
|
||||
shell: bash
|
||||
run: |
|
||||
package="$("${REA_AGENT_ENV_COMMAND%/agent-env}/python" -c 'import agent_env, pathlib; print(pathlib.Path(agent_env.__file__).parent)')"
|
||||
grep -rhoE '^FROM +public\.ecr\.aws/docker/library/[^ ]+' --include='Dockerfile*' "${package}/env/gateway" "${package}/env/envs/service_db" | awk '{print $2}' | sort -u |
|
||||
grep -rhoE '^FROM +public\.ecr\.aws/docker/library/[^ ]+' --include='Dockerfile*' "${package}" | awk '{print $2}' | sort -u |
|
||||
while read -r image; do
|
||||
name=${image#public.ecr.aws/docker/library/}
|
||||
source=mirror.gcr.io/library/$name
|
||||
|
||||
@@ -11,7 +11,6 @@ on:
|
||||
- tests/conformance/android/**
|
||||
- tests/fixtures/android/static-initializers/**
|
||||
- tests/fixtures/android/direct-invokes/**
|
||||
- tests/fixtures/android/external-invokes/**
|
||||
- .nvmrc
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
@@ -45,8 +44,7 @@ jobs:
|
||||
"$JAVA_HOME/bin/javac" -J-Xmx512m -J-XX:ActiveProcessorCount=2 \
|
||||
-classpath "$jar" -d "$classes" \
|
||||
bridge/android/ReaJadxBridge.java \
|
||||
tests/conformance/android/ReaJadxMetadataRegression.java \
|
||||
tests/conformance/android/ReaJadxExternalReferencesRegression.java
|
||||
tests/conformance/android/ReaJadxMetadataRegression.java
|
||||
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
|
||||
-classpath "$classes:$jar" ReaJadxMetadataRegression \
|
||||
"$android_root/ApiDemos-debug.apk" \
|
||||
@@ -55,9 +53,3 @@ jobs:
|
||||
| tee "$android_root/metadata.log"
|
||||
grep -F 'RAW_CLASS_SELECTOR_REGRESSION_PASS nested=true metadata=true references=true' \
|
||||
"$android_root/metadata.log"
|
||||
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
|
||||
-classpath "$classes:$jar" ReaJadxExternalReferencesRegression \
|
||||
"$GITHUB_WORKSPACE/tests/fixtures/android/external-invokes" "$classes" \
|
||||
| tee "$android_root/external-references.log"
|
||||
grep -F 'EXTERNAL_REFERENCE_REGRESSION_PASS prototypes=2 using_nodes=3 sites=4' \
|
||||
"$android_root/external-references.log"
|
||||
@@ -55,12 +55,12 @@ jobs:
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Install AArch64 and ELF32 cross toolchains for source-owned fixtures
|
||||
- name: Install ELF32 cross toolchains for source-owned fixtures
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --yes --no-install-recommends \
|
||||
gcc-aarch64-linux-gnu gcc-i686-linux-gnu gcc-arm-linux-gnueabi gcc-mips-linux-gnu gcc-mipsel-linux-gnu \
|
||||
libc6-dev-arm64-cross libc6-dev-i386-cross libc6-dev-armel-cross libc6-dev-mips-cross libc6-dev-mipsel-cross
|
||||
gcc-i686-linux-gnu gcc-arm-linux-gnueabi gcc-mips-linux-gnu gcc-mipsel-linux-gnu \
|
||||
libc6-dev-i386-cross libc6-dev-armel-cross libc6-dev-mips-cross libc6-dev-mipsel-cross
|
||||
- name: Acquire exact upstream libraries in an isolated runner environment
|
||||
shell: bash
|
||||
run: |
|
||||
|
||||
@@ -53,8 +53,3 @@ jobs:
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
# The preceding verifier builds the runtime used by this request journey.
|
||||
- run: dbus-run-session -- node scripts/verify-browser-request.mjs
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
@@ -1,35 +1,39 @@
|
||||
name: Real packet capture
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/captures/**"
|
||||
- "tests/boundary/cli/captureCleanup.test.ts"
|
||||
- "tests/boundary/mcp/packetCaptureMcp.test.ts"
|
||||
- "src/captures/**"
|
||||
- "src/pcap/**"
|
||||
- "src/domain/pcap/**"
|
||||
- "src/domain/analysisView/pcapCaptureView.ts"
|
||||
- "src/application/pcap/**"
|
||||
- "src/contracts/pcap/**"
|
||||
- "src/server/registerPcapTools.ts"
|
||||
- "tests/boundary/pcap/**"
|
||||
- "src/domain/captures/**"
|
||||
- "src/application/captures/**"
|
||||
- "src/contracts/captures/**"
|
||||
- "src/composition/captures.ts"
|
||||
- "src/cli/captureCommands.ts"
|
||||
- "src/server/registerCaptureTools.ts"
|
||||
- "scripts/verify-packet-capture.mjs"
|
||||
- ".github/workflows/real-packet-capture.yml"
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
concurrency:
|
||||
group: real-packet-capture-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
tshark:
|
||||
runs-on: ubuntu-24.04
|
||||
@@ -44,18 +48,8 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run build:cached
|
||||
- name: Run focused tests and require Linux resource and cleanup cases
|
||||
run: |
|
||||
report=".cache/rea-packet-required-tests.json"
|
||||
mkdir -p .cache
|
||||
rm -f "$report"
|
||||
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
|
||||
tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts tests/acceptance/pcapVerifier.test.ts
|
||||
node scripts/ci/require-provider-tests.mjs packet "$report"
|
||||
- run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
|
||||
- run: node scripts/verify-packet-capture.mjs
|
||||
env:
|
||||
REA_TSHARK_COMMAND: /usr/bin/tshark
|
||||
|
||||
@@ -1,35 +1,32 @@
|
||||
name: Real signal recording
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/signals/**"
|
||||
- "tests/boundary/cli/signalCleanup.test.ts"
|
||||
- "tests/boundary/mcp/signalRecordingMcp.test.ts"
|
||||
- "src/signals/**"
|
||||
- "src/domain/signals/**"
|
||||
- "src/application/signals/**"
|
||||
- "src/contracts/signals/**"
|
||||
- "src/composition/signals.ts"
|
||||
- "src/cli/signalCommands.ts"
|
||||
- "src/server/registerSignalTools.ts"
|
||||
- "scripts/verify-signal-recording.mjs"
|
||||
- ".github/workflows/real-signal-recording.yml"
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
concurrency:
|
||||
group: real-signal-recording-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
sigrok:
|
||||
runs-on: ubuntu-24.04
|
||||
@@ -44,18 +41,8 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run build:cached
|
||||
- name: Run focused tests and require Linux resource and cleanup cases
|
||||
run: |
|
||||
report=".cache/rea-signal-required-tests.json"
|
||||
mkdir -p .cache
|
||||
rm -f "$report"
|
||||
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
|
||||
tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
|
||||
node scripts/ci/require-provider-tests.mjs signal "$report"
|
||||
- run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
|
||||
- run: node scripts/verify-signal-recording.mjs
|
||||
env:
|
||||
REA_SIGROK_COMMAND: /usr/bin/sigrok-cli
|
||||
@@ -21,10 +21,6 @@ jobs:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Set up Node.js for the website Worker checks
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
|
||||
@@ -54,10 +54,6 @@ jobs:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Set up Node.js for the website Worker checks
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
@@ -79,21 +75,10 @@ jobs:
|
||||
PY
|
||||
- name: Configure Pages
|
||||
uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
|
||||
# upload-pages-artifact excludes dot directories, including .well-known.
|
||||
- name: Package the website including agent discovery
|
||||
run: |
|
||||
tar --dereference --hard-dereference --directory website/_site \
|
||||
--exclude=.git --exclude=.github -cf "$RUNNER_TEMP/artifact.tar" .
|
||||
tar -tf "$RUNNER_TEMP/artifact.tar" \
|
||||
./.well-known/ard.json ./.well-known/ai-catalog.json \
|
||||
./.well-known/agent-skills/index.json > /dev/null
|
||||
- name: Upload the website
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0
|
||||
with:
|
||||
name: github-pages
|
||||
path: ${{ runner.temp }}/artifact.tar
|
||||
retention-days: 1
|
||||
if-no-files-found: error
|
||||
path: website/_site
|
||||
|
||||
deploy:
|
||||
needs: prepare
|
||||
@@ -145,24 +130,14 @@ jobs:
|
||||
PUBLISH_TARGET: ${{ inputs.target }}
|
||||
run: |
|
||||
python3 -u - <<'PY'
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from urllib.error import URLError
|
||||
from urllib.parse import urljoin, urlsplit
|
||||
from urllib.parse import urljoin
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
def open_public(url, accept=None):
|
||||
headers = {
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
}
|
||||
if accept:
|
||||
headers["Accept"] = accept
|
||||
return urlopen(Request(url, headers=headers), timeout=10)
|
||||
|
||||
expected = os.environ["GITHUB_SHA"]
|
||||
pages_url = os.environ["PAGES_URL"]
|
||||
if not pages_url:
|
||||
@@ -178,7 +153,11 @@ jobs:
|
||||
for name, base in list(pending.items()):
|
||||
url = urljoin(base, "deployment.json") + "?commit=" + expected
|
||||
try:
|
||||
with open_public(url) as response:
|
||||
request = Request(url, headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
})
|
||||
with urlopen(request, timeout=10) as response:
|
||||
actual = json.load(response).get("commit")
|
||||
if actual == expected:
|
||||
print(f"{name} serves {expected}: {base}")
|
||||
@@ -193,27 +172,6 @@ jobs:
|
||||
time.sleep(5)
|
||||
if pending:
|
||||
sys.exit("Could not verify published revision on: " + ", ".join(pending))
|
||||
for name, base in hosts.items():
|
||||
with open_public(urljoin(base, ".well-known/agent-skills/index.json")) as response:
|
||||
skill, = json.load(response)["skills"]
|
||||
# Fetch from each selected host, including the /rea/ mirror.
|
||||
artifact_url = urljoin(base, urlsplit(skill["url"]).path.lstrip("/"))
|
||||
with open_public(artifact_url) as response:
|
||||
digest = "sha256:" + hashlib.sha256(response.read()).hexdigest()
|
||||
if digest != skill["digest"]:
|
||||
sys.exit(name + " serves a skill archive with the wrong digest")
|
||||
for manifest in ("ard.json", "ai-catalog.json"):
|
||||
with open_public(urljoin(base, ".well-known/" + manifest)) as response:
|
||||
if not json.load(response).get("entries"):
|
||||
sys.exit(name + " serves an empty discovery manifest")
|
||||
if name == "Cloudflare":
|
||||
with open_public(base, accept="text/markdown") as response:
|
||||
vary = {value.strip().lower() for value in response.headers.get("Vary", "").split(",")}
|
||||
if response.headers.get_content_type() != "text/markdown" or "accept" not in vary:
|
||||
sys.exit("Cloudflare did not negotiate Markdown with Vary: Accept")
|
||||
if 'rel="ard"' not in response.headers.get("Link", ""):
|
||||
sys.exit("Cloudflare is missing its discovery Link header")
|
||||
print(name + " serves the discovery manifests and verified skill archive")
|
||||
with open(os.environ["GITHUB_STEP_SUMMARY"], "a", encoding="utf-8") as summary:
|
||||
links = " and ".join(f"[{name}]({url})" for name, url in hosts.items())
|
||||
summary.write(f"Published commit `{expected}` to {links}.\n")
|
||||
|
||||
@@ -14,13 +14,13 @@ Preserve caller-selected inputs and local evidence, including output, URLs, path
|
||||
|
||||
## Architecture and Cleanup
|
||||
|
||||
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture/architecture.mermaid](docs/architecture/architecture.mermaid) when changing composition.
|
||||
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture.mermaid](docs/architecture.mermaid) when changing composition.
|
||||
|
||||
Give each interpretation and resource a clear owner. Consumers must use that owner's parsed facts rather than reinterpret raw source. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
|
||||
|
||||
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
|
||||
|
||||
For lifecycle changes, follow [ownership and completion invariants](docs/development/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/development/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
|
||||
For lifecycle changes, follow [ownership and completion invariants](docs/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
|
||||
|
||||
Before adding complexity for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode, likelihood, and complexity cost to the user and get their buy-in. Do not silently expand scope for theoretical robustness.
|
||||
|
||||
@@ -28,10 +28,6 @@ Assess brittleness in touched code during implementation, debugging, review, and
|
||||
|
||||
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
|
||||
|
||||
After confirming a bug, search sibling adapters, consumers and similar workflows
|
||||
for the same failure. Verify suspected matches, record working comparisons, and
|
||||
group findings by root cause instead of filing duplicate reports.
|
||||
|
||||
When addressing a review thread, inspect the producing boundary and affected consumers for brittleness. Complete directly related refactoring in the same change. Before resolving the thread, explain the root cause, structural fix, validation, and any remaining cleanup.
|
||||
|
||||
Parse unknown boundary values and model expected failures with the tagged error algebra and `Result`. Preserve partial facts and meaningful failure reasons. Keep identity and lookup values separate from display formatting; preserve source values and explicit unknowns when normalization loses information. Absolute paths, file URLs, and HTTP paths have different semantics.
|
||||
@@ -68,12 +64,6 @@ do not establish fewer tokens or faster analysis.
|
||||
|
||||
## Development
|
||||
|
||||
Documentation ownership and authoring conventions live in
|
||||
[docs/development/documentation.md](docs/development/documentation.md). A feature
|
||||
uses `docs/specs/<domain>/<feature-slug>/spec.md`, one `plan.md` and `tasks.md`;
|
||||
individual architectural decisions use `docs/adr/<domain>/<decision-slug>.md`.
|
||||
Consult the [documentation directory](docs/README.md) before adding a guide.
|
||||
|
||||
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`. Prefix `PATH="/usr/local/bin:$PATH"` before `node` and `npm`.
|
||||
|
||||
- `npm ci`: install locked dependencies.
|
||||
@@ -85,24 +75,15 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
|
||||
|
||||
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
|
||||
|
||||
When changing installers or setup recovery, follow
|
||||
[Checking installers and publisher downloads](docs/development/testing.md#checking-installers-and-publisher-downloads).
|
||||
|
||||
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/development/testing.md#developer-commands).
|
||||
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/testing.md#developer-commands).
|
||||
|
||||
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
|
||||
|
||||
## Tool and Test Changes
|
||||
|
||||
For tool design, follow [docs/development/tool-design.md](docs/development/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
|
||||
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
|
||||
|
||||
For user- or agent-facing tool problems, prioritize fixing the owning behavior,
|
||||
contract, and actionable result or recovery at the point of use. Document the
|
||||
result in product docs when explanation is needed. Add skill guidance only for
|
||||
workflow judgment or context that the tool cannot reasonably convey; do not use
|
||||
a skill as the primary repair for a discoverability or contract defect.
|
||||
|
||||
For test changes or provider claims, use [docs/development/testing.md](docs/development/testing.md). Select applicable [boundary regressions](docs/development/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
|
||||
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Select applicable [boundary regressions](docs/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
|
||||
|
||||
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
|
||||
|
||||
@@ -134,8 +115,8 @@ owner to complete. Never attest to human review on their behalf. Draft status
|
||||
is optional; preserve the PR's current draft status unless the user asks to change
|
||||
it. Later pushes do not invalidate the readiness declaration or require another
|
||||
human sign-off. Keep the PR body and affected validation current as work changes.
|
||||
The readiness check validates the filled declaration or the author exemption
|
||||
listed in CONTRIBUTING.md; CI and automated review results do not establish that
|
||||
The readiness check validates
|
||||
the filled declaration; CI and automated review results do not establish that
|
||||
human review occurred, maintainer scope approval, or merge permission.
|
||||
|
||||
<!-- BEGIN:turborepo-agent-rules -->
|
||||
|
||||
+7
-7
@@ -132,8 +132,8 @@ Full contribution record: [4 commits and 1 merged PR](https://github.com/morluto
|
||||
- **Kuna analysis:** Use a caller-installed Kuna for procedure listings, pseudocode, assembly, call relationships, exact-address xrefs and cached string searches. Project exports retain source spans and digests; repeated requests reuse completed work. Recorded real-provider coverage uses Kuna 1.833 on Linux; Windows, PE and universal Mach-O are unsupported. See [the provider guide](docs/kuna-provider.md). ([#3832](https://github.com/morluto/rea/pull/3832), [#3854](https://github.com/morluto/rea/pull/3854), [#4071](https://github.com/morluto/rea/pull/4071), [#4095](https://github.com/morluto/rea/pull/4095)). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Retained JavaScript facts:** Query exact application paths, selected properties and filtered module references through `inspect_analysis_view`, with paging and complete row Evidence. These views reuse completed analysis and preserve source selection and unknown coverage. ([#4265](https://github.com/morluto/rea/pull/4265), [morluto/rea#4190](https://github.com/morluto/rea/issues/4190), [#4097](https://github.com/morluto/rea/pull/4097), [morluto/rea#4092](https://github.com/morluto/rea/issues/4092)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **APK locale tables:** Request several resource locales in one decode with repeatable CLI `--locales` or MCP `locales`. Each table reports its status and coverage; existing single-locale fields remain available. ([#4804](https://github.com/morluto/rea/pull/4804), [morluto/rea#4777](https://github.com/morluto/rea/issues/4777)). Thanks @abhay-2108, @N0zoM1z0.
|
||||
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/providers/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
|
||||
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/providers/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
|
||||
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
|
||||
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
|
||||
- **JADX deadlines:** Set `REA_JADX_OPERATION_TIMEOUT_MS` to extend APK loading and upstream operation deadlines together. Doctor reports the effective value and setup forwards an explicit selection. ([#4466](https://github.com/morluto/rea/pull/4466), [morluto/rea#3918](https://github.com/morluto/rea/issues/3918)). Thanks @jaychou813, @SandstromPL.
|
||||
- **Electron window options:** Resolve stable local constant `BrowserWindow` options while retaining source provenance. Relevant writes and escapes keep the result unknown. ([#4400](https://github.com/morluto/rea/pull/4400), [morluto/rea#4366](https://github.com/morluto/rea/issues/4366)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Language editions:** Add the Greek website edition and Malayalam documentation, with Korean typography and translation corrections across the site. ([#3520](https://github.com/morluto/rea/pull/3520), [#4792](https://github.com/morluto/rea/pull/4792), [#3821](https://github.com/morluto/rea/pull/3821), [morluto/rea#3820](https://github.com/morluto/rea/issues/3820), [#3799](https://github.com/morluto/rea/pull/3799), [morluto/rea#3798](https://github.com/morluto/rea/issues/3798), [#4344](https://github.com/morluto/rea/pull/4344), [#3856](https://github.com/morluto/rea/pull/3856), [#4166](https://github.com/morluto/rea/pull/4166)). Thanks @gensokyo-dev, @hi2242, @kalayciburak, @kaoru0822-kitauji, @kraken-503, @kukinghan, @N0zoM1z0, @yeseong0412.
|
||||
@@ -192,7 +192,7 @@ Full contribution record: [571 commits and 236 merged PRs](https://github.com/mo
|
||||
|
||||
## [6.5.0](https://github.com/morluto/rea/compare/rea-agents-6.4.0...rea-agents-6.5.0) (2026-10-13)
|
||||
|
||||
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/development/testing.md).
|
||||
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/testing.md).
|
||||
|
||||
### Highlights
|
||||
|
||||
@@ -220,7 +220,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
|
||||
- **HarmonyOS packages:** inventory HAP/HSP/App Pack structures without executing code. The recorded real workflow covers one pinned HAP through repository-built CLI/MCP; installed-package and real HSP/App Pack coverage remain unverified. ([#1447](https://github.com/morluto/rea/pull/1447)). Thanks @BaaRai.
|
||||
- **Ghidra raw inputs and references:** compatible Linux/macOS Ghidra profiles accept caller-selected raw architecture/entry information and expose typed incoming references at exact addresses. Windows Ghidra P0 admits the raw NMOS 6502 profile described below; other raw targets and the incoming-reference facet remain outside Windows P0. ([#1868](https://github.com/morluto/rea/pull/1868), [issue morluto/rea#1836](https://github.com/morluto/rea/issues/1836), [#1715](https://github.com/morluto/rea/pull/1715), [issue morluto/rea#1245](https://github.com/morluto/rea/issues/1245)). Thanks @bugstan, @casesolved-co-uk, @mvanhorn, @N0zoM1z0.
|
||||
- **Ghidra MIPS:** Add bounded MIPS32r2/o32 relocatable objects, soft-float integer-register storage, admitted MIPS16 shared ELF, and a narrowly admitted recordless legacy MIPS32 executable profile. The latter pins an endian-matched 32-bit language and records inferred o32 as an analysis assumption. Admission follows each documented ABI/flag/relocation boundary. ([#1949](https://github.com/morluto/rea/pull/1949), [issue morluto/rea#1937](https://github.com/morluto/rea/issues/1937), [#1950](https://github.com/morluto/rea/pull/1950), [issue morluto/rea#1938](https://github.com/morluto/rea/issues/1938), [#1991](https://github.com/morluto/rea/pull/1991), [issue morluto/rea#1851](https://github.com/morluto/rea/issues/1851), [#1898](https://github.com/morluto/rea/pull/1898), [issue morluto/rea#1620](https://github.com/morluto/rea/issues/1620)). Thanks @AurelioB, @BaaRai, @bqw371, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/providers/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
|
||||
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
|
||||
- **PSP static ELF profile:** the Ghidra profile is Linux x64, bounded little-endian ELF32 `ET_EXEC` with the documented PSP/ABI declarations and a compatible caller-installed Allegrex extension. PRX, shared/relocatable images, PBP/ISO/CSO and runtime emulation are outside scope. Historical extension experiments are separate from public-provider verification; real public Ghidra PSP coverage remains unverified. ([#1599](https://github.com/morluto/rea/pull/1599)). Thanks @jaelliot.
|
||||
- **Ghidra analysis settings:** apply validated analyzer overrides before analysis while retaining the selected analysis profile. ([#1913](https://github.com/morluto/rea/pull/1913)). Thanks @akushonkamen, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Attached native IDA MCP:** add an optional native bridge alongside existing legacy/headless profiles. Recorded native runs use Linux IDA Free 9.4 with a navigation correction and Windows IDA Pro 9.3 with a loopback-listener change; stock native switching, native macOS and Windows IDA Free remain unverified. Attached databases remain read-only and are never saved or closed. ([#1544](https://github.com/morluto/rea/pull/1544)). Thanks @imissapixel, @N0zoM1z0.
|
||||
@@ -292,7 +292,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
|
||||
- **Source maps:** original-source and mapping rows preserve a null source name when unavailable. Handle null explicitly instead of expecting an invented unknown-source string; new index fields remain optional when reading older Evidence. ([ee50ed1](https://github.com/morluto/rea/commit/ee50ed121e38cc6e1a2ff0dc8b789f15f71c5402)).
|
||||
- **Streamed filtered JSON:** `--filter-output` now emits JSON `null` instead of `{}` when the selected value is null; consumers must distinguish a present null from a missing selection. Use `--full-output` when an object envelope is required. ([#2333](https://github.com/morluto/rea/pull/2333)). Thanks @Jackmeson1.
|
||||
- **Call graphs:** `get_call_graph.normalized_result` is now `{ graph, limitations }`; read traversal-depth groups under `graph` and retain relationship-specific caveats. Contract/fixture validation does not establish new Ghidra or IDA engine coverage. ([5dfb9d0](https://github.com/morluto/rea/commit/5dfb9d050c2cc44a61b183dae765bef4d22a53b7)).
|
||||
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/reference/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
|
||||
- **Provider admission:** MIPS targets with unresolved GNU ABI attributes are refused. Supply supported ABI declarations. Saved Hopper databases on macOS are unavailable because the launcher cannot run the analysis bridge when opening a database; select an executable target instead. Database support requires a script-capable database launch route. ([d4bd65c](https://github.com/morluto/rea/commit/d4bd65c53c0254d0195cc7ab8da163dcadfd3b9b)).
|
||||
- **Segment extents:** `binary_overview` segment length may be null when Hopper coordinates cannot establish a finite extent in one address space. Preserve unknown rather than converting it to zero or a saturated value. ([36db19c](https://github.com/morluto/rea/commit/36db19c59)).
|
||||
@@ -372,7 +372,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/guides/runtime/process-capture.md) and [browser scenario contract](docs/guides/web/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
|
||||
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/process-capture.md) and [browser scenario contract](docs/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
|
||||
* **browser:** JSON shape paths now contain typed property and array-element segments. WebMCP input_schema_shape is replaced by input_schema and input_schema_sha256, retaining declared examples and defaults.
|
||||
* **process:** Filesystem digest omission entries now require system_code; budget and changed-file omissions use null, and unavailable-file omissions preserve the operating-system error code.
|
||||
* **contracts:** Provider analysis outputs must explicitly supply metadata previously repaired with defaults; unsupported nullable facets must be reported as null.
|
||||
@@ -540,7 +540,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
|
||||
Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
|
||||
|
||||
* **MCP / Evidence / snapshots / artifacts:** Success results are direct canonical Evidence; read `normalized_result` rather than the old `result`/`evidence` wrapper. Evidence requires nullable `analysis_profile`, session provider aliases are removed, snapshots require complete profile-bound query records, and artifact roles/permissions belong to occurrences. Recreate incompatible saved data; adding fields without recomputing their semantic identity is invalid. ([63bf914](https://github.com/morluto/rea/commit/63bf91405cf97b114a84f337fb1272d98a267999))
|
||||
* **Process captures:** Legacy v3 conversion is removed. Explicit executable identity and `event_journal` are required; retain old captures and recapture when current comparisons are needed instead of inventing launch identity or event order. ([2a77469](https://github.com/morluto/rea/commit/2a7746998d47446c4db6d4f524e0f183d6803f3f))
|
||||
|
||||
+18
-39
@@ -11,14 +11,15 @@ incorrect user instructions warrant standalone corrections. Collect other
|
||||
cosmetic-only suggestions in an issue rather than separate PRs.
|
||||
|
||||
For selected architectural owners and dependency paths, use the
|
||||
[architecture map](docs/architecture/architecture.mermaid). It is not a complete directory or
|
||||
[architecture map](docs/architecture.mermaid). It is not a complete directory or
|
||||
import inventory. For current provider wiring, start with
|
||||
[`src/composition/binary.ts`](src/composition/binary.ts) for deep candidates and
|
||||
the other `src/composition/` factories for tool services, including
|
||||
[`gpu.ts`](src/composition/gpu.ts) and [`model.ts`](src/composition/model.ts).
|
||||
Live Frida, Cutter, GDB and Rizin construction also uses shared composition
|
||||
factories. For service ownership, availability and catalog declarations, follow
|
||||
the [capability composition guide](docs/capability-composition.md).
|
||||
Live Frida and Cutter adapters are constructed directly in
|
||||
[`src/cli/fridaCommands.ts`](src/cli/fridaCommands.ts),
|
||||
[`src/cli/cutterCommands.ts`](src/cli/cutterCommands.ts), and
|
||||
[`src/server/createServer.ts`](src/server/createServer.ts).
|
||||
Production factories construct fresh providers without starting engines or
|
||||
acquiring targets; optional adapter failures must preserve successful peers. Run
|
||||
`npm run verify:test-discovery` after adding or moving tests. Resolved import
|
||||
@@ -32,7 +33,7 @@ removing it. Preserve validation, lazy loading, caller policy, and resource
|
||||
acquisition and cleanup supplied by the old boundary. Published APIs, persisted
|
||||
formats, and independently versioned bridges require an explicit compatibility
|
||||
decision; an internal representation alone does not establish that obligation.
|
||||
See [interpreting boundary facts once](docs/development/tool-design.md#interpret-boundary-facts-once)
|
||||
See [interpreting boundary facts once](docs/tool-design.md#interpret-boundary-facts-once)
|
||||
for admission ownership and the distinction between repeated validation and
|
||||
independent checks.
|
||||
|
||||
@@ -44,7 +45,7 @@ Use the [analysis optimization skill](.agents/skills/rea-analysis-optimization/S
|
||||
to locate provider and shared-workflow bottlenecks, preserve Evidence, and
|
||||
compare equivalent real tasks before claiming an improvement.
|
||||
|
||||
When adding or changing an MCP tool, follow the [tool design guide](docs/development/tool-design.md) and preserve the canonical contracts and generated catalog.
|
||||
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
|
||||
When adding or changing an admission ceiling, budget, or deadline, follow
|
||||
[analysis settings, limits, and recovery](docs/design/limits-and-recovery.md)
|
||||
for ownership, operation snapshots, provider integration, and the steps for
|
||||
@@ -93,8 +94,7 @@ Before requesting maintainer review, the human owner must:
|
||||
passing. Follow the [verification requirements](#development-feedback-and-pr-verification)
|
||||
for required checks and real-provider coverage.
|
||||
- Complete the **Author readiness** declaration in the PR template, explicitly
|
||||
stating that the PR is ready for initial review, unless the PR author is
|
||||
exempt from the declaration as listed below.
|
||||
stating that the PR is ready for initial review.
|
||||
|
||||
Keep the PR description concise and current. Explain the problem, observable
|
||||
outcome, approach, and useful evidence; link long logs or put them in optional
|
||||
@@ -115,12 +115,9 @@ validation current as work changes. AI agents must leave human declarations to
|
||||
the human owner and must not attest to a human review on their behalf.
|
||||
|
||||
The `PR readiness` check reports pending for drafts and requires a named human
|
||||
owner and all three template declarations for non-drafts. PRs authored by
|
||||
@morluto, @kaoru0822-kitauji, @N0zoM1z0, or @gensokyo-dev are exempt from
|
||||
the declaration; their human ownership, review, and evidence responsibilities
|
||||
above still apply. The check identifies authors by GitHub account ID. It checks
|
||||
that the declaration is filled, not whether the code or evidence is correct.
|
||||
Existing PRs can copy the **Author readiness** section from the current template.
|
||||
owner and all three template declarations for non-drafts. It checks that the
|
||||
declaration is filled, not whether the code or evidence is correct. Existing PRs
|
||||
can copy the **Author readiness** section from the current template.
|
||||
Open PRs must have distinct head commits because GitHub shares commit statuses.
|
||||
|
||||
Maintainers enable merge enforcement after the workflow is merged: run
|
||||
@@ -234,9 +231,7 @@ guarantee.
|
||||
Use the development toolchain pinned in `.nvmrc` and
|
||||
`package.json#packageManager`. Run `nvm use` before installing dependencies;
|
||||
installed-package runtime support is documented in
|
||||
[installation](docs/getting-started/installation.md#start-setup).
|
||||
Use Node.js and npm for source builds and verifier scripts; the installed
|
||||
package's Bun runtime support does not extend to those development scripts.
|
||||
[installation](docs/installation.md#start-setup).
|
||||
|
||||
```bash
|
||||
npm ci
|
||||
@@ -251,15 +246,6 @@ catalogs, documentation projections, or managed verification evidence. Turbo
|
||||
caches deterministic builds and static checks across Git worktrees. After a package, lockfile, or managed-skill version change, run
|
||||
`npm run metadata:generate` before building.
|
||||
|
||||
Dependabot proposes updates for the npm lockfiles, website Python requirements,
|
||||
and GitHub Actions. Review dependency changes and run the checks relevant to
|
||||
their consumers. For dependencies maintained outside Dependabot, including
|
||||
`third_party/` and the HTTP/3 lab's pinned requirements, the maintainer should
|
||||
review upstream security advisories when updating or using them. Preserve source
|
||||
and version provenance, update license notices when needed, and report which
|
||||
verification was actually repeated. The lab requirements describe a recorded
|
||||
environment; a dependency update does not verify the revised environment.
|
||||
|
||||
Repository invariants live in [AGENTS.md](AGENTS.md). Keep provider-specific
|
||||
protocols in adapters and shared CLI/MCP workflows in the application layer.
|
||||
|
||||
@@ -286,23 +272,16 @@ The general source suite (`npm run test:fast`, `npm run test:changed`, and
|
||||
On a Windows checkout, some source tests fail because fixtures use POSIX paths,
|
||||
`#!/bin/sh` tool stubs, or POSIX rename semantics. Compare failures against the
|
||||
base revision, or run the suite on Linux or WSL, before attributing them to a
|
||||
change. The embedded-artifact parent-swap tests exercise the selected-path
|
||||
replacement when the rename succeeds; if Windows refuses it while the admitted
|
||||
file is open, they instead verify the refusal and cleanup. See
|
||||
[testing](docs/development/testing.md#developer-commands) for command scopes
|
||||
change. See [testing](docs/testing.md#developer-commands) for command scopes
|
||||
and lane selection.
|
||||
|
||||
`git submodule update --init --recursive` fails at the pinned Unblob submodule
|
||||
on Windows and can leave later submodules unchecked out. Apply the
|
||||
[Unblob checkout workaround](docs/guides/firmware/firmware-analysis.md#windows-submodule-checkout),
|
||||
[Unblob checkout workaround](docs/firmware-analysis.md#windows-submodule-checkout),
|
||||
then rerun the command.
|
||||
|
||||
## Documentation website
|
||||
|
||||
Use the [documentation directory](docs/README.md) and
|
||||
[authoring guide](docs/development/documentation.md) for document ownership,
|
||||
domain-based feature specifications and architectural decision records.
|
||||
|
||||
The VitePress site uses the Markdown files in `docs/`. Run `npm run docs:dev`
|
||||
for live editing, `npm run docs:build` to check the production build and
|
||||
links, and `npm run docs:preview` to preview that build at `/rea/`.
|
||||
@@ -356,7 +335,7 @@ npm run check:fast
|
||||
`test:local` runs source tests without building. `check:changed` combines cached
|
||||
static checks with source tests affected since the merge base with `origin/main`.
|
||||
Import-graph selection can miss runtime registration, bridges, and generated data;
|
||||
select relevant boundary tests explicitly. See [testing](docs/development/testing.md#developer-commands)
|
||||
select relevant boundary tests explicitly. See [testing](docs/testing.md#developer-commands)
|
||||
for commands, test placement, pruning, and real-provider prerequisites.
|
||||
|
||||
Before requesting review, run relevant tests and `npm run check`; add
|
||||
@@ -431,8 +410,8 @@ Run `npm run verify:hopper` on a supported macOS host or
|
||||
`npm run verify:hopper:linux` on the supported Linux demo runner. These build
|
||||
native fixtures and verify the installed engine through CLI/MCP; package or
|
||||
simulated-provider checks do not establish engine behavior. See
|
||||
[real-toolchain verification](docs/development/testing.md#end-to-end-integration-and-golden-evidence)
|
||||
for prerequisites and scope, and [Hopper setup](docs/getting-started/installation.md#hopper)
|
||||
[real-toolchain verification](docs/testing.md#end-to-end-integration-and-golden-evidence)
|
||||
for prerequisites and scope, and [Hopper setup](docs/installation.md#hopper)
|
||||
for supported hosts and unattended startup.
|
||||
|
||||
Use the default `build/conformance/manifest.json` unless explicitly verifying
|
||||
@@ -441,7 +420,7 @@ Generated fixtures and manifests remain ignored.
|
||||
|
||||
## Maintainer release checklist
|
||||
|
||||
Use the [release guide](docs/development/releasing.md). Main pushes automatically refresh a
|
||||
Use the [release guide](docs/releasing.md). Main pushes automatically refresh a
|
||||
Release Please PR with the next version and changelog. Review its migration
|
||||
notes, wait for the current CI checks to pass, and merge it into main to publish.
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
**See a feature you like. Understand how it works, down to the binary level.**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](docs/reference/mcp-contracts.md#generated-catalog)
|
||||
[](docs/mcp-contracts.md#generated-catalog)
|
||||
[](#what-you-can-analyze)
|
||||
[](https://skills.sh/morluto/rea/reverse-engineer-anything)
|
||||
[](LICENSE)
|
||||
@@ -20,8 +20,6 @@
|
||||
|
||||
**[Website](https://rea.tools/) · [Guides](https://rea.tools/guides/) · [Showcases](https://rea.tools/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Join the REA community on Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [Build RL environments](#build-rl-environments) · [FAQ](#faq) · [Documentation](#documentation)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
@@ -64,7 +62,7 @@ See a feature in an app that you want in your own product? Ask your agent to inv
|
||||
|
||||
REA connects your agent to tools for inspecting native binaries, JavaScript and Electron apps, .NET assemblies, and websites. You can also use the same tools from your terminal. Analysis runs locally, and results include the evidence and limitations behind each conclusion.
|
||||
|
||||
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, IDA or Kuna installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
|
||||
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, or IDA installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
|
||||
|
||||
> **[Visit the REA website](https://rea.tools/)** for setup instructions, illustrated guides, and real case studies.
|
||||
|
||||
@@ -72,7 +70,7 @@ Setup registers REA with your agent and installs matching workflow instructions.
|
||||
|
||||
On Linux, Ghidra, JADX, firmware tool launches and static JavaScript/Electron
|
||||
analysis require a procps-compatible `ps` on REA's `PATH` for process ownership
|
||||
checks. Recovery steps are in `docs/getting-started/installation.md` under
|
||||
checks. Recovery steps are in `docs/installation.md` under
|
||||
"Linux owned-process prerequisite".
|
||||
|
||||
### Set up your agent
|
||||
@@ -91,8 +89,8 @@ available. Run package acquisition from a trusted directory outside the target
|
||||
project. Installation and recovery details are in the setup guide below.
|
||||
|
||||
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
|
||||
[other agents](docs/getting-started/installation.md#supported-agents). See
|
||||
[installation and setup](docs/getting-started/installation.md) for provider configuration and
|
||||
[other agents](docs/installation.md#supported-agents). See
|
||||
[installation and setup](docs/installation.md) for provider configuration and
|
||||
manual MCP registration and Claude Code/Codex plugin installation.
|
||||
|
||||
MCP uses compact input schemas by default to reduce context overhead. Set
|
||||
@@ -135,7 +133,7 @@ rea --help
|
||||
```
|
||||
|
||||
For native analysis, configure a provider first. See the
|
||||
[CLI and Evidence guide](docs/getting-started/cli.md) for native commands, provider selection,
|
||||
[CLI and Evidence guide](docs/cli.md) for native commands, provider selection,
|
||||
snapshots and scripting.
|
||||
|
||||
### Update REA
|
||||
@@ -191,45 +189,44 @@ CLI commands use the same workflows.
|
||||
REA requires Node.js 22.x (>=22.23.4), 24.x (>=24.22.1), or 26.12.1+, plus npm.
|
||||
Additional tools and host support depend on the target:
|
||||
|
||||
| Target | What REA returns | Requirements and guide |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/guides/native/native-investigation.md) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/guides/native/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
|
||||
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
|
||||
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
|
||||
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
|
||||
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
|
||||
| Binary hardening | PIE, NX, RELRO, CFG and related header flags for ELF, PE and Mach-O | Built-in static header inspection; [binary hardening](docs/binary-hardening.md) |
|
||||
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
|
||||
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/guides/evm/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/guides/native/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/guides/javascript/javascript-application-workflows.md) |
|
||||
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/guides/web/browser-observation.md) |
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/guides/web/web-network-captures.md) |
|
||||
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/guides/managed/managed-code-analysis.md) |
|
||||
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/guides/android/android-analysis.md) |
|
||||
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
|
||||
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/guides/android/adb-device-analysis.md) |
|
||||
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/providers/jeb-analysis.md) |
|
||||
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/guides/android/apktool-resource-analysis.md) |
|
||||
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/guides/android/flutter-analysis.md) |
|
||||
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/guides/firmware/firmware-analysis.md) |
|
||||
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/guides/data/packet-captures.md) |
|
||||
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/guides/data/pcap-analysis.md) |
|
||||
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/guides/signals/signal-recordings.md) |
|
||||
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/guides/javascript/javascript-artifact-reconstruction.md), [Apple applications](docs/guides/apple/apple-application-analysis.md) |
|
||||
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/guides/runtime/process-capture.md) |
|
||||
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
|
||||
| Target | What REA returns | Requirements and guide |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
|
||||
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
|
||||
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
|
||||
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
|
||||
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
|
||||
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
|
||||
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
|
||||
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
|
||||
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
|
||||
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
|
||||
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
|
||||
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
|
||||
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
|
||||
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
|
||||
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
|
||||
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
|
||||
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
|
||||
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
|
||||
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
|
||||
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
|
||||
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
|
||||
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
|
||||
|
||||
Static JavaScript and .NET inspection read the supplied files without running
|
||||
the application. Runtime capture runs or interacts with the selected target
|
||||
@@ -243,15 +240,15 @@ or update either provider.
|
||||
<a id="choosing-a-deep-analysis-provider"></a>
|
||||
|
||||
Native formats and host support vary by provider. See
|
||||
[Hopper and Ghidra setup](docs/getting-started/installation.md#hopper), the
|
||||
[IDA guide](docs/providers/ida-provider.md) (including an optional native MCP bridge), the
|
||||
[Hopper and Ghidra setup](docs/installation.md#hopper), the
|
||||
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), the
|
||||
[Kuna decompiler guide](docs/kuna-provider.md), and
|
||||
[experimental Windows Ghidra support](docs/providers/windows-ghidra-p0.md).
|
||||
Ghidra also supports [16-bit DOS analysis](docs/providers/ghidra-dos.md).
|
||||
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
|
||||
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
|
||||
For large binaries, raise the provider's startup deadline with
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS` or `REA_HOPPER_STARTUP_TIMEOUT_MS`.
|
||||
For provider selection, see the [CLI guide](docs/getting-started/cli.md#choose-a-provider).
|
||||
Check [release availability](docs/getting-started/installation.md#released-package-and-main)
|
||||
For provider selection, see the [CLI guide](docs/cli.md#choose-a-provider).
|
||||
Check [release availability](docs/installation.md#released-package-and-main)
|
||||
for features added since the latest npm release.
|
||||
|
||||
## Showcases
|
||||
@@ -341,19 +338,17 @@ REA exports a runnable environment with a basic test to check that it starts and
|
||||
<summary><strong>Which agents can use REA?</strong></summary>
|
||||
|
||||
Any agent that supports local MCP servers. Setup configures the
|
||||
[supported agents](docs/getting-started/installation.md#supported-agents); other clients can use
|
||||
[manual MCP registration](docs/getting-started/installation.md#mcp-registry).
|
||||
[supported agents](docs/installation.md#supported-agents); other clients can use
|
||||
[manual MCP registration](docs/installation.md#mcp-registry).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Do I need a native analysis engine?</strong></summary>
|
||||
<summary><strong>Do I need Hopper, Ghidra or IDA?</strong></summary>
|
||||
|
||||
Deep native analysis uses Hopper, Ghidra, IDA or Kuna, depending on the target
|
||||
and operation. Static JavaScript and .NET inspection work without a native
|
||||
analysis engine. Setup can install Hopper after approval; Ghidra, IDA and Kuna
|
||||
use your existing installations. See [provider setup](docs/getting-started/installation.md#hopper)
|
||||
for the options and their supported targets.
|
||||
Deep native analysis uses one of them. Static JavaScript and .NET inspection
|
||||
work without a native analysis engine. Setup can install Hopper after approval;
|
||||
Ghidra and IDA use your existing installations. See [provider setup](docs/installation.md#hopper).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -364,7 +359,7 @@ REA starts Hopper when an operation needs it. On macOS, a first-run dialog may
|
||||
ask you to choose demo mode or activate your license. Complete that prompt in the
|
||||
same user session before your first CLI/MCP operation or unattended job. Open
|
||||
Hopper manually if needed. See
|
||||
[Hopper startup and troubleshooting](docs/getting-started/installation.md#launcher-paths-and-troubleshooting).
|
||||
[Hopper startup and troubleshooting](docs/installation.md#launcher-paths-and-troubleshooting).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -373,7 +368,7 @@ Hopper manually if needed. See
|
||||
|
||||
The skill supplies investigation instructions for your agent. Use `npx rea-agents setup`
|
||||
to register REA's MCP server and install the matching instructions, then restart
|
||||
your agent. See [skill-only installation](docs/getting-started/installation.md#skill-only-installation).
|
||||
your agent. See [skill-only installation](docs/installation.md#skill-only-installation).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -422,12 +417,12 @@ with your REA version, target type, steps to reproduce and error output.
|
||||
Start with the website's [worked guides](https://rea.tools/guides/).
|
||||
For exact options, prerequisites and result contracts:
|
||||
|
||||
- [Installation and setup](docs/getting-started/installation.md): agent registration, provider configuration, updates and uninstall.
|
||||
- [Readiness and troubleshooting](docs/getting-started/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
|
||||
- [CLI and Evidence](docs/getting-started/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
|
||||
- [MCP contracts](docs/reference/mcp-contracts.md) and [agent prompts](docs/getting-started/mcp-prompts.md): tool results, sessions and guided investigations.
|
||||
- [Tool catalog](docs/reference/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
|
||||
- [Roadmap](docs/development/roadmap.md): planned work and capability trackers.
|
||||
- [Installation and setup](docs/installation.md): agent registration, provider configuration, updates and uninstall.
|
||||
- [Readiness and troubleshooting](docs/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
|
||||
- [CLI and Evidence](docs/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
|
||||
- [MCP contracts](docs/mcp-contracts.md) and [agent prompts](docs/mcp-prompts.md): tool results, sessions and guided investigations.
|
||||
- [Tool catalog](docs/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
|
||||
- [Roadmap](docs/roadmap.md): planned work and capability trackers.
|
||||
|
||||
Report vulnerabilities through [SECURITY.md](SECURITY.md).
|
||||
|
||||
@@ -462,8 +457,8 @@ report a bug or suggest a feature, or [send a pull request](https://github.com/m
|
||||
to improve the code or docs.
|
||||
|
||||
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
|
||||
[testing](docs/development/testing.md) for verification lanes, and the
|
||||
[architecture map](docs/architecture/architecture.mermaid) for selected layers and dependency paths.
|
||||
[testing](docs/testing.md) for verification lanes, and the
|
||||
[architecture map](docs/architecture.mermaid) for selected layers and dependency paths.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
+1
-1
@@ -27,7 +27,7 @@ provider-dependent behavior to be reproduced safely.
|
||||
REA authenticates its Hopper and Ghidra bridges with per-session random
|
||||
capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses
|
||||
authenticated loopback TCP with a private NTFS runtime whose DACL admits only
|
||||
the current user and SYSTEM. See [Windows Ghidra P0](docs/providers/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
|
||||
the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
|
||||
Tokens are transferred through private session descriptors rather than process
|
||||
arguments or environment variables.
|
||||
This is not a sandbox and does not protect against malicious processes already
|
||||
|
||||
+21
-22
@@ -17,7 +17,6 @@ notices supplied with the software you actually obtain or redistribute.
|
||||
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| npm dependencies | [Complete inventory](licenses/npm-dependencies.txt) of every entry in `package-lock.json`, including transitive, optional platform, development, and website dependencies. [Runtime notices](licenses/npm-runtime-notices.txt) preserve the copyright, license, and NOTICE documents from integrity-verified release archives, plus the MIT notice for Rolldown's generated runtime helpers. Development dependencies are inventoried but are not all distributed in REA's npm package. |
|
||||
| pefile ordinal tables | Import-hash ordinal names are derived from [pefile 2024.8.26](https://github.com/erocarrera/pefile/tree/4b3b1e2e568a88d4f1897d694d684f23d9e270c4), copyright 2004–2024 Ero Carrera. Preserve the [MIT license](licenses/pefile-2024.8.26-LICENSE.txt). The Python package is not a runtime dependency. |
|
||||
| UF2 family names | UF2 family IDs and short names are copied from [microsoft/uf2 `utils/uf2families.json`](https://github.com/microsoft/uf2/blob/cf3bd7f9c274b949528f8cd6a19eef2502a687d2/utils/uf2families.json), copyright Microsoft Corporation. Preserve the [MIT license](licenses/microsoft-uf2-MIT.txt). |
|
||||
| NativeAOT parser | REA's derived parser uses the MIT-licensed [ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot/tree/effeb734fc570c32650f88b159608979dc7b423e) work of Washi and the NativeAOT layout definitions in [dotnet/runtime](https://github.com/dotnet/runtime/tree/a3b2d40328be0be3f8dd43f0e5cc925b8827b044). Preserve [Washi's MIT notice](licenses/nativeaot-parser-MIT.txt) and the [.NET Foundation MIT notice](licenses/dotnet-runtime-MIT.txt). |
|
||||
| EVM analysis | `[email protected]`, MIT, copyright 2023 Maxim Andreev. The published npm archive omits its license document; REA retains the [original upstream license](third_party/evmole/LICENSE) and [provenance](third_party/evmole/README.md). It is also included in the runtime notices. |
|
||||
| Windows native addon | The release addon is compiled with MinGW-w64 and statically links GCC runtime libraries. Preserve the [MinGW-w64 runtime notices](licenses/MinGW-w64-runtime.txt), [GPLv3 text](licenses/GPL-3.0.txt), and [GCC Runtime Library Exception 3.1](licenses/GCC-Runtime-Library-Exception-3.1.txt). The exception governs eligible compiled combinations; it does not relicense the runtime itself under MIT. The Node-API build headers are from Node 22.19.0; its [original license and component notices](licenses/Node-22.19.0-LICENSE.txt) are retained. Node itself is supplied by the user. Snapshot provenance and release-toolchain review requirements are in [licenses/README.md](licenses/README.md). |
|
||||
@@ -54,27 +53,27 @@ REA's adapters do not grant, replace, or waive these licenses. The upstream
|
||||
executables and their dependency graphs are supplied separately. Git submodules
|
||||
pin inspection sources; they are not included in the npm package.
|
||||
|
||||
| Project | License and source | REA relationship |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/getting-started/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
|
||||
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
|
||||
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
|
||||
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
|
||||
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
|
||||
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
|
||||
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
|
||||
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
|
||||
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
|
||||
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
|
||||
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
|
||||
| Project | License and source | REA relationship |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.833/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.833/NOTICE) | Bring your own `kuna`; REA verified 1.833 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
|
||||
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
|
||||
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
|
||||
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
|
||||
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
|
||||
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
|
||||
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
|
||||
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
|
||||
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
|
||||
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
|
||||
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
|
||||
|
||||
Ghidra's original notice credits software developed at the National Security
|
||||
Agency. It explains that US-government-created portions are not subject to US
|
||||
|
||||
@@ -1,262 +0,0 @@
|
||||
"""Run REA inside an agentenv build computer for REA on the host: capture the app, draft its task and export it.
|
||||
|
||||
REA sends this file's source through the computer's ``computer-exec/v1`` route as
|
||||
``python3 -I -c <source> <command> <JSON arguments>``; each command prints one JSON line and exits. ``start``
|
||||
leaves the capture running on its own: its stdin is a FIFO it holds open for reading and writing, so it never
|
||||
sees the end of its input, and ``mark`` and ``stop`` write their requests there and wait for the capture's own
|
||||
acknowledgement on its progress log. A small shell around the capture writes its exit status to a file, the
|
||||
mark a later command reads. REA runs as the session user in the home, with no ``AGENTENV_*`` variable, so
|
||||
neither REA nor the app it launches holds the computer's control token, and with ``APT_CONFIG`` naming the
|
||||
package lists ``start`` downloads as that user, so the capture finds the package's sources.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
POLL_S = 0.2
|
||||
TAIL_BYTES = 4096
|
||||
|
||||
|
||||
def _say(value: dict) -> int:
|
||||
print(json.dumps(value), flush=True)
|
||||
return 0
|
||||
|
||||
|
||||
def _tail(path: str) -> str:
|
||||
try:
|
||||
with open(path, "rb") as handle:
|
||||
handle.seek(max(0, os.path.getsize(path) - TAIL_BYTES))
|
||||
return handle.read().decode("utf-8", "replace")
|
||||
except OSError:
|
||||
return ""
|
||||
|
||||
|
||||
def _environment(work: str) -> dict:
|
||||
environment = {name: value for name, value in os.environ.items() if not name.startswith("AGENTENV_")}
|
||||
environment["APT_CONFIG"] = os.path.join(work, "apt", "apt.conf")
|
||||
return environment
|
||||
|
||||
|
||||
def _apt(work: str, timeout: float) -> dict:
|
||||
"""Download the package lists into the work folder as the session user, until one update succeeds."""
|
||||
root = os.path.join(work, "apt")
|
||||
lists = os.path.join(root, "lists")
|
||||
updated = os.path.join(root, "updated")
|
||||
os.makedirs(os.path.join(lists, "partial"), exist_ok=True)
|
||||
os.makedirs(os.path.join(root, "cache", "archives", "partial"), exist_ok=True)
|
||||
with open(os.path.join(root, "apt.conf"), "w", encoding="utf-8") as conf:
|
||||
conf.write(
|
||||
f'Dir::State::Lists "{lists}";\n'
|
||||
f'Dir::Cache "{root}/cache";\n'
|
||||
'Debug::NoLocking "true";\n'
|
||||
'APT::Update::Error-Mode "any";\n'
|
||||
)
|
||||
# apt keeps lists compressed as it fetched them, such as *_Packages.lz4 on Ubuntu.
|
||||
count = lambda: len([name for name in os.listdir(lists) if re.search(r"_Packages(\.\w+)?$", name)]) # noqa: E731
|
||||
# A marker left by an update that fetched nothing is not a success; download again.
|
||||
if os.path.exists(updated) and count() > 0:
|
||||
return {"lists": count(), "updated": False}
|
||||
log = os.path.join(root, "update.log")
|
||||
try:
|
||||
with open(log, "wb") as output:
|
||||
code = subprocess.run(["apt-get", "update"], env=_environment(work), stdin=subprocess.DEVNULL,
|
||||
stdout=output, stderr=subprocess.STDOUT, timeout=timeout).returncode
|
||||
except (OSError, subprocess.TimeoutExpired) as error:
|
||||
return {"lists": count(), "updated": True, "error": f"{type(error).__name__}: {error}"}
|
||||
lists_fetched = count()
|
||||
if code == 0 and lists_fetched > 0:
|
||||
# Only a whole update that fetched lists is kept, so a failed or empty one runs again.
|
||||
open(updated, "w").close()
|
||||
return {"lists": lists_fetched, "updated": True, "exit_code": code}
|
||||
if code == 0:
|
||||
return {"lists": lists_fetched, "updated": True, "exit_code": code,
|
||||
"error": "apt-get update exited 0 without a package list"}
|
||||
return {"lists": lists_fetched, "updated": True, "exit_code": code,
|
||||
"error": f"apt-get update exited {code}: {_tail(log)[-1500:].strip()}"}
|
||||
|
||||
|
||||
def _exit_code(capture: str):
|
||||
try:
|
||||
with open(os.path.join(capture, "exit"), encoding="utf-8") as handle:
|
||||
return int(handle.read().strip() or "-1")
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _event_line(path: str, offset: int, kinds: set, name):
|
||||
"""The first progress line after ``offset`` whose event is of one of ``kinds``, for ``name`` when given."""
|
||||
try:
|
||||
with open(path, "rb") as handle:
|
||||
handle.seek(offset)
|
||||
lines = handle.read().decode("utf-8", "replace").splitlines()
|
||||
except OSError:
|
||||
return None
|
||||
for line in lines:
|
||||
try:
|
||||
event = json.loads(line)["rea_progress"]["event"]
|
||||
except (ValueError, KeyError, TypeError):
|
||||
continue
|
||||
if isinstance(event, dict) and event.get("kind") in kinds and (name is None or event.get("name") == name):
|
||||
return line
|
||||
return None
|
||||
|
||||
|
||||
def _wait(capture: str, offset: int, kinds: set, name, timeout: float) -> dict:
|
||||
"""Wait for the capture's event, or for it to exit, or for time to run out."""
|
||||
log = os.path.join(capture, "progress.log")
|
||||
deadline = time.monotonic() + timeout
|
||||
while True:
|
||||
line = _event_line(log, offset, kinds, name)
|
||||
if line is not None:
|
||||
return {"line": line}
|
||||
code = _exit_code(capture)
|
||||
if code is not None:
|
||||
return {"exited": code, "progress": _tail(log), "output": _tail(os.path.join(capture, "capture.out"))}
|
||||
if time.monotonic() > deadline:
|
||||
return {"timed_out": True, "progress": _tail(log)}
|
||||
time.sleep(POLL_S)
|
||||
|
||||
|
||||
def start(arguments: dict) -> int:
|
||||
work, capture = os.path.abspath(arguments["work"]), os.path.abspath(arguments["capture"])
|
||||
rea = shutil.which("rea")
|
||||
if rea is None:
|
||||
return _say({"error": "REA is not installed in this computer; start the computer from a request, which installs it."})
|
||||
if os.path.exists(capture):
|
||||
return _say({"error": f"the capture folder {capture} is already used"})
|
||||
apt = _apt(work, arguments.get("apt_timeout_s", 600))
|
||||
try:
|
||||
os.makedirs(capture)
|
||||
except FileExistsError:
|
||||
return _say({"error": f"a stop claimed {capture} before the capture launched", "apt": apt})
|
||||
control = os.path.join(capture, "control")
|
||||
os.mkfifo(control, 0o600)
|
||||
argv = [rea, "capture-desktop-application", "--desktop-entry", arguments["desktop_entry"],
|
||||
"--output", os.path.join(capture, "capture"), "--exclude", work, "--json"]
|
||||
for value in arguments.get("arguments", []):
|
||||
argv += ["--argument", value]
|
||||
if arguments.get("package_file"):
|
||||
argv += ["--package-file", arguments["package_file"]]
|
||||
# Reading and writing, so the capture never reads the end of its input between requests.
|
||||
reader = os.open(control, os.O_RDWR)
|
||||
try:
|
||||
with open(os.path.join(capture, "capture.out"), "wb") as out, \
|
||||
open(os.path.join(capture, "progress.log"), "wb") as progress:
|
||||
process = subprocess.Popen(
|
||||
["sh", "-c", 'done=$1; shift; "$@"; code=$?; printf "%s" "$code" > "$done.tmp" && mv "$done.tmp" "$done"',
|
||||
"sh", os.path.join(capture, "exit"), *argv],
|
||||
cwd=os.path.expanduser("~"), env=_environment(work), stdin=reader, stdout=out, stderr=progress,
|
||||
start_new_session=True)
|
||||
finally:
|
||||
os.close(reader)
|
||||
waited = _wait(capture, 0, {"window_open"}, None, arguments.get("timeout_s", 120))
|
||||
return _say({"pid": process.pid, "apt": apt, **waited})
|
||||
|
||||
|
||||
def _request(capture: str, line: str) -> bool:
|
||||
"""Write one request to the running capture; False when nothing reads its FIFO any more."""
|
||||
try:
|
||||
fd = os.open(os.path.join(capture, "control"), os.O_WRONLY | os.O_NONBLOCK)
|
||||
except OSError:
|
||||
return False
|
||||
try:
|
||||
os.write(fd, (line + "\n").encode("utf-8"))
|
||||
finally:
|
||||
os.close(fd)
|
||||
return True
|
||||
|
||||
|
||||
def mark(arguments: dict) -> int:
|
||||
capture, name = os.path.abspath(arguments["capture"]), arguments["name"]
|
||||
code = _exit_code(capture)
|
||||
log = os.path.join(capture, "progress.log")
|
||||
if code is not None:
|
||||
return _say({"exited": code, "progress": _tail(log)})
|
||||
# A checkpoint already taken, whose answer was lost, is answered from the capture's own log.
|
||||
taken = _event_line(log, 0, {"checkpoint"}, name)
|
||||
if taken is not None:
|
||||
return _say({"line": taken})
|
||||
offset = os.path.getsize(log)
|
||||
if not _request(capture, f"checkpoint {name}"):
|
||||
return _say({"error": "the capture is not reading its requests"})
|
||||
return _say(_wait(capture, offset, {"checkpoint", "checkpoint_refused"}, name, arguments.get("timeout_s", 600)))
|
||||
|
||||
|
||||
def stop(arguments: dict) -> int:
|
||||
capture = os.path.abspath(arguments["capture"])
|
||||
if _exit_code(capture) is None and not os.path.exists(os.path.join(capture, "control")):
|
||||
# Never launched: claim the folder, so a start still on its way finds it used and launches nothing.
|
||||
os.makedirs(capture, exist_ok=True)
|
||||
return _say({"absent": True})
|
||||
if _exit_code(capture) is None:
|
||||
_request(capture, "stop")
|
||||
deadline = time.monotonic() + arguments.get("timeout_s", 600)
|
||||
while (code := _exit_code(capture)) is None and time.monotonic() < deadline:
|
||||
time.sleep(POLL_S)
|
||||
log = os.path.join(capture, "progress.log")
|
||||
if code is None:
|
||||
return _say({"timed_out": True, "progress": _tail(log)})
|
||||
output = os.path.join(capture, "capture.out")
|
||||
return _say({"exit_code": code, "progress": _tail(log), "output_bytes": os.path.getsize(output)})
|
||||
|
||||
|
||||
def run(arguments: dict) -> int:
|
||||
"""Run one REA command with its output in a file, after writing the files it reads."""
|
||||
rea = shutil.which("rea")
|
||||
if rea is None:
|
||||
return _say({"error": "REA is not installed in this computer"})
|
||||
for path, text in arguments.get("files", {}).items():
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "w", encoding="utf-8") as handle:
|
||||
handle.write(text)
|
||||
with open(arguments["stdout"], "wb") as out:
|
||||
try:
|
||||
done = subprocess.run([rea, *arguments["argv"]], cwd=os.path.expanduser("~"),
|
||||
env=_environment(os.path.abspath(arguments["work"])), stdin=subprocess.DEVNULL, stdout=out,
|
||||
stderr=subprocess.PIPE, timeout=arguments.get("timeout_s", 1800))
|
||||
except subprocess.TimeoutExpired:
|
||||
return _say({"timed_out": True})
|
||||
return _say({"exit_code": done.returncode, "stdout_bytes": os.path.getsize(arguments["stdout"]),
|
||||
"stderr": done.stderr[-TAIL_BYTES:].decode("utf-8", "replace")})
|
||||
|
||||
|
||||
def listing(arguments: dict) -> int:
|
||||
"""Every regular file under a folder, by its path relative to the home, with its size and SHA-256.
|
||||
|
||||
The list goes to the file ``output`` names, outside the folder, and the answer gives its size: a large
|
||||
bundle's list outgrows the 1 MiB of stdout a command answers with, so REA reads it through ``data/get``.
|
||||
"""
|
||||
home = os.path.expanduser("~")
|
||||
files, skipped = [], []
|
||||
for directory, folders, names in os.walk(os.path.abspath(arguments["root"])):
|
||||
folders.sort()
|
||||
for name in sorted(names):
|
||||
path = os.path.join(directory, name)
|
||||
relative = os.path.relpath(path, home)
|
||||
if os.path.islink(path) or not os.path.isfile(path):
|
||||
skipped.append(relative)
|
||||
continue
|
||||
digest = hashlib.sha256()
|
||||
with open(path, "rb") as handle:
|
||||
for chunk in iter(lambda: handle.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
files.append({"path": relative, "size": os.path.getsize(path), "sha256": digest.hexdigest()})
|
||||
output = os.path.abspath(arguments["output"])
|
||||
os.makedirs(os.path.dirname(output), exist_ok=True)
|
||||
with open(output, "w", encoding="utf-8") as handle:
|
||||
json.dump({"files": files, "skipped": skipped}, handle)
|
||||
return _say({"listing_bytes": os.path.getsize(output)})
|
||||
|
||||
|
||||
COMMANDS = {"start": start, "mark": mark, "stop": stop, "run": run, "list": listing}
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(COMMANDS[sys.argv[1]](json.loads(sys.argv[2])))
|
||||
@@ -1,83 +0,0 @@
|
||||
"""Keep one agentenv computer up for REA until stdin closes or SIGTERM arrives.
|
||||
|
||||
REA runs ``computer.py <bundle folder> [task]`` with the agentenv runner's own interpreter, so ``agent_env``
|
||||
imports. It deploys the bundle's task, ``smoke`` by default, on this machine with
|
||||
``run_bundle(keep=True, emulate=False)``, prints one JSON line naming the computer's address, MCP URL, control
|
||||
token and sandbox id, or ``{"error": ...}``, and tears the computer down once its stdin closes or it gets SIGTERM
|
||||
or SIGINT. Everything else it or agentenv prints, a build's output included, goes to stderr. It exits 0 once
|
||||
the teardown left nothing up, 1 when it did, and 2 when no computer came up.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import threading
|
||||
|
||||
|
||||
def _protocol():
|
||||
"""A private copy of stdout for the JSON line; fd 1 then points at stderr, as do children that inherit it."""
|
||||
protocol = os.fdopen(os.dup(1), "w", encoding="utf-8")
|
||||
os.dup2(2, 1)
|
||||
sys.stdout = sys.stderr
|
||||
return protocol
|
||||
|
||||
|
||||
def _say(protocol, value: dict) -> None:
|
||||
protocol.write(json.dumps(value) + "\n")
|
||||
protocol.flush()
|
||||
|
||||
|
||||
def _computer(result):
|
||||
"""The run's deployed computer, or why there is none."""
|
||||
run = result.runs[0] if result.runs else None
|
||||
if run is None:
|
||||
return None, "the bundle ran no task"
|
||||
if run.error is not None or run.failed_steps:
|
||||
return None, f"the task failed: {run.error or run.failed_steps}"
|
||||
for env in run.context.deployed_envs:
|
||||
if env.environment_url and env.mcp_url and getattr(env, "sandbox_id", None):
|
||||
return env, None
|
||||
return None, "the task deployed no computer"
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
protocol = _protocol()
|
||||
bundle, task = argv[1], argv[2] if len(argv) > 2 else "smoke"
|
||||
try:
|
||||
from agent_env.bundle import run_bundle
|
||||
except ImportError as error:
|
||||
_say(protocol, {"error": f"agent_env does not import with {sys.executable}: {error}"})
|
||||
return 2
|
||||
try:
|
||||
result = run_bundle(bundle, tasks=[task], keep=True, emulate=False, sandbox="local",
|
||||
on_progress=lambda line: print(line, file=sys.stderr, flush=True))
|
||||
except BaseException as error: # noqa: BLE001 - a signal's RunInterrupted too; run_bundle tore its runs down
|
||||
_say(protocol, {"error": f"{type(error).__name__}: {error}"})
|
||||
return 2
|
||||
stop = threading.Event()
|
||||
for signum in (signal.SIGTERM, signal.SIGINT):
|
||||
signal.signal(signum, lambda *_: stop.set())
|
||||
env, problem = _computer(result)
|
||||
try:
|
||||
if env is None:
|
||||
_say(protocol, {"error": problem})
|
||||
else:
|
||||
_say(protocol, {"base_url": env.environment_url, "mcp_url": env.mcp_url,
|
||||
"control_token": env.control_token, "sandbox_id": env.sandbox_id})
|
||||
threading.Thread(target=lambda: (sys.stdin.buffer.read(), stop.set()), daemon=True).start()
|
||||
while not stop.wait(0.5):
|
||||
pass
|
||||
finally:
|
||||
torn_down = result.teardown()
|
||||
still_up = [sandbox for run in torn_down.runs if run.torn_down is not None for sandbox in run.torn_down.still_up]
|
||||
if still_up:
|
||||
print(f"still up after the teardown: {still_up}", file=sys.stderr, flush=True)
|
||||
return 1
|
||||
return 0 if env is not None else 2
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -15,9 +15,7 @@ import jadx.api.plugins.input.insns.InsnIndexType;
|
||||
import jadx.api.plugins.input.insns.Opcode;
|
||||
import jadx.core.codegen.TypeGen;
|
||||
import jadx.core.dex.info.MethodInfo;
|
||||
import jadx.core.dex.instructions.args.ArgType;
|
||||
import jadx.core.dex.nodes.*;
|
||||
import jadx.core.xmlgen.BinaryXMLParser;
|
||||
import java.io.PrintStream;
|
||||
import java.io.StringReader;
|
||||
import java.io.InputStream;
|
||||
@@ -52,25 +50,17 @@ class ReaJadxBridge {
|
||||
private Map<String, List<ClassSnapshot>> classes = Map.of();
|
||||
private JadxSession manifestSession;
|
||||
private ManifestSnapshot manifestSnapshot;
|
||||
private ExternalInventory externalInventory;
|
||||
|
||||
private record MethodSnapshot(String name, String rawName, String shortId, List<String> bodyAbsenceMarkers,
|
||||
Map<String, Object> summary) {}
|
||||
private record ClassSnapshot(JavaClass node, List<MethodSnapshot> methods, Map<String, Object> summary,
|
||||
Map<String, Object> summaryWithValues) {}
|
||||
private record ManifestSnapshot(String text, Map<String, Object> summary) {}
|
||||
private record ExternalMethod(MethodInfo info, Map<String, Object> summary) {}
|
||||
private record ExternalInventory(JadxSession session, String className, String methodName,
|
||||
List<ExternalMethod> methods) {}
|
||||
|
||||
private static final class InputFailure extends IllegalArgumentException {
|
||||
final String field;
|
||||
InputFailure(String field, String message) { super(message); this.field = field; }
|
||||
}
|
||||
private static final class ReferenceLimit extends IllegalArgumentException {
|
||||
final String stage;
|
||||
ReferenceLimit(String stage) { super("External reference representation limit"); this.stage = stage; }
|
||||
}
|
||||
|
||||
private ReaJadxBridge(SessionHolder holder) { this.holder = holder; }
|
||||
|
||||
@@ -197,7 +187,6 @@ class ReaJadxBridge {
|
||||
private void clearManifest() {
|
||||
manifestSession = null;
|
||||
manifestSnapshot = null;
|
||||
externalInventory = null;
|
||||
}
|
||||
|
||||
// One bounded DEX at a time, beside the already admitted analysis JVM. The
|
||||
@@ -411,7 +400,10 @@ class ReaJadxBridge {
|
||||
selected = resource;
|
||||
}
|
||||
if (selected == null) throw new IllegalArgumentException("Root AndroidManifest.xml not found");
|
||||
String xml = manifestText(selected);
|
||||
var content = selected.loadContent();
|
||||
if (content == null || content.getDataType() != jadx.core.xmlgen.ResContainer.DataType.TEXT)
|
||||
throw new IllegalArgumentException("Root AndroidManifest.xml did not decode as text");
|
||||
String xml = content.getText().getCodeStr();
|
||||
ManifestSnapshot snapshot = new ManifestSnapshot(xml, manifestSummary(xml));
|
||||
manifestSnapshot = snapshot;
|
||||
manifestSession = current;
|
||||
@@ -419,23 +411,6 @@ class ReaJadxBridge {
|
||||
return manifestSnapshot;
|
||||
}
|
||||
|
||||
/** Decode with resource names but without JADX's class alias substitution. */
|
||||
static String manifestText(ResourceFile resource) {
|
||||
var decompiler = resource.getDecompiler();
|
||||
// BinaryXMLParser resolves android:name against its root's class identities
|
||||
// and replaces raw '$' and relative names with Java-style aliases. A fresh
|
||||
// root owns no classes, while its resource-name table preserves reference
|
||||
// rendering from the loaded target.
|
||||
RootNode root = new RootNode(decompiler.getArgs());
|
||||
root.getConstValues().setResourcesNames(decompiler.getRoot().getConstValues().getResourcesNames());
|
||||
try {
|
||||
return ResourcesLoader.decodeStream(resource,
|
||||
(size, input) -> new BinaryXMLParser(root).parse(input).getCodeStr());
|
||||
} catch (jadx.core.utils.exceptions.JadxException failure) {
|
||||
throw new IllegalArgumentException("Root AndroidManifest.xml did not decode as text", failure);
|
||||
}
|
||||
}
|
||||
|
||||
/** One parsed XML owner supplies namespace-aware values alongside the exact decoded text. */
|
||||
static Map<String, Object> manifestSummary(String xml) {
|
||||
Element root;
|
||||
@@ -479,7 +454,7 @@ class ReaJadxBridge {
|
||||
return Collections.unmodifiableMap(summary);
|
||||
}
|
||||
|
||||
private void indexClasses() {
|
||||
private ClassSnapshot selected(JsonObject args) {
|
||||
JadxSession current = session();
|
||||
if (indexedSession != current) {
|
||||
Map<String, List<ClassSnapshot>> index = new LinkedHashMap<>();
|
||||
@@ -555,10 +530,6 @@ class ReaJadxBridge {
|
||||
classes = Collections.unmodifiableMap(index);
|
||||
indexedSession = current;
|
||||
}
|
||||
}
|
||||
|
||||
private ClassSnapshot selected(JsonObject args) {
|
||||
indexClasses();
|
||||
String name = string(args, "class_name");
|
||||
List<ClassSnapshot> candidates = classes.get(name);
|
||||
if (candidates == null) throw new InputFailure("class_name", "Exact class not found: " + name
|
||||
@@ -658,18 +629,12 @@ class ReaJadxBridge {
|
||||
// JSON-RPC/CallToolResult envelope and generated numeric request id, and
|
||||
// account for JSON text being escaped again inside TextContent.
|
||||
private static final int MAX_XREF_TEXT_BYTES = 8 * 1024 * 1024 - 1024;
|
||||
// Leave room for the prototype reply, reference reply, retained raw calls,
|
||||
// normalized Evidence and their text/structured MCP copies, not just one frame.
|
||||
private static final int MAX_EXTERNAL_XREF_BYTES = MAX_XREF_TEXT_BYTES / 8;
|
||||
private static int escapedJsonBytes(Object value) {
|
||||
return JSON.toJson(JSON.toJson(value)).getBytes(StandardCharsets.UTF_8).length;
|
||||
}
|
||||
private static final class InvokeBudget {
|
||||
long remaining;
|
||||
InvokeBudget(long remaining) { this.remaining = Math.max(0, remaining); }
|
||||
boolean fitsText(long characters) {
|
||||
return remaining >= 256 && characters <= (remaining - 256) / 12;
|
||||
}
|
||||
boolean admit(Map<String, Object> site) {
|
||||
// Check a conservative two-layer escape bound before serialization.
|
||||
long bound = 256;
|
||||
@@ -684,55 +649,26 @@ class ReaJadxBridge {
|
||||
}
|
||||
private record DirectInvokes(List<Map<String, Object>> sites, boolean exhaustive) {}
|
||||
|
||||
@FunctionalInterface private interface InvokeVisitor {
|
||||
void accept(MethodInfo target, int offset, int index, Opcode opcode);
|
||||
}
|
||||
|
||||
/** Decode raw invoke identity once at the shared bridge boundary, without generating source. */
|
||||
private static void visitInvokes(MethodNode node, InvokeVisitor visitor) {
|
||||
checkNativeInterrupted();
|
||||
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
|
||||
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
|
||||
MethodNode node = source.getMethodNode();
|
||||
ICodeReader code = node.getCodeReader();
|
||||
if (code == null) return;
|
||||
if (code == null) return null;
|
||||
List<Map<String, Object>> sites = new ArrayList<>();
|
||||
boolean[] exhaustive = {true};
|
||||
code.visitInstructions(insn -> {
|
||||
checkNativeInterrupted();
|
||||
insn.decode();
|
||||
if (insn.getIndexType() != InsnIndexType.METHOD_REF) return;
|
||||
Opcode opcode = insn.getOpcode();
|
||||
if (!opcode.name().startsWith("INVOKE_")) return;
|
||||
IMethodRef ref = insn.getIndexAsMethod();
|
||||
ref.load();
|
||||
visitor.accept(MethodInfo.fromRef(node.root(), ref), insn.getOffset(), insn.getIndex(), opcode);
|
||||
});
|
||||
}
|
||||
|
||||
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
|
||||
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
|
||||
return directInvokes(source.getMethodNode(), target, budget);
|
||||
}
|
||||
|
||||
private static void visitDirectInvokes(MethodNode node, MethodInfo target, InvokeVisitor visitor) {
|
||||
visitInvokes(node, (referenced, offset, index, opcode) -> {
|
||||
if (referenced.getRawFullId().equals(target.getRawFullId()))
|
||||
visitor.accept(referenced, offset, index, opcode);
|
||||
});
|
||||
}
|
||||
|
||||
private static boolean referencesMethod(MethodNode node, MethodInfo target) {
|
||||
boolean[] found = {false};
|
||||
visitDirectInvokes(node, target, (referenced, offset, index, opcode) -> found[0] = true);
|
||||
return found[0];
|
||||
}
|
||||
|
||||
private static DirectInvokes directInvokes(MethodNode node, MethodInfo target, InvokeBudget budget) {
|
||||
if (node.getCodeReader() == null) return null;
|
||||
List<Map<String, Object>> sites = new ArrayList<>();
|
||||
boolean[] exhaustive = {true};
|
||||
visitDirectInvokes(node, target, (referenced, offset, index, opcode) -> {
|
||||
if (!MethodInfo.fromRef(node.root(), ref).getRawFullId().equals(target.getRawFullId())) return;
|
||||
Map<String, Object> site = Map.of(
|
||||
"dex_entry", node.getInputFileName(),
|
||||
"source_raw_method_id", node.getMethodInfo().getRawFullId(),
|
||||
"dex_code_unit_offset", offset,
|
||||
"dex_method_index", index,
|
||||
"dex_code_unit_offset", insn.getOffset(),
|
||||
"dex_method_index", insn.getIndex(),
|
||||
"opcode", opcode.name());
|
||||
if (budget.admit(site)) sites.add(site);
|
||||
else exhaustive[0] = false;
|
||||
@@ -740,143 +676,6 @@ class ReaJadxBridge {
|
||||
return new DirectInvokes(sites, exhaustive[0]);
|
||||
}
|
||||
|
||||
private List<ClassSnapshot> sourceClasses() {
|
||||
indexClasses();
|
||||
Set<ClassSnapshot> seen = Collections.newSetFromMap(new IdentityHashMap<>());
|
||||
List<ClassSnapshot> result = new ArrayList<>();
|
||||
for (List<ClassSnapshot> group : classes.values())
|
||||
for (ClassSnapshot source : group) if (seen.add(source)) result.add(source);
|
||||
result.sort(Comparator.comparing(source -> source.node().getRawName()));
|
||||
return result;
|
||||
}
|
||||
|
||||
private static long displayTypeCharacters(ArgType type) {
|
||||
if (type.isArray())
|
||||
return 2L * type.getArrayDimension() + displayTypeCharacters(type.getArrayRootElement());
|
||||
if (type.isPrimitive()) return type.getPrimitiveType().getLongName().length();
|
||||
if (type.isObject() && !type.isGeneric() && !type.isGenericType()) return type.getObject().length();
|
||||
throw new IllegalArgumentException("External prototype contains a non-DEX type");
|
||||
}
|
||||
|
||||
private static Map<String, Object> externalMethodSummary(MethodInfo info, InvokeBudget budget) {
|
||||
String name = info.getName();
|
||||
long signatureCharacters = name.length() + 4L + displayTypeCharacters(info.getReturnType())
|
||||
+ 2L * Math.max(0, info.getArgsCount() - 1);
|
||||
for (ArgType type : info.getArgumentsTypes()) signatureCharacters += displayTypeCharacters(type);
|
||||
long descriptorCharacters = info.getShortId().length() - (long) name.length();
|
||||
// Use parsed type facts and the existing short-id length before copying/joining strings.
|
||||
if (!budget.fitsText(2L * name.length() + descriptorCharacters + signatureCharacters))
|
||||
throw new ReferenceLimit("method_inventory");
|
||||
String descriptor = info.getShortId().substring(name.length());
|
||||
String signature = name + "(" + info.getArgumentsTypes().stream()
|
||||
.map(Object::toString).collect(Collectors.joining(", "))
|
||||
+ "): " + info.getReturnType();
|
||||
Map<String, Object> summary = Map.of("name", name, "raw_name", name, "signature", signature,
|
||||
"dex_descriptor", descriptor, "is_constructor", info.isConstructor(), "def_pos", 0);
|
||||
if (!budget.admit(summary)) throw new ReferenceLimit("method_inventory");
|
||||
return summary;
|
||||
}
|
||||
|
||||
/** Only directly referenced prototypes, not invented definitions or an external classpath. */
|
||||
private List<ExternalMethod> externalMethods(JsonObject args) {
|
||||
String className = string(args, "class_name");
|
||||
String methodName = string(args, "method_name");
|
||||
JadxSession current = session();
|
||||
if (externalInventory != null && externalInventory.session() == current
|
||||
&& externalInventory.className().equals(className) && externalInventory.methodName().equals(methodName))
|
||||
return externalInventory.methods();
|
||||
// Retain only the latest bounded selection, never results from a replaced APK/session.
|
||||
externalInventory = null;
|
||||
SortedMap<String, ExternalMethod> methods = new TreeMap<>();
|
||||
boolean[] observedClass = {false};
|
||||
InvokeBudget budget = new InvokeBudget(MAX_EXTERNAL_XREF_BYTES
|
||||
- 4096L - 12L * (className.length() + (long) methodName.length()));
|
||||
for (ClassSnapshot source : sourceClasses()) {
|
||||
for (MethodSnapshot selected : source.methods()) {
|
||||
visitInvokes(method(source, selected).getMethodNode(), (info, offset, index, opcode) -> {
|
||||
if (!info.getDeclClass().getRawName().equals(className)) return;
|
||||
observedClass[0] = true;
|
||||
if (!info.getName().equals(methodName) || methods.containsKey(info.getShortId())) return;
|
||||
methods.put(info.getShortId(), new ExternalMethod(info, externalMethodSummary(info, budget)));
|
||||
});
|
||||
}
|
||||
}
|
||||
if (!observedClass[0]) throw new InputFailure("class_name",
|
||||
"No APK definition or direct DEX invoke references the exact class: " + className);
|
||||
if (methods.isEmpty()) throw new InputFailure("method_name",
|
||||
"No direct DEX invoke references " + className + "." + methodName);
|
||||
List<ExternalMethod> result = List.copyOf(methods.values());
|
||||
externalInventory = new ExternalInventory(current, className, methodName, result);
|
||||
return result;
|
||||
}
|
||||
|
||||
private Map<String, Object> externalSummary(JsonObject args) {
|
||||
List<ExternalMethod> methods = externalMethods(args);
|
||||
return Map.of("class_name", string(args, "class_name"), "reference_target", "external",
|
||||
"observed_method_count", methods.size(),
|
||||
"prototypes", methods.stream().map(ExternalMethod::summary).toList());
|
||||
}
|
||||
|
||||
private record ExternalCaller(MethodNode node, Map<String, Object> row) {}
|
||||
|
||||
/** Reserve the complete using-node inventory before optional instruction positions. */
|
||||
private List<ExternalCaller> externalCallers(MethodInfo target, InvokeBudget budget) {
|
||||
List<ExternalCaller> callers = new ArrayList<>();
|
||||
for (ClassSnapshot source : sourceClasses()) {
|
||||
String className = String.valueOf(source.summary().get("full_name"));
|
||||
Map<String, Integer> overloads = new HashMap<>();
|
||||
for (MethodSnapshot candidate : source.methods()) {
|
||||
int overload = overloads.merge(candidate.name(), 1, Integer::sum) - 1;
|
||||
MethodNode node = method(source, candidate).getMethodNode();
|
||||
if (!referencesMethod(node, target)) continue;
|
||||
if (classes.get(className).size() != 1)
|
||||
throw new IllegalArgumentException("External caller has an ambiguous APK class identity: " + className);
|
||||
Map<String, Object> row = new LinkedHashMap<>(Map.of(
|
||||
"kind", "method", "name", candidate.name(), "full_name", className + "." + candidate.name(),
|
||||
"containing_class", className, "top_class", source.node().getTopParentClass().getFullName(),
|
||||
"def_pos", 0, "source_selector", Map.of("class_name", className,
|
||||
"method_name", candidate.name(), "overload_index", overload),
|
||||
"direct_invokes", List.of(), "direct_invokes_exhaustive", false));
|
||||
if (!budget.admit(row)) throw new ReferenceLimit("caller_inventory");
|
||||
callers.add(new ExternalCaller(node, row));
|
||||
}
|
||||
}
|
||||
return List.copyOf(callers);
|
||||
}
|
||||
|
||||
private Map<String, Object> externalReferences(JsonObject args) {
|
||||
List<ExternalMethod> methods = externalMethods(args);
|
||||
if (methods.size() != 1 && !args.has("overload_index"))
|
||||
throw new InputFailure("overload_index", "Select one of " + methods.size() + " observed external prototypes");
|
||||
int index = integer(args, "overload_index", 0);
|
||||
if (index >= methods.size()) throw new InputFailure("overload_index", "External prototype index out of range: " + index);
|
||||
ExternalMethod method = methods.get(index);
|
||||
MethodInfo target = method.info();
|
||||
Map<String, Object> selected = method.summary();
|
||||
InvokeBudget budget = new InvokeBudget(MAX_EXTERNAL_XREF_BYTES
|
||||
- 4096L - escapedJsonBytes(selected));
|
||||
List<ExternalCaller> callers = externalCallers(target, budget);
|
||||
List<Map<String, Object>> items = new ArrayList<>();
|
||||
for (ExternalCaller caller : callers) {
|
||||
DirectInvokes invokes = directInvokes(caller.node(), target, budget);
|
||||
caller.row().put("direct_invokes", invokes.sites());
|
||||
caller.row().put("direct_invokes_exhaustive", invokes.exhaustive());
|
||||
items.add(caller.row());
|
||||
}
|
||||
Map<String, Object> result = new LinkedHashMap<>();
|
||||
result.put("target", string(args, "class_name") + "." + target.getName());
|
||||
result.put("reference_target", "external");
|
||||
result.put("reference_scope", "direct_dex_invokes");
|
||||
result.put("dex_descriptor", selected.get("dex_descriptor"));
|
||||
result.put("reported_signature", selected.get("signature"));
|
||||
result.put("overload_index", index);
|
||||
result.put("total", items.size());
|
||||
result.put("count", items.size());
|
||||
result.put("resolve_line", false);
|
||||
result.put("items", items);
|
||||
return result;
|
||||
}
|
||||
|
||||
private CallToolResult handle(String name, kotlinx.serialization.json.JsonObject input) {
|
||||
JsonObject args = JsonParser.parseString(input.toString()).getAsJsonObject();
|
||||
if (name.equals("rea_jvm_status")) return json(Map.of(
|
||||
@@ -913,12 +712,6 @@ class ReaJadxBridge {
|
||||
"scope", "top_level", "offset", offset, "limit", limit,
|
||||
"items", items));
|
||||
}
|
||||
indexClasses();
|
||||
if (flag(args, "reference_target") && !classes.containsKey(string(args, "class_name"))) {
|
||||
if (name.equals("get_class_summary")) return json(externalSummary(args));
|
||||
if (name.equals("get_xrefs_to_method")) return json(externalReferences(args));
|
||||
throw new InputFailure("class_name", "External prototypes are only available to method reference queries");
|
||||
}
|
||||
ClassSnapshot cls = selected(args);
|
||||
if (name.equals("get_class_summary"))
|
||||
return json(flag(args, "dex_static_values") ? cls.summaryWithValues() : cls.summary());
|
||||
@@ -992,9 +785,6 @@ class ReaJadxBridge {
|
||||
try { return bridge.handle(name, request.getArguments()); }
|
||||
catch (InputFailure error) { return text(JSON.toJson(Map.of(
|
||||
"kind", "invalid-input", "field", error.field, "message", error.getMessage())), true); }
|
||||
catch (ReferenceLimit error) { return text(JSON.toJson(Map.of(
|
||||
"kind", "reference-output-limit", "stage", error.stage,
|
||||
"limit_bytes", MAX_EXTERNAL_XREF_BYTES)), true); }
|
||||
catch (IllegalArgumentException error) { return text(error.getMessage(), true); }
|
||||
});
|
||||
}
|
||||
@@ -1007,7 +797,7 @@ class ReaJadxBridge {
|
||||
SessionHolder holder = new SessionHolder(defaults);
|
||||
ServerCapabilities capabilities = new ServerCapabilities(
|
||||
new ServerCapabilities.Tools(null), null, null, null, null, null, null, Map.of());
|
||||
Server server = new Server(new Implementation("rea-jadx-bridge", "10", null, null, null),
|
||||
Server server = new Server(new Implementation("rea-jadx-bridge", "9", null, null, null),
|
||||
new ServerOptions(capabilities), "", ignored -> Unit.INSTANCE);
|
||||
SessionToolsKt.registerSessionTools(server, holder);
|
||||
ClassToolsKt.registerClassTools(server, holder);
|
||||
@@ -1057,12 +847,6 @@ class ReaJadxBridge {
|
||||
if (name.equals("get_class_summary")) properties.put("dex_static_values",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("boolean"))));
|
||||
if (name.equals("get_class_summary") || name.equals("get_xrefs_to_method"))
|
||||
properties.put("reference_target", new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("boolean"))));
|
||||
if (name.equals("get_class_summary")) properties.put("method_name",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))));
|
||||
String description = switch (name) {
|
||||
case "list_classes" -> "Pages of case-sensitive matching top-level class names; offset and next_offset are inventory positions, limit bounds matching rows. Omitted or empty query inventories all names.";
|
||||
case "get_class_summary" -> "Parsed member metadata for one exact class name; no source generation. Set dex_static_values to include encoded static final field values.";
|
||||
|
||||
+126
-461
@@ -40,7 +40,6 @@ import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.TreeMap;
|
||||
import java.util.TreeSet;
|
||||
import java.util.regex.Pattern;
|
||||
import java.util.regex.PatternSyntaxException;
|
||||
|
||||
@@ -179,17 +178,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private List<FunctionEntry> procedureInventory;
|
||||
private List<InventoryItem> stringInventory;
|
||||
private DecompInterface decompiler;
|
||||
private boolean existingProject;
|
||||
// A refused Program may only report its admission failure to the client.
|
||||
private String identityAdmissionFailure;
|
||||
/** Per-function native decompiler limit from the session; 0 leaves it unbounded. */
|
||||
private int decompileTimeoutSeconds;
|
||||
// DecompileProcess converts seconds to an int millisecond timer delay.
|
||||
private static final int MAX_DECOMPILE_TIMEOUT_SECONDS = Integer.MAX_VALUE / 1000;
|
||||
private final ReaGhidraExtensions analysisExtensions = new ReaGhidraExtensions();
|
||||
private static AddressSpace sessionDefaultAddressSpace;
|
||||
private Path checkpointDirectory;
|
||||
private long checkpointModification;
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
@@ -198,9 +192,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
throw new IllegalArgumentException("REA bridge requires one session descriptor");
|
||||
}
|
||||
Path descriptorPath = Path.of(arguments[0]);
|
||||
checkpointDirectory = descriptorPath.toAbsolutePath().normalize().getParent();
|
||||
SessionDescriptor descriptor = readDescriptor(descriptorPath);
|
||||
existingProject = descriptor.existingProject;
|
||||
decompileTimeoutSeconds = descriptor.decompileTimeoutSeconds();
|
||||
// Windows retains a native immutable bearer descriptor lease until
|
||||
// runtime_close; POSIX removes it after this read. Normal cleanup removes
|
||||
@@ -231,17 +223,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
throw new IllegalStateException("Ghidra provider version does not match the session");
|
||||
}
|
||||
String importedSha256 = currentProgram.getExecutableSHA256();
|
||||
if (importedSha256 == null || !importedSha256.matches("(?i)[0-9a-f]{64}")) {
|
||||
identityAdmissionFailure = "import_identity_unavailable";
|
||||
}
|
||||
else if (!constantTimeEquals(importedSha256.toLowerCase(Locale.ROOT),
|
||||
descriptor.targetSha256)) {
|
||||
identityAdmissionFailure = "import_identity_mismatch";
|
||||
}
|
||||
if (identityAdmissionFailure != null) {
|
||||
end(false);
|
||||
serve(descriptor);
|
||||
return;
|
||||
if (importedSha256 == null ||
|
||||
!constantTimeEquals(
|
||||
importedSha256.toLowerCase(Locale.ROOT),
|
||||
descriptor.targetSha256
|
||||
)) {
|
||||
throw new IllegalStateException(
|
||||
"Ghidra imported-byte digest does not match the admitted target"
|
||||
);
|
||||
}
|
||||
admittedTargetSha256 = descriptor.targetSha256;
|
||||
// GhidraScript wraps run() in a transaction. End it before serving so
|
||||
@@ -252,7 +241,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
throw new IOException("Windows Ghidra P0 does not admit metadata recovery or database mutation.");
|
||||
analysisExtensions.analyze(descriptor.analysisExtensions, currentProgram, monitor);
|
||||
initializeDecompiler();
|
||||
checkpointModification = currentProgram.getModificationNumber();
|
||||
serve(descriptor);
|
||||
}
|
||||
finally {
|
||||
@@ -416,14 +404,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
writeSuccess(writer, request.id, result);
|
||||
return;
|
||||
}
|
||||
if (identityAdmissionFailure != null) {
|
||||
// Keep the producer alive until the client has consumed the
|
||||
// refusal and closed or sent shutdown. Every other request
|
||||
// remains refused, including further startup attempts.
|
||||
writeFailure(writer, request.id, identityAdmissionFailure,
|
||||
"Ghidra imported identity admission failed");
|
||||
continue;
|
||||
}
|
||||
decompileTimeoutSeconds = request.decompileTimeoutSeconds() == null
|
||||
? descriptor.decompileTimeoutSeconds() : request.decompileTimeoutSeconds();
|
||||
writeSuccess(writer, request.id, handleRequest(request, descriptor));
|
||||
@@ -457,7 +437,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (descriptor.existingProject && isMutatingOperation(request.method))
|
||||
throw new RequestFailure("method_unavailable", "Existing Ghidra project sessions are read-only");
|
||||
return switch (request.method) {
|
||||
case "checkpoint_program" -> checkpointProgram(request.params, descriptor);
|
||||
case "read_binary_inventory_summary" -> binaryInventorySummary(request.params);
|
||||
case "inspect_native_load_image" -> inspectNativeLoadImage(request.params);
|
||||
case "read_bytes" -> readMemoryBytes(request.params);
|
||||
@@ -513,31 +492,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return method.equals("annotate_native_function") || method.equals("set_address_name") || method.equals("set_addresses_names");
|
||||
}
|
||||
|
||||
/** Internal lifecycle request: the destination is inside this owned runtime. */
|
||||
private JsonObject checkpointProgram(JsonObject params, SessionDescriptor descriptor) throws Exception {
|
||||
requireKeys(params, Set.of("path", "force"));
|
||||
if (descriptor.existingProject || !descriptor.transport.equals("unix-socket"))
|
||||
throw new RequestFailure("method_unavailable", "This Program has no checkpoint authority");
|
||||
Path output = Path.of(requireString(params, "path")).toAbsolutePath().normalize();
|
||||
if (!output.getParent().equals(checkpointDirectory) ||
|
||||
!output.getFileName().toString().matches("checkpoint-[a-f0-9-]{36}\\.gzf"))
|
||||
throw new RequestFailure("invalid_request", "Checkpoint destination must belong to this runtime");
|
||||
JsonElement forced = params.get("force");
|
||||
if (!forced.isJsonPrimitive() || !forced.getAsJsonPrimitive().isBoolean())
|
||||
throw new RequestFailure("invalid_request", "Checkpoint force must be a boolean");
|
||||
boolean force = forced.getAsBoolean();
|
||||
JsonObject result = new JsonObject();
|
||||
boolean changed = currentProgram.getModificationNumber() != checkpointModification;
|
||||
result.addProperty("saved", force || changed);
|
||||
if (force || changed) {
|
||||
if (Files.exists(output)) throw new RequestFailure("invalid_request", "Checkpoint destination already exists");
|
||||
currentProgram.saveToPackedFile(output.toFile(), monitor);
|
||||
checkpointModification = currentProgram.getModificationNumber();
|
||||
result.addProperty("path", output.toString());
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private JsonObject sessionInfo(SessionDescriptor descriptor) throws Exception {
|
||||
boolean timedOut = analysisTimeoutOccurred();
|
||||
JsonObject provider = new JsonObject();
|
||||
@@ -594,11 +548,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
String softFloat = currentProgram.getOptions(ghidra.program.model.listing.Program.PROGRAM_INFO)
|
||||
.getString("REA MIPS soft-float storage", null);
|
||||
if (softFloat != null) result.add("mips_soft_float_storage", JsonParser.parseString(softFloat));
|
||||
// Written by ReaGhidraRiscvCallingConvention after analysis; REA requires it
|
||||
// exactly when the committed profile declares a RISC-V calling-convention policy.
|
||||
String riscvCallingConvention = currentProgram.getOptions(ghidra.program.model.listing.Program.PROGRAM_INFO)
|
||||
.getString("REA RISC-V calling convention", null);
|
||||
if (riscvCallingConvention != null) result.add("riscv_calling_convention", JsonParser.parseString(riscvCallingConvention));
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -738,7 +687,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
row.addProperty("status", relocation.getStatus().toString());
|
||||
row.addProperty("type", relocation.getType());
|
||||
row.add("symbol", GSON.toJsonTree(relocation.getSymbolName()));
|
||||
row.add("values", GSON.toJsonTree(relocation.getValues() == null ? new long[0] : relocation.getValues()));
|
||||
byte[] original = relocation.getBytes();
|
||||
row.add("original_bytes_hex", original == null ? JsonNull.INSTANCE : GSON.toJsonTree(bytesHex(original)));
|
||||
@@ -816,8 +764,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return result;
|
||||
}
|
||||
|
||||
private record ByteSearchBlock(MemoryBlock block, Address low, Address high) {}
|
||||
|
||||
private JsonObject searchMemoryBytes(JsonObject params, SessionDescriptor descriptor) throws Exception {
|
||||
// mask, start, end, and cursor are omitted when absent; JSON encoding drops those keys.
|
||||
Set<String> keys = params.keySet();
|
||||
@@ -878,24 +824,20 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
JsonArray blocks = new JsonArray();
|
||||
JsonArray skipped = new JsonArray();
|
||||
JsonArray notLoaded = new JsonArray();
|
||||
JsonArray overlays = new JsonArray();
|
||||
List<ByteSearchBlock> eligible = new ArrayList<>();
|
||||
long covered = 0;
|
||||
String next = null;
|
||||
for (MemoryBlock block : ordered) {
|
||||
monitor.checkCancelled();
|
||||
Address low = block.getStart();
|
||||
Address high = block.getEnd();
|
||||
Address bound = start == null ? end : start;
|
||||
if (bound != null && !bound.getAddressSpace().equals(low.getAddressSpace())) continue;
|
||||
// Without bounds, search loaded non-overlay memory. Unloaded file
|
||||
// parts and loaded overlays stay reachable through explicit bounds.
|
||||
if (bound == null && !block.isLoaded()) {
|
||||
// Without bounds, search loaded memory only. Non-loaded file parts
|
||||
// (such as .symtab) and overlays stay reachable through explicit bounds.
|
||||
if (bound == null && (!block.isLoaded() || block.isOverlay())) {
|
||||
notLoaded.add(byteSearchCoverage(block, low, high));
|
||||
continue;
|
||||
}
|
||||
if (bound == null && block.isOverlay()) {
|
||||
overlays.add(byteSearchCoverage(block, low, high));
|
||||
continue;
|
||||
}
|
||||
if (start != null && low.compareTo(start) < 0) low = start;
|
||||
if (end != null && high.compareTo(end) > 0) high = end;
|
||||
if (low.compareTo(high) > 0) continue;
|
||||
@@ -903,73 +845,46 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
skipped.add(byteSearchCoverage(block, low, high));
|
||||
continue;
|
||||
}
|
||||
eligible.add(new ByteSearchBlock(block, low, high));
|
||||
}
|
||||
long covered = 0;
|
||||
String next = null;
|
||||
for (int first = 0; first < eligible.size() && next == null;) {
|
||||
monitor.checkCancelled();
|
||||
int last = first;
|
||||
while (last + 1 < eligible.size()) {
|
||||
monitor.checkCancelled();
|
||||
ByteSearchBlock previous = eligible.get(last);
|
||||
ByteSearchBlock following = eligible.get(last + 1);
|
||||
if (!previous.high().getAddressSpace().equals(following.low().getAddressSpace()) ||
|
||||
previous.high().next() == null || !previous.high().next().equals(following.low())) break;
|
||||
last++;
|
||||
}
|
||||
Address low = eligible.get(first).low();
|
||||
Address high = eligible.get(last).high();
|
||||
if (next != null) continue;
|
||||
if (resume != null && low.compareTo(resume) < 0) low = resume;
|
||||
if (low.compareTo(high) <= 0 && low.getAddressSpace().equals(high.getAddressSpace())) {
|
||||
Address at = low;
|
||||
Address examinedEnd = high;
|
||||
while (at != null && at.compareTo(high) <= 0 && high.subtract(at) >= bytes.length - 1L) {
|
||||
monitor.checkCancelled();
|
||||
// findBytes bounds candidate starts. The entire candidate remains in this eligible run.
|
||||
Address found = currentProgram.getMemory().findBytes(at, high.subtract(bytes.length - 1L), bytes, masks, true, monitor);
|
||||
monitor.checkCancelled();
|
||||
if (found == null || high.subtract(found) < bytes.length - 1L) break;
|
||||
if (matches.size() == limit) {
|
||||
next = binding + ":" + canonicalAddress(found);
|
||||
examinedEnd = found.addNoWrap(bytes.length - 1L);
|
||||
break;
|
||||
}
|
||||
MemoryBlock matchBlock = currentProgram.getMemory().getBlock(found);
|
||||
JsonObject match = new JsonObject();
|
||||
match.addProperty("address", canonicalAddress(found));
|
||||
match.addProperty("segment", matchBlock.getName());
|
||||
Long offset = targetFileOffset(matchBlock, found).offset();
|
||||
match.add("file_offset", offset == null ? JsonNull.INSTANCE : GSON.toJsonTree(offset));
|
||||
Function function = currentProgram.getFunctionManager().getFunctionContaining(found);
|
||||
JsonObject procedure = new JsonObject();
|
||||
if (function != null) {
|
||||
procedure.addProperty("address", canonicalAddress(function.getEntryPoint()));
|
||||
procedure.addProperty("name", function.getName(true));
|
||||
}
|
||||
match.add("procedure", function == null ? JsonNull.INSTANCE : procedure);
|
||||
matches.add(match);
|
||||
at = found.next();
|
||||
if (low.compareTo(high) > 0 || !low.getAddressSpace().equals(high.getAddressSpace())) continue;
|
||||
Address at = low;
|
||||
Address examinedEnd = high;
|
||||
while (at != null && at.compareTo(high) <= 0 && high.subtract(at) >= bytes.length - 1L) {
|
||||
monitor.checkCancelled();
|
||||
// Ghidra's end address bounds candidate starts, not the final matched byte.
|
||||
Address found = currentProgram.getMemory().findBytes(at, high.subtract(bytes.length - 1L), bytes, masks, true, monitor);
|
||||
monitor.checkCancelled();
|
||||
if (found == null || high.subtract(found) < bytes.length - 1L) break;
|
||||
if (matches.size() == limit) {
|
||||
next = binding + ":" + canonicalAddress(found);
|
||||
examinedEnd = found.addNoWrap(bytes.length - 1L);
|
||||
break;
|
||||
}
|
||||
for (int index = first; index <= last; index++) {
|
||||
monitor.checkCancelled();
|
||||
ByteSearchBlock item = eligible.get(index);
|
||||
Address coveredLow = item.low().compareTo(low) < 0 ? low : item.low();
|
||||
Address coveredHigh = item.high().compareTo(examinedEnd) > 0 ? examinedEnd : item.high();
|
||||
if (coveredLow.compareTo(coveredHigh) > 0) continue;
|
||||
JsonObject coverage = byteSearchCoverage(item.block(), coveredLow, coveredHigh);
|
||||
covered = Math.addExact(covered, coverage.get("bytes").getAsLong());
|
||||
blocks.add(coverage);
|
||||
JsonObject match = new JsonObject();
|
||||
match.addProperty("address", canonicalAddress(found));
|
||||
match.addProperty("segment", block.getName());
|
||||
Long offset = targetFileOffset(block, found).offset();
|
||||
match.add("file_offset", offset == null ? JsonNull.INSTANCE : GSON.toJsonTree(offset));
|
||||
Function function = currentProgram.getFunctionManager().getFunctionContaining(found);
|
||||
JsonObject procedure = new JsonObject();
|
||||
if (function != null) {
|
||||
procedure.addProperty("address", canonicalAddress(function.getEntryPoint()));
|
||||
procedure.addProperty("name", function.getName(true));
|
||||
}
|
||||
match.add("procedure", function == null ? JsonNull.INSTANCE : procedure);
|
||||
matches.add(match);
|
||||
at = found.next();
|
||||
}
|
||||
first = last + 1;
|
||||
JsonObject coverage = byteSearchCoverage(block, low, examinedEnd);
|
||||
covered = Math.addExact(covered, coverage.get("bytes").getAsLong());
|
||||
blocks.add(coverage);
|
||||
}
|
||||
JsonObject searched = new JsonObject();
|
||||
searched.add("blocks", blocks);
|
||||
searched.addProperty("bytes", covered);
|
||||
searched.add("skipped_uninitialized", skipped);
|
||||
searched.add("skipped_not_loaded", notLoaded);
|
||||
searched.add("skipped_overlays", overlays);
|
||||
JsonObject result = new JsonObject();
|
||||
result.add("matches", matches);
|
||||
result.add("searched", searched);
|
||||
@@ -1007,33 +922,17 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireKeys(params, Set.of("document", "address"));
|
||||
requireDocument(params);
|
||||
Address address = requireAddress(params, "address");
|
||||
String at = canonicalAddress(address);
|
||||
// A well-formed address can still lack one exact source offset. Each
|
||||
// outcome has its own code, so REA reports the reason rather than
|
||||
// schema repair. Initialization, not the loaded flag, decides whether
|
||||
// source bytes can exist: file-backed overlays are not loaded.
|
||||
MemoryBlock block = currentProgram.getMemory().getBlock(address);
|
||||
if (block == null)
|
||||
throw new RequestFailure("address_unmapped",
|
||||
"Address " + at + " is outside every Ghidra memory block");
|
||||
if (!block.isInitialized())
|
||||
throw new RequestFailure("address_uninitialized",
|
||||
"Address " + at + " is in uninitialized Ghidra memory block " + block.getName());
|
||||
if (block == null || !block.isInitialized())
|
||||
throw new RequestFailure("invalid_request", "Address has no initialized file-backed memory: " + canonicalAddress(address));
|
||||
TargetFileOffset mapping = targetFileOffset(block, address);
|
||||
if (mapping.mappings() == 0)
|
||||
throw new RequestFailure("file_mapping_absent",
|
||||
"Address " + at + " in initialized Ghidra memory block " + block.getName() +
|
||||
" has no authoritative mapping to the admitted target file");
|
||||
if (mapping.mappings() > 1)
|
||||
throw new RequestFailure("file_mapping_ambiguous",
|
||||
"Address " + at + " has " + mapping.mappings() +
|
||||
" authoritative mappings to the admitted target file");
|
||||
if (mapping.mappings() != 1)
|
||||
throw new RequestFailure("invalid_request", "Address has " + mapping.mappings() + " authoritative target-file mappings: " + canonicalAddress(address));
|
||||
Long fileOffset = mapping.offset();
|
||||
if (fileOffset == null)
|
||||
throw new RequestFailure("file_offset_unrepresentable",
|
||||
"Address " + at + " maps to an admitted target-file offset beyond the exact JSON integer range");
|
||||
throw new RequestFailure("invalid_request", "Address lacks a safe file offset: " + canonicalAddress(address));
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("address", at);
|
||||
result.addProperty("address", canonicalAddress(address));
|
||||
result.addProperty("file_offset", fileOffset);
|
||||
return result;
|
||||
}
|
||||
@@ -1088,7 +987,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
} catch (RequestFailure failure) {
|
||||
// Existing names/addresses win. Only an unresolved numeric input
|
||||
// can fall back to source coordinates; ambiguity never falls back.
|
||||
if (!failure.code.equals("procedure_not_found") || !selector.matches("0x[0-9a-fA-F]+"))
|
||||
if (!failure.code.equals("not_found") || !selector.matches("0x[0-9a-fA-F]+"))
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
@@ -1115,17 +1014,17 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
List<Address> candidates = new ArrayList<>(mappedAddresses);
|
||||
if (candidates.size() > 1)
|
||||
throw new RequestFailure("file_offset_ambiguous", "File offset " + coordinate +
|
||||
throw new RequestFailure("ambiguous", "File offset " + coordinate +
|
||||
" has multiple loaded mappings: " + candidates.stream().map(ReaGhidraBridge::canonicalAddress).toList() +
|
||||
"; select an explicit analysis address");
|
||||
if (candidates.isEmpty())
|
||||
throw new RequestFailure("file_offset_unmapped", "File offset " + coordinate +
|
||||
throw new RequestFailure("not_found", "File offset " + coordinate +
|
||||
" has no loaded source mapping for the admitted target file; Ghidra image base: " + canonicalAddress(currentProgram.getImageBase()) +
|
||||
". Inspect mappings with rea inspect-native-load-image <path> --provider ghidra --json or MCP inspect_native_load_image.");
|
||||
Address mapped = candidates.get(0);
|
||||
Function function = functionAtOrContaining(mapped);
|
||||
if (function == null)
|
||||
throw new RequestFailure("procedure_not_found", "File offset " + coordinate +
|
||||
throw new RequestFailure("not_found", "File offset " + coordinate +
|
||||
" maps to analysis address " + canonicalAddress(mapped) +
|
||||
", but no procedure starts at or contains that address in the current Ghidra analysis. " +
|
||||
"Choose an observed procedure with CLI rea search <path> . --kind procedures --mode regex --provider ghidra --json or MCP list_procedures.");
|
||||
@@ -1328,12 +1227,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (component.getDataType() instanceof ghidra.program.model.data.BitFieldDataType bitfield) {
|
||||
field.addProperty("bit_size", bitfield.getBitSize());
|
||||
field.addProperty("bit_offset", bitfield.getBitOffset());
|
||||
JsonObject bitField = new JsonObject();
|
||||
bitField.addProperty("base_type_id", bitfield.getBaseDataType().getPathName());
|
||||
bitField.addProperty("declared_bit_size", bitfield.getDeclaredBitSize());
|
||||
bitField.addProperty("component_type_lookup", typeLookupResult(bitfield));
|
||||
bitField.addProperty("base_type_lookup", typeLookupResult(bitfield.getBaseDataType()));
|
||||
field.add("bit_field", bitField);
|
||||
}
|
||||
field.addProperty("status", "observed");
|
||||
field.addProperty("flexible_tail", "unknown");
|
||||
@@ -1364,7 +1257,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.add("members", members);
|
||||
result.addProperty("total_fields", totalFields);
|
||||
result.addProperty("truncated", totalFields > 20_000);
|
||||
result.add("limitations", GSON.toJsonTree(List.of("Layouts are Ghidra data-type database observations; original debug/source authority and flexible-tail semantics are not inferred. Component type paths do not necessarily identify separately registered types; bit-field lookups report whether the manager resolves equivalent types.", "At most 20000 fields or enum members are serialized per type to bound provider output. total_fields and truncated report omissions. Enum values preserve Ghidra's signed long representation.")));
|
||||
result.add("limitations", GSON.toJsonTree(List.of("Layouts are Ghidra data-type database observations; original debug/source authority and flexible-tail semantics are not inferred. Nested and recursive types retain category-path identities for separate inspection.", "At most 20000 fields or enum members are serialized per type to bound provider output. total_fields and truncated report omissions. Enum values preserve Ghidra's signed long representation.")));
|
||||
if (type != null) {
|
||||
JsonObject metadata = analysisExtensions.inspectDataType(currentProgram, type, address, monitor);
|
||||
if (metadata != null) result.add("metadata_recovery", metadata);
|
||||
@@ -1372,11 +1265,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return result;
|
||||
}
|
||||
|
||||
private String typeLookupResult(DataType type) {
|
||||
DataType resolved = currentProgram.getDataTypeManager().getDataType(type.getPathName());
|
||||
return resolved == null ? "not-found" : resolved.isEquivalent(type) ? "equivalent" : "different";
|
||||
}
|
||||
|
||||
private JsonObject instructionReference(Reference reference) {
|
||||
JsonObject item = new JsonObject();
|
||||
JsonObject kind = referenceKind(reference);
|
||||
@@ -1424,29 +1312,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
flow.addProperty("conditional", false);
|
||||
flow.addProperty("computed", false);
|
||||
if (instruction == null) {
|
||||
Instruction containing = currentProgram.getListing().getInstructionContaining(address);
|
||||
boolean inMemory = currentProgram.getMemory().contains(address);
|
||||
Data containingData = inMemory && containing == null
|
||||
? currentProgram.getListing().getDefinedDataContaining(address) : null;
|
||||
result.addProperty("status", !inMemory ? "outside-memory" : containing != null
|
||||
? "not-instruction-boundary" : containingData != null ? "data" : "not-in-listing");
|
||||
if (inMemory && containing == null && containingData != null) {
|
||||
JsonObject parentData = new JsonObject();
|
||||
parentData.addProperty("address", canonicalAddress(containingData.getAddress()));
|
||||
parentData.addProperty("length", containingData.getLength());
|
||||
parentData.addProperty("type_id", containingData.getDataType().getPathName());
|
||||
parentData.addProperty("provenance", "ghidra-listing");
|
||||
result.add("containing_data", parentData);
|
||||
}
|
||||
if (containing != null && currentProgram.getMemory().contains(address)) {
|
||||
JsonObject containingInstruction = new JsonObject();
|
||||
containingInstruction.addProperty("address", canonicalAddress(containing.getAddress()));
|
||||
containingInstruction.addProperty("length", containing.getLength());
|
||||
containingInstruction.addProperty("mnemonic", containing.getMnemonicString());
|
||||
containingInstruction.addProperty("raw_disassembly", containing.toString());
|
||||
containingInstruction.addProperty("provenance", "ghidra-listing");
|
||||
result.add("containing_instruction", containingInstruction);
|
||||
}
|
||||
result.addProperty("status", !currentProgram.getMemory().contains(address) ? "outside-memory" : currentProgram.getListing().getInstructionContaining(address) != null
|
||||
? "not-instruction-boundary" : currentProgram.getListing().getDefinedDataContaining(address) != null ? "data" : "undecodable");
|
||||
for (String key : List.of("bytes", "length", "mnemonic", "raw_disassembly")) result.add(key, JsonNull.INSTANCE);
|
||||
} else {
|
||||
result.addProperty("status", "decoded");
|
||||
@@ -1518,7 +1385,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.add("incoming_references", incomingReferences);
|
||||
}
|
||||
List<String> limitations = new ArrayList<>(List.of("Operand components are Ghidra Listing decoder tokens; effective memory base/index/displacement roles and per-instruction context mode are unavailable. The mode field is the program language variant."));
|
||||
if (result.get("status").getAsString().equals("not-in-listing")) limitations.add("No listed instruction or defined data covers this in-memory address. REA did not attempt a decode; the memory may be uninitialized or contain valid instructions.");
|
||||
if (flow.get("kind").getAsString().equals("terminal")) limitations.add("No RETURN operation was observed in the provider's instruction p-code; terminal classification remains generic.");
|
||||
result.add("limitations", GSON.toJsonTree(limitations));
|
||||
return result;
|
||||
@@ -2064,62 +1930,41 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return symbol == null ? JsonNull.INSTANCE : GSON.toJsonTree(symbol.getName(true));
|
||||
}
|
||||
|
||||
// Analyst naming: a function entry renames the function; another mapped
|
||||
// address renames its non-dynamic primary label or receives a new global
|
||||
// label. Resolve that destination without editing the program.
|
||||
private AddressNameTarget addressNameTarget(Address address, String name)
|
||||
throws ghidra.util.exception.InvalidInputException {
|
||||
// Analyst naming: a function entry renames the function, other addresses get a primary label.
|
||||
private boolean applyAddressName(Address address, String name) throws Exception {
|
||||
if (!currentProgram.getMemory().contains(address))
|
||||
throw new RequestFailure("invalid_request", "Address is not mapped: " + canonicalAddress(address));
|
||||
validateAnnotationText(name, "name");
|
||||
Symbol edited = editedNameSymbol(address);
|
||||
return edited == null
|
||||
? new AddressNameTarget(address, currentProgram.getGlobalNamespace(), name)
|
||||
: new AddressNameTarget(address, edited.getParentNamespace(), symbolLeafName(edited, name));
|
||||
}
|
||||
|
||||
private Symbol editedNameSymbol(Address address) {
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
if (function != null) return function.getSymbol();
|
||||
Symbol primary = currentProgram.getSymbolTable().getPrimarySymbol(address);
|
||||
return primary == null || primary.isDynamic() ? null : primary;
|
||||
}
|
||||
|
||||
// Every address whose label or function symbol holds the destination.
|
||||
private List<Address> addressNameHolders(AddressNameTarget target) {
|
||||
Set<Address> holders = new TreeSet<>();
|
||||
for (Symbol symbol : currentProgram.getSymbolTable().getLabelOrFunctionSymbols(target.leaf(), target.namespace()))
|
||||
holders.add(symbol.getAddress());
|
||||
return List.copyOf(holders);
|
||||
}
|
||||
|
||||
// Apply one planned destination in its own transaction. Null means the
|
||||
// edit committed with matching primary-symbol readback.
|
||||
private String applyAddressName(AddressNameTarget target) {
|
||||
Address address = target.address();
|
||||
int transaction = currentProgram.startTransaction("REA address name");
|
||||
boolean commit = false;
|
||||
try {
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
Symbol edited = editedNameSymbol(address);
|
||||
SymbolTable table = currentProgram.getSymbolTable();
|
||||
Symbol primary = function == null ? table.getPrimarySymbol(address) : function.getSymbol();
|
||||
Namespace namespace = primary == null
|
||||
? currentProgram.getGlobalNamespace()
|
||||
: primary.getParentNamespace();
|
||||
String expected = primary == null ? name : symbolLeafName(primary, name);
|
||||
for (Symbol symbol : table.getLabelOrFunctionSymbols(expected, namespace)) {
|
||||
if (!address.equals(symbol.getAddress())) return false;
|
||||
}
|
||||
if (function != null) {
|
||||
function.setName(target.leaf(), SourceType.USER_DEFINED);
|
||||
} else if (edited != null) {
|
||||
edited.setName(target.leaf(), SourceType.USER_DEFINED);
|
||||
function.setName(expected, SourceType.USER_DEFINED);
|
||||
} else {
|
||||
Symbol created = currentProgram.getSymbolTable().createLabel(address, target.leaf(), SourceType.USER_DEFINED);
|
||||
created.setPrimary();
|
||||
if (primary != null && !primary.isDynamic()) {
|
||||
primary.setName(expected, SourceType.USER_DEFINED);
|
||||
} else {
|
||||
Symbol created = table.createLabel(address, expected, SourceType.USER_DEFINED);
|
||||
created.setPrimary();
|
||||
}
|
||||
}
|
||||
invalidateAnalysisCaches();
|
||||
Symbol readback = currentProgram.getSymbolTable().getPrimarySymbol(address);
|
||||
commit = readback != null && target.leaf().equals(readback.getName());
|
||||
return commit ? null : "readback_mismatch";
|
||||
commit = readback != null && expected.equals(readback.getName());
|
||||
return commit;
|
||||
}
|
||||
catch (ghidra.util.exception.InvalidInputException exception) {
|
||||
return "invalid_name";
|
||||
}
|
||||
catch (ghidra.util.exception.DuplicateNameException exception) {
|
||||
return "duplicate_name";
|
||||
catch (ghidra.util.exception.InvalidInputException | ghidra.util.exception.DuplicateNameException exception) {
|
||||
return false;
|
||||
}
|
||||
finally {
|
||||
currentProgram.endTransaction(transaction, commit);
|
||||
@@ -2129,17 +1974,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private JsonElement setAddressName(JsonObject params) throws Exception {
|
||||
requireKeys(params, Set.of("document", "address", "name"));
|
||||
requireDocument(params);
|
||||
String key = requireString(params, "address");
|
||||
Address address = requireAddress(params, "address");
|
||||
String name = requireString(params, "name");
|
||||
// A single edit keeps request errors for unmapped addresses and
|
||||
// unrepresentable text; other outcomes share the batch representation.
|
||||
if (!currentProgram.getMemory().contains(address))
|
||||
throw new RequestFailure("invalid_request", "Address is not mapped: " + canonicalAddress(address));
|
||||
validateAnnotationText(name, "name");
|
||||
Map<String, String> requested = new LinkedHashMap<>();
|
||||
requested.put(key, name);
|
||||
return applyAddressNames(requested);
|
||||
return GSON.toJsonTree(applyAddressName(requireAddress(params, "address"), requireString(params, "name")));
|
||||
}
|
||||
|
||||
private JsonElement setAddressesNames(JsonObject params) throws Exception {
|
||||
@@ -2147,129 +1982,34 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireDocument(params);
|
||||
if (!params.has("names") || !params.get("names").isJsonObject())
|
||||
throw new RequestFailure("invalid_request", "names must be an object of address -> name");
|
||||
Map<String, String> requested = new LinkedHashMap<>();
|
||||
for (Map.Entry<String, JsonElement> entry : params.getAsJsonObject("names").entrySet())
|
||||
requested.put(entry.getKey(), entry.getValue().getAsString());
|
||||
return applyAddressNames(requested);
|
||||
}
|
||||
|
||||
// Decide every request from the names held before any edit, so object
|
||||
// member order never selects which address receives a name. Reply with
|
||||
// per-key results and reasons; a contested destination is listed once in
|
||||
// conflicts and failures refer to it by index.
|
||||
private JsonObject applyAddressNames(Map<String, String> requested) {
|
||||
Map<String, Address> addressesByKey = new LinkedHashMap<>();
|
||||
Map<Address, List<AddressNameRequest>> requestsByAddress = new LinkedHashMap<>();
|
||||
for (Map.Entry<String, String> entry : requested.entrySet()) {
|
||||
Address address = null;
|
||||
JsonObject result = new JsonObject();
|
||||
for (Map.Entry<String, JsonElement> entry : params.getAsJsonObject("names").entrySet()) {
|
||||
try {
|
||||
address = parseReaAddress(entry.getKey());
|
||||
Address address = parseReaAddress(entry.getKey());
|
||||
requestsByAddress.computeIfAbsent(address, ignored -> new ArrayList<>())
|
||||
.add(new AddressNameRequest(entry.getKey(), address, entry.getValue()));
|
||||
.add(new AddressNameRequest(entry.getKey(), address, entry.getValue().getAsString()));
|
||||
}
|
||||
catch (RequestFailure failure) {
|
||||
// The key keeps its own invalid_address outcome below.
|
||||
}
|
||||
addressesByKey.put(entry.getKey(), address);
|
||||
}
|
||||
Map<Address, AddressNameFailure> failures = new HashMap<>();
|
||||
Map<AddressNameDestination, List<AddressNameTarget>> destinations = new LinkedHashMap<>();
|
||||
for (Map.Entry<Address, List<AddressNameRequest>> selected : requestsByAddress.entrySet()) {
|
||||
Address address = selected.getKey();
|
||||
String name = selected.getValue().get(0).name;
|
||||
if (selected.getValue().stream().anyMatch(request -> !request.name.equals(name))) {
|
||||
failures.put(address, new AddressNameFailure("conflicting_alias_names", null));
|
||||
continue;
|
||||
}
|
||||
if (!currentProgram.getMemory().contains(address)) {
|
||||
failures.put(address, new AddressNameFailure("unmapped_address", null));
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
AddressNameTarget target = addressNameTarget(address, name);
|
||||
destinations.computeIfAbsent(target.destination(), ignored -> new ArrayList<>()).add(target);
|
||||
}
|
||||
catch (RequestFailure | ghidra.util.exception.InvalidInputException exception) {
|
||||
failures.put(address, new AddressNameFailure("invalid_name", null));
|
||||
result.addProperty(entry.getKey(), false);
|
||||
}
|
||||
}
|
||||
// Reject a destination that several selected addresses request, or
|
||||
// that another address holds, instead of letting order pick a winner.
|
||||
List<AddressNameConflict> conflicts = new ArrayList<>();
|
||||
List<AddressNameTarget> independent = new ArrayList<>();
|
||||
for (List<AddressNameTarget> requesters : destinations.values()) {
|
||||
AddressNameTarget first = requesters.get(0);
|
||||
List<Address> holders = addressNameHolders(first);
|
||||
String reason;
|
||||
if (requesters.size() > 1) {
|
||||
reason = "duplicate_requested_name";
|
||||
} else if (holders.stream().allMatch(first.address()::equals)) {
|
||||
independent.add(first);
|
||||
continue;
|
||||
} else if (holders.stream().anyMatch(holder ->
|
||||
!holder.equals(first.address()) && !requestsByAddress.containsKey(holder))) {
|
||||
reason = "name_held_by_unselected_address";
|
||||
} else {
|
||||
reason = "name_held_by_selected_address";
|
||||
for (List<AddressNameRequest> requests : requestsByAddress.values()) {
|
||||
String name = requests.get(0).name;
|
||||
boolean conflictingAliasNames = requests.stream()
|
||||
.anyMatch(request -> !request.name.equals(name));
|
||||
boolean success = false;
|
||||
if (!conflictingAliasNames) {
|
||||
try {
|
||||
success = applyAddressName(requests.get(0).address, name);
|
||||
}
|
||||
catch (RequestFailure failure) {
|
||||
// Keep per-address failures in the existing batch result shape.
|
||||
}
|
||||
}
|
||||
AddressNameConflict conflict = new AddressNameConflict(
|
||||
first.namespace(),
|
||||
first.leaf(),
|
||||
requesters.stream().map(AddressNameTarget::address).sorted().toList(),
|
||||
holders
|
||||
);
|
||||
conflicts.add(conflict);
|
||||
for (AddressNameTarget target : requesters)
|
||||
failures.put(target.address(), new AddressNameFailure(reason, conflict));
|
||||
for (AddressNameRequest request : requests)
|
||||
result.addProperty(request.inputAddress, success);
|
||||
}
|
||||
// Independent edits have distinct destinations that no other address
|
||||
// holds, so their results do not depend on application order.
|
||||
for (AddressNameTarget target : independent) {
|
||||
String reason = applyAddressName(target);
|
||||
if (reason != null) failures.put(target.address(), new AddressNameFailure(reason, null));
|
||||
}
|
||||
// Requesters belong to exactly one destination, so ordering conflicts
|
||||
// by their first requester gives indexes independent of member order.
|
||||
conflicts.sort(Comparator.comparing((AddressNameConflict conflict) -> conflict.requesters().get(0)));
|
||||
Map<AddressNameConflict, Integer> indexes = new IdentityHashMap<>();
|
||||
JsonArray contested = new JsonArray();
|
||||
for (AddressNameConflict conflict : conflicts) {
|
||||
indexes.put(conflict, contested.size());
|
||||
JsonObject value = new JsonObject();
|
||||
if (conflict.namespace().isGlobal()) value.add("namespace", JsonNull.INSTANCE);
|
||||
else value.addProperty("namespace", conflict.namespace().getName(true));
|
||||
value.addProperty("name", conflict.leaf());
|
||||
value.add("requested_by", canonicalAddresses(conflict.requesters()));
|
||||
value.add("held_by", canonicalAddresses(conflict.holders()));
|
||||
contested.add(value);
|
||||
}
|
||||
JsonObject results = new JsonObject();
|
||||
JsonObject failed = new JsonObject();
|
||||
for (Map.Entry<String, Address> entry : addressesByKey.entrySet()) {
|
||||
Address address = entry.getValue();
|
||||
AddressNameFailure failure = address == null
|
||||
? new AddressNameFailure("invalid_address", null)
|
||||
: failures.get(address);
|
||||
results.addProperty(entry.getKey(), failure == null);
|
||||
if (failure == null) continue;
|
||||
JsonObject value = new JsonObject();
|
||||
if (address == null) value.add("address", JsonNull.INSTANCE);
|
||||
else value.addProperty("address", canonicalAddress(address));
|
||||
value.addProperty("reason", failure.reason());
|
||||
if (failure.conflict() == null) value.add("conflict", JsonNull.INSTANCE);
|
||||
else value.addProperty("conflict", indexes.get(failure.conflict()));
|
||||
failed.add(entry.getKey(), value);
|
||||
}
|
||||
JsonObject result = new JsonObject();
|
||||
result.add("results", results);
|
||||
result.add("failures", failed);
|
||||
result.add("conflicts", contested);
|
||||
return result;
|
||||
}
|
||||
|
||||
private static JsonArray canonicalAddresses(List<Address> addresses) {
|
||||
JsonArray result = new JsonArray();
|
||||
for (Address address : addresses) result.add(canonicalAddress(address));
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -2385,6 +2125,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
throw new RequestFailure("invalid_request", "mode must be literal or regex");
|
||||
}
|
||||
boolean caseSensitive = requireBoolean(params, "case_sensitive");
|
||||
List<InventoryItem> inventory = procedureSearch
|
||||
? procedures().stream().map(FunctionEntry::item).toList()
|
||||
: strings();
|
||||
ValueMatcher matcher;
|
||||
if (mode.equals("literal") && !caseSensitive) {
|
||||
if (!params.has("literal_ascii_candidate"))
|
||||
@@ -2398,65 +2141,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
: regexMatcher(expression, caseSensitive);
|
||||
}
|
||||
JsonArray items = new JsonArray();
|
||||
if (procedureSearch) {
|
||||
// Search the names exact lookup accepts, keeping one row per
|
||||
// procedure under its primary name.
|
||||
for (FunctionEntry entry : procedures()) {
|
||||
monitor.checkCancelled();
|
||||
ProcedureNameMatch match = matchProcedureNames(entry, matcher);
|
||||
if (!match.matched()) {
|
||||
continue;
|
||||
}
|
||||
JsonObject item = resultItem(entry.item, "procedure");
|
||||
if (!match.aliases().isEmpty()) {
|
||||
JsonArray aliases = new JsonArray();
|
||||
for (String alias : match.aliases()) aliases.add(alias);
|
||||
item.add("matched_aliases", aliases);
|
||||
}
|
||||
items.add(item);
|
||||
}
|
||||
return items;
|
||||
}
|
||||
for (InventoryItem item : strings()) {
|
||||
for (InventoryItem item : inventory) {
|
||||
monitor.checkCancelled();
|
||||
if (!matcher.matches(item.value)) {
|
||||
continue;
|
||||
}
|
||||
items.add(resultItem(item, procedureSearch ? "procedure" : "string"));
|
||||
items.add(resultItem(item, null));
|
||||
}
|
||||
return items;
|
||||
}
|
||||
|
||||
/**
|
||||
* The one definition of the names that select a procedure, shared by exact
|
||||
* lookup and search: its primary name and every other symbol at its exact
|
||||
* entry, each in leaf or namespace-qualified spelling. A label inside the
|
||||
* body never names the procedure. Matching secondary symbols are reported
|
||||
* by their qualified names, sorted.
|
||||
*/
|
||||
private ProcedureNameMatch matchProcedureNames(FunctionEntry entry, ValueMatcher matcher) throws Exception {
|
||||
Function function = entry.function;
|
||||
boolean primary = spellingMatches(function.getName(), entry.item.value, matcher);
|
||||
Set<String> aliases = null;
|
||||
for (Symbol symbol : currentProgram.getSymbolTable().getSymbols(function.getEntryPoint())) {
|
||||
monitor.checkCancelled();
|
||||
String qualified = symbol.getName(true);
|
||||
if (!spellingMatches(symbol.getName(), qualified, matcher)) continue;
|
||||
if (qualified.equals(entry.item.value)) {
|
||||
primary = true;
|
||||
continue;
|
||||
}
|
||||
if (aliases == null) aliases = new TreeSet<>();
|
||||
aliases.add(qualified);
|
||||
}
|
||||
return new ProcedureNameMatch(primary, aliases == null ? List.of() : List.copyOf(aliases));
|
||||
}
|
||||
|
||||
/** A leaf is the suffix of its qualified spelling, so a literal found in it is found in both. */
|
||||
private static boolean spellingMatches(String leaf, String qualified, ValueMatcher matcher) {
|
||||
return matcher.matches(qualified) || (!leaf.equals(qualified) && matcher.matches(leaf));
|
||||
}
|
||||
|
||||
private List<InventoryItem> names() throws Exception {
|
||||
if (nameInventory != null) {
|
||||
return nameInventory;
|
||||
@@ -2559,16 +2253,29 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
// exact symbol name, even when every character is a hexadecimal digit.
|
||||
boolean explicitAddress = isExplicitReaAddress(value);
|
||||
if (!explicitAddress) {
|
||||
// Imported labels (for example Mach-O _main) can share a function
|
||||
// entry without becoming its primary symbol; search accepts the same.
|
||||
ValueMatcher selector = value::equals;
|
||||
List<Function> matches = new ArrayList<>();
|
||||
for (FunctionEntry entry : procedures()) {
|
||||
monitor.checkCancelled();
|
||||
if (matchProcedureNames(entry, selector).matched()) matches.add(entry.function);
|
||||
Function function = entry.function;
|
||||
Symbol symbol = function.getSymbol();
|
||||
String qualified = symbol == null ? function.getName() : symbol.getName(true);
|
||||
boolean matchesName = function.getName().equals(value) || qualified.equals(value);
|
||||
if (!matchesName) {
|
||||
// Imported labels (for example Mach-O _main) can share a
|
||||
// function entry without becoming its primary symbol. Only
|
||||
// exact entries qualify; an interior label is not a selector.
|
||||
for (Symbol entrySymbol : currentProgram.getSymbolTable().getSymbols(function.getEntryPoint())) {
|
||||
monitor.checkCancelled();
|
||||
if (entrySymbol.getName().equals(value) || entrySymbol.getName(true).equals(value)) {
|
||||
matchesName = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (matchesName) matches.add(function);
|
||||
}
|
||||
if (matches.size() > 1)
|
||||
throw new RequestFailure("procedure_ambiguous", "Ghidra procedure name is ambiguous: " + value +
|
||||
throw new RequestFailure("ambiguous", "Ghidra procedure name is ambiguous: " + value +
|
||||
"; select an exact entry address from " + matches.stream()
|
||||
.map(function -> canonicalAddress(function.getEntryPoint())).toList());
|
||||
if (matches.size() == 1) return matches.get(0);
|
||||
@@ -2576,11 +2283,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
// Retain legacy bare-hex address input only when no symbol matches.
|
||||
Address address = explicitAddress ? parseReaAddress(value) : tryParseAddress(value);
|
||||
if (address == null)
|
||||
throw new RequestFailure("procedure_not_found", "Unknown Ghidra procedure name or address: " + value);
|
||||
throw new RequestFailure("not_found", "Unknown Ghidra procedure name or address: " + value);
|
||||
Function function = functionAtOrContaining(address);
|
||||
if (function == null)
|
||||
// Other procedure operations retain their analysis-address semantics.
|
||||
throw new RequestFailure("procedure_not_found", "No procedure exists at the requested address: " + value +
|
||||
throw new RequestFailure("not_found", "No procedure exists at the requested address: " + value +
|
||||
"; Ghidra image base: " + canonicalAddress(currentProgram.getImageBase()) +
|
||||
". Choose an observed analysis address. CLI: rea search <path> . --kind procedures --mode regex --provider ghidra --json; " +
|
||||
"MCP: list_procedures. Inspect the loaded base and source mappings with CLI rea analyze <path> --provider ghidra --json " +
|
||||
@@ -3747,7 +3454,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return result;
|
||||
}
|
||||
|
||||
private JsonObject callRelationshipCoverage(
|
||||
private static JsonObject callRelationshipCoverage(
|
||||
CallBoundaryObservation calls, Integer missingCallers) {
|
||||
JsonObject coverage = completeResolvedCallCoverage();
|
||||
Integer omitted = calls == null ? null : calls.unrepresentedCalleeCalls();
|
||||
@@ -3764,7 +3471,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (calls == null && missingCallers == null) directions.add("callers");
|
||||
directions.add("callees");
|
||||
coverage.add("limited_directions", directions);
|
||||
return qualifyProjectCallCoverage(coverage, calls, missingCallers);
|
||||
return coverage;
|
||||
}
|
||||
JsonArray limitations = new JsonArray();
|
||||
if (callerLimitation != null) limitations.add(callerLimitation);
|
||||
@@ -3778,35 +3485,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
coverage.add("limitations", limitations);
|
||||
coverage.add("limited_directions", directions);
|
||||
}
|
||||
return qualifyProjectCallCoverage(coverage, calls, missingCallers);
|
||||
}
|
||||
|
||||
private JsonObject qualifyProjectCallCoverage(
|
||||
JsonObject coverage, CallBoundaryObservation calls, Integer missingCallers) {
|
||||
if (!existingProject) return coverage;
|
||||
String reason = "The selected project's prior analysis completeness is unknown; " +
|
||||
"the recorded call relationships may omit undiscovered callers or callees.";
|
||||
String status = coverage.get("status").getAsString();
|
||||
// A single-direction read with known omissions is still partial; an
|
||||
// unknown project history adds a limitation rather than erasing them.
|
||||
if (status.equals("partial") && (calls == null || missingCallers == null)) {
|
||||
coverage.getAsJsonArray("limitations").add(reason);
|
||||
return coverage;
|
||||
}
|
||||
if (status.equals("unknown")) reason += " " + coverage.get("reason").getAsString();
|
||||
if (status.equals("partial")) {
|
||||
for (JsonElement limitation : coverage.getAsJsonArray("limitations"))
|
||||
reason += " " + limitation.getAsString();
|
||||
}
|
||||
JsonArray directions = new JsonArray();
|
||||
if (missingCallers != null || (calls == null && missingCallers == null))
|
||||
directions.add("callers");
|
||||
if (calls != null || (calls == null && missingCallers == null))
|
||||
directions.add("callees");
|
||||
coverage.addProperty("status", "unknown");
|
||||
coverage.addProperty("reason", reason);
|
||||
coverage.remove("limitations");
|
||||
coverage.add("limited_directions", directions);
|
||||
return coverage;
|
||||
}
|
||||
|
||||
@@ -5055,7 +4733,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
// without retaining its cause. This producer reason is not bad syntax.
|
||||
if (exception.getDescription().equals("Stack overflow during pattern compilation"))
|
||||
throw regexStackExhausted(expression, "compiling");
|
||||
throw new RequestFailure("invalid_regex_pattern", "Invalid regex pattern " + expression + ": " + exception.getDescription() + " at index " + exception.getIndex());
|
||||
throw new RequestFailure("invalid_request", "Invalid regex pattern " + expression + ": " + exception.getDescription() + " at index " + exception.getIndex());
|
||||
}
|
||||
catch (StackOverflowError exhausted) {
|
||||
throw regexStackExhausted(expression, "compiling");
|
||||
@@ -5302,20 +4980,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
|
||||
private record InventoryItem(Address address, String value, JsonObject facts, JsonObject symbol) {}
|
||||
private record AddressNameRequest(String inputAddress, Address address, String name) {}
|
||||
private record AddressNameDestination(long namespace, String leaf) {}
|
||||
private record AddressNameTarget(Address address, Namespace namespace, String leaf) {
|
||||
AddressNameDestination destination() {
|
||||
return new AddressNameDestination(namespace.getID(), leaf);
|
||||
}
|
||||
}
|
||||
private record AddressNameConflict(Namespace namespace, String leaf, List<Address> requesters, List<Address> holders) {}
|
||||
private record AddressNameFailure(String reason, AddressNameConflict conflict) {}
|
||||
private record FunctionEntry(Function function, InventoryItem item) {}
|
||||
private record ProcedureNameMatch(boolean primary, List<String> aliases) {
|
||||
boolean matched() {
|
||||
return primary || !aliases.isEmpty();
|
||||
}
|
||||
}
|
||||
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage, boolean decompilerFailed) {}
|
||||
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
|
||||
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
|
||||
|
||||
@@ -1,283 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.database.SpecExtension;
|
||||
import ghidra.program.model.data.DataType;
|
||||
import ghidra.program.model.data.DoubleDataType;
|
||||
import ghidra.program.model.data.Float16DataType;
|
||||
import ghidra.program.model.data.FloatDataType;
|
||||
import ghidra.program.model.data.FunctionDefinition;
|
||||
import ghidra.program.model.data.IntegerDataType;
|
||||
import ghidra.program.model.data.StructureDataType;
|
||||
import ghidra.program.model.lang.CompilerSpec;
|
||||
import ghidra.program.model.lang.Language;
|
||||
import ghidra.program.model.lang.PrototypeModel;
|
||||
import ghidra.program.model.lang.Register;
|
||||
import ghidra.program.model.listing.Function;
|
||||
import ghidra.program.model.listing.Parameter;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.listing.VariableStorage;
|
||||
import ghidra.program.model.pcode.Varnode;
|
||||
import ghidra.program.model.pcode.XmlEncode;
|
||||
import java.io.StringReader;
|
||||
import java.io.StringWriter;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.regex.Pattern;
|
||||
import javax.xml.parsers.DocumentBuilderFactory;
|
||||
import javax.xml.transform.OutputKeys;
|
||||
import javax.xml.transform.TransformerFactory;
|
||||
import javax.xml.transform.dom.DOMSource;
|
||||
import javax.xml.transform.stream.StreamResult;
|
||||
import org.w3c.dom.Document;
|
||||
import org.w3c.dom.Element;
|
||||
import org.w3c.dom.Node;
|
||||
import org.w3c.dom.NodeList;
|
||||
import org.xml.sax.InputSource;
|
||||
|
||||
/**
|
||||
* Apply the committed RISC-V calling-convention policy after analysis. Ghidra's generic RV32
|
||||
* compiler specification models ilp32d. Each policy derives a program-specific prototype from the
|
||||
* program's own default prototype, as its {@link Policy} row and docs/providers/ghidra-riscv.md state. The
|
||||
* script installs the prototype as a specification extension and assigns it, with dynamic
|
||||
* storage, to every non-thunk function, external function and function definition.
|
||||
*/
|
||||
public final class ReaGhidraRiscvCallingConvention extends GhidraScript {
|
||||
static final String REPORT_KEY = "REA RISC-V calling convention";
|
||||
/** RISC-V floating-point registers by ABI name: ft0-ft11, fs0-fs11 and fa0-fa7. */
|
||||
private static final Pattern FLOAT_REGISTER = Pattern.compile("f[ast]\\d+");
|
||||
|
||||
/**
|
||||
* What a policy changes in the default ilp32d prototype. Every policy also passes values of 9
|
||||
* bytes or more, such as {@code long double}, by reference.
|
||||
*
|
||||
* @param floatEntryBytes the largest value a floating-point register may pass or return; 0
|
||||
* removes those entries and every rule that assigns floating-point storage
|
||||
* @param preservesFloatRegisters whether fs0-fs11 stay preserved across calls
|
||||
*/
|
||||
private record Policy(
|
||||
String id, String prototype, int floatEntryBytes, boolean preservesFloatRegisters) {}
|
||||
|
||||
private static final List<Policy> POLICIES =
|
||||
List.of(new Policy("ilp32-prototype-v1", "__rea_ilp32", 0, false));
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] arguments = getScriptArgs();
|
||||
Policy policy =
|
||||
POLICIES.stream()
|
||||
.filter(candidate -> arguments.length == 1 && candidate.id().equals(arguments[0]))
|
||||
.findFirst()
|
||||
.orElseThrow(
|
||||
() ->
|
||||
new IllegalArgumentException(
|
||||
"REA RISC-V calling convention requires one of the policies "
|
||||
+ POLICIES.stream().map(Policy::id).toList()));
|
||||
Language language = currentProgram.getLanguage();
|
||||
if (!language.getProcessor().toString().equals("RISCV")
|
||||
|| currentProgram.getDefaultPointerSize() != 4)
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.id() + " requires a 32-bit RISC-V program, not "
|
||||
+ language.getLanguageID());
|
||||
CompilerSpec spec = currentProgram.getCompilerSpec();
|
||||
new SpecExtension(currentProgram)
|
||||
.addReplaceCompilerSpecExtension(
|
||||
derive(spec.getDefaultCallingConvention(), spec, language, policy), monitor);
|
||||
PrototypeModel model =
|
||||
currentProgram.getCompilerSpec().getCallingConvention(policy.prototype());
|
||||
if (model == null)
|
||||
throw new IllegalStateException("REA " + policy.prototype() + " was not installed");
|
||||
selfCheck(model, policy, language);
|
||||
int functions = 0;
|
||||
List<Function> callables = new ArrayList<>();
|
||||
currentProgram.getFunctionManager().getFunctions(true).forEach(callables::add);
|
||||
currentProgram.getFunctionManager().getExternalFunctions().forEach(callables::add);
|
||||
for (Function function : callables) {
|
||||
// A thunk's signature is its target's; the target is in this list.
|
||||
if (function.isThunk()) continue;
|
||||
if (function.hasCustomVariableStorage()) function.setCustomVariableStorage(false);
|
||||
function.setCallingConvention(policy.prototype());
|
||||
if (policy.floatEntryBytes() == 0) {
|
||||
List<VariableStorage> storage = new ArrayList<>();
|
||||
storage.add(function.getReturn().getVariableStorage());
|
||||
for (Parameter parameter : function.getParameters())
|
||||
storage.add(parameter.getVariableStorage());
|
||||
// An untyped function has no assigned storage yet; the decompiler applies the
|
||||
// prototype when it recovers one.
|
||||
requireIntegerStorage(
|
||||
storage.toArray(VariableStorage[]::new), function.getName(true), true);
|
||||
}
|
||||
functions += 1;
|
||||
}
|
||||
int definitions = 0;
|
||||
Iterator<DataType> types = currentProgram.getDataTypeManager().getAllDataTypes();
|
||||
while (types.hasNext()) {
|
||||
if (!(types.next() instanceof FunctionDefinition definition)) continue;
|
||||
definition.setCallingConvention(policy.prototype());
|
||||
definitions += 1;
|
||||
}
|
||||
// Written last: any failure above leaves no report, and REA refuses the session.
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setString(
|
||||
REPORT_KEY,
|
||||
"{\"policy\":\"" + policy.id() + "\",\"functions\":" + functions
|
||||
+ ",\"definitions\":" + definitions + "}");
|
||||
println(
|
||||
"REA RISC-V " + policy.prototype() + ": functions=" + functions
|
||||
+ " definitions=" + definitions);
|
||||
}
|
||||
|
||||
/** The default ilp32d prototype changed as the policy states. */
|
||||
private static String derive(
|
||||
PrototypeModel base, CompilerSpec spec, Language language, Policy policy) throws Exception {
|
||||
XmlEncode encoder = new XmlEncode();
|
||||
base.encode(encoder, spec.getPcodeInjectLibrary());
|
||||
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
|
||||
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
|
||||
Document document =
|
||||
factory.newDocumentBuilder().parse(new InputSource(new StringReader(encoder.toString())));
|
||||
Element root = document.getDocumentElement();
|
||||
if (!root.getTagName().equals("prototype"))
|
||||
throw new IllegalStateException("Unexpected default prototype encoding " + root.getTagName());
|
||||
root.setAttribute("name", policy.prototype());
|
||||
List<Element> floatEntries = new ArrayList<>();
|
||||
NodeList entries = root.getElementsByTagName("pentry");
|
||||
for (int index = 0; index < entries.getLength(); index += 1) {
|
||||
Element entry = (Element) entries.item(index);
|
||||
if (entry.getAttribute("storage").equals("float")) floatEntries.add(entry);
|
||||
}
|
||||
if (floatEntries.isEmpty())
|
||||
throw new IllegalStateException("Default prototype has no floating-point entries");
|
||||
if (policy.floatEntryBytes() == 0) {
|
||||
List<Node> removed = new ArrayList<>(floatEntries);
|
||||
NodeList rules = root.getElementsByTagName("rule");
|
||||
for (int index = 0; index < rules.getLength(); index += 1) {
|
||||
Element rule = (Element) rules.item(index);
|
||||
if (usesFloatStorage(rule)) removed.add(rule);
|
||||
}
|
||||
if (removed.size() == floatEntries.size())
|
||||
throw new IllegalStateException("Default prototype has no floating-point rules to remove");
|
||||
for (Node node : removed) node.getParentNode().removeChild(node);
|
||||
} else
|
||||
for (Element entry : floatEntries)
|
||||
entry.setAttribute("maxsize", Integer.toString(policy.floatEntryBytes()));
|
||||
// Values over 2 x XLEN go by reference. Returns need no rule: a value that the return
|
||||
// registers cannot hold already becomes a caller buffer passed in a0.
|
||||
NodeList inputs = root.getElementsByTagName("input");
|
||||
if (inputs.getLength() != 1)
|
||||
throw new IllegalStateException("Default prototype has no single input list");
|
||||
Element input = (Element) inputs.item(0);
|
||||
Element last = null;
|
||||
for (Node child = input.getFirstChild(); child != null; child = child.getNextSibling())
|
||||
if (child instanceof Element element && element.getTagName().equals("rule")) last = element;
|
||||
if (last == null || !joinsAnyValue(last))
|
||||
throw new IllegalStateException("Default prototype's input list does not end with a join");
|
||||
Element byReference = document.createElement("rule");
|
||||
Element datatype = document.createElement("datatype");
|
||||
datatype.setAttribute("name", "any");
|
||||
datatype.setAttribute("minsize", "9");
|
||||
byReference.appendChild(datatype);
|
||||
byReference.appendChild(document.createElement("convert_to_ptr"));
|
||||
input.insertBefore(byReference, last);
|
||||
if (!policy.preservesFloatRegisters()) {
|
||||
NodeList killedLists = root.getElementsByTagName("killedbycall");
|
||||
Element killed = killedLists.getLength() == 0 ? null : (Element) killedLists.item(0);
|
||||
if (killed == null) {
|
||||
killed = document.createElement("killedbycall");
|
||||
root.appendChild(killed);
|
||||
}
|
||||
List<Element> moved = new ArrayList<>();
|
||||
NodeList preservedLists = root.getElementsByTagName("unaffected");
|
||||
for (int list = 0; list < preservedLists.getLength(); list += 1) {
|
||||
NodeList preserved = preservedLists.item(list).getChildNodes();
|
||||
for (int index = 0; index < preserved.getLength(); index += 1) {
|
||||
if (preserved.item(index) instanceof Element varnode
|
||||
&& isFloatRegister(varnode, language))
|
||||
moved.add(varnode);
|
||||
}
|
||||
}
|
||||
for (Element varnode : moved) {
|
||||
varnode.getParentNode().removeChild(varnode);
|
||||
killed.appendChild(varnode);
|
||||
}
|
||||
}
|
||||
StringWriter xml = new StringWriter();
|
||||
var transformer = TransformerFactory.newInstance().newTransformer();
|
||||
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
|
||||
transformer.transform(new DOMSource(document), new StreamResult(xml));
|
||||
return xml.toString();
|
||||
}
|
||||
|
||||
/** The installed prototype places each class of value as its policy states. */
|
||||
private void selfCheck(PrototypeModel model, Policy policy, Language language) {
|
||||
StructureDataType pair = new StructureDataType("rea_float_pair", 0);
|
||||
pair.add(FloatDataType.dataType);
|
||||
pair.add(FloatDataType.dataType);
|
||||
VariableStorage[] storage =
|
||||
model.getStorageLocations(
|
||||
currentProgram,
|
||||
new DataType[] {
|
||||
DoubleDataType.dataType, FloatDataType.dataType, DoubleDataType.dataType,
|
||||
IntegerDataType.dataType, pair, Float16DataType.dataType
|
||||
},
|
||||
false);
|
||||
VariableStorage quad = storage[storage.length - 1];
|
||||
if (quad.size() != currentProgram.getDefaultPointerSize() || quad.getRegister() == null)
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.prototype() + " does not pass a 16-byte value by reference: " + quad);
|
||||
if (policy.floatEntryBytes() == 0)
|
||||
requireIntegerStorage(storage, policy.prototype() + " self-check", false);
|
||||
if (policy.preservesFloatRegisters()) return;
|
||||
for (Varnode preserved : model.getUnaffectedList()) {
|
||||
Register register = language.getRegister(preserved.getAddress(), preserved.getSize());
|
||||
if (register != null && FLOAT_REGISTER.matcher(register.getName()).matches())
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.prototype() + " still preserves " + register.getName());
|
||||
}
|
||||
}
|
||||
|
||||
private static boolean joinsAnyValue(Element rule) {
|
||||
List<Element> children = new ArrayList<>();
|
||||
for (Node child = rule.getFirstChild(); child != null; child = child.getNextSibling())
|
||||
if (child instanceof Element element) children.add(element);
|
||||
return children.size() == 2
|
||||
&& children.get(0).getTagName().equals("datatype")
|
||||
&& children.get(0).getAttribute("name").equals("any")
|
||||
&& children.get(1).getTagName().equals("join");
|
||||
}
|
||||
|
||||
/** Whether a rule's action, at any depth, assigns floating-point storage. */
|
||||
private static boolean usesFloatStorage(Element element) {
|
||||
NodeList children = element.getChildNodes();
|
||||
for (int index = 0; index < children.getLength(); index += 1) {
|
||||
if (!(children.item(index) instanceof Element child)) continue;
|
||||
if (child.getAttribute("storage").equals("float") || usesFloatStorage(child)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private static boolean isFloatRegister(Element varnode, Language language) {
|
||||
if (!varnode.getTagName().equals("varnode")
|
||||
|| !varnode.getAttribute("space").equals("register"))
|
||||
return false;
|
||||
Register register =
|
||||
language.getRegister(
|
||||
language.getAddressFactory().getRegisterSpace()
|
||||
.getAddress(Long.decode(varnode.getAttribute("offset"))),
|
||||
Integer.parseInt(varnode.getAttribute("size")));
|
||||
return register != null && FLOAT_REGISTER.matcher(register.getName()).matches();
|
||||
}
|
||||
|
||||
private static void requireIntegerStorage(
|
||||
VariableStorage[] storage, String owner, boolean allowUnassigned) {
|
||||
for (VariableStorage item : storage) {
|
||||
if (item.isBadStorage() || (!allowUnassigned && item.isUnassignedStorage()))
|
||||
throw new IllegalStateException("REA RISC-V storage unresolved for " + owner);
|
||||
List<Register> registers = item.getRegisters();
|
||||
if (registers == null) continue; // stack-only storage
|
||||
for (Register register : registers) {
|
||||
if (FLOAT_REGISTER.matcher(register.getName()).matches())
|
||||
throw new IllegalStateException(
|
||||
"REA RISC-V storage still uses " + register.getName() + " for " + owner);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+5
-161
@@ -22,9 +22,6 @@ HOPPER_SYMBOL_NAME_UTF16_UNITS = 1024
|
||||
# that does not fit is null, not an older descriptor path.
|
||||
# 536870887 is Node's 536870888-character string ceiling minus the framing LF.
|
||||
# The ceiling is the encoded payload; JSON expansion is measured before it.
|
||||
# Encoded JSON allowance, in bytes, shared by one instruction list, one
|
||||
# native-caller list, and one caller-scan page. This release has no runtime override.
|
||||
CALLER_SCAN_ALLOWANCE_BYTES = 512 * 1024
|
||||
_BRIDGE_MAX_REQUEST_BYTES_SETTING = "REA_BRIDGE_MAX_REQUEST_BYTES"
|
||||
DEFAULT_BRIDGE_MAX_REQUEST_BYTES = 16 * 1024 * 1024
|
||||
MINIMUM_BRIDGE_MAX_REQUEST_BYTES = 7464
|
||||
@@ -235,9 +232,6 @@ EXHAUSTIVE_ANALYSIS_METHODS = frozenset(
|
||||
"procedure_assembly",
|
||||
"procedure_callees",
|
||||
"procedure_callers",
|
||||
"rea_caller_scan_page",
|
||||
"rea_caller_scan_target",
|
||||
"rea_caller_base",
|
||||
"procedure_info",
|
||||
"procedure_pseudo_code",
|
||||
"procedure_references",
|
||||
@@ -491,30 +485,19 @@ def _instruction_addresses(procedure):
|
||||
return result
|
||||
|
||||
|
||||
def _iter_native_calls(procedure, direction):
|
||||
def _native_calls(procedure, direction):
|
||||
references = procedure.getAllCallees() if direction == "outgoing" else procedure.getAllCallers()
|
||||
kinds = {0: "none", 1: "unknown", 2: "direct", 3: "objective_c"}
|
||||
calls = {}
|
||||
unresolved = []
|
||||
for reference in references:
|
||||
source, target, kind = reference.fromAddress(), reference.toAddress(), reference.type()
|
||||
if source in BAD_ADDRESSES:
|
||||
yield None, None, None
|
||||
continue
|
||||
if target in BAD_ADDRESSES:
|
||||
yield source, None, {"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind}
|
||||
unresolved.append({"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind})
|
||||
continue
|
||||
yield source, target, {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
|
||||
|
||||
|
||||
def _native_calls(procedure, direction):
|
||||
calls = {}
|
||||
unresolved = []
|
||||
for source, target, detail in _iter_native_calls(procedure, direction):
|
||||
if detail is None:
|
||||
continue
|
||||
if target is None:
|
||||
unresolved.append(detail)
|
||||
else:
|
||||
calls[(source, target)] = detail
|
||||
calls[(source, target)] = {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
|
||||
return calls, sorted(unresolved, key=lambda item: int(item["address"], 16))
|
||||
|
||||
|
||||
@@ -564,136 +547,6 @@ def _procedure_map(document):
|
||||
return result
|
||||
|
||||
|
||||
def _caller_target_data(procedure):
|
||||
addresses = []
|
||||
seen = set()
|
||||
size = 0
|
||||
for block in procedure.basicBlockIterator():
|
||||
for address in _block_instruction_addresses(procedure, block):
|
||||
if address in seen:
|
||||
continue
|
||||
encoded = _hex(address)
|
||||
size += len(encoded) + 4
|
||||
if size > CALLER_SCAN_ALLOWANCE_BYTES:
|
||||
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": False}
|
||||
addresses.append(encoded)
|
||||
seen.add(address)
|
||||
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": True}
|
||||
|
||||
|
||||
def _caller_scan_target(document, params):
|
||||
return _caller_target_data(_procedure(document, params.get("procedure")))
|
||||
|
||||
|
||||
def _caller_base(document, params):
|
||||
procedure = _procedure(document, params.get("procedure"))
|
||||
callers = []
|
||||
size = 0
|
||||
complete = True
|
||||
for item in procedure.getAllCallerProcedures():
|
||||
address = _hex(item.getEntryPoint())
|
||||
size += len(address) + 4
|
||||
if size > CALLER_SCAN_ALLOWANCE_BYTES:
|
||||
complete = False
|
||||
break
|
||||
callers.append(address)
|
||||
return {
|
||||
**_caller_target_data(procedure),
|
||||
"callers": sorted(callers, key=lambda value: int(value, 16)),
|
||||
"callers_complete": complete,
|
||||
}
|
||||
|
||||
|
||||
def _caller_scan_page(document, params):
|
||||
"""Observe forward relationships in one bounded, stateless bridge request."""
|
||||
start = params.get("start")
|
||||
if isinstance(start, bool) or not isinstance(start, int) or start < 0:
|
||||
raise InvalidRequestError("Caller scan start must be a non-negative integer")
|
||||
segments = document.getSegmentsList()
|
||||
counts = [segment.getProcedureCount() for segment in segments]
|
||||
total = sum(counts)
|
||||
end = min(start + 64, total)
|
||||
if start > total:
|
||||
raise InvalidRequestError("Caller scan start exceeds the procedure inventory")
|
||||
items = []
|
||||
item_bytes = 0
|
||||
page_limit = CALLER_SCAN_ALLOWANCE_BYTES
|
||||
oversized = False
|
||||
identities = {}
|
||||
containing = {}
|
||||
|
||||
def identity(procedure):
|
||||
entry = procedure.getEntryPoint()
|
||||
if entry not in identities:
|
||||
identities[entry] = _procedure_identity(procedure)
|
||||
return identities[entry]
|
||||
|
||||
def owner(address):
|
||||
if address not in containing:
|
||||
procedure, _ = _containing_procedure(document, address)
|
||||
containing[address] = identity(procedure) if procedure is not None else None
|
||||
return containing[address]
|
||||
|
||||
segment_start = 0
|
||||
for segment, count in zip(segments, counts):
|
||||
begin, stop = max(start - segment_start, 0), min(end - segment_start, count)
|
||||
segment_start += count
|
||||
for index in range(begin, max(begin, stop)):
|
||||
candidate = segment.getProcedureAtIndex(index)
|
||||
item = {"caller": identity(candidate), "callees": [], "references": []}
|
||||
size = len(json.dumps(item, ensure_ascii=True).encode("utf-8")) + 1
|
||||
remaining = page_limit - item_bytes
|
||||
too_large = size > remaining
|
||||
if not too_large:
|
||||
for callee in candidate.getAllCalleeProcedures():
|
||||
address = _hex(callee.getEntryPoint())
|
||||
size += len(address) + 4
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
item["callees"].append(address)
|
||||
if not too_large:
|
||||
callee_entries = set(item["callees"])
|
||||
for source, destination, call in _iter_native_calls(candidate, "outgoing"):
|
||||
# Account for every native record, even when its target cannot
|
||||
# be associated with an observed procedure.
|
||||
size += 32
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
if destination is None:
|
||||
continue
|
||||
source_procedure = item["caller"] if _hex(source) == item["caller"]["address"] else owner(source)
|
||||
target_address = _hex(destination)
|
||||
if target_address in callee_entries:
|
||||
target_entry = target_address
|
||||
else:
|
||||
target_procedure = owner(destination)
|
||||
if target_procedure is None:
|
||||
continue
|
||||
target_entry = target_procedure["address"]
|
||||
edge = {
|
||||
"source_address": _hex(source), "target_address": _hex(destination),
|
||||
"source_procedure": "caller" if source_procedure is not None and source_procedure["address"] == item["caller"]["address"] else source_procedure,
|
||||
"target_procedure": target_entry,
|
||||
"call": call,
|
||||
}
|
||||
size += len(json.dumps(edge, ensure_ascii=True).encode("utf-8")) + 1
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
item["references"].append(edge)
|
||||
if too_large:
|
||||
oversized = not items
|
||||
break
|
||||
items.append(item)
|
||||
item_bytes += size
|
||||
else:
|
||||
continue
|
||||
break
|
||||
return {"start": start, "next": start + len(items), "total": total, "items": items, "oversized": oversized}
|
||||
|
||||
|
||||
def _string_record(segment, address, display):
|
||||
"""Read one native typed object, retaining its shortened display separately."""
|
||||
length = segment.getObjectLength(address)
|
||||
@@ -1034,15 +887,6 @@ def _dispatch(method, params):
|
||||
if method in EXHAUSTIVE_ANALYSIS_METHODS:
|
||||
_api().require_analysis_complete(document, method)
|
||||
|
||||
if method == "rea_caller_scan_target":
|
||||
return _caller_scan_target(document, params)
|
||||
|
||||
if method == "rea_caller_base":
|
||||
return _caller_base(document, params)
|
||||
|
||||
if method == "rea_caller_scan_page":
|
||||
return _caller_scan_page(document, params)
|
||||
|
||||
if method == "analyze_function":
|
||||
return _analyze_function(document, params)
|
||||
if method == "read_function_instructions":
|
||||
|
||||
@@ -65,10 +65,7 @@ func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard SHA256.hash(data: executableBytes).map({ String(format: "%02x", $0) }).joined() == request.sha256 else {
|
||||
try fail("target-mismatch", "Running executable bytes differ from the active target digest")
|
||||
}
|
||||
// .optionIncludingWindow omits a selected window while it is off screen, which
|
||||
// is not absence: select from every window, and let capture report what it can.
|
||||
guard let windows = (CGWindowListCopyWindowInfo([.optionAll], kCGNullWindowID) as? [[String: Any]])?
|
||||
.filter({ ($0[kCGWindowNumber as String] as? NSNumber)?.uint32Value == request.window_id }),
|
||||
guard let windows = CGWindowListCopyWindowInfo([.optionIncludingWindow], request.window_id) as? [[String: Any]],
|
||||
windows.count == 1, let window = windows.first,
|
||||
(window[kCGWindowOwnerPID as String] as? NSNumber)?.int32Value == request.pid,
|
||||
(window[kCGWindowLayer as String] as? NSNumber)?.intValue == 0,
|
||||
|
||||
@@ -26,6 +26,7 @@ ARGUMENT_REGISTERS = {
|
||||
"x86_64h": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
|
||||
}
|
||||
MAX_SELECTOR_BYTES = 1024
|
||||
MAX_REPORTED_LOCATIONS = 64
|
||||
MAX_OTHER_STOPS = 256
|
||||
MAX_TRACE_BYTES = 8 * 1024 * 1024
|
||||
MAX_TRACE_FRAMES = 65_536
|
||||
@@ -243,20 +244,6 @@ def _hex(value):
|
||||
return "0x%x" % value
|
||||
|
||||
|
||||
def _source_location(address):
|
||||
"""Preserve available line-entry facets without acquiring source files."""
|
||||
entry = address.GetLineEntry()
|
||||
if not entry.IsValid():
|
||||
return None
|
||||
file_spec = entry.GetFileSpec()
|
||||
file_path = str(file_spec) if file_spec.IsValid() else None
|
||||
line = entry.GetLine() or None
|
||||
column = entry.GetColumn() or None
|
||||
if not file_path and line is None and column is None:
|
||||
return None
|
||||
return {"file_path": file_path or None, "line": line, "column": column}
|
||||
|
||||
|
||||
def _location(address, target):
|
||||
"""Load and file address, module and symbol of one code address."""
|
||||
module = address.GetModule()
|
||||
@@ -270,7 +257,6 @@ def _location(address, target):
|
||||
"module": module.GetFileSpec().GetFilename() if module.IsValid() else None,
|
||||
"module_path": str(module.GetFileSpec()) if module.IsValid() else None,
|
||||
"symbol": symbol.GetName() if symbol.IsValid() else None,
|
||||
"source_location": _source_location(address),
|
||||
}
|
||||
return location
|
||||
|
||||
@@ -519,19 +505,20 @@ def _breakpoint_report(target, created, config):
|
||||
if not _wanted(config["breakpoints"][index], location["symbol"] or ""):
|
||||
continue
|
||||
count += 1
|
||||
remaining_bytes = (
|
||||
MAX_BREAKPOINT_LOCATION_BYTES - retained_location_bytes
|
||||
)
|
||||
if not _fits_trace_budget(location, remaining_bytes):
|
||||
locations_truncated = True
|
||||
continue
|
||||
location_bytes = len(json.dumps(location).encode("utf-8"))
|
||||
addition = location_bytes + (1 if locations else 0)
|
||||
if addition > remaining_bytes:
|
||||
locations_truncated = True
|
||||
continue
|
||||
locations.append(location)
|
||||
retained_location_bytes += addition
|
||||
if len(locations) < MAX_REPORTED_LOCATIONS:
|
||||
remaining_bytes = (
|
||||
MAX_BREAKPOINT_LOCATION_BYTES - retained_location_bytes
|
||||
)
|
||||
if not _fits_trace_budget(location, remaining_bytes):
|
||||
locations_truncated = True
|
||||
continue
|
||||
location_bytes = len(json.dumps(location).encode("utf-8"))
|
||||
addition = location_bytes + (1 if locations else 0)
|
||||
if addition > remaining_bytes:
|
||||
locations_truncated = True
|
||||
continue
|
||||
locations.append(location)
|
||||
retained_location_bytes += addition
|
||||
report.append({"index": index, "location_count": count, "locations": locations})
|
||||
return report, locations_truncated
|
||||
|
||||
|
||||
@@ -25,45 +25,6 @@ spec.loader.exec_module(tracer)
|
||||
|
||||
|
||||
class NativeTraceBudgetTests(unittest.TestCase):
|
||||
def test_source_coordinates_preserve_partial_metadata_without_reading_source(self):
|
||||
def address(valid=True, file_valid=True, line=23, column=7):
|
||||
class SourceFileSpec:
|
||||
def IsValid(self):
|
||||
return file_valid
|
||||
|
||||
def __str__(self):
|
||||
return "/missing/é space/fixture.pas"
|
||||
|
||||
file_spec = SourceFileSpec()
|
||||
entry = types.SimpleNamespace(
|
||||
IsValid=lambda: valid,
|
||||
GetFileSpec=lambda: file_spec,
|
||||
GetLine=lambda: line,
|
||||
GetColumn=lambda: column,
|
||||
)
|
||||
return types.SimpleNamespace(GetLineEntry=lambda: entry)
|
||||
|
||||
self.assertEqual(
|
||||
tracer._source_location(address()),
|
||||
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": 7},
|
||||
)
|
||||
self.assertEqual(
|
||||
tracer._source_location(address(column=0)),
|
||||
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": None},
|
||||
)
|
||||
self.assertEqual(
|
||||
tracer._source_location(address(file_valid=False)),
|
||||
{"file_path": None, "line": 23, "column": 7},
|
||||
)
|
||||
self.assertEqual(
|
||||
tracer._source_location(address(line=0)),
|
||||
{"file_path": "/missing/é space/fixture.pas", "line": None, "column": 7},
|
||||
)
|
||||
self.assertIsNone(tracer._source_location(address(valid=False)))
|
||||
self.assertIsNone(
|
||||
tracer._source_location(address(file_valid=False, line=0, column=0))
|
||||
)
|
||||
|
||||
def test_invalid_launch_process_reports_explicit_no_creation(self):
|
||||
class InvalidProcess:
|
||||
def IsValid(self):
|
||||
@@ -429,9 +390,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
return FileSpec()
|
||||
|
||||
class Address:
|
||||
def GetLineEntry(self):
|
||||
return types.SimpleNamespace(IsValid=lambda: False)
|
||||
|
||||
def GetModule(self):
|
||||
return Module()
|
||||
|
||||
@@ -472,34 +430,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
self.assertEqual(report[0]["locations"], [])
|
||||
self.assertFalse(locations_truncated)
|
||||
|
||||
def test_breakpoint_report_keeps_all_locations_under_aggregate_budget(self):
|
||||
class Location:
|
||||
def __init__(self, index):
|
||||
self.index = index
|
||||
|
||||
def GetAddress(self):
|
||||
return self.index
|
||||
|
||||
class Breakpoint:
|
||||
def GetNumLocations(self):
|
||||
return 65
|
||||
|
||||
def GetLocationAtIndex(self, index):
|
||||
return Location(index)
|
||||
|
||||
with patch.object(
|
||||
tracer,
|
||||
"_location",
|
||||
side_effect=lambda address, _target: {"address": address, "symbol": None},
|
||||
), patch.object(tracer, "_wanted", return_value=True):
|
||||
report, locations_truncated = tracer._breakpoint_report(
|
||||
object(), [[Breakpoint()]], {"breakpoints": [{}]}
|
||||
)
|
||||
|
||||
self.assertEqual(report[0]["location_count"], 65)
|
||||
self.assertEqual(len(report[0]["locations"]), 65)
|
||||
self.assertFalse(locations_truncated)
|
||||
|
||||
def test_breakpoint_report_bounds_aggregate_location_json(self):
|
||||
tracer.lldb.LLDB_INVALID_ADDRESS = -1
|
||||
symbol_name = (
|
||||
@@ -533,24 +463,7 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
def GetFileSpec(self):
|
||||
return FileSpec()
|
||||
|
||||
class SourceFileSpec:
|
||||
def IsValid(self):
|
||||
return True
|
||||
|
||||
def __str__(self):
|
||||
return source_path
|
||||
|
||||
class Address:
|
||||
def GetLineEntry(self):
|
||||
if source_path is None:
|
||||
return types.SimpleNamespace(IsValid=lambda: False)
|
||||
return types.SimpleNamespace(
|
||||
IsValid=lambda: True,
|
||||
GetFileSpec=SourceFileSpec,
|
||||
GetLine=lambda: 15,
|
||||
GetColumn=lambda: 3,
|
||||
)
|
||||
|
||||
def GetModule(self):
|
||||
return Module()
|
||||
|
||||
@@ -577,35 +490,19 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
breakpoint_count = 1000
|
||||
created = [[Breakpoint()] for _ in range(breakpoint_count)]
|
||||
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count}
|
||||
for source_path in (None, "/tmp/" + "源🚀" * 3000 + "/calls.pas"):
|
||||
with self.subTest(source_path_present=source_path is not None):
|
||||
report, locations_truncated = tracer._breakpoint_report(
|
||||
object(), created, config
|
||||
)
|
||||
retained_bytes = sum(
|
||||
len(json.dumps(location).encode("utf-8"))
|
||||
+ (1 if index > 0 else 0)
|
||||
for row in report
|
||||
for index, location in enumerate(row["locations"])
|
||||
)
|
||||
self.assertTrue(locations_truncated)
|
||||
self.assertEqual(
|
||||
sum(row["location_count"] for row in report), 64_000
|
||||
)
|
||||
self.assertLess(
|
||||
sum(len(row["locations"]) for row in report), 64_000
|
||||
)
|
||||
self.assertLessEqual(
|
||||
retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES
|
||||
)
|
||||
if source_path is not None:
|
||||
self.assertTrue(
|
||||
any(
|
||||
location["source_location"]["file_path"] == source_path
|
||||
for row in report
|
||||
for location in row["locations"]
|
||||
)
|
||||
)
|
||||
report, locations_truncated = tracer._breakpoint_report(
|
||||
object(), created, config
|
||||
)
|
||||
retained_bytes = sum(
|
||||
len(json.dumps(location).encode("utf-8"))
|
||||
+ (1 if index > 0 else 0)
|
||||
for row in report
|
||||
for index, location in enumerate(row["locations"])
|
||||
)
|
||||
self.assertTrue(locations_truncated)
|
||||
self.assertEqual(sum(row["location_count"] for row in report), 64_000)
|
||||
self.assertLess(sum(len(row["locations"]) for row in report), 64_000)
|
||||
self.assertLessEqual(retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -15,8 +15,6 @@ struct ProcessIdentityObservation: Encodable {
|
||||
let effective_uid: UInt32?
|
||||
let lifecycle: String?
|
||||
let relation: ProcessRelationObservation?
|
||||
/** Display-only kernel accounting name (p_comm, at most MAXCOMLEN bytes). */
|
||||
let name: String?
|
||||
let reason: String?
|
||||
}
|
||||
|
||||
@@ -33,7 +31,6 @@ struct ProcessMetadata {
|
||||
let parentPid: Int32
|
||||
let processGroupId: Int32
|
||||
let pointerSize: Int
|
||||
let name: String
|
||||
}
|
||||
|
||||
struct ProcessSnapshot {
|
||||
@@ -95,8 +92,7 @@ func readProcessSnapshot(
|
||||
lifecycle: process.kp_proc.p_stat == SZOMB ? "zombie" : "non-zombie",
|
||||
parentPid: process.kp_eproc.e_ppid,
|
||||
processGroupId: process.kp_eproc.e_pgid,
|
||||
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4,
|
||||
name: processName(process)
|
||||
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4
|
||||
)
|
||||
)
|
||||
},
|
||||
@@ -107,13 +103,6 @@ func readProcessSnapshot(
|
||||
return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(ENOMEM)")
|
||||
}
|
||||
|
||||
/** Decode the NUL-terminated accounting name; invalid UTF-8 is replaced for display. */
|
||||
private func processName(_ process: kinfo_proc) -> String {
|
||||
withUnsafeBytes(of: process.kp_proc.p_comm) { bytes in
|
||||
String(decoding: bytes.prefix { $0 != 0 }, as: UTF8.self)
|
||||
}
|
||||
}
|
||||
|
||||
func readProcessIdentities(pids: [Int32]) -> [ProcessIdentityObservation] {
|
||||
readProcessIdentities(pids: pids, snapshot: readProcessSnapshot())
|
||||
}
|
||||
@@ -141,7 +130,6 @@ func readProcessIdentities(
|
||||
effective_uid: nil,
|
||||
lifecycle: nil,
|
||||
relation: nil,
|
||||
name: nil,
|
||||
reason: "not-in-snapshot"
|
||||
)
|
||||
}
|
||||
@@ -156,7 +144,6 @@ func readProcessIdentities(
|
||||
parent_pid: metadata.parentPid,
|
||||
process_group_id: metadata.processGroupId
|
||||
),
|
||||
name: metadata.name,
|
||||
reason: nil
|
||||
)
|
||||
}
|
||||
@@ -170,25 +157,12 @@ private func unavailableIdentity(pid: Int32, reason: String) -> ProcessIdentityO
|
||||
effective_uid: nil,
|
||||
lifecycle: nil,
|
||||
relation: nil,
|
||||
name: nil,
|
||||
reason: reason
|
||||
)
|
||||
}
|
||||
|
||||
enum ProcargsMalformedStage: String, CaseIterable {
|
||||
case header
|
||||
case argumentCount = "argument_count"
|
||||
case executablePath = "executable_path"
|
||||
case pointerSize = "pointer_size"
|
||||
case pathPadding = "path_padding"
|
||||
case arguments
|
||||
case terminator
|
||||
case trailingEntries = "trailing_entries"
|
||||
}
|
||||
|
||||
enum RunTokenReadError: Error {
|
||||
case invalidBuffer
|
||||
case invalidBufferAt(ProcargsMalformedStage)
|
||||
case duplicateToken
|
||||
case invalidTokenEncoding
|
||||
case environmentUnavailable
|
||||
@@ -228,22 +202,6 @@ func processRunTokenObservation(
|
||||
parse: Result<ParsedRunToken, RunTokenReadError>,
|
||||
codeSigningFlags: () -> UInt32?
|
||||
) -> RunTokenObservation {
|
||||
// Native inspection restrictions apply even when argv is incomplete or
|
||||
// overwritten, before parsing reaches the environment. Keep this fact at
|
||||
// the native boundary; cleanup still decides whether the process is related.
|
||||
func withheldEnvironment() -> RunTokenObservation? {
|
||||
guard let flags = codeSigningFlags() else { return nil }
|
||||
if (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func unavailableEnvironment(_ reason: String) -> RunTokenObservation {
|
||||
withheldEnvironment() ?? unavailable(pid: pid, reason: reason)
|
||||
}
|
||||
switch parse {
|
||||
case .success(let parsed):
|
||||
if let runId = parsed.runId {
|
||||
@@ -254,25 +212,38 @@ func processRunTokenObservation(
|
||||
if parsed.environmentEntryCount > 0 {
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
}
|
||||
if let withheld = withheldEnvironment() { return withheld }
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
case .failure(let failure):
|
||||
switch failure {
|
||||
case .environmentUnavailable:
|
||||
return unavailableEnvironment("environment_unavailable")
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "environment_unavailable")
|
||||
case .ambiguousEnvironmentBoundary:
|
||||
return unavailableEnvironment("ambiguous_environment_boundary")
|
||||
// These failures already establish that token bytes were available.
|
||||
if let flags = codeSigningFlags(), (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "ambiguous_environment_boundary")
|
||||
case .duplicateToken:
|
||||
return unavailable(pid: pid, reason: "duplicate_run_token")
|
||||
case .invalidTokenEncoding:
|
||||
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
|
||||
case .appleVectorUnavailable:
|
||||
return unavailableEnvironment("apple_vector_unavailable")
|
||||
return unavailable(pid: pid, reason: "apple_vector_unavailable")
|
||||
case .invalidBuffer:
|
||||
return unavailableEnvironment("malformed_procargs")
|
||||
case .invalidBufferAt(let stage):
|
||||
return unavailableEnvironment("malformed_procargs_\(stage.rawValue)")
|
||||
return unavailable(pid: pid, reason: "malformed_procargs")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -331,30 +302,27 @@ func parseProcessEnvironment(
|
||||
in bytes: [UInt8],
|
||||
pointerSize: Int = MemoryLayout<UnsafeRawPointer>.size
|
||||
) throws -> ParsedRunToken {
|
||||
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBufferAt(.header) }
|
||||
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBuffer }
|
||||
let argc = bytes.withUnsafeBytes { $0.loadUnaligned(as: Int32.self).littleEndian }
|
||||
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBufferAt(.argumentCount) }
|
||||
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBuffer }
|
||||
var offset = MemoryLayout<Int32>.size
|
||||
guard let executablePath = readString(in: bytes, offset: &offset) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.executablePath)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBufferAt(.pointerSize) }
|
||||
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBuffer }
|
||||
let pathRecordSize = Array("executable_path=".utf8).count + executablePath.count + 1
|
||||
let padding = (pointerSize - pathRecordSize % pointerSize) % pointerSize
|
||||
guard offset + padding <= bytes.count,
|
||||
bytes[offset..<(offset + padding)].allSatisfy({ $0 == 0 }) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.pathPadding)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
offset += padding
|
||||
let tokenPrefix = Array("REA_PROCESS_RUN_ID=".utf8)
|
||||
var tokenInArgument = false
|
||||
for _ in 0..<argc {
|
||||
guard let argument = readString(in: bytes, offset: &offset, allowEmpty: true) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.arguments)
|
||||
guard readString(in: bytes, offset: &offset, allowEmpty: true) != nil else {
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
tokenInArgument = tokenInArgument || argument.starts(with: tokenPrefix)
|
||||
}
|
||||
guard bytes.last == 0 else { throw RunTokenReadError.invalidBufferAt(.terminator) }
|
||||
guard bytes.last == 0 else { throw RunTokenReadError.invalidBuffer }
|
||||
if offset == bytes.count {
|
||||
throw RunTokenReadError.environmentUnavailable
|
||||
}
|
||||
@@ -372,7 +340,7 @@ func parseProcessEnvironment(
|
||||
continue
|
||||
}
|
||||
guard let entry = readString(in: bytes, offset: &offset) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.trailingEntries)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
trailingEntries.append(entry)
|
||||
}
|
||||
@@ -393,28 +361,22 @@ func parseProcessEnvironment(
|
||||
throw RunTokenReadError.appleVectorUnavailable
|
||||
}
|
||||
if trailingEntries[appleVectorStart...].contains(where: {
|
||||
$0.starts(with: tokenPrefix)
|
||||
$0.starts(with: Array("REA_PROCESS_RUN_ID=".utf8))
|
||||
}) {
|
||||
throw RunTokenReadError.ambiguousEnvironmentBoundary
|
||||
}
|
||||
let environmentEntries = trailingEntries[..<appleVectorStart]
|
||||
var found: String?
|
||||
for entry in environmentEntries {
|
||||
if entry.starts(with: tokenPrefix) {
|
||||
if entry.starts(with: Array("REA_PROCESS_RUN_ID=".utf8)) {
|
||||
guard found == nil else { throw RunTokenReadError.duplicateToken }
|
||||
let value = entry.dropFirst(tokenPrefix.count)
|
||||
let value = entry.dropFirst("REA_PROCESS_RUN_ID=".utf8.count)
|
||||
guard let decoded = String(bytes: value, encoding: .utf8) else {
|
||||
throw RunTokenReadError.invalidTokenEncoding
|
||||
}
|
||||
found = decoded
|
||||
}
|
||||
}
|
||||
// A changed process title can overwrite the original argument boundaries,
|
||||
// causing a real environment entry to be consumed as an argument. A real
|
||||
// argument can have this form too, so it proves neither ownership nor absence.
|
||||
if tokenInArgument {
|
||||
throw RunTokenReadError.ambiguousEnvironmentBoundary
|
||||
}
|
||||
return ParsedRunToken(runId: found, environmentEntryCount: environmentEntries.count)
|
||||
}
|
||||
|
||||
|
||||
@@ -109,7 +109,7 @@ class CoreContextTests(unittest.TestCase):
|
||||
self.assertFalse(result["ok"])
|
||||
self.assertEqual(result["reason"], "output-limit")
|
||||
self.assertNotIn("exceeds64", result["message"])
|
||||
self.assertIn("64 MiB", result["message"])
|
||||
self.assertEqual(result["message"], "Complete debugger context exceeds the 64 MiB reply budget.")
|
||||
self.assertEqual(reads, [])
|
||||
self.assertEqual(queries, ["maps"])
|
||||
|
||||
|
||||
@@ -16,10 +16,10 @@ PROFILE = "[email protected];[email protected];[email protected]"
|
||||
PACKAGES = {"pwntools": "4.15.0", "pyelftools": "0.33", "unicorn": "2.1.2"}
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
# (e_machine, ELF class bits, byte order) -> architecture pwntools must also report.
|
||||
PROFILES = {(62, 64, "little"): "amd64", (183, 64, "little"): "aarch64", (3, 32, "little"): "i386",
|
||||
(40, 32, "little"): "arm", (8, 32, "little"): "mips", (8, 32, "big"): "mips"}
|
||||
PROFILE_SCOPE = ("Layout profile requires ELF64 x86-64, ELF64 AArch64 little-endian, ELF32 i386, "
|
||||
"ELF32 ARM little-endian or ELF32 MIPS little- or big-endian objects.")
|
||||
PROFILES = {(62, 64, "little"): "amd64", (3, 32, "little"): "i386", (40, 32, "little"): "arm",
|
||||
(8, 32, "little"): "mips", (8, 32, "big"): "mips"}
|
||||
PROFILE_SCOPE = ("Layout profile requires ELF64 x86-64, ELF32 i386, ELF32 ARM little-endian "
|
||||
"or ELF32 MIPS little- or big-endian objects.")
|
||||
|
||||
|
||||
class LayoutFailure(Exception):
|
||||
|
||||
+91
-88
@@ -1,18 +1,4 @@
|
||||
import { defineConfig } from "vitepress";
|
||||
import { dirname, relative, resolve, sep } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import {
|
||||
documentationRewrites,
|
||||
documentationSections,
|
||||
} from "./documentation.js";
|
||||
|
||||
const documentationRoot = fileURLToPath(new URL("..", import.meta.url));
|
||||
const rewrites: Record<string, string> = {
|
||||
...documentationRewrites,
|
||||
"README.md": "documentation.md",
|
||||
"specs/README.md": "specs/index.md",
|
||||
"adr/README.md": "adr/index.md",
|
||||
};
|
||||
|
||||
export default defineConfig({
|
||||
title: "REA",
|
||||
@@ -20,92 +6,109 @@ export default defineConfig({
|
||||
"Reverse engineer binaries, applications, and runtime behavior with one CLI and MCP server.",
|
||||
lang: "en-US",
|
||||
base: "/rea/",
|
||||
rewrites,
|
||||
markdown: {
|
||||
config(md) {
|
||||
// Repository links resolve from their physical source. Render them from
|
||||
// the stable public route instead, including links across categories.
|
||||
md.core.ruler.after("inline", "documentation-source-links", (state) => {
|
||||
const physicalPath =
|
||||
state.env.realPath ??
|
||||
(state.env.relativePath
|
||||
? resolve(documentationRoot, state.env.relativePath)
|
||||
: null);
|
||||
if (physicalPath === null) return;
|
||||
const renderedPath =
|
||||
state.env.path ??
|
||||
resolve(
|
||||
documentationRoot,
|
||||
rewrites[state.env.relativePath] ?? state.env.relativePath,
|
||||
);
|
||||
for (const block of state.tokens) {
|
||||
for (const token of block.children ?? []) {
|
||||
const attribute =
|
||||
token.type === "link_open"
|
||||
? "href"
|
||||
: token.type === "image"
|
||||
? "src"
|
||||
: null;
|
||||
if (attribute === null) continue;
|
||||
const destination = token.attrGet(attribute);
|
||||
if (
|
||||
!destination ||
|
||||
/^(?:[a-z][a-z\d+.-]*:|[/#])/iu.test(destination)
|
||||
)
|
||||
continue;
|
||||
const separator = destination.search(/[?#]/u);
|
||||
const path =
|
||||
separator === -1 ? destination : destination.slice(0, separator);
|
||||
const suffix = separator === -1 ? "" : destination.slice(separator);
|
||||
const source = relative(
|
||||
documentationRoot,
|
||||
resolve(dirname(physicalPath), path),
|
||||
)
|
||||
.split(sep)
|
||||
.join("/");
|
||||
const rendered = relative(
|
||||
dirname(renderedPath),
|
||||
resolve(
|
||||
documentationRoot,
|
||||
source.startsWith("public/")
|
||||
? source.slice("public/".length)
|
||||
: (rewrites[source] ?? source),
|
||||
),
|
||||
)
|
||||
.split(sep)
|
||||
.join("/");
|
||||
token.attrSet(attribute, rendered + suffix);
|
||||
}
|
||||
}
|
||||
});
|
||||
},
|
||||
},
|
||||
srcExclude: ["readme/**"],
|
||||
themeConfig: {
|
||||
nav: [
|
||||
{ text: "Get started", link: "/installation" },
|
||||
{ text: "Documentation", link: "/documentation" },
|
||||
{ text: "Reference", link: "/mcp-contracts" },
|
||||
{ text: "Roadmap", link: "/roadmap" },
|
||||
],
|
||||
sidebar: [
|
||||
{
|
||||
text: "Documentation directory",
|
||||
text: "Getting started",
|
||||
items: [
|
||||
{ text: "All guides", link: "/documentation" },
|
||||
{ text: "Authoring conventions", link: "/development/documentation" },
|
||||
{ text: "Feature specifications", link: "/specs/" },
|
||||
{ text: "Architecture decisions", link: "/adr/" },
|
||||
{ text: "Installation and setup", link: "/installation" },
|
||||
{ text: "Client configuration", link: "/client-configuration" },
|
||||
{ text: "CLI and Evidence", link: "/cli" },
|
||||
{ text: "Agent prompts", link: "/mcp-prompts" },
|
||||
{ text: "Roadmap", link: "/roadmap" },
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Investigation guides",
|
||||
items: [
|
||||
{ text: "Native binaries", link: "/native-investigation" },
|
||||
{ text: "Go build metadata", link: "/go-binaries" },
|
||||
{ text: "IL2CPP metadata", link: "/il2cpp-metadata" },
|
||||
{ text: "Model artifacts", link: "/model-artifacts" },
|
||||
{ text: "Rust dependencies", link: "/rust-binaries" },
|
||||
{ text: "PE certificates", link: "/pe-certificates" },
|
||||
{ text: "PE imports", link: "/pe-imports" },
|
||||
{ text: "PE TLS/load config", link: "/pe-load-config" },
|
||||
{ text: "ELF notes", link: "/elf-notes" },
|
||||
{ text: "Apple IPS crash reports", link: "/apple-ips" },
|
||||
{ text: "Linux kernel modules", link: "/kernel-modules" },
|
||||
{ text: "CPython pyc files", link: "/python-pyc" },
|
||||
{ text: "Erlang BEAM", link: "/erlang-beam" },
|
||||
{
|
||||
text: "JavaScript and Electron",
|
||||
link: "/javascript-artifact-reconstruction",
|
||||
},
|
||||
{
|
||||
text: "JavaScript workflows",
|
||||
link: "/javascript-application-workflows",
|
||||
},
|
||||
{ text: "Browser observation", link: "/browser-observation" },
|
||||
{
|
||||
text: "Web network captures",
|
||||
link: "/web-network-captures",
|
||||
},
|
||||
{
|
||||
text: "Build RL environments",
|
||||
link: "/agent-environments-quickstart",
|
||||
},
|
||||
{ text: "Environment reference", link: "/agent-environments" },
|
||||
{ text: "Electron observation", link: "/electron-observation" },
|
||||
{ text: "Electron fuses", link: "/electron-fuses" },
|
||||
{ text: "Process capture", link: "/process-capture" },
|
||||
{ text: ".NET assemblies", link: "/managed-code-analysis" },
|
||||
{
|
||||
text: "Python packaged applications",
|
||||
link: "/python-packaged-applications",
|
||||
},
|
||||
{ text: "Android applications", link: "/android-analysis" },
|
||||
{ text: "Apple applications", link: "/apple-application-analysis" },
|
||||
{ text: "Desktop installations", link: "/desktop-applications" },
|
||||
{ text: "Firmware", link: "/firmware-analysis" },
|
||||
{ text: "SQLite databases", link: "/sqlite-databases" },
|
||||
{ text: "Packet captures", link: "/packet-captures" },
|
||||
{ text: "Signal recordings", link: "/signal-recordings" },
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Provider guides",
|
||||
collapsed: true,
|
||||
items: [
|
||||
{ text: "IDA", link: "/ida-provider" },
|
||||
{ text: "Kuna", link: "/kuna-provider" },
|
||||
{ text: "Ghidra on Windows", link: "/windows-ghidra-p0" },
|
||||
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
|
||||
{ text: "Ghidra raw binary", link: "/raw-binary" },
|
||||
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
|
||||
{ text: "Ghidra PowerPC", link: "/ghidra-ppc" },
|
||||
{ text: "Ghidra GameCube DOL", link: "/ghidra-dol" },
|
||||
{ text: "Ghidra semantics", link: "/ghidra-provider" },
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Reference and development",
|
||||
collapsed: true,
|
||||
items: [
|
||||
{ text: "MCP contracts", link: "/mcp-contracts" },
|
||||
{ text: "Tool design", link: "/tool-design" },
|
||||
{
|
||||
text: "Admission limits and recovery",
|
||||
link: "/design/limits-and-recovery",
|
||||
},
|
||||
{ text: "Testing", link: "/testing" },
|
||||
{ text: "Client verification", link: "/client-verification" },
|
||||
{
|
||||
text: "Reconstruction readiness",
|
||||
link: "/reconstruction-readiness",
|
||||
},
|
||||
{ text: "Releasing", link: "/releasing" },
|
||||
],
|
||||
},
|
||||
...documentationSections.map((section) => ({
|
||||
text: section.text,
|
||||
collapsed: section.text !== "Getting started",
|
||||
items: section.items.map((item) => ({
|
||||
text: item.text,
|
||||
link: item.route,
|
||||
})),
|
||||
})),
|
||||
],
|
||||
search: { provider: "local" },
|
||||
socialLinks: [
|
||||
|
||||
@@ -1,654 +0,0 @@
|
||||
// Authored guide ownership: physical sources and stable public routes.
|
||||
export const documentationSections = [
|
||||
{
|
||||
text: "Getting started",
|
||||
items: [
|
||||
{
|
||||
text: "Installation and setup",
|
||||
source: "getting-started/installation.md",
|
||||
route: "/installation",
|
||||
},
|
||||
{
|
||||
text: "Client configuration and troubleshooting",
|
||||
source: "getting-started/client-configuration.md",
|
||||
route: "/client-configuration",
|
||||
},
|
||||
{
|
||||
text: "CLI and Evidence",
|
||||
source: "getting-started/cli.md",
|
||||
route: "/cli",
|
||||
},
|
||||
{
|
||||
text: "Guided MCP prompts and completion",
|
||||
source: "getting-started/mcp-prompts.md",
|
||||
route: "/mcp-prompts",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Native binaries",
|
||||
items: [
|
||||
{
|
||||
text: "Native binary investigation",
|
||||
source: "guides/native/native-investigation.md",
|
||||
route: "/native-investigation",
|
||||
},
|
||||
{
|
||||
text: "Go binary build metadata",
|
||||
source: "guides/native/go-binaries.md",
|
||||
route: "/go-binaries",
|
||||
},
|
||||
{
|
||||
text: "Offline binary diagnostics",
|
||||
source: "guides/native/binary-diagnostics.md",
|
||||
route: "/binary-diagnostics",
|
||||
},
|
||||
{
|
||||
text: "Recorded crash evidence",
|
||||
source: "guides/native/recorded-crashes.md",
|
||||
route: "/recorded-crashes",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "JavaScript and Electron",
|
||||
items: [
|
||||
{
|
||||
text: "JavaScript Application Graph",
|
||||
source: "guides/javascript/javascript-application-graph.md",
|
||||
route: "/javascript-application-graph",
|
||||
},
|
||||
{
|
||||
text: "Cross-layer JavaScript application workflows",
|
||||
source: "guides/javascript/javascript-application-workflows.md",
|
||||
route: "/javascript-application-workflows",
|
||||
},
|
||||
{
|
||||
text: "JavaScript artifact reconstruction",
|
||||
source: "guides/javascript/javascript-artifact-reconstruction.md",
|
||||
route: "/javascript-artifact-reconstruction",
|
||||
},
|
||||
{
|
||||
text: "JavaScript source recovery",
|
||||
source: "guides/javascript/javascript-recovery.md",
|
||||
route: "/javascript-recovery",
|
||||
},
|
||||
{
|
||||
text: "Passive Node and Electron runtime observation",
|
||||
source: "guides/javascript/javascript-runtime-observation.md",
|
||||
route: "/javascript-runtime-observation",
|
||||
},
|
||||
{
|
||||
text: "JavaScript static/runtime reconciliation",
|
||||
source: "guides/javascript/javascript-runtime-reconciliation.md",
|
||||
route: "/javascript-runtime-reconciliation",
|
||||
},
|
||||
{
|
||||
text: "Electron file-page observation",
|
||||
source: "guides/javascript/electron-observation.md",
|
||||
route: "/electron-observation",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Browser and captured websites",
|
||||
items: [
|
||||
{
|
||||
text: "Website observation with CDP",
|
||||
source: "guides/web/browser-observation.md",
|
||||
route: "/browser-observation",
|
||||
},
|
||||
{
|
||||
text: "Browser scenario contract",
|
||||
source: "guides/web/browser-scenario-contract.md",
|
||||
route: "/browser-scenario-contract",
|
||||
},
|
||||
{
|
||||
text: "Website runtime source attribution",
|
||||
source: "guides/web/web-runtime.md",
|
||||
route: "/web-runtime",
|
||||
},
|
||||
{
|
||||
text: "Historical web network captures",
|
||||
source: "guides/web/web-network-captures.md",
|
||||
route: "/web-network-captures",
|
||||
},
|
||||
{
|
||||
text: "HTTP/3 capture lab",
|
||||
source: "http3-capture-lab.md",
|
||||
route: "/http3-capture-lab",
|
||||
},
|
||||
{
|
||||
text: "Captured website source locations",
|
||||
source: "guides/web/web-source-location.md",
|
||||
route: "/web-source-location",
|
||||
},
|
||||
{
|
||||
text: "Captured website module relationships",
|
||||
source: "guides/web/website-module-trace.md",
|
||||
route: "/website-module-trace",
|
||||
},
|
||||
{
|
||||
text: "Captured website scripts and static JavaScript analysis",
|
||||
source: "guides/web/website-script-export.md",
|
||||
route: "/website-script-export",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Android applications",
|
||||
items: [
|
||||
{
|
||||
text: "Static Android APK analysis",
|
||||
source: "guides/android/android-analysis.md",
|
||||
route: "/android-analysis",
|
||||
},
|
||||
{
|
||||
text: "Apktool resource decoding",
|
||||
source: "guides/android/apktool-resource-analysis.md",
|
||||
route: "/apktool-resource-analysis",
|
||||
},
|
||||
{
|
||||
text: "ADB device inspection and acquisition",
|
||||
source: "guides/android/adb-device-analysis.md",
|
||||
route: "/adb-device-analysis",
|
||||
},
|
||||
{
|
||||
text: "Flutter build identification",
|
||||
source: "guides/android/flutter-analysis.md",
|
||||
route: "/flutter-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Apple applications",
|
||||
items: [
|
||||
{
|
||||
text: "Apple Application Inventory Projection",
|
||||
source: "guides/apple/apple-application-analysis.md",
|
||||
route: "/apple-application-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: ".NET assemblies",
|
||||
items: [
|
||||
{
|
||||
text: "Managed-code analysis",
|
||||
source: "guides/managed/managed-code-analysis.md",
|
||||
route: "/managed-code-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Firmware",
|
||||
items: [
|
||||
{
|
||||
text: "Firmware analysis",
|
||||
source: "guides/firmware/firmware-analysis.md",
|
||||
route: "/firmware-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "HarmonyOS packages",
|
||||
items: [
|
||||
{
|
||||
text: "Static HarmonyOS package inventory",
|
||||
source: "guides/harmony/harmony-analysis.md",
|
||||
route: "/harmony-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "WebAssembly",
|
||||
items: [
|
||||
{
|
||||
text: "Offline WebAssembly artifacts",
|
||||
source: "guides/wasm/wasm-artifacts.md",
|
||||
route: "/wasm-artifacts",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "EVM contracts",
|
||||
items: [
|
||||
{
|
||||
text: "Offline EVM bytecode",
|
||||
source: "guides/evm/evm-bytecode.md",
|
||||
route: "/evm-bytecode",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Databases and recorded packets",
|
||||
items: [
|
||||
{
|
||||
text: "SQLite database snapshots",
|
||||
source: "guides/data/sqlite-databases.md",
|
||||
route: "/sqlite-databases",
|
||||
},
|
||||
{
|
||||
text: "Recorded packet capture inspection",
|
||||
source: "guides/data/packet-captures.md",
|
||||
route: "/packet-captures",
|
||||
},
|
||||
{
|
||||
text: "Packet capture summaries",
|
||||
source: "guides/data/pcap-analysis.md",
|
||||
route: "/pcap-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Process capture and instrumentation",
|
||||
items: [
|
||||
{
|
||||
text: "Process capture",
|
||||
source: "guides/runtime/process-capture.md",
|
||||
route: "/process-capture",
|
||||
},
|
||||
{
|
||||
text: "Frida instrumentation",
|
||||
source: "guides/runtime/frida-instrumentation.md",
|
||||
route: "/frida-instrumentation",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Provider setup and formats",
|
||||
items: [
|
||||
{
|
||||
text: "Ghidra provider semantics",
|
||||
source: "providers/ghidra-provider.md",
|
||||
route: "/ghidra-provider",
|
||||
},
|
||||
{
|
||||
text: "DOS MZ and COM analysis with Ghidra",
|
||||
source: "providers/ghidra-dos.md",
|
||||
route: "/ghidra-dos",
|
||||
},
|
||||
{
|
||||
text: "Optional NativeAOT metadata recovery",
|
||||
source: "providers/ghidra-nativeaot.md",
|
||||
route: "/ghidra-nativeaot",
|
||||
},
|
||||
{
|
||||
text: "MIPS ELF analysis through Ghidra",
|
||||
source: "providers/ghidra-mips.md",
|
||||
route: "/ghidra-mips",
|
||||
},
|
||||
{
|
||||
text: "PSP static ELF analysis through Ghidra",
|
||||
source: "providers/ghidra-psp.md",
|
||||
route: "/ghidra-psp",
|
||||
},
|
||||
{
|
||||
text: "RISC-V ELF analysis through Ghidra",
|
||||
source: "providers/ghidra-riscv.md",
|
||||
route: "/ghidra-riscv",
|
||||
},
|
||||
{
|
||||
text: "Windows Ghidra P0",
|
||||
source: "providers/windows-ghidra-p0.md",
|
||||
route: "/windows-ghidra-p0",
|
||||
},
|
||||
{
|
||||
text: "Headerless raw-binary analysis with Ghidra",
|
||||
source: "providers/raw-binary.md",
|
||||
route: "/raw-binary",
|
||||
},
|
||||
{
|
||||
text: "IDA provider",
|
||||
source: "providers/ida-provider.md",
|
||||
route: "/ida-provider",
|
||||
},
|
||||
{
|
||||
text: "JEB instrumentation and analysis",
|
||||
source: "providers/jeb-analysis.md",
|
||||
route: "/jeb-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Contracts and evidence",
|
||||
items: [
|
||||
{
|
||||
text: "MCP runtime contracts",
|
||||
source: "reference/mcp-contracts.md",
|
||||
route: "/mcp-contracts",
|
||||
},
|
||||
{
|
||||
text: "Reconstruction obligation ledgers",
|
||||
source: "reference/reconstruction-obligation-ledgers.md",
|
||||
route: "/reconstruction-obligation-ledgers",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Development and verification",
|
||||
items: [
|
||||
{
|
||||
text: "Settings and recovery",
|
||||
source: "design/limits-and-recovery.md",
|
||||
route: "/design/limits-and-recovery",
|
||||
},
|
||||
{
|
||||
text: "MCP tool design",
|
||||
source: "development/tool-design.md",
|
||||
route: "/tool-design",
|
||||
},
|
||||
{
|
||||
text: "Testing REA",
|
||||
source: "development/testing.md",
|
||||
route: "/testing",
|
||||
},
|
||||
{
|
||||
text: "Client and agent verification",
|
||||
source: "development/client-verification.md",
|
||||
route: "/client-verification",
|
||||
},
|
||||
{
|
||||
text: "Reconstruction readiness conformance",
|
||||
source: "development/reconstruction-readiness.md",
|
||||
route: "/reconstruction-readiness",
|
||||
},
|
||||
{
|
||||
text: "Releasing REA",
|
||||
source: "development/releasing.md",
|
||||
route: "/releasing",
|
||||
},
|
||||
{
|
||||
text: "REA roadmap",
|
||||
source: "development/roadmap.md",
|
||||
route: "/roadmap",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Release migrations",
|
||||
items: [
|
||||
{
|
||||
text: "Upgrading from REA 6.1 to 6.2",
|
||||
source: "migrations/migration-6.2.md",
|
||||
route: "/migration-6.2",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Signal recordings",
|
||||
items: [
|
||||
{
|
||||
text: "Recorded signal inspection",
|
||||
source: "guides/signals/signal-recordings.md",
|
||||
route: "/signal-recordings",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Additional guides",
|
||||
items: [
|
||||
{
|
||||
text: "Offline API contract inventory",
|
||||
source: "api-contracts.md",
|
||||
route: "/api-contracts",
|
||||
},
|
||||
{
|
||||
text: "Bun standalone executables",
|
||||
source: "bun-standalone.md",
|
||||
route: "/bun-standalone",
|
||||
},
|
||||
{
|
||||
text: "Capability composition and ownership",
|
||||
source: "capability-composition.md",
|
||||
route: "/capability-composition",
|
||||
},
|
||||
{
|
||||
text: ".NET single-file bundles",
|
||||
source: "dotnet-bundles.md",
|
||||
route: "/dotnet-bundles",
|
||||
},
|
||||
{
|
||||
text: "ESP flash images",
|
||||
source: "esp-flash-image.md",
|
||||
route: "/esp-flash-image",
|
||||
},
|
||||
{
|
||||
text: "U-Boot FIT images",
|
||||
source: "fit-images.md",
|
||||
route: "/fit-images",
|
||||
},
|
||||
{
|
||||
text: "Measure MCP catalog costs in Codex",
|
||||
source: "mcp-catalog-evaluation.md",
|
||||
route: "/mcp-catalog-evaluation",
|
||||
},
|
||||
{
|
||||
text: "Offline Playwright traces",
|
||||
source: "playwright-traces.md",
|
||||
route: "/playwright-traces",
|
||||
},
|
||||
{
|
||||
text: "Testing lanes and provider verification",
|
||||
source: "testing-lanes.md",
|
||||
route: "/testing-lanes",
|
||||
},
|
||||
{
|
||||
text: "UF2 and DfuSe images",
|
||||
source: "uf2-dfuse.md",
|
||||
route: "/uf2-dfuse",
|
||||
},
|
||||
{
|
||||
text: "WeChat mini-program wxapkg archives",
|
||||
source: "wxapkg.md",
|
||||
route: "/wxapkg",
|
||||
},
|
||||
{
|
||||
text: "Binary hardening",
|
||||
source: "binary-hardening.md",
|
||||
route: "/binary-hardening",
|
||||
},
|
||||
{
|
||||
text: "Desktop application capture",
|
||||
source: "desktop-application-capture.md",
|
||||
route: "/desktop-application-capture",
|
||||
},
|
||||
{
|
||||
text: "Desktop application export",
|
||||
source: "desktop-application-export.md",
|
||||
route: "/desktop-application-export",
|
||||
},
|
||||
{
|
||||
text: "Desktop application state",
|
||||
source: "desktop-application-state.md",
|
||||
route: "/desktop-application-state",
|
||||
},
|
||||
{
|
||||
text: "Desktop application tasks",
|
||||
source: "desktop-application-tasks.md",
|
||||
route: "/desktop-application-tasks",
|
||||
},
|
||||
{
|
||||
text: "Desktop build computer",
|
||||
source: "desktop-build-computer.md",
|
||||
route: "/desktop-build-computer",
|
||||
},
|
||||
{
|
||||
text: "Desktop build witness",
|
||||
source: "desktop-build-witness.md",
|
||||
route: "/desktop-build-witness",
|
||||
},
|
||||
{
|
||||
text: "iNES containers",
|
||||
source: "ines-containers.md",
|
||||
route: "/ines-containers",
|
||||
},
|
||||
{
|
||||
text: "Build RL environments with REA",
|
||||
source: "agent-environments-quickstart.md",
|
||||
route: "/agent-environments-quickstart",
|
||||
},
|
||||
{
|
||||
text: "Agent environments from recorded captures",
|
||||
source: "agent-environments.md",
|
||||
route: "/agent-environments",
|
||||
},
|
||||
{
|
||||
text: "APK signature schemes",
|
||||
source: "apk-signature.md",
|
||||
route: "/apk-signature",
|
||||
},
|
||||
{
|
||||
text: "Apple IPS crash reports",
|
||||
source: "apple-ips.md",
|
||||
route: "/apple-ips",
|
||||
},
|
||||
{
|
||||
text: "Static analysis in GitHub Actions",
|
||||
source: "ci-analysis.md",
|
||||
route: "/ci-analysis",
|
||||
},
|
||||
{
|
||||
text: "Desktop installation directories",
|
||||
source: "desktop-applications.md",
|
||||
route: "/desktop-applications",
|
||||
},
|
||||
{
|
||||
text: "Flattened devicetree inspection",
|
||||
source: "devicetree.md",
|
||||
route: "/devicetree",
|
||||
},
|
||||
{
|
||||
text: "Electron fuse state",
|
||||
source: "electron-fuses.md",
|
||||
route: "/electron-fuses",
|
||||
},
|
||||
{ text: "ELF note tables", source: "elf-notes.md", route: "/elf-notes" },
|
||||
{
|
||||
text: "Erlang BEAM files",
|
||||
source: "erlang-beam.md",
|
||||
route: "/erlang-beam",
|
||||
},
|
||||
{ text: "ESP images", source: "esp-image.md", route: "/esp-image" },
|
||||
{
|
||||
text: "GameCube DOL analysis through Ghidra",
|
||||
source: "ghidra-dol.md",
|
||||
route: "/ghidra-dol",
|
||||
},
|
||||
{
|
||||
text: "PowerPC ELF analysis through Ghidra",
|
||||
source: "ghidra-ppc.md",
|
||||
route: "/ghidra-ppc",
|
||||
},
|
||||
{
|
||||
text: "Godot PCK archives",
|
||||
source: "godot-pck.md",
|
||||
route: "/godot-pck",
|
||||
},
|
||||
{
|
||||
text: "Static CUDA and HIP code objects",
|
||||
source: "gpu-code-objects.md",
|
||||
route: "/gpu-code-objects",
|
||||
},
|
||||
{
|
||||
text: "Intel HEX and S-record images",
|
||||
source: "hex-records.md",
|
||||
route: "/hex-records",
|
||||
},
|
||||
{
|
||||
text: "IL2CPP metadata",
|
||||
source: "il2cpp-metadata.md",
|
||||
route: "/il2cpp-metadata",
|
||||
},
|
||||
{
|
||||
text: "Static JavaScript route inventory",
|
||||
source: "javascript-route-inventory.md",
|
||||
route: "/javascript-route-inventory",
|
||||
},
|
||||
{
|
||||
text: "Linux kernel modules",
|
||||
source: "kernel-modules.md",
|
||||
route: "/kernel-modules",
|
||||
},
|
||||
{
|
||||
text: "Kuna provider",
|
||||
source: "kuna-provider.md",
|
||||
route: "/kuna-provider",
|
||||
},
|
||||
{
|
||||
text: "Model file declarations",
|
||||
source: "model-artifacts.md",
|
||||
route: "/model-artifacts",
|
||||
},
|
||||
{
|
||||
text: "Match native functions across builds",
|
||||
source: "native-function-matching.md",
|
||||
route: "/native-function-matching",
|
||||
},
|
||||
{
|
||||
text: "Node.js single executable applications",
|
||||
source: "node-sea.md",
|
||||
route: "/node-sea",
|
||||
},
|
||||
{
|
||||
text: "PE Authenticode certificate tables",
|
||||
source: "pe-certificates.md",
|
||||
route: "/pe-certificates",
|
||||
},
|
||||
{
|
||||
text: "PE import and delay-load tables",
|
||||
source: "pe-imports.md",
|
||||
route: "/pe-imports",
|
||||
},
|
||||
{
|
||||
text: "PE TLS callbacks and load configuration",
|
||||
source: "pe-load-config.md",
|
||||
route: "/pe-load-config",
|
||||
},
|
||||
{
|
||||
text: "Python packaged applications",
|
||||
source: "python-packaged-applications.md",
|
||||
route: "/python-packaged-applications",
|
||||
},
|
||||
{
|
||||
text: "CPython pyc inspection",
|
||||
source: "python-pyc.md",
|
||||
route: "/python-pyc",
|
||||
},
|
||||
{
|
||||
text: "Reconstruction coverage",
|
||||
source: "reconstruction-coverage.md",
|
||||
route: "/reconstruction-coverage",
|
||||
},
|
||||
{
|
||||
text: "Reverse-engineering tool integrations",
|
||||
source: "reverse-engineering-tools.md",
|
||||
route: "/reverse-engineering-tools",
|
||||
},
|
||||
{
|
||||
text: "Rust binary dependency metadata",
|
||||
source: "rust-binaries.md",
|
||||
route: "/rust-binaries",
|
||||
},
|
||||
{
|
||||
text: "U-Boot legacy uImage containers",
|
||||
source: "uimage.md",
|
||||
route: "/uimage",
|
||||
},
|
||||
{
|
||||
text: "UnityFS AssetBundle archives",
|
||||
source: "unityfs.md",
|
||||
route: "/unityfs",
|
||||
},
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
export const documentationRewrites = Object.fromEntries(
|
||||
documentationSections.flatMap((section) =>
|
||||
section.items.flatMap((item) => {
|
||||
const target = `${item.route.slice(1)}.md`;
|
||||
return item.source === target ? [] : [[item.source, target]];
|
||||
}),
|
||||
),
|
||||
);
|
||||
-195
@@ -1,195 +0,0 @@
|
||||
# Documentation directory
|
||||
|
||||
REA documentation is organized by reader task and document owner. This directory
|
||||
links every authored guide; the [site](https://rea.tools/) also serves separate
|
||||
product pages. The VitePress documentation source is this `docs/` tree.
|
||||
|
||||
## Getting started
|
||||
|
||||
- [Installation and setup](getting-started/installation.md)
|
||||
- [Client configuration and troubleshooting](getting-started/client-configuration.md)
|
||||
- [CLI and Evidence](getting-started/cli.md)
|
||||
- [Guided MCP prompts and completion](getting-started/mcp-prompts.md)
|
||||
|
||||
## Native binaries
|
||||
|
||||
- [Native binary investigation](guides/native/native-investigation.md)
|
||||
- [Go binary build metadata](guides/native/go-binaries.md)
|
||||
- [Offline binary diagnostics](guides/native/binary-diagnostics.md)
|
||||
- [Recorded crash evidence](guides/native/recorded-crashes.md)
|
||||
|
||||
## JavaScript and Electron
|
||||
|
||||
- [JavaScript Application Graph](guides/javascript/javascript-application-graph.md)
|
||||
- [Cross-layer JavaScript application workflows](guides/javascript/javascript-application-workflows.md)
|
||||
- [JavaScript artifact reconstruction](guides/javascript/javascript-artifact-reconstruction.md)
|
||||
- [JavaScript source recovery](guides/javascript/javascript-recovery.md)
|
||||
- [Passive Node and Electron runtime observation](guides/javascript/javascript-runtime-observation.md)
|
||||
- [JavaScript static/runtime reconciliation](guides/javascript/javascript-runtime-reconciliation.md)
|
||||
- [Electron file-page observation](guides/javascript/electron-observation.md)
|
||||
|
||||
## Browser and captured websites
|
||||
|
||||
- [Website observation with CDP](guides/web/browser-observation.md)
|
||||
- [Browser scenario contract](guides/web/browser-scenario-contract.md)
|
||||
- [Website runtime source attribution](guides/web/web-runtime.md)
|
||||
- [Historical web network captures](guides/web/web-network-captures.md)
|
||||
- [HTTP/3 capture lab](http3-capture-lab.md)
|
||||
- [Captured website source locations](guides/web/web-source-location.md)
|
||||
- [Captured website module relationships](guides/web/website-module-trace.md)
|
||||
- [Captured website scripts and static JavaScript analysis](guides/web/website-script-export.md)
|
||||
|
||||
## Android applications
|
||||
|
||||
- [Static Android APK analysis](guides/android/android-analysis.md)
|
||||
- [Apktool resource decoding](guides/android/apktool-resource-analysis.md)
|
||||
- [ADB device inspection and acquisition](guides/android/adb-device-analysis.md)
|
||||
- [Flutter build identification](guides/android/flutter-analysis.md)
|
||||
|
||||
## Apple applications
|
||||
|
||||
- [Apple Application Inventory Projection](guides/apple/apple-application-analysis.md)
|
||||
|
||||
## .NET assemblies
|
||||
|
||||
- [Managed-code analysis](guides/managed/managed-code-analysis.md)
|
||||
|
||||
## Firmware
|
||||
|
||||
- [Firmware analysis](guides/firmware/firmware-analysis.md)
|
||||
|
||||
## HarmonyOS packages
|
||||
|
||||
- [Static HarmonyOS package inventory](guides/harmony/harmony-analysis.md)
|
||||
|
||||
## WebAssembly
|
||||
|
||||
- [Offline WebAssembly artifacts](guides/wasm/wasm-artifacts.md)
|
||||
|
||||
## EVM contracts
|
||||
|
||||
- [Offline EVM bytecode](guides/evm/evm-bytecode.md)
|
||||
|
||||
## Databases and recorded packets
|
||||
|
||||
- [SQLite database snapshots](guides/data/sqlite-databases.md)
|
||||
- [Recorded packet capture inspection](guides/data/packet-captures.md)
|
||||
- [Packet capture summaries](guides/data/pcap-analysis.md)
|
||||
|
||||
## Process capture and instrumentation
|
||||
|
||||
- [Process capture](guides/runtime/process-capture.md)
|
||||
- [Frida instrumentation](guides/runtime/frida-instrumentation.md)
|
||||
|
||||
## Signal recordings
|
||||
|
||||
- [Recorded signal inspection](guides/signals/signal-recordings.md)
|
||||
|
||||
## Provider setup and formats
|
||||
|
||||
- [Ghidra provider semantics](providers/ghidra-provider.md)
|
||||
- [DOS MZ and COM analysis with Ghidra](providers/ghidra-dos.md)
|
||||
- [Optional NativeAOT metadata recovery](providers/ghidra-nativeaot.md)
|
||||
- [MIPS ELF analysis through Ghidra](providers/ghidra-mips.md)
|
||||
- [PSP static ELF analysis through Ghidra](providers/ghidra-psp.md)
|
||||
- [RISC-V ELF analysis through Ghidra](providers/ghidra-riscv.md)
|
||||
- [Windows Ghidra P0](providers/windows-ghidra-p0.md)
|
||||
- [Headerless raw-binary analysis with Ghidra](providers/raw-binary.md)
|
||||
- [IDA provider](providers/ida-provider.md)
|
||||
- [JEB instrumentation and analysis](providers/jeb-analysis.md)
|
||||
|
||||
## Contracts and evidence
|
||||
|
||||
- [MCP runtime contracts](reference/mcp-contracts.md)
|
||||
- [Reconstruction obligation ledgers](reference/reconstruction-obligation-ledgers.md)
|
||||
|
||||
## Development and verification
|
||||
|
||||
- [Settings and recovery](design/limits-and-recovery.md)
|
||||
- [MCP tool design](development/tool-design.md)
|
||||
- [Testing REA](development/testing.md)
|
||||
- [Client and agent verification](development/client-verification.md)
|
||||
- [Reconstruction readiness conformance](development/reconstruction-readiness.md)
|
||||
- [Releasing REA](development/releasing.md)
|
||||
- [REA roadmap](development/roadmap.md)
|
||||
|
||||
## Release migrations
|
||||
|
||||
- [Upgrading from REA 6.1 to 6.2](migrations/migration-6.2.md)
|
||||
|
||||
## Architecture and authoring
|
||||
|
||||
- [Architecture map](https://github.com/morluto/rea/blob/main/docs/architecture/architecture.mermaid)
|
||||
- [Documentation ownership and contribution model](development/documentation.md)
|
||||
- [Feature specifications](specs/README.md)
|
||||
- [Architectural decision records](adr/README.md)
|
||||
|
||||
## Machine-owned documentation
|
||||
|
||||
| Path | Owner and update command |
|
||||
| ----------------------------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| `public/product-catalog.json` | Ignored contract projection; `npm run docs:generate` |
|
||||
| `verification/managed-conformance-*.json` | Ignored managed conformance projections; `npm run evidence:generate` |
|
||||
| `error-contract.schema.json` | Tracked schema from contracts; `npm run docs:error-schema` |
|
||||
| `user-visible-errors.csv` | Authored error classification inventory; preserve its schema |
|
||||
| `releases/<version>.contributions.json` | Durable release contribution record; follow the [release guide](development/releasing.md) |
|
||||
| `assets/` | Shared authored images referenced by guides |
|
||||
|
||||
Do not edit generated catalogs or evidence to change product facts. Change their
|
||||
canonical contracts or producer and regenerate. Historical release records retain
|
||||
the exact commit messages and references from the release they describe.
|
||||
|
||||
## Additional guides
|
||||
|
||||
- [Offline API contract inventory](api-contracts.md)
|
||||
- [Bun standalone executables](bun-standalone.md)
|
||||
- [Capability composition and ownership](capability-composition.md)
|
||||
- [.NET single-file bundles](dotnet-bundles.md)
|
||||
- [ESP flash images](esp-flash-image.md)
|
||||
- [U-Boot FIT images](fit-images.md)
|
||||
- [Measure MCP catalog costs in Codex](mcp-catalog-evaluation.md)
|
||||
- [Offline Playwright traces](playwright-traces.md)
|
||||
- [Testing lanes and provider verification](testing-lanes.md)
|
||||
- [UF2 and DfuSe images](uf2-dfuse.md)
|
||||
- [WeChat mini-program wxapkg archives](wxapkg.md)
|
||||
- [Binary hardening](binary-hardening.md)
|
||||
- [Desktop application capture](desktop-application-capture.md)
|
||||
- [Desktop application export](desktop-application-export.md)
|
||||
- [Desktop application state](desktop-application-state.md)
|
||||
- [Desktop application tasks](desktop-application-tasks.md)
|
||||
- [Desktop build computer](desktop-build-computer.md)
|
||||
- [Desktop build witness](desktop-build-witness.md)
|
||||
- [iNES containers](ines-containers.md)
|
||||
- [Build RL environments with REA](agent-environments-quickstart.md)
|
||||
- [Agent environments from recorded captures](agent-environments.md)
|
||||
- [APK signature schemes](apk-signature.md)
|
||||
- [Apple IPS crash reports](apple-ips.md)
|
||||
- [Static analysis in GitHub Actions](ci-analysis.md)
|
||||
- [Desktop installation directories](desktop-applications.md)
|
||||
- [Flattened devicetree inspection](devicetree.md)
|
||||
- [Electron fuse state](electron-fuses.md)
|
||||
- [ELF note tables](elf-notes.md)
|
||||
- [Erlang BEAM files](erlang-beam.md)
|
||||
- [ESP images](esp-image.md)
|
||||
- [GameCube DOL analysis through Ghidra](ghidra-dol.md)
|
||||
- [PowerPC ELF analysis through Ghidra](ghidra-ppc.md)
|
||||
- [Godot PCK archives](godot-pck.md)
|
||||
- [Static CUDA and HIP code objects](gpu-code-objects.md)
|
||||
- [Intel HEX and S-record images](hex-records.md)
|
||||
- [IL2CPP metadata](il2cpp-metadata.md)
|
||||
- [Static JavaScript route inventory](javascript-route-inventory.md)
|
||||
- [Linux kernel modules](kernel-modules.md)
|
||||
- [Kuna provider](kuna-provider.md)
|
||||
- [Model file declarations](model-artifacts.md)
|
||||
- [Match native functions across builds](native-function-matching.md)
|
||||
- [Node.js single executable applications](node-sea.md)
|
||||
- [PE Authenticode certificate tables](pe-certificates.md)
|
||||
- [PE import and delay-load tables](pe-imports.md)
|
||||
- [PE TLS callbacks and load configuration](pe-load-config.md)
|
||||
- [Python packaged applications](python-packaged-applications.md)
|
||||
- [CPython pyc inspection](python-pyc.md)
|
||||
- [Reconstruction coverage](reconstruction-coverage.md)
|
||||
- [Reverse-engineering tool integrations](reverse-engineering-tools.md)
|
||||
- [Rust binary dependency metadata](rust-binaries.md)
|
||||
- [U-Boot legacy uImage containers](uimage.md)
|
||||
- [UnityFS AssetBundle archives](unityfs.md)
|
||||
@@ -25,21 +25,7 @@ REA never installs the Android SDK, an emulator, or a system image.
|
||||
Readiness is `rea inspect-adb-client`: it runs `adb version` and reports the
|
||||
resolved binary, its version, and where it was selected from. A missing
|
||||
binary reports `capability_unavailable` with the `REA_ADB_PATH` remediation
|
||||
rather than attempting an installation. A path that cannot resolve to a file
|
||||
(`ENOENT`, `ENOTDIR`, `ELOOP` or `ENAMETOOLONG`) is reported as missing. A
|
||||
selected path the host refuses to execute, such as a directory or a file
|
||||
without execute permission (`EACCES`) or one the host does not permit
|
||||
(`EPERM`), and any other spawn error report `capability_unavailable` naming
|
||||
that path. Exhausted host capacity (`EAGAIN`, `EMFILE`, `ENFILE` or `ENOMEM`)
|
||||
is not a configuration failure. `inspect_adb_client`, and any device
|
||||
operation whose adb cannot start at its version probe or at a later adb
|
||||
command, report a retryable `capacity` resource constraint rather than a
|
||||
capability refusal or an adb protocol failure; the `binary_session` inventory
|
||||
reports the provider as `runtime_unavailable`. Both say the host could not
|
||||
start adb and ask for a retry after freeing processes or descriptors, without
|
||||
blaming `REA_ADB_PATH`. When adb never started, the refusal diagnostics record
|
||||
the spawn error as `filesystem_error` with `path_source`, never as an
|
||||
`exit_code`.
|
||||
rather than attempting an installation.
|
||||
The `binary_session` inventory uses the same bounded client-version probe; it
|
||||
does not query devices, start the adb server, or establish that a requested
|
||||
serial is reachable. Device state is checked by each device operation.
|
||||
@@ -117,10 +103,6 @@ existing analysis families directly, for example
|
||||
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
|
||||
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
|
||||
|
||||
Screen capture validates the PNG signature, IHDR framing, dimensions, and IHDR
|
||||
checksum before publishing. It does not validate encoding fields or decode and
|
||||
validate image contents; that limitation is retained in Evidence.
|
||||
|
||||
When `push_adb_file` receives an existing directory, adb places the local file
|
||||
inside it under the local basename; Evidence reports that effective file path
|
||||
while retaining the caller's directory path in the input parameters. The
|
||||
@@ -217,18 +199,12 @@ and during shutdown.
|
||||
`uninstall_adb_package` declare `mutatesTarget` and never run implicitly:
|
||||
pushes check for existing device files before transfer and refuse them
|
||||
unless `overwrite` is set. A failed or unexpected existence probe stops
|
||||
transfer even with `overwrite`. A completed `test -e` exit of 1 establishes
|
||||
absence only with empty stdout and either empty stderr or a recognized,
|
||||
completed adb-server startup notice sequence. Startup notices are host
|
||||
observations, not remote-path diagnostics; successful push receipts retain
|
||||
them in `raw_result.device_path_probes`. Incomplete startup, extra diagnostics,
|
||||
truncation and cancellation do not establish absence. The check is not atomic, so a different
|
||||
transfer even with `overwrite`; only a clean `test -e` exit of 1 with no
|
||||
output is interpreted as absence. The check is not atomic, so a different
|
||||
process can create the path before `adb push` writes it. When available, the device's
|
||||
`sha256sum` verifies the transfer. A missing command, output without a
|
||||
recognizable digest, or host capacity that prevents the digest check from
|
||||
starting reports `device_digest_source: "unavailable"` with an
|
||||
unverified-transfer limitation rather than a retryable error; installs
|
||||
preserve the
|
||||
`sha256sum` verifies the transfer. A missing command or output without a
|
||||
recognizable digest reports `device_digest_source: "unavailable"` with an
|
||||
unverified-transfer limitation; installs preserve the
|
||||
device's `Failure [...]` reason verbatim.
|
||||
- **Producer identity stays attached.** Every result embeds the producing
|
||||
adb client (`binary_path`, `path_source`, `version`, `installed_path`).
|
||||
@@ -277,18 +253,18 @@ beyond already-connected devices (`adb connect`) are not exposed yet.
|
||||
|
||||
## Real verification
|
||||
|
||||
`npm run verify:adb` exercises observation and transfer operations against a live device. It
|
||||
`npm run verify:adb` exercises every operation against a live device. It
|
||||
requires `adb` (or `REA_ADB_PATH`) and any attached emulator or phone;
|
||||
`--serial` selects one when several are attached, `--pull PACKAGE` pulls a
|
||||
real package and verifies the returned digests against the files on disk.
|
||||
The optional `--install-apk` lifecycle is unavailable until the selected APK
|
||||
identity and device cleanup ownership can be established safely.
|
||||
real package and verifies the returned digests against the files on disk,
|
||||
and `--install-apk PATH` drives a full install → resolve → uninstall
|
||||
lifecycle with a lane-owned probe.
|
||||
|
||||
Record: verified against adb 34.0.5 on Linux against an Android 14
|
||||
(API 34) x86_64 emulator — all operations above, a 224-package inventory,
|
||||
287 services, 92 features, a real two-APK split set pulled with byte-exact
|
||||
digests, a verified push/pull roundtrip, a 1.3 MB screen capture, and the
|
||||
complete package lifecycle; see `docs/development/testing.md` for the lane summary.
|
||||
complete package lifecycle; see `docs/testing.md` for the lane summary.
|
||||
|
||||
Intent integer extras use signed decimal text. The ADB argument encoder removes
|
||||
redundant leading zeros for Android's radix-sensitive `--ei` parser while retained
|
||||
@@ -1,9 +0,0 @@
|
||||
# Architectural decision records
|
||||
|
||||
Store one actual decision per `docs/adr/<domain>/<decision-slug>.md`, without a
|
||||
global sequence number. Use meaningful names and the Michael Nygard sections:
|
||||
Status, Context, Decision and Consequences. Link each decision from its owning
|
||||
feature specification.
|
||||
|
||||
See [the authoring guide](../development/documentation.md#feature-lifecycle) for
|
||||
the distinction between specifications, roadmaps and isolated decisions.
|
||||
@@ -11,15 +11,15 @@ with your RL framework to set task goals, check rewards, and run training.
|
||||
|
||||
## Before you start
|
||||
|
||||
Install [REA and its agent integration](getting-started/installation.md). To record an app,
|
||||
Install [REA and its agent integration](installation.md). To record an app,
|
||||
you also need a running test instance and a Chrome-family browser. Choose an
|
||||
instance where the agent is allowed to create, edit, and delete data. REA
|
||||
records the app you provide; you start it and prepare its test data.
|
||||
|
||||
To run the exported environment, install the
|
||||
[AgentEnv framework](https://github.com/scaleapi/agentenv-framework#install)
|
||||
and start Docker. REA's [compatibility check](development/testing.md#exported-environment-bundles)
|
||||
uses AgentEnv version `0.9.1431` with Python 3.12.
|
||||
and start Docker. REA's [compatibility check](testing.md#exported-environment-bundles)
|
||||
uses AgentEnv version `0.9.1334` with Python 3.12.
|
||||
|
||||
There are two ways to build an environment: ask your agent to record the app,
|
||||
or start from a recording you already have.
|
||||
@@ -52,7 +52,7 @@ work and which need more evidence. The app is used during recording; the
|
||||
exported environment runs from the saved data.
|
||||
|
||||
After your agent exports the folder, skip to
|
||||
[run the environment](#run-the-environment). The next section is an
|
||||
[run the environment](#3-run-the-environment). The next section is an
|
||||
alternative for an existing recording.
|
||||
|
||||
## 2. Or build from a saved recording
|
||||
@@ -91,8 +91,6 @@ A successful build means the folder was created. Check the report to confirm
|
||||
it supports the operations your tasks need. `coverage: null` means the build
|
||||
could not prepare state for this environment and it will use replay.
|
||||
|
||||
<a id="run-the-environment"></a>
|
||||
|
||||
## 3. Run the environment
|
||||
|
||||
Whether your agent exported the folder or you built it from a HAR, run it
|
||||
@@ -160,12 +158,12 @@ assigned by the app.
|
||||
|
||||
Use the report's hints to decide what to record next:
|
||||
|
||||
| Reason | Next step |
|
||||
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `unconfirmed` | Read the item and its list after the change. |
|
||||
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
|
||||
| `no-observation` | Read back a created item. For an edit or delete, also read it before changing it. |
|
||||
| `contradicted` or `side-effect` | Inspect the reported disagreement. The operation may need behavior REA cannot model yet. |
|
||||
| Reason | Next step |
|
||||
| ------------------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| `unconfirmed` | Read the item and its list after the change. |
|
||||
| `no-state-target` | Record reads of the collection or item the operation changes. |
|
||||
| `no-observation` | Read back a created item. For an edit or delete, also read it before changing it. |
|
||||
| `contradicted` or `side-effect` | Inspect the reported disagreement. The operation may need behavior REA cannot model yet. |
|
||||
|
||||
See the [full reason table](agent-environments.md#build-a-bundle-in-one-step)
|
||||
for the other cases. Rebuild from the new recording and check the report
|
||||
|
||||
+24
-34
@@ -30,7 +30,7 @@ rea environment-serve manifest.json --evidence capture.json --port 8765
|
||||
```
|
||||
|
||||
The first command inspects the capture; the second projects the
|
||||
[environment manifest](guides/web/web-network-captures.md#environment-manifest) view from
|
||||
[environment manifest](web-network-captures.md#environment-manifest) view from
|
||||
it, where `view.json` wraps the capture Evidence as the inline source:
|
||||
|
||||
```json
|
||||
@@ -163,7 +163,7 @@ at info level; the report is the only output on stdout:
|
||||
|
||||
`coverage` is the state's read views and write plans summarized, and `null`
|
||||
when the state is unavailable and the bundle serves replay. The
|
||||
[environment coverage](guides/web/web-network-captures.md#environment-coverage) view
|
||||
[environment coverage](web-network-captures.md#environment-coverage) view
|
||||
reports the same summary for any capture without building, so an agent can
|
||||
check it after each recording. The view summarizes the state as derived; it
|
||||
does not apply the serve-time admission checks, so for an environment whose
|
||||
@@ -174,7 +174,7 @@ what to record next; hints never change what is served:
|
||||
|
||||
| Reason | Hint |
|
||||
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `no-state-target` | The write could not be linked to a modelled target. Check the path template and recorded views; record missing reads. Some route forms may not be supported. |
|
||||
| `no-state-target` | The write's path names no collection or item the capture reads. Read the collection or item it changes. |
|
||||
| `no-call` | No recorded call of this tool was modelled, for example because every call failed. Record a successful call. |
|
||||
| `body-unusable` | The retained request or response cannot be modelled. Record a successful call with a complete supported request and reproducible response. |
|
||||
| `no-observation` | Nothing showed the entity or document this write changed. Read a created entity back by id right after the write; for an update or delete the state must already hold the entity, so read it before the write or record the create that made it. |
|
||||
@@ -235,7 +235,7 @@ name `export_environment_bundle`; the others name their stage as
|
||||
## From a browser scenario
|
||||
|
||||
The capture can also come from REA itself: a
|
||||
[browser scenario](guides/web/browser-scenario-contract.md) records a scripted session
|
||||
[browser scenario](browser-scenario-contract.md) records a scripted session
|
||||
in a browser REA launches, and its Evidence is a manifest parent like a HAR
|
||||
inspection. Select the `network` event family and response bodies. A replayed
|
||||
reply carries up to 256 KiB of a recorded body, but the scenario keeps at most
|
||||
@@ -321,7 +321,7 @@ declared secret replaced by its redaction marker (a body that redaction had to
|
||||
empty replays no body).
|
||||
|
||||
To record writes, sign in through the UI and then send the application's API
|
||||
calls with [request steps](guides/web/browser-scenario-contract.md#request-steps): each
|
||||
calls with [request steps](browser-scenario-contract.md#request-steps): each
|
||||
sends one request to the start origin through the page's own `fetch`, with the
|
||||
session's cookies, and a header can carry the CSRF cookie or stored token the
|
||||
application's own requests send. A write is served only when the capture shows
|
||||
@@ -481,7 +481,7 @@ captured text without changing the tool names or schema fields clients use.
|
||||
|
||||
Replay answers each call from one recording, so the answers reflect whichever
|
||||
moment of the capture recorded them. The
|
||||
[environment state](guides/web/web-network-captures.md#environment-state) view extracts
|
||||
[environment state](web-network-captures.md#environment-state) view extracts
|
||||
instead the data the capture's reads observed before the first related write:
|
||||
entity types with an identity field, entities with their recorded values,
|
||||
documents for reads that are not entities, and a render plan per read tool,
|
||||
@@ -503,14 +503,14 @@ see [Serve from the state](#serve-from-the-state).
|
||||
|
||||
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the
|
||||
environment, then to `127.0.0.1` and an ephemeral port, exactly as
|
||||
[`rea mcp --http`](getting-started/cli.md#serve-mcp-over-streamable-http) resolves them. The
|
||||
[`rea mcp --http`](cli.md#serve-mcp-over-streamable-http) resolves them. The
|
||||
bound MCP URL is written to stderr as one JSON log line
|
||||
(`"msg":"MCP transport listening"` with `url`); `REA_LOG_LEVEL` overrides that
|
||||
log level. `ENVIRONMENT_NAME` renames the served environment; otherwise the
|
||||
manifest's `name` is used, and a set but blank value is refused as
|
||||
`configuration_invalid` before binding rather than silently falling back. The
|
||||
listener serves one MCP client at a time with the same session, `Host` header
|
||||
and shutdown contract as [transports](reference/mcp-contracts.md#transports) describes.
|
||||
and shutdown contract as [transports](mcp-contracts.md#transports) describes.
|
||||
It runs until SIGINT or SIGTERM, then closes the listener, prints one
|
||||
`{"status":"stopped", …}` document whose `state` member says whether it served
|
||||
from the state, and exits `130` or `143`.
|
||||
@@ -585,7 +585,7 @@ names the method and path template, then what the tool returns, one level deep:
|
||||
`returns a JSON object with fields …`, `returns a JSON array of objects with
|
||||
fields …` or `returns a JSON object whose <member> lists objects with fields …`,
|
||||
where a list is an array of objects found by the
|
||||
[environment state](guides/web/web-network-captures.md#environment-state)'s list rule (the
|
||||
[environment state](web-network-captures.md#environment-state)'s list rule (the
|
||||
root array, or the one array of objects an envelope carries). Field and member
|
||||
names are written as JSON strings, so a captured name cannot read as more of
|
||||
the description; a member name too long to show whole is a quoted prefix marked
|
||||
@@ -641,7 +641,7 @@ rea environment-serve manifest.json --evidence capture.json
|
||||
```
|
||||
|
||||
Before it binds, the server derives the
|
||||
[environment state](guides/web/web-network-captures.md#environment-state) the capture
|
||||
[environment state](web-network-captures.md#environment-state) the capture
|
||||
observed: it runs the `environment-state` view on the capture in-process with
|
||||
the manifest's `template_rules` in the same order, exactly as
|
||||
`inspect-analysis-view` projects it from the same capture, so the state
|
||||
@@ -723,8 +723,6 @@ and `POST` tools the capture shows only reading (a lookup answering as a `GET`
|
||||
did, an RPC tunnel naming read operations, a session renewal), stay served and
|
||||
answer from their recordings with the reason `query-body`; they are not write
|
||||
tools.
|
||||
`HEAD`, `OPTIONS` and `TRACE` tools also replay their captured responses; their
|
||||
state views report `method-replay` and coverage counts them as recorded reads.
|
||||
|
||||
A line establishes whole-collection membership only from the beginning:
|
||||
`offset`, `start`, `_start` and `page` must be zero when present. Positive or
|
||||
@@ -733,20 +731,20 @@ query model may describe supported paging; an empty later page cannot start
|
||||
returning earlier entities merely because it was empty in the capture.
|
||||
|
||||
`POST`, `PUT`, `PATCH` and `DELETE` tools are served only when the state's
|
||||
[write plan](guides/web/web-network-captures.md#environment-state) for them is served:
|
||||
[write plan](web-network-captures.md#environment-state) for them is served:
|
||||
the capture's later reads confirmed the effect the plan models. Other write
|
||||
tools are not served, so no call claims a change it does not make; each one's
|
||||
plan states why in its `reason`. A served write changes the live state, which
|
||||
every MCP session and `data/get` then see:
|
||||
|
||||
| Effect | Call |
|
||||
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
|
||||
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, nested clock, increment, slug, stamp, derived, composed, reference, unique and renewed changes the plan models. |
|
||||
| `update` (action) | A `POST`, `PUT` or `PATCH` to an item path plus an action segment (`/{type}/{id}/archive`) sets, on the entity before the segment, each scalar the recorded calls showed the action set (`set`), or flips each boolean they showed it flipping both ways (`toggle`), with any sent members; it is served only when a recorded call found what it sets already set, or flipped back what it toggles. |
|
||||
| `delete` | Removes the entity the path names. |
|
||||
| bulk `update` or `delete` | A `DELETE`, `PUT` or `PATCH` at a collection, or a `POST` to it plus `delete`, `remove` or `destroy`, applies its effect to every entity the body names, as recorded: an array of ids or of objects carrying the id field, at the root or as the body's one array member (`{"IDs": […], "Read": true}`). |
|
||||
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
|
||||
| Effect | Call |
|
||||
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
|
||||
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, nested clock, increment, slug, stamp, derived, composed, reference, unique and renewed changes the plan models. |
|
||||
| `update` (action) | A `POST`, `PUT` or `PATCH` to an item path plus an action segment (`/{type}/{id}/archive`) sets, on the entity before the segment, each scalar the recorded calls showed the action set (`set`), with any sent members; it is served only when a recorded call found those fields already set, so a toggle never is. |
|
||||
| `delete` | Removes the entity the path names. |
|
||||
| bulk `update` or `delete` | A `DELETE`, `PUT` or `PATCH` at a collection, or a `POST` to it plus `delete`, `remove` or `destroy`, applies its effect to every entity the body names, as recorded: an array of ids or of objects carrying the id field, at the root or as the body's one array member (`{"IDs": […], "Read": true}`). |
|
||||
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
|
||||
|
||||
An update's unsent changes follow the plan's rules. A `held-clock` is a clock
|
||||
the client sends back as it read it and the application stamps anew (Ghost's
|
||||
@@ -757,9 +755,7 @@ the entity already names under its `created…by` counterpart into a
|
||||
`modified…by` or `updated…by` field. A `derived` field takes a value showing
|
||||
the one sent member's text that the entity stores nowhere (a field Ghost
|
||||
derives from `html`), only with that member's recorded value. An action's
|
||||
`set` rule sets a scalar to the value the recorded calls left it at, and its
|
||||
`toggle` rule flips a boolean, refusing a call on an entity that holds none
|
||||
there (Miniflux's `PUT /v1/entries/{id}/bookmark` stars and unstars).
|
||||
`set` rule sets a scalar to the value the recorded calls left it at.
|
||||
|
||||
Its answer has the status and content type of the plan's recorded response,
|
||||
with the written entity or its identity rendered in, no body, or the recorded
|
||||
@@ -1049,9 +1045,8 @@ unknown recorded status stays unknown; it is not borrowed from another
|
||||
record). A bundle with no such tool yields a verifier that checks the count
|
||||
only and says so. `tasks/smoke.json` deploys the env on the local
|
||||
sandbox and runs that verifier against the env's MCP endpoint; the first run
|
||||
also pulls the framework's published infrastructure images from `ghcr.io`,
|
||||
building one only if its pull fails, and the run tears everything down when it
|
||||
ends.
|
||||
also builds the framework's own infrastructure, and the run tears everything
|
||||
down when it ends.
|
||||
|
||||
The README written into the bundle records the capture artifact's SHA-256, the
|
||||
capture and manifest Evidence ids, the manifest view digest, the served tool
|
||||
@@ -1150,12 +1145,7 @@ global conflict involving an equal pair is not used to normalize either pair; th
|
||||
|
||||
A step agrees when the statuses and the normalised bodies are equal. The report gives agreement overall, per probe and
|
||||
per tool, `agreement_ignoring_order` for steps that differ only in item order, and each step's differences, at most 20,
|
||||
with a JSON pointer and both values cut short. Each step reports `differences_total` before that cap,
|
||||
`differences_omitted`, and `differences_truncated` (true when entries were omitted). Agreement and categories use the
|
||||
complete comparison. Omitted entries are not retained, so `--full-output` cannot recover them; inspect the original
|
||||
capture and probe Evidence to investigate beyond the returned pointers.
|
||||
|
||||
The difference categories are:
|
||||
with a JSON pointer and both values cut short:
|
||||
|
||||
| Category | Meaning |
|
||||
| --------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
@@ -1187,7 +1177,7 @@ read, not that the write had no other effect.
|
||||
## Compatibility verification
|
||||
|
||||
`npm run verify:agentenv:environment` is the hosted `real-agentenv` lane
|
||||
described in [testing](development/testing.md#exported-environment-bundles). It projects
|
||||
described in [testing](testing.md#exported-environment-bundles). It projects
|
||||
the Grafana-shaped source-owned capture through the built CLI, packs the
|
||||
checkout, exports a bundle with `--package-tarball`, runs it with the pinned
|
||||
`agentenv-framework` release on the runner's Docker and requires the smoke task
|
||||
|
||||
@@ -6,12 +6,12 @@ kept as an unmodified, commit-pinned Git submodule; see
|
||||
application workflow and return artifact-bound Evidence inline.
|
||||
|
||||
This family is present on repository main and in npm 4.1.0. Check the
|
||||
[released package boundary](../../getting-started/installation.md#released-package-and-main) and the
|
||||
[released package boundary](installation.md#released-package-and-main) and the
|
||||
connected server's tool list before selecting it.
|
||||
|
||||
On Linux, JADX launches an REA-owned process and requires a procps-compatible
|
||||
`ps` on REA's `PATH`; see the shared
|
||||
[Linux process prerequisite](../../getting-started/installation.md#linux-owned-process-prerequisite).
|
||||
[Linux process prerequisite](installation.md#linux-owned-process-prerequisite).
|
||||
|
||||
## Optional package delivery
|
||||
|
||||
@@ -39,9 +39,7 @@ unverified.
|
||||
|
||||
`inspect_android_package` also returns a static inventory of declared
|
||||
activities, aliases, services, receivers and providers from the decoded
|
||||
manifest. The binary XML decoder preserves component class spelling, including
|
||||
`$` in nested classes, without applying JADX class aliases. `declared_name`
|
||||
preserves the decoded source attribute; `name` resolves a
|
||||
manifest. `declared_name` preserves the source attribute; `name` resolves a
|
||||
relative name only when the manifest package is known and the resolved copy
|
||||
fits the UTF-8 byte budget. An unresolved package or exhausted name budget
|
||||
retains the declared name and marks the inventory partial. Empty component
|
||||
@@ -73,9 +71,6 @@ spellings; the original SDK declaration strings remain unchanged.
|
||||
it remains null for unknown provider target SDKs, nonliteral declarations, or
|
||||
components with filters targeting API 31 or later that omit the now-required
|
||||
explicit declaration. A limitation records the latter validation error.
|
||||
As in Android's package parser, an activity, alias or receiver filter without
|
||||
an `<action>` is discarded and does not count toward either rule; a service
|
||||
keeps such a filter. `intent_filter_count` still counts every declared filter.
|
||||
Provider authorities and all inventory dimensions are bounded. `status:
|
||||
"partial"` and `limitations` identify omitted declarations or values; a
|
||||
capacity limitation appears only when an actual collection limit was reached.
|
||||
@@ -255,35 +250,6 @@ queries accept the same `overload_index`; the bridge resolves the selected
|
||||
method node and the result reports its index and displayed signature. These
|
||||
indices are tied to the current artifact and engine, not stable across builds.
|
||||
|
||||
Method reference queries also accept an external class, such as
|
||||
`android.app.NotificationManager --method-name notify`, when the APK directly
|
||||
invokes that exact DEX class and method name. If multiple prototypes are observed,
|
||||
the input error lists their exact DEX descriptors and local overload indices;
|
||||
repeat with an explicit `--overload-index`. These are the prototypes observed in
|
||||
the APK, not the complete method inventory of an Android framework or library.
|
||||
|
||||
Such results report `target_definition: "external"` and
|
||||
`reference_scope: "direct_dex_invokes"`. Each using method has `source_selector`
|
||||
with `class_name`, `method_name` and `overload_index`; pass those fields with the
|
||||
same APK path to `inspect_android_method` to inspect the caller. Repeated direct
|
||||
invokes in one method remain one using node with multiple instruction offsets.
|
||||
This scans raw invoke instructions in JADX's admitted APK methods without
|
||||
generating their source. It does not resolve reflection, JNI, method handles,
|
||||
inherited method lookup or possible runtime dispatch. Class-only queries and
|
||||
class/method inspection still require an APK definition; an external target has
|
||||
no invented body. Unknown class/method names remain input errors.
|
||||
|
||||
External prototype and caller metadata use a bridge-frame-derived representation
|
||||
allowance that reserves space for retained raw calls and CLI/MCP copies. Exhausting
|
||||
it returns a typed `resource_constraint`, rather than a truncated caller list
|
||||
reported as complete. Direct instruction offsets have their separate explicit
|
||||
`direct_invokes_exhaustive` coverage. Older retained Evidence may omit the new
|
||||
definition, scope and caller-selector fields; omission does not establish them.
|
||||
The bridge checks prototype expansion before copying descriptors or joining
|
||||
displayed signatures. It reuses one bounded prototype inventory for the exact
|
||||
class/method selection in the current APK session; replacing the session or
|
||||
selection invalidates that retained result.
|
||||
|
||||
`inspect_android_class` also reports `dex_static_value` for each `static final`
|
||||
field whose class definition encodes a value in its DEX static values, so
|
||||
compile-time constants are readable even when the class has no `<clinit>`.
|
||||
@@ -687,14 +653,14 @@ bounds each upstream call. Timeout recovery reports the effective milliseconds
|
||||
and range, with `details.limit` identifying the runtime setting. An invalid
|
||||
provider-local launch value can also be repaired by a runtime override; reset
|
||||
exposes that invalid launch value again. See the
|
||||
[settings design](../../design/limits-and-recovery.md#invalid-launch-configuration).
|
||||
[settings design](design/limits-and-recovery.md#invalid-launch-configuration).
|
||||
A passing JADX configuration check in `rea doctor` reports its launch-derived
|
||||
`operation_timeout_ms`; it does not inspect another running server's overrides. Explicit
|
||||
cancellation still ends the operation immediately, and partial observations
|
||||
and owned-process cleanup are unchanged. Raising the deadline does not extend
|
||||
the bridge's 30-second startup handshake or the upstream **90-second**
|
||||
decompilation limit. MCP clients enforce their own tool-call timeouts, and
|
||||
setup's derived [client tool timeout](../../getting-started/client-configuration.md) does not include
|
||||
setup's derived [client tool timeout](client-configuration.md) does not include
|
||||
this setting, so also raise the client timeout when selecting a longer
|
||||
deadline.
|
||||
|
||||
@@ -768,7 +734,3 @@ containing node's unknown `dex_offset`. `direct_invokes_exhaustive: false`
|
||||
means instruction rows were bounded by the local bridge frame budget; `null`
|
||||
means the producer did not establish instruction coverage. Using-node totals
|
||||
and coverage do not imply complete instruction evidence.
|
||||
`direct_invokes_coverage` reports counts of using nodes with complete, partial,
|
||||
or unknown positions, plus the number of retained instruction sites. This
|
||||
aggregate makes position truncation visible without scanning every caller;
|
||||
older retained Evidence may omit it, which means the aggregate was unexamined.
|
||||
@@ -1,178 +0,0 @@
|
||||
# Offline API contract inventory
|
||||
|
||||
`inspect_api_contract` / `rea inspect-api-contract` supports local OpenAPI JSON
|
||||
and GraphQL SDL (`format: "graphql-sdl"`). Neither profile contacts endpoints.
|
||||
|
||||
## OpenAPI JSON
|
||||
|
||||
The default profile reads one local OpenAPI
|
||||
3.0, 3.1 or 3.2 JSON document. It inventories inline operation slots under
|
||||
`paths`; the facts describe declarations, not implemented, reachable or
|
||||
authorized runtime behavior. YAML, AsyncAPI, callbacks/webhooks, schema graphs,
|
||||
contract diffs and traffic correlation are outside this version.
|
||||
|
||||
The OpenAPI importer never resolves references, opens sibling documents, evaluates
|
||||
examples or contacts declared servers. OpenAPI 3.2 `query` and
|
||||
`additionalOperations` follow the [Path Item contract](https://spec.openapis.org/oas/v3.2.1.html#path-item-object),
|
||||
including case-sensitive additional HTTP method spelling.
|
||||
|
||||
## Select a useful answer
|
||||
|
||||
MCP defaults to a summary retaining the complete inventory. An optional view
|
||||
returns selected facts immediately, taking precedence over `detail`:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/artifacts/openapi.json",
|
||||
"view": {
|
||||
"kind": "api-contract",
|
||||
"collection": "operations",
|
||||
"methods": ["GET"],
|
||||
"limit": 20
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`operations` supports exact `operation_ids`, `operation_keys`, `methods`,
|
||||
`paths`, `declared_ids` and `states` filters. Filters combine by intersection;
|
||||
empty arrays select nothing. `fields` chooses columns; identity, source pointer,
|
||||
state and unknown reason remain. The other collections are
|
||||
`path-item-references` (optional `reference_ids`) and `diagnostics` (optional
|
||||
`codes`), also with fields and offset/limit. The default page contains 64 rows.
|
||||
|
||||
Query a retained result with `inspect_analysis_view`, using the same view and
|
||||
`source: {"kind":"retained-evidence","evidence_id":"..."}`. Copy the complete
|
||||
parent ID from `normalized_result.parent_evidence_id`, not the summary's own ID.
|
||||
These pages use immutable retained facts without rereading the specification.
|
||||
Complete Evidence remains available through `get_evidence_bundle`.
|
||||
|
||||
CLI defaults to complete Evidence. For a selected answer, save that complete
|
||||
Evidence in an absent bundle path:
|
||||
|
||||
```sh
|
||||
rea inspect-api-contract ./openapi.json --detail summary \
|
||||
--view '{"kind":"api-contract","collection":"operations","methods":["GET"],"limit":20}' \
|
||||
--evidence_output ./contract.evidence.json --json
|
||||
```
|
||||
|
||||
Later `rea inspect-analysis-view` can read that bundle through an exact
|
||||
`evidence-file` source selection, even after the original specification is
|
||||
unavailable. Publication never overwrites an existing file. CLI preserves its
|
||||
original path spelling in `caller_input_path`; analysis locations and MCP inputs
|
||||
use absolute filesystem paths.
|
||||
|
||||
## Identity, provenance and coverage
|
||||
|
||||
Each operation has an artifact-scoped `operation_id`, an exact method/path
|
||||
`operation_key`, an optional `declared_id` from `operationId`, and its original
|
||||
JSON Pointer. Keys can correlate exact declarations in a caller-selected pair
|
||||
of revisions; they do not establish equivalence across unrelated APIs, renamed
|
||||
path templates or runtime handlers. Declared IDs are not assumed unique.
|
||||
The artifact includes the verified byte length, SHA-256 and selected path.
|
||||
|
||||
Malformed siblings retain useful declarations plus diagnostics. A Path Item
|
||||
`$ref`, including a local fragment, remains unresolved; adjacent inline slots
|
||||
are unknown because referenced fields may overlap. Missing or malformed paths
|
||||
are explicit, and missing required OpenAPI 3.0 paths does not become a complete
|
||||
empty inventory. `source_coverage` is limited to inline paths and records that
|
||||
reference resolution and full schema validation were not performed. Page
|
||||
exhaustion describes the selected rows and never upgrades source coverage.
|
||||
|
||||
Structured transport userinfo credentials are excluded from reference URIs;
|
||||
`credential_redacted` reports that change, and an ambiguous credential span leaves
|
||||
the displayed reference unknown. Ordinary reference query/fragment text and
|
||||
names are retained. Original bytes remain in the caller's artifact; complete
|
||||
inventory Evidence does not embed the entire source JSON. Duplicate JSON members
|
||||
use the shared parser's last-value-wins behavior, recorded in source coverage.
|
||||
|
||||
If decoding or later validation fails after a verified read, the typed error
|
||||
retains a compact `file-snapshot-hash` observation with the exact path, digest
|
||||
and byte length. Syntax diagnostics do not quote source JSON. Cancellation
|
||||
does not replace a parser failure that has already completed.
|
||||
|
||||
## Resource bounds and verification
|
||||
|
||||
The file-byte allowance and conservative expanded-facts allowance are each
|
||||
32 MiB resource policies, not OpenAPI format limits. `max_file_bytes` can lower
|
||||
the input allowance. Native JSON parsing additionally checks available heap
|
||||
against the shared representation-expansion estimate before decoding. The
|
||||
shared JSON nesting limit is 1,000, including uninterpreted schema regions.
|
||||
Capacity refusals are typed; reply paging does not increase parser allowances.
|
||||
|
||||
The committed [Petstore fixture](https://github.com/morluto/rea/blob/b7c8caedd4166e6032c49024cdc6fb1154f1a2df/tests/fixtures/openapi/README.md) is an
|
||||
original document from the published application, replayed offline through
|
||||
public CLI/MCP tests. These check advertised schemas, complete/selected parity,
|
||||
retained paging, partial references, zero requests to a loopback reference
|
||||
server and unchanged source bytes. They do not establish runtime endpoint
|
||||
behavior or full OpenAPI conformance.
|
||||
|
||||
## GraphQL SDL and operation documents
|
||||
|
||||
Select `format: "graphql-sdl"` for SDL. Optionally supply `operations_path` to
|
||||
one local document containing operations and fragments. Both MCP paths are
|
||||
absolute; CLI accepts relative spellings:
|
||||
|
||||
```sh
|
||||
rea inspect-api-contract ./schema.graphql --contractFormat graphql-sdl \
|
||||
--operationsPath ./queries.graphql --detail summary \
|
||||
--view '{"kind":"graphql-contract","collection":"references","limit":20}' \
|
||||
--evidence_output ./graphql.evidence.json --json
|
||||
```
|
||||
|
||||
Collections are `types`, `fields`, `definitions`, `references`, `fragment-spreads`
|
||||
and `diagnostics`. Each supports `fields`, `offset` and `limit` (default 64).
|
||||
Exact batch filters include type/field/definition/reference IDs, type names,
|
||||
`field_keys` such as `Query.viewer`, owning type names, binding/spread states
|
||||
and diagnostic codes. Filters intersect; an empty array selects nothing.
|
||||
Identity, source and applicable uncertainty fields always remain. Retained pages
|
||||
and bundle-based queries use the same `graphql-contract` views without rereading
|
||||
SDL or rebuilding its schema model.
|
||||
|
||||
Types and fields retain individual declaration occurrences, including extensions,
|
||||
list/non-null wrappers, implemented interfaces and union member names. Operation
|
||||
and fragment definitions own field references with aliases and parent-reference
|
||||
IDs. Each reference links to a supplied SDL field occurrence, remains `model-only`
|
||||
for a library-provided field such as `__typename`, or records an exact unknown
|
||||
reason. Rejected schema construction keeps declarations but cannot supply trusted
|
||||
bindings. Missing or duplicate fragment names remain explicit. Fragments are
|
||||
visited once in their declared type scope and linked by spread edges; cycles
|
||||
are never recursively expanded.
|
||||
|
||||
The versioned `graphql-sdl-field-bindings-v1` profile records the GraphQL library
|
||||
version. It performs syntax parsing and schema construction for field resolution;
|
||||
it does not establish whole-schema/operation validity, spread applicability,
|
||||
reachability, authorization or endpoint behavior. Arguments, defaults,
|
||||
descriptions, directive behavior, enum value inventory, introspection JSON,
|
||||
federation composition and version comparison remain outside this profile.
|
||||
|
||||
Every fact has a document role, SHA-256, UTF-16 start/end offsets and parser token
|
||||
line/column. Offsets address decoded UTF-8 text, including an optional BOM, rather
|
||||
than byte positions. The two files are independently verified snapshots, not an
|
||||
atomic pair. Fact IDs identify source occurrences; `field_key` separately supports
|
||||
exact name-based joins and does not assert equivalence across revisions. CLI
|
||||
retains its auxiliary path spelling in `caller_input_paths.operations_path`,
|
||||
outside semantic identity, as well as its primary `caller_input_path`.
|
||||
|
||||
Malformed operation text retains the SDL inventory with partial coverage and a
|
||||
source-located diagnostic. A primary parser refusal or later resource/cancellation
|
||||
failure retains known source identities and available interpreted Evidence;
|
||||
syntax diagnostics do not quote source text. Complete Evidence contains facts,
|
||||
not original document bytes. An exhausted page never upgrades source coverage.
|
||||
|
||||
Each file has the same configurable maximum of 32 MiB. Conservatively charged
|
||||
facts share a 32 MiB allowance; text/AST allocation is admitted against available
|
||||
heap. The parser admits at most 262,144 lexical tokens and 65,536 comments per
|
||||
document; subsequent iterative AST admission limits traversal depth to 128.
|
||||
These are REA resource policies, not GraphQL language limits. Very deep input can
|
||||
be refused by the parser before AST admission. Selected replies reduce transport
|
||||
and retained queries reuse immutable facts; they do not avoid the initial parse.
|
||||
|
||||
The committed SWAPI SDL is an unchanged public application fixture with provenance
|
||||
in `tests/fixtures/graphql/README.md`. It supplies 53 types and 243 fields;
|
||||
verification does not execute its API.
|
||||
|
||||
Large complete inventories can exceed the default MCP response budget even when
|
||||
selected pages fit. Use `export_evidence_bundle` with a caller-selected local
|
||||
path for complete retained Evidence, or CLI complete JSON/bundle publication.
|
||||
A transport refusal does not erase a completed retained analysis. Narrow pages
|
||||
and source coverage remain available on the same continuing connection.
|
||||
+12
-17
@@ -179,31 +179,26 @@ share the application workflow, projection, retention and publication owners.
|
||||
- `schemes_agree_on_signer` (observed): every present block scheme presents
|
||||
the same _set_ of signer certificates. An extra signer in one scheme is
|
||||
disagreement; a signer without a certificate makes it `null` (unknown).
|
||||
A single present block scheme with known signers agrees with itself;
|
||||
v1 is compared separately below.
|
||||
Build-tools 33+ rotate through v3.1 by default, so a rotated app usually
|
||||
reads `false` here and `true` below.
|
||||
- `signer_change_declared_by_rotation` (declared): the signers differ, but
|
||||
each appears in a lineage the APK declares. Read the two together: block
|
||||
schemes with different signers covered by a declared lineage show `false`
|
||||
/ `true`; differing signers without such a lineage show `false` / `false`.
|
||||
A lineage alone does not establish a change between block schemes. It is
|
||||
`null` when a signer's identity is unknown, or when the signers differ and
|
||||
a declared lineage could not be decoded.
|
||||
each appears in a lineage the APK declares. Read the two together: a
|
||||
rotated APK shows `false` / `true`; signers that differ with no declared
|
||||
lineage containing them show `false` / `false`. It is `null` when a
|
||||
signer's identity is unknown, or when the signers differ and a declared
|
||||
lineage could not be decoded.
|
||||
- `v1_signers_match_block_signers` (observed): the v1 signer certificates
|
||||
equal what the lowest block scheme that has signers (v2, else v3, else
|
||||
v3.1) expects v1 to carry. That is its signer set, except that a v3/v3.1
|
||||
signer declaring a rotation lineage expects the lineage's first
|
||||
(original) certificate, as apksig requires. `false` is notable: the JAR
|
||||
signature names a different key than the block signature. When v1 uses
|
||||
the lineage's original key and v3 its newer key, this comparison reads
|
||||
`true`. If v3 is the only present block scheme, agreement is `true` and
|
||||
declared inter-block signer change is `false` even though rotation is
|
||||
present. If v2 also uses the original key, agreement is `false` and the
|
||||
declared inter-block change is `true`. `null` when
|
||||
v1 or a populated block scheme is absent, any signer's identity is unknown (a
|
||||
signature names a different key than the block signature. A key rotation
|
||||
keeps v1 (and v2) on the original key, so it reads `true` while
|
||||
`schemes_agree_on_signer` reads `false`, with or without v2. `null` when
|
||||
v1 or a block scheme is absent, any signer's identity is unknown (a
|
||||
`decode_error`, a `null` certificate or a `subjectKeyIdentifier`
|
||||
`SignerInfo`), or the lineage's original certificate cannot be
|
||||
identified. When identities or a lineage origin are unknown, limitations
|
||||
explain why; absence is shown in the scheme records.
|
||||
identified; a limitation then says why.
|
||||
- `strongest_scheme`, `rotation_present`, `source_stamp_present`,
|
||||
`self_consistent`.
|
||||
|
||||
|
||||
+1
-1
@@ -174,7 +174,7 @@ OpenJDK 25 against a signed aapt2-built probe APK carrying two resource
|
||||
locales — launcher version, on-disk digest agreement, apktool.yml metadata
|
||||
(1.2.3, SDK 24–34), manifest package agreement, default and `de` string
|
||||
tables projected. Parser goldens in the test suite come from the same
|
||||
decode. See `docs/development/testing.md` for the lane summary.
|
||||
decode. See `docs/testing.md` for the lane summary.
|
||||
|
||||
Multi-locale and table-only projection were also verified through real CLI and
|
||||
stdio MCP workflows on Linux with Apktool 3.0.3/JDK 21 against NewPipe 0.29.1,
|
||||
+2
-7
@@ -101,8 +101,8 @@ limitation reports how many inventoried entries were left out. IPA projection
|
||||
still lists every archive component, including `SwiftSupport/`.
|
||||
|
||||
`platforms` lists `macos` for application roots with `Contents/`. It lists `ios`
|
||||
for shallow application roots that have an `Info.plist` at their root. An
|
||||
application root with neither, such as an empty inventory, has no platform.
|
||||
for shallow application roots that have an `Info.plist`. An application root
|
||||
with neither, such as an empty inventory, has no platform.
|
||||
|
||||
## macOS bundle anatomy
|
||||
|
||||
@@ -149,11 +149,6 @@ directories rather than `Versions/Current`. A framework's `info_plist_path` is
|
||||
because `Versions/Current` decides which one applies) or when its single
|
||||
version directory has no `Resources/Info.plist`.
|
||||
|
||||
A `shallow` bundle's `info_plist_path` is its root `Info.plist`. Without one,
|
||||
it is `Resources/Info.plist`: a flattened macOS framework keeps its binary,
|
||||
`_CodeSignature/`, and `Resources/` directly in the `.framework`, like a single
|
||||
`Versions/<version>/` directory without the `Versions/` wrapper.
|
||||
|
||||
AppleDouble sidecar files (`._*` and `__MACOSX/`) are inventory facts, but they
|
||||
describe neighbouring files. They are excluded from roots, bundle roles, and the
|
||||
anatomy components.
|
||||
+1
-1
@@ -87,7 +87,7 @@ that matches none of those rules is `unclassified`.
|
||||
|
||||
Opened Mach-O and dSYM UUID joins remain unobserved. Symbol names printed
|
||||
inside the report are recorded claims and are not verified. Linux ELF cores
|
||||
use the separate [recorded crash](guides/native/recorded-crashes.md) workflow.
|
||||
use the separate [recorded crash](recorded-crashes.md) workflow.
|
||||
|
||||
## Limits and failures
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Production Composition"
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>typed capability bindings: construction, availability and retryable lifetimes<br/>pure capability declarations feed provider catalogs"]
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>static WASM/EVM descriptors: named contracts, factories, retryable lifetimes"]
|
||||
end
|
||||
|
||||
subgraph "Adapters"
|
||||
@@ -23,7 +23,6 @@ graph TD
|
||||
SESSION["SessionProviderRouter + BinarySession<br/>one immutable deep binding per target<br/>disjoint auxiliary routing"]
|
||||
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback<br/>ProviderInspectionScope: prerequisite observation lifetime"]
|
||||
RECORDS["src/application/investigation/<br/>composed InvestigationRecords + narrow ports<br/>EvidenceLedger and Unknown ownership"]
|
||||
LIFETIMES["CapabilityScope<br/>host-owned service close capabilities<br/>released peers removed; failed cleanup retained for retry"]
|
||||
SNAPSHOT["BinarySessionRecords + AnalysisSnapshotCache<br/>binary target/profile snapshots and session file ownership<br/>record coordination and post-commit notifications"]
|
||||
JS_RECON["src/application/javascript/ + src/artifacts/javascript/<br/>JavaScript artifact reconstruction<br/>directory/ASAR reads<br/>AST-only bundle projection"]
|
||||
end
|
||||
@@ -100,7 +99,6 @@ graph TD
|
||||
MAIN --> COMPOSITION
|
||||
CLI --> COMPOSITION
|
||||
SERVER --> COMPOSITION
|
||||
COMPOSITION --> LIFETIMES
|
||||
COMPOSITION --> SESSION
|
||||
APP --> FIRMWARE
|
||||
FIRMWARE --> PROCESS
|
||||
@@ -151,7 +149,8 @@ graph TD
|
||||
OBJDUMP --> DOMAIN
|
||||
COMPOSITION --> RIZIN
|
||||
RIZIN --> DOMAIN
|
||||
COMPOSITION --> GDB
|
||||
CLI --> GDB
|
||||
SERVER --> GDB
|
||||
GDB --> PROCESS
|
||||
GDB --> DOMAIN
|
||||
HOPPER --> HOPPER_BRIDGE
|
||||
@@ -207,7 +206,8 @@ graph TD
|
||||
NATIVE_READERS --> DOMAIN
|
||||
COMPOSITION --> FORMAT_READERS
|
||||
FORMAT_READERS --> DOMAIN
|
||||
COMPOSITION --> RUNTIME
|
||||
SERVER --> RUNTIME
|
||||
CLI --> RUNTIME
|
||||
RUNTIME --> DOMAIN
|
||||
PROCESS --> FILESYSTEM
|
||||
AUX --> FILESYSTEM
|
||||
@@ -9,11 +9,9 @@ ordinary binary inspection use the same modular capability.
|
||||
## Bring your own engine
|
||||
|
||||
Real verification covers **Linux x64** hosts and **EXEC/DYN/REL** objects in
|
||||
these profiles: ELF64 x86-64, ELF64 little-endian AArch64, ELF32 i386, ELF32
|
||||
little-endian ARM and ELF32 little- or big-endian MIPS. Other machines, classes
|
||||
and byte orders, including big-endian ARM, return `unsupported_target`. Other
|
||||
hosts, including Linux arm64, return `capability_unavailable` naming the
|
||||
detected host before any target is read. Provide an absolute
|
||||
these profiles: ELF64 x86-64, ELF32 i386, ELF32 little-endian ARM and ELF32
|
||||
little- or big-endian MIPS. Other machines, classes and byte orders, including
|
||||
AArch64 and big-endian ARM, return `unsupported_target`. Provide an absolute
|
||||
Python executable whose environment already contains unchanged pwntools 4.15.0,
|
||||
pyelftools 0.33 and Unicorn 2.1.2:
|
||||
|
||||
@@ -125,7 +123,7 @@ reply budget applies to the decoder record; the CLI/MCP Evidence envelope adds
|
||||
copies and encoding overhead. The
|
||||
pinned MCP SDK defaults to a 10 MiB receive buffer. Large complete results need
|
||||
a caller-configured `StdioClientTransport({ maxBufferSize: ... })` and a matching
|
||||
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](../../reference/mcp-contracts.md#tool-results).
|
||||
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](mcp-contracts.md#tool-results).
|
||||
These settings affect delivery, not decoder coverage or limits. The large
|
||||
extended-index verification uses 256 MiB and an explicit five-minute request
|
||||
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to at most 3 GiB virtual address space,
|
||||
@@ -426,10 +424,8 @@ rea inspect-native-exports ./libexample.so --detail summary --evidence_output ./
|
||||
without an export directory is a complete empty inventory.
|
||||
- ELF rows list defined `.dynsym` symbols with global, weak or GNU-unique
|
||||
binding and default or protected visibility. Each row keeps `value`
|
||||
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`,
|
||||
`binding` and `visibility` (`"default"` or `"protected"`; `"unknown"` in older
|
||||
historical Evidence where visibility was not recorded). ELF string tables do
|
||||
not declare a text encoding. Names that
|
||||
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`
|
||||
and `binding`. ELF string tables do not declare a text encoding. Names that
|
||||
contain non-ASCII bytes are decoded as UTF-8 when those bytes form valid
|
||||
UTF-8; `name_utf8_decodable` records whether the non-ASCII name bytes are
|
||||
valid UTF-8 and `name_bytes_base64` retains those exact bytes (`null` for
|
||||
@@ -1,62 +0,0 @@
|
||||
# Binary hardening
|
||||
|
||||
Use `inspect_binary_hardening` to read compile-time and link-time hardening from
|
||||
an explicit local ELF, PE, or Mach-O file. No disassembler, pwntools install, or
|
||||
target execution is required. Universal Mach-O files return one image per slice.
|
||||
|
||||
```sh
|
||||
rea inspect-binary-hardening /absolute/path/to/program --json
|
||||
rea inspect-binary-hardening /absolute/path/to/program --max-file-bytes 1048576 --json
|
||||
```
|
||||
|
||||
MCP accepts `{ "path": "/absolute/path/to/program" }`. CLI paths may be relative
|
||||
to the current directory; MCP paths must be absolute.
|
||||
|
||||
Each image has one row per property: `property`, `state`, `confidence`, `basis`,
|
||||
`location`, `note`, and `next_probe`. States are `enabled`, `disabled`,
|
||||
`partial`, `unknown`, and `not_applicable`. `confidence` is `observed` for a
|
||||
single header or import, and `derived` when the state combines several fields,
|
||||
such as ELF PIE or full RELRO. An image's `architecture` is its CPU family, or
|
||||
null when REA has no family for it. `machine` and `slice.architecture` keep the
|
||||
exact identity, such as `arm64e`.
|
||||
|
||||
ELF rows cover PIE (`ET_DYN` with `PT_INTERP` or `DF_1_PIE`), NX (`PT_GNU_STACK`;
|
||||
a missing header is `unknown` and the architecture default is only noted), RELRO
|
||||
(`PT_GNU_RELRO` plus `BIND_NOW`), stack protector and FORTIFY undefined dynamic
|
||||
imports, CET and AArch64 BTI/PAC GNU property notes, `DT_RPATH`, `DT_RUNPATH`, and
|
||||
whether a `SHT_SYMTAB` section is present. Only an undefined dynamic import
|
||||
counts. Imports come from the table `DT_SYMTAB` declares. A `SHT_DYNSYM` section
|
||||
only supplies the symbol count when `DT_HASH` and `DT_GNU_HASH` leave it open,
|
||||
and only if it names the same symbol and string tables. Without that count the
|
||||
import rows are `unknown` and give the reason. A missing `__stack_chk_fail` or fortify import is `unknown`. That
|
||||
absence does not establish that the protection was left out of a static or
|
||||
stripped build. A present import is a sign that some code uses the protection,
|
||||
not proof that all code does. NX, RELRO, `DT_RPATH`, and `DT_RUNPATH` are
|
||||
`not_applicable` when `e_type` is neither `ET_EXEC` nor `ET_DYN`, and `unknown`
|
||||
for a linked image with no readable program headers.
|
||||
|
||||
PE rows read `DllCharacteristics` for dynamic base, high-entropy VA, NX
|
||||
compatibility, force integrity, no SEH, and guard CF. `cfg_instrumented` is the
|
||||
load-config `GuardFlags` linker claim when `Size` covers that field. An
|
||||
unreadable load configuration, including a TLS failure in that parser, leaves
|
||||
only `cfg_instrumented` `unknown`. Authenticode and debug rows report certificate
|
||||
and debug directory presence, not signature validity or debug content. A
|
||||
directory whose address and size disagree is `unknown`.
|
||||
|
||||
Mach-O rows read `MH_PIE`, `MH_ALLOW_STACK_EXECUTION`, `MH_NO_HEAP_EXECUTION`,
|
||||
an `__RESTRICT` segment, `LC_CODE_SIGNATURE` presence, encryption from the
|
||||
existing encryption reader, the arm64e subtype, and a stack-protector import.
|
||||
`hardened_runtime` is `unknown` with `next_probe` `inspect_signature`.
|
||||
CodeDirectory flags stay with that tool.
|
||||
|
||||
The report states presence of build-time features. It does not state
|
||||
exploitability. `inspect_binary_layout` is unchanged.
|
||||
|
||||
The default read budget is 64 MiB (`max_file_bytes`, ceiling 512 MiB). A changed
|
||||
file is refused. A malformed header, a Mach-O segment or encryption command that fails admission, or a program, section,
|
||||
or segment payload declared outside the file returns no partial report, except that one bad slice in a universal Mach-O leaves the other slices. A symbol table or
|
||||
symbol name over its budget leaves the import rows `unknown` and keeps the
|
||||
header rows. A GNU property note over its budget, or a compressed note section
|
||||
that may hold one, leaves the CET and BTI/PAC rows `unknown` and keeps the header
|
||||
rows. A dynamic-table budget still fails the ELF report. The image is never
|
||||
executed.
|
||||
@@ -31,7 +31,7 @@ authority are validated independently before selection and connection.
|
||||
|
||||
For a compact route query, select `detail: "summary"` and a
|
||||
`view: {kind: "javascript-routes", collection: "routes", limit: 20}`. The
|
||||
shared [route inventory](../../javascript-route-inventory.md) preserves source IDs,
|
||||
shared [route inventory](javascript-route-inventory.md) preserves source IDs,
|
||||
precise locations, partial coverage, and the complete parent Evidence for
|
||||
subsequent filtered pages. Static candidates never establish complete application
|
||||
coverage or reachability.
|
||||
@@ -52,11 +52,7 @@ observations may omit the coverage field; treat omission as `unknown` rather
|
||||
than inferring completeness from the array.
|
||||
|
||||
Passive DOM comparison checks complete node inventories and complete frame
|
||||
parent/child structure independently. Node value lengths are comparable only
|
||||
when both nonempty snapshots declare the same `node_value_length_unit`.
|
||||
Missing or different units leave length comparison unknown; independent node
|
||||
or frame structural differences can still prove change. Empty inventories have
|
||||
no value lengths to compare. Frame IDs and sibling enumeration order
|
||||
parent/child structure independently. Frame IDs and sibling enumeration order
|
||||
are capture-local and do not establish a change; duplicate IDs, missing parents,
|
||||
and cyclic ancestry remain unknown. A complete frame difference can still prove
|
||||
change when DOM nodes are incomplete, and vice versa. Partial DOM or metadata
|
||||
@@ -147,7 +143,7 @@ Caller-owned JSON/base64 strings, input parsing and validation, transport
|
||||
buffers, garbage-collector behavior, and other runtime memory are outside this
|
||||
estimate; the budget does not promise a bound on total process memory.
|
||||
|
||||
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](../javascript/electron-observation.md).
|
||||
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](electron-observation.md).
|
||||
|
||||
## Provider authorities
|
||||
|
||||
@@ -177,7 +173,7 @@ See [the reproduction and Chromium ownership limitation](https://github.com/morl
|
||||
|
||||
Existing static application Evidence and passive web/Electron captures can be
|
||||
combined later through
|
||||
[JavaScript static/runtime reconciliation](../javascript/javascript-runtime-reconciliation.md).
|
||||
[JavaScript static/runtime reconciliation](javascript-runtime-reconciliation.md).
|
||||
|
||||
## Start a browser
|
||||
|
||||
@@ -404,19 +400,7 @@ contextual evidence with `timestamp: null`, accompanied by a limitation stating
|
||||
that it does not establish activity in the selected window. Main-frame scope
|
||||
enforcement still applies to contextual navigation events. Network redirect
|
||||
events retain their paired prior-hop response context when the timestamped
|
||||
next-request event falls inside the window. A redirect row's optional
|
||||
`redirect_response` contains the prior response `url`, CDP-reported `status`,
|
||||
and `scope`. The URL has transport credentials removed; URL and status are
|
||||
withheld unless its origin is approved. Missing or malformed response URLs have
|
||||
null scope and add `invalid_protocol_value` timeline coverage; a valid URL with an
|
||||
unsupported protocol has `unsupported` scope. Outside-policy and unsupported
|
||||
responses mark the timeline as policy-filtered with `disallowed_origin` or
|
||||
`unsupported_url`. Missing status remains null rather than zero. A supplied
|
||||
non-numeric status is also null and records `invalid_protocol_value` coverage;
|
||||
the other valid response facts remain available. Historical rows may omit this
|
||||
field; omission does not establish that no prior response existed. Response
|
||||
headers and bodies are discarded. The row timestamp belongs to the next request,
|
||||
not an exact prior response time. The pre-arm timeline buffer is bounded
|
||||
next-request event falls inside the window. The pre-arm timeline buffer is bounded
|
||||
by original CDP message bytes. If clock sampling completes after that budget is
|
||||
exhausted, the operation returns `payload_limit` instead of a partial window.
|
||||
Cancellation and clock-request failures retain their own outcomes. Session
|
||||
@@ -465,7 +449,7 @@ not bound the whole process heap.
|
||||
REA retains local observation data and removes only URL userinfo credentials, explicitly declared scenario secrets, and values from structured credential/storage fields:
|
||||
|
||||
- URL observations preserve the complete local URL, including query values, fragments, duplicate parameters, and their order. Only username/password userinfo is removed; malformed original URL strings remain available with `origin: null`.
|
||||
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values. `dom.node_value_length_unit` is `utf16-code-units` for web observations and applies to every node's `node_value_length`; it counts JavaScript string code units, not Unicode code points or UTF-8 bytes. Electron observations declare `utf8-bytes` instead. Historical captures without the unit leave it unknown; reading them does not infer a unit or convert their stored lengths.
|
||||
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values.
|
||||
- Supported DOM URL attributes are projected separately as resolved destinations. Empty `action` on HTML or XHTML forms and empty `formaction` on HTML or XHTML submit controls resolve to the document URL; other relative URL attributes resolve against the captured document base.
|
||||
- Accessibility structure and reported roles are retained by default, while names and descriptions require `include_accessibility_text: true`. The browser protocol may omit an individual role; that role remains unknown for comparison. Other observed accessibility differences can still establish a change.
|
||||
- Network observations retain method, status, MIME type, size, type, initiator stack location, and complete URLs with userinfo credentials removed. Redirects reported by CDP are attached as ordered `redirects` on the one request ID, with the prior request URL, CDP response URL, method/type, response status/MIME/encoded length, request timestamp, and the next `requestWillBeSent` event timestamp. CDP delivers `redirectResponse` with that next request event, so the latter is an event boundary rather than an exact response timestamp. Redirect response headers and bodies are discarded; hops outside the selected origins are discarded with the request chain. Headers are discarded after an allowlisted projection of length/encoding, structured CSP/Link/policy fields, and untrusted agent hints. Cookies and authorization headers are never retained.
|
||||
@@ -500,7 +484,7 @@ REA retains local observation data and removes only URL userinfo credentials, ex
|
||||
connection/frame inventories; matching direction/opcode sequences alone do
|
||||
not prove alignment after drops. Redirect resource types use only shared known metadata:
|
||||
an omitted type leaves equality unknown rather than proving a modification.
|
||||
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. Source locations are CDP's zero-based lines and UTF-16 columns. A negative column, which Chrome reports on the first line of some inline scripts, is kept as reported and does not locate a column in the source URL; an invalid line or a non-integer column is `null` (WebMCP `registration_source` is `null`) and is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
|
||||
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
|
||||
- WebSocket observations retain direction, opcode, and payload byte length. A missing or malformed CDP opcode is returned as `null`, with opcode-dependent payload byte length and shape also `null`, and marks `websocket_frames` incomplete. Missing or wrong-typed payload data is not treated as empty; its byte length is `null`, and a text payload shape is unknown. Numeric opcode `0` and an observed empty payload remain distinct from missing data. Opcode is nullable in the public inspection schema, so clients must handle unknown protocol values. When shape capture is selected, text messages are classified as text or complete value-free JSON shape; binary bytes, hashes, prefixes, and raw payloads are never retained.
|
||||
- Storage observations always redact values. Key names, IndexedDB names, and cache names require `include_storage_keys`; stable content fingerprints require the additional `include_storage_fingerprints` selection. Cookie fingerprints cover cookies applicable to the current main-frame URL, including its path, rather than every cookie path on that origin. Other storage remains scoped to the origin. Cache fingerprints hash all decoded response bytes, including bodies above 64 KiB, subject to the [64 MiB per-message CDP limit](#request-boundary). Missing or undecodable cache bodies and partial IndexedDB remote objects make fingerprint coverage incomplete, so identical observations remain `unknown`.
|
||||
- Unavailable or malformed storage inventories mark `storage_keys` unavailable, even without fingerprints. Comparisons suppress inferred key additions and removals while preserving modifications proven by complete fingerprints on both sides. Previously exported captures are not retroactively corrected; recapture them to obtain these coverage markers.
|
||||
@@ -9,10 +9,7 @@ liveness deadlines. `INPUT_JSON` may be inline JSON or a JSON file path.
|
||||
|
||||
The minimal request contains only browser launch/connect selection, `start_url`,
|
||||
and at least one `actions` entry. Launch `headless` defaults to `true` and can
|
||||
be set to `false` where the host supports a visible browser. Launched Chromium
|
||||
keeps its sandbox; on hosts that cannot provide one, such as root in a
|
||||
container, set `REA_BROWSER_NO_SANDBOX=true` for the REA process. Termux on
|
||||
Android always launches without it. Navigations follow
|
||||
be set to `false` where the host supports a visible browser. Navigations follow
|
||||
the destinations named by the scenario and the page's ordinary browser requests,
|
||||
and [request steps](#request-steps) send only to the start URL's origin;
|
||||
there is no separate origin allowlist. Request steps use a temporary dispatch
|
||||
@@ -414,10 +411,8 @@ data you may change. This scenario creates one item:
|
||||
- Request steps follow no redirects: a redirect answer fails the step, and
|
||||
nothing is sent to the location it names, though a write the redirect
|
||||
answered may already have been applied. `expected_status` cannot name a 3xx
|
||||
status. Manual redirects can be opaque to the page and may not produce a
|
||||
response event; their status and `Location` can therefore be unavailable even
|
||||
when network events and header values were selected. Such captures report
|
||||
incomplete event coverage.
|
||||
status. Give the path the redirect names; with header values selected, the
|
||||
capture records its `Location`.
|
||||
- Header values are literals, declared secrets, a cookie the browser holds for
|
||||
the request URL (`{"source": "cookie", "name"}`, HttpOnly cookies included,
|
||||
whose value the page's scripts can then see while the request runs), or a
|
||||
@@ -1,62 +0,0 @@
|
||||
# Bun standalone executables
|
||||
|
||||
`inspect-artifact` inventories embedded Bun module graphs in Mach-O `__BUN,__bun`,
|
||||
PE `.bun`, ELF `.bun`, and appended trailers. `extract-artifact` writes their stored
|
||||
module contents through the existing safe output tree. Neither operation executes
|
||||
the executable or decompiles bytecode.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./hello --json
|
||||
rea extract-artifact ./hello --json
|
||||
```
|
||||
|
||||
In MCP, open the executable with `open_binary`, then call `inspect_artifact`.
|
||||
Use `detail: "summary"` for large graphs and `inspect_analysis_view` to page retained
|
||||
Evidence. The root metadata's `entry_point_logical_path` is the reader-owned
|
||||
location of its entry module. Use it as an inventory `logical_paths` filter,
|
||||
then read selected node fields with the returned content identity. This also
|
||||
works for names mapped to indexed extraction paths. The complete Evidence stays
|
||||
retained; no source-name normalization is needed in the caller.
|
||||
|
||||
The root reports the carrier, layout, module count, entry point, raw flags and
|
||||
compile exec-argv. Each module retains its source name, loader and encoding values,
|
||||
entry-point status, sizes and SHA-256. Sourcemaps, bytecode and module-info regions
|
||||
are separate children. Source names such as `/$bunfs/root/main.js` remain in
|
||||
metadata; extraction uses a relative path. Unsafe or non-UTF-8 names receive an
|
||||
indexed path and retain their source spelling or bytes. Virtual names deeper
|
||||
than 256 logical path parts also use an indexed path with an explicit limitation,
|
||||
preventing unbounded path-tree work while retaining the source fact. Duplicate names remain
|
||||
separate occurrences; ordinary extraction collision rules apply.
|
||||
|
||||
The recognized record shapes are the 36-byte Bun 1.3.3 layout and the 52-byte
|
||||
Bun 1.3.9 layout. These labels identify structural layouts, not a detected compiler
|
||||
version. Bun does not store an explicit revision identifier: unknown table sizes
|
||||
and ambiguous known layouts are unsupported. A future layout that is byte-for-byte
|
||||
compatible cannot be distinguished. Invalid pointers and truncated claimed sections
|
||||
are malformed. An absent trailer without a named section does not establish a Bun
|
||||
graph. Unknown enum values remain raw, with a null name.
|
||||
|
||||
Sourcemaps are stored Bun regions, not necessarily standalone JSON source maps.
|
||||
Extracted contents retain their original encoding. Use JavaScript analysis only on
|
||||
compatible source files; bytecode is retained as evidence, not decoded.
|
||||
|
||||
Module parsing reads exact metadata ranges rather than loading stored asset
|
||||
bodies or whole pages around small pointers. A cache retains at most 256 KiB
|
||||
and 1,024 ranges for one reader preparation. Cumulative uncached metadata reads
|
||||
are limited to 256 MiB, checked before allocation or I/O. The module table,
|
||||
decoded strings and combined logical path tree have separate admission checks;
|
||||
at most 250,000 distinct path prefixes are admitted before inventory construction.
|
||||
Stored bodies are range-streamed for hashing and extraction. Header/suffix observations are reused
|
||||
only for the same admitted file state, with selected-path checks before publication.
|
||||
|
||||
See [limits and recovery](design/limits-and-recovery.md) for admission ceilings.
|
||||
Synthetic tests cover the carriers and malformed declarations. Run
|
||||
`node scripts/verify-bun-standalone.mjs` after `npm run build:cached` for local Bun
|
||||
verification, and add `--docker` to compile a Linux fixture with `oven/bun:1.4.0`.
|
||||
Bun and Docker are fixture producers only and are not installed by REA.
|
||||
|
||||
Layout references: [Bun 1.3.3 serializer](https://github.com/oven-sh/bun/blob/bun-v1.3.3/src/StandaloneModuleGraph.zig)
|
||||
and [Bun 1.3.9 serializer](https://github.com/oven-sh/bun/blob/bun-v1.3.9/src/StandaloneModuleGraph.zig).
|
||||
|
||||
The current 52-byte record order can also be checked in the
|
||||
[Bun 1.4.4 Rust serializer](https://github.com/oven-sh/bun/blob/bun-v1.4.4/src/standalone_graph/StandaloneModuleGraph.rs).
|
||||
@@ -1,107 +0,0 @@
|
||||
# Capability composition and ownership
|
||||
|
||||
REA keeps domain contracts and shared workflows separate from provider adapters
|
||||
and CLI/MCP translation. A capability can have several provider identities; a
|
||||
provider can contribute operations to several capabilities. Those associations
|
||||
are source facts, while host availability is an observation.
|
||||
|
||||
## Source declarations and runtime bindings
|
||||
|
||||
`src/composition/capabilityDeclarations.ts` contains static provider identities,
|
||||
named contract-group references and precise operation subsets. Complete groups
|
||||
refer to `TOOL_CONTRACT_GROUPS` in the existing contract owner; family definitions
|
||||
and provider catalogs read the same arrays, so a new group operation is included
|
||||
without updating a copied provider list. Metadata imports only the group ID type
|
||||
and pure metadata leaves; it must not load factories, schemas, engines or
|
||||
generated catalogs. Catalog generation resolves groups and subsets against the
|
||||
canonical contracts and rejects unknown groups/operations and conflicting
|
||||
identities. New provider provenance must come from the implementation
|
||||
that actually produces the Evidence, including format-specific capture adapters
|
||||
and derived workflows.
|
||||
|
||||
`src/composition/capabilityServices.ts` binds typed service factories to those
|
||||
capability keys. Each runtime entry owns construction, cleanup and any available
|
||||
host/configuration facts or live probes. A service entry requires a close
|
||||
capability; entries adapting managers retain their existing `closeAll` operation.
|
||||
The host maps entries without erasing their individual service/port types. There
|
||||
is no generic JSON operation dispatcher.
|
||||
|
||||
These two declarations have different loading needs. Static facts can be read
|
||||
without loading a schema or provider graph. The full runtime composition is
|
||||
materialized only when the MCP runtime needs it. One-shot CLI adapters import the
|
||||
selected factory rather than constructing a complete capability host. CLI and MCP
|
||||
reuse the same provider/service factories and application workflows, while
|
||||
keeping their respective argument translation and lifetime.
|
||||
|
||||
## Resource lifetime
|
||||
|
||||
`CapabilityScope` registers a module's actual close capability when its instance
|
||||
is composed. There is no independent server provider-close list to update.
|
||||
Concurrent shutdown calls join the same cleanup attempt. A successful peer is
|
||||
removed from pending cleanup; failed peers remain owned and are retried on the
|
||||
next close. Synchronous throws and rejections without a reason remain failures,
|
||||
and cannot prevent other peers from attempting cleanup. Shutdown seals further
|
||||
ownership registration.
|
||||
|
||||
The host distinguishes these lifetimes:
|
||||
|
||||
- Host-owned service instances are released by that host.
|
||||
- Caller-owned optional observations remain borrowed by SDK server instances;
|
||||
their process-level owner closes them, including discarded discovery probes.
|
||||
- CLI invocations use their existing finalizers to join cleanup and preserve the
|
||||
original observation, cancellation and cleanup errors.
|
||||
- Binary engines, bundle readers and investigation records retain their existing
|
||||
owners. The `BinarySession` transition/drain barrier still coordinates target
|
||||
changes, reset and disposal. A module refactor must not create independent
|
||||
barriers that allow late work to repopulate reset records.
|
||||
|
||||
An application workflow may borrow a shared resource scope, such as byte entropy
|
||||
using the host's artifact scope. Its wrapper does not acquire a second ownership
|
||||
claim over the borrowed scope.
|
||||
|
||||
## Availability and registration
|
||||
|
||||
Module bindings provide host support, selected configuration and precisely typed
|
||||
live availability probes through `CapabilityAvailabilityFacts`. Constructors do
|
||||
not run these probes or launch engines. Session discovery invokes the existing
|
||||
provider-owned probes only for advertised families; live failures remain fresh
|
||||
observations. Caller policy, selected environments and injected providers retain
|
||||
their existing precedence.
|
||||
|
||||
MCP registration remains in the MCP adapter. It consumes composed services,
|
||||
record ports, admission and logging; it must not add fallback constructors that
|
||||
can escape the host owner. Named contracts bind handlers and schemas. The complete
|
||||
tool-family registrar map is checked by TypeScript, and catalog order never
|
||||
selects a handler or schema.
|
||||
|
||||
## Extending a capability
|
||||
|
||||
1. Add or reuse the domain contract and application workflow, with explicit
|
||||
Evidence provenance, coverage, unknowns and caller choices.
|
||||
2. Put concrete construction in `src/composition/` and keep its default factory
|
||||
lazy. Reference adapter-owned pure identity metadata from both the provider
|
||||
and its static capability declaration.
|
||||
3. Add the runtime binding if a host-owned service is needed. Bind cleanup and
|
||||
availability/probes there, preserving borrowed versus owned resources.
|
||||
4. Add the CLI/MCP argument adapters and their named registrations. A one-shot
|
||||
command uses the shared selected factory and existing finalization helpers.
|
||||
5. Run source-boundary, discovery, SDK-schema and relevant public workflow checks.
|
||||
Exercise partial cleanup, retry, cancellation and caller-owned peers at the
|
||||
applicable production seam; use actual applications/providers for routing and
|
||||
complete Evidence verification. Recording ports do not prove real providers.
|
||||
|
||||
The constructor boundary guard covers the centralized Frida, Cutter, GDB and
|
||||
Rizin constructors without rejecting legitimate type-only injected interfaces.
|
||||
Other settled pure/provider boundaries continue checking type imports too. The
|
||||
shared parser records module-edge erasure once. Whole-declaration type imports
|
||||
are erased; inline type bindings retain their runtime module edge under
|
||||
`verbatimModuleSyntax`. Static dynamic imports remain runtime edges too.
|
||||
Tests belong with their semantic owner and must be
|
||||
found exactly once; provider roots need the corresponding adapter discovery lane.
|
||||
|
||||
This is an internal composition contract. External plugin distribution, automatic
|
||||
provider installation and a universal provider interface are separate concerns.
|
||||
|
||||
The [initial verification report](verification/capability-composition-2026-11-01.md)
|
||||
records the executed public workflows, cleanup boundaries, resource measurements
|
||||
and remaining provider qualification gaps for this composition increment.
|
||||
+2
-2
@@ -65,14 +65,14 @@ cancellation can still prevent upload.
|
||||
that analysis was complete. A successful operation can retain partial analysis,
|
||||
unknowns, and limitations. Exit zero is not a security verdict, proof of complete
|
||||
coverage, or a policy gate for those findings. Inspect the saved Evidence and
|
||||
coverage before making such a decision. See [output and exit status](getting-started/cli.md#output-and-exit-status).
|
||||
coverage before making such a decision. See [output and exit status](cli.md#output-and-exit-status).
|
||||
|
||||
A nonzero exit can represent invalid input, unavailable tooling, an unsupported
|
||||
host/capability, or another operation failure. Keep the original JSON error (when
|
||||
emitted), stderr, and exit code, and use their category/message to diagnose it.
|
||||
An early startup failure may leave `analysis.json` empty; do not parse that as
|
||||
successful analysis. Select a supported operation or explicitly provision its
|
||||
[documented prerequisites](getting-started/installation.md); do not silently switch providers,
|
||||
[documented prerequisites](installation.md); do not silently switch providers,
|
||||
execute the artifact, or ignore the failure to obtain a green job.
|
||||
|
||||
To exercise the failure path, temporarily set `ARTIFACT_PATH` to
|
||||
|
||||
@@ -4,7 +4,7 @@ Use the CLI for a direct inspection or a script. It uses the same application
|
||||
workflows and evidence contracts as REA's MCP server. Each CLI invocation is a
|
||||
separate process; save results when you want to use them in a later command.
|
||||
For a pinned, noninteractive workflow with retained reports, see
|
||||
[static analysis in GitHub Actions](../ci-analysis.md).
|
||||
[static analysis in GitHub Actions](ci-analysis.md).
|
||||
|
||||
## Run a command
|
||||
|
||||
@@ -19,7 +19,7 @@ Use an extracted application directory or an ASAR as the target. Static
|
||||
JavaScript analysis returns the application graph and Evidence directly.
|
||||
Generic `rea analyze PATH` selects this workflow for directories and `.asar`
|
||||
files when neither `--provider` nor `--snapshot` is supplied. See
|
||||
[JavaScript artifact reconstruction](../guides/javascript/javascript-artifact-reconstruction.md)
|
||||
[JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
|
||||
for results, integrity checks and coverage.
|
||||
|
||||
The complete application Evidence can be hundreds of megabytes for a real
|
||||
@@ -39,17 +39,6 @@ pass an absolute path. REA reads and authenticates the file, then projects it
|
||||
like retained Evidence, so a large file is bounded by the file reader rather
|
||||
than charged as inline input.
|
||||
|
||||
An argument that accepts inline JSON or a JSON file path also accepts `-`, which
|
||||
reads the request from standard input, so a generated request needs no
|
||||
temporary file:
|
||||
|
||||
```bash
|
||||
jq -nc '{source: {kind: "evidence-file", path: "app-evidence.json"}, view: {kind: "summary"}}' | rea inspect-analysis-view -
|
||||
```
|
||||
|
||||
Standard input is read once as a single UTF-8 JSON document without a
|
||||
byte-order mark, and it is limited to 8 MiB. Put a larger request in a file.
|
||||
|
||||
Local file inspection commands reject empty paths and NUL bytes before resolving
|
||||
relative inputs, auxiliary files or output destinations. An empty string never
|
||||
selects the working directory; use an explicit `.` when selecting that directory
|
||||
@@ -59,7 +48,7 @@ For example, a 334 MB Obsidian application Evidence file yields a summary view
|
||||
of about 10 KB. Replace the view with
|
||||
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
|
||||
module page. See
|
||||
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md) for
|
||||
[JavaScript application workflows](javascript-application-workflows.md) for
|
||||
views, feature traces and comparisons that take the same saved Evidence.
|
||||
|
||||
Add `--timings` to a command to write one JSON timing record to stderr. For
|
||||
@@ -176,15 +165,6 @@ into the current shell:
|
||||
eval "$(rea completions bash)"
|
||||
```
|
||||
|
||||
For zsh, initialize completion before loading the hook. If your shell already
|
||||
runs `compinit`, place the hook after that existing initialization:
|
||||
|
||||
```zsh
|
||||
autoload -Uz compinit
|
||||
compinit
|
||||
eval "$(rea completions zsh)"
|
||||
```
|
||||
|
||||
The Bash hook supports macOS system Bash 3.2. It uses optional completion
|
||||
features when the shell provides them and falls back to portable registration.
|
||||
Completion includes standard global options and the active command's options;
|
||||
@@ -226,7 +206,7 @@ saving the complete original Evidence in a new bundle. Summary requires that
|
||||
explicit destination; existing files are preserved. Later
|
||||
`inspect-analysis-view` calls can page the exact parent from the saved bundle
|
||||
without reopening the assembly. See
|
||||
[CLI summary and saved member rows](../guides/managed/managed-code-analysis.md#cli-summary-and-saved-member-rows)
|
||||
[CLI summary and saved member rows](managed-code-analysis.md#cli-summary-and-saved-member-rows)
|
||||
for the complete command sequence. Omitted or `--detail complete` retains the
|
||||
full inline response.
|
||||
|
||||
@@ -234,8 +214,8 @@ Choose the workflow from the artifact's contents rather than its `.exe` or
|
||||
`.dll` extension. The managed parser requires PE/CLI metadata; it does not
|
||||
unpack .NET single-file hosts or decode IL2CPP. For the image, type and method
|
||||
names in a Unity `global-metadata.dat`, use
|
||||
[IL2CPP metadata](../il2cpp-metadata.md). NativeAOT method bodies require native
|
||||
analysis. See the [managed-code guide](../guides/managed/managed-code-analysis.md)
|
||||
[IL2CPP metadata](il2cpp-metadata.md). NativeAOT method bodies require native
|
||||
analysis. See the [managed-code guide](managed-code-analysis.md)
|
||||
for supported deployment forms, selection semantics, and coverage limits.
|
||||
|
||||
## SQLite snapshots
|
||||
@@ -243,7 +223,7 @@ for supported deployment forms, selection semantics, and coverage limits.
|
||||
For a local database snapshot, use `rea inspect-sqlite-database PATH --json`.
|
||||
Add `--table NAME --row-limit 20` to read a bounded sample from an ordinary
|
||||
table. This command does not require a native analysis provider. See
|
||||
[SQLite database snapshots](../guides/data/sqlite-databases.md) for database/WAL identity,
|
||||
[SQLite database snapshots](sqlite-databases.md) for database/WAL identity,
|
||||
source-file handling and row-value semantics.
|
||||
|
||||
## Native analysis
|
||||
@@ -256,7 +236,7 @@ rea inspect-go-binary /absolute/path/to/program --json
|
||||
```
|
||||
|
||||
The shared CLI/MCP workflow reports compiler version, modules, replacements,
|
||||
build settings, original bytes and SHA-256. See [Go binaries](../guides/native/go-binaries.md)
|
||||
build settings, original bytes and SHA-256. See [Go binaries](go-binaries.md)
|
||||
for supported containers and absent or malformed metadata.
|
||||
|
||||
Inspect a Godot PCK without running the game:
|
||||
@@ -267,7 +247,7 @@ rea extract-artifact game.pck --json
|
||||
```
|
||||
|
||||
Formats 2, 3 and 4 retain header, directory and integrity facts. Compact delivery
|
||||
keeps complete Evidence in the requested bundle. See [Godot PCK](../godot-pck.md)
|
||||
keeps complete Evidence in the requested bundle. See [Godot PCK](godot-pck.md)
|
||||
for embedded packs, batch queries and unknown encrypted coverage.
|
||||
|
||||
Inspect a UnityFS AssetBundle without running Unity:
|
||||
@@ -278,43 +258,9 @@ rea extract-artifact content.bundle --json
|
||||
```
|
||||
|
||||
Format versions 6 through 8 retain header, block and directory facts with
|
||||
none/LZ4/LZ4HC block decompression. See [UnityFS](../unityfs.md) for encryption
|
||||
none/LZ4/LZ4HC block decompression. See [UnityFS](unityfs.md) for encryption
|
||||
and LZMA coverage boundaries.
|
||||
|
||||
Inspect a WeChat mini-program package without WeChat or an unpacker:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact app.wxapkg --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact app.wxapkg --json
|
||||
```
|
||||
|
||||
Plaintext packages retain header, table and member facts with verbatim member
|
||||
bytes. `V1MMWX` packages are detected as encrypted coverage and refused on
|
||||
extraction. See [wxapkg](../wxapkg.md) for layout and coverage boundaries.
|
||||
|
||||
Inspect an ESP32-family app or bootloader image without esptool:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact firmware.bin --json
|
||||
rea extract-artifact firmware.bin --json
|
||||
```
|
||||
|
||||
The inventory lists each segment with its ESP-IDF load classification, the
|
||||
checksum and SHA-256 results, the descriptor and any Secure Boot v2 signature
|
||||
blocks. See [ESP images](../esp-image.md) for recognition, coverage and refusals.
|
||||
|
||||
Inspect an ESP flash dump, or one read from a flash address other than 0:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact flash.bin --json
|
||||
rea inspect-artifact dump.bin --target-format esp-flash-image --flash-offset 0x1000 --json
|
||||
rea extract-artifact flash.bin --json
|
||||
```
|
||||
|
||||
The inventory lists the bootloader, the partition table and each partition,
|
||||
with ESP image facts for app partitions and the OTA records. See
|
||||
[ESP flash images](../esp-flash-image.md) for table location, inputs and limits.
|
||||
|
||||
Inspect a Node.js SEA carrier without running it:
|
||||
|
||||
```sh
|
||||
@@ -322,7 +268,7 @@ rea inspect-artifact ./application --json
|
||||
rea extract-artifact ./application --json
|
||||
```
|
||||
|
||||
See [Node.js SEA](../node-sea.md) for supported layouts, script and asset paths,
|
||||
See [Node.js SEA](node-sea.md) for supported layouts, script and asset paths,
|
||||
fuse observations, and the real-fixture verification lane.
|
||||
|
||||
Inventory and extract Intel HEX or S-record regions:
|
||||
@@ -332,7 +278,7 @@ rea inspect-artifact ./firmware.hex --json
|
||||
rea extract-artifact ./firmware.hex --json
|
||||
```
|
||||
|
||||
See [Intel HEX and S-record images](../hex-records.md) for recognition, region
|
||||
See [Intel HEX and S-record images](hex-records.md) for recognition, region
|
||||
names, metadata and limits.
|
||||
|
||||
Inspect and extract stored U-Boot legacy uImage payloads and components:
|
||||
@@ -342,38 +288,9 @@ rea inspect-artifact ./firmware.bin --detail summary --evidence_output firmware.
|
||||
rea extract-artifact ./firmware.bin --json
|
||||
```
|
||||
|
||||
See [U-Boot uImage](../uimage.md) for verified CRCs, physical component ranges,
|
||||
See [U-Boot uImage](uimage.md) for verified CRCs, physical component ranges,
|
||||
unknown names, resource limits and retained views. Compressed bytes stay stored.
|
||||
|
||||
`inspect-artifact` and `extract-artifact` also read ZIP members compressed with
|
||||
Zstandard (standard method 93). They retain the shared ZIP CRC, declared-size,
|
||||
overlap and archive expansion checks. The native streaming decoder keeps the
|
||||
existing 128 MiB Zstandard window cap; frames requiring a larger window are
|
||||
reported as a resource limit (`reason: limit`, `boundary: memory`), rather than
|
||||
as archive corruption. The cap cannot be raised at runtime: recompress with a
|
||||
window of at most 128 MiB, or decompress externally before retrying.
|
||||
Deprecated method 20 and other unregistered ZIP codecs remain unsupported.
|
||||
|
||||
Inventory and extract UF2 or DfuSe regions:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./firmware.uf2 --json
|
||||
rea extract-artifact ./firmware.dfu --json
|
||||
```
|
||||
|
||||
See [UF2 and DfuSe images](../uf2-dfuse.md) for recognition, families, numbering,
|
||||
the DfuSe suffix CRC and limits.
|
||||
|
||||
Inventory and extract U-Boot FIT images:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./boot.itb --json
|
||||
rea extract-artifact ./boot.itb --json
|
||||
```
|
||||
|
||||
See [U-Boot FIT images](../fit-images.md) for recognition, data sources, hashes
|
||||
and limits.
|
||||
|
||||
Inspect local model declarations without loading weights:
|
||||
|
||||
```bash
|
||||
@@ -385,7 +302,7 @@ Default hashing reads only headers and leaves full-file identity unknown.
|
||||
`--hash full` streams the complete file for SHA-256. Both return typed metadata,
|
||||
original tensor declarations and derived totals. MCP summaries retain complete
|
||||
Evidence for filtered metadata, tensor and type-total pages; CLI compact delivery
|
||||
uses `--detail summary --evidence_output FILE`. See [model files](../model-artifacts.md).
|
||||
uses `--detail summary --evidence_output FILE`. See [model files](model-artifacts.md).
|
||||
|
||||
Inspect Unity IL2CPP metadata names without Unity or a disassembler:
|
||||
|
||||
@@ -402,7 +319,7 @@ method instances and `--ghidra-seeds PATH` then writes Ghidra function seeds. It
|
||||
with retained Evidence; query indexed rows with filters, batch indices and
|
||||
selected fields. The CLI defaults to complete Evidence;
|
||||
`--detail summary --evidence_output FILE` exports the full result to a new file
|
||||
and returns a compact overview. See [IL2CPP metadata](../il2cpp-metadata.md).
|
||||
and returns a compact overview. See [IL2CPP metadata](il2cpp-metadata.md).
|
||||
|
||||
Inspect an embedded cargo-auditable Rust dependency list without installing
|
||||
Rust or configuring a disassembler:
|
||||
@@ -414,7 +331,7 @@ rea inspect-rust-binary /absolute/path/to/program --json
|
||||
The same workflow reports package names, versions, sources, the root package,
|
||||
dependency edges, and the section's file range. A missing `.dep-v0` section is
|
||||
null metadata, not proof that the binary has no dependencies. See
|
||||
[Rust binaries](../rust-binaries.md).
|
||||
[Rust binaries](rust-binaries.md).
|
||||
|
||||
Inspect a PE Authenticode certificate table without executing the image or
|
||||
verifying signatures:
|
||||
@@ -426,7 +343,7 @@ rea inspect-pe-certificates /absolute/path/to/application.exe --json
|
||||
The workflow reports `WIN_CERTIFICATE` entries and, for PKCS#7 signedData,
|
||||
embedded X.509 serial, issuer, subject, and validity fields. A missing security
|
||||
directory is observed absence of that table. See
|
||||
[PE Authenticode certificate tables](../pe-certificates.md).
|
||||
[PE Authenticode certificate tables](pe-certificates.md).
|
||||
|
||||
Inspect PE import and delay-load tables without executing the image:
|
||||
|
||||
@@ -437,7 +354,7 @@ rea inspect-pe-imports /absolute/path/to/application.exe --json
|
||||
The workflow reports DLL names, named and ordinal functions, delay-load
|
||||
descriptors, and a pefile-compatible import hash of the standard import table.
|
||||
A missing import directory is observed absence of that table. See
|
||||
[PE import and delay-load tables](../pe-imports.md).
|
||||
[PE import and delay-load tables](pe-imports.md).
|
||||
|
||||
Inspect ELF PT_NOTE and SHT_NOTE tables without executing the image:
|
||||
|
||||
@@ -447,7 +364,7 @@ rea inspect-elf-notes /absolute/path/to/program --json
|
||||
|
||||
The workflow reports note owners, types, descriptor hashes, and GNU build-id
|
||||
or ABI-tag fields when those descriptors match. A missing note table is
|
||||
observed absence of those declarations. See [ELF note tables](../elf-notes.md).
|
||||
observed absence of those declarations. See [ELF note tables](elf-notes.md).
|
||||
|
||||
Inspect an Apple `.ips` crash report without executing a target:
|
||||
|
||||
@@ -458,7 +375,7 @@ rea inspect-apple-ips /absolute/path/to/report.ips --json
|
||||
The workflow reports first-line metadata, process/exception/termination facts,
|
||||
threads, usedImages, lossless integer registers, and documented crash classes
|
||||
when `bug_type` is `"309"`. Other DiagnosticReports types return no crash
|
||||
result. See [Apple IPS crash reports](../apple-ips.md).
|
||||
result. See [Apple IPS crash reports](apple-ips.md).
|
||||
Inspect Linux kernel module metadata without executing the module or requiring
|
||||
`modinfo`:
|
||||
|
||||
@@ -474,10 +391,9 @@ not decoded and the signature is not validated. Other ELF types are not
|
||||
applicable. gzip, xz, and zstd containers are unsupported and name
|
||||
`external_decompression` as the next probe: decompress with an external
|
||||
utility, then inspect the decompressed file. See
|
||||
[Linux kernel modules](../kernel-modules.md).
|
||||
[Linux kernel modules](kernel-modules.md).
|
||||
|
||||
Inspect a CPython 3.7–3.13 `.pyc` header and nested marshal code objects (3.14 and
|
||||
3.15 headers are qualified but their marshal layout is not pinned)
|
||||
Inspect a CPython 3.7–3.13 `.pyc` header and nested marshal code objects
|
||||
without executing bytecode or installing Python:
|
||||
|
||||
```bash
|
||||
@@ -486,7 +402,7 @@ rea inspect-python-pyc /absolute/path/to/module.pyc --json
|
||||
|
||||
The workflow reports the pinned CPython version for the magic, PEP 552 hash or
|
||||
mtime/size fields, nested function names, and bytecode SHA-256. Bytecode is
|
||||
not decompiled. See [CPython pyc inspection](../python-pyc.md).
|
||||
not decompiled. See [CPython pyc inspection](python-pyc.md).
|
||||
|
||||
Inspect an Erlang BEAM file without executing bytecode:
|
||||
|
||||
@@ -496,19 +412,10 @@ rea inspect-erlang-beam /absolute/path/to/app.beam --json
|
||||
|
||||
The workflow reports IFF chunk names, atom tables, exports, imports, lambdas,
|
||||
and the Code chunk header. Missing optional tables stay null. See
|
||||
[Erlang BEAM files](../erlang-beam.md).
|
||||
|
||||
Complete `rea decompile --json` Evidence can be saved and inspected through
|
||||
`inspect-analysis-view` with an inline source and
|
||||
`view: {kind: "native", facet: "pseudocode", offset: 0, limit: 20000}`.
|
||||
The projection reads that saved observation without starting a provider;
|
||||
offsets/limits count UTF-16 code units. A summary view preserves the original
|
||||
selector and distinguishes null from empty text. The MCP tool additionally
|
||||
supports `detail: "summary"` on its first call to retain the complete source
|
||||
without returning the body inline; see [native investigation](../guides/native/native-investigation.md).
|
||||
[Erlang BEAM files](erlang-beam.md).
|
||||
|
||||
Configure [Hopper or Ghidra](installation.md#hopper), the
|
||||
[IDA adapter](../providers/ida-provider.md), or [Kuna](../kuna-provider.md) before analyzing a
|
||||
[IDA adapter](ida-provider.md), or [Kuna](kuna-provider.md) before analyzing a
|
||||
native target. Substitute
|
||||
your target, search text and function name or address in these examples:
|
||||
|
||||
@@ -536,11 +443,11 @@ MCP `analyze_function` defaults to the same summary and retains the complete
|
||||
dossier in the session for `inspect_analysis_view`.
|
||||
|
||||
Run `rea --help` or a command's `--help` for arguments and output options.
|
||||
The [generated catalog](../reference/mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
|
||||
The [generated catalog](mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
|
||||
tools. Provider-specific target support is described in the
|
||||
[native guide](../guides/native/native-investigation.md), [DOS guide](../providers/ghidra-dos.md),
|
||||
[raw binary guide](../providers/raw-binary.md), [GameCube DOL guide](../ghidra-dol.md), and
|
||||
[Windows Ghidra guide](../providers/windows-ghidra-p0.md).
|
||||
[native guide](native-investigation.md), [DOS guide](ghidra-dos.md),
|
||||
[raw binary guide](raw-binary.md), [GameCube DOL guide](ghidra-dol.md), and
|
||||
[Windows Ghidra guide](windows-ghidra-p0.md).
|
||||
|
||||
## Choose a provider
|
||||
|
||||
@@ -587,34 +494,15 @@ For Hopper, Ghidra, IDA, JADX, and Kuna, run
|
||||
such as `pwntools-elf`, report their configuration requirements directly and are
|
||||
not accepted by `doctor --provider`. See
|
||||
[task readiness](installation.md#check-readiness-for-your-task) and
|
||||
[provider selection](../reference/mcp-contracts.md#identity-and-discovery).
|
||||
[provider selection](mcp-contracts.md#identity-and-discovery).
|
||||
|
||||
The session reports work still in progress through `analysis_activity`.
|
||||
A client timeout can end its wait while the provider continues analyzing.
|
||||
`cleanup_incomplete` identifies resources whose shutdown or removal could not
|
||||
be verified. See [MCP contracts](../reference/mcp-contracts.md) for session lifecycle and
|
||||
[Ghidra first-query deadlines](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
|
||||
be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and
|
||||
[Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
|
||||
for import, client timeouts and recovery.
|
||||
|
||||
A one-shot command releases its provider session and REA runtime before it
|
||||
prints a failure, so it omits the same-session reuse and
|
||||
`update_analysis_settings` guidance an MCP connection receives. Its
|
||||
`details.limit` reports `scope: "rea-invocation"` and
|
||||
`applies_at: "next_invocation"`; set the named `environment` (or the per-call
|
||||
`parameter`) for a new invocation and rerun. For example, a Ghidra decompile
|
||||
deadline names `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS`.
|
||||
|
||||
`rea debug-with-gdb` and `rea debug-with-rizin` hold their debugger only for the
|
||||
command's own process. If cleanup cannot confirm the debugger stopped, they print
|
||||
`cleanup_incomplete` and exit nonzero without stopping it. `remediation.action`
|
||||
names the process, as does `retained_debugger_pids` in `details.diagnostics`, or
|
||||
in `details.diagnostics.cleanup_error.details.diagnostics` when the command also
|
||||
failed or was interrupted. Exiting closes the debugger's command and output pipes.
|
||||
REA does not establish what the debugger or its target does next, so check that
|
||||
process. An MCP server that shuts down likewise exits with status 1 instead of
|
||||
waiting on a debugger it could not stop. It names the process on stderr and as
|
||||
`retained_debugger_pids` in its error log.
|
||||
|
||||
If one-shot analysis fails and closing its session also fails, the original
|
||||
error details remain inline and `details.cleanup_error` describes the later
|
||||
close failure. Each diagnostic retains its own reported resources; the later
|
||||
@@ -647,9 +535,6 @@ Saved snapshots include only cache entries whose subject identifies the selected
|
||||
target. Unbound observations and observations of child artifacts remain available
|
||||
in the open session and its Evidence bundle; they are excluded from the target's
|
||||
saved replay entries without changing their subjects.
|
||||
The snapshot also retains available Evidence linked by selected records, including
|
||||
transitive sources with an unbound or different subject. Those sources are not
|
||||
replay entries for the selected target. Unrelated records remain excluded.
|
||||
|
||||
Snapshots retain eligible target-scoped question histories in full, including
|
||||
mutation Evidence recorded while another target was active. Observations and
|
||||
@@ -664,11 +549,6 @@ rea analyze /absolute/path/to/program --provider ghidra --snapshot /absolute/pat
|
||||
```
|
||||
|
||||
An exact CLI cache hit is normally read before starting a provider process.
|
||||
`decompile --provider ghidra --snapshot ...` also retains its single selected
|
||||
Program's pseudocode result, even without an explicit document selector. Old
|
||||
snapshots that contain only the Evidence need one new query to add that replay
|
||||
entry. This does not retain every body from `decompile-all` or make a persistent
|
||||
session's implicit document replayable.
|
||||
On Windows, Ghidra first admits the selected source through its native snapshot
|
||||
owner, so replay can start that provider.
|
||||
In MCP, `open_binary` accepts `snapshot_path` to import a snapshot atomically
|
||||
@@ -677,30 +557,6 @@ cached result. `close_binary` accepts `snapshot_path` and optional
|
||||
`overwrite: true` to save before releasing provider resources. A failed save
|
||||
leaves the session open so the caller can resolve the output failure.
|
||||
|
||||
When a validated snapshot does not match the selected binding, the error's
|
||||
`details.binding_mismatch` reports confirmed target identities, provider
|
||||
identities, or profile digests that differ. `snapshot` is the saved value and
|
||||
`selected` is the requested value. Multiple differences can be reported. A
|
||||
`profile_comparison: "not_compared"` means the profile was not available at
|
||||
that boundary; it does not establish a match. A profileless snapshot can report
|
||||
`workflow_scope: "incompatible"` even when its target and profile match.
|
||||
Choose a matching target, provider, and profile, or analyze without the snapshot
|
||||
and save a new one for the intended binding.
|
||||
|
||||
For a profile mismatch, the saved snapshot contains
|
||||
`binding.analysis_profile.parameters`. If `binary_session` reports an active
|
||||
profile whose digest matches `binding_mismatch.profile_digest.selected`, its
|
||||
`analysis_provider_binding.analysis_profile.parameters` can be compared with
|
||||
those saved parameters. A rejected switch leaves the previous session active,
|
||||
so its profile may be different from the attempted selection; check the digest
|
||||
before using it for this comparison. A configuration difference does not by
|
||||
itself establish why analysis results differ.
|
||||
|
||||
If no target is open and you intend to inspect the selected binding, open that
|
||||
target without the snapshot, then check its profile digest in `binary_session`
|
||||
before comparing parameters. This may start the selected provider; the saved
|
||||
snapshot remains a separate file.
|
||||
|
||||
If a selected snapshot's file descriptor cannot close, `open_binary` reports
|
||||
`cleanup_incomplete`. `binary_session` reports `cleanup_pending` even when no
|
||||
target opened. Retry `close_binary` to release the retained descriptor before
|
||||
@@ -721,10 +577,6 @@ results and Evidence. These are cached observations, not a saved provider
|
||||
database: only eligible exact queries can reuse a result, and new or live
|
||||
queries can still require provider startup. The CLI uses the same snapshot
|
||||
format and preserves these records when loading and updating it.
|
||||
When an MCP session replays a provider observation, its delivered Evidence identifies
|
||||
the original observation through `evidence_links` and reports that the
|
||||
provider was not rerun. The original and its available source records remain
|
||||
in the investigation bundle.
|
||||
|
||||
## Import, export and compare Evidence
|
||||
|
||||
@@ -742,23 +594,10 @@ rea compare left.json right.json --record_pairs '[{"left_evidence_id":"ev_<left-
|
||||
Migration: replace `rea compare-bundles` with `rea compare`; the former command
|
||||
has been removed. The MCP tool remains `compare_bundles`.
|
||||
|
||||
`rea evidence-import` performs process-local bundle authentication and
|
||||
validation, reporting record and Unknown-revision counts. `unknowns_added`
|
||||
counts revisions, not distinct Unknown IDs. It does not retain records in a
|
||||
separate or background MCP server process. To retain records across calls in an
|
||||
active MCP session, use the MCP tool `import_evidence_bundle` in that session.
|
||||
|
||||
`rea compare` compares records by Evidence identity and explicit pairs. It does
|
||||
not read artifact contents: the inventories of two builds are unresolved
|
||||
records, or one opaque change when you pair them. Use `compare-artifacts`,
|
||||
described at the end of this section, to compare their contents.
|
||||
|
||||
`--record_pairs` accepts an inline JSON array or the path to a local JSON
|
||||
file containing that array, matching MCP `compare_bundles.record_pairs`. For a
|
||||
file, use `rea compare left.json right.json --record_pairs pairs.json --json`.
|
||||
Pair IDs must exist on their named sides and be one-to-one. Without pairs,
|
||||
one-sided membership remains unknown; REA does not infer correspondence from
|
||||
position or display names.
|
||||
`--record_pairs` supplies an explicit JSON array of observation pairs, matching
|
||||
MCP `compare_bundles.record_pairs`. Pair IDs must exist on their named sides and
|
||||
be one-to-one. Without pairs, one-sided membership remains unknown; REA does
|
||||
not infer correspondence from position or display names.
|
||||
|
||||
Exports preserve an existing destination unless `--overwrite` is explicit.
|
||||
|
||||
@@ -769,66 +608,14 @@ protection still applies. A genuine pathname refusal uses `resource_constraint`
|
||||
with `details.reason: path-limit` and `boundary: filesystem-path`.
|
||||
|
||||
For application traces and version comparisons, see
|
||||
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md).
|
||||
MCP [retained Evidence references](../reference/mcp-contracts.md#retained-application-evidence-inputs)
|
||||
belong to one connection. Separate CLI calls can consume full saved records or
|
||||
resolve exact retained IDs from an explicitly selected local bundle:
|
||||
|
||||
```bash
|
||||
rea trace-application-feature request.json --evidence-bundle /absolute/path/evidence.json --json
|
||||
```
|
||||
|
||||
Here `request.json` uses the same logical Evidence input as MCP, for example
|
||||
`"application": {"kind":"retained-evidence","evidence_id":"ev_<64 lowercase hex characters>"}`.
|
||||
Inline records and references may be mixed in the request's declared source
|
||||
fields, including arrays and nested inputs. The bundle is read and authenticated
|
||||
once for that command; no target is reopened, no producer is rerun, and no
|
||||
record is selected by file order. Source identity, provenance, coverage and
|
||||
partial or unknown facts agree with the complete inline form. When starting
|
||||
from summary or inspection Evidence, select its
|
||||
`normalized_result.parent_evidence_id` for a workflow requiring complete source
|
||||
facts. Omit `--evidence-bundle` when all sources are inline.
|
||||
|
||||
The JSON workflow commands supporting this option are:
|
||||
|
||||
- `trace-application-feature`, `trace-javascript-semantics`,
|
||||
`compare-application-versions`, `compare-javascript-export-shapes`,
|
||||
`compare-source-to-bundle`, and `reconcile-javascript-runtime`.
|
||||
- `import-managed-reconstruction`, `verify-managed-native-boundaries`, and
|
||||
`project-managed-application-graph`.
|
||||
- `project-android-application-graph`, `project-apple-application-graph`, and
|
||||
`project-harmony-application-graph`.
|
||||
- `compare-wasm-interfaces` and `inspect-analysis-view`.
|
||||
|
||||
For `inspect-analysis-view`, the request's source reference selects a parent
|
||||
from the bundle for every supported view kind. Bundles exported from MCP work
|
||||
for these commands. Its inline source form cannot also select a bundle.
|
||||
A missing ID is refused with `details.reason: "missing"`;
|
||||
supplying a bundle does not make incompatible Evidence valid for a workflow.
|
||||
`compare-managed-members` keeps its two assembly paths and an optional
|
||||
repeatable `--type` selection applied to both sides, and
|
||||
`compare-process-captures` keeps its two capture Evidence JSON paths. The bundle
|
||||
option above selects an input; producer options that write a new bundle, such
|
||||
as `inspect-managed-members --evidence-bundle`, retain their existing meaning.
|
||||
File selectors remain available only on contracts that explicitly support them.
|
||||
|
||||
Compare two artifact inventories with `compare-artifacts`. It takes the same
|
||||
`left` and `right` input as MCP `compare_artifacts`: complete Evidence, or an
|
||||
`evidence-file` source with an optional `evidence_id` that selects one record
|
||||
from a bundle. Relative paths resolve from the working directory:
|
||||
|
||||
```bash
|
||||
rea inspect-artifact old.zip --json > old.json
|
||||
rea inspect-artifact new.zip --json > new.json
|
||||
rea compare-artifacts '{"left":{"kind":"evidence-file","path":"old.json"},"right":{"kind":"evidence-file","path":"new.json"}}' --json
|
||||
```
|
||||
|
||||
A one-shot `compare-artifacts` command keeps no session Evidence, so it refuses a
|
||||
`retained-evidence` source with `details.reason: "missing"`. Select the record
|
||||
from a saved bundle instead.
|
||||
For an artifact summary selected from a bundle, include its complete parent
|
||||
inventory in the same bundle. The command resolves that parent from the
|
||||
authenticated file records; a summary-only file cannot supply missing facts.
|
||||
[JavaScript application workflows](javascript-application-workflows.md).
|
||||
MCP [retained Evidence references](mcp-contracts.md#retained-application-evidence-inputs)
|
||||
belong to one connection. Separate CLI calls can consume full saved records.
|
||||
`inspect-analysis-view REQUEST.json --evidence-bundle BUNDLE.json` resolves
|
||||
the request's `retained-evidence` source ID from that authenticated bundle,
|
||||
including bundles exported from an MCP session. The same view JSON works for
|
||||
every supported parent kind; no target is reopened and no record is selected
|
||||
by file order. Omit `--evidence-bundle` when the request supplies inline Evidence.
|
||||
|
||||
## Import historical source
|
||||
|
||||
@@ -844,7 +631,6 @@ Import an older source tree to compare with the current artifact:
|
||||
```bash
|
||||
rea import-reference-source /absolute/path/to/source
|
||||
rea import-reference-source /absolute/path/to/source --detail summary --outline-depth 2 --json
|
||||
rea import-reference-source /absolute/path/to/source --exclude-paths '["vendor/","*.generated","!keep.generated"]'
|
||||
```
|
||||
|
||||
The default (`--detail full`) returns the complete `HistoricalSourceGraph`.
|
||||
@@ -978,23 +764,19 @@ for percent signs, query marks and fragments.
|
||||
Historical-source import requires safe no-follow file opens on Linux or
|
||||
macOS. Native Windows returns `unsupported_host`; use Linux REA inside WSL
|
||||
or another supported host. See
|
||||
[source-to-bundle comparison](../guides/javascript/javascript-application-workflows.md#historical-source-to-bundle-comparison)
|
||||
[source-to-bundle comparison](javascript-application-workflows.md#historical-source-to-bundle-comparison)
|
||||
for mapping identities, inferences and unknowns.
|
||||
|
||||
## Capture runtime behavior
|
||||
|
||||
Choose the guide for the target and intended interaction:
|
||||
|
||||
- [Browser observation](../guides/web/browser-observation.md): inspect a selected page through CDP.
|
||||
- [Browser scenarios](../guides/web/browser-scenario-contract.md): run declared interactions and capture their results.
|
||||
- [Electron observation](../guides/javascript/electron-observation.md): inspect an Electron renderer or capture an application scenario.
|
||||
- [Electron fuses](../electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
|
||||
- [Node/Electron Inspector](../guides/javascript/javascript-runtime-observation.md): record script locations and execution contexts.
|
||||
- [Process capture](../guides/runtime/process-capture.md): run an executable and compare terminal, exit and filesystem observations.
|
||||
- [Desktop application capture](../desktop-application-capture.md): launch a Linux desktop application and record its windows and home-directory state at checkpoints while it is used.
|
||||
- [Desktop application state](../desktop-application-state.md): read a capture folder into the application's state files, typed views of each version and kinds of change.
|
||||
- [Desktop application export](../desktop-application-export.md): write a capture folder as an agentenv computer app bundle with its class, skill and seed.
|
||||
- [Desktop application tasks](../desktop-application-tasks.md): describe how a capture's files changed, then validate an agent-written task against the capture and export it with its verifier.
|
||||
- [Browser observation](browser-observation.md): inspect a selected page through CDP.
|
||||
- [Browser scenarios](browser-scenario-contract.md): run declared interactions and capture their results.
|
||||
- [Electron observation](electron-observation.md): inspect an Electron renderer or capture an application scenario.
|
||||
- [Electron fuses](electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
|
||||
- [Node/Electron Inspector](javascript-runtime-observation.md): record script locations and execution contexts.
|
||||
- [Process capture](process-capture.md): run an executable and compare terminal, exit and filesystem observations.
|
||||
|
||||
Runtime requests name the target and actions. Launched targets run with your
|
||||
user permissions; consult the chosen guide for host requirements and effects.
|
||||
@@ -1012,13 +794,7 @@ rea mcp --http --port 8765
|
||||
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the environment,
|
||||
then to `127.0.0.1` and an ephemeral port. A bind outside loopback requires
|
||||
`MCP_ALLOWED_HOSTS` or `--allowed-hosts`, a comma-separated list of client host
|
||||
names or IP addresses without URLs or ports.
|
||||
|
||||
Repeat `--allowed-hosts` to append names, or put multiple names in one
|
||||
comma-separated value. All occurrences are validated together and deduplicated
|
||||
after hostname normalization; the explicit list replaces `MCP_ALLOWED_HOSTS`.
|
||||
|
||||
For example:
|
||||
names or IP addresses without URLs or ports. For example:
|
||||
|
||||
```sh
|
||||
rea mcp --http --host 0.0.0.0 --allowed-hosts 127.0.0.1,rea.lan
|
||||
@@ -1033,7 +809,7 @@ one client at a time: a session ends with the client's `DELETE`, and a client
|
||||
that connects while another session is open takes it over, so a client that
|
||||
exited without ending its session needs no restart. The listener runs until
|
||||
SIGINT or SIGTERM and does not stop when stdin closes. See
|
||||
[transports](../reference/mcp-contracts.md#transports) for the session, `Host` header and
|
||||
[transports](mcp-contracts.md#transports) for the session, `Host` header and
|
||||
shutdown contract. Client registrations written by setup and the MCP registry
|
||||
entry keep stdio.
|
||||
|
||||
@@ -1058,7 +834,7 @@ answers every method with JSON-RPC `-32601`. The stopped document's `state`
|
||||
member is `{"served": true, "evidence_id": …}`,
|
||||
`{"served": false, "reason": "replay-only"}` or
|
||||
`{"served": false, "reason": "unavailable", "code": …, "message": …}`; see
|
||||
[serve from the state](../agent-environments.md#serve-from-the-state).
|
||||
[serve from the state](agent-environments.md#serve-from-the-state).
|
||||
|
||||
```sh
|
||||
rea environment-serve manifest.json --evidence capture.json --port 8765
|
||||
@@ -1078,7 +854,7 @@ MCP_ALLOWED_HOSTS=127.0.0.1,rea.lan rea environment-serve manifest.json --eviden
|
||||
the served environment; set but blank, it is refused as `configuration_invalid`
|
||||
like a blank `MCP_HOST`. The admitted client URL is written to stderr as one JSON log line
|
||||
(`"msg":"MCP transport listening"`). See
|
||||
[agent environments](../agent-environments.md) for what the state and replay
|
||||
[agent environments](agent-environments.md) for what the state and replay
|
||||
return and do not establish.
|
||||
|
||||
## Build an environment bundle
|
||||
@@ -1103,7 +879,7 @@ command. `--dry-run` writes nothing and may omit the directory.
|
||||
rea environment-build grafana.har ./grafana --json
|
||||
```
|
||||
|
||||
See [agent environments](../agent-environments.md#build-a-bundle-in-one-step)
|
||||
See [agent environments](agent-environments.md#build-a-bundle-in-one-step)
|
||||
for the report, the hints and what the bundle ships.
|
||||
|
||||
## Export an environment bundle
|
||||
@@ -1136,7 +912,7 @@ name Windows reserves (`con`, `nul`, `com1` and the like) is refused. The
|
||||
command prints one JSON document listing the written files with their sizes,
|
||||
the pinned package source, the served `tool_count`, and `state` in the shape
|
||||
of the serve command's stopped document. See
|
||||
[agent environments](../agent-environments.md#export-a-bundle) for the folder
|
||||
[agent environments](agent-environments.md#export-a-bundle) for the folder
|
||||
layout and what the smoke task checks.
|
||||
|
||||
## Compare an environment with its application
|
||||
@@ -1149,7 +925,7 @@ answer is compared with the application's after ids are mapped and clocks and ge
|
||||
makes no network call. It prints the report and exits `0`, or prints a structured `invalid_request` error and exits `1`
|
||||
for the same pair refusals as `environment-serve`, a probe that is not capture Evidence, or a browser probe captured
|
||||
without response bodies, or without the request bodies its writes need. See
|
||||
[agent environments](../agent-environments.md#measure-fidelity-against-the-application) for the report.
|
||||
[agent environments](agent-environments.md#measure-fidelity-against-the-application) for the report.
|
||||
|
||||
```sh
|
||||
rea environment-compare manifest.json --evidence capture.json --probe probe.json --json
|
||||
@@ -1206,10 +982,6 @@ such as `UNSUPPORTED_OUTPUT_COMBINATION` for `--json` with `--token-limit`. Use
|
||||
Commands reject surplus positional arguments before execution, including commands
|
||||
that accept no positionals. Use `rea <command> --help` for the declared argument
|
||||
shape; a final variadic argument continues to accept multiple values.
|
||||
`rea mcp add` accepts options only and rejects unknown options before reading or
|
||||
backing up client settings or starting registration. Its `--agent` and
|
||||
`--command`/`-c` options accept spaced and equals forms. Values must be nonempty; a following flag does not satisfy a spaced option's
|
||||
missing value.
|
||||
|
||||
`rea mcp doctor --json` checks production MCP startup and reports child logs in
|
||||
`diagnostics`. It drains stderr while retaining at most its first 64 KiB of raw
|
||||
@@ -1227,8 +999,8 @@ Malformed format values and token windows fail with `INVALID_OUTPUT_VALUE` in th
|
||||
selected structured format. Token limits and offsets must be nonnegative safe
|
||||
integers; zero selects an empty window. These checks run before command execution. `--json` is indented on a terminal and compact when piped or
|
||||
redirected, which keeps large results smaller for agents and scripts; pipe
|
||||
through `jq .` for indented files. A formatting refusal exits nonzero even when
|
||||
the underlying operation succeeded.
|
||||
through `jq .` for indented files. Output selection and formatting do not change
|
||||
operation status.
|
||||
|
||||
`--format md` escapes backslashes, pipes, and inline Markdown syntax in table
|
||||
headers and cells so source text is displayed literally, and renders cell line
|
||||
@@ -1240,22 +1012,13 @@ values instead of replacing them with an object label. Table widths are measured
|
||||
iteratively, so an admitted row page is not limited by the JavaScript function
|
||||
argument count.
|
||||
|
||||
Markdown refuses selected NUL characters and unpaired UTF-16 surrogates because
|
||||
renderers or UTF-8 output would substitute them. TOON refuses selected unpaired
|
||||
surrogates for the same UTF-8 reason. Use JSON or YAML to retain those values.
|
||||
An authored replacement character and valid supplementary characters are not
|
||||
treated as malformed.
|
||||
|
||||
`--format csv` and `--format tsv` export one selected row collection. Select an
|
||||
exact array path with `--filter-output` and provide comma-separated direct
|
||||
scalar columns with `--columns`; column order is the output column order:
|
||||
|
||||
For an artifact inventory request that selects the `occurrences` collection
|
||||
and includes `logical_path` and `artifact_id` in its view fields:
|
||||
|
||||
```bash
|
||||
rea inspect-analysis-view request.json --format csv \
|
||||
--filter-output normalized_result.items --columns logical_path,artifact_id
|
||||
--filter-output normalized_result.rows --columns name,address
|
||||
```
|
||||
|
||||
The first line is a REA metadata record,
|
||||
@@ -1268,8 +1031,7 @@ delimiter, quotes, or line breaks are quoted and embedded quotes are doubled.
|
||||
Values are emitted as UTF-8 without Unicode normalization. Missing and `null`
|
||||
cells are empty; booleans and finite numbers use their ordinary text forms.
|
||||
String cells and headers that begin with spreadsheet formula syntax are refused
|
||||
with a structured error; unpaired surrogates in selected cells or headers are
|
||||
also refused. JSON retains those values unchanged.
|
||||
with a structured error; JSON retains those values unchanged.
|
||||
|
||||
Delimited output does not flatten nested Evidence or nested field values.
|
||||
It returns a structured error and a nonzero status if the filter does not
|
||||
@@ -1350,9 +1112,7 @@ the command's signal handling. Commands routed through the shared
|
||||
`withCommandCancellation` handler in the CLI set status `130` for SIGINT or
|
||||
`143` for SIGTERM after their operation settles. Current source registrations
|
||||
use that handler for process capture and comparison (`capture-process`,
|
||||
`compare-process-captures`), desktop application capture and state
|
||||
(`capture-desktop-application`, `inspect-desktop-application-state`,
|
||||
`export-desktop-application`, `describe-desktop-application-task`), JavaScript application analysis (`analyze` when it
|
||||
`compare-process-captures`), JavaScript application analysis (`analyze` when it
|
||||
selects the JavaScript lane, and `analyze-javascript-application`), selected
|
||||
browser, web and Electron commands (including `analyze-web-bundle`,
|
||||
`observe-web-execution`, `capture-browser-scenario` and
|
||||
@@ -1375,18 +1135,6 @@ new file. A publication that has committed can win a later cancellation; the
|
||||
after commit, the `cleanup_incomplete` error reports the same file as
|
||||
`details.published_path` and `details.bytes`.
|
||||
|
||||
When a failure retains a completed observation and no bundle committed,
|
||||
`details.source_closure` states what that observation cites. `publication` is
|
||||
`not_published`. `records` holds authentic source Evidence this invocation
|
||||
captured, within 64 KiB. `omitted` lists up to 64 other identities and
|
||||
`omitted_count` the total: `not_captured` means the invocation holds no such
|
||||
record, so nothing is shown for it or its ancestors; `not_inlined` means it was
|
||||
captured but is not in this response. `complete` is true only when every cited
|
||||
record is in `records` or embedded in the observation. If examining the closure
|
||||
itself fails, `collection_failure` names the error type and the closure state is
|
||||
unknown; the original failure is still returned. Nothing is retained after exit,
|
||||
and running the command again is a new observation.
|
||||
|
||||
When the reader of piped output closes early, as `rea ... | head` does, the
|
||||
CLI stops writing, prints no error and exits `141` (`128 + 13`, the status of a
|
||||
SIGPIPE-terminated writer). The operation itself may have completed; only its
|
||||
@@ -1398,11 +1146,7 @@ status with the analysis failure status, so it exits `1`. Setup has its own cont
|
||||
structured result and process status as separate signals when scripting
|
||||
cancellation.
|
||||
|
||||
`setup --dry-run` returns `planned` and exits `0`. For `setup` and
|
||||
`environment-build`, explicit `--dry-run=true` and `--dry-run=false` are
|
||||
accepted; empty or other values fail with `PARSE_ERROR` before any changes.
|
||||
Every supplied value is checked even when a later occurrence overrides it.
|
||||
A cancelled setup also exits
|
||||
`setup --dry-run` returns `planned` and exits `0`; a cancelled setup also exits
|
||||
`0`. Setup returns `1` for `needs_confirmation` or `needs_human`.
|
||||
`doctor` returns `1` when required checks in its selected readiness scope fail;
|
||||
unavailable optional providers remain informational for unrelated tasks.
|
||||
@@ -1429,7 +1173,7 @@ the total token count and have no next offset.
|
||||
|
||||
Inspect an explicit local FDT v16/v17 blob without execution or an external tool.
|
||||
Returns header, reservations, node/property tree, derived standard values and
|
||||
selected phandle references. See [devicetree inspection](../devicetree.md) for
|
||||
selected phandle references. See [devicetree inspection](devicetree.md) for
|
||||
limits and retained-view pagination.
|
||||
|
||||
```bash
|
||||
@@ -1445,19 +1189,4 @@ rea inspect-pe-load-config /absolute/path/to/application.exe --json
|
||||
|
||||
Reports TLS callbacks and size-covered load-config fields without execution.
|
||||
Use `--max-callbacks` and `--max-file-bytes` to set read budgets. See
|
||||
[PE TLS and load configuration](../pe-load-config.md).
|
||||
|
||||
### Binary hardening
|
||||
|
||||
```sh
|
||||
rea inspect-binary-hardening /absolute/path/to/program --json
|
||||
```
|
||||
|
||||
Reports ELF, PE, and Mach-O hardening rows without execution. A missing import
|
||||
is `unknown`, not `disabled`. Use `--max-file-bytes` to set the read budget. See
|
||||
[binary hardening](../binary-hardening.md).
|
||||
|
||||
Bun standalone executables use the existing `inspect-artifact` and
|
||||
`extract-artifact` commands to inventory and materialize embedded source,
|
||||
sourcemaps and bytecode without running the executable. See
|
||||
[Bun standalone executables](../bun-standalone.md) for layout coverage and limits.
|
||||
[PE TLS and load configuration](pe-load-config.md).
|
||||
@@ -24,8 +24,8 @@ the registration uses them. Clearing either makes startup fail without resolving
|
||||
another package from the client's project. Restore the same installation before
|
||||
repairing it; an unidentified registration remains untouched. See
|
||||
[installation and recovery](installation.md#package-runners-and-global-installation).
|
||||
Setup forwards supported, explicitly selected REA settings into client
|
||||
configuration, but does not copy unrelated ambient environment values. To refresh an older registration, preview
|
||||
Setup does not copy your ambient environment into client
|
||||
settings. To refresh an older registration, preview
|
||||
`rea setup --client <client-id> --skill=false --dry-run`, then run the same command
|
||||
without `--dry-run` and approve the displayed changes. Setup preserves unrelated
|
||||
settings, backs up changed configuration and leaves an aligned registration
|
||||
@@ -43,36 +43,7 @@ Rerun setup to review the registration and restart the client.
|
||||
executable and environment of the terminal running doctor. It does not launch
|
||||
the client's configured command or reproduce the client's environment. If doctor
|
||||
passes but the client disconnects, inspect the client's startup error and registered
|
||||
launcher; see [verification scope](../development/client-verification.md#launch-environment-verification).
|
||||
|
||||
`rea doctor` has the same limit for provider checks, and reports a provider setting
|
||||
the registration's file does not carry as a
|
||||
[non-blocking unknown](installation.md#check-readiness-for-your-task).
|
||||
|
||||
## Setting launch variables in PowerShell
|
||||
|
||||
Shell examples that prefix a command with `REA_MCP_INPUT_SCHEMA_PROFILE=compact`
|
||||
use POSIX shell syntax. In PowerShell, set the process environment before
|
||||
launching REA:
|
||||
|
||||
```powershell
|
||||
$env:REA_MCP_INPUT_SCHEMA_PROFILE = "compact"
|
||||
rea mcp doctor --json
|
||||
```
|
||||
|
||||
The assignment applies to the current PowerShell process and commands it launches.
|
||||
Restore the previous value when finished, or use
|
||||
`Remove-Item Env:REA_MCP_INPUT_SCHEMA_PROFILE` if it was previously unset. It does
|
||||
not update an already running MCP server or desktop client.
|
||||
|
||||
To select this profile for an agent, set `REA_MCP_INPUT_SCHEMA_PROFILE` in that
|
||||
client's existing REA registration `env` object (`environment` for OpenCode) and
|
||||
restart the client, preserving its other settings. A successful terminal doctor run does not verify the client's
|
||||
launch environment. See [launch-environment verification](../development/client-verification.md#launch-environment-verification)
|
||||
and the client-specific instructions below. REA setup forwards supported selections,
|
||||
including `REA_MCP_INPUT_SCHEMA_PROFILE`, from its environment into the registration;
|
||||
it does not copy unrelated environment values. Restoring the shell's previous
|
||||
value does not undo a selection already saved by setup.
|
||||
launcher; see [verification scope](client-verification.md#launch-environment-verification).
|
||||
|
||||
## Runtime settings and client deadlines
|
||||
|
||||
@@ -83,7 +54,7 @@ raising a REA startup or operation deadline, separately ensure the client allows
|
||||
enough time for the full request. Use `inspect_analysis_settings` for the running
|
||||
server's defaults; `rea doctor` checks launch configuration and cannot observe
|
||||
another process's runtime overrides. See the
|
||||
[settings design](../design/limits-and-recovery.md) for ownership and lifetime.
|
||||
[settings design](design/limits-and-recovery.md) for ownership and lifetime.
|
||||
|
||||
## Client tool names and discovery
|
||||
|
||||
@@ -102,86 +73,11 @@ does not repair a client's naming behavior or infer permission drift from an
|
||||
unfamiliar prefix. The unavailable fork reported in
|
||||
[#5512](https://github.com/morluto/rea/issues/5512) has not been reverified.
|
||||
|
||||
<a id="hotcold-catalogs"></a>
|
||||
|
||||
## Hot/cold catalogs
|
||||
|
||||
For a client that supports MCP catalog refresh, set
|
||||
`REA_MCP_TOOL_CATALOG_PROFILE=hot-cold` in REA's server environment and restart
|
||||
the connection. REA then advertises a small initial catalog and loads other
|
||||
named tools through `discover_tools`. Setup preserves this explicit setting.
|
||||
See the [discovery contract](../reference/mcp-contracts.md#hotcold-catalog-discovery) for
|
||||
pagination, identity and family-scope semantics.
|
||||
|
||||
Analysis tools, including `inspect_analysis_view`, load on demand. For example,
|
||||
load `inspect_api_contract` and supply its `view` to answer an initial OpenAPI
|
||||
or GraphQL question directly. When later questions need retained facts, load
|
||||
`inspect_analysis_view` and use the complete `parent_evidence_id`; source files
|
||||
need not be reopened. If both tools are needed from the start, request both
|
||||
names in one `discover_tools` batch to avoid another catalog refresh. The same
|
||||
pattern applies to selected Playwright trace and endpoint-comparison results.
|
||||
This defers unused schema delivery and client validator preparation; it does
|
||||
not reduce parsing work, alter Evidence or shorten requested facts. Clients
|
||||
without catalog refresh should keep the default `full` profile.
|
||||
|
||||
Codex CLI 0.165.0 requires its experimental `mcp_tool_list_changed` feature for
|
||||
this workflow. Merge these entries into the existing `~/.codex/config.toml`
|
||||
without replacing other feature or REA settings:
|
||||
|
||||
```toml
|
||||
[features]
|
||||
mcp_tool_list_changed = true
|
||||
|
||||
[mcp_servers.rea.env]
|
||||
REA_MCP_TOOL_CATALOG_PROFILE = "hot-cold"
|
||||
```
|
||||
|
||||
The server setting alone does not enable client refresh. If a successful
|
||||
discovery call is followed by a missing tool, check the client's refresh feature
|
||||
and tool allow/deny lists. Clients without refresh should use the default
|
||||
`REA_MCP_TOOL_CATALOG_PROFILE=full`; reconnect after changing it. The
|
||||
[FastMCP HTTP trial](#fastmcp-and-unsloth-sse-catalog-limits) below covers
|
||||
explicit refreshes; other clients' automatic refresh remains unverified.
|
||||
|
||||
## FastMCP and Unsloth SSE catalog limits
|
||||
|
||||
If an HTTP client reports `Server-sent event exceeded the 1048576 byte limit`
|
||||
during `tools/list`, merge these settings into the **REA server process's**
|
||||
environment, then reconnect:
|
||||
|
||||
```text
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE=compact
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE=compact
|
||||
REA_MCP_TOOL_CATALOG_PROFILE=full
|
||||
```
|
||||
|
||||
This advertises all tools without requiring catalog refresh. Compact output
|
||||
schemas widen client-side validation; REA still validates complete results
|
||||
against the canonical contracts. See [output schemas](../reference/mcp-contracts.md#output-schemas).
|
||||
|
||||
On main `3e0599a94`, a real FastMCP 4.1.6 / Python MCP 2.3.0 HTTP client with
|
||||
the default 1 MiB SSE event limit rejected the 4,108,975-byte full catalog.
|
||||
The settings above delivered all 240 tools in 611,305 bytes and a
|
||||
`binary_session` call succeeded. These are measured SSE response sizes for
|
||||
that catalog, not a size guarantee for future catalogs or other profiles.
|
||||
|
||||
For a client that refreshes tools, combine `hot-cold` with the compact schema
|
||||
settings to reduce initial context further: the same trial delivered six
|
||||
initial tools in 33,903 bytes, then explicitly refreshed through 34 loaded
|
||||
tools. Hot/cold alone later exceeded the SSE limit during refresh. Follow
|
||||
the [refresh requirements](#hotcold-catalogs).
|
||||
|
||||
The Unsloth report [#9093](https://github.com/morluto/rea/issues/9093) fails at
|
||||
catalog delivery; the Unsloth application itself was not exercised in this
|
||||
trial. Schema compaction does not bound analysis results. Continue to use
|
||||
summary/detail selection and tool-specific paging for larger results; client
|
||||
receive limits and full Evidence access still apply.
|
||||
|
||||
## Fiddler Everywhere alongside REA
|
||||
|
||||
Connect Fiddler Everywhere's MCP server separately for HTTP traffic debugging,
|
||||
preserving your REA client entry. See the
|
||||
[setup instructions and public skills](../guides/web/web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
|
||||
[setup instructions and public skills](web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
|
||||
|
||||
## VS Code
|
||||
|
||||
@@ -219,7 +115,7 @@ tool inventory on Linux using a loopback model fixture. After setup, add
|
||||
settings, then restart the client. Compact advertisements can reduce nested
|
||||
schema constraints; REA still validates complete canonical inputs. Full-profile
|
||||
`gpt-5.4` metadata is also verified locally. Live model API acceptance and
|
||||
Windows remain unverified; see [native client verification](../development/testing.md).
|
||||
Windows remain unverified; see [native client verification](testing.md).
|
||||
|
||||
## Claude Desktop
|
||||
|
||||
@@ -230,7 +126,7 @@ in the file the installed app reads:
|
||||
|
||||
- Without Claude Desktop's MSIX package storage, setup uses
|
||||
`%APPDATA%\Claude\claude_desktop_config.json`.
|
||||
- The legacy MSIX storage layout (package family `Claude_pzs8sxrjxfjjc`, installed under
|
||||
- The MSIX build (package family `Claude_pzs8sxrjxfjjc`, installed under
|
||||
`C:\Program Files\WindowsApps\Claude_<version>_x64__pzs8sxrjxfjjc`)
|
||||
virtualizes AppData writes. Once it has saved its configuration, it reads
|
||||
the package-private copy
|
||||
@@ -239,11 +135,6 @@ in the file the installed app reads:
|
||||
private copy when it exists. Before the package has saved one, they use the
|
||||
`%APPDATA%` file it still reads; with neither file, setup creates the
|
||||
private copy.
|
||||
- If `%LOCALAPPDATA%\Claude-Data\claude_pzs8sxrjxfjjc` exists, setup, doctor
|
||||
and uninstall report an unresolved location. Migrated MSIX installations
|
||||
have been observed reading `claude_desktop_config.json` in this directory.
|
||||
Quit Claude Desktop and verify its active configuration before registering
|
||||
manually. See [#8968](https://github.com/morluto/rea/issues/8968).
|
||||
- If `%LOCALAPPDATA%\Packages` holds a `Claude_<publisher ID>` package from
|
||||
another publisher, an `AnthropicPBC.Claude_<publisher ID>` package, or cannot
|
||||
be read, REA cannot establish which file Claude
|
||||
@@ -331,7 +222,7 @@ and Claude Code reports that autocompact is thrashing. To avoid this:
|
||||
override.
|
||||
- To remove entire tool families from the upfront catalog, run
|
||||
`rea setup --client claude_code --tool-families <ids>` with the families the
|
||||
workflow needs (see [the advertising scope contract](../reference/mcp-contracts.md#tool-family-advertising-scope)).
|
||||
workflow needs (see [the advertising scope contract](mcp-contracts.md#tool-family-advertising-scope)).
|
||||
Out-of-scope tools are not advertised and their calls fail with the SDK's
|
||||
disabled-tool error, so widen the selection when a task needs another
|
||||
family. Setup writes the scope into the entry's `env` as
|
||||
@@ -348,7 +239,7 @@ Both keep canonical result validation and complete structured results; `compact`
|
||||
keeps a widened root shape and `off` advertises nothing. On catalogs large
|
||||
enough that full input schemas alone approach the frame limit, also keep the
|
||||
input profile at its `compact` default. See [MCP output schema
|
||||
profiles](../reference/mcp-contracts.md#output-schemas).
|
||||
profiles](mcp-contracts.md#output-schemas).
|
||||
|
||||
## Kimi Code
|
||||
|
||||
@@ -382,7 +273,7 @@ This headroom is a practical default, not a guarantee for every analysis
|
||||
duration. Setup preserves positive timeout values already in the `rea` entry.
|
||||
|
||||
Setup also forwards explicitly selected input/output schema profiles,
|
||||
`REA_MCP_MAX_RESPONSE_BYTES`, `REA_MCP_MAX_INPUT_ELEMENTS`, the Windows ChangeTime settle settings `REA_WINDOWS_CHANGE_TIME_DRIFT_MS`, `REA_WINDOWS_CHANGE_TIME_SETTLE_POLL_MS`, and `REA_WINDOWS_CHANGE_TIME_SETTLE_MAX_POLLS`, provider paths and supported Ghidra, JADX and Kuna
|
||||
`REA_MCP_MAX_RESPONSE_BYTES`, provider paths and supported Ghidra, JADX and Kuna
|
||||
settings. It also forwards explicit executable and resource selections for ADB,
|
||||
apktool, packet capture, GDB/Rizin/objdump, Cutter, WABT, Wakaru, firmware and
|
||||
signal tools, browser inspection, and pwntools/pwndbg. This includes their
|
||||
@@ -392,9 +283,8 @@ patterns. A provider being unavailable does not discard its selected paths;
|
||||
doctor reports its readiness separately. Explicit selections override discovered
|
||||
defaults, including when setup runs again with a changed selection. Unrelated
|
||||
environment variables, process ownership tokens and one-shot target paths are not
|
||||
copied. For an explicitly configured startup target, see the
|
||||
[initial-target settings](../reference/mcp-contracts.md#initial-target-at-startup). Existing
|
||||
client environment overrides remain intact unless setup supplies the same key.
|
||||
copied. Existing client environment overrides remain intact unless setup supplies
|
||||
the same key.
|
||||
You can edit `startupTimeoutMs` and `toolTimeoutMs` there; doctor keeps positive
|
||||
caller-selected values aligned and reports a non-blocking advisory when a tool
|
||||
timeout is shorter than the known provider allowance.
|
||||
@@ -415,22 +305,6 @@ Kimi Code also discovers personal skills in `~/.agents/skills` and project-local
|
||||
and verifies the matching skill files through the normal shared-skill flow.
|
||||
Start a new Kimi Code session to pick up MCP and skill changes.
|
||||
|
||||
## LM Studio
|
||||
|
||||
For [LM Studio](https://lmstudio.ai/docs/app/mcp), setup writes `mcpServers.rea`
|
||||
into `mcp.json` in the LM Studio home directory. Setup finds that directory as
|
||||
LM Studio does: the path stored in `~/.lmstudio-home-pointer`, otherwise
|
||||
`~/.cache/lm-studio` when it exists, otherwise `~/.lmstudio`. Setup only reads
|
||||
the pointer file. LM Studio parses `mcp.json` as strict JSON, so setup rejects
|
||||
a file with comments, trailing commas, or a byte order mark without a backup
|
||||
or a write. The entry carries `command`, `args`, and `env`, and setup keeps
|
||||
every other server. Local models often have small context windows, so setup
|
||||
selects `REA_MCP_INPUT_SCHEMA_PROFILE=compact` unless the stored entry or the
|
||||
setup environment selects a profile explicitly. Doctor reports an entry with no
|
||||
explicit profile as stale, as for Kimi Code. LM Studio 0.4.26 on macOS loaded
|
||||
this registration, and a local model called an REA tool through it. Linux and
|
||||
Windows remain unverified.
|
||||
|
||||
## Pi
|
||||
|
||||
For [Pi](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/mcp.md),
|
||||
@@ -504,7 +378,7 @@ standalone CLI check.
|
||||
A model API's `400 INVALID_ARGUMENT` alone does not identify a rejected schema,
|
||||
tool-count limit or skill defect. Preserve the response's parameter path and
|
||||
diagnostic, the actual running REA version, Gemini CLI version and selected
|
||||
model. For schema rejection, review the [input presentation profiles](../reference/mcp-contracts.md#compact-input-schema-profile)
|
||||
model. For schema rejection, review the [input presentation profiles](mcp-contracts.md#compact-input-schema-profile)
|
||||
and reconnect after changing REA's registration environment. Compact is the
|
||||
current default; an explicitly stored `full` choice is preserved. The generic
|
||||
error in [#4620](https://github.com/morluto/rea/issues/4620) does not establish
|
||||
@@ -585,7 +459,7 @@ Responses-compatible Kimi gateway as “infinite recursion”; expanding those
|
||||
references succeeded. A separate full-profile input failed the provider's
|
||||
schema-size limit. These are distinct reported failures, not proof that the
|
||||
canonical schema is recursive. Current REA inputs are already self-contained;
|
||||
see [schema advertisement](../reference/mcp-contracts.md#schema-advertisement).
|
||||
see [schema advertisement](mcp-contracts.md#schema-advertisement).
|
||||
|
||||
For this failure, inspect the active MCP launch command and refresh the REA
|
||||
installation that command actually runs. A configured package selector or the
|
||||
@@ -711,18 +585,6 @@ The user-level MCP paths follow the editor's VS Code layout. Trae's official
|
||||
documentation describes the raw JSON editor without specifying those paths;
|
||||
loading this registration in Trae remains unverified.
|
||||
|
||||
## CodeBuddy
|
||||
|
||||
For CodeBuddy, setup writes `mcpServers.rea` into the global
|
||||
`~/.codebuddy/mcp.json` (a standard `mcpServers` JSON file at the user home
|
||||
root on every platform). Entries carry `command`, `args`, and `env`; stdio is
|
||||
the default transport. CodeBuddy discovers personal skills from
|
||||
`~/.codebuddy/skills`, so setup installs its skill copy there.
|
||||
|
||||
The global MCP path is a plain JSON file; setup preserves every other server
|
||||
and top-level key when it upserts `rea`. Loading this registration, the
|
||||
bundled skill, and real REA tool calls through CodeBuddy are verified.
|
||||
|
||||
## WorkBuddy
|
||||
|
||||
For [WorkBuddy](https://www.workbuddy.cn/docs/workbuddy/From-Beginner-to-Expert-Guide/Function-Description/MCP-Guide), setup writes `mcpServers.rea` into the
|
||||
@@ -754,10 +616,9 @@ the fallback file untouched. Existing REA environment overrides are preserved
|
||||
when refreshing its command. A malformed or unreadable fallback file fails setup instead of
|
||||
being silently suppressed; repair or remove it and rerun. Uninstall removes an
|
||||
REA-owned `rea` entry from both the native table and the fallback, backing up
|
||||
each changed file. If removing the last native server would activate a retained
|
||||
fallback server, uninstall refuses before changing either file; review the
|
||||
fallback configuration before retrying. Other fallback servers, settings and a
|
||||
foreign `rea` entry are kept. ZCode disables a
|
||||
each changed file, so an emptied native table cannot load a leftover fallback
|
||||
registration; other fallback servers, settings and a foreign `rea` entry are
|
||||
kept. ZCode disables a
|
||||
server with `enable: false` on its entry; doctor reports such a registration
|
||||
as stale, and rerunning setup removes the flag to repair it. Native configuration
|
||||
uses strict JSON; setup refuses comments, trailing commas and a BOM rather than
|
||||
@@ -765,20 +626,21 @@ writing a registration ZCode cannot load.
|
||||
|
||||
## FreeBuff
|
||||
|
||||
For FreeBuff, `rea setup --client freebuff` writes `mcpServers.rea` into the
|
||||
user-scope `~/.agents/mcp.json` shared-agent file. Entries carry `command`,
|
||||
`args`, and `env`; stdio is the default transport. The personal skill location
|
||||
is `~/.agents/skills`, so setup installs its skill copy there. Because the same
|
||||
For FreeBuff, the proposed integration writes `mcpServers.rea` into the user-scope
|
||||
`~/.agents/mcp.json` shared-agent file. Entries carry `command`, `args`, and
|
||||
`env`; stdio is the default transport. The proposed personal skill location is
|
||||
`~/.agents/skills`, so setup installs its skill copy there. Because the same
|
||||
file doubles as ZCode's user-scope fallback (see above), a `freebuff`
|
||||
registration is also in the file ZCode uses until ZCode defines its own native
|
||||
server. Setup and doctor detect FreeBuff from its own profile directory
|
||||
(`~/.config/freebuff-desktop`), not from the shared `.agents` folder that REA's
|
||||
skill installs create for every client.
|
||||
registration is also in the file ZCode uses until ZCode defines its own native server.
|
||||
Setup and doctor detect FreeBuff from its own profile directory
|
||||
(`~/.config/freebuff-desktop`), not from the shared `.agents` folder that
|
||||
REA's skill installs create for every client.
|
||||
|
||||
FreeBuff MCP loading and skill discovery have not been verified with an
|
||||
installed native FreeBuff client/version; filesystem tests cover REA's editing,
|
||||
discovery, and ownership behavior only. Automatic applicability requires the
|
||||
FreeBuff profile directory even when the shared MCP file exists.
|
||||
The configuration write and personal skill locations are proposed integration
|
||||
boundaries. FreeBuff MCP loading and skill discovery have not been verified
|
||||
with an installed native FreeBuff client/version; filesystem tests cover REA's
|
||||
editing, discovery, and ownership behavior only. Automatic applicability
|
||||
requires the FreeBuff profile directory even when the shared MCP file exists.
|
||||
When uninstall cannot establish FreeBuff or another shared-storage reader from
|
||||
product discovery, a malformed or unreadable shared MCP file is retained
|
||||
unchanged and does not stop other REA removals, including `--purge-data`. When
|
||||
@@ -786,9 +648,8 @@ FreeBuff or ZCode product applicability is established or unresolved, the same
|
||||
malformed shared file stops uninstall before removals. See
|
||||
[Diagnose, update, and remove](installation.md#diagnose-update-and-remove).
|
||||
|
||||
FreeBuff setup support requires `[email protected]` or later (including the
|
||||
current npm `latest` release). Older versions reject
|
||||
`rea setup --client freebuff` as an unsupported client. See the
|
||||
FreeBuff setup support requires `[email protected]` or later. Older versions
|
||||
reject `rea setup --client freebuff` as an unsupported client. See the
|
||||
[installation guide](installation.md#supported-agents) for client availability.
|
||||
|
||||
## DeepSeek Harness
|
||||
@@ -834,110 +695,13 @@ in `~/.agents/skills` (`$DSH_AGENTS_HOME/skills` when configured); Harness
|
||||
discovers that directory, and its native `skill` tool can load
|
||||
`reverse-engineer-anything`.
|
||||
|
||||
REA's optional [native client verification](../development/testing.md#end-to-end-integration-and-golden-evidence)
|
||||
REA's optional [native client verification](testing.md#end-to-end-integration-and-golden-evidence)
|
||||
checks headless chat, skill loading, all tool input schemas and actual analysis
|
||||
using the installed Harness and a local model fixture. This establishes the
|
||||
Harness integration, with live DeepSeek API compatibility remaining unverified.
|
||||
See the [Harness MCP guide](https://github.com/deepseek-ai/deepseek-harness/blob/639ed015397290b3745d163aafe02ffee4aa3f84/packages/mcp/mcp-client/README.md)
|
||||
for its native configuration contract.
|
||||
|
||||
<a id="llamacpp"></a>
|
||||
|
||||
## llama.cpp
|
||||
|
||||
Use REA from llama.cpp's local Web UI through `llama-server`'s native stdio
|
||||
MCP support. REA supplies the analysis tools; llama.cpp runs your chosen model.
|
||||
It is not a replacement for a native analysis provider such as Ghidra or Hopper.
|
||||
|
||||
Install REA in a persistent location so the configuration below can use a stable
|
||||
entrypoint. For a project-local installation, run this in your chosen directory:
|
||||
|
||||
```bash
|
||||
npm install --save-exact rea-agents
|
||||
```
|
||||
|
||||
Or install it globally:
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
```
|
||||
|
||||
See [package runners and global installation](installation.md#package-runners-and-global-installation)
|
||||
for runtime requirements and installation options. Running `npx rea-agents setup`
|
||||
alone does not create either installation; do not use its temporary package-cache path.
|
||||
Use a llama.cpp build whose `llama-server --help` lists
|
||||
`--mcp-servers-config`; older builds without that option cannot use this route.
|
||||
The configuration below uses an already installed REA package, so tool discovery
|
||||
does not depend on downloading an npm package during server startup.
|
||||
|
||||
Save a JSON file, for example `rea-mcp.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "/absolute/path/to/node",
|
||||
"args": ["/absolute/path/to/rea-agents/scripts/rea.mjs", "mcp"],
|
||||
"timeout_ms": 450000
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Replace both paths with your installation's actual paths. For a project-local npm
|
||||
installation, run `npm root` in that project; the script is under
|
||||
`$(npm root)/rea-agents/scripts/rea.mjs`. For a global installation, use
|
||||
`$(npm root -g)/rea-agents/scripts/rea.mjs` instead. Write the resulting absolute
|
||||
paths as JSON strings; llama-server does not expand shell expressions there.
|
||||
On Windows, use forward slashes, such as `"C:/Program Files/nodejs/node.exe"`,
|
||||
or double each backslash, as in `"C:\\Program Files\\nodejs\\node.exe"`.
|
||||
Apply this to both `command` and the script path in `args`. Single backslashes
|
||||
can make the JSON invalid or change the path through escape sequences.
|
||||
`node -p "JSON.stringify(process.execPath)"` prints your current Node.js executable
|
||||
as a quoted, escaped JSON string that can be used as the `command` value.
|
||||
Use the version supported by REA. Preserve any other
|
||||
servers if you are adding REA to an existing configuration. `timeout_ms` is
|
||||
llama.cpp's per-tool-call timeout, including cold engine startup. The example
|
||||
allows REA's default 330-second Ghidra startup budget plus 120 seconds for the
|
||||
operation. Increase it if you configure a longer provider allowance or need more
|
||||
operation time; see [provider deadlines](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
|
||||
Start the server with your own tool-capable GGUF model:
|
||||
|
||||
```bash
|
||||
llama-server -m /absolute/path/to/model.gguf \
|
||||
--host 127.0.0.1 --port 8080 --parallel 1 --ctx-size 16384 \
|
||||
--mcp-servers-config /absolute/path/to/rea-mcp.json
|
||||
```
|
||||
|
||||
Open `http://127.0.0.1:8080`. REA tools appear with a `rea_` prefix, such as
|
||||
`rea_analyze_javascript_application`. Select only the tools needed for the
|
||||
conversation in the Web UI's tool settings: discovery of REA's complete catalog
|
||||
does not mean that its schemas and results fit the selected model's context.
|
||||
Choose a context size your model and machine support. For a first check, enable
|
||||
`rea_analyze_javascript_application` and ask it to inspect a small JavaScript
|
||||
directory you own. This task needs no native analysis engine. Inspect the
|
||||
returned Evidence and limitations rather than relying only on the model's summary.
|
||||
|
||||
If the model prints a proposed call as ordinary text, REA has not been called.
|
||||
Check the selected model and its chat template against llama.cpp's function-calling
|
||||
guidance. A working MCP connection does not establish reliable tool selection by
|
||||
every model.
|
||||
|
||||
`--tools all` is not needed; it enables llama.cpp's separate built-in tools.
|
||||
`--ui-mcp-proxy` is also not needed for this server-side stdio connection.
|
||||
REA's setup wizard does not register llama.cpp automatically; pass the JSON
|
||||
file to llama-server explicitly and restart it after changes.
|
||||
|
||||
The MCP child runs with llama-server's permissions. Loopback binding does not
|
||||
sandbox it or make every REA operation read-only. Enable only trusted commands,
|
||||
keep the server local, and select targets and operations you are authorized to use.
|
||||
|
||||
See llama.cpp's [MCP server configuration](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md#mcp-servers)
|
||||
and [function-calling guidance](https://github.com/ggml-org/llama.cpp/blob/master/docs/function-calling.md).
|
||||
The [verification record](../development/client-verification.md#llamacpp) distinguishes connection
|
||||
checks from model behavior and lists the tested versions.
|
||||
|
||||
## Other client settings
|
||||
|
||||
For OpenCode, setup writes the V1 `mcp.rea` entry, which OpenCode V1 and V2
|
||||
@@ -19,7 +19,7 @@ initialization, discovery and target-free request flow; it does not verify a
|
||||
client's configured launcher or its inherited environment. Test the generated
|
||||
registration through the actual client, including a reduced `PATH` when checking
|
||||
desktop or service startup. Record the client version and launch environment
|
||||
separately from terminal doctor results. See [launcher troubleshooting](../getting-started/client-configuration.md#launchers-and-path).
|
||||
separately from terminal doctor results. See [launcher troubleshooting](client-configuration.md#launchers-and-path).
|
||||
|
||||
## Published Pi code-mode renderer limit
|
||||
|
||||
@@ -33,7 +33,7 @@ shared form would leave later fields unknown instead. The existing
|
||||
its character limit by advertising reduced nested validation structure, while
|
||||
REA continues to validate calls against the complete canonical input schema.
|
||||
Compact is the default for the server's advertised inputs; an explicit `full`
|
||||
choice is preserved. See the [compact input schema profile](../reference/mcp-contracts.md#compact-input-schema-profile)
|
||||
choice is preserved. See the [compact input schema profile](mcp-contracts.md#compact-input-schema-profile)
|
||||
for details of its reduced forms.
|
||||
|
||||
Custom renderer hosts can raise `inputMaxChars` through `renderToolSignature` or
|
||||
@@ -43,30 +43,6 @@ does not establish broader Pi compatibility. See the pinned upstream
|
||||
[`declarations.ts`](https://github.com/earendil-works/pi/blob/b45e0ebcc5ae2fc42a6f2f23104d10e656e6bedb/packages/codemode/src/declarations.ts)
|
||||
and [renderer usage notes](https://github.com/earendil-works/pi/blob/b45e0ebcc5ae2fc42a6f2f23104d10e656e6bedb/packages/codemode/README.md#declarations-for-the-model).
|
||||
|
||||
<a id="llamacpp"></a>
|
||||
|
||||
## llama.cpp
|
||||
|
||||
The [manual stdio connection](../getting-started/client-configuration.md#llamacpp) was checked on
|
||||
macOS arm64 with Node.js 24.18.0, llama-server b11715 (`9d1878a43`), and the
|
||||
published `[email protected]` package. `GET /tools` exposed 207 REA tools without
|
||||
enabling llama.cpp's built-in tools. With the source checkout at `9e372c6`, all
|
||||
207 forwarded names, descriptions and input schemas matched a direct MCP client.
|
||||
|
||||
Calls through `POST /tools` returned session metadata and JavaScript analysis
|
||||
Evidence for a fixture at a Unicode path. A missing target returned REA's input
|
||||
error, and a subsequent session call still succeeded. The owned llama-server
|
||||
process exited successfully after the checks. No native analysis engine or
|
||||
Windows host was exercised.
|
||||
|
||||
A local Qwen3-1.7B Q8_0 run selected the JavaScript analysis tool and received its
|
||||
Evidence, but its follow-up answer reached the output-token limit. Other small
|
||||
Qwen3 attempts returned proposed calls as ordinary JSON text rather than tool
|
||||
calls. These runs do not establish dependable model-driven investigations.
|
||||
A request with the full catalog and task history exceeded a 16,384-token context;
|
||||
select tools for the task instead of assuming that discovery implies model fit.
|
||||
The checks exercised HTTP and stdio, not browser interaction with the Web UI.
|
||||
|
||||
## End-to-end, integration and golden evidence
|
||||
|
||||
The optional `verify:qwen-client`, `verify:pi-client`, and `verify:hermes-client`
|
||||
@@ -486,7 +462,7 @@ existing static JavaScript analysis through CLI and stdio MCP, including
|
||||
manifest readback, competing query variants, and resolved relative imports.
|
||||
It uses an installed browser and accepts an optional installed REA entrypoint.
|
||||
The complete `verify:browser` lane also exercises passive script export through
|
||||
both public adapters. See [website script export](../guides/web/website-script-export.md).
|
||||
both public adapters. See [website script export](website-script-export.md).
|
||||
|
||||
`verify:browser:modules` compares CLI and stdio MCP traces against an independent
|
||||
real Chromium module-loading fixture: import-map scopes, package prefixes,
|
||||
@@ -494,7 +470,7 @@ null/backtracking rejection, query/fragment identities, repeated module instance
|
||||
lazy and computed unknowns, exact source/map identities and no implicit refetch.
|
||||
Set `REA_BROWSER_EXECUTABLE`; the optional script argument selects an installed
|
||||
package entrypoint. The complete `verify:browser` lane and existing conditional
|
||||
Chrome CI include this verifier. See [module relationships](../guides/web/website-module-trace.md).
|
||||
Chrome CI include this verifier. See [module relationships](website-module-trace.md).
|
||||
|
||||
## Agent evaluation and conformance records
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
This guide explains the runtime settings design for maintainers and developers:
|
||||
who owns each value, when an update takes effect, how providers enforce it, and
|
||||
how to extend the system. The public tool contract is in
|
||||
[MCP contracts](../reference/mcp-contracts.md#editable-analysis-settings); recorded runtime
|
||||
[MCP contracts](../mcp-contracts.md#editable-analysis-settings); recorded runtime
|
||||
checks and their limitations are in the
|
||||
[verification report](../verification/runtime-analysis-settings.md).
|
||||
|
||||
@@ -121,22 +121,9 @@ launch environment affects the next REA process.
|
||||
is the byte-budget shape: a decimal safe integer with a 10 MiB floor
|
||||
(`MINIMUM_RESPONSE_BYTES`), the same shared schema, and no silent fallback. A
|
||||
change needs a REA restart and a matching client receive buffer. It does not
|
||||
raise REA's reserved string limit or Node's string ceiling. Like the regex
|
||||
deadline, it is in `SELECTED_REGISTRATION_SETTINGS`, so setup forwards an
|
||||
explicit selection into the client registration.
|
||||
|
||||
Windows ChangeTime admission waits out a fresh timestamp before recording a
|
||||
baseline. `REA_WINDOWS_CHANGE_TIME_DRIFT_MS` (default 2000 ms, floor 1 ms,
|
||||
maximum 60000 ms), `REA_WINDOWS_CHANGE_TIME_SETTLE_POLL_MS` (default 25 ms,
|
||||
floor 1 ms, maximum 1000 ms), and `REA_WINDOWS_CHANGE_TIME_SETTLE_MAX_POLLS`
|
||||
(default 40, floor 1, maximum 1000) are launch settings. Unset keeps those
|
||||
defaults. An invalid value is a configuration error: it names the setting, the
|
||||
rejected value, source `setting`, the default, the floor, and the maximum, and
|
||||
says to set a positive decimal integer in the environment that starts REA and
|
||||
then restart REA. There is no runtime override. Setup forwards an explicit
|
||||
selection. When the poll budget runs out, admission keeps the last observation
|
||||
instead of failing the readable file. Raising a setting only changes how long
|
||||
that wait can last. It does not change Windows ChangeTime itself.
|
||||
raise REA's reserved string limit or Node's string ceiling. The setup
|
||||
allowlist does not copy this setting. A new headroom setting should, following
|
||||
the regex deadline.
|
||||
|
||||
A per-call budget can have a runtime default without having an environment
|
||||
setting. JavaScript `max_heap_mb`
|
||||
@@ -235,25 +222,6 @@ Error copying, attribution, redaction, and delivery must preserve these facts.
|
||||
Human-readable remediation explains the same failure; clients should not need
|
||||
to parse that sentence to identify an editable setting.
|
||||
|
||||
`scope` and `applies_at` describe a continuing caller, such as an MCP
|
||||
connection. A one-shot invocation, such as a CLI command, has released its
|
||||
provider session and exited its runtime before its caller reads the failure,
|
||||
so its adapter projects with an explicit `"invocation"` caller lifetime.
|
||||
`details.limit` then keeps every other field and reports
|
||||
`scope: "rea-invocation"`, `applies_at: "next_invocation"` and the launch
|
||||
`environment` seed when one exists. `agent_editable: true` still holds, because
|
||||
the caller chooses that environment value or per-call `parameter` for the new
|
||||
invocation; `rea_restart_required: false` means no running REA process needs a
|
||||
restart. A producer keeps guidance that holds only while its caller retains the
|
||||
session, such as "the Ghidra session remains usable" or a same-session runtime
|
||||
update, in `continuingCallerGuidance`; the shared presentation adds it only for
|
||||
a continuing caller. The producer's own message and remediation must stay true
|
||||
after the session is released. A composed error records each embedded child
|
||||
failure through `childFailureDiagnostics`, which snapshots it at composition
|
||||
for both lifetimes when they differ, so cleanup history keeps its state and
|
||||
the final projection selects the caller's form. Failures retained inside
|
||||
Evidence keep the continuing form, so CLI and MCP Evidence stay identical.
|
||||
|
||||
Cache policy is part of enforcement. Successful reusable analysis can survive
|
||||
a budget update. Transient Ghidra decompile failures remain uncached so a raised
|
||||
deadline can recover on the same Program. Admission checks still apply to reused
|
||||
@@ -318,9 +286,7 @@ whether live repair is possible.
|
||||
## What agents do
|
||||
|
||||
Use `details.limit` for a direct runtime update when it names an editable
|
||||
setting with `scope: "rea-runtime"`. For `scope: "rea-invocation"`, set its
|
||||
`environment` or `parameter` for a new invocation instead.
|
||||
Follow `remediation.action` for per-call, fixed and external controls.
|
||||
setting. Follow `remediation.action` for per-call, fixed and external controls.
|
||||
Inspect settings only when bounds or application details are missing. Restart only when it says the value is
|
||||
read at startup. A sentence that a control does not raise the limit is not a
|
||||
request to change it.
|
||||
@@ -350,12 +316,8 @@ request to change it.
|
||||
`runtime-configuration:` check. Re-run `rea setup` for that client when the
|
||||
setting is on the registration allowlist, then reconnect so the new process
|
||||
sees it.
|
||||
- `invalid_request`: follow the specific recovery and reported issue. Inspect
|
||||
the schema or CLI help when the input violates the contract or the response
|
||||
asks for `inspect_input_contract`. A valid selector can instead require a
|
||||
different choice, such as selecting a reported address; an inventory miss
|
||||
does not establish that the target lacks the item. Do not repeat the
|
||||
unchanged call when the response says it cannot succeed unchanged.
|
||||
- `invalid_request`: correct the reported fields from the current schema or
|
||||
CLI help. Do not repeat the unchanged call.
|
||||
|
||||
A restart is required to adopt a changed launch environment or registration.
|
||||
The curated runtime settings above require no REA restart. It is not the first step for a transient
|
||||
@@ -399,7 +361,7 @@ application points, and resulting behavior. Check that adapters deliver the
|
||||
canonical recovery guidance, with independent assertions for its underlying
|
||||
facts. Avoid freezing prose or deriving all expected values from the same
|
||||
registry used by the implementation; either can hide the wrong failure.
|
||||
See [test design](../development/testing.md#test-design-and-pruning) for the assertion policy.
|
||||
See [test design](../testing.md#test-design-and-pruning) for the assertion policy.
|
||||
|
||||
Runtime checks cover atomic validation, stale writes, reset, runtime isolation,
|
||||
accepted-operation snapshots, and explicit argument precedence. Public MCP
|
||||
@@ -416,17 +378,3 @@ The [verification report](../verification/runtime-analysis-settings.md) separate
|
||||
public MCP checks, simulated provider coverage, real Ghidra execution, and
|
||||
measured task costs. It does not establish real-provider behavior for every
|
||||
editable setting or active worker telemetry.
|
||||
|
||||
## Bun standalone module graphs
|
||||
|
||||
`MAX_BUN_MODULE_COUNT` admits at most 100,000 module records.
|
||||
`MAX_BUN_RETAINED_BYTES` admits at most 256 MiB of serialized graph payload and
|
||||
separately bounds retained entry metadata. Physical metadata read work is also
|
||||
limited to 256 MiB per preparation, including repeated uncached ranges. A
|
||||
256 KiB / 1,024-entry exact-range cache avoids full-page amplification. The
|
||||
combined logical path tree admits at most 250,000 distinct prefixes; this is
|
||||
checked before adding nodes, with shared prefixes counted once. The shared
|
||||
artifact expansion budget also bounds the sum of referenced member lengths. Refusals name the boundary.
|
||||
There is no runtime override; additional host memory or `NODE_OPTIONS` does not
|
||||
raise these ceilings. Use a smaller producer fixture or a separately reviewed
|
||||
reader for larger graphs. No partial extraction is claimed on admission failure.
|
||||
@@ -1,379 +0,0 @@
|
||||
# Desktop application capture
|
||||
|
||||
`rea capture-desktop-application` launches an installed Linux desktop
|
||||
application, watches it while a person or a driver uses it, and records what
|
||||
it is, which windows it opened and how the files under the home directory
|
||||
changed. The result is the starting point for turning the application into an
|
||||
agent environment: the inventory says how to install and launch it, the windows
|
||||
say how to recognise it, and the checkpoints hold the state it read and wrote.
|
||||
|
||||
The application runs with the current user's permissions, in the current
|
||||
desktop session, against the user's real files. This is not a sandbox. The
|
||||
capture folder keeps copies of real files from the home directory, so treat it
|
||||
as private data and keep it local.
|
||||
|
||||
## Host support
|
||||
|
||||
Capture runs on Linux with an X11 display: `DISPLAY` must be set, and `xprop`
|
||||
(from `x11-utils`) must be installed, or `REA_XPROP_COMMAND` must name it.
|
||||
The window manager must support the EWMH client list (`_NET_CLIENT_LIST`,
|
||||
listed in `_NET_SUPPORTED`), as xfwm4, Mutter, KWin and Openbox do; without it
|
||||
the capture is refused, because no window could be seen. A supported list that
|
||||
is unset, as xfwm4 leaves it while no window is open, means no windows. Applications that run under XWayland are visible;
|
||||
native Wayland windows are not. Other hosts are refused before anything is
|
||||
launched.
|
||||
|
||||
## Capture an application
|
||||
|
||||
Name the application by its desktop entry id, the entry's absolute path, or an
|
||||
executable:
|
||||
|
||||
```sh
|
||||
rea capture-desktop-application --desktop-entry org.xfce.mousepad --output ./mousepad-capture
|
||||
rea capture-desktop-application --executable gnucash --argument /home/ada/books.gnucash --output ./gnucash-capture
|
||||
```
|
||||
|
||||
Desktop entries are looked up in `$XDG_DATA_HOME/applications` and then each
|
||||
`$XDG_DATA_DIRS` directory; an entry installed as a symbolic link, as flatpak
|
||||
and Nix do, is followed to its file. The entry's `Exec` line is expanded with no files;
|
||||
`--argument` values are appended in its place, and the application runs in the
|
||||
entry's `Path` or the home directory. When the line starts with
|
||||
`env NAME=value`, the program is found on the `PATH` those assignments set,
|
||||
with an empty or relative entry read from the folder the application runs
|
||||
in, and their XDG folders name the state directories. Otherwise the program is
|
||||
looked up on the host's `PATH` in the order the launch searches it, and one
|
||||
found only through a relative or empty entry is refused, since the exported
|
||||
computer would not have that entry. An `--executable` runs from
|
||||
the current directory, as it would from a shell, and a relative path to it is
|
||||
resolved there.
|
||||
|
||||
Before writing anything the command refuses to start if a window of the
|
||||
application is already open, or if the window list cannot be read. A launch
|
||||
beside a running instance can hand off to it, and that instance's unsaved
|
||||
state would be missing from the baseline. It then records the starting state,
|
||||
launches the application in its own process group with a run token in its
|
||||
environment, and reads commands from stdin, one per line. The application gets
|
||||
REA's environment without the `AGENTENV_*` variables an agent environment sets,
|
||||
such as its control token: an app the exported computer launches never has
|
||||
them, and an app captured inside a computer must not hold a credential to it.
|
||||
|
||||
| Line | Effect |
|
||||
| ------------------------------------------- | ----------------------------------------------------------- |
|
||||
| empty, `checkpoint`, or `checkpoint <name>` | Record the current state as a checkpoint |
|
||||
| any other text except `stop` | Record a checkpoint named by that text |
|
||||
| `stop`, or the end of stdin | Record a final checkpoint, close the application and finish |
|
||||
|
||||
An unnamed checkpoint is named `checkpoint-<ms>` after its time in the
|
||||
session. A name that repeats an earlier checkpoint, or is one of the capture's
|
||||
own names (`baseline`, `stop`, `time-limit`, `exit`), is refused with a
|
||||
progress line on stderr; no checkpoint is taken and the session continues.
|
||||
|
||||
Name the checkpoints a later step reads by what they mark. The
|
||||
[export](desktop-application-export.md) seeds the bundle from the baseline
|
||||
unless `--seed-checkpoint` names another, and a
|
||||
[task](desktop-application-tasks.md) is checked against the last checkpoint
|
||||
unless `--end-checkpoint` names another. So one capture can build a seed and
|
||||
then the change: launch the app, create and save the document, send
|
||||
`checkpoint seed`, make the edit and save it, send `checkpoint end`, then
|
||||
`stop`.
|
||||
|
||||
The session also finishes when the launched process has exited and the
|
||||
application's processes and windows are all gone in two consecutive samples,
|
||||
or when `--max-duration-seconds` passes. Before deciding the processes are
|
||||
gone, the capture looks for any process still carrying the run token, so a
|
||||
worker whose launcher exited before it was sampled keeps the session open. A
|
||||
sample whose window list could not be read is not taken as an empty one; the
|
||||
limitations say how many there were. One queued line is handled between
|
||||
samples, and reading stdin pauses while 16 lines wait.
|
||||
|
||||
Progress lines on stderr are JSON, `{"rea_progress": {"phase":
|
||||
"desktop_capture", "message": ...}}`. Those a driver waits on also carry an
|
||||
`event`: `{"kind": "window_open"}` once the window opens; `{"kind":
|
||||
"checkpoint", "name", "trigger", "at_ms", "entries", "changes",
|
||||
"retained_files", "retained_bytes"}` once a checkpoint has been taken, the
|
||||
baseline, `stop` and `exit` included, with the paths walked, the paths
|
||||
changed since the checkpoint before and the files retained; and `{"kind":
|
||||
"checkpoint_refused", "name", "reason"}` for a requested name it refused. A
|
||||
driver that sends `checkpoint <name>` waits for the event naming it rather
|
||||
than for a fixed time.
|
||||
|
||||
When the session ends the capture sends SIGTERM to the launched process, its
|
||||
process group and every process that carries the run token, waits up to five
|
||||
seconds for them to exit, and then removes every process that still carries
|
||||
the token, whether or not it stayed in the process group. If that cleanup
|
||||
cannot be verified after a second attempt, the capture is still written, and
|
||||
the command then fails, naming the run token and process group that may
|
||||
remain. Interrupting the command (Ctrl-C) ends the session the same way
|
||||
without a further checkpoint; the capture is still written with end reason
|
||||
`cancelled`, and the command exits with status 130. Interrupted before the
|
||||
application is launched, the command writes no capture and removes the
|
||||
content it had retained.
|
||||
|
||||
If the capture fails for any other reason before `capture.json` is written,
|
||||
the application is closed the same way and the content already retained is
|
||||
removed from the capture folder. The error names any process or file that
|
||||
could not be removed.
|
||||
|
||||
A person types these lines in the terminal. An agent can drive the
|
||||
application with a tool such as `xdotool` and write the lines to a FIFO that
|
||||
it passes as stdin. Because the end of stdin stops the session, the driver
|
||||
must keep the FIFO open for writing between checkpoints, for example with
|
||||
`exec 3>capture.fifo` in a shell, rather than open and close it for each
|
||||
line.
|
||||
|
||||
| Option | Default | Meaning |
|
||||
| ------------------------ | ------- | ----------------------------------------------------------------------------------------- |
|
||||
| `--state-path` | none | File or directory whose starting content is retained too (repeatable) |
|
||||
| `--exclude` | none | Path left out of every checkpoint walk (repeatable); the output folder is always left out |
|
||||
| `--max-duration-seconds` | 3600 | Stop the session after this long |
|
||||
| `--max-entries` | 250,000 | Paths recorded per checkpoint walk, its roots included |
|
||||
| `--max-file-bytes` | 64 MiB | Largest file whose content is retained |
|
||||
| `--max-content-bytes` | 1 GiB | Total retained content |
|
||||
| `--package-file` | none | The `.deb` file the application was installed from; its facts are recorded, not the file |
|
||||
|
||||
`--state-path`, `--exclude` and `--output` refuse an empty value, which
|
||||
would otherwise name the current directory.
|
||||
|
||||
## The capture folder
|
||||
|
||||
The output folder must be new, or empty and owned by the current user; the
|
||||
command makes it private (mode 0700). It holds:
|
||||
|
||||
- `capture.json`: the Evidence record, also printed on stdout.
|
||||
- `content/<sha256>`: the bytes of each retained file, stored once per
|
||||
distinct content.
|
||||
|
||||
The Evidence operation is `capture_desktop_application` and its predicate type
|
||||
is `rea.desktop-application-capture`. Its `host` names the platform,
|
||||
architecture, display and home directory, and the host's name (`hostname`)
|
||||
and the user's (`user`), which captured state can hold, as in a mail
|
||||
template's `ada@e81232be457d`; the export removes them from the bundle. A
|
||||
capture written before they were recorded has neither, and one is left out
|
||||
when the system cannot say, as for a user with no passwd entry. The rest of
|
||||
the result has four parts.
|
||||
|
||||
**Application.** The desktop entry (id, path, name, `Exec`, icon,
|
||||
`StartupWMClass`, categories, MIME types), the launch command and working
|
||||
directory, the executable as found on `PATH` and after resolving links, and the
|
||||
package that installed it. A Debian package records its name, version,
|
||||
architecture and the apt sources that list the installed version, with the
|
||||
origin (`o=`) apt reports for each source in `source_origins`, such as `Ubuntu`
|
||||
for the Ubuntu archive and its mirrors or `LP-PPA-<name>` for a PPA; an empty
|
||||
source list means the version came from a local `.deb` or the package lists
|
||||
are missing, which `package_lists_present` tells apart. apt's lists folder,
|
||||
and the trusted keys below, are where `apt-config` resolves
|
||||
`Dir::State::Lists`, `Dir::Etc::TrustedParts` and `Dir::Etc::Trusted` under
|
||||
`Dir`, so an `APT_CONFIG` file that moves them is followed: on a computer
|
||||
without lists, an ordinary user can run `apt-get update` with an `APT_CONFIG`
|
||||
naming writable `Dir::State::Lists` and `Dir::Cache` folders and capture with
|
||||
the same `APT_CONFIG`. Without `apt-config`, or when it fails, they are
|
||||
`/var/lib/apt/lists`, `/etc/apt/trusted.gpg.d` and `/etc/apt/trusted.gpg`.
|
||||
|
||||
The Debian package is the one `dpkg-query -S` names as the owner of the
|
||||
desktop entry the capture launched, or else of the executable, since the
|
||||
entry's package is what installs the application: LibreOffice Writer's entry
|
||||
belongs to `libreoffice-writer`, while the `/usr/bin/libreoffice` it runs
|
||||
belongs to `libreoffice-common`, which does not bring Writer with it. When the
|
||||
two differ, `executable_package` names the executable's package, which the
|
||||
export installs beside the entry's, since the entry's package need not depend
|
||||
on it (`vim-common` owns Vim's entry but does not depend on `vim`).
|
||||
`executable_package_reason` says why the computer could not install it by
|
||||
name, by the rule for library packages below: neither the Ubuntu 24.04 archive
|
||||
nor a vendor repository the export declares offers it, or `apt-cache policy`
|
||||
did not say. When a package owns the entry but none owns its program, such as
|
||||
a `~/.local/bin/<name>` found first on `PATH`, the entry's package is recorded
|
||||
without `executable_package`, and `executable_package_reason` says no package
|
||||
owns the program, so the export refuses rather than install a package that
|
||||
does not bring it. A program link that dpkg does not own counts as owned when
|
||||
dpkg owns the file it resolves to, as an alternatives link does. An
|
||||
`--executable` launch, or an entry no package owns, such as
|
||||
one in `~/.local/share/applications`, records the executable's package. When
|
||||
`dpkg-query` does not finish or exits with an error for the executable or the
|
||||
entry, the package is `unknown` and the limitations say which path, since a
|
||||
failed lookup is not proof that no package owns it. The sources, vendor
|
||||
repositories and `--package-file` match below are the recorded package's; the
|
||||
library walk still reads the executable.
|
||||
|
||||
Each source whose origin is not `Ubuntu` is also recorded in
|
||||
`apt_repositories`. `apt-get indextargets` names the sources file entry apt
|
||||
read it from and its release, so flat repositories match too, whether their
|
||||
suite is `./` (Typora's `deb https://downloads.typora.io/linux ./`) or `/`
|
||||
(pkgs.k8s.io's). The record holds that entry (`sources_file`, `uri`, `suite`,
|
||||
`components`, none for a flat repository, and `architectures`, the entry's
|
||||
`arch=` or `Architectures:` restriction, empty when it has none), `release`,
|
||||
the release's Suite (apt's `a=`) or else its Codename, and the key that signs
|
||||
it. A user name or password in an entry's URI (`https://user:password@host/…`)
|
||||
is never recorded: REA removes it from every URI it reads, in the sources file
|
||||
and in `apt-get indextargets` and `apt-cache policy` output alike, so entries
|
||||
match whether or not apt prints it, and sets `uri_credentials_removed` on a repository whose URI held
|
||||
one (a capture written before this was recorded omits the field). The key is
|
||||
found by verification, not by name: the candidates are the
|
||||
entry's `signed-by` keyrings, or else every `.gpg` and `.asc` key in apt's trusted
|
||||
parts folder and its trusted keyring, and each is checked with
|
||||
`gpgv` against the release's `InRelease` (or `Release` with `Release.gpg`)
|
||||
beside its index in apt's lists folder. A deb822 entry whose `Signed-By` field
|
||||
holds the armored key itself, as `add-apt-repository ppa:<owner>/<name>`
|
||||
writes it on Ubuntu 24.04, has that key as its only candidate, as apt trusts
|
||||
no other for it: REA reads the field's continuation lines as deb822 writes
|
||||
them, a `.` line standing for an empty one, from the key block's `BEGIN` line
|
||||
to its last `END` line, and records the key's path as the entry's sources
|
||||
file. A key counts when gpgv reports a valid signature by it and no bad,
|
||||
expired, revoked or unverifiable signature by it, whatever gpgv's exit status:
|
||||
a release also signed by a key the host lacks makes gpgv fail. Each candidate
|
||||
is copied, an armored key decoded to a binary keyring, into a private
|
||||
temporary folder that is removed afterwards; a folder REA cannot remove
|
||||
fails the capture with the folder as its cleanup resource. The key that
|
||||
verifies is recorded in `signing_key` with its path,
|
||||
SHA-256, format (`binary` or `armored`) and primary fingerprint, and its bytes
|
||||
are retained under `content/`. A key file holding one key is retained as it
|
||||
is; from one holding several, such as a legacy `/etc/apt/trusted.gpg`, only
|
||||
the transferable key with the verifying fingerprint is retained, in binary, so
|
||||
the bundle trusts no other key. When no candidate verifies, `gpgv` is
|
||||
missing, apt holds no signed release, no sources entry matches, the key the
|
||||
entry's `Signed-By` holds does not verify the release, or the key is larger
|
||||
than `--max-file-bytes` or the content budget, `signing_key` is null and
|
||||
`signing_key_reason` says why. Ubuntu Pro's ESM repositories (origins such as
|
||||
`UbuntuESMApps`, host `esm.ubuntu.com`) are not checked: apt reaches them only
|
||||
with the host's Ubuntu Pro credentials, which a computer does not have.
|
||||
|
||||
Unless the installed version comes from the Ubuntu 24.04 archive, so for a
|
||||
vendor repository's package and for a local install that no apt source lists,
|
||||
`library_packages` lists the packages that own the shared libraries the
|
||||
executable loads at start; the walk runs beside the repository check, and not
|
||||
at all for an archive package, whose export does not use it. REA reads
|
||||
DT_NEEDED from the resolved executable and from each library it finds, and
|
||||
looks each soname up as ld.so does: in the RPATH of the needing image and of
|
||||
each image up its chain of loaders to the executable when the needing image
|
||||
has no RUNPATH, then in its RUNPATH (with `$ORIGIN` expanded and paths
|
||||
normalized), and then in the `ldconfig -p` cache entries for the executable's
|
||||
machine. It asks `dpkg-query -S` for each library's package. Libraries in the
|
||||
executable's own folder belong to the application: they are walked but not
|
||||
listed, and neither is the application's own package. One batched
|
||||
`apt-cache policy` then keeps only the packages the Ubuntu 24.04 archive or a
|
||||
recorded vendor repository with a verified key offers, in any component its
|
||||
sources entry lists, since the export declares them all; the rest, such as
|
||||
packages from a PPA or a local `.deb`, are listed in
|
||||
`uninstallable_library_packages`. The same query checks `executable_package`.
|
||||
Each recorded repository's `supplies` lists the kept packages, and the
|
||||
executable's package, that the archive does not offer and its package files
|
||||
list, so the export can leave out those only a repository it does not declare
|
||||
supplies, such as one whose key the content budget later dropped. Sonames
|
||||
found nowhere are listed in
|
||||
`missing_libraries`, and libraries no package owns in `unpackaged_libraries`.
|
||||
`library_packages_reason` says why the lists are incomplete: the executable is
|
||||
not an ELF image (a launcher script, for example), `ldconfig`, `dpkg-query` or
|
||||
`apt-cache` is missing, a library could not be read or names more than 512
|
||||
libraries, or the walk stopped after 512 libraries.
|
||||
|
||||
A sources file, key or library whose handle REA cannot close fails the capture
|
||||
with the file as its cleanup resource. When the repository check and the
|
||||
library walk both fail, the capture reports both failures and every folder or
|
||||
file either could not release.
|
||||
|
||||
An application installed from a `.deb` file you downloaded, such as VariCAD
|
||||
from varicad.com, has no apt source a computer could install it from. Name
|
||||
that file with `--package-file` so the export can have the computer download
|
||||
it instead. REA reads the file's control fields with
|
||||
`dpkg-deb --field FILE Package Version Architecture Depends Pre-Depends`, and
|
||||
when its Package, Version and Architecture are the installed package's, it
|
||||
reads the file once more for its SHA-256 and size and records `package_file`:
|
||||
the file's `name` without its folder, `sha256`, `size`, `package`, `version`,
|
||||
`architecture`, and the `depends` and `pre_depends` entries as the file
|
||||
declares them. Otherwise `package_file_reason` says why, for example that
|
||||
`dpkg-deb` is not installed, that the file holds another version than the
|
||||
installed one, or that the file changed between the two reads (its device,
|
||||
inode, size and times are compared), since its fields and its hash could then
|
||||
describe different files. The capture never copies the file: it is the vendor's, and only
|
||||
these facts are kept.
|
||||
|
||||
Snaps, flatpaks and
|
||||
AppImages are recognised from their paths, and flatpaks also from a
|
||||
`flatpak run` launch, whose application id is the first operand after
|
||||
flatpak's options and their values. Electron applications are marked by
|
||||
the fuse sentinel in their binary or a `resources/app.asar` beside it. State
|
||||
directories are the XDG config, data, state and cache folders and the
|
||||
dot-folder named after the package, the executable, the desktop entry's full
|
||||
id and its last part, and the entry's name, with whether each existed at the
|
||||
start. A folder in those places whose name is one of these in another letter
|
||||
case joins them, such as GIMP's `~/.config/GIMP` for the `gimp` package; only
|
||||
the whole name is compared, so `~/.config/GIMPshop` does not. Dotfiles in the
|
||||
home named after the application (`.<name>rc`, or
|
||||
`.<name>` followed by `-`, `.` or `_`, for a name of three or more characters)
|
||||
that exist at the start join them, since an application may only read such a
|
||||
file during the capture. A `flatpak run` launch is named after its application
|
||||
id rather than the `flatpak` launcher, and adds `~/.var/app/<id>`.
|
||||
|
||||
**Session.** When it started, how long it ran and why it ended; the launched
|
||||
process's exit status; every process seen in the application's tree; every
|
||||
client window with its class pair, titles, owner pid and type, marked when it
|
||||
belongs to the application; when the first application window appeared; the
|
||||
most common class pair of the application's windows, for matching its window
|
||||
later; files the application's processes held open under the walked roots; the first
|
||||
8 KiB of the launched process's stderr; and whether cleanup was verified. Each
|
||||
process records whether the capture started it: the launched process, a
|
||||
process carrying the capture's run token, or a descendant of one, even when
|
||||
its environment cannot be read, as for a zombie or a sandboxed renderer.
|
||||
Processes are told apart by their start time, so a pid reused by another
|
||||
process is not counted as the application's.
|
||||
|
||||
A window belongs to the application when its owner pid is one of the
|
||||
application's live processes or its class matches the desktop entry's
|
||||
`StartupWMClass`, the executable's name, the entry's full id or its last part,
|
||||
or a flatpak's application id. A flatpak's windows report pids from its
|
||||
sandbox, so they are matched by class only.
|
||||
Ownership is checked again at every sample, because a window can appear before
|
||||
the process that owns it is found. A matching window's pid joins the
|
||||
application's processes, so a launch handed to a process the capture did not
|
||||
start, such as a D-Bus activated service, is still followed. That process does
|
||||
not carry the run token, so the capture never signals it and leaves it
|
||||
running.
|
||||
|
||||
**State.** The walked roots (the home directory plus any content root
|
||||
outside it), the content roots (state directories, `--state-path` values and
|
||||
absolute `--argument` values, except `/` or a directory that contains the home
|
||||
directory, which are named in the limitations instead; a selected path that is
|
||||
not a directory brings the `-wal` and `-journal` paths beside it, so a database
|
||||
selected on its own is captured with its logs), the state directories
|
||||
as walked (their physical spelling, which differs from the inventory's when one
|
||||
is a symlink), the paths every walk left out (the output folder and
|
||||
`--exclude` values), and the checkpoints in order: `baseline` before
|
||||
launch, each requested checkpoint, `stop` or `time-limit`, and `exit` after the
|
||||
application has gone. Each checkpoint records how many paths its walk saw,
|
||||
whether it was truncated, how many paths it could not read or directories on
|
||||
another device it did not enter (`unchecked_paths`), which paths were created, modified, deleted or
|
||||
changed type since the previous checkpoint, and which files it retained. A
|
||||
path counts as created or deleted only where the walk listed its nearest
|
||||
existing directory completely; differences under a directory that could not be
|
||||
listed, or that a truncated walk did not reach, are counted as unconfirmed
|
||||
instead. A walk does not enter a directory on another filesystem than its
|
||||
root, such as an NFS, sshfs or other FUSE mount under the home directory; the
|
||||
limitations list each such mount point. The
|
||||
baseline retains every file under the content roots; later checkpoints retain
|
||||
every changed file. A change records the file's type, size, mode,
|
||||
modification time and, when known, the SHA-256 of its content before and
|
||||
after, plus whether it lies in a state directory and whether the application
|
||||
held it open. A file is retained only if it is still the version the walk
|
||||
recorded and does not change while it is read. A file larger than the
|
||||
remaining content budget is not read, unless stored content of the same size
|
||||
could make it a duplicate. Files that could not be
|
||||
retained are listed with the reason: too large, over the content budget,
|
||||
changed since the walk, changed during the read, or unreadable.
|
||||
Directories the application held files open in, but whose starting content was
|
||||
not retained, are listed so a second capture can pass them with
|
||||
`--state-path`.
|
||||
|
||||
**Limitations.** What the capture could not establish, such as a truncated
|
||||
walk or a missing window.
|
||||
|
||||
## What the capture does not establish
|
||||
|
||||
Changes are found by comparing walks, using each path's size, modification
|
||||
time, inode and mode, so a change undone between two checkpoints is not seen
|
||||
and a rewrite that keeps all four is missed. A change is not attributed to the
|
||||
process that made it: other desktop services write under the home directory
|
||||
too, such as the dconf database, the window manager's settings and GTK's
|
||||
recently used list. The markers for state directories and open files are
|
||||
evidence for attribution, not proof.
|
||||
|
||||
Clicks and keystrokes are not recorded. Window titles are sampled every half
|
||||
second, so short-lived titles can be missed.
|
||||
@@ -1,342 +0,0 @@
|
||||
# Desktop application export
|
||||
|
||||
`rea export-desktop-application <capture-folder> <directory>` turns a folder
|
||||
written by [desktop application capture](desktop-application-capture.md) into an
|
||||
agentenv bundle holding one computer app environment: the real application,
|
||||
installed from the distribution archive, its vendor's apt repository or the
|
||||
`.deb` file it was installed from on the `ubuntu-24.04` computer base, launched
|
||||
at boot, seeded with the user's state and serving that state through agentenv's
|
||||
data operations.
|
||||
|
||||
```sh
|
||||
rea export-desktop-application ./mousepad-capture ./mousepad-bundle
|
||||
agent-env run ./mousepad-bundle --dry-run
|
||||
```
|
||||
|
||||
The export reads the capture and its [state view](desktop-application-state.md);
|
||||
it does not run the application. `--name` sets the environment name; otherwise
|
||||
it comes from the desktop entry id or the program, with an `app-` prefix when
|
||||
that name would not be a portable bundle name (for example all digits). The
|
||||
output directory must be new or empty. `--task` adds a task an agent wrote for
|
||||
the capture, once REA has validated it against the capture; see
|
||||
[desktop application tasks](desktop-application-tasks.md). `--package-url`
|
||||
says where the computer downloads the package of an application installed from
|
||||
a `.deb` file; see [Applications from a .deb file](#applications-from-a-deb-file).
|
||||
`--seed-checkpoint` names the checkpoint whose state seeds the bundle, the
|
||||
baseline by default, and `--end-checkpoint` the one a task must pass on, the
|
||||
capture's last by default; see [The seed checkpoint](#the-seed-checkpoint).
|
||||
|
||||
The command returns Evidence with operation `export_desktop_application` and
|
||||
predicate type `rea.desktop-application-export`, listing the environment name,
|
||||
the seed and end `checkpoints`, owned paths, seed files, the seeds it
|
||||
rewrote to remove the capture host's name and user (`scrubbed_files`), the
|
||||
bundle files that still hold them (`unscrubbed_files`), and written files,
|
||||
the last three by their path in the bundle. A failed export, cancellation
|
||||
included, removes the bundle it had not yet published, so no partial copy of the
|
||||
user's data remains. If the bundle was published but its output handles could
|
||||
not be closed, the command reports a cleanup error that carries the export
|
||||
Evidence; the bundle itself is complete.
|
||||
|
||||
## The bundle
|
||||
|
||||
| Path | Contents |
|
||||
| -------------------------------- | -------------------------------------------------------------------------------------------------- |
|
||||
| `envs/<name>/env.toml` | `type = "computer_app"`, the module and class, `computer = "ubuntu-24.04"`, `driver = "reference"` |
|
||||
| `envs/<name>/<file>.py` | The `ComputerAppEnvironment` class |
|
||||
| `envs/<name>/SKILL.md` | What the app is, its window class and where its files live |
|
||||
| `envs/<name>/seed/` | The application's files at the seed checkpoint, by home-relative path |
|
||||
| `envs/<name>/apt-keys/` | The signing key of each vendor apt repository the app installs from |
|
||||
| `envs/<name>/install-package.sh` | With `--package-url`: the run script that installs the downloaded package with dpkg |
|
||||
| `tasks/smoke.json` | A task that deploys the app |
|
||||
| `tasks/<task>.json` | With `--task`: the agent's task, its verifier in `artifacts/` and the eval in `evals/` |
|
||||
| `README.md` | What was exported, from which capture, and what to keep in mind |
|
||||
|
||||
The class file is `<name>_app.py` with hyphens as underscores, for example
|
||||
`mousepad_app.py`: agentenv imports it under its stem, so a plain `<name>.py`
|
||||
could shadow a standard library module such as `calendar`. A class name that
|
||||
would collide with a name the module imports or defines, such as `Path`, takes
|
||||
an `App` suffix.
|
||||
|
||||
The class declares:
|
||||
|
||||
- `Packages(apt=[...])` with the package the capture found. A vendor package
|
||||
is named `<package>/<release>`, its repository's release (the Suite apt
|
||||
reports as `a=`, or else its Codename), so apt takes it from that repository
|
||||
even when the Ubuntu archive offers a higher version or epoch, as noble's
|
||||
transitional `firefox` does over packages.mozilla.org's; it is named alone
|
||||
when the release names neither, or when the capture was written before
|
||||
releases were recorded. Its library packages follow, so a package that
|
||||
leaves out dependencies on the libraries it loads, as Typora's Electron
|
||||
package does, still starts; library packages the capture found neither the
|
||||
Ubuntu 24.04 archive nor a declared repository offers are left out, with a
|
||||
limitation, and so are those only repositories the export leaves out supply
|
||||
(a repository's `supplies`), such as one whose key the capture could not
|
||||
retain. An archive package's declared dependencies are trusted as they
|
||||
are. When the desktop entry's package is not the one that owns the program
|
||||
it runs, as `vim-common`'s entry runs `vim`'s program, the program's package
|
||||
(`executable_package`) follows the app's by name, since the entry's package
|
||||
need not depend on it; a capture that found no archive or declared
|
||||
repository offering it records why, and the export refuses it with that
|
||||
reason, as it does when only repositories it leaves out supply it, or when
|
||||
no package owns the program at all (`executable_package_reason` without
|
||||
`executable_package`). The package must come from the Ubuntu 24.04 archive
|
||||
or a mirror of it,
|
||||
or from a vendor apt repository whose signing key the capture verified and
|
||||
retained. A source counts as the archive when apt reports its origin as
|
||||
`Ubuntu` for a noble suite; a capture made before origins were recorded
|
||||
counts only Canonical's own archive hosts.
|
||||
- For a vendor repository, `sources=[AptSource(apt=..., suite=...,
|
||||
components=[...], key="apt-keys/<name>.gpg")]`, with the retained key copied
|
||||
to that path (`.asc` when it is armored, which agentenv tells by its
|
||||
content). A PPA added with `add-apt-repository`, whose sources entry holds its
|
||||
key in `Signed-By`, exports that key as `apt-keys/<name>.asc`. apt takes one
|
||||
key per URI and suite, so sources that share them, such as one repository
|
||||
listed for two architectures, are declared once, with their components
|
||||
merged; one with a different key is left out. A repository
|
||||
whose key the capture did not verify, an Ubuntu Pro (ESM) repository, one
|
||||
whose sources entry's URI held a user name or password (which the capture
|
||||
did not record and the export never writes; the reason names neither them
|
||||
nor the URI), one not reached over http or https, or one whose `arch=`
|
||||
restriction leaves out the capture host's architecture is refused with its
|
||||
reason (a capture
|
||||
written before restrictions were recorded counts as unrestricted); when
|
||||
another usable repository lists the same version it is left out instead,
|
||||
with a limitation. The export also refuses snaps, flatpaks, AppImages, and a
|
||||
package no recorded repository lists unless it was installed from a `.deb`
|
||||
file the capture recorded and `--package-url` is given, since the computer
|
||||
could not install them by name, and a capture written before credentials
|
||||
were removed whose repository URI still holds them, since it counted that
|
||||
repository's libraries as installable; capture it again.
|
||||
- `@computer_app(supports=...)` with both `amd64` and `arm64` for an archive
|
||||
package, which the archive serves for both, and only the capture host's
|
||||
architecture for a vendor package, since its repository may serve no other.
|
||||
A vendor package captured on any other architecture is refused. A package
|
||||
installed from its `.deb` file supports only that file's architecture.
|
||||
- A launch at boot with the captured command, ready when a window of the
|
||||
captured class appears and one of them is listed in the taskbar and is not
|
||||
typed as a splash: a splash screen or a dialog of the class skips the
|
||||
taskbar or says it is a splash, and an app whose splash shows first would
|
||||
otherwise count as ready before its own window opens. Arguments that are the captured home or under it, and
|
||||
`<option>=<path>` values under it, are respelled under the computer user's
|
||||
`$HOME`. agentenv starts the app in its own folder, where an open or save
|
||||
dialog would show the framework's files, so the command first changes to the
|
||||
captured working directory (`"$HOME"` when that was the home, as for a
|
||||
desktop entry without `Path`), which a relative argument or program path
|
||||
such as `./app` also needs, and still starts if that folder is missing. The
|
||||
export refuses a capture that saw no application window.
|
||||
- For an Electron application (the capture found its fuse sentinel or a
|
||||
`resources/app.asar`), `--no-sandbox` right after the program unless the
|
||||
command already has it: Chromium's sandbox needs user namespaces the
|
||||
computer's container does not grant, and Electron exits without them. The
|
||||
README and the result's limitations say the sandbox is off.
|
||||
- `paths`: the state directories holding the application's state files and the
|
||||
folders holding its documents, nested folders collapsed. A folder beside a
|
||||
state directory whose name differs from it only in letter case is owned
|
||||
whole in the files' spelling, so a `~/.config/GIMP` profile GIMP created
|
||||
during the capture beside the candidate `~/.config/gimp` is owned as
|
||||
`.config/GIMP`, not as the folder of its first file. The home itself and
|
||||
the shared `.config`, `.local`, `.local/share`, `.local/state` and `.cache`
|
||||
folders are never owned whole: a file directly in one is owned as itself.
|
||||
Files the capture recorded under a symlinked state directory's physical
|
||||
spelling are mapped back to the innermost such directory's home spelling,
|
||||
unless that spelling is outside the captured home, as when the home itself is
|
||||
a symbolic link. When two files map to one home path, the export uses the one
|
||||
present before the capture, else the first by path, and a limitation names
|
||||
the other.
|
||||
- Data hooks, because declaring any hook switches off agentenv's default data
|
||||
plane: the class seeds the home once per boot and keeps it as the initial
|
||||
state with `keep_initial`, `data/add` and `data/reset` call agentenv's `add`
|
||||
and `reset`, and `data/get` returns the files under `paths` plus a data part
|
||||
with a typed JSON view of each, read in a worker thread. An application with
|
||||
no observed state declares no paths and no hooks.
|
||||
|
||||
A file under the owned paths whose bytes at the seed the capture did not
|
||||
retain is not seeded: one in the state view without retained content at that
|
||||
checkpoint, or one the capture omitted (for example too large or past the
|
||||
content budget). The result lists them in `unseeded_files` and the README
|
||||
names them. When the application held files open in folders whose starting
|
||||
content the capture did not retain, a limitation names those folders. The
|
||||
desktop Trash, `~/.local/share/Trash`, is never the application's, so a file
|
||||
the user moved there is neither owned nor seeded, even when a window title
|
||||
names it; a limitation lists the files the Trash held at the seed.
|
||||
|
||||
The application's dconf keys at the seed, if any, are written once when the
|
||||
app first starts, also for an application that owns no files, and the class adds
|
||||
`dconf-cli` to its apt packages for that. The server starts before the
|
||||
desktop's D-Bus session, so the class does not use `dconf load`: it dumps any
|
||||
existing user database through a `file-db:` profile, then runs `dconf compile`
|
||||
on that dump and the captured keys, in that name order so the captured keys
|
||||
win, and replaces `~/.config/dconf/user`. If that fails, the server prints a
|
||||
one-line warning and the app starts without the captured keys. `data/reset` does
|
||||
not restore them.
|
||||
|
||||
## The seed checkpoint
|
||||
|
||||
By default the seed is the state before the capture, its `baseline`, and a
|
||||
task's end state is the capture's last checkpoint. `--seed-checkpoint` and
|
||||
`--end-checkpoint` name others, so one capture can make a seed and then a
|
||||
change: the user creates and saves a document and takes `checkpoint seed`,
|
||||
then edits and saves it and takes `checkpoint end`. Exported with
|
||||
`--seed-checkpoint seed --end-checkpoint end`, the bundle seeds the document
|
||||
as it was saved at `seed`, owned paths, dconf keys and unseeded files
|
||||
included, and a task must fail on that seed and pass on the state at `end`.
|
||||
A file's state at a checkpoint is its last version recorded at or before it.
|
||||
The export refuses a name the capture does not hold exactly once, and an end
|
||||
checkpoint taken before the seed checkpoint; `inspect-desktop-application-state`
|
||||
lists the names in `capture.checkpoints`.
|
||||
|
||||
## Host name and user
|
||||
|
||||
A seed can hold the capture host's name and the user's: Geany saved its mail
|
||||
template as `pref_template_mail=ada@e81232be457d`, the user `ada` on a
|
||||
container named `e81232be457d`, and many files hold the home path
|
||||
`/home/ada`. The capture records `host.hostname` and `host.user`, and the
|
||||
export removes them from what the bundle ships:
|
||||
|
||||
- In seeded text files (text, JSON, key files and XML in the state view), in
|
||||
UTF-16 files that start with a byte-order mark (keeping the mark, byte order
|
||||
and a following U+FEFF content character), and in the dconf keys,
|
||||
`user@host` becomes `user@localhost`, the
|
||||
host name standing alone becomes `localhost`, and the captured home becomes
|
||||
`/home/user` wherever it is followed by `/`, the end of the text, or a
|
||||
character that cannot continue a file name. The host name counts only as a
|
||||
whole name: not inside a longer word, before a `-`, or as a label of a
|
||||
domain such as `e81232be457d.local`, so a host named `ubuntu` leaves
|
||||
`ports.ubuntu.com/ubuntu-ports` alone. A host named `localhost` is left
|
||||
alone, but its `user@localhost` still takes the computer's user. The user
|
||||
name alone is never replaced, since a short one such as `ada` or `user` is
|
||||
also ordinary text. A rewritten seed is written with its own SHA-256, and
|
||||
`scrubbed_files` and the README list it.
|
||||
- JSON and XML keep their syntax. In JSON only string values change, each
|
||||
read as its decoded value, so `\u0065` escapes cannot hide the host, and is
|
||||
written back as JSON only when it changed; numbers and `true`, `false` and
|
||||
`null` stay as they are even for a host named `true`. Member names are never
|
||||
rewritten, since `{"ubuntu": 1, "localhost": 2}` on a host named `ubuntu`
|
||||
would get two `localhost` keys and lose a setting; a file whose names hold
|
||||
the host is listed in `unscrubbed_files` instead. In XML only character
|
||||
data, attribute values, comments and CDATA change, character references
|
||||
decoded the same way; element and attribute names and processing
|
||||
instructions stay. A UTF-16 file is scrubbed by the format its text would
|
||||
have in UTF-8, so JSON or XML in UTF-16 follows these rules too.
|
||||
- In the dconf keys only GVariant strings change, each read as its decoded
|
||||
value and written back in its own quotes only when it changed. Group names,
|
||||
keys, dictionary keys, bytestrings and everything outside quotes stay, so a
|
||||
host named `true` leaves `wrap=true` alone. A value whose string does not
|
||||
parse, or that holds an object path or signature, ships as captured.
|
||||
- `SEED_HOME` in the class is `/home/user`, the home the seed spells, which
|
||||
the class respells to the computer's own home when it seeds the home at
|
||||
boot, as it did the captured home before. The replacements read as the
|
||||
computer's own: its user is `user`, with its home in `/home/user`, and
|
||||
`localhost` is the name every host has for itself.
|
||||
- Binary files, SQLite databases among them, and text files over 256 MiB are
|
||||
copied as captured. Then every file the bundle ships, the class module with
|
||||
its dconf keys and the README included, is searched by a looser rule than
|
||||
the replacement's: the host name and `user@host` anywhere, as in
|
||||
`e81232be457d.local` or a seed named `e81232be457d-notes.txt`, and the
|
||||
captured home as a whole path, in its path and its content, JSON and dconf
|
||||
strings and XML references decoded and the replacements themselves set
|
||||
aside. Each one found is listed in `unscrubbed_files`, and in the README
|
||||
unless it is the README, to check before sharing the bundle. Bytes that are not text are
|
||||
searched as they are, so a binary file that holds the host in another
|
||||
encoding is not listed.
|
||||
- A task check whose values name `user@host` or the host name anywhere, by
|
||||
that rule, is refused, as one naming the captured home is: the bundle does
|
||||
not hold them. Its field names, kind and id are not values, so a host named
|
||||
`file` or `kind` refuses no check. A task's checks must fail on the seed as
|
||||
the bundle ships it, scrubbed, so a check the scrub itself satisfies, such
|
||||
as a mail setting the user changed from `ada@e81232be457d` to
|
||||
`user@localhost`, does not count as failing there.
|
||||
|
||||
A capture written before the host's name and user were recorded has only its
|
||||
home removed, with a limitation saying so; capture again to remove the rest.
|
||||
|
||||
## Applications from a .deb file
|
||||
|
||||
A closed-source application you installed from a `.deb` file you downloaded,
|
||||
such as VariCAD from varicad.com, is not in any apt repository the computer can
|
||||
use. Capture it with `--package-file` naming that file (see
|
||||
[desktop application capture](desktop-application-capture.md)), then pass
|
||||
`--package-url` to the export with an `http://` or `https://` URL the computer
|
||||
can download that same file from:
|
||||
|
||||
```sh
|
||||
rea capture-desktop-application --executable varicad --package-file ~/Downloads/varicad2026-en_2.03_amd64.deb --output ./varicad-capture
|
||||
rea export-desktop-application ./varicad-capture ./varicad-bundle --package-url https://files.example.com/varicad2026-en_2.03_amd64.deb
|
||||
```
|
||||
|
||||
The class then declares, beside `Packages`:
|
||||
|
||||
- `downloads=[Download(url=..., sha256=..., to="vendor/<file>", arch=...)]`
|
||||
with the captured file's SHA-256. agentenv downloads the URL once, when the
|
||||
bundle is put, and refuses the file unless its SHA-256 matches, so the
|
||||
computer installs exactly the captured version. The bundle does not contain
|
||||
the package.
|
||||
- `Packages(apt=[...])` with the first alternative of each of the file's
|
||||
Depends and Pre-Depends entries, without version constraints or `:any`
|
||||
qualifiers, then the library packages the capture found, since a vendor's
|
||||
binary may load libraries its Depends leaves out.
|
||||
- `run=[RunScript(script="install-package.sh")]`, a script the export writes
|
||||
into the environment folder. agentenv runs it as root from the app's folder
|
||||
once the apt packages are installed and the download is in place, with the
|
||||
network off: it runs `dpkg -i` on the download and then removes it.
|
||||
|
||||
The bundle supports only the file's architecture. Vendor sites often refuse
|
||||
automated downloads, as varicad.com does, so the URL may need to point at a
|
||||
copy you keep, such as a private mirror; you must be allowed to use that copy,
|
||||
and the package's licence, such as a 30-day trial's terms, is yours to check.
|
||||
The README and the result's limitations say so.
|
||||
|
||||
The export refuses a package no recorded repository lists when `--package-url`
|
||||
is missing, saying to capture with `--package-file` if the capture lacks it. It
|
||||
also refuses `--package-url` when the capture recorded no package file, giving
|
||||
the capture's `package_file_reason` when there is one; when the archive or a
|
||||
declared vendor repository installs the package already; when the URL is not
|
||||
written as `http://` or `https://` or holds a user name or password; and when
|
||||
the file is built for an architecture other than `amd64`, `arm64` or `all`.
|
||||
|
||||
## Typed views
|
||||
|
||||
The views follow the [state view](desktop-application-state.md)'s formats and
|
||||
shapes: SQLite tables (at most 64 ordinary tables, `tables_truncated` past
|
||||
that, and per table `columns`, `primary_key`, at most 500 `rows` and
|
||||
`truncated`, with blobs, unsafe integers and text that is not UTF-8 tagged as
|
||||
REA tags them), JSON, XML as `{root, truncated}` with REA's node, depth and
|
||||
whitespace rules and its `doctype` when the document declares one, key file
|
||||
groups, text as
|
||||
`{lines, line_count, truncated}` split on `\n` and `\r\n`, and zip archives
|
||||
as their entries, read with Python's `zipfile` from the file in place, with
|
||||
REA's entry, size, compression ratio and archive limits and its wording for an
|
||||
entry it does not read. The module reads the central directory itself, as REA
|
||||
does, and shows `zipfile` only the records it lists, reading each entry in
|
||||
bounded chunks no further than a byte past its declared size. A JSON file
|
||||
holding a number JavaScript cannot read exactly, such as an integer above 2^53,
|
||||
or whose retained value nests more than 500 levels deep, gets an `error` instead of a view, as in
|
||||
REA. Duplicate keys keep their last value before the depth check. Text that is not one JSON
|
||||
document, such as JSON Lines, stays text. Format detection follows REA's
|
||||
rules. Files larger than 64 MiB get an `error` instead of a view
|
||||
and are not read. XML is read with Python's `expat` twice: with namespace
|
||||
processing, which refuses what REA's parser refuses, such as an undeclared
|
||||
prefix, then without it, so element and attribute names keep the prefixes the
|
||||
document spells and namespace declarations stay attributes, as in REA's view;
|
||||
text splits into nodes where REA's splits it, at CDATA sections, comments and
|
||||
processing instructions. Both passes read a DOCTYPE as REA does: with
|
||||
parameter entity parsing off, so the DTD it names is never read, and handlers
|
||||
that refuse an internal subset, an external entity and an entity expat would
|
||||
skip; the first pass also checks each start tag for a reference to an
|
||||
undeclared entity, which expat drops from an attribute value without a word
|
||||
when a DOCTYPE names a DTD. Known differences remain:
|
||||
|
||||
- dconf databases are reported as binary, since REA reads them with `dconf`;
|
||||
inside an archive, one is `dconf` with an error, as in REA.
|
||||
- Zip entry names: a name in an Info-ZIP Unicode Path extra field is used only
|
||||
where Python's `zipfile` reads it (3.12 and later), and an entry whose local
|
||||
header names it differently from the central directory is damaged rather
|
||||
than read.
|
||||
- SQLite databases are read in place, read-only, rather than from a staged copy;
|
||||
a write lock the running app holds is waited out for a quarter of a second
|
||||
and then gives the file an `error`, and so does a table or column name that
|
||||
is not UTF-8.
|
||||
|
||||
The seed is the user's real data. Keep the bundle local and do not publish it.
|
||||
The class was checked against agentenv-framework 0.9.1431 with
|
||||
`python -m agentenv_protocol.computer._declare` and `agent-env run --dry-run`.
|
||||
@@ -1,211 +0,0 @@
|
||||
# Desktop application state
|
||||
|
||||
`rea inspect-desktop-application-state <capture-folder>` reads a folder written
|
||||
by [desktop application capture](desktop-application-capture.md) and describes
|
||||
the application's state: which observed files are the application's, what each
|
||||
retained version contains as typed JSON, and how each file changed between
|
||||
checkpoints. It reads only the capture folder; the application is not run
|
||||
again.
|
||||
|
||||
```sh
|
||||
rea inspect-desktop-application-state ./mousepad-capture --format json
|
||||
```
|
||||
|
||||
The result is Evidence with operation `inspect_desktop_application_state` and
|
||||
predicate type `rea.desktop-application-state`. Its confidence is `inferred`,
|
||||
because roles come from heuristics; the views themselves are read from the
|
||||
retained bytes, which are checked against their SHA-256 before use.
|
||||
|
||||
## Paths and roles
|
||||
|
||||
The view covers every file retained at the baseline and every file that
|
||||
changed later. Each path records the evidence the capture holds about it:
|
||||
|
||||
| Reason | Meaning |
|
||||
| ------------------------ | ------------------------------------------------------------------------- |
|
||||
| `state_directory` | It lies in one of the application's state directories, in either spelling |
|
||||
| `opened_by_application` | An application process held it open during a sample |
|
||||
| `named_in_window_title` | An application window title spelled its name or `~/` path on its own |
|
||||
| `selected_path` | It lies under a `--state-path` or absolute `--argument` of the capture |
|
||||
| `dconf_application_keys` | It is a dconf database holding groups whose path names the application |
|
||||
| `desktop_service` | It lies where desktop services and toolkits keep shared files, or caches |
|
||||
| `desktop_trash` | It lies in the desktop Trash, `~/.local/share/Trash` |
|
||||
|
||||
A folder beside a state directory whose name differs from it only in letter
|
||||
case counts as that state directory, so a profile the application first
|
||||
creates during the capture, such as GIMP's `~/.config/GIMP` beside the
|
||||
candidate `~/.config/gimp`, is its state too.
|
||||
|
||||
The role follows from the reasons, in this order: `shared_desktop` for a file
|
||||
in the desktop Trash, which the user deleted, whatever else names it;
|
||||
`application_state` for application keys in dconf, or a state directory outside shared desktop
|
||||
locations; `document` for a file named in a window title or selected by the
|
||||
caller, even if the application holds it open; `application_state` for another
|
||||
open file outside shared locations; `shared_desktop` for other files in shared
|
||||
locations; `unrelated` otherwise. A window title names a file only when no
|
||||
other file name character adjoins the name, so `day.txt` does not match
|
||||
`today.txt - Notes`.
|
||||
|
||||
Application names, which select dconf groups, come from the package, the
|
||||
executable, the desktop entry's id and `StartupWMClass`, and the window match.
|
||||
The entry's display name is left out, since a generic name such as `Settings`
|
||||
would claim other applications' groups.
|
||||
|
||||
## Versions and views
|
||||
|
||||
Each path lists its versions: the baseline, then its state after each
|
||||
checkpoint where it, or a database's `-wal` or `-journal` file, changed. A
|
||||
checkpoint that leaves an absent file absent adds no version. A file's state
|
||||
at any checkpoint is its last version recorded at or before it, which is how
|
||||
the [export](desktop-application-export.md#the-seed-checkpoint) and the
|
||||
[task draft](desktop-application-tasks.md) read it at the seed and end
|
||||
checkpoints they are given; `capture.checkpoints` lists the names in order.
|
||||
Views hold the captured bytes as they are, the capture host's name and user
|
||||
included; the export removes those from the bundle. A version records
|
||||
whether the file was present, the SHA-256 of its retained bytes, and a view:
|
||||
|
||||
| Format | Detected by | View |
|
||||
| --------- | ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
|
||||
| `sqlite` | The `SQLite format 3` header | Every ordinary table's columns, primary key and up to 500 rows |
|
||||
| `dconf` | The `GVariant` header | Groups whose path names the application, plus the names of the other groups |
|
||||
| `zip` | A zip local file header, `PK\x03\x04`, whatever the name | Its file entries in archive order, each with its own format and view |
|
||||
| `json` | A `.json` name or a leading `{` or `[` that parses | The parsed value, from files up to 4 MiB nested up to 500 levels |
|
||||
| `xml` | An XML name or declaration that parses | An element tree (`name`, `attributes`, `children`); text kept exactly, DTDs never read |
|
||||
| `keyfile` | Only `[group]`, `key=value` and comment lines, with at least one group | Groups of `key` → string value, from files up to 4 MiB |
|
||||
| `text` | Other UTF-8 text | Lines, up to 5,000 |
|
||||
| `binary` | NUL bytes or invalid UTF-8 | None |
|
||||
|
||||
A JSON value holding a number JavaScript would round, such as an integer above
|
||||
2^53, is not read rather than shown rounded.
|
||||
|
||||
An XML view names elements and attributes as the document spells them, prefix
|
||||
included: a saved Inkscape drawing's root is `svg`, in its default namespace,
|
||||
with `sodipodi:docname` and the declarations `xmlns` and `xmlns:sodipodi`
|
||||
among its attributes, and a Writer paragraph is `text:p`. An element's
|
||||
`children` hold its elements and text in order, text split where a CDATA
|
||||
section, comment or processing instruction falls. Whitespace-only text between
|
||||
child elements is formatting, and dropped, in an element that holds no other
|
||||
text, unless `xml:space="preserve"` applies; in mixed content, an element with
|
||||
text of its own beside its children such as a paragraph split into runs, it is
|
||||
kept, so `<p>Meet on <s>Sunday</s> <s>evening</s></p>` keeps the space between
|
||||
the runs. Whitespace here is XML's: space, tab, carriage return and line feed;
|
||||
a no-break space is text. The view keeps up to 5,000 nodes, comments and
|
||||
processing instructions counted, nested up to 64 levels, and sets `truncated`
|
||||
when it leaves some out.
|
||||
|
||||
A document that begins with a DOCTYPE naming its DTD, as AbiWord's `.abw`
|
||||
does, reads as the same element tree, and the view records the declaration
|
||||
beside `root`, its public identifier normalized as XML matches it:
|
||||
|
||||
```json
|
||||
{
|
||||
"doctype": {
|
||||
"name": "abiword",
|
||||
"public_id": "-//ABISOURCE//DTD AWML 1.0 Strict//EN",
|
||||
"system_id": "http://www.abisource.com/awml.dtd"
|
||||
},
|
||||
"root": { "name": "abiword", "attributes": {}, "children": [] },
|
||||
"truncated": false
|
||||
}
|
||||
```
|
||||
|
||||
An identifier the DOCTYPE leaves out is `null`, and a document without one
|
||||
has no `doctype`. The DTD is never fetched or read, so it adds no default
|
||||
attributes and declares no entities. A DOCTYPE with an internal subset
|
||||
(`<!DOCTYPE a [ … ]>`), which could declare entities, is not read as XML, nor is
|
||||
a document that refers to any entity but XML's five (`&`, `<`, `>`,
|
||||
`'`, `"`); such a file is text.
|
||||
|
||||
Exported XML views use REA's same formatting-whitespace classification, so
|
||||
retained XML text and the task checks that address it stay aligned.
|
||||
|
||||
SQLite databases are read through REA's SQLite worker from a private copy,
|
||||
with any retained `-wal` or `-journal` file staged beside it; the worker
|
||||
accepts an inactive rollback journal and refuses a hot one. A version is not
|
||||
read when its `-wal` or `-journal` file existed but was not retained, or when
|
||||
the capture did not record whether either existed, since the database alone
|
||||
could misstate its committed rows. The capture records them under its content
|
||||
roots, which include both beside a database selected on its own, when its
|
||||
baseline walk listed every directory, read every path, entered no other
|
||||
device, did not stop early and recorded what it left out, and wherever they change; an excluded log, and any elsewhere, is
|
||||
unknown, since an unchanged file and a missing one look alike. A table
|
||||
that cannot be read keeps its `error` in place of its rows, and changes to that
|
||||
database are marked incomplete. dconf
|
||||
databases are read with `dconf dump /` against a private file profile, so the
|
||||
live database and the dconf service are not touched; this needs `dconf-cli`, or
|
||||
`REA_DCONF_COMMAND`.
|
||||
|
||||
A version has a `view_error` when its bytes were not retained, are over the 64
|
||||
MiB read limit, are missing or no longer match their digest, exceed a format's
|
||||
limit above, or its reader failed.
|
||||
|
||||
## Archives
|
||||
|
||||
Many applications save a document as a zip archive of XML and other files:
|
||||
FreeCAD's `.FCStd` and `.FCBak`, OpenDocument (`.odt`, `.ods`, whose
|
||||
`content.xml` is the document), Office Open XML, EPUB and JAR. Such a file is
|
||||
`zip`, and its view lists its file entries in archive order, leaving out
|
||||
directories: an entry whose name ends in `/`, or whose attributes mark a Unix
|
||||
directory or, from an MS-DOS host, the directory attribute:
|
||||
|
||||
```json
|
||||
{
|
||||
"entries": [
|
||||
{
|
||||
"name": "Document.xml",
|
||||
"size": 11923,
|
||||
"sha256": "…",
|
||||
"format": "xml",
|
||||
"view": {
|
||||
"root": { "name": "Document", "attributes": {}, "children": [] },
|
||||
"truncated": false
|
||||
},
|
||||
"view_error": null
|
||||
}
|
||||
],
|
||||
"entry_count": 15,
|
||||
"truncated": false
|
||||
}
|
||||
```
|
||||
|
||||
Each entry's bytes are read from the archive's verified bytes in memory,
|
||||
checked against its CRC-32 and declared size, and given a `format` and `view`
|
||||
by the rules and limits above, with the entry's name in place of a file name.
|
||||
Nothing is extracted to disk. An entry has a `view_error`, with `sha256`,
|
||||
`format` and `view` null, when it is encrypted, compressed other than stored or
|
||||
deflated, larger than 16 MiB, more than 100 times its compressed size, past the
|
||||
64 MiB read from one archive, or its data is damaged. A SQLite or dconf
|
||||
database, or an archive, inside an archive keeps its format and `sha256` and
|
||||
has a `view_error` instead of a view: it is not read. The view lists at most
|
||||
1,000 entries; `entry_count` counts them all, and `truncated` is true when some
|
||||
are not listed. REA reads the central directory itself, a record at a time,
|
||||
through the Zip64 end record whenever a Zip64 locator precedes the end record,
|
||||
as Python's `zipfile` does, and hands the zip reader only the records it lists,
|
||||
so an archive of hundreds of thousands of entries costs no more to read
|
||||
than one of a thousand. An archive whose central directory cannot be read has a
|
||||
`view_error` of its own.
|
||||
|
||||
## Kinds of change
|
||||
|
||||
Between consecutive versions a change is `created`, `deleted`, or a
|
||||
format-specific kind with what was added, removed and changed (up to 100 of
|
||||
each, with counts):
|
||||
|
||||
- `lines` for text: lines added and removed, as a multiset.
|
||||
- `values` for JSON and XML: JSON pointers to values, containers included, so
|
||||
an array that becomes an object with the same members is changed. A pointer
|
||||
longer than 4096 characters is counted but not listed.
|
||||
- `values` for zip archives too, with array members named by
|
||||
[selectors](desktop-application-tasks.md#pointers-and-selectors) rather than
|
||||
positions wherever a selector names one member in both versions: entries by
|
||||
name, as `/entries/[name=Document.xml]`, and XML elements by tag and an
|
||||
attribute, as `/view/root/children/[name=Object][@name=Hole]`, so an object
|
||||
inserted before another does not shift it. A comparison is incomplete when
|
||||
either view leaves out entries, or an entry's view is missing or truncated.
|
||||
- `keys` for key files and dconf: `group/key`, with `~` and `/` in the key
|
||||
written `~0` and `~1` as in a JSON pointer, so `[a/b]` `c` and `[a]` `b/c`
|
||||
stay apart.
|
||||
- `rows` for SQLite: `table:` and the row's primary key, or the whole row when
|
||||
the table has none.
|
||||
- `content` for binary files, and `unreadable` when either version has no
|
||||
view.
|
||||
@@ -1,343 +0,0 @@
|
||||
# Desktop application tasks
|
||||
|
||||
A [desktop application export](desktop-application-export.md) can carry one
|
||||
task: a prompt for a computer-use agent and checks that grade what the agent
|
||||
left in the application's files. An agent, not REA, writes the prompt and the
|
||||
checks from the change the capture recorded. REA validates the task against
|
||||
the capture and keeps it only when its checks fail on the seed and all pass on
|
||||
the captured end state. REA runs no model and no code the agent wrote: the
|
||||
checks are declarative, and the exported verifier evaluates them the same way
|
||||
REA does.
|
||||
|
||||
```sh
|
||||
rea describe-desktop-application-task ./mousepad-capture --format json > draft.json
|
||||
# edit the draft's task into task.json: a name, the user's request, the checks
|
||||
rea export-desktop-application ./mousepad-capture ./mousepad-bundle --task task.json
|
||||
agent-env run ./mousepad-bundle --task add-milk --agent claude-code
|
||||
```
|
||||
|
||||
## Describe the change
|
||||
|
||||
`rea describe-desktop-application-task <capture-folder>` reads the capture and
|
||||
its [state view](desktop-application-state.md) and returns Evidence with
|
||||
operation `describe_desktop_application_task` and predicate type
|
||||
`rea.desktop-application-task-draft`:
|
||||
|
||||
- `application`: the environment name the export would use, the title and the
|
||||
window class.
|
||||
- `owned_paths`: the paths the export owns, the only ones whose files the
|
||||
app's `data/get` serves and checks can read.
|
||||
- `checkpoints`: the two checkpoints compared, `baseline` for the seed and
|
||||
`final` for the end. They are the capture's first and last unless
|
||||
`--seed-checkpoint` and `--end-checkpoint` name others, as for the
|
||||
[export](desktop-application-export.md#the-seed-checkpoint), which takes
|
||||
the same options; a name the capture does not hold, or an end before the
|
||||
seed, is refused.
|
||||
- `files`: each file under them whose state differs between those
|
||||
checkpoints, with its home path, role, its versions at both (`baseline` and
|
||||
`final`, the last recorded at or before each) and the net change between them, from the state view's own diff (`lines`, `values`,
|
||||
`keys` or `rows` with what was added, removed and changed, or `created`,
|
||||
`deleted`, `format`, `content`, `unreadable`; `content` also when the
|
||||
views hold only part of a file and match while the file changed). A
|
||||
database is compared by its rows, since its hash leaves out the `-wal`
|
||||
file. Text changes list each line
|
||||
once and add `line_counts`, the lines found at both ends whose number of
|
||||
occurrences changed. Views appear only for the files the suggestions cover,
|
||||
with arrays past 200 items and strings past 2,000 characters cut
|
||||
(`view_cut`).
|
||||
- `task`: a draft with the name `change`, a placeholder prompt the export
|
||||
refuses, and suggested checks derived from the net change of the files with
|
||||
role `document`, or of the app's own state files when no document changed.
|
||||
A text file gains `lines_present` for its new lines and `lines_absent` for
|
||||
lines it lost (when either is at most ten), and `line_count` for each line
|
||||
holding a letter whose count changed; JSON and key files gain
|
||||
`value_equals` and `value_absent` by pointer; SQLite tables gain rows added,
|
||||
changed and removed, by primary key, and their new row count. An XML file,
|
||||
and the JSON, key file and XML views of a zip archive's entries, gain checks
|
||||
by [selectors](#pointers-and-selectors) and never through an element by
|
||||
position: in XML, first [`text_equals`](#whole-text) for each element whose
|
||||
text changed, then an element added whole that it exists, one removed whole
|
||||
that it is absent, then values changed, added and removed, an entry or a
|
||||
JSON value added whole through its members, and an empty array or object as
|
||||
it is. The text check names the paragraph rather than a run inside it: it
|
||||
goes up from an element sitting in its parent's text, and from one named by
|
||||
its tag or position alone to the nearest ancestor named by an attribute, as
|
||||
AbiWord's `p` by its `xid`; the text nodes and runs inside it get no other
|
||||
check, but an element holding no text that is added or removed there, such
|
||||
as an image or a line break, keeps its own, since the text check never
|
||||
reads it. Whitespace an app lays out between elements is not checked, and
|
||||
values that look like a time the app stamps, in ISO 8601 or as C's
|
||||
`asctime` writes it (`Thu Nov 5 09:20:31 2026`), are left out.
|
||||
|
||||
The suggestions are a starting point. Keep the ones that say what the user
|
||||
did; drop the ones that only follow from how the app saves its file. A LibreCAD
|
||||
capture that added four holes to a drawing rewrote it from DXF R12 to DXF 2007,
|
||||
so most of its lines changed; the draft suggests counts of `SECTION`, `ENDSEC`
|
||||
and `CIRCLE`, and only `CIRCLE` going from 5 to 9 is the user's change.
|
||||
|
||||
## The task file
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "add-milk",
|
||||
"prompt": "Add buy milk to my to-do list in Mousepad and save it.",
|
||||
"checks": [
|
||||
{
|
||||
"id": "milk",
|
||||
"kind": "lines_present",
|
||||
"file": "Documents/todo.txt",
|
||||
"lines": ["buy milk"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
`name` is a portable bundle name other than `smoke` (lowercase letters, digits
|
||||
and hyphens); `prompt` is what the user asks for; `checks` holds 1 to 32
|
||||
checks. Each check has a `kind`, a `file` given as a path under the home such as
|
||||
`Documents/todo.txt`, and an optional `id` (`check-1`, `check-2` and so on by
|
||||
position otherwise). A check reads the typed view `data/get` returns for that
|
||||
file, in the shapes of the state view.
|
||||
|
||||
| Kind | Fields | Passes when |
|
||||
| --------------- | ------------------------------------------------------------ | ------------------------------------------------------------------------ |
|
||||
| `file_present` | | The file exists |
|
||||
| `file_absent` | | The file does not exist |
|
||||
| `lines_present` | `lines` | Every listed line is a whole line of the text view |
|
||||
| `lines_absent` | `lines` | None of the listed lines is |
|
||||
| `line_count` | `exact` or `contains`; `equals`, `at_least` or `at_most` | The number of lines equal to `exact`, or holding `contains`, compares so |
|
||||
| `value_equals` | `pointer`, `value`; optional `tolerance` | The pointer names a value equal to `value`, or within `tolerance` of it |
|
||||
| `value_absent` | `pointer` | The pointer names no value |
|
||||
| `text_equals` | `pointer`, `value`; optional `normalize_whitespace` | The XML element's [whole text](#whole-text) equals `value` |
|
||||
| `text_contains` | `pointer`, `value`; optional `normalize_whitespace` | The XML element's whole text holds `value` |
|
||||
| `row_present` | `table`, `where` | A row of the SQLite table has every column in `where` equal |
|
||||
| `row_absent` | `table`, `where` | No row does |
|
||||
| `row_count` | `table`; optional `where`; `equals`, `at_least` or `at_most` | The number of rows, or of rows matching `where`, compares so |
|
||||
|
||||
The rules both evaluators follow:
|
||||
|
||||
- An absent file reads as empty: no lines, values or rows. So `lines_absent`,
|
||||
`value_absent` and `row_absent` pass on it and a count is 0.
|
||||
- A file that has no view (binary, over the read limit, or unreadable) fails
|
||||
every check but `file_present` and `file_absent`. Line checks need a text
|
||||
view, row checks a SQLite view and text checks an XML view, in a file or an
|
||||
archive entry; another format fails.
|
||||
- Equality is JSON equality: numbers by value, never a boolean for a number,
|
||||
objects by their keys whatever their order, arrays in order.
|
||||
- Pointers follow RFC 6901 (`~1` for `/`, `~0` for `~`) and must start with
|
||||
`/`; on an array a segment may be a selector, below. Key files read as
|
||||
`{"groups": {"GROUP": {"KEY": "VALUE"}}}`, so a setting is
|
||||
`/groups/GROUP/KEY`; XML as `{"root": {"name", "attributes", "children"}}`;
|
||||
SQLite as `{"tables": {"TABLE": {"columns", "rows"}}}`; a zip archive as
|
||||
`{"entries": [{"name", "format", "view", …}]}`.
|
||||
- With `tolerance`, a non-negative number, `value` must be a number, and the
|
||||
value found passes when it is a number, or a string that is a decimal
|
||||
numeral such as `"5.0000000000000000"`, at most `tolerance` from `value`.
|
||||
`tolerance: 0` compares numerically, so `"1.2500"` equals `1.25`. Anything
|
||||
else fails as not a number.
|
||||
- `contains` matches a line that holds it anywhere, case and all. Checks take
|
||||
no regular expressions: a pattern run on what the agent wrote could stall
|
||||
the verifier, and JavaScript and Python read some patterns differently.
|
||||
- A check decided by what a view leaves out (`lines_absent`, `value_absent`,
|
||||
`row_absent`, the counts, `value_equals` of a whole object or array, and
|
||||
the text checks of an element) fails when that view is truncated; a check
|
||||
satisfied by what the view holds is decided on it.
|
||||
|
||||
The export refuses, naming the check, a file outside the owned paths, a file
|
||||
spelled with a leading slash, `.` or `..`, a pointer into a file that is text
|
||||
at the seed, as the bundle ships it, or at the end checkpoint, the empty
|
||||
pointer (the whole file), a value that names the captured home, or the
|
||||
capture host's name or `user@host` anywhere (which the
|
||||
[export removes from the seed](desktop-application-export.md#host-name-and-user)), a
|
||||
`tolerance` with a value that is not a number, an empty `text_contains`
|
||||
value, or one of only whitespace with `normalize_whitespace`, and a value that
|
||||
looks like a date and time, in ISO 8601 or `asctime`'s form. Other volatile
|
||||
content, such as a
|
||||
counter the app bumps on every save, is for the author to leave out.
|
||||
|
||||
## Pointers and selectors
|
||||
|
||||
An app that rewrites a document moves things: FreeCAD 1.0 records more
|
||||
properties than 0.21, so a hole's radius sits at another position among its
|
||||
siblings, and a plate's entry moves once a hole is added before it. A pointer
|
||||
segment applied to an array may therefore name a member by what it holds
|
||||
rather than by its position:
|
||||
|
||||
- `[KEY=VALUE]` names the member whose string field `KEY` is `VALUE`, such as
|
||||
a zip entry by `[name=Document.xml]` or an XML element by its tag,
|
||||
`[name=Object]`.
|
||||
- `[@KEY=VALUE]` names the member whose `attributes` hold `KEY="VALUE"`, as
|
||||
the XML view keeps an element's attributes.
|
||||
- Conditions can follow one another, `[name=Object][@name=Hole]`, and all must
|
||||
hold. They compare strings only, the selector is escaped like any pointer
|
||||
segment (`~1` for `/`), and a value cannot hold `][`.
|
||||
|
||||
A selector must name exactly one member. When several match, or none does
|
||||
before the last segment, the check fails and says which selector matched what.
|
||||
When the last segment's selector matches none, the pointer names no value:
|
||||
`value_absent` passes, which is how a check says an element was removed, and
|
||||
`value_equals` fails. On an object, a bracketed segment is an ordinary key.
|
||||
|
||||
A FreeCAD task that checks the user cut a 10 mm hole through the plate, in
|
||||
FreeCAD 0.21 and 1.0 alike:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "cut-hole",
|
||||
"prompt": "Cut a 10 mm hole through the middle of the plate and save it.",
|
||||
"checks": [
|
||||
{
|
||||
"id": "cut",
|
||||
"kind": "value_equals",
|
||||
"file": "Documents/plate.FCStd",
|
||||
"pointer": "/entries/[name=Document.xml]/view/root/children/[name=Objects]/children/[name=Object][@type=Part::Cut]/attributes/type",
|
||||
"value": "Part::Cut"
|
||||
},
|
||||
{
|
||||
"id": "radius",
|
||||
"kind": "value_equals",
|
||||
"file": "Documents/plate.FCStd",
|
||||
"pointer": "/entries/[name=Document.xml]/view/root/children/[name=ObjectData]/children/[name=Object][@name=Hole]/children/[name=Properties]/children/[name=Property][@name=Radius]/children/[name=Float]/attributes/value",
|
||||
"value": 5,
|
||||
"tolerance": 0.01
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Before the edit the archive has no `Part::Cut` and no `Hole`, so both checks
|
||||
fail on the seed; after it, `Radius` is `"5.0000000000000000"` and both pass.
|
||||
A check decided by what a view leaves out depends, in an archive, on the entry
|
||||
its pointer goes into: it fails when that entry's view is missing or
|
||||
truncated, or when the entry is not listed and the listing is truncated. A
|
||||
selector must see every member to know it names one, so a check whose pointer
|
||||
searches an array the view may hold only part of fails as undecided: any array
|
||||
of a truncated XML view, of a truncated entry's view, or the listing when it
|
||||
is cut. A position is decided on what the view holds. Like a file, an entry
|
||||
that has no view (binary, not read, or a database or archive inside the
|
||||
archive) fails every check into its `view`; its `null` there is no value, while
|
||||
an entry holding the JSON document `null` reads as `null`. Checks on its `name`,
|
||||
`size`, `sha256` and `format` still read.
|
||||
|
||||
XML that uses namespaces is named as the document spells it, in both
|
||||
evaluators: an element or attribute by its prefixed name, such as `text:p`
|
||||
or `inkscape:label`, an element in a default namespace by its bare name, such
|
||||
as `svg`, and namespace declarations as the attributes `xmlns` and
|
||||
`xmlns:PREFIX`. Text is a string among an element's `children`. These checks
|
||||
hold a poster's headline in an Inkscape SVG and a date in a LibreOffice Writer
|
||||
document:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"id": "headline",
|
||||
"kind": "value_equals",
|
||||
"file": "Pictures/poster.svg",
|
||||
"pointer": "/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0",
|
||||
"value": "Summer Bake Sale"
|
||||
},
|
||||
{
|
||||
"id": "date",
|
||||
"kind": "value_equals",
|
||||
"file": "Documents/flyer.odt",
|
||||
"pointer": "/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Subtitle]/children/0",
|
||||
"value": "Monday 24 November"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
Inkscape writes a line of text as a `tspan` inside its `text` element, and
|
||||
Writer keeps a paragraph's style in `text:style-name`. A prefix is the one the
|
||||
document uses, so a check names the prefixes the app writes.
|
||||
|
||||
## Whole text
|
||||
|
||||
A text node is only the stretch of text between two pieces of markup, and
|
||||
apps split a paragraph into runs as they save it. AbiWord wraps text typed
|
||||
into a reopened document in `c` elements, so changing a time to `3:30` saves
|
||||
`<p>The team meeting starts at <c>3</c>:<c>30</c> in room 4.</p>`; Writer,
|
||||
which records revision IDs by default, writes text typed in one session as
|
||||
`text:span` runs. A `value_equals` on the paragraph's `children/0` then reads
|
||||
`"The team meeting starts at "` and fails. `text_equals` and `text_contains`
|
||||
read the element's whole text instead:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "time",
|
||||
"kind": "text_equals",
|
||||
"file": "Documents/meeting.abw",
|
||||
"pointer": "/root/children/[name=section]/children/[name=p][@xid=3]",
|
||||
"value": "The team meeting starts at 3:30 in room 4."
|
||||
}
|
||||
```
|
||||
|
||||
- The pointer names an element of an XML view, in a file or in an archive
|
||||
entry's `view`, or a text node or attribute value in one, whose text is its
|
||||
value. Another format, a pointer into an archive outside an XML entry's
|
||||
view, or any other value fails, an element's `name` and the `doctype`'s
|
||||
fields included: they are strings of the view, not text of the document.
|
||||
- An element's text is XPath's `string()` of what the view keeps: every text
|
||||
node in it and in its descendants, CDATA included, in document order, with
|
||||
nothing between them. The view drops whitespace-only text between child
|
||||
elements only in an element that holds no other text, outside
|
||||
`xml:space="preserve"`, so the space between two runs of a paragraph is
|
||||
part of its text. Elements that stand for characters, such as Writer's
|
||||
`text:s`, `text:tab` and `text:line-break`, add none.
|
||||
- `text_equals` passes when the text equals `value` exactly, case and all;
|
||||
`text_contains` when it holds `value`, as a time within a sentence, so the
|
||||
check holds whatever the rest of the sentence says.
|
||||
- `normalize_whitespace: true` collapses each run of XML whitespace (space,
|
||||
tab, carriage return, line feed) in both the text and `value` to one space
|
||||
and trims both ends before comparing, as XPath's `normalize-space()` does,
|
||||
for an app that reflows text or a value written across lines. A no-break
|
||||
space is not whitespace there.
|
||||
|
||||
## Validation and export
|
||||
|
||||
`rea export-desktop-application <capture> <directory> --task <task.json>`
|
||||
evaluates the checks on the capture's state view: on each file as it was at
|
||||
the seed checkpoint, which is the seed, as the bundle ships it with the
|
||||
capture host's name and user removed, and at the end checkpoint, the
|
||||
captured end state. Those are the capture's first and last checkpoints unless
|
||||
`--seed-checkpoint` and `--end-checkpoint` name others.
|
||||
It refuses a task whose every check passes on the seed, and a check that fails
|
||||
on the end state, with that check's reason. Only a failure on what the seed's
|
||||
view shows counts: a task whose checks fail on the seed only because a view
|
||||
could not be read or may have left out what they look for, such as a line past
|
||||
the 5,000 a text view keeps, is refused, naming each failure. A capture yields
|
||||
one task, so the option is taken once. The result's `task` names the task, its
|
||||
number of checks and those that fail on what the seed shows. A refused task
|
||||
writes no bundle.
|
||||
|
||||
The bundle then holds, besides the app and `tasks/smoke.json`:
|
||||
|
||||
| Path | Contents |
|
||||
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| `tasks/<name>.json` | Deploy the app, deploy the agent and its skills, record the screen, prompt the agent, run the verifier, stop recording |
|
||||
| `artifacts/<name>-check/verify.py` | Reads the app's `data/get` through the computer and evaluates the checks, one result per check |
|
||||
| `evals/<env>.toml` | The task, so a run of the bundle runs it; the smoke task runs with `--task smoke` |
|
||||
|
||||
The verifier finds the app among the computer's child environments, calls its
|
||||
`data/get` under the run's grant of the computer's control token, and reads
|
||||
the `state` data part. It raises when the app cannot be reached, so the run
|
||||
fails rather than scoring the agent. The README lists the task, its prompt, its
|
||||
checks and the recipe below.
|
||||
|
||||
## Solve rate
|
||||
|
||||
REA does not run agents. To measure how often an agent solves the task, run it
|
||||
k times, three by default, and count the runs whose summary line says
|
||||
`passed`:
|
||||
|
||||
```sh
|
||||
for run in 1 2 3; do
|
||||
agent-env run ./mousepad-bundle --task add-milk --agent claude-code --model MODEL
|
||||
done
|
||||
```
|
||||
|
||||
The agent defaults to Claude Code and the model is the caller's choice (Claude
|
||||
Sonnet 5.5 is a reasonable reference). A run reported as `scored below 1`
|
||||
failed the checks and counts as a miss; one reported as `failed` did not finish,
|
||||
as when the computer or the app could not be reached, and is run again rather
|
||||
than counted. Each run's instance keeps the verifier's results per check under
|
||||
`verifications`.
|
||||
@@ -110,15 +110,6 @@ not establish full metadata validity or correspondence to a native binary;
|
||||
version 24 inspection still applies only to supported header layouts. See
|
||||
[IL2CPP metadata](il2cpp-metadata.md).
|
||||
|
||||
In a managed directory with a Unity Mono witness, `Assembly-CSharp.dll` and
|
||||
`Assembly-CSharp-firstpass.dll` get an `inspect_managed_members` probe listed
|
||||
first; project-named asmdef assemblies keep only the ordinary managed probes.
|
||||
The same probe is offered for those names in `Managed/` of a
|
||||
`<Product>_BackUpThisFolder_ButDontShipItWithYourGame` folder (or when that
|
||||
folder is the root); whether its IL matches the shipped IL2CPP code is unknown.
|
||||
Point the scan at the game folder: a root that is the `<Product>_Data` or
|
||||
`Managed` folder itself has no Mono witness path and gets no member probe.
|
||||
|
||||
Candidates are derived or inferred from these facts and do not prove loading or
|
||||
execution. BepInEx, MelonLoader, Easy Anti-Cheat and BattlEye labels identify
|
||||
path conventions only; runtime presence stays unknown. No bypass is provided.
|
||||
@@ -144,13 +135,11 @@ observations, not an atomic directory snapshot. Unread bytes remain unknown.
|
||||
## Resource bounds and reuse
|
||||
|
||||
Defaults are 20,000 examined entries and 8 MiB of total probe reads. Callers may
|
||||
select up to 100,000 entries and 64 MiB. A separate 32 MiB representation
|
||||
budget bounds retained rows, paths and facts before growth. Each retained path
|
||||
is charged at the larger of its UTF-16 heap size and its escaped JSON size, so
|
||||
typical macOS bundle paths fit about 20,000 rows, and control-character paths
|
||||
fewer. Limits and failed probes produce partial coverage. Traversal and reads
|
||||
are serial. Each reader admits one inspection at a time; overlapping requests
|
||||
receive a retryable capacity refusal without cancelling the admitted caller.
|
||||
select up to 100,000 entries and 64 MiB. A separate 32 MiB conservative
|
||||
representation budget bounds retained paths and facts before growth. Limits
|
||||
and failed probes produce partial coverage. Traversal and reads are serial.
|
||||
Each reader admits one inspection at a time; overlapping requests receive a
|
||||
retryable capacity refusal without cancelling the admitted caller.
|
||||
|
||||
Unity reference decoding stops after a witness in each `Data/Managed` directory;
|
||||
other assemblies' references remain unexamined, while their headers are still
|
||||
|
||||
@@ -1,199 +0,0 @@
|
||||
# Desktop build computer
|
||||
|
||||
A build computer is a local agentenv computer that an agent drives while it
|
||||
builds a desktop app environment. Claude Code cannot attach a new MCP server
|
||||
in the middle of a session, so REA keeps the computer up and forwards the
|
||||
agent's computer-use actions to it through three MCP tools:
|
||||
|
||||
| Tool | What it does |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------ |
|
||||
| `start_build_computer` | Starts a computer from a request or a bundle folder; returns its id and a first screenshot |
|
||||
| `act_on_build_computer` | Sends one action to the computer's driver and returns the driver's answer, images included |
|
||||
| `close_build_computer` | Tears the computer down, by agentenv's teardown or by removing its sandbox's containers |
|
||||
|
||||
The tools are MCP-only, like the debugger sessions: the computer lives as long
|
||||
as the REA MCP server that started it, or until it is closed. Capturing the
|
||||
app, exporting its bundle and witnessing the task on such a computer are in
|
||||
[Building and witnessing an environment](desktop-build-witness.md).
|
||||
|
||||
## Requirements
|
||||
|
||||
- Docker, and an agentenv-framework install whose `agent-env` program is named
|
||||
by the absolute `REA_AGENT_ENV_COMMAND`. Bundles target agentenv
|
||||
0.9.1431. REA runs its helper, `bridge/agentenv/computer.py`, with that
|
||||
runner's Python: the interpreter on the program's `#!` line, or the
|
||||
`python3` beside it in its virtual environment.
|
||||
- `REA_DOCKER_COMMAND`, the absolute path of the Docker CLI, lets REA remove a
|
||||
computer whose agentenv teardown was not confirmed. Without it, REA names the
|
||||
command to run instead.
|
||||
- For a request, `REA_BUILD_COMPUTER_REA_ARCHIVE`: the absolute path of a tar
|
||||
archive of a runnable REA, with its production dependencies, whose one top
|
||||
folder holds `scripts/rea.mjs`. The computer runs REA from it. From a
|
||||
checkout, `npm run pack:build-computer-runtime -- /path/to/rea-prod.tar.xz`
|
||||
writes one: `npm pack`, its package folder as
|
||||
`rea-prod` with the lockfile, `npm ci --omit=dev --omit=optional`, then
|
||||
`rea-prod` as `.tar.xz`.
|
||||
|
||||
Set these where the MCP server starts; a shell export does not reach a server
|
||||
that is already running.
|
||||
|
||||
## Start a computer
|
||||
|
||||
From a request, REA renders a bundle in a private temporary folder: a computer
|
||||
app on the `ubuntu-24.04` base with the `reference` driver that installs the
|
||||
app, opens it on demand with `gtk-launch <desktop entry>` from the computer
|
||||
user's home, so its open and save dialogs start there rather than in the
|
||||
framework's folder for the app, and keeps no seed and no data hooks. The app is not open when the computer starts: agentenv lists it
|
||||
in the desktop's Applications menu under Other by the environment's name, such
|
||||
as `mousepad`, and that entry launches it through the app's server.
|
||||
|
||||
When the app is Electron, set `electron` to true. The install rewrites that
|
||||
desktop entry's command with the same policy an export uses: `--no-sandbox`
|
||||
immediately after the program, unless the command already has it. The
|
||||
computer's container does not grant the user namespaces Chromium's sandbox
|
||||
needs, and Electron exits without the flag. With the rewritten command, the
|
||||
Applications menu entry can open the app.
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "request",
|
||||
"desktop_entry": "org.xfce.mousepad",
|
||||
"window_class": "Mousepad",
|
||||
"install": { "kind": "archive", "package": "mousepad" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`install` takes one of:
|
||||
|
||||
| `kind` | Installs |
|
||||
| ---------------- | ---------------------------------------------------------------------------------------------------------- |
|
||||
| `archive` | `package` from the Ubuntu 24.04 archive, on amd64 and arm64 |
|
||||
| `apt_repository` | `package` from the repository at `uri` with `suite` and `components`, trusting the key file `key_file` |
|
||||
| `package_file` | The `.deb` at `url`, checked against `sha256`, for one `architecture`, after its apt `dependencies` are in |
|
||||
|
||||
Build steps run with the network off, so a `.deb`'s dependencies must be
|
||||
listed. The bundle adds REA in a build-only layer: Node.js
|
||||
24.22.1, downloaded per architecture and checked against its release checksum,
|
||||
and REA from `REA_BUILD_COMPUTER_REA_ARCHIVE`, both unpacked into `/opt/rea` by
|
||||
a root build step, with `node` and `rea` on `PATH`. An exported bundle carries
|
||||
none of it.
|
||||
|
||||
From a bundle folder, such as one `export-desktop-application` wrote, REA
|
||||
starts that bundle's `tasks/smoke.json` as it is:
|
||||
|
||||
```json
|
||||
{ "source": { "kind": "bundle", "path": "/home/ada/mousepad-bundle" } }
|
||||
```
|
||||
|
||||
REA runs its helper with `PYTHONDONTWRITEBYTECODE=1`, which agentenv's own
|
||||
import of the bundle's app class inherits, so a start leaves no `__pycache__`
|
||||
in the folder and its files stay as they were shared.
|
||||
|
||||
REA runs the deploy with `run_bundle(keep=True, emulate=False)` on the local
|
||||
sandbox provider, so only the host's own architecture is built. The first start
|
||||
of a bundle builds its image and can take minutes; later starts reuse it.
|
||||
While it waits, REA sends an MCP progress notification every five seconds with
|
||||
agentenv's latest line, so a client that resets its request timeout on
|
||||
progress keeps waiting. REA then waits until the computer's readiness checks
|
||||
pass and opens one MCP session to its driver. Readiness can pass before the
|
||||
desktop has drawn, so REA takes a whole-screen PNG screenshot every 500 ms
|
||||
while it is one colour throughout. The retries share 10 s, each given only
|
||||
what is left of it, and when that runs out REA returns the latest frame;
|
||||
progress notifications continue meanwhile. Given `screenshot` with a
|
||||
`region`, `format` or `quality`, REA then takes one with them, a `region`
|
||||
cropping that frame. The result holds
|
||||
`computer_id`, agentenv's `sandbox_id`, the environment name or bundle folder,
|
||||
and the screenshot's frame as structured content beside its image.
|
||||
|
||||
A failed start returns the last 16 KiB of the helper's stderr, where a failing
|
||||
build prints its error, and its exit status; a cancelled one stops the helper,
|
||||
which tears down whatever agentenv brought up. When that teardown is not
|
||||
confirmed, the error's `computer_id` names what REA keeps for
|
||||
`close_build_computer` to retry. Closing the server cancels starts still in
|
||||
progress and waits for them before closing every computer.
|
||||
|
||||
## Act
|
||||
|
||||
`act_on_build_computer` takes `computer_id`, an `action` and the driver's own
|
||||
arguments for `computer_<action>`, which REA passes through:
|
||||
|
||||
| `action` | Arguments |
|
||||
| ------------ | --------------------------------------------------------------- |
|
||||
| `screenshot` | `region`, `format` (`png` or `jpeg`), `quality` |
|
||||
| `click` | `x`, `y`, `button`, `count` (2 double-clicks), `modifiers` |
|
||||
| `drag` | `path` of `[x, y]` points, `button`, `duration_ms`, `modifiers` |
|
||||
| `scroll` | `direction`, `x`, `y`, `amount`, `modifiers` |
|
||||
| `type` | `text`, where `\n` presses Enter and `\t` Tab |
|
||||
| `key` | `keys`, one chord pressed in order, and `repeat` |
|
||||
| `wait` | `ms` |
|
||||
| `reset` | `screenshot` |
|
||||
|
||||
`reset` is not a driver tool: REA calls the app's `data/reset`, which restores
|
||||
the bundle's seed and relaunches the app, then waits until the computer's
|
||||
readiness checks pass again, sending an MCP progress notification every five
|
||||
seconds while it waits. With `screenshot: true` it then takes a screenshot; if
|
||||
that fails, the reset still counts as applied and its result has no `frame`
|
||||
but a `screenshot_error` with the computer's message. A computer started from
|
||||
a request has no data operations, so it refuses a reset.
|
||||
|
||||
Pointer actions also take `frame_id`, `space` and `norm_scale`, and every
|
||||
action but `screenshot` takes `screenshot: true` for an image afterwards. That
|
||||
image is taken at once, so after an action that opens a menu or a window, ask
|
||||
for it with a `wait` of a few hundred milliseconds instead.
|
||||
Points are pixels of the latest frame, so take a screenshot first; with
|
||||
`space: "norm"` they are on a `0..norm_scale` grid over it. Keys are W3C names
|
||||
or one character; REA translates model spellings such as `ctrl+s`, `Return`,
|
||||
`esc` and `cmd` and leaves other names for the driver to judge. An argument the
|
||||
action does not take is refused before anything is sent.
|
||||
|
||||
The result carries the driver's content unchanged, its text and images, after
|
||||
REA's own JSON: `action_index` and the driver's structured result. A refusal,
|
||||
such as a stale frame, is an error carrying the computer's message. A response
|
||||
larger than the MCP budget is refused with how to ask for a smaller image: a
|
||||
`region`, or `jpeg` with a lower `quality`. That error keeps REA's own result,
|
||||
`computer_id` included, as `details.result`, since the action or start has
|
||||
happened.
|
||||
|
||||
Actions, the capture steps and the task checks on one computer run one at a
|
||||
time, in the order REA receives them; one that has to wait for the call
|
||||
before it sends an MCP progress notification every five seconds meanwhile.
|
||||
Different computers run side by side.
|
||||
|
||||
Every action is logged on the computer in order with the arguments sent, its
|
||||
start and end and whether it was applied, refused or failed. The log keeps the
|
||||
latest 256 actions within 256 KiB of JSON, dropping the oldest first. Each
|
||||
string argument, such as `type`'s `text`, is kept to its first 1 KiB and each
|
||||
list, such as `drag`'s `path`, to its first 64 items; an entry names the
|
||||
arguments it cut in `truncated_arguments`, and the driver gets them whole. The
|
||||
log counts every action, so it says how many it dropped, and keeps the index of
|
||||
the last applied action of each kind for rules over the whole session. The log
|
||||
and the computer's recording hooks (`computer-recording/v1` start and end under
|
||||
REA's own request id, and the segments fetched through the driver's `data/get`
|
||||
one at a time, each read within its manifest size and checked against it) are
|
||||
what the witness records; they are not tools of their own. A recording larger
|
||||
than a quarter of the V8 heap is refused before any segment is read.
|
||||
|
||||
## Close
|
||||
|
||||
`close_build_computer` ends REA's MCP session and closes the helper's stdin,
|
||||
on which the helper runs agentenv's teardown. When agentenv confirms nothing is
|
||||
left, the result says `teardown: "agentenv"`. Otherwise REA removes the
|
||||
containers of that exact sandbox: agentenv runs a computer as
|
||||
`agent-<sandbox id>` without a label and labels its other containers
|
||||
`agentenv.sandbox=<sandbox id>`, so REA lists both with `docker ps -aq
|
||||
--filter`, removes them with `docker rm -f`, and says
|
||||
`teardown: "container_label"`. When neither confirms it, the error names what
|
||||
is left and the computer stays listed, so closing it again retries. A close
|
||||
cancels the call in progress on the computer, refuses the calls waiting for
|
||||
it, and tears down once that call has ended. Closing the
|
||||
MCP server closes every computer the same way, and a helper whose REA server
|
||||
exits sees its stdin close and tears down.
|
||||
|
||||
The control token that guards the computer's framework-only routes stays in
|
||||
REA: no result, error or log carries it. Code running in the computer as its
|
||||
user can still read it, which is acceptable for a local build.
|
||||
|
||||
See the [specification](specs/desktop-app-environments/build-computer/spec.md)
|
||||
for scope and verification.
|
||||
@@ -1,175 +0,0 @@
|
||||
# Building and witnessing an environment
|
||||
|
||||
An agent such as Claude Code builds a desktop app environment from a request
|
||||
on a [build computer](desktop-build-computer.md): it makes the seed through
|
||||
the app, makes the change the task asks for, exports the bundle with the task
|
||||
it writes, and then witnesses the task in a fresh computer of that bundle.
|
||||
REA runs its own capture, task draft and export inside the build computer and
|
||||
grades the witness with the same checks the exported verifier runs.
|
||||
|
||||
| Tool | What it does |
|
||||
| -------------------------- | ------------------------------------------------------------------------------------------- |
|
||||
| `start_build_capture` | Downloads the package lists as the computer's user and launches the app under REA's capture |
|
||||
| `mark_build_checkpoint` | Takes a named checkpoint and returns the capture's acknowledgement with its counts |
|
||||
| `stop_build_capture` | Stops the capture and returns its checkpoints and the task draft between `seed` and `end` |
|
||||
| `export_build_environment` | Exports the bundle with the agent's task inside the computer and copies it to a folder here |
|
||||
| `check_build_task` | Grades a task against the app's live state; with a `stage`, files the check in the witness |
|
||||
|
||||
Like the computer tools, these are MCP-only and act on a `computer_id` from
|
||||
`start_build_computer`.
|
||||
|
||||
## Build
|
||||
|
||||
1. `start_build_computer` from a request. REA is installed in the computer.
|
||||
2. `start_build_capture`. REA writes an apt configuration under
|
||||
`~/.rea-build/apt` and runs `apt-get update` as the computer's user, so the
|
||||
capture finds the package's sources (`package_lists` counts the lists,
|
||||
and `package_lists_error` gives apt's exit status and last lines when the
|
||||
download failed; a later start runs the update again until one succeeds). It then starts `rea capture-desktop-application
|
||||
--desktop-entry <entry>`
|
||||
in the computer, leaving `~/.rea-build` out of every checkpoint, and returns
|
||||
once the app's window is open, with a screenshot. For an app installed
|
||||
from a `.deb` file, the build layer keeps that file at
|
||||
`/opt/rea/package.deb` and the capture records it with `--package-file`. Do not open the app from
|
||||
the menu first: the capture refuses an app whose window is already open.
|
||||
3. Make the seed through the app with `act_on_build_computer`, saving it
|
||||
where the user keeps such files, such as `~/Documents`, then
|
||||
`mark_build_checkpoint` `seed`.
|
||||
4. Make the change the task will ask for, save it, and
|
||||
`mark_build_checkpoint` `end`.
|
||||
5. `stop_build_capture`. The app is closed and the capture finishes. The
|
||||
result lists the checkpoints with their counts, the package and the
|
||||
sources the capture found, its limitations and the draft: each file that
|
||||
changed between `seed` and `end` with its net change, and suggested
|
||||
checks with a placeholder prompt.
|
||||
6. Write the task: its name, the prompt in the user's own words, and checks
|
||||
that hold after the change and not on the seed.
|
||||
7. `export_build_environment` with the task and a new or empty `directory`.
|
||||
The export runs inside the computer with `--seed-checkpoint seed
|
||||
--end-checkpoint end`, so it seeds what was saved at `seed`, scrubs the
|
||||
computer's host name and user from it, and refuses a task whose checks
|
||||
all pass on the seed. A task larger than 96 KiB as REA passes it to the
|
||||
computer is refused before anything runs. A file moved to the desktop Trash is not seeded,
|
||||
and the export's limitations name what the Trash held at `seed`. For a
|
||||
`.deb` app that no apt source the capture found offers, the export takes
|
||||
the request's URL with `--package-url`; one apt offers installs from apt. REA lists the bundle's files with their sizes
|
||||
and SHA-256 in the computer, into a file beside the bundle that it reads
|
||||
through `data/get`, since a large bundle's list outgrows a command's
|
||||
output; it then brings the files back through the driver's
|
||||
`data/get` in answers of at most 8 MiB, and checks and writes each answer
|
||||
into the folder before asking for the next, so one answer at a time is
|
||||
held; a failure leaves no partial folder.
|
||||
8. `close_build_computer`.
|
||||
|
||||
A checkpoint name is lowercase letters, digits, dot, dash and underscore; the
|
||||
capture's own `baseline`, `stop`, `time-limit` and `exit` are refused, and so
|
||||
is a name already taken. The acknowledgement comes from the capture after it
|
||||
has walked the home and retained what changed: `entries` walked, `changes`
|
||||
since the checkpoint before, `retained_files` and `retained_bytes`.
|
||||
|
||||
Each step can be asked again after its answer was lost:
|
||||
|
||||
- A start whose answer is lost, or whose window never opened, stops the
|
||||
capture it may have left running. Until the computer confirms that stop,
|
||||
REA keeps the capture and the next `start_build_capture` retries the stop
|
||||
before starting another.
|
||||
- A checkpoint the capture took whose acknowledgement was lost comes back
|
||||
from the capture's own log when the same name is marked again.
|
||||
- A stop whose summary or draft was not read finishes when stop is asked
|
||||
again, and a finished stop repeats its result. The draft and the export
|
||||
trust the checkpoints the capture's own Evidence records.
|
||||
- A step that saved something, such as an export's bundle or the witness's
|
||||
record, returns it even when its call is cancelled or the computer closes
|
||||
as the save finishes. `start_build_capture` saves nothing, so a close or
|
||||
cancellation during its screenshot of the opened app cancels it.
|
||||
|
||||
The capture, the draft and the export run as the computer's user in its home,
|
||||
without any `AGENTENV_*` variable, so neither REA nor the app holds the
|
||||
computer's control token. REA sends each command through agentenv's
|
||||
`computer-exec/v1` route with that token, which stays in REA, and reads larger
|
||||
outputs, such as the draft, the export's Evidence and the bundle's file
|
||||
list, from files under `~/.rea-build` through `data/get`, since a command's
|
||||
output is capped at 1 MiB.
|
||||
|
||||
Calls on one computer run one at a time, in the order REA receives them:
|
||||
an action, a capture step, a check or a close waits until the call before
|
||||
it on that computer has answered, and sends progress notifications while
|
||||
it waits. So a check reads the app with no action half done, and a witness
|
||||
stage is admitted, read and filed before anything else on that computer
|
||||
runs. Calls on different computers run side by side. Closing a computer
|
||||
cancels the call in progress on it and refuses the calls still waiting.
|
||||
|
||||
## Witness
|
||||
|
||||
Start a fresh computer from the exported folder with `start_build_computer`
|
||||
and `{"kind": "bundle", "path": ...}`; agentenv boots the app with the seed.
|
||||
Then, with the task exactly as exported:
|
||||
|
||||
| `stage` | Before it | Its checks must |
|
||||
| ------------- | ----------------------------------------------------------------- | ---------------------------- |
|
||||
| `seed` | nothing; give `directory`, a new or empty folder for the record | fail at least once |
|
||||
| `after_edit` | the task done through the app from its prompt alone | all pass |
|
||||
| `near_miss` | a change that is almost right, such as a misspelled line | fail at least once |
|
||||
| `after_reset` | `act_on_build_computer` with `action: "reset"` as the last change | each come out as on the seed |
|
||||
|
||||
`check_build_task` reads the app's `data/get`, takes the views it returns
|
||||
beside the files and evaluates each check with REA's evaluator, the one the
|
||||
export validates tasks with and the exported verifier mirrors, one result per
|
||||
check. Without `stage` it only grades. The `seed` stage begins a recording
|
||||
of the display under REA's own request id, one per computer; a seed check
|
||||
that fails after it, such as on a failed read of the app's state or a lost
|
||||
reply to the recording's start, can be asked again, and asks for the
|
||||
recording under the same id, which the computer answers with the recording
|
||||
it already began. The stages come in order. The edit and the near miss each
|
||||
need an applied click, drag, scroll, type or key since the stage before, with
|
||||
no reset after the last of them, since a reset puts the seed back rather than
|
||||
changing the app; `after_reset` needs the reset as the last change, and a
|
||||
reset that leaves any check unlike on the seed fails the witness, its
|
||||
stage's `remediation` naming those checks. On a computer started from a
|
||||
bundle folder, the seed refuses a task whose `tasks/<name>.json` or verifier
|
||||
in the bundle differs from what the export writes for it. A reset
|
||||
relaunches the app, and REA waits until the computer is ready again before
|
||||
answering, sending progress notifications while it waits.
|
||||
|
||||
Only a check that fails on what the app's view shows counts as a failure. A
|
||||
check that fails because its file or table could not be read, such as an
|
||||
SQLite file locked past the reader's wait, or because what it looks for may
|
||||
lie in what its view cut, is `undecided` in its result: it shows nothing
|
||||
about the task. That covers a line `lines_present` misses in a text view cut
|
||||
at 5,000 lines, a row `row_present` misses in a table whose view keeps only
|
||||
its first rows, a `value_equals` pointer that names nothing in a truncated
|
||||
view or past a cut zip listing, and the absence checks and counts on a cut
|
||||
view. A check that passes on what the view kept stays decided, and so does a
|
||||
value found there that differs from the one expected. The result's `passed`
|
||||
and description are the exported verifier's, which has no undecided results.
|
||||
A stage that only undecided checks would settle, such as a near miss whose
|
||||
one failing check could not read its file, files nothing and is refused;
|
||||
check it again once the files read whole.
|
||||
|
||||
A stage whose checks do not show what it must ends the witness, and so does
|
||||
`after_reset`. REA then ends the recording, fetches its segments through
|
||||
`data/get`, each checked against the recording's manifest, and writes into
|
||||
`directory`:
|
||||
|
||||
- `recording/NNNN.mp4`, the recording's segments;
|
||||
- `witness.json`, REA Evidence (`rea.desktop-build-witness`) holding the
|
||||
verdict, the stage that failed if one did, the task's name and SHA-256,
|
||||
each stage's results with how many logged actions came before it, the
|
||||
computer's [action log](desktop-build-computer.md#act) as it keeps it, each
|
||||
action with its arguments, timing and outcome, with `actions_dropped`
|
||||
counting the earliest actions it no longer keeps, and the recording's
|
||||
segments with their sizes and SHA-256. The record's limitations say when
|
||||
the log dropped actions or cut arguments, which each entry names in
|
||||
`truncated_arguments`.
|
||||
|
||||
The stages' rules read the log's count of actions and the last applied
|
||||
action of each kind, which it keeps for the whole session, so they hold
|
||||
however many entries it dropped. The result returns the same record. When writing it fails, such as for a
|
||||
folder something else wrote into, the witness keeps its stages: check the
|
||||
stage that ended it again, with the same task, and REA writes the record
|
||||
without checking again. Once it is saved, the same check returns the saved
|
||||
record again, so a record saved as its call was cancelled is not lost. A
|
||||
computer is witnessed once; witness again from a fresh computer.
|
||||
|
||||
See the [specification](specs/desktop-app-environments/build-witness/spec.md)
|
||||
for scope and verification.
|
||||
@@ -1,82 +0,0 @@
|
||||
# Documentation ownership and authoring
|
||||
|
||||
Use the [documentation directory](../README.md) to find existing material before
|
||||
adding another guide. Preserve one canonical owner for each contract or workflow;
|
||||
link to that owner instead of copying its claims into several guides.
|
||||
|
||||
## Choose a destination
|
||||
|
||||
| Document | Destination |
|
||||
| -------------------------------------------------------- | -------------------------------------- |
|
||||
| Installation, client setup and first CLI/MCP steps | `docs/getting-started/` |
|
||||
| Analyst workflows and evidence interpretation | `docs/guides/<domain>/` |
|
||||
| Analysis-engine configuration and format prerequisites | `docs/providers/` |
|
||||
| Shared runtime contracts and evidence semantics | `docs/reference/` |
|
||||
| Development, testing, release and authoring instructions | `docs/development/` |
|
||||
| Composition diagrams | `docs/architecture/` |
|
||||
| Compatibility changes for a released version | `docs/migrations/` |
|
||||
| One feature specification, plan and tasks | `docs/specs/<domain>/<feature-slug>/` |
|
||||
| One architectural decision | `docs/adr/<domain>/<decision-slug>.md` |
|
||||
|
||||
Use meaningful, lowercase domain and feature slugs. Global numeric prefixes are
|
||||
unnecessary: different domains can add features independently, and branches do
|
||||
not compete for the next sequence number. Check the existing domain directory
|
||||
and linked issues before creating a feature with the same slug.
|
||||
|
||||
Guides describe existing behavior and limitations. Specs define one proposed
|
||||
feature or domain change; an existing guide does not become a specification by
|
||||
moving it into `specs/`.
|
||||
|
||||
## Feature lifecycle
|
||||
|
||||
Create the specification before implementation. Each feature directory contains:
|
||||
|
||||
- `spec.md`: problem, desired behavior, requirements, acceptance and non-goals.
|
||||
- `plan.md`: the single implementation plan, contracts and architectural choices.
|
||||
- `tasks.md`: incremental, testable steps linked to requirements and verification.
|
||||
|
||||
Split independent domains or separately reviewable features before writing their
|
||||
specs. A broad `ROADMAP.md`, when useful, only lists child specifications, order
|
||||
and dependencies; requirements and implementation tasks belong to each child.
|
||||
Cross-repository contracts have one producer-owned specification. Consumers link
|
||||
to that contract instead of maintaining competing copies.
|
||||
|
||||
Record an isolated architectural decision separately in
|
||||
`docs/adr/<domain>/<decision-slug>.md`, with **Status**, **Context**,
|
||||
**Decision** and **Consequences**. Link the ADR from affected specifications.
|
||||
Do not create decision records for routine file moves or invent past decisions.
|
||||
|
||||
Specs do not replace the contribution-scope decision required for a new provider
|
||||
or expanded execution authority. Follow
|
||||
[contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
|
||||
|
||||
## Links and public routes
|
||||
|
||||
Use relative links to physical Markdown files so GitHub readers can follow them.
|
||||
Link repository files outside `docs/` with their full GitHub URLs. Preserve
|
||||
existing heading anchors when moving a guide.
|
||||
|
||||
[`documentation.ts`](https://github.com/morluto/rea/blob/main/docs/.vitepress/documentation.ts)
|
||||
owns categorized site navigation and the existing guides' stable public routes.
|
||||
Update it when adding or moving a guide. VitePress rewrites categorized sources
|
||||
to the existing public paths; its Markdown hook resolves repository links from
|
||||
the physical source and renders them against those public routes. Keep basenames
|
||||
and route assignments unique. Edit links use the physical source file.
|
||||
|
||||
For a move, search for both the basename and its old full path. Update README
|
||||
translations, authored skills, verification instructions, documentation-fact
|
||||
readers and release configuration when they consume that path. Preserve
|
||||
historical release records and references bound to earlier commits or tags.
|
||||
|
||||
## Generated ownership and checks
|
||||
|
||||
The [documentation directory](../README.md#machine-owned-documentation) identifies
|
||||
the machine-owned outputs and their producers. Their existing paths are retained
|
||||
because scripts, package consumers and release history depend on them. Change
|
||||
canonical sources and regenerate; do not fix a generated projection by hand.
|
||||
|
||||
Run `npm run docs:check` for documentation facts and generated consistency, and
|
||||
`npm run verify:docs:routes` to build the complete production site and check local
|
||||
links, anchors and physical edit targets.
|
||||
When a physical path moves, also run the tests that read it. Follow
|
||||
[development checks](testing.md#developer-commands) before requesting review.
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,110 +0,0 @@
|
||||
# .NET single-file bundles
|
||||
|
||||
Use `rea inspect-artifact /absolute/path/host --json` to inventory an ordinary
|
||||
.NET single-file host and `rea extract-artifact /absolute/path/host --json` to
|
||||
materialize its members into a fresh, REA-owned output directory. With a host
|
||||
selected through `open_binary`, the corresponding MCP tools are
|
||||
`inspect_artifact` and `extract_artifact`. No .NET installation is needed for
|
||||
inspection or extraction.
|
||||
|
||||
For a compact inventory, call `inspect_artifact` with `detail: "summary"`,
|
||||
then use `inspect_analysis_view` with the returned `parent_evidence_id`.
|
||||
Inventory occurrence pages accept `logical_paths`, `occurrence_ids`, selected
|
||||
`fields`, `offset`, and `limit`; node pages accept batches of `artifact_ids`
|
||||
to retrieve expanded-byte digests. Include `format_metadata` and
|
||||
`source_location` when member declarations and stored extents matter. Identity
|
||||
fields survive projection, and the complete inspection Evidence stays retained.
|
||||
|
||||
REA detects the apphost marker and its preceding header offset from PE, ELF,
|
||||
and thin Mach-O bytes, regardless of filename. An unbundled apphost's zero
|
||||
offset remains an ordinary native artifact. Universal Mach-O slice inventory
|
||||
keeps its existing behavior; bundled contents inside slices are not expanded.
|
||||
When a native file has a recognized PyInstaller CArchive, apphost-marker
|
||||
scanning stops before its payload. A .NET helper stored inside that archive
|
||||
does not replace the outer container's format; an outer .NET host can still
|
||||
contain a PyInstaller helper as a member.
|
||||
|
||||
## Supported layout
|
||||
|
||||
| Header | Producer generation | Member encoding |
|
||||
| ------ | ---------------------------------- | --------------------------- |
|
||||
| 1.0 | .NET Core 3.x | Uncompressed |
|
||||
| 2.0 | .NET 5 | Uncompressed |
|
||||
| 6.0 | .NET 6 and later using this layout | Uncompressed or raw DEFLATE |
|
||||
|
||||
Versions, minor versions, and flags outside this set are refused explicitly;
|
||||
they do not silently become an ordinary PE or a successful empty inventory.
|
||||
Flags 0 and the v2+ compatibility flag 1 are admitted. Unknown file-type bytes
|
||||
are retained with a limitation, rather than treated as executable authority.
|
||||
Encrypted/protected wrappers and recovering their payloads are outside this
|
||||
reader's scope.
|
||||
|
||||
The host keeps its whole-file SHA-256. Each embedded file gets its expanded-byte
|
||||
SHA-256, path, declared size, compressed size, and `format_metadata` declaration
|
||||
(`major_version`, `minor_version`, `file_type`). Its `source_location` describes
|
||||
the stored host extent, including the compressed length when applicable.
|
||||
The root manifest's `root_format` is `dotnet-bundle`; the root content node
|
||||
still records its native PE/ELF/Mach-O format. A file-type declaration or `.dll`
|
||||
suffix does not establish a valid assembly. Inspect an extracted component
|
||||
explicitly with the managed tools to establish PE/CLI facts.
|
||||
|
||||
## Admission and containment
|
||||
|
||||
Metadata is range-read with a 64 KiB cache. Policy budgets are 16 MiB of
|
||||
manifest metadata, 100,000 files, 64 KiB per UTF-8 string, 512 MiB of stored or
|
||||
expanded bytes per file, 256 path parts per member, 250,000 distinct in-memory
|
||||
path-tree nodes, and 2 GiB of aggregate expanded bytes. The node budget counts
|
||||
shared directory prefixes only once. These are REA
|
||||
resource policies, not limits imposed by Microsoft's format. Marker scanning
|
||||
streams 64 KiB chunks and reuses completed probe facts during extraction;
|
||||
ordinary native files without a marker incur one extra
|
||||
read pass before host hashing.
|
||||
|
||||
Extraction hashes the whole host again through the same descriptor used to read
|
||||
members and compares it with the inventory's root digest before creating an
|
||||
output directory. This adds a full-host read pass to extraction, including when
|
||||
the embedded members themselves are small.
|
||||
|
||||
Malformed/truncated headers, unsafe or conflicting paths, overlapping member
|
||||
ranges, members crossing the manifest, contradictory configuration extents,
|
||||
invalid UTF-8, corrupt/trailing compressed data, and incorrect expanded sizes
|
||||
fail before output publication. Source descriptor/path identity is checked
|
||||
during reading; extraction verifies the host digest and member bytes against
|
||||
the inventory. Existing output roots are never overwritten. Cancellation and
|
||||
failures release owned streams/descriptors through the shared extraction
|
||||
rollback scope.
|
||||
|
||||
## Reproduce producer fixtures
|
||||
|
||||
The source-owned generator uses the installed .NET 8 SDK's **Microsoft
|
||||
HostModel** producer to emit x64 v1, v2, plain v6, and compressed v6 bundles
|
||||
for Windows, Linux, or macOS. It does not run generated hosts or download runtime packs. Supply an
|
||||
SDK apphost template from
|
||||
`packs/Microsoft.NETCore.App.Host.<rid>/<version>/runtimes/<rid>/native/apphost`
|
||||
(`apphost.exe` on Windows). Pass `windows` (the default), `linux`, or `osx`
|
||||
as the optional third generator argument to match that template.
|
||||
|
||||
```bash
|
||||
dotnet build scripts/fixtures/dotnet-bundle/BundleFixture.csproj -c Release --configfile scripts/fixtures/dotnet-bundle/NuGet.Config -o /absolute/path/producer -p:BaseIntermediateOutputPath=/absolute/path/producer-obj/
|
||||
dotnet /absolute/path/producer/BundleFixture.dll /absolute/path/apphost.exe /absolute/path/absent-bundles-directory
|
||||
rea inspect-artifact /absolute/path/absent-bundles-directory/net8-compressed/Probe.exe --json
|
||||
rea extract-artifact /absolute/path/absent-bundles-directory/net8-compressed/Probe.exe --json
|
||||
```
|
||||
|
||||
The generator writes `oracle.json` beside each host. An independent C#
|
||||
`BinaryReader`/`DeflateStream` pass records version, member types and stored
|
||||
extents, then compares every expanded member digest with its source file.
|
||||
Compare those records with inspection Evidence and the extracted bytes.
|
||||
This verifies Microsoft's compression writer separately from synthetic parser
|
||||
fixtures; it is not evidence that the bundled host executes on a target runtime.
|
||||
|
||||
Deterministic boundary and public-entrypoint coverage:
|
||||
|
||||
```bash
|
||||
npm run test:focused -- tests/boundary/filesystem/dotnetBundleArtifacts.test.ts tests/acceptance/dotnetBundleJourney.test.ts
|
||||
```
|
||||
|
||||
Layout references: Microsoft's [manifest writer](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/Bundle/Manifest.cs),
|
||||
[file entry writer](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/Bundle/FileEntry.cs),
|
||||
[apphost marker handling](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/AppHost/HostWriter.cs),
|
||||
and [compression producer](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/Bundle/Bundler.cs).
|
||||
@@ -6,7 +6,7 @@ Disconnecting the inspection's page session can clear caller-owned
|
||||
`navigator.platform` emulation inside Chromium. REA cannot restore another
|
||||
client's complete user-agent/client-hint configuration. Use a disposable target
|
||||
or let the owning client reapply its configuration; see the shared
|
||||
[CDP state limitation](../web/browser-observation.md#provider-authorities).
|
||||
[CDP state limitation](browser-observation.md#provider-authorities).
|
||||
|
||||
This passive runtime surface is distinct from the target-free static
|
||||
[`analyze_javascript_application`](javascript-artifact-reconstruction.md)
|
||||
@@ -14,7 +14,7 @@ workflow. Static analysis reads a local ASAR or extracted directory without
|
||||
executing application code. Passive observation inspects an existing Electron
|
||||
page. The results keep static inferences and runtime observations distinct.
|
||||
Packaged fuse state is a separate static read:
|
||||
[`inspect_electron_fuses`](../../electron-fuses.md) reports whether this binary will
|
||||
[`inspect_electron_fuses`](electron-fuses.md) reports whether this binary will
|
||||
honor Inspector flags, `ELECTRON_RUN_AS_NODE`, embedded ASAR integrity, and
|
||||
`app.asar`-only loading.
|
||||
Use the separate
|
||||
@@ -75,13 +75,6 @@ DOM node indices retain their snapshot positions. Invalid rows are omitted with
|
||||
completeness diagnostics, so `total_nodes` can exceed the retained node count
|
||||
and a parent index can refer to an omitted source node.
|
||||
|
||||
`dom.node_value_length_unit` is `utf8-bytes`: each node's `node_value_length`
|
||||
counts the bytes needed to encode its value as UTF-8, not characters. Web
|
||||
observations instead declare `utf16-code-units`. These existing measurements
|
||||
are preserved; the unit is reported once for the whole DOM, without retaining
|
||||
text values. Historical captures without the unit leave it unknown; reading
|
||||
them does not infer a unit or convert their stored lengths.
|
||||
|
||||
Script metadata retains its execution-context frame ID when CDP supplies one and
|
||||
records admission separately in `frame_scope`. An `authorized` scope refers to
|
||||
the observed document generation; `unknown` retains the known frame ID without
|
||||
@@ -147,14 +140,6 @@ window-targeted renderer reload/crash actions, and synthetic `open-url` or
|
||||
WebContents identities, process metrics, and IPC channel and value-shape metadata.
|
||||
Payload values are not retained.
|
||||
|
||||
For the exact Playwright launch sequence, the active hook removes only REA's
|
||||
bootstrap arguments from the application-visible `process.argv`. The vector keeps
|
||||
the selected entry point and every caller argument, in order, including repeated
|
||||
hook paths, `.asar` filenames, and debug-like options. The private launch handoff
|
||||
is consumed before application code runs. If the observed sequence cannot be
|
||||
matched exactly, argv stays unchanged and capture coverage reports `hook_conflict`;
|
||||
REA does not guess an entry point or remove arguments by filename patterns.
|
||||
|
||||
`window_index` selects from Playwright's current Page list at the start of each action. REA resolves that Page to its corresponding BrowserWindow once; the action's `target` uses the BrowserWindow's capture-scoped `window_id`, matching the window inventory. Reordering either enumeration after selection does not retarget the action. Handle cleanup failures retain the completed action records and remain owned for provider cleanup retry.
|
||||
|
||||
Run the real fixture verifier with an operator-provided Electron runtime:
|
||||
@@ -201,9 +186,6 @@ neither result means the requested host operation occurred. The existing
|
||||
`openExternal`, `openPath`, and `trashItem` blocks reject their promises.
|
||||
The `trashItem` block prevents that call from reaching the OS trash operation;
|
||||
it does not intercept other ways the application can remove files.
|
||||
Process rejection and uncaught-exception events are observed at dispatch without
|
||||
adding exception handlers. The runtime and application retain their warning,
|
||||
handler, and fatal-exit policies, including for rejected shell-block promises.
|
||||
CLI SIGINT/SIGTERM requests cancellation and awaits owned-process cleanup.
|
||||
Window IDs, URLs, and cached titles come from one main-process Electron snapshot,
|
||||
so a crashed renderer does not require a successful renderer-side title query.
|
||||
@@ -328,7 +310,7 @@ budget used by static ASAR reads: 16 MiB / 16777216 bytes by default, seeded by
|
||||
`update_analysis_settings` can override `asar.max_header_bytes` for new
|
||||
operations without restarting REA. Cached headers are checked against each
|
||||
operation's admission, including after a decrease. See the
|
||||
[settings design](../../design/limits-and-recovery.md#failure-and-retry).
|
||||
[settings design](design/limits-and-recovery.md#failure-and-retry).
|
||||
An oversized header is
|
||||
not parsed; selecting a member path does not bypass admission, and the page
|
||||
stays unauthorized. The header cache separately retains at most that many bytes
|
||||
+1
-7
@@ -38,12 +38,6 @@ were examined; an unavailable section name does not erase those note facts.
|
||||
For GNU notes, `gnu.build_id_hex`, `gnu.abi_tag`, or `gnu.gold_version` are
|
||||
filled when the descriptor matches that type. Other owners keep `gnu` null.
|
||||
|
||||
For Linux `CORE` notes, `core.prpsinfo`, `core.file`, `core.siginfo`, or
|
||||
`core.auxv` are decoded for process arguments and status (`NT_PRPSINFO`), mapped
|
||||
file memory regions and paths (`NT_FILE`), fault signal and code (`NT_SIGINFO`), and
|
||||
auxiliary vectors (`NT_AUXV`) across 32-bit and 64-bit images. Other owners or unparsed
|
||||
notes keep `core` null.
|
||||
|
||||
`PT_NOTE` and `SHT_NOTE` are examined independently. The same file range may
|
||||
appear twice when both tables declare it.
|
||||
|
||||
@@ -54,7 +48,7 @@ the image never carried notes in another container or overlay.
|
||||
|
||||
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
|
||||
32 note entries are examined by default across every table. Retained note
|
||||
names, section names, GNU descriptor text, and core dump string representations share a scan budget based on
|
||||
names, section names, and GNU descriptor text share a scan budget based on
|
||||
the source bytes plus any decoded section-name table, capped at
|
||||
`max_file_bytes`. The raw source and decoded table must also fit together
|
||||
within that ceiling. If admission fails, the error reports the required
|
||||
|
||||
@@ -52,15 +52,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -124,15 +115,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -196,15 +178,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -268,15 +241,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -340,15 +304,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -412,15 +367,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -484,15 +430,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -556,15 +493,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -628,15 +556,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -700,15 +619,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -772,15 +682,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -800,7 +701,7 @@
|
||||
"calculated_sha256",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": {}
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
@@ -826,33 +727,7 @@
|
||||
"calculated_md5",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": {}
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"logical_path": { "type": "string" },
|
||||
"algorithm": { "type": "string", "const": "dfu-crc32" },
|
||||
"declared_crc32": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-f0-9]{8}$"
|
||||
},
|
||||
"calculated_crc32": {
|
||||
"anyOf": [
|
||||
{ "type": "string", "pattern": "^[a-f0-9]{8}$" },
|
||||
{ "type": "null" }
|
||||
]
|
||||
},
|
||||
"unpacked": { "type": "boolean" }
|
||||
},
|
||||
"required": [
|
||||
"logical_path",
|
||||
"algorithm",
|
||||
"declared_crc32",
|
||||
"calculated_crc32",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": {}
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -918,15 +793,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -990,15 +856,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1062,15 +919,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1134,15 +982,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1206,15 +1045,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1278,15 +1108,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1350,15 +1171,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1422,15 +1234,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1494,15 +1297,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -1,348 +0,0 @@
|
||||
# ESP flash images
|
||||
|
||||
REA reads a dump of an ESP32-family SPI flash, such as the output of
|
||||
`esptool read-flash 0 ALL dump.bin` or `esptool merge-bin`, through
|
||||
`inspect_artifact` and `extract_artifact`. The reader is static: no device,
|
||||
esptool or ESP-IDF is needed, and no code is run.
|
||||
|
||||
The container format is `esp-flash-image`. Entries are the bootloader region,
|
||||
the partition table sector and one file per partition. The bootloader and
|
||||
each app partition are also decoded as [ESP images](esp-image.md), so their
|
||||
headers, segments, integrity and descriptors appear in the root
|
||||
`format_metadata`. Layout facts are cited to ESP-IDF v6.1
|
||||
(`esp_flash_partitions.h`, `flash_partitions.c`, `gen_esp32part.py`,
|
||||
`bootloader_common_loader.c`, `bootloader_utility.c` and each chip's
|
||||
`esp_rom_caps.h`).
|
||||
|
||||
## Recognition
|
||||
|
||||
A file is a flash image when REA finds a partition table in the entry
|
||||
structure ESP-IDF's bootloader verifies (`esp_partition_table_verify`):
|
||||
32-byte `0x50AA` entries, at most one `0xEBEB` MD5 entry over the entries
|
||||
before it, and a terminating entry within `0xC00` bytes. ESP-IDF accepts a
|
||||
table with no partition entries, but REA requires one, so that neither
|
||||
erased flash nor a lone MD5 entry is read as a table. A table whose MD5 does not match, or that has a second MD5 entry, is
|
||||
still read, with `md5.status` `invalid` and a limitation, because the
|
||||
bootloader would refuse it. The table is located in this order:
|
||||
|
||||
1. at the caller's `partition_table_offset`, when given; no scan follows;
|
||||
2. at the default `0x8000`;
|
||||
3. by scanning every `0x1000`-aligned flash sector for a table whose MD5
|
||||
verifies. This location is derived, and the inventory says so.
|
||||
|
||||
A table never lies inside the located bootloader image, so the default
|
||||
offset and the scan are tried only after the image's last segment. Without
|
||||
a caller selection, the scan runs only when an ESP image sits at a ROM
|
||||
bootloader offset, and a single app or bootloader image ends at its last
|
||||
segment, so neither ordinary files nor single images are scanned. A C3 or
|
||||
S3 dump begins with its bootloader, so flash layouts are detected before a
|
||||
file is read as one ESP image.
|
||||
|
||||
Unless the caller names the table with `partition_table_offset`, REA
|
||||
refuses two kinds of dump before looking for a table:
|
||||
|
||||
- an image of a known chip at another chip's ROM offset, such as an ESP32
|
||||
bootloader at the start of a dump read from `0x1000`. The dump's origin
|
||||
is not the flash address REA assumed, given or default, and a table found
|
||||
there would place every partition at the wrong file offset; pass the
|
||||
`flash_offset` where the dump starts. When a partition table sits where
|
||||
the image's own ROM offset puts `0x8000`, after the image's segments, the
|
||||
refusal names that offset.
|
||||
Without a selected format such a dump opens as its bootloader image, whose
|
||||
[ESP image](esp-image.md) facts say how to reopen it.
|
||||
- an ESP8266 image at a ROM offset. ESP8266 dumps can hold the same table
|
||||
format, and they keep the [ESP image](esp-image.md) reader's ESP8266
|
||||
refusal.
|
||||
|
||||
When no table is found the layout is not recognized. This is an unknown
|
||||
outcome, not a malformed file: flash encryption, which encrypts the
|
||||
bootloader and the table, is one cause. When a table is found without a
|
||||
located bootloader and `flash_offset` is omitted, a limitation says that
|
||||
each partition's file offset assumes the dump starts at flash `0x0`.
|
||||
|
||||
## Caller inputs
|
||||
|
||||
| Input | Meaning |
|
||||
| ------------------------- | ----------------------------------------------------------------------------- |
|
||||
| `format: esp-flash-image` | Read the file as a flash dump; with no table found, refuse it |
|
||||
| `flash_offset` | Flash address (`0x…`) of the dump's first byte; default 0 |
|
||||
| `partition_table_offset` | Flash address (`0x…`, a multiple of `0x1000`) of the table; disables the scan |
|
||||
|
||||
MCP takes these on `open_binary`; the CLI takes `--target-format
|
||||
esp-flash-image`, `--flash-offset` and `--partition-table-offset` on
|
||||
`inspect-artifact` and `extract-artifact`. The offsets require the format, and
|
||||
the format takes no `architecture` or `entry_point`. The inputs are part of
|
||||
the target's identity, so reopening the same bytes with other inputs reads
|
||||
them again instead of reusing the earlier reading. The `open_binary` summary
|
||||
reports them as `flash_offset` and `partition_table_offset`, null when
|
||||
omitted.
|
||||
|
||||
## Layout facts
|
||||
|
||||
The root `format_metadata` (`esp-flash-image`) records:
|
||||
|
||||
- `flash_offset`: its `value` and whether the `caller` gave it or it is the
|
||||
`default`;
|
||||
- `chip`: from the bootloader's `chip_id`, or null without a located
|
||||
bootloader;
|
||||
- `bootloader`: the region from the chip's ROM bootloader offset to the
|
||||
table, with its ESP image facts;
|
||||
- `partition_table`: its flash address, the `method` that found it
|
||||
(`default`, `caller` or `scan`), the partition count and the MD5 `status`
|
||||
(`valid`, `invalid` or `absent`) with the stored and computed digests;
|
||||
- `apps`: ESP image facts for each app partition the dump holds; app
|
||||
partitions declared over the same bytes share one decode;
|
||||
- `otadata`: the otadata partition's records, or null without one;
|
||||
- `boot_selection`: the app the bootloader would select, for every table;
|
||||
- `partitions_outside_dump`: declared partitions that begin before the
|
||||
dump's first byte, which have no entry;
|
||||
- `undescribed`: ranges outside every region, each from a byte that is not
|
||||
erased flash to the end of its gap.
|
||||
|
||||
The ROM loads the bootloader from `ESP_ROM_BOOTLOADER_OFFSET_FLASH`:
|
||||
`0x1000` for ESP32 and ESP32-S2; `0x0` for ESP32-S3, C2, C3, C6, C61, H2 and
|
||||
H21; and `0x2000` for ESP32-C5, H4, P4 and S31. A bootloader is located only
|
||||
where its own `chip_id` says the ROM loads it.
|
||||
|
||||
### Partitions
|
||||
|
||||
Each partition entry is `partitions/<label>.bin` when the label is a safe
|
||||
name of up to 16 letters, digits, `.`, `_` and `-` that starts with a
|
||||
letter, digit or underscore and has at least one letter or underscore, is
|
||||
not a Windows device name such as `NUL` or `COM1`, alone or before a dot,
|
||||
and no other label repeats it, ignoring case as some file systems do; it is
|
||||
`partitions/<index>.bin` otherwise. `source_path` keeps the label, the
|
||||
table's own name for the partition. Its `format_metadata`
|
||||
(`esp-flash-partition`) records the table `index`, `label` (null when not
|
||||
UTF-8), `label_hex` (the label's bytes when it is not UTF-8, else null),
|
||||
`type`, `subtype`, ESP-IDF's `type_name` and `subtype_name`,
|
||||
`flash_address`, `size`, the `encrypted` and `readonly` flags and any
|
||||
`unknown_flags`. The `encrypted` flag says how a device with flash
|
||||
encryption stores the partition; it does not establish that the dump's
|
||||
bytes are ciphertext.
|
||||
|
||||
Each partition entry also records where the bytes the dump holds are
|
||||
written, relative to the partition's start: `written_offset`, the first
|
||||
byte that is not erased flash (`0xFF`), `written_end`, just past the last
|
||||
such byte, and `written_bytes`, how many bytes are not `0xFF`. A partition
|
||||
erased throughout has null offsets and `written_bytes` `0`; a partition the
|
||||
file holds none of has all three null. The range bounds the written
|
||||
bytes and does not say whether erased flash lies between them; extract the
|
||||
partition to read them. REA reads the bytes the partitions cover once, so
|
||||
overlapping partitions share that read however many the table declares.
|
||||
|
||||
ESP-IDF 5.3 and later may declare primary `bootloader` and `partition_table`
|
||||
entries over the bootloader region and the table sector. They are
|
||||
partitions like any other, whose bytes are also `bootloader.bin` and
|
||||
`partition-table.bin`, and they are exempt from the rule that partitions
|
||||
follow the table, as in `gen_esp32part.py`.
|
||||
|
||||
Every partition from the dump's first byte on is an entry holding the bytes
|
||||
the file has. One that the file holds only in part, as when an `esptool
|
||||
merge-bin` dump ends inside its last partition, has an entry limitation that
|
||||
gives the size the table declares; one that lies past the end of the file is
|
||||
an entry with no bytes and a limitation, and has no image facts. A partition
|
||||
that begins before the dump's first byte has no entry and is listed in
|
||||
`partitions_outside_dump`. One unreadable partition does not hide the
|
||||
others: partitions outside the dump or overlapping another are each reported
|
||||
while the rest stay readable.
|
||||
|
||||
### App partitions and the bootloader
|
||||
|
||||
Each app partition and the bootloader region carry their entry's `path`
|
||||
and ESP image `facts`:
|
||||
`status` is `esp-image`, `erased` (the image header bytes are erased flash,
|
||||
as in an empty OTA slot) or `unrecognized` (ciphertext or other data). For an
|
||||
image they hold the [ESP image](esp-image.md) root facts, `segments`,
|
||||
`signature_blocks`, `coverage` and `limitations`, with offsets relative to
|
||||
the region. Extract a partition and inspect it as an ESP image for its
|
||||
segments as entries.
|
||||
|
||||
The summary view (`inspect_artifact` with `detail: "summary"`, or the
|
||||
inventory summary of `inspect_analysis_view`) reports the root facts under
|
||||
their own kind, `esp-flash-image-summary`, and gives each of these images a
|
||||
`summary` instead of its `facts`: `status`, `chip`, `entry_addr`,
|
||||
`segment_count`, `load_ranges` (the load address, declared length and
|
||||
classification of each segment that is mapped, loaded or unclassified),
|
||||
`signature_block_count`, the `descriptor` kind with the fields that tell
|
||||
builds apart (an app's `project_name`, `version`, `idf_ver`,
|
||||
`secure_version` and `app_elf_sha256`; a bootloader's `version`, `idf_ver`
|
||||
and `secure_version`), the `checksum` and `appended_sha256` statuses,
|
||||
`coverage` and `limitations`. The default complete detail keeps the full
|
||||
facts, every segment included, and the summary's `inventory_evidence_ids`
|
||||
name the Evidence that holds them.
|
||||
|
||||
`erased` is the bootloader's view: with no image header at the start of the
|
||||
slot, it finds no app there. An erased slot can still hold data further in,
|
||||
even a complete app image. When it does, its entry's `written_offset` and
|
||||
`written_end` say where, and a limitation says that the slot holds bytes the
|
||||
bootloader does not read; REA does not look for images in them.
|
||||
|
||||
### OTA data
|
||||
|
||||
The otadata partition holds two `esp_ota_select_entry_t` records, one at the
|
||||
start of each 4 KiB sector. Like the bootloader, REA reads the last otadata
|
||||
partition the table declares, and the last factory app and OTA app of each
|
||||
subtype; a limitation names any repeated declaration. REA reports each
|
||||
record the partition and the dump hold: its `ota_seq`, `seq_label`,
|
||||
`ota_state` and its name, `crc`, whether the CRC matches (CRC-32 of
|
||||
`ota_seq` seeded with `0xFFFFFFFF`) and whether the bootloader would treat
|
||||
it as valid.
|
||||
|
||||
### Boot selection
|
||||
|
||||
`boot_selection` is present for every table. It is derived as
|
||||
`bootloader_utility_get_selected_boot_partition` does without app rollback
|
||||
or anti-rollback, and `basis` names its branch:
|
||||
|
||||
- `otadata`: the valid record with the higher `ota_seq` (record 0 on a tie)
|
||||
selects slot `(ota_seq - 1) % ota_app_count`, indexed by OTA subtype;
|
||||
- `factory-fallback`: no record is usable, or the table has no OTA app, and
|
||||
a factory app is declared; or records are set but none is valid, as after
|
||||
a CRC mismatch, which selects the factory index whether or not a factory
|
||||
app is declared;
|
||||
- `ota_0-fallback`: no record is usable, or the table has no OTA app, and no
|
||||
factory app is declared, so the bootloader selects boot index 0;
|
||||
- `no-otadata`: the table declares no otadata partition, or one at offset 0,
|
||||
which the bootloader takes for none, so it selects the factory index, as in
|
||||
the common factory-only layout;
|
||||
- `none`: the bootloader rejects the table, for an MD5 mismatch or a second
|
||||
MD5 entry, or cannot read an otadata partition smaller than two 4 KiB
|
||||
sectors, and resets without selecting an app;
|
||||
- `unknown`: the dump lacks an otadata record the bootloader reads.
|
||||
|
||||
Without an OTA app the records do not matter, so a dump that lacks them
|
||||
still selects a fallback. REA does not compare the partitions with the
|
||||
flash size in the bootloader header, which the bootloader also checks.
|
||||
|
||||
`active_record` and `slot` are set for `otadata` and `slot` for
|
||||
`ota_0-fallback`. `partition_index` is null when the table declares no
|
||||
partition for the selected slot or fallback; the bootloader then tries the
|
||||
other apps. `assumptions` list what the derivation leaves out: records used
|
||||
as written (for the branches that read them), anti-rollback, load fallback,
|
||||
and factory-reset and test-app GPIO holds and a wake from deep sleep; they
|
||||
are empty for `none` and `unknown`. `running_partition` is always
|
||||
`unknown`: at boot the bootloader may first mark a pending-verify record
|
||||
aborted, check the app's secure version against eFuse, and fall back to
|
||||
another partition when an app fails to load.
|
||||
|
||||
## Coverage and extraction
|
||||
|
||||
Erased flash (`0xFF`) between regions is described. In a gap between
|
||||
regions, the first byte that is not erased starts an `undescribed` range to
|
||||
the end of the gap, and makes coverage `unknown`; the rest of that range is
|
||||
not examined. Coverage is also `unknown` when no bootloader is located, or
|
||||
when a declared partition begins before the dump's first byte, since it has
|
||||
no entry. A partition cut short or left out by the end of the file does not
|
||||
make it `unknown`: its entry holds the bytes the file has, none for one past
|
||||
the end, and its limitation says what the file lacks. Coverage describes the
|
||||
inventory of the file, not the observation of every declared flash byte.
|
||||
|
||||
So artifact comparison classifies every partition entry of two dumps. Where
|
||||
both lack the same bytes, as two `merge-bin` dumps of the same layout do,
|
||||
the entries are equal; a dump against a longer one reports each partition
|
||||
it cuts short or lacks as `changed`, and that entry's limitation says the
|
||||
file did not hold the bytes.
|
||||
|
||||
`extract_artifact` writes every entry as an ordinary file, `nvs_keys`
|
||||
partitions included. On a device without flash encryption an `nvs_keys`
|
||||
partition holds the NVS encryption keys in plain text, and a dump of it
|
||||
contains them. REA decrypts nothing, recovers no keys and does not interpret
|
||||
NVS.
|
||||
|
||||
## Comparison
|
||||
|
||||
Artifact comparison uses each partition's `label`, `label_hex`, `type`,
|
||||
`subtype`, `flash_address`, `size` and flags as its declaration, so a
|
||||
partition that moves, changes type or flags, or is renamed with unchanged
|
||||
bytes is `changed` in `metadata`. Partitions pair by path. The root facts
|
||||
are not compared as a declaration: they derive from the dump's bytes, which
|
||||
content comparison covers, and from the caller's `flash_offset` and
|
||||
`partition_table_offset`, which are part of the target's identity and
|
||||
decide each partition's byte range.
|
||||
|
||||
## Limits
|
||||
|
||||
- A flash dump inside another container, such as a ZIP, is classified from
|
||||
its first bytes and so reads as an ESP image, not as a flash layout; extract
|
||||
it first.
|
||||
- ESP8266 dumps are refused; Secure Boot v1 bootloader digests and
|
||||
flash-encrypted layouts are not read.
|
||||
- Partitions are not checked against the flash chip size, which the
|
||||
bootloader also verifies.
|
||||
- NVS, FAT, SPIFFS and other data partitions are extracted, not decoded.
|
||||
- An image nested in a partition is decoded for its facts; its segments are
|
||||
not inventory entries.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic fixtures with computed MD5s and CRCs, and one CRC otatool wrote,
|
||||
cover the table rules, every way the table is located, the scan's chunk and
|
||||
sector edges, the misplaced-origin guard with the origin omitted or given,
|
||||
the ESP8266 refusal, the `0x0`, `0x1000` and `0x2000` ROM bootloader
|
||||
offsets, single images that are not scanned, overlaps, primary entries,
|
||||
truncation and partitions outside the dump, index names for unsafe labels,
|
||||
repeated declarations, the otadata states and records, and each branch of
|
||||
the bootloader's selection. CLI and stdio MCP tests exercise the inputs and
|
||||
their identity and extraction. MCP `compare_artifacts` tests compare
|
||||
partition declarations, labels that are not UTF-8 included, two dumps that
|
||||
end inside their last partition, and such a dump against a full one.
|
||||
|
||||
The optional real lane builds its dumps with esptool and ESP-IDF's own tools:
|
||||
|
||||
```sh
|
||||
REA_RISCV_CLANG=/path/to/clang REA_RISCV_LLD=/path/to/ld.lld \
|
||||
REA_ESPTOOL_PYTHON=/path/to/python-with-esptool \
|
||||
REA_ESP_IDF_PATH=/path/to/esp-idf \
|
||||
npm run verify:esp-flash
|
||||
```
|
||||
|
||||
It converts the conformance ELFs with esptool, builds a default, a
|
||||
relocated and a primary-entry partition table with `gen_esp32part.py`,
|
||||
assembles dumps with `esptool merge-bin`, one of them written from flash
|
||||
`0x1000` with `--target-offset`, and writes OTA records with `otatool`
|
||||
through `tests/conformance/esp/flash_file_tool.py`. That script replaces
|
||||
parttool's esptool calls with reads and writes of the dump file, so
|
||||
otatool's own OTA logic runs unchanged. The lane then checks:
|
||||
|
||||
- the bootloader and app facts against `esptool image-info`;
|
||||
- every partition against `gen_esp32part.py`'s decoding of the table REA
|
||||
extracted;
|
||||
- both OTA records against `otatool read_otadata` after switching to slot 0,
|
||||
then to slot 1, then erasing the otadata, and the derived selection after
|
||||
each: slot 0, slot 1, then the factory fallback;
|
||||
- that each extracted file is the dump's bytes at its range and that the
|
||||
source is unchanged;
|
||||
- a table with the primary `bootloader` and `partition_table` entries that
|
||||
`--primary-bootloader-offset` fills, read with complete coverage and no
|
||||
overlap or misplacement limitation; it has no otadata, so its boot
|
||||
selection is `no-otadata` with the factory app;
|
||||
- the relocated table, found by scanning and named with
|
||||
`partition_table_offset`, and the dump written from `0x1000`, read with
|
||||
`flash_offset` and, without it, left an ESP image whose `flash_dump`
|
||||
names `flash_offset` `0x1000`;
|
||||
- an app with a flipped byte beside valid partitions, and a truncated dump,
|
||||
read with complete coverage and its missing partitions as entries with no
|
||||
bytes.
|
||||
|
||||
The lane ran on macOS arm64 with Node 24.22.1, Homebrew Clang and LLD 21.1.8
|
||||
and ESP-IDF v5.5.2's `gen_esp32part.py`, `otatool.py` and `parttool.py`,
|
||||
once with esptool v5.6.0 and once with 4.12.0. Its `gen_esp32part.py` is
|
||||
identical to v6.1's, and its otatool and parttool differ only in import style
|
||||
and esptool command spelling. All checks passed in both runs. With v5.6.0 and
|
||||
the build already cached, the run took 24.1 s elapsed, 16.9 s user CPU and
|
||||
6.3 s system CPU under `/usr/bin/time -l`; the largest single process peaked
|
||||
at a 278 MiB resident set, and the combined peak of all processes was not
|
||||
measured.
|
||||
|
||||
The lane does not establish:
|
||||
|
||||
- dumps read from a device: its dumps come from `merge-bin`;
|
||||
- chips other than ESP32-C3 and ESP32;
|
||||
- `ota_state` and record validity, which `read_otadata` does not print, or
|
||||
OTA states the bootloader writes itself, app rollback or anti-rollback;
|
||||
- selection with gaps or repeated OTA subtypes, which otatool does not
|
||||
write;
|
||||
- flash-encrypted dumps, which are not recognized by design.
|
||||
|
||||
It does not run in CI.
|
||||
@@ -1,333 +0,0 @@
|
||||
# ESP images
|
||||
|
||||
REA reads ESP32-family firmware images through `inspect-artifact` /
|
||||
`inspect_artifact` and `extract-artifact` / `extract_artifact`. OTA downloads,
|
||||
app partitions and second-stage bootloaders are all ESP images. Reading is
|
||||
static: no esptool, device or flash access is needed, and no code is run or
|
||||
disassembled.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./firmware.bin --json
|
||||
rea inspect-artifact ./ota.bin --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact ./firmware.bin --json
|
||||
```
|
||||
|
||||
For MCP, open the file with `open_binary`, then call `inspect_artifact`. The
|
||||
summary's `root_format_metadata` holds the image facts. Use its
|
||||
`parent_evidence_id` with `inspect_analysis_view` to select segment rows.
|
||||
`extract_artifact` writes one file per segment and per signature block.
|
||||
|
||||
For agent sessions, advertise the `session,artifact,application` tool families
|
||||
and consider setting both `REA_MCP_INPUT_SCHEMA_PROFILE` and
|
||||
`REA_MCP_OUTPUT_SCHEMA_PROFILE` to `compact`. The output profile bounds each
|
||||
advertised output schema to 4,096 bytes by relaxing nested client validation;
|
||||
REA still validates complete results against the canonical contracts. Pair
|
||||
this with retained summaries and selected inventory views to limit the context
|
||||
needed for one question. See [MCP contracts](reference/mcp-contracts.md) for the profiles
|
||||
and their client compatibility tradeoffs.
|
||||
|
||||
The layout facts follow ESP-IDF v6.1 (`esp_app_format.h`, `esp_app_desc.h`,
|
||||
`esp_bootloader_desc.h`, `esp_image_format.c`, the ROM `secure_boot.h`
|
||||
headers and `docs/en/security/secure-boot-v2.rst`) and esptool v5.6.0
|
||||
(`bin_image.py`, `cmds.py`, `loader.py`, espsecure).
|
||||
|
||||
## Recognition
|
||||
|
||||
The container format is `esp-image`. A file is an ESP image when its bytes
|
||||
show all of these; the file name does not matter:
|
||||
|
||||
- magic `0xE9` at offset 0 and 1 to 16 segments. A count of 0 is valid but
|
||||
loads nothing, and accepting it would admit x86 programs that begin
|
||||
`E9 00`;
|
||||
- a flash mode and flash speed code that ESP-IDF's enums define;
|
||||
- a `hash_appended` byte of 0 or 1, as esptool requires;
|
||||
- when the file holds the first segment header, a segment length that is a
|
||||
multiple of 4.
|
||||
|
||||
Later segments, the checksum and the digest are integrity facts. A truncated
|
||||
download is still an ESP image, with its integrity reported as unavailable.
|
||||
|
||||
| Input | Outcome |
|
||||
| --------------------------------------------------- | ---------------------------------------------------------------- |
|
||||
| ESP8266 image (v1 or v3): same magic, 8-byte header | Refused: "ESP8266 firmware images are not supported…" |
|
||||
| Flash-encrypted image (ciphertext) | Not recognized: the bytes carry no magic. Unknown, not malformed |
|
||||
| ESP8266 v2 image (magic `0xEA`) | Not recognized |
|
||||
|
||||
An ESP8266 image has its first segment header at offset 8, and that segment
|
||||
loads into ESP8266 DRAM, IRAM (Arduino's eboot links at `0x4010F000`) or the
|
||||
IROM flash window, where v3 and Arduino app images begin. In an ESP32-family
|
||||
header the same bytes hold `wp_pin` and the drive settings, which esptool and
|
||||
ESP-IDF write as `0xEE` and zeros. Inside another container, an ESP8266 image
|
||||
is classified as a plain file.
|
||||
|
||||
Recognition cannot exclude every file that starts with an x86 near jump
|
||||
(`0xE9`), such as a DOS `.com` program. An explicit `--target-format dos-com`
|
||||
or `raw-binary` selection skips ESP recognition.
|
||||
|
||||
## Image facts
|
||||
|
||||
Root `format_metadata` has `kind: "esp-image"`:
|
||||
|
||||
| Field | Label | Meaning |
|
||||
| ------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
|
||||
| `header` | observed | Every `esp_image_header_t` field, with ESP-IDF's names, for example `entry_addr`, `chip_id`, `min_chip_rev_full` |
|
||||
| `chip` | derived | `target`, `isa` and the table's `source` (`ESP-IDF v6.1`) from `chip_id`; `null` for an ID the table lacks |
|
||||
| `checksum` | derived | `offset`, `stored`, `computed` and `status`: `valid`, `invalid` or `unavailable` |
|
||||
| `appended_sha256` | derived | The digest `hash_appended` adds: `offset`, `stored`, `computed` and `status`, including `not-appended` |
|
||||
| `image_end` | derived | End of the checksum block and any appended digest; `null` when the file ends before it |
|
||||
| `signature_sector_offset` | derived | The Secure Boot v2 signature sector, when it holds at least one valid block |
|
||||
| `undescribed_trailing` | derived | Byte ranges after the image that nothing describes; empty when every byte is described |
|
||||
| `descriptor` | observed | `esp_app_desc_t` or `esp_bootloader_desc_t` fields, with ESP-IDF's names, and the descriptor's file `offset` |
|
||||
|
||||
The descriptor's `kind`, derived from its magic, classifies the image as an
|
||||
`app` or a `bootloader`. Without a descriptor the role is unknown.
|
||||
|
||||
The ISA comes from `chip_id`, never from the series letter: ESP32-S31 is
|
||||
RISC-V. The chip table covers every `esp_chip_id_t` in ESP-IDF v6.1:
|
||||
|
||||
| `chip_id` | Target | ISA | `chip_id` | Target | ISA |
|
||||
| --------- | --------- | ------ | --------- | ---------- | ----- |
|
||||
| `0x0000` | `esp32` | xtensa | `0x0012` | `esp32p4` | riscv |
|
||||
| `0x0002` | `esp32s2` | xtensa | `0x0014` | `esp32c61` | riscv |
|
||||
| `0x0005` | `esp32c3` | riscv | `0x0017` | `esp32c5` | riscv |
|
||||
| `0x0009` | `esp32s3` | xtensa | `0x0019` | `esp32h21` | riscv |
|
||||
| `0x000C` | `esp32c2` | riscv | `0x001C` | `esp32h4` | riscv |
|
||||
| `0x000D` | `esp32c6` | riscv | `0x0020` | `esp32s31` | riscv |
|
||||
| `0x0010` | `esp32h2` | riscv | | | |
|
||||
|
||||
Other IDs, for example esptool v5.6.0's ESP32-E22 (`0x1F`), have `chip: null`
|
||||
and unclassified segments.
|
||||
|
||||
**Integrity.** The checksum is the XOR of every segment data byte, seeded with
|
||||
`0xEF`, in the last byte of the next 16-byte block. The appended SHA-256 covers
|
||||
every byte through the checksum. Both are facts, not admission rules: a
|
||||
mismatch is reported with a limitation and the segments stay readable.
|
||||
|
||||
**Descriptors.** Both descriptors begin segment 0: ESP-IDF's bootloader reads
|
||||
the app descriptor there, and esptool places the bootloader descriptor there.
|
||||
Strings stop at the first NUL and are omitted, with a limitation, when they
|
||||
are not valid UTF-8. The bootloader descriptor's magic is a single byte
|
||||
(`0x50`), so its reserved bytes, which ESP-IDF zero-fills, must also be zero.
|
||||
A `--ram-only-header` image puts RAM segments first, so its app descriptor is
|
||||
not in segment 0 and its role is unknown. `mmu_page_size` keeps ESP-IDF's
|
||||
name and value: the MMU page size as a base-2 logarithm, as
|
||||
`esp_app_desc_t` declares it, so `16` means 64 KiB. It is `0` in images built
|
||||
before ESP-IDF v5.4. `spi_flash_mode` is the next byte, which ESP-IDF v6.1
|
||||
names, kept as stored and not interpreted; releases that reserve it leave it
|
||||
`0`. A file can end inside the app descriptor: every field it holds in full
|
||||
is kept, a field it cuts off is `null`, and `observed_length` gives the
|
||||
descriptor bytes the file holds. The segment's truncation is reported as usual.
|
||||
|
||||
## Segments
|
||||
|
||||
Each header-declared segment is one entry named `segment-<index>.bin`, for
|
||||
example `segment-00.bin`, so two builds pair segments by position. Its
|
||||
`source_location` is the segment data in the image file. `format_metadata`
|
||||
(`esp-image-segment`) records `index`, `load_addr`, `data_len`,
|
||||
`header_offset` and `classification`.
|
||||
|
||||
`classification` follows ESP-IDF's bootloader (`should_map` and
|
||||
`should_load` in `esp_image_format.c`) on a cold boot:
|
||||
|
||||
| Value | Meaning |
|
||||
| ------------ | ------------------------------------------------------------------------------------------------- |
|
||||
| `mapped` | Inside the chip's IROM, DROM or, where the MMU is per memory type, PSRAM range (`soc.h`) |
|
||||
| `loaded` | `should_load` selects it: not mapped, at or above `0x10000000`. RAM ranges are not checked |
|
||||
| `not-loaded` | Below `0x10000000`: the padding segments esptool writes at address 0, including `--secure-pad-v2` |
|
||||
| `unknown` | The chip is unknown, or the mapping depends on the bootloader's MMU page size |
|
||||
|
||||
`loaded` does not mean the bootloader accepts the segment. Before loading,
|
||||
ESP-IDF's `verify_load_addresses` rejects the image when a loaded segment is
|
||||
not entirely in the chip's DRAM, IRAM, RTC memory or SPM, or overlaps the
|
||||
bootloader's own stack, data or loader code. For example, an ESP32-C6
|
||||
segment at `0x43000000` is `loaded` here and rejected there. REA models no
|
||||
RAM ranges.
|
||||
|
||||
Padding segments are inventoried and extractable, but are never load regions.
|
||||
When waking from deep sleep, ESP-IDF also skips RTC-memory segments; that
|
||||
runtime case is not modelled. ESP32-C6 and ESP32-H2 map flash from
|
||||
`0x42000000` through 256 pages of the bootloader's configured MMU page size,
|
||||
which no image records. The app descriptor's page size describes the app's
|
||||
build, not the bootloader's. With page sizes from 8 KiB (which the hardware
|
||||
supports and esptool builds for) to 64 KiB, segments from `0x42200000` to
|
||||
`0x43000000` are `unknown`. Ranges follow ESP-IDF's `should_map`, not esptool's
|
||||
wider placement ranges.
|
||||
|
||||
A segment cut short by the end of the file is the bytes the file holds, with
|
||||
an entry limitation that gives the declared length.
|
||||
|
||||
Artifact comparison uses each segment's `load_addr` and `data_len` as its
|
||||
declaration, so a segment that moves with unchanged bytes is `changed` in
|
||||
`metadata`. Segments pair by index: inserting or removing one shifts the
|
||||
paths of the segments after it. Header offsets and `classification` are not
|
||||
compared; the image's own change covers the header and chip. The image's and
|
||||
the signature blocks' `format_metadata` carry no declaration: their fields
|
||||
come from their own bytes, which content comparison covers, and a block's
|
||||
`image_digest_matches` follows the image's content.
|
||||
|
||||
## Secure Boot v2 signatures
|
||||
|
||||
A signature sector starts at the next 4 KiB boundary after the image and
|
||||
holds up to three 1216-byte slots. A slot holds a block when it has the `0xE7`
|
||||
magic and a matching CRC32 at offset 1196, the rule ESP-IDF and espsecure
|
||||
apply. Each slot is examined independently, so an erased or damaged slot does
|
||||
not hide a later block. Each block is an entry, `signature-block-<slot>.bin`,
|
||||
with `esp-image-signature-block` metadata:
|
||||
|
||||
| Field | Meaning |
|
||||
| ---------------------------------- | ------------------------------------------------------------------------- |
|
||||
| `index`, `version`, `scheme` | Slot 0 to 2, version byte, and `rsa` (`0x02`) or `ecdsa` (`0x03`) |
|
||||
| `sha_version`, `curve_id` | For ECDSA, the third byte (0 is SHA-256, 1 SHA-384) and the curve ID byte |
|
||||
| `digest_algorithm`, `image_digest` | The digest the block records |
|
||||
| `image_digest_matches` | Whether it equals the digest of every byte before the sector |
|
||||
| `public_key_sha256` | SHA-256 of the block's public key, the value eFuse key digests hold |
|
||||
| `signature_status` | Always `unverified` |
|
||||
|
||||
Signatures are observed, not verified: REA checks no public key, signature or
|
||||
eFuse. `public_key_sha256` is the digest espsecure `signature-info-v2`
|
||||
prints, so it can be compared with `espefuse summary`. An unknown version or
|
||||
`sha_version` leaves the layout-dependent fields `null`. A slot with the magic
|
||||
but no matching CRC32 is not a block: it is reported in a limitation, whether
|
||||
or not another slot holds a block, and its bytes are undescribed. Secure Boot
|
||||
v1 signatures are not decoded.
|
||||
|
||||
## Coverage and limitations
|
||||
|
||||
Bytes after the image are described when they are erased flash (`0xFF`) or a
|
||||
signature sector's blocks; erased slots and slack in that sector count as
|
||||
erased flash. Every other byte is in `undescribed_trailing`.
|
||||
|
||||
| Situation | Outcome |
|
||||
| ---------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
|
||||
| Segment data or a segment header beyond the end of the file | Truncation limitation; checksum and SHA-256 `unavailable`; coverage `unknown` |
|
||||
| File ends before the checksum or the appended digest | That status is `unavailable`, with a limitation; coverage `unknown`, since a signature sector could follow |
|
||||
| Checksum or SHA-256 mismatch | `invalid`, with a limitation; coverage is unchanged |
|
||||
| Erased flash after the image that runs past the three slots | Described; no limitation |
|
||||
| Erased flash that ends before the three slots are complete | Limitation; coverage `unknown`, since a signed image cut there would have lost its sector |
|
||||
| A signature sector the file cuts short | Limitation; coverage `unknown` |
|
||||
| Any other bytes after the image | Ranges in `undescribed_trailing`, a limitation, and coverage `unknown` |
|
||||
| Segment length that is not a multiple of 4 | Limitation: ESP-IDF rejects such an image |
|
||||
| Segment of 16 MiB or more, except on chips with 32-bit cache addresses | Limitation: ESP-IDF rejects such an image |
|
||||
|
||||
ESP32-S3, ESP32-P4, ESP32-C5, ESP32-C61, ESP32-H4 and ESP32-S31 map flash
|
||||
through 32-bit cache addresses (`SOC_SPI_MEM_SUPPORT_CACHE_32BIT_ADDR_MAP`), so
|
||||
ESP-IDF sets no 16 MiB segment limit for them. For an unknown chip the limit
|
||||
is not applied.
|
||||
|
||||
Undescribed bytes include other flash contents such as an app partition's
|
||||
stale data, a Secure Boot v1 signature, and the flash segments that esptool
|
||||
`--ram-only-header` writes after the checksum without counting them. REA
|
||||
reports their ranges and does not guess their structure. In the padding, in a
|
||||
sector without a valid block, in the sector's slack and after the sector, the
|
||||
first byte that is not erased starts a range to the end of that region. A
|
||||
slot that is neither a block nor erased is reported whole. Adjacent ranges
|
||||
are merged.
|
||||
|
||||
A whole-flash dump whose partition table REA finds is read as an
|
||||
[ESP flash image](esp-flash-image.md). Without one, a dump of a chip whose
|
||||
bootloader is at offset 0, such as ESP32-C3 or ESP32-S3, is recognized as that
|
||||
bootloader image, and the partition table, apps and data follow as
|
||||
undescribed bytes.
|
||||
|
||||
A dump read from a later flash address, such as an ESP32 dump read from
|
||||
`0x1000` with `read-flash 0x1000 …` or `merge-bin --target-offset 0x1000`,
|
||||
opens as its bootloader image. REA says so when the file shows it: the image
|
||||
at file offset 0 is not an app, its chip's ROM loads the bootloader from a
|
||||
later address (`0x1000` for ESP32 and S2, `0x2000` for C5, H4, P4 and S31),
|
||||
and a partition table sits where that origin puts `0x8000` (file offset
|
||||
`0x7000` for `0x1000`), after the image ends. Then `flash_dump` records
|
||||
`flash_offset` and
|
||||
`table_file_offset`, a limitation says to reopen the file with
|
||||
`format: "esp-flash-image"` and that `flash_offset`, and `inspect_artifact`
|
||||
lists the same `open_binary` call as a next probe. `flash_dump` is null for
|
||||
every other image. Evidence that REA recorded before it reported the field
|
||||
omits it; REA still reads and compares such Evidence, and the omission says
|
||||
nothing about where the file was read from.
|
||||
|
||||
## Limits
|
||||
|
||||
An image has at most 16 segments and 3 signature blocks. Decoding reads each
|
||||
byte once and retains only fixed-size fields. It reads through erased flash to
|
||||
find the first byte that is not erased, and stops there. There is no
|
||||
file-size limit.
|
||||
|
||||
## Analyzing a segment
|
||||
|
||||
Inventory does not disassemble code or build a memory map. `open_binary`
|
||||
admits neither RISC-V nor Xtensa as a raw-binary architecture, so a segment
|
||||
cannot yet be opened as a raw image. Each segment's `source_location` and
|
||||
`load_addr` give the file offset and address to map it at by hand, for
|
||||
example with the Rizin tool's `om` command.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic fixtures with computed checksums, digests and CRCs cover
|
||||
recognition, ESP8266 layouts, the ESP32, ESP32-C3, ESP32-C6 and ESP32-P4
|
||||
memory maps, truncation at each structure, integrity mismatches, erased and
|
||||
undescribed trailing bytes, signature slots including damaged and unknown
|
||||
layouts, and the refusals. CLI and stdio MCP tests exercise the same artifact
|
||||
provider and advertised schemas.
|
||||
|
||||
The optional real lane builds ELF files from source, converts them with
|
||||
esptool and compares REA's facts with `esptool image-info`:
|
||||
|
||||
```sh
|
||||
REA_RISCV_CLANG=/path/to/clang REA_RISCV_LLD=/path/to/ld.lld \
|
||||
REA_ESPTOOL_PYTHON=/path/to/python-with-esptool \
|
||||
npm run verify:esp-images
|
||||
```
|
||||
|
||||
It needs a Clang with the RISC-V target, `ld.lld` and a Python that can run
|
||||
`esptool` and `espsecure`. It checks each tool before building anything and
|
||||
names the variable to set when one is missing, and it warns when esptool is
|
||||
not v5.6.0, the version whose sources the layout facts cite. It builds an
|
||||
ESP32-C3 app, the same app with `--dont-append-digest`, `--secure-pad-v2`,
|
||||
`--ram-only-header` and `--pad-to-size`, a bootloader, and ESP8266 v1 and v3
|
||||
images. It then checks:
|
||||
|
||||
- entry point, chip ID, each segment's length, address and header offset,
|
||||
checksum, digest and descriptor fields against `image-info`;
|
||||
- each segment's classification in the app, its variants and the
|
||||
bootloader, whose linker scripts fix every section's region, including the
|
||||
`not-loaded` padding that `--secure-pad-v2` adds;
|
||||
- after signing with RSA-3072 and appending ECDSA P-256 and P-384 blocks,
|
||||
each block's `public_key_sha256` against espsecure `signature-info-v2`,
|
||||
and that its recorded digest matches;
|
||||
- extraction of the app and the signed image: each segment file equals the
|
||||
bytes after esptool's segment header offset, each signature block equals
|
||||
its slot in the sector espsecure appended, and the source's SHA-256 is
|
||||
unchanged and equals the reported `root_sha256`;
|
||||
- the `--pad-to-size` erased tail is described; the `--ram-only-header`
|
||||
flash segments are undescribed;
|
||||
- a flipped checksum, a flipped digest byte, a truncated segment (where
|
||||
`image-info` stops with "End of file reading segment"), and both ESP8266
|
||||
refusals.
|
||||
|
||||
`REA_XTENSA_ESP32_GCC` adds an ESP32 (Xtensa) app built with Espressif's
|
||||
`xtensa-esp32-elf-gcc`. Without it the report marks that image as skipped.
|
||||
|
||||
Each command runs as a process tree owned by the shared verifier command
|
||||
owner, so a timeout also stops the linker or assembler a compiler driver
|
||||
starts. The lane removes its workspace only after every tree has stopped and
|
||||
keeps it when that cleanup cannot be confirmed.
|
||||
|
||||
The lane ran on macOS arm64 with Node 24.22.1, Homebrew Clang and LLD 21.1.8,
|
||||
and `xtensa-esp32-elf-gcc` 14.2.0 (crosstool-NG esp-14.2.0_20251107), once
|
||||
with esptool and espsecure v5.6.0 and once with 4.12.0. All checks passed for
|
||||
all 13 images in both runs. With v5.6.0 the complete run, including the builds
|
||||
and every esptool call, took 21.7 s elapsed, 16.9 s user CPU and 6.1 s system
|
||||
CPU under `/usr/bin/time -l` with warm caches. The largest single process
|
||||
peaked at a 253 MiB resident set; the combined peak of all processes was not
|
||||
measured.
|
||||
|
||||
The lane does not establish:
|
||||
|
||||
- the other chips: only ESP32-C3 (RISC-V) and ESP32 (Xtensa) images come
|
||||
from a real producer; the other rows of the chip table are checked against
|
||||
the ESP-IDF v6.1 headers by reading them;
|
||||
- images from a real ESP-IDF project build, an OTA download or a device's
|
||||
flash, none of which it inspects;
|
||||
- that a signature is valid: it compares digests, and nothing verifies a
|
||||
signature.
|
||||
|
||||
It does not run in CI.
|
||||
@@ -28,15 +28,6 @@ The host must already provide util-linux `prlimit`, normally `/usr/bin/prlimit`;
|
||||
no system tools and changes no user configuration. [Upstream provenance and
|
||||
license](https://github.com/morluto/rea/blob/main/third_party/evmole/README.md) are packaged.
|
||||
|
||||
Limiter discovery validates the selected path without launching it. A relative
|
||||
path or directory is reported as `not_configured`, a missing file as
|
||||
`executable_missing`, and denied access or another filesystem-probe failure as
|
||||
`runtime_unavailable`. Recovery names the selected path and
|
||||
`REA_EVM_PRLIMIT_COMMAND`; denied access asks for execute permission and
|
||||
filesystem access. The original admission or system code stays in diagnostics.
|
||||
An available filesystem probe does not verify the selected tool's version or
|
||||
util-linux compatibility, and REA does not substitute another command.
|
||||
|
||||
## Evidence semantics
|
||||
|
||||
- `encoding` is required: raw bytes or UTF-8 hex, with optional `0x` prefix and
|
||||
@@ -89,10 +80,8 @@ without bytecode text or duplicated raw stdout:
|
||||
}
|
||||
```
|
||||
|
||||
The result is distinct view Evidence linked to its complete parent. Like the
|
||||
parent, it has `confidence: inferred`; projection never strengthens inferred
|
||||
candidates. Its `normalized_result.summary` preserves every producer-returned
|
||||
selector, decoded
|
||||
The result is distinct derived Evidence linked to its complete parent. Its
|
||||
`normalized_result.summary` preserves every producer-returned selector, decoded
|
||||
byte offset, dispatch, argument/mutability string including null, carrier/decoded
|
||||
identities, original limitations and explicit unknowns. Payload `evidence_kind`
|
||||
remains `inferred`. It preserves stderr/truncation and records stdout text length.
|
||||
@@ -11,20 +11,7 @@ select a returned ELF separately for supported Ghidra analysis.
|
||||
|
||||
On Linux, Binwalk and Unblob run as REA-owned processes and require a
|
||||
procps-compatible `ps` on REA's `PATH`; see the shared
|
||||
[Linux process prerequisite](../../getting-started/installation.md#linux-owned-process-prerequisite).
|
||||
|
||||
## Compare stored firmware components
|
||||
|
||||
For supported FIT and legacy uImage containers, use the built-in
|
||||
`inspect-artifact` / `inspect_artifact` inventory and shared
|
||||
`compare-artifacts` / `compare_artifacts` workflow. This route needs no Binwalk,
|
||||
Unblob or extraction: exact logical paths, stored-byte hashes and declared
|
||||
metadata establish component changes, with source Evidence and explicit unknowns.
|
||||
The outer container at `.` is separate from its children. See the
|
||||
[FIT version comparison recipe](../../fit-images.md#comparison) for CLI,
|
||||
MCP, summary delivery and selected supporting facts. Its real-data verification
|
||||
covers a pair of OpenWrt FIT releases; it does not establish comparison of
|
||||
decoded root filesystems or arbitrary firmware formats.
|
||||
[Linux process prerequisite](installation.md#linux-owned-process-prerequisite).
|
||||
|
||||
## Caller-supplied tools
|
||||
|
||||
@@ -39,9 +26,6 @@ REA does not install these tools, their dependencies or system packages. It
|
||||
checks the tool's `--version` banner when executing a request. Binwalk must
|
||||
print `binwalk <version>` and Unblob must print the version alone. Inspection and
|
||||
extraction have independent prerequisites; one does not require the other.
|
||||
Capability discovery recommends `REA_BINWALK_COMMAND` for missing inspection
|
||||
and `REA_UNBLOB_COMMAND` for missing extraction. Configuring the other engine
|
||||
does not enable the unavailable operation.
|
||||
Both require util-linux `prlimit`, normally `/usr/bin/prlimit`; override its
|
||||
absolute path with `REA_FIRMWARE_PRLIMIT_COMMAND` if needed. `rea doctor` does
|
||||
not currently inspect firmware readiness. Engine selection/version checks run
|
||||
@@ -98,11 +82,6 @@ Published file `relative_path` preserves producer spelling for lineage; `path`
|
||||
uses the writer's actual normalized filename. An extracted filename that cannot
|
||||
be represented losslessly as UTF-8 fails publication explicitly with its raw
|
||||
name bytes in hexadecimal instead of silently omitting the entry.
|
||||
Unpublished symlinks retain the extracted tree's target bytes in lowercase
|
||||
`link_target_hex`; `link_target` is null when those bytes are not valid UTF-8.
|
||||
Older retained Evidence may omit the hexadecimal field. Unblob may have rewritten
|
||||
the target before REA observed it, so these bytes are not necessarily those stored
|
||||
in the firmware image.
|
||||
|
||||
## CLI and MCP
|
||||
|
||||
@@ -209,7 +188,7 @@ Select a verified native child for the existing tools:
|
||||
|
||||
Then use `analyze_function`, `inspect_native_api` or the other native primitives.
|
||||
Native architecture and executable validation still apply. Generic
|
||||
[MIPS ELF](../../providers/ghidra-mips.md) and [RISC-V ELF](../../providers/ghidra-riscv.md) lanes are separate
|
||||
[MIPS ELF](ghidra-mips.md) and [RISC-V ELF](ghidra-riscv.md) lanes are separate
|
||||
documents with their own admission boundaries. This milestone does
|
||||
not add PPC decompilation, raw flash load profiles, bootloader/device
|
||||
tree semantics, vulnerability scanning, emulation or hardware runtime capture.
|
||||
@@ -258,12 +237,9 @@ tree semantics, vulnerability scanning, emulation or hardware runtime capture.
|
||||
3, allowed range 1–10.
|
||||
|
||||
Staging budgets are polled and rechecked after exit; short-lived overshoot is
|
||||
possible. The per-file process limit is one byte above the larger of the selected
|
||||
payload budget and the fixed report/diagnostic file allowances, so a small payload
|
||||
selection still permits the mandatory report, and a staged file the kernel stops
|
||||
at the limit exceeds its budget instead of passing as complete. A tighter
|
||||
inherited host limit still applies and can remove that byte, so a staged file
|
||||
that reaches the installed limit is rejected as possibly truncated.
|
||||
possible. The per-file process limit is the larger of the selected payload budget
|
||||
and the fixed report/diagnostic file allowances, so a small payload selection
|
||||
still permits the mandatory report. A tighter inherited host limit still applies.
|
||||
The inherited file-size limit bounds each file, not total storage;
|
||||
address-space limits are per process, not an aggregate memory quota. Worker
|
||||
count is not CPU affinity for every external extractor. These limits do not
|
||||
@@ -310,29 +286,9 @@ REA_FIRMWARE_VERIFY_GHIDRA=1 npm run verify:firmware
|
||||
generated gzip/USTAR/ext4 with a Linux x64 ELF child; SquashFS, UBIFS, proprietary
|
||||
firmware, other hosts and architectures remain unverified.
|
||||
|
||||
## Inspect ESP images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on an ESP32-family app or
|
||||
bootloader image, such as an OTA download or app partition, for its chip,
|
||||
classified segments, checksum and SHA-256 results, descriptor and Secure Boot
|
||||
v2 signature blocks. `extract-artifact` / `extract_artifact` writes one file
|
||||
per segment and signature block. Reading is portable and needs no external
|
||||
tool. See [ESP images](../../esp-image.md).
|
||||
|
||||
## Inspect ESP flash dumps
|
||||
|
||||
Use the same tools on a whole-flash dump, such as `esptool read-flash 0 ALL`
|
||||
output. The inventory reports the bootloader, the partition table and how it
|
||||
was found, each partition with its declared type, flags and digest, ESP image
|
||||
facts for the bootloader and each app partition, and the OTA records with a
|
||||
derived selection; the running partition stays unknown. Extraction writes
|
||||
each partition, `nvs_keys` included, as an ordinary file. For a dump that
|
||||
does not start at flash address 0, select `esp-flash-image` with
|
||||
`flash_offset`. See [ESP flash images](../../esp-flash-image.md).
|
||||
|
||||
## Inspect extracted devicetrees
|
||||
|
||||
Use [`inspect_devicetree`](../../devicetree.md) / `rea inspect-devicetree` on an
|
||||
Use [`inspect_devicetree`](devicetree.md) / `rea inspect-devicetree` on an
|
||||
extracted DTB for its node/property tree, reservations, hardware summary and
|
||||
selected phandle relationships. FDT v16/v17 inspection is portable and requires
|
||||
no external tool or disassembler.
|
||||
@@ -343,7 +299,7 @@ Use `inspect-artifact` / `inspect_artifact` on an Intel HEX or Motorola S-record
|
||||
file for its address regions, declared start address and record facts, then
|
||||
`extract-artifact` / `extract_artifact` for one raw file per region. Decoding is
|
||||
portable and requires no external tool. See
|
||||
[Intel HEX and S-record images](../../hex-records.md).
|
||||
[Intel HEX and S-record images](hex-records.md).
|
||||
|
||||
## Inspect U-Boot legacy uImages
|
||||
|
||||
@@ -352,26 +308,9 @@ portable and requires no external tool. See
|
||||
The root facts record the 64-byte header (architecture, OS, type, and
|
||||
compression as raw values and names, load and entry addresses, image name,
|
||||
and timestamp) with the header and data CRC32 verified; the stored payload
|
||||
is extractable as `payload.bin`, and multi-file and script images additionally
|
||||
expose each sub-image as `image-<n>.bin` from the stored size table. Declared
|
||||
is extractable as `payload.bin`, and multi-file images additionally expose
|
||||
each sub-image as `image-<n>.bin` from the stored size table. Declared
|
||||
addresses are facts from the header, not proven entry points, and compressed
|
||||
payloads stay as stored. Intel HEX/S-record and FIT images are separate
|
||||
readers and are not covered here. See [U-Boot legacy uImage](../../uimage.md) for
|
||||
readers and are not covered here. See [U-Boot legacy uImage](uimage.md) for
|
||||
component padding, preserved names, resource policies and verification scope.
|
||||
|
||||
## Inspect UF2 and DfuSe images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on a UF2 file for each familyID's
|
||||
address regions and its block numbering, or on a DfuSe file for its targets,
|
||||
elements and DFU suffix, then `extract-artifact` / `extract_artifact` for one
|
||||
raw file per region or element. A DfuSe suffix CRC mismatch follows
|
||||
`integrity_policy`. Decoding is portable and requires no external tool. See
|
||||
[UF2 and DfuSe images](../../uf2-dfuse.md).
|
||||
|
||||
## Inspect U-Boot FIT images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on a U-Boot FIT (`.itb`) for its
|
||||
images, declared load and entry addresses, configurations and verified hashes,
|
||||
then `extract-artifact` / `extract_artifact` for one raw file per image.
|
||||
Decoding is portable and requires no external tool. See
|
||||
[U-Boot FIT images](../../fit-images.md).
|
||||
@@ -1,293 +0,0 @@
|
||||
# U-Boot FIT images
|
||||
|
||||
REA inventories U-Boot Flattened Image Tree (FIT, `.itb`) files through
|
||||
`inspect-artifact` / `inspect_artifact`, and materializes the stored bytes of
|
||||
each image through `extract-artifact` / `extract_artifact`. Decoding is static
|
||||
and needs no external tool.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./boot.itb --json
|
||||
rea inspect-artifact ./boot.itb --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact ./boot.itb --json
|
||||
```
|
||||
|
||||
For MCP, open the file with `open_binary`, then call `inspect_artifact`. Summary
|
||||
delivery retains the complete Evidence; use `parent_evidence_id` with
|
||||
`inspect_analysis_view` to read selected images and fields. `extract_artifact`
|
||||
returns a fresh `output_root` containing one raw file per image. FIT summaries
|
||||
use the current projection identity; there is no pre-coverage FIT summary variant.
|
||||
|
||||
## Recognition
|
||||
|
||||
A FIT is a flattened devicetree (FDT v16 or v17) with a top-level `/images`
|
||||
node. Recognition walks the structure block within the first 4 KiB of the file
|
||||
only; the file name does not matter. A devicetree whose `/images` node begins
|
||||
after 4 KiB, a devicetree without `/images`, and any other FDT version stay on
|
||||
the plain devicetree route: `open_binary` refuses them and
|
||||
[`inspect_devicetree`](devicetree.md) still parses them. A U-Boot control DTB
|
||||
that contains `/images` is treated as a FIT. The container format is `fit`.
|
||||
|
||||
Recognition is a probe. The complete devicetree parse decides, so a file whose
|
||||
prefix declares `/images` but whose structure is malformed fails as a
|
||||
malformed FIT rather than falling back to plain classification.
|
||||
|
||||
## Images and metadata
|
||||
|
||||
Each child of `/images` is one inventory entry named `images/<unit-name>.bin`,
|
||||
listed in devicetree order. Its bytes come from exactly one data source:
|
||||
|
||||
| Source | Stored bytes |
|
||||
| --------------- | --------------------------------------------------------------------------------------- |
|
||||
| `data` | The embedded property value |
|
||||
| `data-offset` | `data-size` bytes at `data-offset` past the devicetree end, aligned to 4 (`mkimage -E`) |
|
||||
| `data-position` | `data-size` bytes at `data-position` from the start of the file (`mkimage -p`) |
|
||||
|
||||
`data-position` is read as U-Boot reads it: an offset from the start of the
|
||||
FIT, not a machine address.
|
||||
|
||||
Image `format_metadata` (`fit-image`) records `index`, `name`, `description`,
|
||||
`type`, `arch`, `os`, `compression`, `load` and `entry`, the `data_source`,
|
||||
the raw `declared_offset` for external data, `image_data` (the shared-data
|
||||
link), `cipher_present`, `cipher_algo`, `hash_algorithms`, `hashes_verified`,
|
||||
`hashes_unverified`, `signature_count`, `signatures_unreadable` and
|
||||
`signature_algorithms`. Algorithm lists are arrays. Nullable declaration fields
|
||||
are absent unless named in `unreadable_fields`, which records parsing failures
|
||||
per field. Hash or signature warnings do not make unrelated declarations unknown.
|
||||
`load` and `entry` fold one or two 32-bit cells into a lowercase
|
||||
`0x…` value, as U-Boot does, independent of `#address-cells`.
|
||||
|
||||
Root `format_metadata` (`fit`) records `fdt_version`, `fdt_totalsize`,
|
||||
`external_data_base` (the aligned devicetree end), the root `description`,
|
||||
`timestamp` (seconds since 1970) and `address_cells`, plus
|
||||
`default_configuration` and `configurations`. A missing description or
|
||||
timestamp is `null`.
|
||||
|
||||
Image bytes are reported as stored. Compressed images are not decompressed and
|
||||
ciphered images are not decrypted: entries report `encrypted: false` and the
|
||||
cipher declaration in `cipher_present` and algorithm in `cipher_algo`. A missing
|
||||
or malformed algorithm stays `null`; it does not establish unciphered bytes.
|
||||
Embedded `image-data` links require byte-identical stored properties; external
|
||||
links require matching physical ranges.
|
||||
|
||||
The extracted file holds the stored bytes. Direct-analysis suggestions require
|
||||
`compression: "none"` and `cipher_present: false`.
|
||||
|
||||
## Hashes and signatures
|
||||
|
||||
Hash subnodes are children whose name starts with `hash`. Inventory verifies
|
||||
`crc32` (4 bytes, big-endian), `md5`, `sha1`, `sha256`, `sha384` and `sha512`
|
||||
over the stored bytes, the same bytes mkimage hashed after any ciphering. All
|
||||
hashes of an image are computed in one streamed pass while the inventory is
|
||||
prepared, so a mismatch fails the whole inventory, naming the image, hash
|
||||
node, algorithm, and the declared and computed values. A supported algorithm
|
||||
whose value has the wrong length also fails.
|
||||
|
||||
Other algorithms such as `crc16-ccitt`, `crc8` or `sm3_256`, a hash without a
|
||||
readable `algo`, and a hash without a `value` are not verified; each is
|
||||
counted in `hashes_unverified` and reported as an image limitation.
|
||||
|
||||
Signature subnodes (names starting with `signature`) on images and
|
||||
configurations retain the total declaration count, readable algorithms and
|
||||
`signatures_unreadable`. Missing or malformed algorithms carry source limitations
|
||||
so unknown declarations remain distinguishable from absence. Signatures are never
|
||||
verified.
|
||||
|
||||
## Configurations
|
||||
|
||||
`/configurations` is optional. Each child records its `name`, `description`,
|
||||
and the image references `kernel`, `firmware`, `fdt` (base plus overlays),
|
||||
`ramdisk` and `loadables`, always as lists, plus its signature algorithms.
|
||||
`compatible`, `fpga`, `script`, `cmdline` and `load-only` are not reported. A
|
||||
reference to a name that is not an `/images` child, and a `default` that names
|
||||
no configuration, are reported as limitations and kept as declared. Each
|
||||
configuration carries `unresolved_reference_count`; dangling references share one
|
||||
configuration-level diagnostic while every original reference remains available.
|
||||
|
||||
## Facts, limitations and failures
|
||||
|
||||
| Situation | Outcome |
|
||||
| ---------------------------------------------------------------------------------------------- | ------------------------------------ |
|
||||
| Malformed devicetree, truncated header, `totalsize` past the end of the file | Fails as malformed |
|
||||
| No data source, more than one data source, external data without `data-size` | Fails as malformed |
|
||||
| `data-offset`, `data-position` or `data-size` not one 32-bit cell, `image-data` not one string | Fails as malformed |
|
||||
| External data outside the file, overlapping the devicetree or another image | Fails as malformed |
|
||||
| Two images with an identical external range linked by `image-data` | Accepted as shared data |
|
||||
| A dangling, chained or mismatched `image-data` link | Fails as malformed |
|
||||
| Hash mismatch, or a supported hash value with the wrong length | Fails integrity |
|
||||
| Unsupported, algorithm-less or value-less hashes | Unverified, reported as a limitation |
|
||||
| Malformed `description`, `type`, `arch`, `os`, `compression`, `load`, `entry` or `timestamp` | `null`, reported as a limitation |
|
||||
| Dangling configuration reference, missing default configuration | Reported as a limitation |
|
||||
|
||||
Failures apply to the whole file and return no partial inventory.
|
||||
|
||||
## Limits
|
||||
|
||||
The devicetree, including any embedded image data, must be at most 16 MiB; a
|
||||
larger `totalsize` fails as a memory limit. Store large images externally
|
||||
(`mkimage -E`): external data of any size is streamed from the file and never
|
||||
loaded into memory. A FIT may declare at most 1,024 images and 1,024
|
||||
configurations, and an image at most 8 hash nodes; the devicetree parser's
|
||||
node, property and depth bounds also apply. The sum of declared image sizes
|
||||
is bounded by the shared archive expansion budget. These are fixed resource
|
||||
bounds with no runtime override. Configurable ceilings are tracked in
|
||||
[#6176](https://github.com/morluto/rea/issues/6176).
|
||||
|
||||
## Analyzing an image
|
||||
|
||||
Inventory does not execute anything, and `load` and `entry` are declarations.
|
||||
After extraction:
|
||||
|
||||
- inspect `flat_dt` images with [`inspect_devicetree`](devicetree.md);
|
||||
- open an uncompressed image with `open_binary` using format `raw-binary`, an
|
||||
architecture matching its declared `arch`, and `entry_point` set to its
|
||||
declared `load` address; raw-binary maps file offset 0 and seeds analysis at
|
||||
`entry_point`. Automatic suggestions require nonempty bytes, an admitted
|
||||
architecture/address/extent, and a declared entry that is absent or equal to
|
||||
load. Distinct or unreadable entry declarations remain in Evidence without
|
||||
a raw-loader suggestion, since this loader cannot represent separate mapping
|
||||
and analysis entry addresses.
|
||||
|
||||
## Comparison
|
||||
|
||||
Use the shared `compare-artifacts` / `compare_artifacts` workflow for exact
|
||||
component paths and stored content identities. Neither extraction nor an external
|
||||
firmware engine is required. This compares the bytes exposed by this FIT reader;
|
||||
it does not inspect files inside a compressed root filesystem, match renamed
|
||||
components, or establish runtime equivalence.
|
||||
|
||||
### CLI
|
||||
|
||||
Retain each complete inspection while returning a summary:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact /absolute/baseline.itb --detail summary \
|
||||
--evidence_output /absolute/baseline-evidence.json --json
|
||||
rea inspect-artifact /absolute/candidate.itb --detail summary \
|
||||
--evidence_output /absolute/candidate-evidence.json --json
|
||||
```
|
||||
|
||||
The CLI returns a `{summary, evidence_bundle}` receipt. Use each response's
|
||||
`summary.normalized_result.parent_evidence_id` as the corresponding
|
||||
`evidence_id` below. Save this request as `comparison.json`, replacing the paths
|
||||
and IDs with the returned values, then run `rea compare-artifacts comparison.json
|
||||
--json`. Evidence-file paths must be absolute. Naming the exact complete parent
|
||||
avoids ambiguity when a bundle contains several inspection or inventory records.
|
||||
|
||||
```json
|
||||
{
|
||||
"left": {
|
||||
"kind": "evidence-file",
|
||||
"path": "/absolute/baseline-evidence.json",
|
||||
"evidence_id": "COMPLETE_BASELINE_EVIDENCE_ID"
|
||||
},
|
||||
"right": {
|
||||
"kind": "evidence-file",
|
||||
"path": "/absolute/candidate-evidence.json",
|
||||
"evidence_id": "COMPLETE_CANDIDATE_EVIDENCE_ID"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### MCP
|
||||
|
||||
For each file, call `open_binary` with its path, then `inspect_artifact` with
|
||||
`{"detail":"summary"}`. Opening the second file retains the first inspection's
|
||||
Evidence. Compare the two complete parent IDs without resending their graphs:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "compare_artifacts",
|
||||
"arguments": {
|
||||
"left": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "COMPLETE_BASELINE_EVIDENCE_ID"
|
||||
},
|
||||
"right": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "COMPLETE_CANDIDATE_EVIDENCE_ID"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The result counts unchanged, added, removed, changed, unknown and contradictory
|
||||
paths, returning every delta inline with source Evidence links. The root `.` is
|
||||
the outer FIT and is counted separately from `images/<unit-name>.bin`. A changed
|
||||
outer hash therefore does not imply every component changed. Additions and
|
||||
removals require complete container-entry coverage; ambiguous duplicate paths or
|
||||
missing observations cannot prove absence or equivalence.
|
||||
|
||||
Artifact comparison includes an image's payload interpretation, including load
|
||||
and entry declarations. Equivalent external offset forms and inventory positions
|
||||
leave those declarations unchanged. Unavailable interpretations remain unknown,
|
||||
while independently known changes survive; a changed row with an `availability`
|
||||
dimension does not establish the unavailable field's value.
|
||||
|
||||
For supporting facts, batch selected paths rather than requesting the entire
|
||||
inspection again:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_analysis_view",
|
||||
"arguments": {
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "COMPLETE_BASELINE_EVIDENCE_ID"
|
||||
},
|
||||
"view": {
|
||||
"kind": "inventory",
|
||||
"collection": "occurrences",
|
||||
"logical_paths": ["images/kernel-1.bin", "images/fdt-1.bin"],
|
||||
"fields": [
|
||||
"artifact_id",
|
||||
"logical_path",
|
||||
"hash_status",
|
||||
"source_location",
|
||||
"source_path"
|
||||
],
|
||||
"offset": 0,
|
||||
"limit": 64
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Read `collection: "nodes"` with those `artifact_ids` and `fields: ["sha256",
|
||||
"size"]` to obtain content facts. Join by `artifact_id`, not array position.
|
||||
Continue using the returned page metadata when necessary. Source locations are
|
||||
ranges in the original FIT, not decoded filesystem offsets. Complete Evidence
|
||||
remains available through the retained parent and `export_evidence_bundle`.
|
||||
Summary and field selection reduce delivery only; they do not skip inventory
|
||||
hashing or change coverage.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic fixtures with computed hashes cover embedded, `data-offset` and
|
||||
`data-position` layouts, shared data, overlap and bounds failures, every
|
||||
verified hash algorithm, configurations, resource limits, inventory,
|
||||
extraction and reader lifecycle. CLI and stdio MCP tests exercise the same
|
||||
artifact provider and advertised schemas. Real U-Boot `mkimage` 2023.01 output
|
||||
has also been verified for embedded, `-E -B 0x200`, and
|
||||
`-E -p 0x2000 -B 0x200` layouts, with byte-exact extraction checked against
|
||||
`dumpimage`. The OpenWrt 24.10.3 ABT ASR3000 recovery FIT was checked against
|
||||
`mkimage`, `fdtget` and `dumpimage`, including its kernel, initrd and devicetree.
|
||||
No image was executed.
|
||||
|
||||
For small containers, complete delivery avoids an additional view request.
|
||||
Choose summary and selected views when the omitted facts justify that extra
|
||||
request; summary delivery does not reduce the underlying inventory work.
|
||||
|
||||
The component comparison workflow was additionally checked on the official
|
||||
[OpenWrt 24.10.0](https://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic/)
|
||||
and [24.10.1](https://downloads.openwrt.org/releases/24.10.1/targets/mediatek/filogic/)
|
||||
`abt_asr3000-initramfs-recovery.itb` images (8,519,680 and 8,585,216 bytes).
|
||||
Their SHA-256 values are
|
||||
`083b174e81ed373f2d2f2ea5366f9291767d95375383528fd6b9955ad815b85a` and
|
||||
`41129cce68ad6269e5c8c48ef110f97682d6c4202ff2198bfd0d300c3dac132a`.
|
||||
An independent FDT byte-range/hash oracle found changed kernel and initrd bytes
|
||||
and an identical DTB; REA reported those two changed components, the changed
|
||||
outer `.` and one unchanged component. A controlled copy changing only the root
|
||||
timestamp reported only `.` changed and all three components unchanged. Public
|
||||
CLI comparison of the exported source Evidence matched the MCP Evidence exactly;
|
||||
summary and selected inventory views matched the independent ranges and hashes.
|
||||
These checks did not decompress either initrd or execute the firmware.
|
||||
@@ -109,13 +109,9 @@ server; older servers keep the complete request after schema discovery.
|
||||
header check does not establish valid table contents or exported symbols.
|
||||
- **`libflutter` facts** are per-ABI observations: byte size, SHA-256
|
||||
digest, GNU build IDs from ELF-declared `PT_NOTE` / `SHT_NOTE` ranges,
|
||||
and Dart/toolchain strings the build carries. `dart_version` accepts the
|
||||
labeled `Dart VM version:` form and the unlabeled Dart
|
||||
`Version::String()` form (`<version> (<channel>) (<commit time>)` with an
|
||||
optional ` on "<os>_<arch>"` suffix) that release engines embed. An
|
||||
unlabeled match is a candidate based on its shape; it is not bound to a
|
||||
Dart symbol and another string can match. `null` means no supported pattern
|
||||
was found, not proof that no version exists. Android clang/LLVM toolchain
|
||||
and any _labeled_ strings the build
|
||||
carries. Release engine builds often strip the `Dart VM version` string —
|
||||
absence is reported as `null`, not guessed. Android clang/LLVM toolchain
|
||||
lines are reported verbatim as build provenance.
|
||||
`build_id_candidates` preserves distinct IDs. `build_id` is populated only
|
||||
when all declared note ranges were examined and one ID was found;
|
||||
@@ -126,13 +122,10 @@ server; older servers keep the complete request after schema discovery.
|
||||
Compressed `SHT_NOTE` sections are skipped with an unsupported-compression
|
||||
limitation and partial coverage; IDs from other note ranges remain candidates.
|
||||
Encoded note storage is never interpreted as note headers or inflated.
|
||||
- Version and toolchain strings must fit their complete grammar.
|
||||
`dart_version_capped` and `toolchain_capped` independently report labels
|
||||
omitted by grammar or projection bounds; either makes identification
|
||||
coverage partial. Values split across read chunks are retained without
|
||||
publishing truncated prefixes. Unlabeled runs that miss the
|
||||
`Version::String()` grammar are ignored and do not set
|
||||
`dart_version_capped`.
|
||||
- Labeled strings must fit their complete grammar. `dart_version_capped`
|
||||
and `toolchain_capped` independently report labels omitted by grammar or
|
||||
projection bounds; either makes identification coverage partial. Values
|
||||
split across read chunks are retained without publishing truncated prefixes.
|
||||
- **`coverage: "partial"`** also appears when an ABI carries `libapp.so`
|
||||
without `libflutter.so` (or vice versa).
|
||||
- The APK itself is digested before parsing, so `target.sha256` binds the
|
||||
File renamed without changes.
+1
-1
@@ -119,7 +119,7 @@ installation's language catalog, the same catalog REA uses to admit
|
||||
and still succeeds on the stock checks.
|
||||
|
||||
The ELF lane (`npm run verify:ghidra:ppc`) is separate. See
|
||||
[PowerPC ELF analysis through Ghidra](ghidra-ppc.md).
|
||||
[PowerPC ELF analysis through Ghidra](/ghidra-ppc).
|
||||
|
||||
## Not supported yet
|
||||
|
||||
|
||||
@@ -175,9 +175,8 @@ are separate from the initialized MZ module. Headerless COM requires the
|
||||
from the filename or unrecognized bytes. DOS extenders without an admitted
|
||||
MZ real-mode entry remain outside this import boundary.
|
||||
|
||||
DOS MZ is available through Ghidra, with no Hopper support claim. Windows Ghidra
|
||||
P0 admits DOS MZ read-only: function annotations and naming edits are refused
|
||||
there, and explicit COM is not admitted. macOS x64 DOS verification remains unverified.
|
||||
DOS MZ is available through Ghidra, with no Hopper support claim. It is outside
|
||||
the Windows native PE P0 boundary. macOS x64 DOS verification remains unverified.
|
||||
|
||||
## Real verification
|
||||
|
||||
@@ -39,9 +39,9 @@ bridge-reported `language_id` after import. A caller-selected
|
||||
pin; combining the pinned profile with an environment override is a typed
|
||||
refusal. A header that declares an ABI field but carries no record is still
|
||||
refused. Two further boundaries apply: an inspection that could not examine
|
||||
every ABI declaration source does not establish the recordless admission
|
||||
(target opening already refuses a carved ELF whose section table extends past
|
||||
the captured bytes, as described below), and an `EF_MIPS_ARCH_1` or
|
||||
every ABI declaration source (for example a carved ELF whose section table
|
||||
extends past the captured bytes without a program-header record) does not
|
||||
establish the recordless admission, and an `EF_MIPS_ARCH_1` or
|
||||
`EF_MIPS_ARCH_32` header that _declares_ an ABI is refused rather than
|
||||
relabeled, because the declared-ABI lane is verified for
|
||||
`EF_MIPS_ARCH_32R2` headers only. The recordless MIPS-I lane pins the same
|
||||
@@ -77,16 +77,13 @@ argument ABI and unknown nonzero MSA values remain unsupported. Without a
|
||||
separate ABI flags record, FP attribute values are admitted only when absent,
|
||||
`ANY` (0), double precision (1), or FPXX (5), the values covered by the
|
||||
recordless profile. Parsed values are retained in the MIPS analysis profile.
|
||||
A carved ELF whose section header table extends beyond the captured file is
|
||||
refused when the target is opened, before provider selection, as is every ELF
|
||||
declaration beyond EOF ([installation](../getting-started/installation.md#ghidra)). The refusal
|
||||
names the declared range and the file size. It applies even when a
|
||||
program-header ABI record was captured: an uncaptured section header could
|
||||
carry an unsupported or conflicting declaration, so the captured bytes are not
|
||||
analyzed as a complete smaller program. The provider also refuses any ABI
|
||||
declaration inspection that reports unexamined headers. A missing
|
||||
program-header record does not make ABI declarations in uncaptured sections
|
||||
absent or supply a default ABI.
|
||||
For carved ELFs whose section table extends
|
||||
beyond the captured file, REA examines every complete captured section header
|
||||
and records the unavailable headers as a limitation. The provider refuses an
|
||||
incomplete ABI declaration inspection even when a program-header ABI record is
|
||||
valid, because an unavailable section header could carry an unsupported or
|
||||
conflicting declaration. A missing program-header record does not make ABI
|
||||
declarations in uncaptured sections absent or supply a default ABI.
|
||||
The field layout follows
|
||||
[`Elf_MIPS_ABIFlags_v0`](https://github.com/bminor/glibc/blob/master/elf/elf.h).
|
||||
|
||||
@@ -260,13 +257,11 @@ observations and does not relabel the upstream result as verified.
|
||||
|
||||
CLI/MCP discovery and decompilation, direct calls, target/profile/Evidence
|
||||
identity, snapshot round-trip and final process cleanup remain covered.
|
||||
For both byte orders, the lane also cuts the image one byte before its section
|
||||
header table, after checking that every program header and payload stays
|
||||
captured. `open_binary` and the CLI must both refuse that copy before provider
|
||||
selection as `target_unavailable` with constraint `file_format_rejected`,
|
||||
naming the declared section-table range and the captured size, and leave it
|
||||
unchanged. Caller-selected Ghidra heap limits are passed to the lane's CLI and
|
||||
MCP child workflows.
|
||||
For both byte orders, the lane also carves away the section table while retaining
|
||||
all program-header payloads. It checks the actual Ghidra language, entry-address
|
||||
decompilation, distinct artifact digest and missing-section limitations through
|
||||
CLI Evidence and an MCP-produced snapshot. Caller-selected Ghidra heap limits
|
||||
are passed to these child workflows.
|
||||
For soft-float fixtures the lane also checks, through `inspect_native_api`, the
|
||||
exact parameter and return types and storage of float, double, mixed (pair
|
||||
alignment and stack spill) and struct-returning functions; that a caller's
|
||||
@@ -8,7 +8,7 @@ labels, functions, or memory bytes. The hydration semantics are adapted from
|
||||
`effeb734fc570c32650f88b159608979dc7b423e` (MIT), cross-checked against
|
||||
.NET's `MethodTable.h` at runtime commit
|
||||
`a3b2d40328be0be3f8dd43f0e5cc925b8827b044` (MIT). The required notice ships in
|
||||
[`licenses/nativeaot-parser-MIT.txt`](https://github.com/morluto/rea/blob/main/licenses/nativeaot-parser-MIT.txt).
|
||||
[`licenses/nativeaot-parser-MIT.txt`](../licenses/nativeaot-parser-MIT.txt).
|
||||
|
||||
The optional headless Ghidra adapter remains available for database annotations
|
||||
and broader recovery. It uses the same pinned upstream commit as an opt-in
|
||||
@@ -158,7 +158,7 @@ does not imply that ordinary native disassembly/decompilation is impossible.
|
||||
|
||||
## Verification
|
||||
|
||||
See [the dedicated lane](../development/testing.md#optional-nativeaot-ghidra-analysis).
|
||||
See [the dedicated lane](testing.md#optional-nativeaot-ghidra-analysis).
|
||||
Fixtures are source-built benign .NET programs with inheritance, an interface,
|
||||
virtual dispatch, representative arithmetic and frozen strings. Independent
|
||||
compiler/linker symbols and raw directory bytes provide the oracle. Stripped
|
||||
@@ -2,28 +2,19 @@
|
||||
|
||||
REA imports one target into an ephemeral Ghidra database. Linux/macOS support
|
||||
read-only analysis and atomic function-name/entry-comment edits; Windows P0 is
|
||||
read-only. GUI controls and writes to caller-selected source databases are unavailable.
|
||||
read-only. GUI controls and persistent database mutation are unavailable.
|
||||
Executable bytes remain unchanged.
|
||||
|
||||
An opt-in [owned Program checkpoint](../getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program)
|
||||
retains analysis across Linux/macOS processes. It saves initial completed analysis
|
||||
and normal-close metadata changes to `REA_GHIDRA_CHECKPOINT_DIR`; cancellation
|
||||
and process failure recover only the last successfully saved state. The restored
|
||||
packed Program's digest is reported with restored observations. Initial analysis
|
||||
keeps one stable generation through its first save; later saved states bind their
|
||||
content digest in profile identity. Restoration does not turn prior
|
||||
database state into a fresh discovery. The default remains ephemeral.
|
||||
|
||||
Use [installation](../getting-started/installation.md#ghidra) for supported hosts, prerequisites,
|
||||
Use [installation](installation.md#ghidra) for supported hosts, prerequisites,
|
||||
configuration, startup deadlines, resource controls, and cleanup recovery.
|
||||
[Linux Ghidra launches](../getting-started/installation.md#linux-owned-process-prerequisite) also
|
||||
[Linux Ghidra launches](installation.md#linux-owned-process-prerequisite) also
|
||||
require a procps-compatible `ps` on REA's `PATH` for process ownership checks.
|
||||
[Windows P0](windows-ghidra-p0.md) has its own admission and lifecycle boundary;
|
||||
[NativeAOT observations](ghidra-nativeaot.md) include a built-in read-only PE
|
||||
snapshot parser and a separate optional annotation extension. Check the
|
||||
[release boundary](../getting-started/installation.md#released-package-and-main) when using npm.
|
||||
[release boundary](installation.md#released-package-and-main) when using npm.
|
||||
|
||||
See [import language and analysis seeds](../getting-started/installation.md#ghidra-import-language-and-analysis-seeds)
|
||||
See [import language and analysis seeds](installation.md#ghidra-import-language-and-analysis-seeds)
|
||||
to override the header-detected language or seed analysis. An override that conflicts with a fixed raw, DOS, PSP, MIPS, RISC-V or GameCube DOL import
|
||||
is a configuration error naming the conflicting settings. It does not make an
|
||||
otherwise admitted target unsupported; REA preserves the requested settings.
|
||||
@@ -40,8 +31,7 @@ operation-scoped JVM `OutOfMemoryError` observations retain their separate resou
|
||||
classification. Automatic kernel or cgroup OOM attribution is not implemented.
|
||||
|
||||
After the authenticated startup handshake, rejected inputs (`invalid_request`,
|
||||
`not_found`, `ambiguous`, an invalid function name, or an address without one
|
||||
exact file offset) use
|
||||
`not_found`, `ambiguous`, or an invalid function name) use
|
||||
`diagnostics.diagnostic_scope: "request"` when their active output checkpoint is
|
||||
available. They preserve the active request's
|
||||
`failure_output`, target/profile identity and runtime/log coordinates, while
|
||||
@@ -52,7 +42,7 @@ not clear it or alter session readiness. Log paths refer to the owned ephemeral
|
||||
runtime and are removed by normal cleanup.
|
||||
|
||||
Provider selection and profile identity follow
|
||||
[MCP contracts](../reference/mcp-contracts.md#identity-and-discovery). One deep
|
||||
[MCP contracts](mcp-contracts.md#identity-and-discovery). One deep
|
||||
provider stays bound to the target; runtime failure never selects another
|
||||
provider automatically. The contracts below describe Ghidra observations,
|
||||
not equivalence with another engine's pseudocode or analysis database.
|
||||
@@ -89,13 +79,6 @@ does not prove text is absent. Both list and search Evidence carry this
|
||||
limitation. When an address is known, use `read_bytes` and `xrefs` to inspect
|
||||
bytes and references.
|
||||
|
||||
String searches and `trace_feature` retain each matched row's Ghidra-reported
|
||||
encoding, termination and byte length. These describe the database's defined
|
||||
data, not independent proof of the original source encoding. Procedure searches
|
||||
and trace matches likewise retain Ghidra's external and thunk classification.
|
||||
The Ghidra analysis profile identifies this search projection, so older
|
||||
snapshots with unqualified search rows cannot replay under the new profile.
|
||||
|
||||
## PE debug-symbol coverage
|
||||
|
||||
The post-analysis handshake reports Ghidra's PDB Program Information attributes:
|
||||
@@ -140,11 +123,6 @@ also accept source-file byte offsets. Existing symbols and analysis addresses
|
||||
take precedence; a `0x`-prefixed number that resolves to no procedure falls back
|
||||
to its observed source mapping. Use `--file-offset=0x...` on the CLI or
|
||||
`procedure: "file-offset:0x..."` through MCP to select file coordinates explicitly.
|
||||
To select an analysis address without the bare-hex fallback, qualify it with its
|
||||
Ghidra address space, for example `ram:0x...`. The `inspect-native-load-image`
|
||||
CLI command or MCP `inspect_native_load_image` reports Ghidra's observed default
|
||||
address space; use the space appropriate to the address, and retain any
|
||||
load-image verification limitations.
|
||||
Multiple mapping candidates fail as ambiguous rather than selecting a bank or
|
||||
address space. A successful conversion retains the file offset, mapped address
|
||||
and procedure entry in raw Evidence and `locations`.
|
||||
@@ -193,11 +171,6 @@ have no file backing. ELF mappings are observed
|
||||
without independent format-specific load-image verification, as the result's
|
||||
status and limitations state.
|
||||
|
||||
Relocation rows retain Ghidra's `symbol` value, including an empty string or
|
||||
null when returned by Ghidra. It is a provider observation, not proof of the
|
||||
runtime-resolved address. Older Evidence may not have recorded this field;
|
||||
a relocation page requesting it reports that uncertainty.
|
||||
|
||||
For large relocation inventories, select MCP `inspect_native_load_image` with
|
||||
`detail: "summary"`. It returns verification state, observed identity and counts,
|
||||
while retaining complete Evidence. Use `normalized_result.parent_evidence_id`
|
||||
@@ -276,21 +249,9 @@ Runtime updates apply to newly accepted operations on the existing Program.
|
||||
They preserve accepted budgets and do not clear cleanup blockers or change the
|
||||
client deadline. Transient decompile timeout failures remain uncached so the
|
||||
same function can be retried after raising the limit. See the
|
||||
[settings design](../design/limits-and-recovery.md) for snapshots, precedence,
|
||||
[settings design](design/limits-and-recovery.md) for snapshots, precedence,
|
||||
and provider integration.
|
||||
|
||||
When `inspect_native_instruction` selects a byte inside a listed instruction,
|
||||
`containing_instruction` reports its separate start, length and Ghidra Listing
|
||||
text, retaining any non-default address space. The requested address remains
|
||||
unchanged, and a containing function may be absent. For defined data, the same
|
||||
tool reports its containing object's start, length and type path; use the start
|
||||
with `inspect_native_data_type` to inspect its layout. This does not select a
|
||||
field or union member. These are listing observations, not proof of execution or
|
||||
an alternate decode. Earlier Evidence can
|
||||
omit the field. The analysis-profile revision prevents older snapshots from
|
||||
replaying results without it; open the target without the old snapshot and save
|
||||
a fresh snapshot when needed.
|
||||
|
||||
Function extents and instruction completeness refer to Ghidra's analyzed body.
|
||||
An inferred no-return call can terminate that body before subsequent bytes.
|
||||
Pseudocode, assembly, procedure info, instruction windows, incoming and outgoing
|
||||
@@ -338,7 +299,7 @@ host/provider acceptance run with only the host compiler.
|
||||
`--dll`. These static fixtures contain source-owned encoded instructions; they
|
||||
are not compiler-produced programs. See the [Windows verification scope](windows-ghidra-p0.md#native-authority-and-target-identity)
|
||||
for real-provider coverage and exclusions.
|
||||
These lanes check all 27 read-only operations, target/snapshot/import SHA-256 linkage,
|
||||
These lanes check all 26 read-only operations, target/snapshot/import SHA-256 linkage,
|
||||
and project, endpoint, process, and runtime cleanup. The independent native
|
||||
lane checks DACLs, handle admission, cancellation, and Job Object lifecycle.
|
||||
`npm run verify:ghidra:windows:package` packs and installs REA into an isolated
|
||||
@@ -359,22 +320,13 @@ must be constrained.
|
||||
|
||||
Optional `start` and `end` addresses are inclusive and select one address space.
|
||||
A match must end at or before `end`, so `end` set inside a match excludes it.
|
||||
Without bounds, loaded non-overlay memory blocks are eligible, in Ghidra address
|
||||
order, including across adjacent initialized blocks. Blocks that are not loaded
|
||||
(file parts such as `.symtab`) are listed in `searched.skipped_not_loaded`;
|
||||
loaded overlay blocks are listed separately in `searched.skipped_overlays`.
|
||||
Neither is searched by default; pass `start` or `end` in its address space to
|
||||
search it. A block that is both unloaded and an overlay is listed once under
|
||||
`skipped_not_loaded`.
|
||||
The separate overlay field changes the byte-search result contract and the
|
||||
Ghidra analysis profile. Refresh cached MCP schemas; snapshots from the older
|
||||
profile are not replayed. Previously exported Evidence remains importable as
|
||||
historical data with its original result shape.
|
||||
Overlapping matches are included, including across adjacent eligible initialized
|
||||
blocks in the same address space. Gaps and excluded or uninitialized blocks break
|
||||
a search run. Each match reports its address, starting segment, unique source file
|
||||
offset for that starting address (or null), and containing procedure (or null).
|
||||
A starting offset does not imply the full match is contiguous in the file.
|
||||
Without bounds, loaded memory blocks are eligible, in Ghidra address order.
|
||||
Blocks that are not loaded (file parts such as `.symtab`) and overlay blocks are
|
||||
listed in `searched.skipped_not_loaded` and are not searched; pass `start` or
|
||||
`end` in their address space to search them.
|
||||
Overlapping matches are included; matches never cross block boundaries.
|
||||
Each match reports its address, segment, unique source file offset (or null),
|
||||
and containing procedure (or null).
|
||||
|
||||
Pages default to 100 matches and accept 1 to 1000. Pass `next_cursor` as `cursor`
|
||||
with the same target, profile, pattern, mask and bounds to continue. `complete`
|
||||
Loaded 100 of 3571 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user