mirror of
https://github.com/morluto/rea.git
synced 2026-11-08 08:11:31 -10:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9cb63fd1d1 | ||
|
|
0567e6bc7b | ||
|
|
5736bd9f94 |
No files matched your search
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"name": "rea",
|
||||
"interface": {
|
||||
"displayName": "REA"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "rea",
|
||||
"source": {
|
||||
"source": "npm",
|
||||
"package": "rea-agents",
|
||||
"version": "6.8.0"
|
||||
},
|
||||
"policy": {
|
||||
"installation": "AVAILABLE",
|
||||
"authentication": "ON_INSTALL"
|
||||
},
|
||||
"category": "Developer Tools"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: rea-tool-design
|
||||
description: Audit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
|
||||
description: Design or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
|
||||
---
|
||||
|
||||
# REA tool design
|
||||
@@ -10,20 +10,6 @@ contract semantics, provider ownership, and discoverability. Inspect the current
|
||||
contract and nearest existing tool relevant to the analyst question; source,
|
||||
documentation, and live catalogs may describe different versions.
|
||||
|
||||
For provider-backed work, exercise the analyst's task through REA and compare
|
||||
relevant native provider and independent artifact observations where available.
|
||||
Check applicable partial results, recovery and session boundaries; report what
|
||||
was not exercised. For failure guidance, follow [design recovery around the
|
||||
failure](../../../docs/tool-design.md#design-recovery-around-the-failure).
|
||||
Use [workflow verification](../../../docs/testing.md#provider-backed-workflow-verification)
|
||||
to select evidence for the proposed contract instead of seeking provider API
|
||||
parity.
|
||||
|
||||
For an audit, follow [audit existing tools](../../../docs/tool-design.md#audit-existing-tools):
|
||||
scope representative tasks, exercise the live workflow, and distinguish
|
||||
reproduced bugs, supported limitations, design opportunities and unverified
|
||||
claims. Report working alternatives and evidence limits as well as findings.
|
||||
|
||||
For a design request, return the proposed boundary, input/result shape, Evidence
|
||||
semantics, nearest alternative, and verification approach. Implement when
|
||||
requested, preserving the user's scope and existing authorization.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "59"
|
||||
version: "49"
|
||||
---
|
||||
|
||||
# REA
|
||||
@@ -17,9 +17,6 @@ readiness or provider commands.
|
||||
|
||||
Installing this skill supplies instructions; it does not register the REA MCP
|
||||
server, install analysis engines, or add tools to an already-running agent.
|
||||
The exception is a Claude Code or Codex plugin install: the plugin bundles
|
||||
this skill and registers its own `rea` server, outside the state that
|
||||
`rea setup` and `rea doctor` manage.
|
||||
|
||||
If REA tools are available and their registration is not known to be stale,
|
||||
proceed directly to the target. Do not run diagnostics or setup before every
|
||||
@@ -40,23 +37,42 @@ repeat or known not to have started. If it may already have changed the
|
||||
target, or it fails again, tell the user and ask them to restart the MCP
|
||||
client. If REA could not start its connection, retry once, then tell the user
|
||||
and ask them to restart the MCP client. Do not retry a startup failure that
|
||||
returns the same unchanged result. For plugin installs, follow
|
||||
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin).
|
||||
For setup-owned registrations, run `rea doctor` when REA could not start
|
||||
returns the same unchanged result. Run `rea doctor` when REA could not start
|
||||
or tools stay missing after that restart. For `cleanup_incomplete`, a retryable
|
||||
capacity limit, or `configuration_invalid`, follow the recovery guide's ladder
|
||||
before asking for a restart. Read that guide before a second repair attempt.
|
||||
|
||||
Choose the installation owner before repairing registration. Plugin installs use
|
||||
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin);
|
||||
only setup-owned registrations use the guide's
|
||||
[registration repair plan](references/connection-and-recovery.md#repair-registration-when-needed).
|
||||
A plugin-only doctor result of `missing` does not call for `rea setup`.
|
||||
1. Diagnose without changing REA or client configuration. `npx` may download
|
||||
the package and write to npm's cache. For Codex, run
|
||||
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
|
||||
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
|
||||
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
|
||||
`commandcode`, `vscode`, `grok_build`, `omp`, or `grok_bot`. `grok_bot` has no local
|
||||
file registration: doctor reports the Grok Bot chat step, and setup does not
|
||||
write `mcp.json`. If the client is unknown, use `doctor --json` and
|
||||
inspect its registration results before choosing a setup scope.
|
||||
2. Distinguish the reason. Missing, malformed, or stale registration needs a
|
||||
scoped configuration repair. An aligned registration with no tools in the
|
||||
active session needs a restart/reconnection; doctor cannot prove that the
|
||||
current agent has connected. A missing provider affects only tasks requiring
|
||||
that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
|
||||
and Android inspection has separate bring-your-own JADX/Java prerequisites.
|
||||
3. For a configuration repair, prepare the read-only plan:
|
||||
`npx -y rea-agents@latest setup --client codex --dry-run --json`.
|
||||
Use the current client's ID, show its exact proposed paths, backups, and
|
||||
changes, and obtain approval before setup writes configuration or installs
|
||||
Hopper. Setup normally installs the matching bundled skill too; include that
|
||||
replacement in the reviewed plan. After approval, apply the same scope with
|
||||
`npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
|
||||
only if that separate installation was needed and explicitly approved.
|
||||
4. Restart/reconnect the affected agent. Verify that REA tools actually appear
|
||||
in the session, then resume the original investigation. If they remain
|
||||
absent, inspect the client's MCP launch error rather than repeating setup.
|
||||
|
||||
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
|
||||
JADX, Kuna, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
|
||||
software to repair MCP registration. For an unsupported client, use manual stdio
|
||||
registration with an existing installation’s absolute runtime and entry script, or continue through the CLI.
|
||||
registration with a version-pinned `rea-agents` package or continue through the CLI.
|
||||
|
||||
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
|
||||
|
||||
@@ -69,11 +85,8 @@ graph, limitations, and unknowns with the same care as an MCP result; the CLI
|
||||
returns the Evidence record directly. This fallback does not establish that
|
||||
MCP is configured. Native CLI tasks still require their selected engine.
|
||||
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
|
||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place from a
|
||||
trusted directory outside the target project, with absolute target paths. Managed
|
||||
setup binds the current installation directly; retain its runtime and package
|
||||
cache entry, or restore that installation before repairing its registration.
|
||||
For setup-owned registration, configuration repair starts with a read-only setup plan; show paths, backups and
|
||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
|
||||
Configuration repair starts with a read-only setup plan; show paths, backups and
|
||||
skill replacement before applying authorized changes. Preserve existing approval
|
||||
and target-execution boundaries.
|
||||
|
||||
@@ -94,9 +107,6 @@ explicit path or endpoint and do not need it.
|
||||
than extra calls and schemas. Runtime options are metadata, not files.
|
||||
Compact delivery still inventories and hashes members. See the
|
||||
[Python packaged applications guide](https://github.com/morluto/rea/blob/main/docs/python-packaged-applications.md).
|
||||
- Bun `--compile` executable: `open_binary`, then `inspect_artifact`; the Bun
|
||||
module graph is the application layer. Extract stored modules and analyze the
|
||||
JavaScript. See [Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
|
||||
`open_bundle_resources`, then the selected resource tool. This does not
|
||||
require `CFBundleExecutable` or a deep provider. Resource sessions have no
|
||||
@@ -109,8 +119,6 @@ explicit path or endpoint and do not need it.
|
||||
`extract_firmware` when extraction is requested, with a caller-selected new
|
||||
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
|
||||
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
|
||||
- ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact`
|
||||
without a provider; see the ESP section of `references/native-and-artifacts.md`.
|
||||
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
|
||||
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
|
||||
optional metadata recovery and its supported host/layout boundary.
|
||||
@@ -150,9 +158,7 @@ explicit path or endpoint and do not need it.
|
||||
and field names/types before choosing fields for `inspect_packet_capture`.
|
||||
Query/page the relevant records; registrations do not establish observed traffic.
|
||||
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and
|
||||
TShark 4.x on an absolute `PATH` entry or via the optional absolute
|
||||
`REA_TSHARK_COMMAND` override; see
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
|
||||
Packet limits count input packets before filtering. Output defaults to 50 matching
|
||||
frames; use `match_offset` / `match_limit` for later pages, or explicit null
|
||||
@@ -338,11 +344,6 @@ leaves its database open. The modern headless profile creates and closes an
|
||||
owned private database without saving. Installation and registration reuse
|
||||
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
|
||||
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
|
||||
For migration to the separate native IDA MCP plugin, follow the guide's
|
||||
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/ida-provider.md#migrate-a-legacy-attached-registration).
|
||||
Replace the upstream registration explicitly; modern mrexodia tools are not the
|
||||
native profile. Native attachment preserves the database but does not provide
|
||||
direct callers; check session availability and the guide's verification limits.
|
||||
Proceed directly to `analyze_function`, pseudocode, or function/string searches
|
||||
after opening; IDA does not supply `binary_overview`. Consult session availability
|
||||
when composing broader workflows. Modern direct callers and unsupported dossier
|
||||
@@ -377,9 +378,8 @@ unless the group overrides them. The grouped name union is the complete tool
|
||||
inventory; use `tools/list` for canonical tool schemas. The default summary keeps
|
||||
each provider candidate's availability and target-support status, code and
|
||||
reason; `omitted_fields` lists what it leaves out. Request `{"detail": "complete"}`
|
||||
only when you need an unavailable candidate's remediation, provider
|
||||
diagnostics, candidate capability sets, or tool surface, effect and annotation
|
||||
flags; in those closed flag vocabularies, listed
|
||||
only when you need provider diagnostics, candidate capability sets, or tool
|
||||
surface, effect and annotation flags; in those closed flag vocabularies, listed
|
||||
flags are true and omitted known flags are false. Provider candidate facts
|
||||
describe possible providers, while top-level session capabilities describe the
|
||||
current route and `analysis_provider_binding` names the selected provider. Do
|
||||
@@ -402,18 +402,9 @@ call `analyze_javascript_application` or `inspect_binary_layout` with
|
||||
a flag: its default reply is the summary, and the complete dossier stays
|
||||
retained under `parent_evidence_id` for facet paging. Pass that ID as a
|
||||
`retained-evidence` reference to `compare_functions`, or inside
|
||||
`match_native_functions` `left_observations`/`right_observations` or
|
||||
`build_call_path.functions`; request `"detail": "complete"` only when the
|
||||
dossier must travel inline.
|
||||
Use the same exact-reference form in other advertised logical Evidence inputs,
|
||||
including optional sources, arrays and nested sources across domains. Each
|
||||
source can independently be inline or retained; preserve its source identity,
|
||||
coverage, partial facts and unknowns. A summary or inspection page has its own
|
||||
Evidence ID: use `normalized_result.parent_evidence_id` when the consumer needs
|
||||
the complete source. For portable CLI follow-ups, commands advertising
|
||||
`--evidence-bundle` resolve references from that selected exported bundle.
|
||||
See [Evidence composition](references/evidence-workflows.md) for scope and
|
||||
recovery; file selectors are supported only where explicitly advertised.
|
||||
`match_native_functions` `left_observations`/`right_observations`; request
|
||||
`"detail": "complete"` only when the dossier must travel inline, as
|
||||
`build_call_path` requires.
|
||||
Page module identities and select the returned `node_id` before requesting an
|
||||
item; a single file path can represent several graph roles. Do not repeat an
|
||||
identical analysis or rejected call without a changed input or resolved blocker.
|
||||
@@ -432,10 +423,7 @@ output also streams without a single MCP frame.
|
||||
When the retained parent ID is unknown, use `get_evidence_bundle` with
|
||||
`detail: "summary"` and exact filters for operation, target SHA-256,
|
||||
analysis-profile digest or canonical procedure address. Select its returned
|
||||
Evidence ID for the next view instead of requesting the complete bundle. When
|
||||
that record has no supported view (for example an annotation), read it with
|
||||
`detail: "records"` and an `evidence_id` filter; `missing_evidence_links`
|
||||
names linked sources this session does not retain.
|
||||
Evidence ID for the next view instead of requesting the complete bundle.
|
||||
`retention: "complete-record"` means the whole record is stored; native
|
||||
value-flow truncation and unknown metadata remain separate. For an exported
|
||||
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
|
||||
@@ -518,8 +506,4 @@ as complete while required questions remain open.
|
||||
Explain findings in plain language and tie them to returned evidence. When the
|
||||
user asks to build something, use normal coding tools and separate observed
|
||||
behavior from design choices. Close an opened native session with
|
||||
`close_binary` when its engine is no longer needed. Evidence references and Unknown
|
||||
history remain in memory on the same MCP connection. End the investigation with
|
||||
`reset_investigation`; supply `save_path` to checkpoint the complete bundle before
|
||||
clearing. Import the saved bundle to restore original references and Unknown
|
||||
history in a new process. Retention is bounded and is not automatic persistence.
|
||||
`close_binary` when the investigation is complete.
|
||||
@@ -13,27 +13,6 @@ own APK path and explicit selectors; do not open a native session first.
|
||||
Choose an `overload_index` from the class inventory when methods are ambiguous
|
||||
instead of guessing.
|
||||
|
||||
For JNI implementations, discover `inspect_android_jni_bindings`. Select raw
|
||||
DEX class/method identities and optional descriptors, ABI or logical library
|
||||
paths. Prefer `detail: "summary"`; complete Evidence remains retained and
|
||||
`android-jni` views page declarations, libraries, candidates or binding rows
|
||||
with exact fields. Global native-pair ambiguity survives class selection.
|
||||
Preserve slot/relocation provenance and source coverage: matched means observed
|
||||
candidate presence; partial scans do not establish uniqueness or absence.
|
||||
This reads DEX/ELF bytes without loading the APK into the decompiler or executing
|
||||
target code. Runtime registration and loading remain unknown. See the
|
||||
[JNI guide](https://github.com/morluto/rea/blob/main/docs/android-analysis.md#link-native-declarations-to-jni-candidates).
|
||||
|
||||
For an R8/ProGuard release build with its `mapping.txt`, pass the absolute
|
||||
`mapping_path` to `inspect_android_class` or `inspect_android_method`. Prefer
|
||||
`detail: "summary"` to receive normalized symbols or the selected body without
|
||||
duplicate raw class metadata. Complete Evidence stays retained under the linked
|
||||
parent ID and can be exported; CLI summary delivery requires `--evidence_output`.
|
||||
Omitted or complete detail preserves the full inline response. Keep
|
||||
selecting by observed names; the separate `mapping` block reports each symbol
|
||||
as `mapped`, `ambiguous` or `unmapped` after checking DEX types and prototypes.
|
||||
REA does not verify that the mapping belongs to the APK build.
|
||||
|
||||
The provider requires a separately supplied headless JADX MCP JAR 0.7.1 and
|
||||
Java 17 or newer. `REA_JADX_MCP_JAR` selects the absolute JAR path; `JAVA_HOME`
|
||||
can select Java. Use `rea doctor --provider jadx --json` to check the JADX
|
||||
|
||||
@@ -17,7 +17,7 @@ does not establish which server an approval wildcard covers. See
|
||||
|
||||
| Observation | Meaning and next step |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| No REA tools in the active session | If REA was installed as a client plugin, follow the plugin section below. Otherwise inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| MCP connection dropped or REA could not start | If the connection drops, retry that call once only when it is safe to repeat or known not to have started. If it may already have changed the target, or the same failure returns, tell the user REA's connection failed and ask them to restart the MCP client. If REA could not start its connection, retry once, then tell the user and ask them to restart the MCP client. Do not retry a startup failure that returns the same unchanged result. Run `rea doctor` when REA could not start or tools stay missing after restart. Doctor cannot restore a live connection by itself. |
|
||||
@@ -116,32 +116,6 @@ is not a recommendation to change it. Do not invent a setting. Host memory and
|
||||
- `invalid_request`: correct the reported fields from the current schema. Do
|
||||
not repeat the unchanged call.
|
||||
|
||||
## REA installed as a client plugin
|
||||
|
||||
A Claude Code or Codex plugin owns its `rea` server and this skill. Doctor's
|
||||
`--client` and `--skill` checks and `rea setup` read and write only
|
||||
setup-owned state, so for a plugin-only install they report `missing`, and
|
||||
applying setup would add a second `rea` server. If this skill was loaded from
|
||||
a plugin, or the client lists a `rea` plugin, do not prepare a setup plan for
|
||||
missing tools. Instead:
|
||||
|
||||
- Check that the plugin is installed and enabled with the client's own
|
||||
listing: `claude plugin list` and `/mcp` in Claude Code, `codex plugin list`
|
||||
in Codex.
|
||||
- Restart or reconnect the client, then read its launch error for the `rea`
|
||||
server if tools are still absent. The plugin starts its bundled runtime; a missing build or version mismatch
|
||||
requires a reinstall, or a fresh build when developing from source.
|
||||
- Check analysis engines through the connected server's
|
||||
`binary_session.result.tool_availability`. `rea doctor --provider <id> --json`
|
||||
from a shell reports that shell's environment, which can differ from the
|
||||
environment the client gave the plugin server. If you still run it through
|
||||
the package runner, use the version pinned in the plugin manifest's server
|
||||
entry (`rea-agents@<version>`), not `@latest`, so it matches the server the
|
||||
plugin starts.
|
||||
|
||||
Use the setup repair below only when the user wants a setup-owned registration
|
||||
instead of the plugin, and remove or disable the plugin first.
|
||||
|
||||
## Repair registration when needed
|
||||
|
||||
For Codex:
|
||||
@@ -156,10 +130,7 @@ Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
|
||||
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `trae`, `grok_build`, `omp`, `pi`,
|
||||
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, `freebuff`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
||||
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
|
||||
For unsupported clients, use manual stdio registration with the selected
|
||||
installation’s absolute runtime and entry script, or continue through the CLI.
|
||||
Acquire packages from a trusted directory outside the target project. Keep a
|
||||
package-runner installation and its runtime available while registrations use it.
|
||||
For unsupported clients, use version-pinned manual stdio registration or CLI.
|
||||
|
||||
Show the plan's exact paths, backups, changes and bundled-skill replacement.
|
||||
Obtain approval for writes when that concrete scope is not already authorized,
|
||||
|
||||
@@ -23,14 +23,6 @@ again if a hint names a gap, then writes the bundle with
|
||||
- Map the application first with `inspect_web_page` (and `capture_web_screenshot`
|
||||
when layout matters): the login form's locators, and the API calls behind the
|
||||
pages that list and edit the entities the user asked about.
|
||||
- `inspect_web_page` cannot sign in. To see pages behind sign-in, run a short
|
||||
`capture_browser_scenario` that signs in and opens them, with
|
||||
`"capture": {"after_each_step": ["accessibility"]}` (`"dom"` for markup).
|
||||
Read a step's tree with `inspect_analysis_view`, `view: {kind:
|
||||
"browser-scenario", collection: "text", step_index, artifact:
|
||||
"accessibility", offset: 0, limit: 2048}`, paging with `offset`. Capture
|
||||
options are only `after_each_step`, `at_end`, `events`, `network` and
|
||||
`limits`.
|
||||
|
||||
## Record a session an environment can learn writes from
|
||||
|
||||
@@ -88,123 +80,62 @@ shape, with the application's own locators, paths and headers:
|
||||
"path": "/api/folders",
|
||||
"expected_status": [200]
|
||||
},
|
||||
{
|
||||
"step_id": "new-folder-1",
|
||||
"action": "click",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "role", "role": "button", "name": "New folder" }
|
||||
},
|
||||
{
|
||||
"step_id": "folder-title-1",
|
||||
"action": "fill",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=title]" },
|
||||
"value": { "source": "literal", "value": "Scratch folder 1" }
|
||||
},
|
||||
{
|
||||
"step_id": "create-folder-1",
|
||||
"action": "click",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "role", "role": "button", "name": "Create" },
|
||||
"save_response": {
|
||||
"method": "POST",
|
||||
"path": "/api/folders",
|
||||
"save": [{ "name": "folder_1", "json_pointer": "/id" }]
|
||||
}
|
||||
},
|
||||
{
|
||||
"step_id": "read-folder-1",
|
||||
"action": "request",
|
||||
"timeout_ms": 15000,
|
||||
"method": "GET",
|
||||
"path": "/api/folders/{saved:folder_1}",
|
||||
"expected_status": [200]
|
||||
"method": "POST",
|
||||
"path": "/api/folders",
|
||||
"json": { "title": "Scratch folder 1" },
|
||||
"headers": [
|
||||
{
|
||||
"name": "X-CSRFToken",
|
||||
"value": { "source": "cookie", "name": "csrftoken" }
|
||||
}
|
||||
],
|
||||
"expected_status": [200, 201]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Give each step that accepts `timeout_ms` a deadline. Without one, a step waits
|
||||
until it completes or the scenario is cancelled, so a locator that never
|
||||
matches hangs the capture and the client with it. The `wait_for_timeout` pause
|
||||
instead uses `duration_ms` and does not accept `timeout_ms`. Match `wait_for`
|
||||
locators to exactly one element.
|
||||
Give every step a `timeout_ms`. A step without one waits until the scenario
|
||||
is cancelled, so a locator that never matches hangs the capture, and the
|
||||
client with it. Match `wait_for` locators to exactly one element.
|
||||
|
||||
1. Sign in through the UI: `fill` the user and password fields (the password
|
||||
as a secret reference), submit, then `wait_for` an element only a signed-in
|
||||
user sees.
|
||||
2. Read with `request` steps: each collection the environment needs, twice
|
||||
before any write, so fields that change on their own show up; then each
|
||||
object a list shows by id, up to ten per collection. An object known only
|
||||
from a list that leaves out fields its own read shows cannot be read by id
|
||||
or updated in the environment.
|
||||
3. Write the way the application's UI does: fill its forms and click its
|
||||
buttons, so each request carries the fields the UI sends, including ones
|
||||
the server would otherwise derive. Give the click or key press that submits
|
||||
a create `save_response` to keep the new object's ID, then `goto` its page
|
||||
(`/items/{saved:item_1}`) or read it with a `request` step. Never guess an
|
||||
ID, and never create a probe object to learn the next one. Write with
|
||||
`request` steps instead only when the UI cannot reach the operation (an
|
||||
API-only resource, or a UI that does not call the JSON API), or when coverage
|
||||
withholds the UI-shaped write as `unexplained-change` or `contradicted`;
|
||||
that counts as one of the two re-recordings. A `request` step write that
|
||||
coverage withholds that way usually lacks fields the UI sends: record it
|
||||
through the UI instead. If a journey cannot be scripted, leave the write
|
||||
unserved and report the limitation.
|
||||
before any write, so fields that change on their own show up; one item of
|
||||
each.
|
||||
3. Use `request` steps only when the API lets the caller choose each scratch
|
||||
object's ID or a prior read already established it. Request paths are fixed
|
||||
strings; no action substitutes a value from an earlier response. For
|
||||
server-assigned IDs, create through the UI and open the new object's detail
|
||||
view to record its read, edit and delete through UI actions. If that journey
|
||||
cannot be scripted, leave the write unserved and report the limitation;
|
||||
never guess an ID or create objects with no supported cleanup path.
|
||||
Record two numbered scratch objects per write type,
|
||||
with varied free text. After each write, and before the next one, read the
|
||||
object back by id and then its list: create 1, read 1, list, create 2,
|
||||
read 2, list; never several writes before their reads.
|
||||
4. Leave more than a second between two creates and between a create and its
|
||||
first edit (`wait_for_timeout` with `duration_ms: 1100`), so lists sorted
|
||||
by creation time show them in order.
|
||||
5. Update both objects and read each back, changing the fields a person
|
||||
would change (a name, an email address, a status, a due date) rather than
|
||||
only a description: a served update accepts only the members a recorded
|
||||
update stored. Run an action that turns something on or off (star,
|
||||
bookmark, archive) twice in a row on one object, reading it back after
|
||||
each: the second call shows whether it sets or flips the field, and the
|
||||
environment serves the action only once a call has shown it. Delete both
|
||||
and read each back (expect 404, or the object marked deleted) and the list.
|
||||
6. Vary what the two creates send, however you write them: the first every
|
||||
field the application takes, the second only what a person must give (its
|
||||
title or name), with the other fields left out or empty. A served create
|
||||
requires each member both creates filled in and lets a caller leave out one
|
||||
a create left out or sent empty, so an agent that names only a title is not
|
||||
refused for fields it had no reason to set.
|
||||
4. Leave more than a second between a create and its first edit
|
||||
(`wait_for_timeout` with `duration_ms: 1100`).
|
||||
5. Update both objects and read each back; delete both and read each back
|
||||
(expect 404) and the list.
|
||||
|
||||
About `request` steps:
|
||||
|
||||
- `path` is on the start URL's origin; the page must be on that origin.
|
||||
- Header values are literals, declared secrets, a cookie the browser holds
|
||||
(`{source: "cookie", name}`), a local storage entry
|
||||
(`{source: "local_storage", key, json_pointer}`), a saved value
|
||||
(`{source: "saved", name}`), or Basic credentials
|
||||
(`{source: "basic_credentials", parts: [username, password]}`, sent as the
|
||||
base64 of `username:password` and redacted), with an optional `prefix` such as
|
||||
`"Bearer "` or `"Basic "`. Each credential part is a literal, secret, saved
|
||||
value, cookie or local storage entry, so a header the frontend builds from a
|
||||
login token (the username and the token) can be copied too; the password is
|
||||
a secret or saved value, never a literal. Copy the
|
||||
headers the application's own requests send (a CSRF header such as
|
||||
`X-CSRFToken` from the `csrftoken` cookie, a bearer token). Values read from the page are redacted from records made after they
|
||||
are read, not from the login response that returned them, and values
|
||||
shorter than 8 characters are not redacted at all.
|
||||
- `save: [{name, json_pointer, secret}]` keeps a string or number from the
|
||||
step's JSON response, such as the created object's `/id`, for later steps;
|
||||
`secret: true` redacts it, as for a token the API returns, so mark every
|
||||
saved token, key or session value secret. This also protects the selected
|
||||
source response before its body or headers are retained. Compressed or
|
||||
unknown-size UI responses cannot supply saved values. On a `click` or `press`,
|
||||
`save_response: {method, path, save}` keeps values from the response to the
|
||||
new request observed while the step waits (`{*}` in `path` matches one segment).
|
||||
Matching does not prove UI causality: choose a path distinct from polling;
|
||||
multiple observed matches fail as ambiguous. The step
|
||||
fails unless a 2xx JSON response arrives before its `timeout_ms`. Later
|
||||
steps send a saved value as `{saved:name}` in a request `path` (and its
|
||||
query) or a `goto` URL; in the `json` body, where a string that is only
|
||||
`{saved:name}` keeps the saved JSON type; or as `{source: "saved", name}` in
|
||||
a header or credential part.
|
||||
(`{source: "cookie", name}`), or a local storage entry
|
||||
(`{source: "local_storage", key, json_pointer}`), with an optional
|
||||
`prefix` such as `"Bearer "`. Copy the headers the application's own
|
||||
requests send (a CSRF header, a bearer token). Values read from the page
|
||||
are redacted from records made after they are read, not from the login
|
||||
response that returned them, and values shorter than 8 characters are not
|
||||
redacted at all.
|
||||
- Reading a page value makes later bodies costlier to keep: after a CSRF
|
||||
token, bodies over about 47 KiB are kept as `unavailable`. When the API
|
||||
returns larger responses, raise `capture.limits`
|
||||
@@ -241,22 +172,20 @@ Re-record a withheld write at most twice, and only for a recording gap. Stop at
|
||||
once when the reason is application behaviour, and report the write as
|
||||
withheld with its hint:
|
||||
|
||||
| Reason | What to do |
|
||||
| --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
|
||||
| `no-observation` | Re-record: read the entity back by id after the write; read it first for an update or delete. |
|
||||
| `unconfirmed` | Re-record: read the entity and its list after the write. |
|
||||
| `incomplete-entities` | Re-record: read the entity through every view that lists it. |
|
||||
| `unexplained-change` | Re-record once with a second, different call of the same write. |
|
||||
| `no-call` | Re-record once with a write that succeeds. |
|
||||
| `side-effect` | Stop: the application changes other entities. |
|
||||
| `contradicted` | Stop: a later read disagrees with the modelled effect. |
|
||||
| `body-unusable` | Re-record a UI write once with `request` steps (its body was not exposed); else stop. |
|
||||
| Reason | What to do |
|
||||
| --------------------- | --------------------------------------------------------------------------------------------- |
|
||||
| `no-state-target` | Re-record: read the collection or item the write changes. |
|
||||
| `no-observation` | Re-record: read the entity back by id after the write; read it first for an update or delete. |
|
||||
| `unconfirmed` | Re-record: read the entity and its list after the write. |
|
||||
| `incomplete-entities` | Re-record: read the entity through every view that lists it. |
|
||||
| `unexplained-change` | Re-record once with a second, different call of the same write. |
|
||||
| `no-call` | Re-record once with a write that succeeds. |
|
||||
| `side-effect` | Stop: the application changes other entities. |
|
||||
| `contradicted` | Stop: a later read disagrees with the modelled effect. |
|
||||
| `body-unusable` | Stop: the body is not a JSON object. |
|
||||
|
||||
Use new scratch names on each attempt, and delete what an earlier attempt
|
||||
created before recording again (through the UI, or a `request` step with the
|
||||
id that capture shows): the environment starts from what the recording first
|
||||
reads, so a leftover object becomes seeded data that write tasks then find.
|
||||
Use new scratch names on each attempt; the first attempt's objects may still
|
||||
exist.
|
||||
New browser scenario captures include request times, so a fully timed capture
|
||||
can learn a write clock and serve updates that stamp modification times. Older
|
||||
captures or ones missing a request start keep record order and learn no clock.
|
||||
|
||||
@@ -11,44 +11,6 @@ returned inline. When complete `inspect_binary_layout` or
|
||||
`inspect_analysis_view` for a summary, one section/symbol/module, or a stable
|
||||
page instead of repeating the producer.
|
||||
|
||||
For any logical Evidence source that advertises retained references, pass
|
||||
`{kind: "retained-evidence", evidence_id}` instead of copying the complete
|
||||
record back into a call. The same contract covers individual, optional, array
|
||||
and nested sources across native, process, JavaScript, managed, mobile and WASM
|
||||
workflows. Sources can independently mix inline records and exact references.
|
||||
Resolution preserves the source's identity, provenance, coverage, partial facts
|
||||
and unknowns; it neither reruns analysis nor makes incomplete facts complete.
|
||||
Keep domain role and identity requirements: a native dossier cannot substitute
|
||||
for a managed member observation, for example.
|
||||
|
||||
When starting from a summary or inspection page, use its
|
||||
`normalized_result.parent_evidence_id` to reference the complete source.
|
||||
The view's own `evidence_id` names
|
||||
only that view. `compare_artifacts` explicitly supports artifact summary inputs
|
||||
and resolves their parents; do not assume every consumer does. When a workflow
|
||||
follows source links, keep those linked records available too.
|
||||
|
||||
MCP references belong to the current connection's ledger. If the ID is known,
|
||||
pass it directly; no discovery call or provider setup is needed. If it is missing,
|
||||
import the exported bundle or supply complete inline Evidence. Do not retry an
|
||||
unchanged reference or rerun a producer merely to move an existing record.
|
||||
Export before closing the session when the source must survive it. For CLI JSON
|
||||
workflows that advertise `--evidence-bundle`, the same reference JSON resolves
|
||||
against one explicitly selected authenticated bundle:
|
||||
|
||||
```bash
|
||||
rea trace-application-feature request.json --evidence-bundle /absolute/path/evidence.json --json
|
||||
```
|
||||
|
||||
This includes JavaScript traces/comparisons and runtime reconciliation, managed
|
||||
reconstruction/native verification/graph projection, Android/Apple/Harmony graph
|
||||
projection, WASM interface comparison and retained analysis views. The managed
|
||||
member comparison CLI still takes two assembly paths; process comparison takes
|
||||
two capture Evidence JSON paths. Check command help for the installed version.
|
||||
An input bundle grants no implicit file lookup: `evidence-file` and historical
|
||||
graph file selectors are accepted only by contracts advertising them. See the
|
||||
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/cli.md#import-export-and-compare-evidence).
|
||||
|
||||
Completed `inspect_artifact` inventory also supports a summary and
|
||||
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
|
||||
Omit `logical_paths` to page all available occurrences; `[]` selects none.
|
||||
@@ -73,14 +35,6 @@ Parent Evidence ID plus source index preserves same-address
|
||||
alternatives. Follow `coverage.next_offset` only as far as the question needs;
|
||||
exhausting retained rows does not establish complete program discovery.
|
||||
|
||||
`batch_decompile` returns every body unless you select `detail: "summary"` when
|
||||
the connected tool advertises it. Use its parent Evidence ID with
|
||||
`view: {kind: "batch-decompile", collection: "items"}` for body-free rows and
|
||||
exact `where.indices`, `addresses`, `procedure_addresses` or `statuses`. Add
|
||||
`include_pseudocode: true` for several chosen bodies, or use
|
||||
`collection: "pseudocode"` with one row's `index` to page a large body. Batch
|
||||
limitations aggregate every item, so do not attribute them to one successful row.
|
||||
|
||||
Use `record_unknown` to track unresolved questions in the user's investigation
|
||||
and name the authority or environment still required. Supply supporting and
|
||||
contradicting evidence IDs. It updates the session registry directly.
|
||||
@@ -92,22 +46,18 @@ Use comparisons with complete, compatible page sets when claiming equivalence
|
||||
or absence:
|
||||
|
||||
- `compare_artifacts` compares inventory pages by occurrence path, content,
|
||||
metadata, and graph relations. Name each side as
|
||||
`{kind: "retained-evidence", evidence_id}` (an `inspect_artifact` result,
|
||||
its nested inventory or its summary) or
|
||||
`{kind: "evidence-file", path, evidence_id?}` instead of resending Evidence.
|
||||
metadata, and graph relations.
|
||||
- `compare_functions` compares explicit function Evidence, not fuzzy
|
||||
whole-binary matches. It rejects an `analyze_function` summary reply; pass
|
||||
the summary's `parent_evidence_id` as a `retained-evidence` reference or
|
||||
the complete dossier.
|
||||
- `compare_bundles` compares canonical bundle membership and unknown history.
|
||||
- `find_changed_behavior` combines existing comparisons; static differences
|
||||
remain candidates, not causal proof. Its `comparisons` accept exact references.
|
||||
remain candidates, not causal proof.
|
||||
- `build_call_path` needs explicit function dossiers from one artifact/provider;
|
||||
pass complete parent IDs in `functions`. An incomplete frontier makes absence
|
||||
unknown.
|
||||
an incomplete frontier makes absence unknown.
|
||||
- `correlate_static_and_runtime` uses explicit mappings; matching patterns are
|
||||
hypotheses, not causality. Both comparison arrays accept exact references.
|
||||
hypotheses, not causality.
|
||||
- `verify_reconstruction` evaluates a finite typed specification. A pass covers
|
||||
only declared comparable claims, not global equivalence.
|
||||
|
||||
|
||||
@@ -169,10 +169,7 @@ leaving it, and `both` when the question needs both. Trace traversal includes
|
||||
containment relationships, so even a narrow seed can expand many nodes; a
|
||||
bidirectional trace can reach unrelated sibling modules. A module item or page
|
||||
is preferable when its observations already answer the question. Do not treat
|
||||
all reached nodes as participants in the feature. For a broad trace, pass
|
||||
`view: {"kind": "feature-trace", "collection": "paths"}` (or `summary`) to
|
||||
retain the complete trace and read terminal paths first; page `nodes` or `edges`
|
||||
with exact `where.node_ids` through `inspect_analysis_view` as needed.
|
||||
all reached nodes as participants in the feature.
|
||||
|
||||
Reuse retained parent Evidence when the server advertises it, or supply complete
|
||||
Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest
|
||||
@@ -239,19 +236,10 @@ This notation is
|
||||
a template: replace it with the actual returned ID. Versions before 4.1.0 accept only full
|
||||
inline Evidence. Use the exact ID
|
||||
returned by the producer on the same MCP connection. Resolution does not run
|
||||
analysis or select a provider; findings remain inline. For summary or inspection
|
||||
replies, select the complete `normalized_result.parent_evidence_id`. Native
|
||||
observation arrays and runtime reconciliation's `static_layers[].analysis` and
|
||||
`runtime_observations[]` also accept exact references where advertised. Each
|
||||
source can independently be inline or retained, preserving identity, coverage
|
||||
and unknowns. CLI workflows advertising `--evidence-bundle` resolve these inputs
|
||||
from the explicitly selected bundle. `close_binary` releases the engine while
|
||||
retaining references and complete Unknown history on the same MCP connection.
|
||||
`reset_investigation` explicitly clears them; optional `save_path` checkpoints the
|
||||
complete bundle before clearing. Import that bundle in another connection to
|
||||
restore its original references. Retention is bounded, in-memory and never silently
|
||||
evicted; `binary_session.investigation` reports the budget. A missing reference
|
||||
includes its ID and recovery guidance.
|
||||
analysis or select a provider; findings remain inline. Native observation
|
||||
arrays still take full Evidence. `close_binary` clears retained references;
|
||||
export a bundle before closing, or supply portable inline Evidence in another
|
||||
connection. A missing reference includes its ID and recovery guidance.
|
||||
|
||||
For exact file/native metadata or processing gaps, use
|
||||
`view: {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"], limit: 32}`.
|
||||
|
||||
@@ -1,11 +1,8 @@
|
||||
# Native, managed, and packaged artifacts
|
||||
|
||||
For release comparisons on one MCP connection, retain `list_procedures` Evidence
|
||||
from each target and reuse its exact IDs after target switches or `close_binary`.
|
||||
For a handoff or new process, export the complete bundle and import it there,
|
||||
including its source closure. `reset_investigation` explicitly clears retained
|
||||
records; optional `save_path` checkpoints them before clearing.
|
||||
`match_native_functions` pairs unique exact names and, when supplied,
|
||||
For release comparisons, export `list_procedures` Evidence from each target before
|
||||
closing it. Close clears session records: import both bundles after collecting the
|
||||
two targets, or supply portable inline Evidence. `match_native_functions` pairs unique exact names and, when supplied,
|
||||
complete observed body bytes. Reuse optional same-context `list_names`,
|
||||
`procedure_info`, `read_bytes` and existing `analyze_function` dossiers as
|
||||
`left_observations`/`right_observations`; an MCP `analyze_function` summary
|
||||
@@ -16,13 +13,6 @@ Select columns, batch IDs and page individual facts; aliases remain unresolved.
|
||||
Unpaired observations do not prove additions/removals. Existing matched dossier
|
||||
pairs include inputs for `compare_functions`; otherwise analyze just chosen pairs.
|
||||
|
||||
Aggregate completed comparisons with `find_changed_behavior`, selecting each
|
||||
comparison's exact `retained-evidence` ID instead of resending full records.
|
||||
Inline comparison Evidence remains accepted. References belong to the current
|
||||
server: after handoff, import the original bundle including its source closure.
|
||||
Do not substitute dossier IDs or summary views. Static differences remain
|
||||
behavior candidates, with runtime behavior unknown unless separately observed.
|
||||
|
||||
For a matching representation refusal, inspect `details.limit` and raise
|
||||
`native.matching_representation_bytes` with `update_analysis_settings` before
|
||||
retrying the unchanged MCP request. CLI launch seed:
|
||||
@@ -346,9 +336,8 @@ the complete tool inventory grouped by current status. Expand each group's
|
||||
overrides them; availability, reasons, remediation, and modes remain inline.
|
||||
Provider candidates keep availability and target-support status, code, and
|
||||
reason. `omitted_fields` names what the summary leaves out; `{"detail":
|
||||
"complete"}` adds each unavailable candidate's `remediation`, provider
|
||||
diagnostics, candidate capability sets, and tool surface, `effects`, and
|
||||
`annotations`. Those arrays name true flags from fixed
|
||||
"complete"}` adds provider diagnostics, candidate capability sets, and tool
|
||||
surface, `effects`, and `annotations`. Those arrays name true flags from fixed
|
||||
vocabularies, so an omitted known flag is false. `tools/list` remains the
|
||||
canonical source for full tool schemas when target or provider state changes.
|
||||
|
||||
@@ -527,11 +516,6 @@ names), coverage and per-collection counts. Page rows with
|
||||
`view: {kind: "managed-members", collection: "call_edges", offset: 0, limit: 64}`
|
||||
over `types`, `exported_types`, `fields`, `methods`, `member_refs`, `call_edges`,
|
||||
`field_accesses` or `user_strings`.
|
||||
Use exact `names` and `declaring_types` filters before paging supported
|
||||
collections. For a compact method lookup, request `fields: ["name",
|
||||
"declaring_type", "signature"]`; omit `body` to avoid anchors. Identity tokens,
|
||||
original `source_index`, module/MVID and source coverage remain. Check
|
||||
`unresolved_filter_rows` before interpreting an empty match as absence.
|
||||
|
||||
### .NET NativeAOT
|
||||
|
||||
@@ -589,16 +573,6 @@ REA. Symlinks and encrypted entries are inventory facts, not extractable files.
|
||||
The result reports the materialized directory; callers do not choose the
|
||||
destination. No separate permission grant is needed.
|
||||
|
||||
For classic iNES cartridges, `open_binary` recognizes the byte signature without
|
||||
an executable architecture. `inspect_artifact` retains header facts and each
|
||||
PRG/CHR file unit's offset, length and SHA-256; its nested inventory's
|
||||
`ines.reconstruction` verifies the actual ordered region bytes against the
|
||||
whole-file digest. Extraction includes the original header, optional trainer
|
||||
and uninterpreted trailing bytes. These file units are not mapper/CPU bank
|
||||
maps. NES 2.0, UNIF, dirty legacy headers and truncated payloads are explicitly
|
||||
refused. No emulator or 6502 analysis is performed. See the
|
||||
[iNES guide](https://github.com/morluto/rea/blob/main/docs/ines-containers.md).
|
||||
|
||||
Native DMG traversal automatically uses a read-only mount on supported macOS
|
||||
hosts, with an owned temporary mount directory and cleanup. It needs no approval
|
||||
flag. Unsupported hosts or mount failures remain explicit limitations; do not
|
||||
@@ -627,27 +601,10 @@ semantics are unsupported. `m68k` requires an even entry and decodes the 68040
|
||||
superset of the 68000 family; Amiga/Atari containers and vector-table entry
|
||||
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
|
||||
|
||||
Intel HEX, S-record, UF2 and DfuSe files are containers, not raw images:
|
||||
`inspect_artifact` lists address regions (UF2 per familyID, DfuSe per element
|
||||
with its alternate setting), and `extract_artifact` writes each region. Open an
|
||||
extracted region as `raw-binary` with its start address as `entry_point`. UF2
|
||||
numbering gaps leave coverage unknown; a DfuSe suffix CRC mismatch follows
|
||||
`integrity_policy`. See [UF2 and DfuSe](https://github.com/morluto/rea/blob/main/docs/uf2-dfuse.md).
|
||||
|
||||
### Function annotations in Ghidra
|
||||
|
||||
Fresh-import sessions admit annotations; existing-project reuse is read-only.
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier. Ghidra rejects batch addresses that request the same name or a name another selected address holds; `raw_result.failures` gives each failure's reason.
|
||||
|
||||
For `compare_functions`, obtain `analyze_function` Evidence before and after the
|
||||
edit while the MCP session is open; annotation Evidence is not a comparison
|
||||
input. Export the needed Evidence before close if comparison will happen later.
|
||||
Authenticated same-artifact coordinates preserve renamed entries as
|
||||
`matched / same-entry`, with the canonical `entry_address` inline; name and
|
||||
exact-text changes remain separate dimensions. This requires compatible
|
||||
architecture, provider-owned mapping and selected-image commitments. Equal
|
||||
numeric addresses alone cannot establish correspondence; inspect retained
|
||||
source Evidence and coverage when the match remains ambiguous.
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier.
|
||||
|
||||
## Offline PE resource inventory
|
||||
|
||||
@@ -675,9 +632,7 @@ For exact core-module import/export differences, use `compare_wasm_interfaces`
|
||||
with two complete inspection Evidence records (or current-session retained
|
||||
references) and explicit scope. Preserve unknown types/duplicate names; static
|
||||
interface equality is not runtime compatibility. CLI accepts complete inline
|
||||
Evidence in `rea compare-wasm-interfaces input.json --json`, or exact references
|
||||
with `--evidence-bundle /absolute/path/evidence.json`. Use the complete parent
|
||||
ID when the inspection returned a summary.
|
||||
Evidence in `rea compare-wasm-interfaces input.json --json`.
|
||||
|
||||
## Godot PCK
|
||||
|
||||
@@ -697,84 +652,3 @@ page cannot prove absence in unknown coverage. Retained views are historical,
|
||||
not fresh filesystem checks. No decryption, Godot resource decoding, GDScript
|
||||
decompilation or target execution is provided. See
|
||||
[Godot PCK](https://github.com/morluto/rea/blob/main/docs/godot-pck.md).
|
||||
|
||||
UnityFS AssetBundle containers (`.bundle`, `.ab`, YooAsset packages and other
|
||||
byte-identified files) list `CAB-*` SerializedFiles and `.resS` members with
|
||||
header and node facts; none/LZ4/LZ4HC member blocks stream on demand. Bundles
|
||||
declaring asset-bundle encryption keep header facts with unknown directory
|
||||
coverage; LZMA member bytes retain named occurrences with unavailable hashes.
|
||||
Virtual member offsets are not physical carrier locations. Format 6–8 readers
|
||||
cache one decoded block and support retained filtered/paged views. No
|
||||
decryption or SerializedFile object decoding is provided. See
|
||||
[UnityFS](https://github.com/morluto/rea/blob/main/docs/unityfs.md).
|
||||
|
||||
U-Boot legacy uImages use the same inspection/extraction workflows. Prefer a
|
||||
retained summary and selected inventory views for header facts or component
|
||||
IDs. Header addresses/CPU/OS are declarations; payloads remain as stored.
|
||||
Multi-file components retain their aligned physical ranges; the full payload
|
||||
is also materialized. Unknown name text retains its original 32 bytes. See
|
||||
[U-Boot uImage](https://github.com/morluto/rea/blob/main/docs/uimage.md).
|
||||
|
||||
## ESP images
|
||||
|
||||
For an ESP32-family app or bootloader image, `open_binary` then
|
||||
`inspect_artifact` with `detail: "summary"`. `root_format_metadata` holds the
|
||||
observed `header` and `descriptor` (its `kind` is the app or bootloader role)
|
||||
and the derived `chip`, `checksum`, `appended_sha256` and
|
||||
`undescribed_trailing` ranges. Segment rows (`segment-NN.bin`) carry
|
||||
`load_addr`, `classification` (`mapped`, `loaded`, `not-loaded` or `unknown`,
|
||||
per ESP-IDF `should_map`/`should_load`; RAM ranges are not checked) and
|
||||
`source_location`; select them with `inspect_analysis_view` and
|
||||
`logical_paths`. Signature blocks are `unverified`; `public_key_sha256`
|
||||
matches eFuse key digests.
|
||||
|
||||
Unknown `source_coverage` means truncation or undescribed trailing bytes, such
|
||||
as the flash segments of a `--ram-only-header` image; do not infer absence
|
||||
from it. Erased flash is described. Integrity mismatches are facts, not
|
||||
refusals. ESP8266 images are refused, and flash-encrypted images are not
|
||||
recognized. Segments are not disassembled, and RISC-V or Xtensa segments
|
||||
cannot be opened as raw-binary. See
|
||||
[ESP images](https://github.com/morluto/rea/blob/main/docs/esp-image.md).
|
||||
|
||||
## ESP flash images
|
||||
|
||||
A whole-flash dump opens as `esp-flash-image`. `root_format_metadata` holds
|
||||
`partition_table` (its `method`: `default`, `caller` or a derived
|
||||
`scan`), `bootloader` and `apps` ESP image facts, the `otadata` records,
|
||||
`partitions_outside_dump` and `boot_selection`. Every table has a
|
||||
`boot_selection`: its `basis` names the bootloader's branch (`no-otadata`
|
||||
without otadata, `none` when the bootloader rejects the table, `unknown`
|
||||
when a record is missing), and it lists its `assumptions`;
|
||||
`running_partition` is always `unknown`. Each image names its entry
|
||||
`path`. With `detail: "summary"` the root kind is `esp-flash-image-summary`
|
||||
and each image is a compact `summary` (chip, entry point, segment count,
|
||||
classified load ranges, signature block count, descriptor identity,
|
||||
integrity); complete detail keeps every segment. Partition rows
|
||||
(`partitions/<label>.bin`) carry the declared `type`, `subtype`, names,
|
||||
`flash_address`, `size` and flags; `encrypted` is a storage flag, not
|
||||
proof of ciphertext. `written_offset`/`written_end` bound the bytes that
|
||||
are not erased: an app slot whose facts say `erased` can still hold an older
|
||||
image, and a limitation says so. For a dump read from another flash address, call
|
||||
`open_binary` with `format: "esp-flash-image"` and `flash_offset`; an
|
||||
`esp-image` whose `flash_dump` is set names that offset in a next probe; a
|
||||
relocated table can be named with `partition_table_offset`. No table found
|
||||
means unknown, often flash encryption. Extraction writes `nvs_keys` as
|
||||
ordinary data. See
|
||||
[ESP flash images](https://github.com/morluto/rea/blob/main/docs/esp-flash-image.md).
|
||||
|
||||
## Bun standalone executables
|
||||
|
||||
A `bun build --compile` executable stores its module graph in Mach-O
|
||||
`__BUN,__bun`, a PE or ELF `.bun` section, or an appended trailer.
|
||||
`open_binary` then `inspect_artifact` reports root format `bun-standalone` with
|
||||
the carrier, structural layout, module count, entry point, raw flags and
|
||||
compile exec-argv; each module keeps its `/$bunfs/...` source name, loader,
|
||||
encoding, sizes and SHA-256, with sourcemap, bytecode and module-info regions as
|
||||
separate children. Use summary root metadata `entry_point_logical_path` as the
|
||||
exact inventory `logical_paths` selector, then read the selected content node by
|
||||
its artifact ID; the reader owns virtual-name normalization and indexed fallback. `extract-artifact` writes stored contents through the safe
|
||||
output tree; route extracted JavaScript to `analyze_javascript_application`.
|
||||
Layout labels describe record shapes, not a detected compiler version; unknown
|
||||
or ambiguous layouts are unsupported. Bytecode is retained, not decoded, and the
|
||||
executable is never run. See
|
||||
[Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
|
||||
"name": "rea",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"owner": {
|
||||
"name": "morluto",
|
||||
"url": "https://github.com/morluto"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "rea",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"source": {
|
||||
"source": "npm",
|
||||
"package": "rea-agents",
|
||||
"version": "6.8.0"
|
||||
},
|
||||
"category": "development"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
{
|
||||
"name": "rea",
|
||||
"version": "6.8.0",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"author": {
|
||||
"name": "morluto",
|
||||
"url": "https://github.com/morluto"
|
||||
},
|
||||
"homepage": "https://rea.tools/",
|
||||
"repository": "https://github.com/morluto/rea",
|
||||
"license": "MIT",
|
||||
"keywords": [
|
||||
"reverse-engineering",
|
||||
"binary-analysis",
|
||||
"decompiler",
|
||||
"mcp-server"
|
||||
],
|
||||
"skills": "./skills",
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "node",
|
||||
"args": [
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/plugin-mcp-launcher.mjs",
|
||||
"6.8.0"
|
||||
]
|
||||
}
|
||||
},
|
||||
"icon": "./assets/rea-plugin.png"
|
||||
}
|
||||
@@ -1,13 +0,0 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "node",
|
||||
"args": [
|
||||
"./scripts/plugin-mcp-launcher.mjs",
|
||||
"6.8.0"
|
||||
],
|
||||
"cwd": ".",
|
||||
"startup_timeout_sec": 30
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,40 +0,0 @@
|
||||
{
|
||||
"name": "rea",
|
||||
"version": "6.8.0",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"author": {
|
||||
"name": "morluto",
|
||||
"url": "https://github.com/morluto"
|
||||
},
|
||||
"homepage": "https://rea.tools/",
|
||||
"repository": "https://github.com/morluto/rea",
|
||||
"license": "MIT",
|
||||
"keywords": [
|
||||
"reverse-engineering",
|
||||
"binary-analysis",
|
||||
"decompiler",
|
||||
"mcp-server"
|
||||
],
|
||||
"skills": "./skills",
|
||||
"mcpServers": "./.codex-plugin/mcp.json",
|
||||
"interface": {
|
||||
"displayName": "REA",
|
||||
"shortDescription": "Reverse engineer anything",
|
||||
"longDescription": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"developerName": "morluto",
|
||||
"category": "Developer Tools",
|
||||
"capabilities": [
|
||||
"Interactive",
|
||||
"Read",
|
||||
"Write"
|
||||
],
|
||||
"websiteURL": "https://rea.tools/",
|
||||
"defaultPrompt": [
|
||||
"Use REA to explain how this binary implements a feature.",
|
||||
"Use REA to compare two versions of this application.",
|
||||
"Use REA to recover the sources from this JavaScript bundle."
|
||||
],
|
||||
"composerIcon": "./assets/rea-plugin.png",
|
||||
"logo": "./assets/rea-plugin.png"
|
||||
}
|
||||
}
|
||||
@@ -1,12 +0,0 @@
|
||||
# Review changes to security-sensitive automation and dependency inputs.
|
||||
/.github/CODEOWNERS @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/dependabot.yml @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/workflows/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/actions/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/verify-tools/skills/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/verify-tools/skills/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/fixtures/http3-lab-requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/website/requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/third_party/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
@@ -1,21 +0,0 @@
|
||||
name: REA production code
|
||||
|
||||
# Match the existing default-setup scan's local-input threat model.
|
||||
threat-models: [local]
|
||||
|
||||
# Retain root traversal, including future production roots. Hidden directories
|
||||
# need their own root; use a directory so Python can share this configuration.
|
||||
paths:
|
||||
- .
|
||||
- docs/.vitepress
|
||||
|
||||
# Keep the default query suite and all production roots, including bridge/,
|
||||
# install.sh, runtime scripts, release controllers, and GitHub Actions.
|
||||
# Exclude test inputs at extraction time so they cannot taint production callers.
|
||||
paths-ignore:
|
||||
- tests
|
||||
- scripts/fixtures
|
||||
- "**/*.test.*"
|
||||
- "**/*.fixture.*"
|
||||
- "**/test_*.py"
|
||||
- "**/*_test.py"
|
||||
@@ -1,33 +0,0 @@
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: npm
|
||||
directories:
|
||||
- /
|
||||
- /scripts/verify-tools/skills
|
||||
schedule:
|
||||
interval: weekly
|
||||
cooldown:
|
||||
default-days: 7
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
npm-minor-and-patch:
|
||||
update-types:
|
||||
- minor
|
||||
- patch
|
||||
|
||||
- package-ecosystem: pip
|
||||
directory: /website
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
|
||||
- package-ecosystem: github-actions
|
||||
directory: /
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
actions-minor-and-patch:
|
||||
update-types:
|
||||
- minor
|
||||
- patch
|
||||
@@ -106,8 +106,7 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
|
||||
|
||||
## Author readiness
|
||||
|
||||
<!-- The human owner completes these declarations before requesting review,
|
||||
unless the PR author is exempt as listed in CONTRIBUTING.md.
|
||||
<!-- The human owner completes these declarations before requesting review.
|
||||
The PR author is the owner by default. A different owner must explicitly accept
|
||||
ownership on this PR.
|
||||
AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA.
|
||||
|
||||
+17
-87
@@ -61,9 +61,6 @@ jobs:
|
||||
qualification_run_id: ${{ steps.qualification.outputs.run_id }}
|
||||
code: ${{ steps.plan.outputs.code }}
|
||||
reverse_tools: ${{ steps.plan.outputs.reverse_tools }}
|
||||
python: ${{ steps.plan.outputs.python }}
|
||||
packet_capture: ${{ steps.plan.outputs.packet_capture }}
|
||||
signal_recording: ${{ steps.plan.outputs.signal_recording }}
|
||||
sqlite: ${{ steps.plan.outputs.sqlite }}
|
||||
runtime: ${{ steps.plan.outputs.runtime }}
|
||||
package: ${{ steps.plan.outputs.package }}
|
||||
@@ -147,9 +144,6 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Validate source architecture before dependency installation
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
run: node scripts/verify/architecture-guards.mjs
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
@@ -270,27 +264,6 @@ jobs:
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
python:
|
||||
name: Python producer tests
|
||||
needs: changes
|
||||
if: needs.changes.outputs.python == 'true' && needs.changes.outputs.tier == 'fast'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
# Analysis-efficiency checks import compiled process modules.
|
||||
- name: Reuse portable build outputs across runs
|
||||
uses: ./.github/actions/portable-build-cache
|
||||
- run: npm run test:prepare
|
||||
- run: npm run test:python
|
||||
|
||||
test-source:
|
||||
name: Source tests
|
||||
needs: changes
|
||||
@@ -418,9 +391,7 @@ jobs:
|
||||
needs.changes.outputs.web_network == 'true' ||
|
||||
needs.changes.outputs.web_runtime == 'true' ||
|
||||
needs.changes.outputs.web_source_map == 'true' ||
|
||||
needs.changes.outputs.agentenv == 'true' ||
|
||||
needs.changes.outputs.packet_capture == 'true' ||
|
||||
needs.changes.outputs.signal_recording == 'true'
|
||||
needs.changes.outputs.agentenv == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
@@ -779,20 +750,9 @@ jobs:
|
||||
npm run verify:legacy-simulator
|
||||
npm run verify:apple-dispatch
|
||||
npm run verify:interface-builder
|
||||
- name: Verify portable iNES CLI, MCP and source boundaries
|
||||
run: >-
|
||||
npm run test:focused --
|
||||
tests/acceptance/inesContainer.test.ts
|
||||
tests/boundary/filesystem/inesArtifact.test.ts
|
||||
tests/boundary/filesystem/inesCancellation.test.ts
|
||||
- name: Run native Apple and process boundary regressions
|
||||
run: >-
|
||||
npm run test:focused --
|
||||
tests/boundary/providers/hopper/hopperMacLaunchArguments.test.ts
|
||||
tests/acceptance/providers/hopperStartupRecovery.test.ts
|
||||
tests/boundary/providers/hopper/hopperClient.helperFailure.test.ts
|
||||
tests/boundary/providers/hopper/bridgeBootstrapCleanup.test.ts
|
||||
tests/boundary/providers/hopper/bridgeTargetLease.integration.test.ts
|
||||
tests/acceptance/artifactDmgAttachmentRecoveryCli.test.ts
|
||||
tests/boundary/providers/native/nativeDmgArtifactReader.test.ts
|
||||
tests/process-global/dmgCancellationReason.test.ts
|
||||
@@ -805,13 +765,9 @@ jobs:
|
||||
tests/boundary/cli/nativeDemangleCli.test.ts
|
||||
tests/boundary/cli/nativeDispatchPropertyAttributes.test.ts
|
||||
tests/boundary/mcp/nativeDemangleMcp.test.ts
|
||||
tests/boundary/mcp/nativeMachoSummaryMcp.test.ts
|
||||
tests/boundary/providers/native/nativeSignatureTargetBinding.test.ts
|
||||
tests/boundary/providers/native/nativeUiOffscreenWindow.test.ts
|
||||
tests/boundary/process/nativeUiVerifierLane.test.ts
|
||||
tests/boundary/cli/nativeSignatureCli.test.ts
|
||||
tests/boundary/cli/nativeMachoArm64_32Cli.test.ts
|
||||
tests/boundary/cli/nativeMachoEncryptionCli.test.ts
|
||||
tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts
|
||||
src/process/capture/FilesystemSnapshot.test.ts
|
||||
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
|
||||
@@ -899,24 +855,6 @@ jobs:
|
||||
# The matching native artifact establishes real Job Object and private
|
||||
# runtime boundaries for these fixtures. Ghidra and Java are not needed;
|
||||
# real engine claims remain in the separate controlled-provider workflow.
|
||||
# Run native proofs before the broad curated suite so an unrelated
|
||||
# fixture failure cannot hide the real control and recovery checks.
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-process-boundary-fixture
|
||||
path: .cache/windows-process-boundary/
|
||||
# Passing the fixture adds the breakaway, caller/child token, environment
|
||||
# semantics and ancestor reparse observations, which the lane skips
|
||||
# entirely when it is absent.
|
||||
- run: >-
|
||||
npm run verify:windows-native -- .
|
||||
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-native-inspection-fixture
|
||||
path: .cache/windows-native-inspection-fixture/
|
||||
- name: Verify native inspection failure ownership and recovery
|
||||
run: node scripts/verify-windows-native-inspection.mjs
|
||||
- name: Run curated Windows capability tests
|
||||
run: >-
|
||||
npx vitest run --coverage.enabled=false
|
||||
@@ -945,7 +883,6 @@ jobs:
|
||||
tests/boundary/process/packageNpmInvocation.test.ts
|
||||
tests/boundary/process/packedBridgeVerification.test.ts
|
||||
tests/boundary/process/windowsOutputCapture.test.ts
|
||||
tests/boundary/process/windowsExecutableSelection.test.ts
|
||||
tests/acceptance/webModuleCli.test.ts
|
||||
tests/acceptance/webSourceLocationCli.test.ts
|
||||
tests/boundary/cli/referenceSourceImportStatus.test.ts
|
||||
@@ -970,6 +907,22 @@ jobs:
|
||||
tests/process-global/runnerCancellation.test.ts
|
||||
tests/process-global/windowsOwnedCommand.test.ts
|
||||
tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-process-boundary-fixture
|
||||
path: .cache/windows-process-boundary/
|
||||
# Passing the fixture adds the breakaway, caller/child token, environment
|
||||
# semantics and ancestor reparse observations, which the lane skips
|
||||
# entirely when it is absent.
|
||||
- run: >-
|
||||
npm run verify:windows-native -- .
|
||||
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-native-inspection-fixture
|
||||
path: .cache/windows-native-inspection-fixture/
|
||||
- name: Verify native inspection failure ownership and recovery
|
||||
run: node scripts/verify-windows-native-inspection.mjs
|
||||
- name: Verify real Node Inspector discovery and observation
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
@@ -1130,26 +1083,6 @@ jobs:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
packet_capture:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.packet_capture == 'true'
|
||||
secrets:
|
||||
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
|
||||
uses: ./.github/workflows/real-packet-capture.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
signal_recording:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.signal_recording == 'true'
|
||||
secrets:
|
||||
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
|
||||
uses: ./.github/workflows/real-signal-recording.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
nativeaot:
|
||||
needs: [changes, static]
|
||||
if: needs.changes.outputs.nativeaot == 'true'
|
||||
@@ -1176,9 +1109,6 @@ jobs:
|
||||
static-deep,
|
||||
build,
|
||||
reverse-tools-quality,
|
||||
python,
|
||||
packet_capture,
|
||||
signal_recording,
|
||||
test-shard,
|
||||
test-source,
|
||||
sqlite-runtime,
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
name: CodeQL production
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
schedule:
|
||||
- cron: "37 5 * * 1"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: codeql-production-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
# A repository administrator must disable default setup before enabling
|
||||
# this workflow; default setup rejects advanced CodeQL result uploads.
|
||||
if: vars.REA_CODEQL_ADVANCED == 'true'
|
||||
name: Analyze production (${{ matrix.language }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write
|
||||
strategy:
|
||||
fail-fast: false
|
||||
max-parallel: 1
|
||||
matrix:
|
||||
language: [javascript-typescript, python, actions]
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Initialize production analysis
|
||||
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
build-mode: none
|
||||
config-file: .github/codeql/production.yml
|
||||
threads: 2
|
||||
ram: 4096
|
||||
- name: Analyze production code
|
||||
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
|
||||
with:
|
||||
category: /language:${{ matrix.language }}
|
||||
output: ${{ runner.temp }}/codeql-production-results
|
||||
threads: 2
|
||||
ram: 4096
|
||||
- name: Retain complete analysis evidence
|
||||
if: always()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: codeql-production-${{ matrix.language }}
|
||||
path: ${{ runner.temp }}/codeql-production-results
|
||||
if-no-files-found: error
|
||||
retention-days: 7
|
||||
@@ -62,36 +62,3 @@ jobs:
|
||||
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
|
||||
cat "${probe_root}/output.log"
|
||||
grep -F 'REA_ARM_CONTEXT_OK arm=0 thumb=1 unavailable=null (injected provider value)' "${probe_root}/output.log"
|
||||
# Each probe imports source-owned bytes at 0x1000, away from the Cortex-M
|
||||
# vector symbols that ARMCortex.pspec defines at 0, and decodes them from
|
||||
# the language's default context, so M-profile Thumb state must hold at
|
||||
# the first byte. The bytes are never executed.
|
||||
- name: Decode M-profile ARM and Xtensa bytes in their default context
|
||||
shell: bash
|
||||
run: |
|
||||
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
|
||||
export JAVA_HOME="${!java_home_variable}"
|
||||
probe_root="${RUNNER_TEMP}/rea-decode"
|
||||
mkdir -p "${probe_root}"
|
||||
# mrs r0, msp; bx lr
|
||||
printf '\357\363\010\200\160\107' > "${probe_root}/cortex.bin"
|
||||
# mrs r0, msplim (ARMv8-M only); bx lr
|
||||
printf '\357\363\012\200\160\107' > "${probe_root}/v8m.bin"
|
||||
# entry a1, 32; retw.n
|
||||
printf '\066\101\000\035\360' > "${probe_root}/xtensa.bin"
|
||||
decode() {
|
||||
local name="$1" language="$2"
|
||||
shift 2
|
||||
mkdir -p "${probe_root}/${name}-project"
|
||||
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/${name}-project" Decode \
|
||||
-import "${probe_root}/${name}.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
|
||||
-processor "${language}" -noanalysis -readOnly -deleteProject \
|
||||
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra" \
|
||||
-postScript ReaDecodeProbe.java "${language}" "$@" \
|
||||
> "${probe_root}/${name}.log" 2>&1 || { cat "${probe_root}/${name}.log"; exit 1; }
|
||||
cat "${probe_root}/${name}.log"
|
||||
grep -F "REA_DECODE_OK ${language} $*" "${probe_root}/${name}.log"
|
||||
}
|
||||
decode cortex ARM:LE:32:Cortex mrs/4/msp bx/2/lr
|
||||
decode v8m ARM:LE:32:v8-m mrs/4/msplim bx/2/lr
|
||||
decode xtensa Xtensa:LE:32:default entry/3/a1 retw.n/2
|
||||
@@ -11,7 +11,6 @@ on:
|
||||
- tests/conformance/android/**
|
||||
- tests/fixtures/android/static-initializers/**
|
||||
- tests/fixtures/android/direct-invokes/**
|
||||
- tests/fixtures/android/external-invokes/**
|
||||
- .nvmrc
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
@@ -45,8 +44,7 @@ jobs:
|
||||
"$JAVA_HOME/bin/javac" -J-Xmx512m -J-XX:ActiveProcessorCount=2 \
|
||||
-classpath "$jar" -d "$classes" \
|
||||
bridge/android/ReaJadxBridge.java \
|
||||
tests/conformance/android/ReaJadxMetadataRegression.java \
|
||||
tests/conformance/android/ReaJadxExternalReferencesRegression.java
|
||||
tests/conformance/android/ReaJadxMetadataRegression.java
|
||||
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
|
||||
-classpath "$classes:$jar" ReaJadxMetadataRegression \
|
||||
"$android_root/ApiDemos-debug.apk" \
|
||||
@@ -55,9 +53,3 @@ jobs:
|
||||
| tee "$android_root/metadata.log"
|
||||
grep -F 'RAW_CLASS_SELECTOR_REGRESSION_PASS nested=true metadata=true references=true' \
|
||||
"$android_root/metadata.log"
|
||||
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
|
||||
-classpath "$classes:$jar" ReaJadxExternalReferencesRegression \
|
||||
"$GITHUB_WORKSPACE/tests/fixtures/android/external-invokes" "$classes" \
|
||||
| tee "$android_root/external-references.log"
|
||||
grep -F 'EXTERNAL_REFERENCE_REGRESSION_PASS prototypes=2 using_nodes=3 sites=4' \
|
||||
"$android_root/external-references.log"
|
||||
@@ -55,12 +55,12 @@ jobs:
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Install AArch64 and ELF32 cross toolchains for source-owned fixtures
|
||||
- name: Install ELF32 cross toolchains for source-owned fixtures
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --yes --no-install-recommends \
|
||||
gcc-aarch64-linux-gnu gcc-i686-linux-gnu gcc-arm-linux-gnueabi gcc-mips-linux-gnu gcc-mipsel-linux-gnu \
|
||||
libc6-dev-arm64-cross libc6-dev-i386-cross libc6-dev-armel-cross libc6-dev-mips-cross libc6-dev-mipsel-cross
|
||||
gcc-i686-linux-gnu gcc-arm-linux-gnueabi gcc-mips-linux-gnu gcc-mipsel-linux-gnu \
|
||||
libc6-dev-i386-cross libc6-dev-armel-cross libc6-dev-mips-cross libc6-dev-mipsel-cross
|
||||
- name: Acquire exact upstream libraries in an isolated runner environment
|
||||
shell: bash
|
||||
run: |
|
||||
|
||||
@@ -53,8 +53,3 @@ jobs:
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
# The preceding verifier builds the runtime used by this request journey.
|
||||
- run: dbus-run-session -- node scripts/verify-browser-request.mjs
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
@@ -1,35 +1,39 @@
|
||||
name: Real packet capture
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/captures/**"
|
||||
- "tests/boundary/cli/captureCleanup.test.ts"
|
||||
- "tests/boundary/mcp/packetCaptureMcp.test.ts"
|
||||
- "src/captures/**"
|
||||
- "src/pcap/**"
|
||||
- "src/domain/pcap/**"
|
||||
- "src/domain/analysisView/pcapCaptureView.ts"
|
||||
- "src/application/pcap/**"
|
||||
- "src/contracts/pcap/**"
|
||||
- "src/server/registerPcapTools.ts"
|
||||
- "tests/boundary/pcap/**"
|
||||
- "src/domain/captures/**"
|
||||
- "src/application/captures/**"
|
||||
- "src/contracts/captures/**"
|
||||
- "src/composition/captures.ts"
|
||||
- "src/cli/captureCommands.ts"
|
||||
- "src/server/registerCaptureTools.ts"
|
||||
- "scripts/verify-packet-capture.mjs"
|
||||
- ".github/workflows/real-packet-capture.yml"
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
concurrency:
|
||||
group: real-packet-capture-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
tshark:
|
||||
runs-on: ubuntu-24.04
|
||||
@@ -44,18 +48,8 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run build:cached
|
||||
- name: Run focused tests and require Linux resource and cleanup cases
|
||||
run: |
|
||||
report=".cache/rea-packet-required-tests.json"
|
||||
mkdir -p .cache
|
||||
rm -f "$report"
|
||||
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
|
||||
tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts tests/acceptance/pcapVerifier.test.ts
|
||||
node scripts/ci/require-provider-tests.mjs packet "$report"
|
||||
- run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
|
||||
- run: node scripts/verify-packet-capture.mjs
|
||||
env:
|
||||
REA_TSHARK_COMMAND: /usr/bin/tshark
|
||||
|
||||
@@ -1,35 +1,32 @@
|
||||
name: Real signal recording
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/signals/**"
|
||||
- "tests/boundary/cli/signalCleanup.test.ts"
|
||||
- "tests/boundary/mcp/signalRecordingMcp.test.ts"
|
||||
- "src/signals/**"
|
||||
- "src/domain/signals/**"
|
||||
- "src/application/signals/**"
|
||||
- "src/contracts/signals/**"
|
||||
- "src/composition/signals.ts"
|
||||
- "src/cli/signalCommands.ts"
|
||||
- "src/server/registerSignalTools.ts"
|
||||
- "scripts/verify-signal-recording.mjs"
|
||||
- ".github/workflows/real-signal-recording.yml"
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
concurrency:
|
||||
group: real-signal-recording-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
sigrok:
|
||||
runs-on: ubuntu-24.04
|
||||
@@ -44,18 +41,8 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run build:cached
|
||||
- name: Run focused tests and require Linux resource and cleanup cases
|
||||
run: |
|
||||
report=".cache/rea-signal-required-tests.json"
|
||||
mkdir -p .cache
|
||||
rm -f "$report"
|
||||
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
|
||||
tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
|
||||
node scripts/ci/require-provider-tests.mjs signal "$report"
|
||||
- run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
|
||||
- run: node scripts/verify-signal-recording.mjs
|
||||
env:
|
||||
REA_SIGROK_COMMAND: /usr/bin/sigrok-cli
|
||||
@@ -21,10 +21,6 @@ jobs:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Set up Node.js for the website Worker checks
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
|
||||
@@ -54,10 +54,6 @@ jobs:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Set up Node.js for the website Worker checks
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
@@ -79,21 +75,10 @@ jobs:
|
||||
PY
|
||||
- name: Configure Pages
|
||||
uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
|
||||
# upload-pages-artifact excludes dot directories, including .well-known.
|
||||
- name: Package the website including agent discovery
|
||||
run: |
|
||||
tar --dereference --hard-dereference --directory website/_site \
|
||||
--exclude=.git --exclude=.github -cf "$RUNNER_TEMP/artifact.tar" .
|
||||
tar -tf "$RUNNER_TEMP/artifact.tar" \
|
||||
./.well-known/ard.json ./.well-known/ai-catalog.json \
|
||||
./.well-known/agent-skills/index.json > /dev/null
|
||||
- name: Upload the website
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0
|
||||
with:
|
||||
name: github-pages
|
||||
path: ${{ runner.temp }}/artifact.tar
|
||||
retention-days: 1
|
||||
if-no-files-found: error
|
||||
path: website/_site
|
||||
|
||||
deploy:
|
||||
needs: prepare
|
||||
@@ -145,24 +130,14 @@ jobs:
|
||||
PUBLISH_TARGET: ${{ inputs.target }}
|
||||
run: |
|
||||
python3 -u - <<'PY'
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from urllib.error import URLError
|
||||
from urllib.parse import urljoin, urlsplit
|
||||
from urllib.parse import urljoin
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
def open_public(url, accept=None):
|
||||
headers = {
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
}
|
||||
if accept:
|
||||
headers["Accept"] = accept
|
||||
return urlopen(Request(url, headers=headers), timeout=10)
|
||||
|
||||
expected = os.environ["GITHUB_SHA"]
|
||||
pages_url = os.environ["PAGES_URL"]
|
||||
if not pages_url:
|
||||
@@ -178,7 +153,11 @@ jobs:
|
||||
for name, base in list(pending.items()):
|
||||
url = urljoin(base, "deployment.json") + "?commit=" + expected
|
||||
try:
|
||||
with open_public(url) as response:
|
||||
request = Request(url, headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
})
|
||||
with urlopen(request, timeout=10) as response:
|
||||
actual = json.load(response).get("commit")
|
||||
if actual == expected:
|
||||
print(f"{name} serves {expected}: {base}")
|
||||
@@ -193,27 +172,6 @@ jobs:
|
||||
time.sleep(5)
|
||||
if pending:
|
||||
sys.exit("Could not verify published revision on: " + ", ".join(pending))
|
||||
for name, base in hosts.items():
|
||||
with open_public(urljoin(base, ".well-known/agent-skills/index.json")) as response:
|
||||
skill, = json.load(response)["skills"]
|
||||
# Fetch from each selected host, including the /rea/ mirror.
|
||||
artifact_url = urljoin(base, urlsplit(skill["url"]).path.lstrip("/"))
|
||||
with open_public(artifact_url) as response:
|
||||
digest = "sha256:" + hashlib.sha256(response.read()).hexdigest()
|
||||
if digest != skill["digest"]:
|
||||
sys.exit(name + " serves a skill archive with the wrong digest")
|
||||
for manifest in ("ard.json", "ai-catalog.json"):
|
||||
with open_public(urljoin(base, ".well-known/" + manifest)) as response:
|
||||
if not json.load(response).get("entries"):
|
||||
sys.exit(name + " serves an empty discovery manifest")
|
||||
if name == "Cloudflare":
|
||||
with open_public(base, accept="text/markdown") as response:
|
||||
vary = {value.strip().lower() for value in response.headers.get("Vary", "").split(",")}
|
||||
if response.headers.get_content_type() != "text/markdown" or "accept" not in vary:
|
||||
sys.exit("Cloudflare did not negotiate Markdown with Vary: Accept")
|
||||
if 'rel="ard"' not in response.headers.get("Link", ""):
|
||||
sys.exit("Cloudflare is missing its discovery Link header")
|
||||
print(name + " serves the discovery manifests and verified skill archive")
|
||||
with open(os.environ["GITHUB_STEP_SUMMARY"], "a", encoding="utf-8") as summary:
|
||||
links = " and ".join(f"[{name}]({url})" for name, url in hosts.items())
|
||||
summary.write(f"Published commit `{expected}` to {links}.\n")
|
||||
|
||||
+1
-1
@@ -50,7 +50,7 @@ Thumbs.db
|
||||
|
||||
# Local reference sources and analysis artifacts
|
||||
/_reference/
|
||||
/experiments/analysis-efficiency/
|
||||
/experiments/analysis-efficiency/results/
|
||||
|
||||
# Generated website download
|
||||
/website/public/examples/notes-example.zip
|
||||
|
||||
+1
-4
@@ -10,10 +10,7 @@
|
||||
"node_modules/**",
|
||||
"CHANGELOG.md",
|
||||
"package-lock.json",
|
||||
".claude-plugin/plugin.json",
|
||||
".codex-plugin/plugin.json",
|
||||
"_reference/**",
|
||||
"third_party/**",
|
||||
".codex-plugin/mcp.json"
|
||||
"third_party/**"
|
||||
]
|
||||
}
|
||||
@@ -28,10 +28,6 @@ Assess brittleness in touched code during implementation, debugging, review, and
|
||||
|
||||
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
|
||||
|
||||
After confirming a bug, search sibling adapters, consumers and similar workflows
|
||||
for the same failure. Verify suspected matches, record working comparisons, and
|
||||
group findings by root cause instead of filing duplicate reports.
|
||||
|
||||
When addressing a review thread, inspect the producing boundary and affected consumers for brittleness. Complete directly related refactoring in the same change. Before resolving the thread, explain the root cause, structural fix, validation, and any remaining cleanup.
|
||||
|
||||
Parse unknown boundary values and model expected failures with the tagged error algebra and `Result`. Preserve partial facts and meaningful failure reasons. Keep identity and lookup values separate from display formatting; preserve source values and explicit unknowns when normalization loses information. Absolute paths, file URLs, and HTTP paths have different semantics.
|
||||
@@ -79,9 +75,6 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
|
||||
|
||||
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
|
||||
|
||||
When changing installers or setup recovery, follow
|
||||
[Checking installers and publisher downloads](docs/testing.md#checking-installers-and-publisher-downloads).
|
||||
|
||||
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/testing.md#developer-commands).
|
||||
|
||||
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
|
||||
@@ -90,12 +83,6 @@ Coordinate expensive checks across agents. If a test unexpectedly consumes heavy
|
||||
|
||||
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
|
||||
|
||||
For user- or agent-facing tool problems, prioritize fixing the owning behavior,
|
||||
contract, and actionable result or recovery at the point of use. Document the
|
||||
result in product docs when explanation is needed. Add skill guidance only for
|
||||
workflow judgment or context that the tool cannot reasonably convey; do not use
|
||||
a skill as the primary repair for a discoverability or contract defect.
|
||||
|
||||
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Select applicable [boundary regressions](docs/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
|
||||
|
||||
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
|
||||
@@ -128,8 +115,8 @@ owner to complete. Never attest to human review on their behalf. Draft status
|
||||
is optional; preserve the PR's current draft status unless the user asks to change
|
||||
it. Later pushes do not invalidate the readiness declaration or require another
|
||||
human sign-off. Keep the PR body and affected validation current as work changes.
|
||||
The readiness check validates the filled declaration or the author exemption
|
||||
listed in CONTRIBUTING.md; CI and automated review results do not establish that
|
||||
The readiness check validates
|
||||
the filled declaration; CI and automated review results do not establish that
|
||||
human review occurred, maintainer scope approval, or merge permission.
|
||||
|
||||
<!-- BEGIN:turborepo-agent-rules -->
|
||||
|
||||
+2
-2
@@ -2683,7 +2683,7 @@ Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. R
|
||||
* enforce conventional pull request titles ([3a21f16](https://github.com/morluto/rea/commit/3a21f16c1832430765083636511a8c12dbccf2dd))
|
||||
* verify installed package with Linux Hopper ([4f97f69](https://github.com/morluto/rea/commit/4f97f699e4d5471557d1ce0ff91e144065d06a61))
|
||||
|
||||
## [0.3.0](https://github.com/morluto/rea/compare/rea-0.2.1...rea-agents-0.3.0) (2026-07-12)
|
||||
## [0.3.0](https://github.com/morluto/rea/compare/rea-agents-0.2.1...rea-agents-0.3.0) (2026-07-12)
|
||||
|
||||
|
||||
### Features
|
||||
@@ -2736,7 +2736,7 @@ Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. R
|
||||
* redesign and localize README ([051b6db](https://github.com/morluto/rea/commit/051b6db65687fd14826b6462cd623feb2504f123))
|
||||
* redesign and localize README ([910eda6](https://github.com/morluto/rea/commit/910eda6f323ba1151b0771fe5de6c60a91e3d46d))
|
||||
|
||||
## [0.2.0](https://github.com/morluto/rea/releases/tag/rea-0.2.0) (2026-07-12)
|
||||
## [0.2.0](https://github.com/morluto/rea/compare/rea-0.1.0...rea-0.2.0) (2026-07-12)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
+9
-26
@@ -16,9 +16,10 @@ import inventory. For current provider wiring, start with
|
||||
[`src/composition/binary.ts`](src/composition/binary.ts) for deep candidates and
|
||||
the other `src/composition/` factories for tool services, including
|
||||
[`gpu.ts`](src/composition/gpu.ts) and [`model.ts`](src/composition/model.ts).
|
||||
Live Frida, Cutter, GDB and Rizin construction also uses shared composition
|
||||
factories. For service ownership, availability and catalog declarations, follow
|
||||
the [capability composition guide](docs/capability-composition.md).
|
||||
Live Frida and Cutter adapters are constructed directly in
|
||||
[`src/cli/fridaCommands.ts`](src/cli/fridaCommands.ts),
|
||||
[`src/cli/cutterCommands.ts`](src/cli/cutterCommands.ts), and
|
||||
[`src/server/createServer.ts`](src/server/createServer.ts).
|
||||
Production factories construct fresh providers without starting engines or
|
||||
acquiring targets; optional adapter failures must preserve successful peers. Run
|
||||
`npm run verify:test-discovery` after adding or moving tests. Resolved import
|
||||
@@ -93,8 +94,7 @@ Before requesting maintainer review, the human owner must:
|
||||
passing. Follow the [verification requirements](#development-feedback-and-pr-verification)
|
||||
for required checks and real-provider coverage.
|
||||
- Complete the **Author readiness** declaration in the PR template, explicitly
|
||||
stating that the PR is ready for initial review, unless the PR author is
|
||||
exempt from the declaration as listed below.
|
||||
stating that the PR is ready for initial review.
|
||||
|
||||
Keep the PR description concise and current. Explain the problem, observable
|
||||
outcome, approach, and useful evidence; link long logs or put them in optional
|
||||
@@ -115,12 +115,9 @@ validation current as work changes. AI agents must leave human declarations to
|
||||
the human owner and must not attest to a human review on their behalf.
|
||||
|
||||
The `PR readiness` check reports pending for drafts and requires a named human
|
||||
owner and all three template declarations for non-drafts. PRs authored by
|
||||
@morluto, @kaoru0822-kitauji, @N0zoM1z0, or @gensokyo-dev are exempt from
|
||||
the declaration; their human ownership, review, and evidence responsibilities
|
||||
above still apply. The check identifies authors by GitHub account ID. It checks
|
||||
that the declaration is filled, not whether the code or evidence is correct.
|
||||
Existing PRs can copy the **Author readiness** section from the current template.
|
||||
owner and all three template declarations for non-drafts. It checks that the
|
||||
declaration is filled, not whether the code or evidence is correct. Existing PRs
|
||||
can copy the **Author readiness** section from the current template.
|
||||
Open PRs must have distinct head commits because GitHub shares commit statuses.
|
||||
|
||||
Maintainers enable merge enforcement after the workflow is merged: run
|
||||
@@ -235,8 +232,6 @@ Use the development toolchain pinned in `.nvmrc` and
|
||||
`package.json#packageManager`. Run `nvm use` before installing dependencies;
|
||||
installed-package runtime support is documented in
|
||||
[installation](docs/installation.md#start-setup).
|
||||
Use Node.js and npm for source builds and verifier scripts; the installed
|
||||
package's Bun runtime support does not extend to those development scripts.
|
||||
|
||||
```bash
|
||||
npm ci
|
||||
@@ -251,15 +246,6 @@ catalogs, documentation projections, or managed verification evidence. Turbo
|
||||
caches deterministic builds and static checks across Git worktrees. After a package, lockfile, or managed-skill version change, run
|
||||
`npm run metadata:generate` before building.
|
||||
|
||||
Dependabot proposes updates for the npm lockfiles, website Python requirements,
|
||||
and GitHub Actions. Review dependency changes and run the checks relevant to
|
||||
their consumers. For dependencies maintained outside Dependabot, including
|
||||
`third_party/` and the HTTP/3 lab's pinned requirements, the maintainer should
|
||||
review upstream security advisories when updating or using them. Preserve source
|
||||
and version provenance, update license notices when needed, and report which
|
||||
verification was actually repeated. The lab requirements describe a recorded
|
||||
environment; a dependency update does not verify the revised environment.
|
||||
|
||||
Repository invariants live in [AGENTS.md](AGENTS.md). Keep provider-specific
|
||||
protocols in adapters and shared CLI/MCP workflows in the application layer.
|
||||
|
||||
@@ -286,10 +272,7 @@ The general source suite (`npm run test:fast`, `npm run test:changed`, and
|
||||
On a Windows checkout, some source tests fail because fixtures use POSIX paths,
|
||||
`#!/bin/sh` tool stubs, or POSIX rename semantics. Compare failures against the
|
||||
base revision, or run the suite on Linux or WSL, before attributing them to a
|
||||
change. The embedded-artifact parent-swap tests exercise the selected-path
|
||||
replacement when the rename succeeds; if Windows refuses it while the admitted
|
||||
file is open, they instead verify the refusal and cleanup. See
|
||||
[testing](docs/testing.md#developer-commands) for command scopes
|
||||
change. See [testing](docs/testing.md#developer-commands) for command scopes
|
||||
and lane selection.
|
||||
|
||||
`git submodule update --init --recursive` fails at the pinned Unblob submodule
|
||||
|
||||
@@ -20,8 +20,6 @@
|
||||
|
||||
**[Website](https://rea.tools/) · [Guides](https://rea.tools/guides/) · [Showcases](https://rea.tools/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Join the REA community on Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [Build RL environments](#build-rl-environments) · [FAQ](#faq) · [Documentation](#documentation)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
@@ -64,7 +62,7 @@ See a feature in an app that you want in your own product? Ask your agent to inv
|
||||
|
||||
REA connects your agent to tools for inspecting native binaries, JavaScript and Electron apps, .NET assemblies, and websites. You can also use the same tools from your terminal. Analysis runs locally, and results include the evidence and limitations behind each conclusion.
|
||||
|
||||
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, IDA or Kuna installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
|
||||
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, or IDA installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
|
||||
|
||||
> **[Visit the REA website](https://rea.tools/)** for setup instructions, illustrated guides, and real case studies.
|
||||
|
||||
@@ -85,15 +83,12 @@ npx rea-agents setup
|
||||
|
||||
Choose your agents, review the proposed changes, and approve them. Setup adds
|
||||
REA's MCP server and matching workflow instructions, with backups of existing
|
||||
configuration. Restart your agent afterward. Setup records the selected runtime and REA
|
||||
entry script directly. If acquired through a package runner, keep its cache entry
|
||||
available. Run package acquisition from a trusted directory outside the target
|
||||
project. Installation and recovery details are in the setup guide below.
|
||||
configuration. Restart your agent afterward.
|
||||
|
||||
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
|
||||
[other agents](docs/installation.md#supported-agents). See
|
||||
[installation and setup](docs/installation.md) for provider configuration and
|
||||
manual MCP registration and Claude Code/Codex plugin installation.
|
||||
manual MCP registration.
|
||||
|
||||
MCP uses compact input schemas by default to reduce context overhead. Set
|
||||
`REA_MCP_INPUT_SCHEMA_PROFILE=full` in your MCP registration when you need the
|
||||
@@ -194,7 +189,7 @@ Additional tools and host support depend on the target:
|
||||
| Target | What REA returns | Requirements and guide |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/native-investigation.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
|
||||
@@ -346,13 +341,11 @@ Any agent that supports local MCP servers. Setup configures the
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Do I need a native analysis engine?</strong></summary>
|
||||
<summary><strong>Do I need Hopper, Ghidra or IDA?</strong></summary>
|
||||
|
||||
Deep native analysis uses Hopper, Ghidra, IDA or Kuna, depending on the target
|
||||
and operation. Static JavaScript and .NET inspection work without a native
|
||||
analysis engine. Setup can install Hopper after approval; Ghidra, IDA and Kuna
|
||||
use your existing installations. See [provider setup](docs/installation.md#hopper)
|
||||
for the options and their supported targets.
|
||||
Deep native analysis uses one of them. Static JavaScript and .NET inspection
|
||||
work without a native analysis engine. Setup can install Hopper after approval;
|
||||
Ghidra and IDA use your existing installations. See [provider setup](docs/installation.md#hopper).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -432,7 +425,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Star history
|
||||
|
||||
🎉 **400,000 GitHub stars — thank you!**
|
||||
🎉 **300,000 GitHub stars — thank you!**
|
||||
|
||||
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
|
||||
|
||||
|
||||
@@ -17,7 +17,6 @@ notices supplied with the software you actually obtain or redistribute.
|
||||
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| npm dependencies | [Complete inventory](licenses/npm-dependencies.txt) of every entry in `package-lock.json`, including transitive, optional platform, development, and website dependencies. [Runtime notices](licenses/npm-runtime-notices.txt) preserve the copyright, license, and NOTICE documents from integrity-verified release archives, plus the MIT notice for Rolldown's generated runtime helpers. Development dependencies are inventoried but are not all distributed in REA's npm package. |
|
||||
| pefile ordinal tables | Import-hash ordinal names are derived from [pefile 2024.8.26](https://github.com/erocarrera/pefile/tree/4b3b1e2e568a88d4f1897d694d684f23d9e270c4), copyright 2004–2024 Ero Carrera. Preserve the [MIT license](licenses/pefile-2024.8.26-LICENSE.txt). The Python package is not a runtime dependency. |
|
||||
| UF2 family names | UF2 family IDs and short names are copied from [microsoft/uf2 `utils/uf2families.json`](https://github.com/microsoft/uf2/blob/cf3bd7f9c274b949528f8cd6a19eef2502a687d2/utils/uf2families.json), copyright Microsoft Corporation. Preserve the [MIT license](licenses/microsoft-uf2-MIT.txt). |
|
||||
| NativeAOT parser | REA's derived parser uses the MIT-licensed [ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot/tree/effeb734fc570c32650f88b159608979dc7b423e) work of Washi and the NativeAOT layout definitions in [dotnet/runtime](https://github.com/dotnet/runtime/tree/a3b2d40328be0be3f8dd43f0e5cc925b8827b044). Preserve [Washi's MIT notice](licenses/nativeaot-parser-MIT.txt) and the [.NET Foundation MIT notice](licenses/dotnet-runtime-MIT.txt). |
|
||||
| EVM analysis | `[email protected]`, MIT, copyright 2023 Maxim Andreev. The published npm archive omits its license document; REA retains the [original upstream license](third_party/evmole/LICENSE) and [provenance](third_party/evmole/README.md). It is also included in the runtime notices. |
|
||||
| Windows native addon | The release addon is compiled with MinGW-w64 and statically links GCC runtime libraries. Preserve the [MinGW-w64 runtime notices](licenses/MinGW-w64-runtime.txt), [GPLv3 text](licenses/GPL-3.0.txt), and [GCC Runtime Library Exception 3.1](licenses/GCC-Runtime-Library-Exception-3.1.txt). The exception governs eligible compiled combinations; it does not relicense the runtime itself under MIT. The Node-API build headers are from Node 22.19.0; its [original license and component notices](licenses/Node-22.19.0-LICENSE.txt) are retained. Node itself is supplied by the user. Snapshot provenance and release-toolchain review requirements are in [licenses/README.md](licenses/README.md). |
|
||||
@@ -60,7 +59,7 @@ pin inspection sources; they are not included in the npm package.
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.833/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.833/NOTICE) | Bring your own `kuna`; REA verified 1.833 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
# Plugin icon
|
||||
|
||||
`rea-plugin.png` is a square adaptation of the left panel of REA's GitHub
|
||||
social preview, retrieved on 2026-10-28:
|
||||
|
||||
https://repository-images.githubusercontent.com/1209966933/1a3fef9a-be4b-41f4-8830-a94c0e8288cc
|
||||
|
||||
The image was adapted with imagegen, preserving the cyan REA lettering,
|
||||
pixel typography, orange rule, and taglines. Both plugin manifests reference
|
||||
this shared asset; only the PNG is included in the npm package.
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 1.1 MiB |
@@ -1,9 +1,5 @@
|
||||
import com.atxx.jhmcp.*;
|
||||
import com.google.gson.*;
|
||||
import com.android.tools.smali.dexlib2.Opcodes;
|
||||
import com.android.tools.smali.dexlib2.dexbacked.DexBackedDexFile;
|
||||
import com.android.tools.smali.dexlib2.dexbacked.DexBackedClassDef;
|
||||
import com.android.tools.smali.dexlib2.dexbacked.DexBackedMethod;
|
||||
import io.modelcontextprotocol.kotlin.sdk.server.*;
|
||||
import io.modelcontextprotocol.kotlin.sdk.types.*;
|
||||
import jadx.api.*;
|
||||
@@ -13,23 +9,14 @@ import jadx.api.plugins.input.data.ICodeReader;
|
||||
import jadx.api.plugins.input.data.IMethodRef;
|
||||
import jadx.api.plugins.input.insns.InsnIndexType;
|
||||
import jadx.api.plugins.input.insns.Opcode;
|
||||
import jadx.core.codegen.TypeGen;
|
||||
import jadx.core.dex.info.MethodInfo;
|
||||
import jadx.core.dex.instructions.args.ArgType;
|
||||
import jadx.core.dex.nodes.*;
|
||||
import jadx.core.xmlgen.BinaryXMLParser;
|
||||
import java.io.PrintStream;
|
||||
import java.io.StringReader;
|
||||
import java.io.InputStream;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.CountDownLatch;
|
||||
import java.util.stream.Collectors;
|
||||
import java.util.zip.CRC32;
|
||||
import java.util.zip.ZipEntry;
|
||||
import java.util.zip.ZipFile;
|
||||
import kotlin.Unit;
|
||||
import kotlin.ResultKt;
|
||||
import kotlin.coroutines.Continuation;
|
||||
@@ -52,25 +39,17 @@ class ReaJadxBridge {
|
||||
private Map<String, List<ClassSnapshot>> classes = Map.of();
|
||||
private JadxSession manifestSession;
|
||||
private ManifestSnapshot manifestSnapshot;
|
||||
private ExternalInventory externalInventory;
|
||||
|
||||
private record MethodSnapshot(String name, String rawName, String shortId, List<String> bodyAbsenceMarkers,
|
||||
Map<String, Object> summary) {}
|
||||
private record ClassSnapshot(JavaClass node, List<MethodSnapshot> methods, Map<String, Object> summary,
|
||||
Map<String, Object> summaryWithValues) {}
|
||||
private record ManifestSnapshot(String text, Map<String, Object> summary) {}
|
||||
private record ExternalMethod(MethodInfo info, Map<String, Object> summary) {}
|
||||
private record ExternalInventory(JadxSession session, String className, String methodName,
|
||||
List<ExternalMethod> methods) {}
|
||||
|
||||
private static final class InputFailure extends IllegalArgumentException {
|
||||
final String field;
|
||||
InputFailure(String field, String message) { super(message); this.field = field; }
|
||||
}
|
||||
private static final class ReferenceLimit extends IllegalArgumentException {
|
||||
final String stage;
|
||||
ReferenceLimit(String stage) { super("External reference representation limit"); this.stage = stage; }
|
||||
}
|
||||
|
||||
private ReaJadxBridge(SessionHolder holder) { this.holder = holder; }
|
||||
|
||||
@@ -197,206 +176,6 @@ class ReaJadxBridge {
|
||||
private void clearManifest() {
|
||||
manifestSession = null;
|
||||
manifestSnapshot = null;
|
||||
externalInventory = null;
|
||||
}
|
||||
|
||||
// One bounded DEX at a time, beside the already admitted analysis JVM. The
|
||||
// response allowance accounts for worst-case JSON escaping before retention.
|
||||
private static final int NATIVE_DEX_BYTES = 64 * 1024 * 1024;
|
||||
private static final long NATIVE_TOTAL_DEX_BYTES = 256L * 1024 * 1024;
|
||||
// This JSON is escaped again inside an 8 MiB MCP frame.
|
||||
private static final long NATIVE_JSON_BYTES = 4L * 1024 * 1024;
|
||||
|
||||
/** The bundled decoder allocates its declared UTF-16 length before validating bytes. */
|
||||
private static final class NativeDex extends DexBackedDexFile {
|
||||
private OptionalIndexedSection<String> boundedStrings;
|
||||
NativeDex(byte[] bytes) { super(Opcodes.getDefault(), bytes); }
|
||||
|
||||
@Override public OptionalIndexedSection<String> getStringSection() {
|
||||
if (boundedStrings == null) {
|
||||
var source = super.getStringSection();
|
||||
boundedStrings = new OptionalIndexedSection<>() {
|
||||
@Override public int size() { return source.size(); }
|
||||
@Override public int getOffset(int index) { return source.getOffset(index); }
|
||||
@Override public String getOptional(int index) { return index == -1 ? null : get(index); }
|
||||
@Override public String get(int index) {
|
||||
int offset = getBuffer().readSmallUint(source.getOffset(index));
|
||||
int units = getDataBuffer().readerAt(offset).readSmallUleb128();
|
||||
if (units > NATIVE_JSON_BYTES / 6
|
||||
|| units > getDataBuffer().getBuf().length - offset)
|
||||
throw new IllegalArgumentException("DEX string exceeds the native decoding allowance");
|
||||
return source.get(index);
|
||||
}
|
||||
};
|
||||
}
|
||||
return boundedStrings;
|
||||
}
|
||||
}
|
||||
|
||||
private static void checkNativeInterrupted() {
|
||||
if (Thread.currentThread().isInterrupted())
|
||||
throw new java.util.concurrent.CancellationException("Native declaration inventory interrupted");
|
||||
}
|
||||
|
||||
private static final class NativeInventory {
|
||||
final List<Map<String, Object>> methods = new ArrayList<>();
|
||||
final List<String> unknowns = new ArrayList<>();
|
||||
long retainedBytes = 1024;
|
||||
int dexEntries;
|
||||
int examinedDexEntries;
|
||||
int examinedClasses;
|
||||
int examinedMethods;
|
||||
boolean partial;
|
||||
boolean diagnosticLimit;
|
||||
|
||||
void unknown(String reason) {
|
||||
partial = true;
|
||||
long bytes = 64L + reason.length() * 6L;
|
||||
if (unknowns.size() < 16 && retainedBytes + bytes <= NATIVE_JSON_BYTES - 4096) {
|
||||
unknowns.add(reason);
|
||||
retainedBytes += bytes;
|
||||
} else if (!diagnosticLimit) {
|
||||
diagnosticLimit = true;
|
||||
unknowns.add("Additional native-inventory limitations exceed the retained diagnostic allowance.");
|
||||
}
|
||||
}
|
||||
|
||||
boolean retain(Map<String, Object> method, long bytes) {
|
||||
if (retainedBytes + bytes > NATIVE_JSON_BYTES - 4096) {
|
||||
unknown("Native declarations exceed the 4 MiB retained JSON allowance; remaining declarations are unknown.");
|
||||
return false;
|
||||
}
|
||||
retainedBytes += bytes;
|
||||
methods.add(method);
|
||||
return true;
|
||||
}
|
||||
|
||||
Map<String, Object> result() {
|
||||
return Map.of("scope", "apk-root-dex-declarations", "methods", List.copyOf(methods),
|
||||
"coverage", Map.of("status", partial ? "partial" : "complete",
|
||||
"dex_entries", dexEntries, "examined_dex_entries", examinedDexEntries,
|
||||
"examined_classes", examinedClasses, "examined_methods", examinedMethods),
|
||||
"unknowns", List.copyOf(unknowns));
|
||||
}
|
||||
}
|
||||
|
||||
/** Match Android's standard root multidex carriers, without a basename guess. */
|
||||
private static boolean nativeDexEntry(String name) {
|
||||
return name.equals("classes.dex") || name.matches("classes[0-9]+\\.dex");
|
||||
}
|
||||
|
||||
private static byte[] readNativeDex(ZipFile archive, ZipEntry entry) throws IOException {
|
||||
long size = entry.getSize();
|
||||
if (size < 0 || size > NATIVE_DEX_BYTES)
|
||||
throw new IllegalArgumentException("DEX exceeds the 64 MiB single-member read allowance");
|
||||
byte[] bytes = new byte[(int) size];
|
||||
CRC32 crc = new CRC32();
|
||||
try (InputStream input = archive.getInputStream(entry)) {
|
||||
int offset = 0;
|
||||
while (offset < bytes.length) {
|
||||
checkNativeInterrupted();
|
||||
int read = input.read(bytes, offset, Math.min(65536, bytes.length - offset));
|
||||
if (read < 0) throw new IOException("DEX is shorter than its ZIP declaration");
|
||||
if (read == 0) continue;
|
||||
crc.update(bytes, offset, read);
|
||||
offset += read;
|
||||
}
|
||||
if (input.read() != -1) throw new IOException("DEX exceeds its ZIP declaration");
|
||||
}
|
||||
if (entry.getCrc() < 0 || entry.getCrc() != crc.getValue())
|
||||
throw new IOException("DEX does not match its ZIP CRC32 declaration");
|
||||
return bytes;
|
||||
}
|
||||
|
||||
/** Raw DEX ownership precedes JADX's duplicate-class selection and aliases. */
|
||||
static Map<String, Object> nativeDeclarations(Path apk) {
|
||||
NativeInventory inventory = new NativeInventory();
|
||||
try (ZipFile archive = new ZipFile(apk.toFile())) {
|
||||
Map<String, Integer> names = new HashMap<>();
|
||||
var directory = archive.entries();
|
||||
while (directory.hasMoreElements()) {
|
||||
checkNativeInterrupted();
|
||||
ZipEntry entry = directory.nextElement();
|
||||
if (!entry.isDirectory() && nativeDexEntry(entry.getName())) {
|
||||
inventory.dexEntries++;
|
||||
names.merge(entry.getName(), 1, Integer::sum);
|
||||
}
|
||||
}
|
||||
long totalBytes = 0;
|
||||
int entryIndex = -1;
|
||||
directory = archive.entries();
|
||||
while (directory.hasMoreElements()) {
|
||||
checkNativeInterrupted();
|
||||
ZipEntry entry = directory.nextElement();
|
||||
entryIndex++;
|
||||
String path = entry.getName();
|
||||
if (entry.isDirectory() || !nativeDexEntry(path)) continue;
|
||||
if (names.get(path) != 1) {
|
||||
inventory.unknown("Duplicate root DEX entry name cannot be selected uniquely by the ZIP reader: " + path);
|
||||
continue;
|
||||
}
|
||||
if (entry.getSize() < 0 || entry.getSize() > NATIVE_DEX_BYTES
|
||||
|| entry.getSize() > NATIVE_TOTAL_DEX_BYTES - totalBytes) {
|
||||
inventory.unknown("Root DEX member exceeds the 64 MiB member or 256 MiB total read allowance: " + path);
|
||||
continue;
|
||||
}
|
||||
totalBytes += entry.getSize();
|
||||
try {
|
||||
DexBackedDexFile dex = new NativeDex(readNativeDex(archive, entry));
|
||||
var classes = dex.getClassSection();
|
||||
for (int classIndex = 0; classIndex < classes.size(); classIndex++) {
|
||||
checkNativeInterrupted();
|
||||
DexBackedClassDef cls = classes.get(classIndex);
|
||||
String type = cls.getType();
|
||||
if (!type.startsWith("L") || !type.endsWith(";"))
|
||||
throw new IllegalArgumentException("Class definition has no reference type");
|
||||
String className = type.substring(1, type.length() - 1).replace('/', '.');
|
||||
// Passing false retains duplicate encoded method references too.
|
||||
for (Iterable<? extends DexBackedMethod> group : List.of(
|
||||
cls.getDirectMethods(false), cls.getVirtualMethods(false))) {
|
||||
for (DexBackedMethod method : group) {
|
||||
checkNativeInterrupted();
|
||||
inventory.examinedMethods++;
|
||||
if ((method.getAccessFlags() & 0x100) == 0) continue;
|
||||
String methodName = method.getName();
|
||||
String returnType = method.getReturnType();
|
||||
long characters = 2L + returnType.length();
|
||||
for (CharSequence parameter : method.getParameterTypes()) {
|
||||
characters += parameter.length();
|
||||
if (characters * 6L > NATIVE_JSON_BYTES) break;
|
||||
}
|
||||
long bytes = 512L + 6L * (className.length() + methodName.length()
|
||||
+ path.length() + characters);
|
||||
if (inventory.retainedBytes + bytes > NATIVE_JSON_BYTES - 4096) {
|
||||
inventory.unknown("Native declarations exceed the 4 MiB retained JSON allowance; remaining declarations are unknown.");
|
||||
return inventory.result();
|
||||
}
|
||||
StringBuilder descriptor = new StringBuilder((int) characters).append('(');
|
||||
for (CharSequence parameter : method.getParameterTypes()) descriptor.append(parameter);
|
||||
descriptor.append(')').append(returnType);
|
||||
Map<String, Object> row = Map.of("source_index", inventory.methods.size(),
|
||||
"class_name", className, "method_name", methodName, "descriptor", descriptor.toString(),
|
||||
"dex_entry", path, "dex_entry_index", entryIndex, "class_index", classIndex,
|
||||
"method_index", method.getMethodIndex());
|
||||
if (!inventory.retain(row, bytes)) return inventory.result();
|
||||
}
|
||||
}
|
||||
inventory.examinedClasses++;
|
||||
}
|
||||
inventory.examinedDexEntries++;
|
||||
} catch (IOException | RuntimeException failure) {
|
||||
if (failure instanceof java.util.concurrent.CancellationException) throw (RuntimeException) failure;
|
||||
inventory.unknown("Root DEX native declarations could not be fully read from " + path + ": "
|
||||
+ failure.getClass().getSimpleName()
|
||||
+ (failure.getMessage() == null ? "" : " (" + failure.getMessage() + ")"));
|
||||
}
|
||||
}
|
||||
} catch (IOException failure) {
|
||||
// A failed archive close must fail the operation so the provider retires
|
||||
// its owned JVM; do not cache a result with lost file-resource ownership.
|
||||
throw new IllegalArgumentException("APK ZIP native declaration inventory could not finish its reader", failure);
|
||||
}
|
||||
return inventory.result();
|
||||
}
|
||||
|
||||
/** Original archive identity, never a deobfuscated basename or name suffix. */
|
||||
@@ -411,7 +190,10 @@ class ReaJadxBridge {
|
||||
selected = resource;
|
||||
}
|
||||
if (selected == null) throw new IllegalArgumentException("Root AndroidManifest.xml not found");
|
||||
String xml = manifestText(selected);
|
||||
var content = selected.loadContent();
|
||||
if (content == null || content.getDataType() != jadx.core.xmlgen.ResContainer.DataType.TEXT)
|
||||
throw new IllegalArgumentException("Root AndroidManifest.xml did not decode as text");
|
||||
String xml = content.getText().getCodeStr();
|
||||
ManifestSnapshot snapshot = new ManifestSnapshot(xml, manifestSummary(xml));
|
||||
manifestSnapshot = snapshot;
|
||||
manifestSession = current;
|
||||
@@ -419,23 +201,6 @@ class ReaJadxBridge {
|
||||
return manifestSnapshot;
|
||||
}
|
||||
|
||||
/** Decode with resource names but without JADX's class alias substitution. */
|
||||
static String manifestText(ResourceFile resource) {
|
||||
var decompiler = resource.getDecompiler();
|
||||
// BinaryXMLParser resolves android:name against its root's class identities
|
||||
// and replaces raw '$' and relative names with Java-style aliases. A fresh
|
||||
// root owns no classes, while its resource-name table preserves reference
|
||||
// rendering from the loaded target.
|
||||
RootNode root = new RootNode(decompiler.getArgs());
|
||||
root.getConstValues().setResourcesNames(decompiler.getRoot().getConstValues().getResourcesNames());
|
||||
try {
|
||||
return ResourcesLoader.decodeStream(resource,
|
||||
(size, input) -> new BinaryXMLParser(root).parse(input).getCodeStr());
|
||||
} catch (jadx.core.utils.exceptions.JadxException failure) {
|
||||
throw new IllegalArgumentException("Root AndroidManifest.xml did not decode as text", failure);
|
||||
}
|
||||
}
|
||||
|
||||
/** One parsed XML owner supplies namespace-aware values alongside the exact decoded text. */
|
||||
static Map<String, Object> manifestSummary(String xml) {
|
||||
Element root;
|
||||
@@ -479,7 +244,7 @@ class ReaJadxBridge {
|
||||
return Collections.unmodifiableMap(summary);
|
||||
}
|
||||
|
||||
private void indexClasses() {
|
||||
private ClassSnapshot selected(JsonObject args) {
|
||||
JadxSession current = session();
|
||||
if (indexedSession != current) {
|
||||
Map<String, List<ClassSnapshot>> index = new LinkedHashMap<>();
|
||||
@@ -505,7 +270,7 @@ class ReaJadxBridge {
|
||||
.map(Object::toString).collect(Collectors.joining(", "))
|
||||
+ "): " + method.getReturnType().toString();
|
||||
return new MethodSnapshot(name, rawName, shortId, List.copyOf(bodyAbsenceMarkers), Map.of(
|
||||
"name", name, "raw_name", rawName, "signature", signature,
|
||||
"name", name, "signature", signature,
|
||||
"dex_descriptor", descriptor,
|
||||
"is_constructor", method.isConstructor(), "def_pos", 0));
|
||||
}).toList();
|
||||
@@ -515,11 +280,8 @@ class ReaJadxBridge {
|
||||
List<Map<String, Object>> valuedFields = new ArrayList<>();
|
||||
boolean anyValue = false;
|
||||
for (FieldNode field : orderedFields) {
|
||||
// Raw DEX identity for mapping lookups; the display type can carry generics.
|
||||
Map<String, Object> entry = Map.of("name", field.getAlias(),
|
||||
"raw_name", field.getFieldInfo().getName(),
|
||||
"type", field.getType().toString(),
|
||||
"dex_type", TypeGen.signature(field.getFieldInfo().getType()), "def_pos", 0);
|
||||
"type", field.getType().toString(), "def_pos", 0);
|
||||
fields.add(entry);
|
||||
Map<String, Object> value = dexStaticValue(field);
|
||||
if (value == null) valuedFields.add(entry);
|
||||
@@ -531,11 +293,10 @@ class ReaJadxBridge {
|
||||
}
|
||||
}
|
||||
List<String> inner = node.getInnerClasses().stream().map(ClassNode::getFullName).toList();
|
||||
List<String> innerRaw = node.getInnerClasses().stream().map(ClassNode::getRawName).toList();
|
||||
Map<String, Object> summary = Map.of(
|
||||
"full_name", cls.getFullName(), "raw_name", cls.getRawName(), "name", cls.getName(),
|
||||
"method_count", methods.size(), "field_count", fields.size(),
|
||||
"inner_class_count", inner.size(), "inner_classes", inner, "inner_class_raw_names", innerRaw,
|
||||
"inner_class_count", inner.size(), "inner_classes", inner,
|
||||
"methods", methods.stream().map(MethodSnapshot::summary).toList(), "fields", List.copyOf(fields));
|
||||
// DEX static values are opt-in, so method and reference lookups keep the member-only reply.
|
||||
Map<String, Object> summaryWithValues = summary;
|
||||
@@ -555,10 +316,6 @@ class ReaJadxBridge {
|
||||
classes = Collections.unmodifiableMap(index);
|
||||
indexedSession = current;
|
||||
}
|
||||
}
|
||||
|
||||
private ClassSnapshot selected(JsonObject args) {
|
||||
indexClasses();
|
||||
String name = string(args, "class_name");
|
||||
List<ClassSnapshot> candidates = classes.get(name);
|
||||
if (candidates == null) throw new InputFailure("class_name", "Exact class not found: " + name
|
||||
@@ -658,18 +415,12 @@ class ReaJadxBridge {
|
||||
// JSON-RPC/CallToolResult envelope and generated numeric request id, and
|
||||
// account for JSON text being escaped again inside TextContent.
|
||||
private static final int MAX_XREF_TEXT_BYTES = 8 * 1024 * 1024 - 1024;
|
||||
// Leave room for the prototype reply, reference reply, retained raw calls,
|
||||
// normalized Evidence and their text/structured MCP copies, not just one frame.
|
||||
private static final int MAX_EXTERNAL_XREF_BYTES = MAX_XREF_TEXT_BYTES / 8;
|
||||
private static int escapedJsonBytes(Object value) {
|
||||
return JSON.toJson(JSON.toJson(value)).getBytes(StandardCharsets.UTF_8).length;
|
||||
}
|
||||
private static final class InvokeBudget {
|
||||
long remaining;
|
||||
InvokeBudget(long remaining) { this.remaining = Math.max(0, remaining); }
|
||||
boolean fitsText(long characters) {
|
||||
return remaining >= 256 && characters <= (remaining - 256) / 12;
|
||||
}
|
||||
boolean admit(Map<String, Object> site) {
|
||||
// Check a conservative two-layer escape bound before serialization.
|
||||
long bound = 256;
|
||||
@@ -684,55 +435,26 @@ class ReaJadxBridge {
|
||||
}
|
||||
private record DirectInvokes(List<Map<String, Object>> sites, boolean exhaustive) {}
|
||||
|
||||
@FunctionalInterface private interface InvokeVisitor {
|
||||
void accept(MethodInfo target, int offset, int index, Opcode opcode);
|
||||
}
|
||||
|
||||
/** Decode raw invoke identity once at the shared bridge boundary, without generating source. */
|
||||
private static void visitInvokes(MethodNode node, InvokeVisitor visitor) {
|
||||
checkNativeInterrupted();
|
||||
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
|
||||
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
|
||||
MethodNode node = source.getMethodNode();
|
||||
ICodeReader code = node.getCodeReader();
|
||||
if (code == null) return;
|
||||
if (code == null) return null;
|
||||
List<Map<String, Object>> sites = new ArrayList<>();
|
||||
boolean[] exhaustive = {true};
|
||||
code.visitInstructions(insn -> {
|
||||
checkNativeInterrupted();
|
||||
insn.decode();
|
||||
if (insn.getIndexType() != InsnIndexType.METHOD_REF) return;
|
||||
Opcode opcode = insn.getOpcode();
|
||||
if (!opcode.name().startsWith("INVOKE_")) return;
|
||||
IMethodRef ref = insn.getIndexAsMethod();
|
||||
ref.load();
|
||||
visitor.accept(MethodInfo.fromRef(node.root(), ref), insn.getOffset(), insn.getIndex(), opcode);
|
||||
});
|
||||
}
|
||||
|
||||
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
|
||||
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
|
||||
return directInvokes(source.getMethodNode(), target, budget);
|
||||
}
|
||||
|
||||
private static void visitDirectInvokes(MethodNode node, MethodInfo target, InvokeVisitor visitor) {
|
||||
visitInvokes(node, (referenced, offset, index, opcode) -> {
|
||||
if (referenced.getRawFullId().equals(target.getRawFullId()))
|
||||
visitor.accept(referenced, offset, index, opcode);
|
||||
});
|
||||
}
|
||||
|
||||
private static boolean referencesMethod(MethodNode node, MethodInfo target) {
|
||||
boolean[] found = {false};
|
||||
visitDirectInvokes(node, target, (referenced, offset, index, opcode) -> found[0] = true);
|
||||
return found[0];
|
||||
}
|
||||
|
||||
private static DirectInvokes directInvokes(MethodNode node, MethodInfo target, InvokeBudget budget) {
|
||||
if (node.getCodeReader() == null) return null;
|
||||
List<Map<String, Object>> sites = new ArrayList<>();
|
||||
boolean[] exhaustive = {true};
|
||||
visitDirectInvokes(node, target, (referenced, offset, index, opcode) -> {
|
||||
if (!MethodInfo.fromRef(node.root(), ref).getRawFullId().equals(target.getRawFullId())) return;
|
||||
Map<String, Object> site = Map.of(
|
||||
"dex_entry", node.getInputFileName(),
|
||||
"source_raw_method_id", node.getMethodInfo().getRawFullId(),
|
||||
"dex_code_unit_offset", offset,
|
||||
"dex_method_index", index,
|
||||
"dex_code_unit_offset", insn.getOffset(),
|
||||
"dex_method_index", insn.getIndex(),
|
||||
"opcode", opcode.name());
|
||||
if (budget.admit(site)) sites.add(site);
|
||||
else exhaustive[0] = false;
|
||||
@@ -740,143 +462,6 @@ class ReaJadxBridge {
|
||||
return new DirectInvokes(sites, exhaustive[0]);
|
||||
}
|
||||
|
||||
private List<ClassSnapshot> sourceClasses() {
|
||||
indexClasses();
|
||||
Set<ClassSnapshot> seen = Collections.newSetFromMap(new IdentityHashMap<>());
|
||||
List<ClassSnapshot> result = new ArrayList<>();
|
||||
for (List<ClassSnapshot> group : classes.values())
|
||||
for (ClassSnapshot source : group) if (seen.add(source)) result.add(source);
|
||||
result.sort(Comparator.comparing(source -> source.node().getRawName()));
|
||||
return result;
|
||||
}
|
||||
|
||||
private static long displayTypeCharacters(ArgType type) {
|
||||
if (type.isArray())
|
||||
return 2L * type.getArrayDimension() + displayTypeCharacters(type.getArrayRootElement());
|
||||
if (type.isPrimitive()) return type.getPrimitiveType().getLongName().length();
|
||||
if (type.isObject() && !type.isGeneric() && !type.isGenericType()) return type.getObject().length();
|
||||
throw new IllegalArgumentException("External prototype contains a non-DEX type");
|
||||
}
|
||||
|
||||
private static Map<String, Object> externalMethodSummary(MethodInfo info, InvokeBudget budget) {
|
||||
String name = info.getName();
|
||||
long signatureCharacters = name.length() + 4L + displayTypeCharacters(info.getReturnType())
|
||||
+ 2L * Math.max(0, info.getArgsCount() - 1);
|
||||
for (ArgType type : info.getArgumentsTypes()) signatureCharacters += displayTypeCharacters(type);
|
||||
long descriptorCharacters = info.getShortId().length() - (long) name.length();
|
||||
// Use parsed type facts and the existing short-id length before copying/joining strings.
|
||||
if (!budget.fitsText(2L * name.length() + descriptorCharacters + signatureCharacters))
|
||||
throw new ReferenceLimit("method_inventory");
|
||||
String descriptor = info.getShortId().substring(name.length());
|
||||
String signature = name + "(" + info.getArgumentsTypes().stream()
|
||||
.map(Object::toString).collect(Collectors.joining(", "))
|
||||
+ "): " + info.getReturnType();
|
||||
Map<String, Object> summary = Map.of("name", name, "raw_name", name, "signature", signature,
|
||||
"dex_descriptor", descriptor, "is_constructor", info.isConstructor(), "def_pos", 0);
|
||||
if (!budget.admit(summary)) throw new ReferenceLimit("method_inventory");
|
||||
return summary;
|
||||
}
|
||||
|
||||
/** Only directly referenced prototypes, not invented definitions or an external classpath. */
|
||||
private List<ExternalMethod> externalMethods(JsonObject args) {
|
||||
String className = string(args, "class_name");
|
||||
String methodName = string(args, "method_name");
|
||||
JadxSession current = session();
|
||||
if (externalInventory != null && externalInventory.session() == current
|
||||
&& externalInventory.className().equals(className) && externalInventory.methodName().equals(methodName))
|
||||
return externalInventory.methods();
|
||||
// Retain only the latest bounded selection, never results from a replaced APK/session.
|
||||
externalInventory = null;
|
||||
SortedMap<String, ExternalMethod> methods = new TreeMap<>();
|
||||
boolean[] observedClass = {false};
|
||||
InvokeBudget budget = new InvokeBudget(MAX_EXTERNAL_XREF_BYTES
|
||||
- 4096L - 12L * (className.length() + (long) methodName.length()));
|
||||
for (ClassSnapshot source : sourceClasses()) {
|
||||
for (MethodSnapshot selected : source.methods()) {
|
||||
visitInvokes(method(source, selected).getMethodNode(), (info, offset, index, opcode) -> {
|
||||
if (!info.getDeclClass().getRawName().equals(className)) return;
|
||||
observedClass[0] = true;
|
||||
if (!info.getName().equals(methodName) || methods.containsKey(info.getShortId())) return;
|
||||
methods.put(info.getShortId(), new ExternalMethod(info, externalMethodSummary(info, budget)));
|
||||
});
|
||||
}
|
||||
}
|
||||
if (!observedClass[0]) throw new InputFailure("class_name",
|
||||
"No APK definition or direct DEX invoke references the exact class: " + className);
|
||||
if (methods.isEmpty()) throw new InputFailure("method_name",
|
||||
"No direct DEX invoke references " + className + "." + methodName);
|
||||
List<ExternalMethod> result = List.copyOf(methods.values());
|
||||
externalInventory = new ExternalInventory(current, className, methodName, result);
|
||||
return result;
|
||||
}
|
||||
|
||||
private Map<String, Object> externalSummary(JsonObject args) {
|
||||
List<ExternalMethod> methods = externalMethods(args);
|
||||
return Map.of("class_name", string(args, "class_name"), "reference_target", "external",
|
||||
"observed_method_count", methods.size(),
|
||||
"prototypes", methods.stream().map(ExternalMethod::summary).toList());
|
||||
}
|
||||
|
||||
private record ExternalCaller(MethodNode node, Map<String, Object> row) {}
|
||||
|
||||
/** Reserve the complete using-node inventory before optional instruction positions. */
|
||||
private List<ExternalCaller> externalCallers(MethodInfo target, InvokeBudget budget) {
|
||||
List<ExternalCaller> callers = new ArrayList<>();
|
||||
for (ClassSnapshot source : sourceClasses()) {
|
||||
String className = String.valueOf(source.summary().get("full_name"));
|
||||
Map<String, Integer> overloads = new HashMap<>();
|
||||
for (MethodSnapshot candidate : source.methods()) {
|
||||
int overload = overloads.merge(candidate.name(), 1, Integer::sum) - 1;
|
||||
MethodNode node = method(source, candidate).getMethodNode();
|
||||
if (!referencesMethod(node, target)) continue;
|
||||
if (classes.get(className).size() != 1)
|
||||
throw new IllegalArgumentException("External caller has an ambiguous APK class identity: " + className);
|
||||
Map<String, Object> row = new LinkedHashMap<>(Map.of(
|
||||
"kind", "method", "name", candidate.name(), "full_name", className + "." + candidate.name(),
|
||||
"containing_class", className, "top_class", source.node().getTopParentClass().getFullName(),
|
||||
"def_pos", 0, "source_selector", Map.of("class_name", className,
|
||||
"method_name", candidate.name(), "overload_index", overload),
|
||||
"direct_invokes", List.of(), "direct_invokes_exhaustive", false));
|
||||
if (!budget.admit(row)) throw new ReferenceLimit("caller_inventory");
|
||||
callers.add(new ExternalCaller(node, row));
|
||||
}
|
||||
}
|
||||
return List.copyOf(callers);
|
||||
}
|
||||
|
||||
private Map<String, Object> externalReferences(JsonObject args) {
|
||||
List<ExternalMethod> methods = externalMethods(args);
|
||||
if (methods.size() != 1 && !args.has("overload_index"))
|
||||
throw new InputFailure("overload_index", "Select one of " + methods.size() + " observed external prototypes");
|
||||
int index = integer(args, "overload_index", 0);
|
||||
if (index >= methods.size()) throw new InputFailure("overload_index", "External prototype index out of range: " + index);
|
||||
ExternalMethod method = methods.get(index);
|
||||
MethodInfo target = method.info();
|
||||
Map<String, Object> selected = method.summary();
|
||||
InvokeBudget budget = new InvokeBudget(MAX_EXTERNAL_XREF_BYTES
|
||||
- 4096L - escapedJsonBytes(selected));
|
||||
List<ExternalCaller> callers = externalCallers(target, budget);
|
||||
List<Map<String, Object>> items = new ArrayList<>();
|
||||
for (ExternalCaller caller : callers) {
|
||||
DirectInvokes invokes = directInvokes(caller.node(), target, budget);
|
||||
caller.row().put("direct_invokes", invokes.sites());
|
||||
caller.row().put("direct_invokes_exhaustive", invokes.exhaustive());
|
||||
items.add(caller.row());
|
||||
}
|
||||
Map<String, Object> result = new LinkedHashMap<>();
|
||||
result.put("target", string(args, "class_name") + "." + target.getName());
|
||||
result.put("reference_target", "external");
|
||||
result.put("reference_scope", "direct_dex_invokes");
|
||||
result.put("dex_descriptor", selected.get("dex_descriptor"));
|
||||
result.put("reported_signature", selected.get("signature"));
|
||||
result.put("overload_index", index);
|
||||
result.put("total", items.size());
|
||||
result.put("count", items.size());
|
||||
result.put("resolve_line", false);
|
||||
result.put("items", items);
|
||||
return result;
|
||||
}
|
||||
|
||||
private CallToolResult handle(String name, kotlinx.serialization.json.JsonObject input) {
|
||||
JsonObject args = JsonParser.parseString(input.toString()).getAsJsonObject();
|
||||
if (name.equals("rea_jvm_status")) return json(Map.of(
|
||||
@@ -885,8 +470,6 @@ class ReaJadxBridge {
|
||||
"available_processors", Runtime.getRuntime().availableProcessors(),
|
||||
"java_version", System.getProperty("java.version"),
|
||||
"metadata_scope", "parsed_members"));
|
||||
if (name.equals("rea_native_declarations"))
|
||||
return json(nativeDeclarations(Path.of(string(args, "path"))));
|
||||
JadxSession current = session();
|
||||
if (name.equals("get_app_info")) return json(manifest(current).summary());
|
||||
if (name.equals("get_android_manifest"))
|
||||
@@ -913,12 +496,6 @@ class ReaJadxBridge {
|
||||
"scope", "top_level", "offset", offset, "limit", limit,
|
||||
"items", items));
|
||||
}
|
||||
indexClasses();
|
||||
if (flag(args, "reference_target") && !classes.containsKey(string(args, "class_name"))) {
|
||||
if (name.equals("get_class_summary")) return json(externalSummary(args));
|
||||
if (name.equals("get_xrefs_to_method")) return json(externalReferences(args));
|
||||
throw new InputFailure("class_name", "External prototypes are only available to method reference queries");
|
||||
}
|
||||
ClassSnapshot cls = selected(args);
|
||||
if (name.equals("get_class_summary"))
|
||||
return json(flag(args, "dex_static_values") ? cls.summaryWithValues() : cls.summary());
|
||||
@@ -992,9 +569,6 @@ class ReaJadxBridge {
|
||||
try { return bridge.handle(name, request.getArguments()); }
|
||||
catch (InputFailure error) { return text(JSON.toJson(Map.of(
|
||||
"kind", "invalid-input", "field", error.field, "message", error.getMessage())), true); }
|
||||
catch (ReferenceLimit error) { return text(JSON.toJson(Map.of(
|
||||
"kind", "reference-output-limit", "stage", error.stage,
|
||||
"limit_bytes", MAX_EXTERNAL_XREF_BYTES)), true); }
|
||||
catch (IllegalArgumentException error) { return text(error.getMessage(), true); }
|
||||
});
|
||||
}
|
||||
@@ -1007,7 +581,7 @@ class ReaJadxBridge {
|
||||
SessionHolder holder = new SessionHolder(defaults);
|
||||
ServerCapabilities capabilities = new ServerCapabilities(
|
||||
new ServerCapabilities.Tools(null), null, null, null, null, null, null, Map.of());
|
||||
Server server = new Server(new Implementation("rea-jadx-bridge", "10", null, null, null),
|
||||
Server server = new Server(new Implementation("rea-jadx-bridge", "7", null, null, null),
|
||||
new ServerOptions(capabilities), "", ignored -> Unit.INSTANCE);
|
||||
SessionToolsKt.registerSessionTools(server, holder);
|
||||
ClassToolsKt.registerClassTools(server, holder);
|
||||
@@ -1027,11 +601,6 @@ class ReaJadxBridge {
|
||||
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of()), List.of(), null),
|
||||
"Observed worker JVM and metadata scope", null, null, null, null, null, null);
|
||||
server.addTool(diagnostics, (connection, request, continuation) -> bridge.handle("rea_jvm_status", request.getArguments()));
|
||||
registerBridgeTool(server, bridge, "rea_native_declarations",
|
||||
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of("path",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of("type",
|
||||
kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))))), List.of("path"), null),
|
||||
"Original ACC_NATIVE declarations from the admitted APK's standard root DEX members, without loading or decompiling the APK.");
|
||||
for (String name : List.of("get_app_info", "get_android_manifest")) {
|
||||
String description = name.equals("get_app_info")
|
||||
? "Namespace-aware summary of the exact root AndroidManifest.xml resource."
|
||||
@@ -1057,12 +626,6 @@ class ReaJadxBridge {
|
||||
if (name.equals("get_class_summary")) properties.put("dex_static_values",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("boolean"))));
|
||||
if (name.equals("get_class_summary") || name.equals("get_xrefs_to_method"))
|
||||
properties.put("reference_target", new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("boolean"))));
|
||||
if (name.equals("get_class_summary")) properties.put("method_name",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))));
|
||||
String description = switch (name) {
|
||||
case "list_classes" -> "Pages of case-sensitive matching top-level class names; offset and next_offset are inventory positions, limit bounds matching rows. Omitted or empty query inventories all names.";
|
||||
case "get_class_summary" -> "Parsed member metadata for one exact class name; no source generation. Set dex_static_values to include encoded static final field values.";
|
||||
|
||||
@@ -40,7 +40,6 @@ import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.TreeMap;
|
||||
import java.util.TreeSet;
|
||||
import java.util.regex.Pattern;
|
||||
import java.util.regex.PatternSyntaxException;
|
||||
|
||||
@@ -549,11 +548,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
String softFloat = currentProgram.getOptions(ghidra.program.model.listing.Program.PROGRAM_INFO)
|
||||
.getString("REA MIPS soft-float storage", null);
|
||||
if (softFloat != null) result.add("mips_soft_float_storage", JsonParser.parseString(softFloat));
|
||||
// Written by ReaGhidraRiscvCallingConvention after analysis; REA requires it
|
||||
// exactly when the committed profile declares a RISC-V calling-convention policy.
|
||||
String riscvCallingConvention = currentProgram.getOptions(ghidra.program.model.listing.Program.PROGRAM_INFO)
|
||||
.getString("REA RISC-V calling convention", null);
|
||||
if (riscvCallingConvention != null) result.add("riscv_calling_convention", JsonParser.parseString(riscvCallingConvention));
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -693,7 +687,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
row.addProperty("status", relocation.getStatus().toString());
|
||||
row.addProperty("type", relocation.getType());
|
||||
row.add("symbol", GSON.toJsonTree(relocation.getSymbolName()));
|
||||
row.add("values", GSON.toJsonTree(relocation.getValues() == null ? new long[0] : relocation.getValues()));
|
||||
byte[] original = relocation.getBytes();
|
||||
row.add("original_bytes_hex", original == null ? JsonNull.INSTANCE : GSON.toJsonTree(bytesHex(original)));
|
||||
@@ -929,33 +922,17 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireKeys(params, Set.of("document", "address"));
|
||||
requireDocument(params);
|
||||
Address address = requireAddress(params, "address");
|
||||
String at = canonicalAddress(address);
|
||||
// A well-formed address can still lack one exact source offset. Each
|
||||
// outcome has its own code, so REA reports the reason rather than
|
||||
// schema repair. Initialization, not the loaded flag, decides whether
|
||||
// source bytes can exist: file-backed overlays are not loaded.
|
||||
MemoryBlock block = currentProgram.getMemory().getBlock(address);
|
||||
if (block == null)
|
||||
throw new RequestFailure("address_unmapped",
|
||||
"Address " + at + " is outside every Ghidra memory block");
|
||||
if (!block.isInitialized())
|
||||
throw new RequestFailure("address_uninitialized",
|
||||
"Address " + at + " is in uninitialized Ghidra memory block " + block.getName());
|
||||
if (block == null || !block.isInitialized())
|
||||
throw new RequestFailure("invalid_request", "Address has no initialized file-backed memory: " + canonicalAddress(address));
|
||||
TargetFileOffset mapping = targetFileOffset(block, address);
|
||||
if (mapping.mappings() == 0)
|
||||
throw new RequestFailure("file_mapping_absent",
|
||||
"Address " + at + " in initialized Ghidra memory block " + block.getName() +
|
||||
" has no authoritative mapping to the admitted target file");
|
||||
if (mapping.mappings() > 1)
|
||||
throw new RequestFailure("file_mapping_ambiguous",
|
||||
"Address " + at + " has " + mapping.mappings() +
|
||||
" authoritative mappings to the admitted target file");
|
||||
if (mapping.mappings() != 1)
|
||||
throw new RequestFailure("invalid_request", "Address has " + mapping.mappings() + " authoritative target-file mappings: " + canonicalAddress(address));
|
||||
Long fileOffset = mapping.offset();
|
||||
if (fileOffset == null)
|
||||
throw new RequestFailure("file_offset_unrepresentable",
|
||||
"Address " + at + " maps to an admitted target-file offset beyond the exact JSON integer range");
|
||||
throw new RequestFailure("invalid_request", "Address lacks a safe file offset: " + canonicalAddress(address));
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("address", at);
|
||||
result.addProperty("address", canonicalAddress(address));
|
||||
result.addProperty("file_offset", fileOffset);
|
||||
return result;
|
||||
}
|
||||
@@ -1336,7 +1313,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
flow.addProperty("computed", false);
|
||||
if (instruction == null) {
|
||||
result.addProperty("status", !currentProgram.getMemory().contains(address) ? "outside-memory" : currentProgram.getListing().getInstructionContaining(address) != null
|
||||
? "not-instruction-boundary" : currentProgram.getListing().getDefinedDataContaining(address) != null ? "data" : "not-in-listing");
|
||||
? "not-instruction-boundary" : currentProgram.getListing().getDefinedDataContaining(address) != null ? "data" : "undecodable");
|
||||
for (String key : List.of("bytes", "length", "mnemonic", "raw_disassembly")) result.add(key, JsonNull.INSTANCE);
|
||||
} else {
|
||||
result.addProperty("status", "decoded");
|
||||
@@ -1408,7 +1385,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.add("incoming_references", incomingReferences);
|
||||
}
|
||||
List<String> limitations = new ArrayList<>(List.of("Operand components are Ghidra Listing decoder tokens; effective memory base/index/displacement roles and per-instruction context mode are unavailable. The mode field is the program language variant."));
|
||||
if (result.get("status").getAsString().equals("not-in-listing")) limitations.add("No listed instruction or defined data covers this in-memory address. REA did not attempt a decode; the memory may be uninitialized or contain valid instructions.");
|
||||
if (flow.get("kind").getAsString().equals("terminal")) limitations.add("No RETURN operation was observed in the provider's instruction p-code; terminal classification remains generic.");
|
||||
result.add("limitations", GSON.toJsonTree(limitations));
|
||||
return result;
|
||||
@@ -1954,62 +1930,41 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return symbol == null ? JsonNull.INSTANCE : GSON.toJsonTree(symbol.getName(true));
|
||||
}
|
||||
|
||||
// Analyst naming: a function entry renames the function; another mapped
|
||||
// address renames its non-dynamic primary label or receives a new global
|
||||
// label. Resolve that destination without editing the program.
|
||||
private AddressNameTarget addressNameTarget(Address address, String name)
|
||||
throws ghidra.util.exception.InvalidInputException {
|
||||
// Analyst naming: a function entry renames the function, other addresses get a primary label.
|
||||
private boolean applyAddressName(Address address, String name) throws Exception {
|
||||
if (!currentProgram.getMemory().contains(address))
|
||||
throw new RequestFailure("invalid_request", "Address is not mapped: " + canonicalAddress(address));
|
||||
validateAnnotationText(name, "name");
|
||||
Symbol edited = editedNameSymbol(address);
|
||||
return edited == null
|
||||
? new AddressNameTarget(address, currentProgram.getGlobalNamespace(), name)
|
||||
: new AddressNameTarget(address, edited.getParentNamespace(), symbolLeafName(edited, name));
|
||||
}
|
||||
|
||||
private Symbol editedNameSymbol(Address address) {
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
if (function != null) return function.getSymbol();
|
||||
Symbol primary = currentProgram.getSymbolTable().getPrimarySymbol(address);
|
||||
return primary == null || primary.isDynamic() ? null : primary;
|
||||
}
|
||||
|
||||
// Every address whose label or function symbol holds the destination.
|
||||
private List<Address> addressNameHolders(AddressNameTarget target) {
|
||||
Set<Address> holders = new TreeSet<>();
|
||||
for (Symbol symbol : currentProgram.getSymbolTable().getLabelOrFunctionSymbols(target.leaf(), target.namespace()))
|
||||
holders.add(symbol.getAddress());
|
||||
return List.copyOf(holders);
|
||||
}
|
||||
|
||||
// Apply one planned destination in its own transaction. Null means the
|
||||
// edit committed with matching primary-symbol readback.
|
||||
private String applyAddressName(AddressNameTarget target) {
|
||||
Address address = target.address();
|
||||
int transaction = currentProgram.startTransaction("REA address name");
|
||||
boolean commit = false;
|
||||
try {
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
Symbol edited = editedNameSymbol(address);
|
||||
SymbolTable table = currentProgram.getSymbolTable();
|
||||
Symbol primary = function == null ? table.getPrimarySymbol(address) : function.getSymbol();
|
||||
Namespace namespace = primary == null
|
||||
? currentProgram.getGlobalNamespace()
|
||||
: primary.getParentNamespace();
|
||||
String expected = primary == null ? name : symbolLeafName(primary, name);
|
||||
for (Symbol symbol : table.getLabelOrFunctionSymbols(expected, namespace)) {
|
||||
if (!address.equals(symbol.getAddress())) return false;
|
||||
}
|
||||
if (function != null) {
|
||||
function.setName(target.leaf(), SourceType.USER_DEFINED);
|
||||
} else if (edited != null) {
|
||||
edited.setName(target.leaf(), SourceType.USER_DEFINED);
|
||||
function.setName(expected, SourceType.USER_DEFINED);
|
||||
} else {
|
||||
Symbol created = currentProgram.getSymbolTable().createLabel(address, target.leaf(), SourceType.USER_DEFINED);
|
||||
created.setPrimary();
|
||||
if (primary != null && !primary.isDynamic()) {
|
||||
primary.setName(expected, SourceType.USER_DEFINED);
|
||||
} else {
|
||||
Symbol created = table.createLabel(address, expected, SourceType.USER_DEFINED);
|
||||
created.setPrimary();
|
||||
}
|
||||
}
|
||||
invalidateAnalysisCaches();
|
||||
Symbol readback = currentProgram.getSymbolTable().getPrimarySymbol(address);
|
||||
commit = readback != null && target.leaf().equals(readback.getName());
|
||||
return commit ? null : "readback_mismatch";
|
||||
commit = readback != null && expected.equals(readback.getName());
|
||||
return commit;
|
||||
}
|
||||
catch (ghidra.util.exception.InvalidInputException exception) {
|
||||
return "invalid_name";
|
||||
}
|
||||
catch (ghidra.util.exception.DuplicateNameException exception) {
|
||||
return "duplicate_name";
|
||||
catch (ghidra.util.exception.InvalidInputException | ghidra.util.exception.DuplicateNameException exception) {
|
||||
return false;
|
||||
}
|
||||
finally {
|
||||
currentProgram.endTransaction(transaction, commit);
|
||||
@@ -2019,17 +1974,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private JsonElement setAddressName(JsonObject params) throws Exception {
|
||||
requireKeys(params, Set.of("document", "address", "name"));
|
||||
requireDocument(params);
|
||||
String key = requireString(params, "address");
|
||||
Address address = requireAddress(params, "address");
|
||||
String name = requireString(params, "name");
|
||||
// A single edit keeps request errors for unmapped addresses and
|
||||
// unrepresentable text; other outcomes share the batch representation.
|
||||
if (!currentProgram.getMemory().contains(address))
|
||||
throw new RequestFailure("invalid_request", "Address is not mapped: " + canonicalAddress(address));
|
||||
validateAnnotationText(name, "name");
|
||||
Map<String, String> requested = new LinkedHashMap<>();
|
||||
requested.put(key, name);
|
||||
return applyAddressNames(requested);
|
||||
return GSON.toJsonTree(applyAddressName(requireAddress(params, "address"), requireString(params, "name")));
|
||||
}
|
||||
|
||||
private JsonElement setAddressesNames(JsonObject params) throws Exception {
|
||||
@@ -2037,129 +1982,34 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireDocument(params);
|
||||
if (!params.has("names") || !params.get("names").isJsonObject())
|
||||
throw new RequestFailure("invalid_request", "names must be an object of address -> name");
|
||||
Map<String, String> requested = new LinkedHashMap<>();
|
||||
for (Map.Entry<String, JsonElement> entry : params.getAsJsonObject("names").entrySet())
|
||||
requested.put(entry.getKey(), entry.getValue().getAsString());
|
||||
return applyAddressNames(requested);
|
||||
}
|
||||
|
||||
// Decide every request from the names held before any edit, so object
|
||||
// member order never selects which address receives a name. Reply with
|
||||
// per-key results and reasons; a contested destination is listed once in
|
||||
// conflicts and failures refer to it by index.
|
||||
private JsonObject applyAddressNames(Map<String, String> requested) {
|
||||
Map<String, Address> addressesByKey = new LinkedHashMap<>();
|
||||
Map<Address, List<AddressNameRequest>> requestsByAddress = new LinkedHashMap<>();
|
||||
for (Map.Entry<String, String> entry : requested.entrySet()) {
|
||||
Address address = null;
|
||||
JsonObject result = new JsonObject();
|
||||
for (Map.Entry<String, JsonElement> entry : params.getAsJsonObject("names").entrySet()) {
|
||||
try {
|
||||
address = parseReaAddress(entry.getKey());
|
||||
Address address = parseReaAddress(entry.getKey());
|
||||
requestsByAddress.computeIfAbsent(address, ignored -> new ArrayList<>())
|
||||
.add(new AddressNameRequest(entry.getKey(), address, entry.getValue()));
|
||||
.add(new AddressNameRequest(entry.getKey(), address, entry.getValue().getAsString()));
|
||||
}
|
||||
catch (RequestFailure failure) {
|
||||
// The key keeps its own invalid_address outcome below.
|
||||
}
|
||||
addressesByKey.put(entry.getKey(), address);
|
||||
}
|
||||
Map<Address, AddressNameFailure> failures = new HashMap<>();
|
||||
Map<AddressNameDestination, List<AddressNameTarget>> destinations = new LinkedHashMap<>();
|
||||
for (Map.Entry<Address, List<AddressNameRequest>> selected : requestsByAddress.entrySet()) {
|
||||
Address address = selected.getKey();
|
||||
String name = selected.getValue().get(0).name;
|
||||
if (selected.getValue().stream().anyMatch(request -> !request.name.equals(name))) {
|
||||
failures.put(address, new AddressNameFailure("conflicting_alias_names", null));
|
||||
continue;
|
||||
}
|
||||
if (!currentProgram.getMemory().contains(address)) {
|
||||
failures.put(address, new AddressNameFailure("unmapped_address", null));
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
AddressNameTarget target = addressNameTarget(address, name);
|
||||
destinations.computeIfAbsent(target.destination(), ignored -> new ArrayList<>()).add(target);
|
||||
}
|
||||
catch (RequestFailure | ghidra.util.exception.InvalidInputException exception) {
|
||||
failures.put(address, new AddressNameFailure("invalid_name", null));
|
||||
result.addProperty(entry.getKey(), false);
|
||||
}
|
||||
}
|
||||
// Reject a destination that several selected addresses request, or
|
||||
// that another address holds, instead of letting order pick a winner.
|
||||
List<AddressNameConflict> conflicts = new ArrayList<>();
|
||||
List<AddressNameTarget> independent = new ArrayList<>();
|
||||
for (List<AddressNameTarget> requesters : destinations.values()) {
|
||||
AddressNameTarget first = requesters.get(0);
|
||||
List<Address> holders = addressNameHolders(first);
|
||||
String reason;
|
||||
if (requesters.size() > 1) {
|
||||
reason = "duplicate_requested_name";
|
||||
} else if (holders.stream().allMatch(first.address()::equals)) {
|
||||
independent.add(first);
|
||||
continue;
|
||||
} else if (holders.stream().anyMatch(holder ->
|
||||
!holder.equals(first.address()) && !requestsByAddress.containsKey(holder))) {
|
||||
reason = "name_held_by_unselected_address";
|
||||
} else {
|
||||
reason = "name_held_by_selected_address";
|
||||
for (List<AddressNameRequest> requests : requestsByAddress.values()) {
|
||||
String name = requests.get(0).name;
|
||||
boolean conflictingAliasNames = requests.stream()
|
||||
.anyMatch(request -> !request.name.equals(name));
|
||||
boolean success = false;
|
||||
if (!conflictingAliasNames) {
|
||||
try {
|
||||
success = applyAddressName(requests.get(0).address, name);
|
||||
}
|
||||
catch (RequestFailure failure) {
|
||||
// Keep per-address failures in the existing batch result shape.
|
||||
}
|
||||
}
|
||||
AddressNameConflict conflict = new AddressNameConflict(
|
||||
first.namespace(),
|
||||
first.leaf(),
|
||||
requesters.stream().map(AddressNameTarget::address).sorted().toList(),
|
||||
holders
|
||||
);
|
||||
conflicts.add(conflict);
|
||||
for (AddressNameTarget target : requesters)
|
||||
failures.put(target.address(), new AddressNameFailure(reason, conflict));
|
||||
for (AddressNameRequest request : requests)
|
||||
result.addProperty(request.inputAddress, success);
|
||||
}
|
||||
// Independent edits have distinct destinations that no other address
|
||||
// holds, so their results do not depend on application order.
|
||||
for (AddressNameTarget target : independent) {
|
||||
String reason = applyAddressName(target);
|
||||
if (reason != null) failures.put(target.address(), new AddressNameFailure(reason, null));
|
||||
}
|
||||
// Requesters belong to exactly one destination, so ordering conflicts
|
||||
// by their first requester gives indexes independent of member order.
|
||||
conflicts.sort(Comparator.comparing((AddressNameConflict conflict) -> conflict.requesters().get(0)));
|
||||
Map<AddressNameConflict, Integer> indexes = new IdentityHashMap<>();
|
||||
JsonArray contested = new JsonArray();
|
||||
for (AddressNameConflict conflict : conflicts) {
|
||||
indexes.put(conflict, contested.size());
|
||||
JsonObject value = new JsonObject();
|
||||
if (conflict.namespace().isGlobal()) value.add("namespace", JsonNull.INSTANCE);
|
||||
else value.addProperty("namespace", conflict.namespace().getName(true));
|
||||
value.addProperty("name", conflict.leaf());
|
||||
value.add("requested_by", canonicalAddresses(conflict.requesters()));
|
||||
value.add("held_by", canonicalAddresses(conflict.holders()));
|
||||
contested.add(value);
|
||||
}
|
||||
JsonObject results = new JsonObject();
|
||||
JsonObject failed = new JsonObject();
|
||||
for (Map.Entry<String, Address> entry : addressesByKey.entrySet()) {
|
||||
Address address = entry.getValue();
|
||||
AddressNameFailure failure = address == null
|
||||
? new AddressNameFailure("invalid_address", null)
|
||||
: failures.get(address);
|
||||
results.addProperty(entry.getKey(), failure == null);
|
||||
if (failure == null) continue;
|
||||
JsonObject value = new JsonObject();
|
||||
if (address == null) value.add("address", JsonNull.INSTANCE);
|
||||
else value.addProperty("address", canonicalAddress(address));
|
||||
value.addProperty("reason", failure.reason());
|
||||
if (failure.conflict() == null) value.add("conflict", JsonNull.INSTANCE);
|
||||
else value.addProperty("conflict", indexes.get(failure.conflict()));
|
||||
failed.add(entry.getKey(), value);
|
||||
}
|
||||
JsonObject result = new JsonObject();
|
||||
result.add("results", results);
|
||||
result.add("failures", failed);
|
||||
result.add("conflicts", contested);
|
||||
return result;
|
||||
}
|
||||
|
||||
private static JsonArray canonicalAddresses(List<Address> addresses) {
|
||||
JsonArray result = new JsonArray();
|
||||
for (Address address : addresses) result.add(canonicalAddress(address));
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -2249,7 +2099,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireKeys(params, Set.of("document", "address"));
|
||||
requireDocument(params);
|
||||
Address query = requireAddress(params, "address");
|
||||
Function function = functionAtOrContaining(query);
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(query);
|
||||
if (function == null && query.isMemoryAddress()) {
|
||||
function = currentProgram.getFunctionManager().getFunctionContaining(query);
|
||||
}
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("query_address", canonicalAddress(query));
|
||||
if (function != null) {
|
||||
@@ -2265,9 +2118,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
|
||||
private JsonArray search(JsonObject params, boolean procedureSearch) throws Exception {
|
||||
Set<String> keys = new HashSet<>(Set.of("pattern", "mode", "case_sensitive", "document"));
|
||||
if (params.has("literal_ascii_candidate")) keys.add("literal_ascii_candidate");
|
||||
requireKeys(params, keys);
|
||||
requireKeys(
|
||||
params,
|
||||
Set.of("pattern", "mode", "case_sensitive", "document")
|
||||
);
|
||||
requireDocument(params);
|
||||
String expression = requireString(params, "pattern");
|
||||
String mode = requireString(params, "mode");
|
||||
@@ -2275,39 +2129,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
throw new RequestFailure("invalid_request", "mode must be literal or regex");
|
||||
}
|
||||
boolean caseSensitive = requireBoolean(params, "case_sensitive");
|
||||
ValueMatcher matcher;
|
||||
if (mode.equals("literal") && !caseSensitive) {
|
||||
if (!params.has("literal_ascii_candidate"))
|
||||
throw new RequestFailure("invalid_request", "Insensitive literal search requires an ASCII candidate");
|
||||
matcher = literalCandidateMatcher(optionalString(params, "literal_ascii_candidate"));
|
||||
} else {
|
||||
if (params.has("literal_ascii_candidate"))
|
||||
throw new RequestFailure("invalid_request", "ASCII candidate applies only to insensitive literals");
|
||||
matcher = mode.equals("literal")
|
||||
? literalMatcher(expression)
|
||||
: regexMatcher(expression, caseSensitive);
|
||||
}
|
||||
List<InventoryItem> inventory = procedureSearch
|
||||
? procedures().stream().map(FunctionEntry::item).toList()
|
||||
: strings();
|
||||
ValueMatcher matcher = mode.equals("literal")
|
||||
? literalMatcher(expression, caseSensitive)
|
||||
: regexMatcher(expression, caseSensitive);
|
||||
JsonArray items = new JsonArray();
|
||||
if (procedureSearch) {
|
||||
// Search the names exact lookup accepts, keeping one row per
|
||||
// procedure under its primary name.
|
||||
for (FunctionEntry entry : procedures()) {
|
||||
monitor.checkCancelled();
|
||||
ProcedureNameMatch match = matchProcedureNames(entry, matcher);
|
||||
if (!match.matched()) {
|
||||
continue;
|
||||
}
|
||||
JsonObject item = resultItem(entry.item, null);
|
||||
if (!match.aliases().isEmpty()) {
|
||||
JsonArray aliases = new JsonArray();
|
||||
for (String alias : match.aliases()) aliases.add(alias);
|
||||
item.add("matched_aliases", aliases);
|
||||
}
|
||||
items.add(item);
|
||||
}
|
||||
return items;
|
||||
}
|
||||
for (InventoryItem item : strings()) {
|
||||
for (InventoryItem item : inventory) {
|
||||
monitor.checkCancelled();
|
||||
if (!matcher.matches(item.value)) {
|
||||
continue;
|
||||
@@ -2317,36 +2146,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return items;
|
||||
}
|
||||
|
||||
/**
|
||||
* The one definition of the names that select a procedure, shared by exact
|
||||
* lookup and search: its primary name and every other symbol at its exact
|
||||
* entry, each in leaf or namespace-qualified spelling. A label inside the
|
||||
* body never names the procedure. Matching secondary symbols are reported
|
||||
* by their qualified names, sorted.
|
||||
*/
|
||||
private ProcedureNameMatch matchProcedureNames(FunctionEntry entry, ValueMatcher matcher) throws Exception {
|
||||
Function function = entry.function;
|
||||
boolean primary = spellingMatches(function.getName(), entry.item.value, matcher);
|
||||
Set<String> aliases = null;
|
||||
for (Symbol symbol : currentProgram.getSymbolTable().getSymbols(function.getEntryPoint())) {
|
||||
monitor.checkCancelled();
|
||||
String qualified = symbol.getName(true);
|
||||
if (!spellingMatches(symbol.getName(), qualified, matcher)) continue;
|
||||
if (qualified.equals(entry.item.value)) {
|
||||
primary = true;
|
||||
continue;
|
||||
}
|
||||
if (aliases == null) aliases = new TreeSet<>();
|
||||
aliases.add(qualified);
|
||||
}
|
||||
return new ProcedureNameMatch(primary, aliases == null ? List.of() : List.copyOf(aliases));
|
||||
}
|
||||
|
||||
/** A leaf is the suffix of its qualified spelling, so a literal found in it is found in both. */
|
||||
private static boolean spellingMatches(String leaf, String qualified, ValueMatcher matcher) {
|
||||
return matcher.matches(qualified) || (!leaf.equals(qualified) && matcher.matches(leaf));
|
||||
}
|
||||
|
||||
private List<InventoryItem> names() throws Exception {
|
||||
if (nameInventory != null) {
|
||||
return nameInventory;
|
||||
@@ -2449,13 +2248,26 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
// exact symbol name, even when every character is a hexadecimal digit.
|
||||
boolean explicitAddress = isExplicitReaAddress(value);
|
||||
if (!explicitAddress) {
|
||||
// Imported labels (for example Mach-O _main) can share a function
|
||||
// entry without becoming its primary symbol; search accepts the same.
|
||||
ValueMatcher selector = value::equals;
|
||||
List<Function> matches = new ArrayList<>();
|
||||
for (FunctionEntry entry : procedures()) {
|
||||
monitor.checkCancelled();
|
||||
if (matchProcedureNames(entry, selector).matched()) matches.add(entry.function);
|
||||
Function function = entry.function;
|
||||
Symbol symbol = function.getSymbol();
|
||||
String qualified = symbol == null ? function.getName() : symbol.getName(true);
|
||||
boolean matchesName = function.getName().equals(value) || qualified.equals(value);
|
||||
if (!matchesName) {
|
||||
// Imported labels (for example Mach-O _main) can share a
|
||||
// function entry without becoming its primary symbol. Only
|
||||
// exact entries qualify; an interior label is not a selector.
|
||||
for (Symbol entrySymbol : currentProgram.getSymbolTable().getSymbols(function.getEntryPoint())) {
|
||||
monitor.checkCancelled();
|
||||
if (entrySymbol.getName().equals(value) || entrySymbol.getName(true).equals(value)) {
|
||||
matchesName = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (matchesName) matches.add(function);
|
||||
}
|
||||
if (matches.size() > 1)
|
||||
throw new RequestFailure("ambiguous", "Ghidra procedure name is ambiguous: " + value +
|
||||
@@ -2479,7 +2291,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return function;
|
||||
}
|
||||
|
||||
/** Exact entries include externals; containment applies only to memory addresses. */
|
||||
private Function functionAtOrContaining(Address address) {
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
return function == null && address.isMemoryAddress()
|
||||
@@ -3184,7 +2995,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (!currentProgram.getLanguageID().getIdAsString().startsWith("AARCH64:")) {
|
||||
return null;
|
||||
}
|
||||
Function function = functionAtOrContaining(jumpTable.getSwitchAddress());
|
||||
Function function = containingFunction(jumpTable.getSwitchAddress());
|
||||
if (function == null) return null;
|
||||
|
||||
List<Instruction> instructions = new ArrayList<>();
|
||||
@@ -3684,7 +3495,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
|
||||
private boolean unrepresentedCaller(Reference reference, Set<Function> callers) {
|
||||
if (!reference.getReferenceType().isCall()) return false;
|
||||
Function caller = functionAtOrContaining(reference.getFromAddress());
|
||||
Function caller = containingFunction(reference.getFromAddress());
|
||||
return caller == null || !callers.contains(caller);
|
||||
}
|
||||
|
||||
@@ -3831,7 +3642,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
void observe(Reference reference) {
|
||||
if (!reference.getReferenceType().isCall()) return;
|
||||
// Reference identity follows the same containing-function owner as referenceEdge.
|
||||
Function target = functionAtOrContaining(reference.getToAddress());
|
||||
Function target = containingFunction(reference.getToAddress());
|
||||
if (returnedCallees != null && (target == null || !returnedCallees.contains(target)))
|
||||
unrepresented++;
|
||||
if (target == null) unresolved(reference.getFromAddress(),
|
||||
@@ -4001,8 +3812,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("source_address", canonicalAddress(reference.getFromAddress()));
|
||||
result.addProperty("target_address", canonicalAddress(reference.getToAddress()));
|
||||
Function source = functionAtOrContaining(reference.getFromAddress());
|
||||
Function target = functionAtOrContaining(reference.getToAddress());
|
||||
Function source = containingFunction(reference.getFromAddress());
|
||||
Function target = containingFunction(reference.getToAddress());
|
||||
result.add(
|
||||
"source_procedure",
|
||||
source == null ? JsonNull.INSTANCE : procedureIdentity(source)
|
||||
@@ -4037,6 +3848,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return result;
|
||||
}
|
||||
|
||||
private Function containingFunction(Address address) {
|
||||
Function result = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
if (result == null && address.isMemoryAddress()) {
|
||||
result = currentProgram.getFunctionManager().getFunctionContaining(address);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private static String procedureName(Function function) {
|
||||
Symbol symbol = function.getSymbol();
|
||||
return symbol == null ? function.getName() : symbol.getName(true);
|
||||
@@ -4888,18 +4707,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return source.name().toLowerCase(Locale.ROOT);
|
||||
}
|
||||
|
||||
private static ValueMatcher literalMatcher(String pattern) {
|
||||
return value -> value.contains(pattern);
|
||||
}
|
||||
|
||||
// REA's host owns Unicode simple folding. Keep all non-ASCII source values
|
||||
// for its final predicate, and filter ASCII-only values without Java regexes.
|
||||
private static ValueMatcher literalCandidateMatcher(String asciiPattern) {
|
||||
if (asciiPattern != null && asciiPattern.chars().anyMatch(character -> character > 127))
|
||||
throw new RequestFailure("invalid_request", "ASCII candidate contains a non-ASCII character");
|
||||
private static ValueMatcher literalMatcher(String pattern, boolean caseSensitive) {
|
||||
String needle = caseSensitive ? pattern : pattern.toLowerCase(Locale.ROOT);
|
||||
return value -> {
|
||||
if (value.chars().anyMatch(character -> character > 127)) return true;
|
||||
return asciiPattern != null && value.toLowerCase(Locale.ROOT).contains(asciiPattern);
|
||||
String candidate = caseSensitive ? value : value.toLowerCase(Locale.ROOT);
|
||||
return candidate.contains(needle);
|
||||
};
|
||||
}
|
||||
|
||||
@@ -5163,20 +4975,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
|
||||
private record InventoryItem(Address address, String value, JsonObject facts, JsonObject symbol) {}
|
||||
private record AddressNameRequest(String inputAddress, Address address, String name) {}
|
||||
private record AddressNameDestination(long namespace, String leaf) {}
|
||||
private record AddressNameTarget(Address address, Namespace namespace, String leaf) {
|
||||
AddressNameDestination destination() {
|
||||
return new AddressNameDestination(namespace.getID(), leaf);
|
||||
}
|
||||
}
|
||||
private record AddressNameConflict(Namespace namespace, String leaf, List<Address> requesters, List<Address> holders) {}
|
||||
private record AddressNameFailure(String reason, AddressNameConflict conflict) {}
|
||||
private record FunctionEntry(Function function, InventoryItem item) {}
|
||||
private record ProcedureNameMatch(boolean primary, List<String> aliases) {
|
||||
boolean matched() {
|
||||
return primary || !aliases.isEmpty();
|
||||
}
|
||||
}
|
||||
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage, boolean decompilerFailed) {}
|
||||
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
|
||||
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
|
||||
|
||||
@@ -1,283 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.database.SpecExtension;
|
||||
import ghidra.program.model.data.DataType;
|
||||
import ghidra.program.model.data.DoubleDataType;
|
||||
import ghidra.program.model.data.Float16DataType;
|
||||
import ghidra.program.model.data.FloatDataType;
|
||||
import ghidra.program.model.data.FunctionDefinition;
|
||||
import ghidra.program.model.data.IntegerDataType;
|
||||
import ghidra.program.model.data.StructureDataType;
|
||||
import ghidra.program.model.lang.CompilerSpec;
|
||||
import ghidra.program.model.lang.Language;
|
||||
import ghidra.program.model.lang.PrototypeModel;
|
||||
import ghidra.program.model.lang.Register;
|
||||
import ghidra.program.model.listing.Function;
|
||||
import ghidra.program.model.listing.Parameter;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.listing.VariableStorage;
|
||||
import ghidra.program.model.pcode.Varnode;
|
||||
import ghidra.program.model.pcode.XmlEncode;
|
||||
import java.io.StringReader;
|
||||
import java.io.StringWriter;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.regex.Pattern;
|
||||
import javax.xml.parsers.DocumentBuilderFactory;
|
||||
import javax.xml.transform.OutputKeys;
|
||||
import javax.xml.transform.TransformerFactory;
|
||||
import javax.xml.transform.dom.DOMSource;
|
||||
import javax.xml.transform.stream.StreamResult;
|
||||
import org.w3c.dom.Document;
|
||||
import org.w3c.dom.Element;
|
||||
import org.w3c.dom.Node;
|
||||
import org.w3c.dom.NodeList;
|
||||
import org.xml.sax.InputSource;
|
||||
|
||||
/**
|
||||
* Apply the committed RISC-V calling-convention policy after analysis. Ghidra's generic RV32
|
||||
* compiler specification models ilp32d. Each policy derives a program-specific prototype from the
|
||||
* program's own default prototype, as its {@link Policy} row and docs/ghidra-riscv.md state. The
|
||||
* script installs the prototype as a specification extension and assigns it, with dynamic
|
||||
* storage, to every non-thunk function, external function and function definition.
|
||||
*/
|
||||
public final class ReaGhidraRiscvCallingConvention extends GhidraScript {
|
||||
static final String REPORT_KEY = "REA RISC-V calling convention";
|
||||
/** RISC-V floating-point registers by ABI name: ft0-ft11, fs0-fs11 and fa0-fa7. */
|
||||
private static final Pattern FLOAT_REGISTER = Pattern.compile("f[ast]\\d+");
|
||||
|
||||
/**
|
||||
* What a policy changes in the default ilp32d prototype. Every policy also passes values of 9
|
||||
* bytes or more, such as {@code long double}, by reference.
|
||||
*
|
||||
* @param floatEntryBytes the largest value a floating-point register may pass or return; 0
|
||||
* removes those entries and every rule that assigns floating-point storage
|
||||
* @param preservesFloatRegisters whether fs0-fs11 stay preserved across calls
|
||||
*/
|
||||
private record Policy(
|
||||
String id, String prototype, int floatEntryBytes, boolean preservesFloatRegisters) {}
|
||||
|
||||
private static final List<Policy> POLICIES =
|
||||
List.of(new Policy("ilp32-prototype-v1", "__rea_ilp32", 0, false));
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] arguments = getScriptArgs();
|
||||
Policy policy =
|
||||
POLICIES.stream()
|
||||
.filter(candidate -> arguments.length == 1 && candidate.id().equals(arguments[0]))
|
||||
.findFirst()
|
||||
.orElseThrow(
|
||||
() ->
|
||||
new IllegalArgumentException(
|
||||
"REA RISC-V calling convention requires one of the policies "
|
||||
+ POLICIES.stream().map(Policy::id).toList()));
|
||||
Language language = currentProgram.getLanguage();
|
||||
if (!language.getProcessor().toString().equals("RISCV")
|
||||
|| currentProgram.getDefaultPointerSize() != 4)
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.id() + " requires a 32-bit RISC-V program, not "
|
||||
+ language.getLanguageID());
|
||||
CompilerSpec spec = currentProgram.getCompilerSpec();
|
||||
new SpecExtension(currentProgram)
|
||||
.addReplaceCompilerSpecExtension(
|
||||
derive(spec.getDefaultCallingConvention(), spec, language, policy), monitor);
|
||||
PrototypeModel model =
|
||||
currentProgram.getCompilerSpec().getCallingConvention(policy.prototype());
|
||||
if (model == null)
|
||||
throw new IllegalStateException("REA " + policy.prototype() + " was not installed");
|
||||
selfCheck(model, policy, language);
|
||||
int functions = 0;
|
||||
List<Function> callables = new ArrayList<>();
|
||||
currentProgram.getFunctionManager().getFunctions(true).forEach(callables::add);
|
||||
currentProgram.getFunctionManager().getExternalFunctions().forEach(callables::add);
|
||||
for (Function function : callables) {
|
||||
// A thunk's signature is its target's; the target is in this list.
|
||||
if (function.isThunk()) continue;
|
||||
if (function.hasCustomVariableStorage()) function.setCustomVariableStorage(false);
|
||||
function.setCallingConvention(policy.prototype());
|
||||
if (policy.floatEntryBytes() == 0) {
|
||||
List<VariableStorage> storage = new ArrayList<>();
|
||||
storage.add(function.getReturn().getVariableStorage());
|
||||
for (Parameter parameter : function.getParameters())
|
||||
storage.add(parameter.getVariableStorage());
|
||||
// An untyped function has no assigned storage yet; the decompiler applies the
|
||||
// prototype when it recovers one.
|
||||
requireIntegerStorage(
|
||||
storage.toArray(VariableStorage[]::new), function.getName(true), true);
|
||||
}
|
||||
functions += 1;
|
||||
}
|
||||
int definitions = 0;
|
||||
Iterator<DataType> types = currentProgram.getDataTypeManager().getAllDataTypes();
|
||||
while (types.hasNext()) {
|
||||
if (!(types.next() instanceof FunctionDefinition definition)) continue;
|
||||
definition.setCallingConvention(policy.prototype());
|
||||
definitions += 1;
|
||||
}
|
||||
// Written last: any failure above leaves no report, and REA refuses the session.
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setString(
|
||||
REPORT_KEY,
|
||||
"{\"policy\":\"" + policy.id() + "\",\"functions\":" + functions
|
||||
+ ",\"definitions\":" + definitions + "}");
|
||||
println(
|
||||
"REA RISC-V " + policy.prototype() + ": functions=" + functions
|
||||
+ " definitions=" + definitions);
|
||||
}
|
||||
|
||||
/** The default ilp32d prototype changed as the policy states. */
|
||||
private static String derive(
|
||||
PrototypeModel base, CompilerSpec spec, Language language, Policy policy) throws Exception {
|
||||
XmlEncode encoder = new XmlEncode();
|
||||
base.encode(encoder, spec.getPcodeInjectLibrary());
|
||||
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
|
||||
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
|
||||
Document document =
|
||||
factory.newDocumentBuilder().parse(new InputSource(new StringReader(encoder.toString())));
|
||||
Element root = document.getDocumentElement();
|
||||
if (!root.getTagName().equals("prototype"))
|
||||
throw new IllegalStateException("Unexpected default prototype encoding " + root.getTagName());
|
||||
root.setAttribute("name", policy.prototype());
|
||||
List<Element> floatEntries = new ArrayList<>();
|
||||
NodeList entries = root.getElementsByTagName("pentry");
|
||||
for (int index = 0; index < entries.getLength(); index += 1) {
|
||||
Element entry = (Element) entries.item(index);
|
||||
if (entry.getAttribute("storage").equals("float")) floatEntries.add(entry);
|
||||
}
|
||||
if (floatEntries.isEmpty())
|
||||
throw new IllegalStateException("Default prototype has no floating-point entries");
|
||||
if (policy.floatEntryBytes() == 0) {
|
||||
List<Node> removed = new ArrayList<>(floatEntries);
|
||||
NodeList rules = root.getElementsByTagName("rule");
|
||||
for (int index = 0; index < rules.getLength(); index += 1) {
|
||||
Element rule = (Element) rules.item(index);
|
||||
if (usesFloatStorage(rule)) removed.add(rule);
|
||||
}
|
||||
if (removed.size() == floatEntries.size())
|
||||
throw new IllegalStateException("Default prototype has no floating-point rules to remove");
|
||||
for (Node node : removed) node.getParentNode().removeChild(node);
|
||||
} else
|
||||
for (Element entry : floatEntries)
|
||||
entry.setAttribute("maxsize", Integer.toString(policy.floatEntryBytes()));
|
||||
// Values over 2 x XLEN go by reference. Returns need no rule: a value that the return
|
||||
// registers cannot hold already becomes a caller buffer passed in a0.
|
||||
NodeList inputs = root.getElementsByTagName("input");
|
||||
if (inputs.getLength() != 1)
|
||||
throw new IllegalStateException("Default prototype has no single input list");
|
||||
Element input = (Element) inputs.item(0);
|
||||
Element last = null;
|
||||
for (Node child = input.getFirstChild(); child != null; child = child.getNextSibling())
|
||||
if (child instanceof Element element && element.getTagName().equals("rule")) last = element;
|
||||
if (last == null || !joinsAnyValue(last))
|
||||
throw new IllegalStateException("Default prototype's input list does not end with a join");
|
||||
Element byReference = document.createElement("rule");
|
||||
Element datatype = document.createElement("datatype");
|
||||
datatype.setAttribute("name", "any");
|
||||
datatype.setAttribute("minsize", "9");
|
||||
byReference.appendChild(datatype);
|
||||
byReference.appendChild(document.createElement("convert_to_ptr"));
|
||||
input.insertBefore(byReference, last);
|
||||
if (!policy.preservesFloatRegisters()) {
|
||||
NodeList killedLists = root.getElementsByTagName("killedbycall");
|
||||
Element killed = killedLists.getLength() == 0 ? null : (Element) killedLists.item(0);
|
||||
if (killed == null) {
|
||||
killed = document.createElement("killedbycall");
|
||||
root.appendChild(killed);
|
||||
}
|
||||
List<Element> moved = new ArrayList<>();
|
||||
NodeList preservedLists = root.getElementsByTagName("unaffected");
|
||||
for (int list = 0; list < preservedLists.getLength(); list += 1) {
|
||||
NodeList preserved = preservedLists.item(list).getChildNodes();
|
||||
for (int index = 0; index < preserved.getLength(); index += 1) {
|
||||
if (preserved.item(index) instanceof Element varnode
|
||||
&& isFloatRegister(varnode, language))
|
||||
moved.add(varnode);
|
||||
}
|
||||
}
|
||||
for (Element varnode : moved) {
|
||||
varnode.getParentNode().removeChild(varnode);
|
||||
killed.appendChild(varnode);
|
||||
}
|
||||
}
|
||||
StringWriter xml = new StringWriter();
|
||||
var transformer = TransformerFactory.newInstance().newTransformer();
|
||||
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
|
||||
transformer.transform(new DOMSource(document), new StreamResult(xml));
|
||||
return xml.toString();
|
||||
}
|
||||
|
||||
/** The installed prototype places each class of value as its policy states. */
|
||||
private void selfCheck(PrototypeModel model, Policy policy, Language language) {
|
||||
StructureDataType pair = new StructureDataType("rea_float_pair", 0);
|
||||
pair.add(FloatDataType.dataType);
|
||||
pair.add(FloatDataType.dataType);
|
||||
VariableStorage[] storage =
|
||||
model.getStorageLocations(
|
||||
currentProgram,
|
||||
new DataType[] {
|
||||
DoubleDataType.dataType, FloatDataType.dataType, DoubleDataType.dataType,
|
||||
IntegerDataType.dataType, pair, Float16DataType.dataType
|
||||
},
|
||||
false);
|
||||
VariableStorage quad = storage[storage.length - 1];
|
||||
if (quad.size() != currentProgram.getDefaultPointerSize() || quad.getRegister() == null)
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.prototype() + " does not pass a 16-byte value by reference: " + quad);
|
||||
if (policy.floatEntryBytes() == 0)
|
||||
requireIntegerStorage(storage, policy.prototype() + " self-check", false);
|
||||
if (policy.preservesFloatRegisters()) return;
|
||||
for (Varnode preserved : model.getUnaffectedList()) {
|
||||
Register register = language.getRegister(preserved.getAddress(), preserved.getSize());
|
||||
if (register != null && FLOAT_REGISTER.matcher(register.getName()).matches())
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.prototype() + " still preserves " + register.getName());
|
||||
}
|
||||
}
|
||||
|
||||
private static boolean joinsAnyValue(Element rule) {
|
||||
List<Element> children = new ArrayList<>();
|
||||
for (Node child = rule.getFirstChild(); child != null; child = child.getNextSibling())
|
||||
if (child instanceof Element element) children.add(element);
|
||||
return children.size() == 2
|
||||
&& children.get(0).getTagName().equals("datatype")
|
||||
&& children.get(0).getAttribute("name").equals("any")
|
||||
&& children.get(1).getTagName().equals("join");
|
||||
}
|
||||
|
||||
/** Whether a rule's action, at any depth, assigns floating-point storage. */
|
||||
private static boolean usesFloatStorage(Element element) {
|
||||
NodeList children = element.getChildNodes();
|
||||
for (int index = 0; index < children.getLength(); index += 1) {
|
||||
if (!(children.item(index) instanceof Element child)) continue;
|
||||
if (child.getAttribute("storage").equals("float") || usesFloatStorage(child)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private static boolean isFloatRegister(Element varnode, Language language) {
|
||||
if (!varnode.getTagName().equals("varnode")
|
||||
|| !varnode.getAttribute("space").equals("register"))
|
||||
return false;
|
||||
Register register =
|
||||
language.getRegister(
|
||||
language.getAddressFactory().getRegisterSpace()
|
||||
.getAddress(Long.decode(varnode.getAttribute("offset"))),
|
||||
Integer.parseInt(varnode.getAttribute("size")));
|
||||
return register != null && FLOAT_REGISTER.matcher(register.getName()).matches();
|
||||
}
|
||||
|
||||
private static void requireIntegerStorage(
|
||||
VariableStorage[] storage, String owner, boolean allowUnassigned) {
|
||||
for (VariableStorage item : storage) {
|
||||
if (item.isBadStorage() || (!allowUnassigned && item.isUnassignedStorage()))
|
||||
throw new IllegalStateException("REA RISC-V storage unresolved for " + owner);
|
||||
List<Register> registers = item.getRegisters();
|
||||
if (registers == null) continue; // stack-only storage
|
||||
for (Register register : registers) {
|
||||
if (FLOAT_REGISTER.matcher(register.getName()).matches())
|
||||
throw new IllegalStateException(
|
||||
"REA RISC-V storage still uses " + register.getName() + " for " + owner);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+5
-161
@@ -22,9 +22,6 @@ HOPPER_SYMBOL_NAME_UTF16_UNITS = 1024
|
||||
# that does not fit is null, not an older descriptor path.
|
||||
# 536870887 is Node's 536870888-character string ceiling minus the framing LF.
|
||||
# The ceiling is the encoded payload; JSON expansion is measured before it.
|
||||
# Encoded JSON allowance, in bytes, shared by one instruction list, one
|
||||
# native-caller list, and one caller-scan page. This release has no runtime override.
|
||||
CALLER_SCAN_ALLOWANCE_BYTES = 512 * 1024
|
||||
_BRIDGE_MAX_REQUEST_BYTES_SETTING = "REA_BRIDGE_MAX_REQUEST_BYTES"
|
||||
DEFAULT_BRIDGE_MAX_REQUEST_BYTES = 16 * 1024 * 1024
|
||||
MINIMUM_BRIDGE_MAX_REQUEST_BYTES = 7464
|
||||
@@ -235,9 +232,6 @@ EXHAUSTIVE_ANALYSIS_METHODS = frozenset(
|
||||
"procedure_assembly",
|
||||
"procedure_callees",
|
||||
"procedure_callers",
|
||||
"rea_caller_scan_page",
|
||||
"rea_caller_scan_target",
|
||||
"rea_caller_base",
|
||||
"procedure_info",
|
||||
"procedure_pseudo_code",
|
||||
"procedure_references",
|
||||
@@ -491,30 +485,19 @@ def _instruction_addresses(procedure):
|
||||
return result
|
||||
|
||||
|
||||
def _iter_native_calls(procedure, direction):
|
||||
def _native_calls(procedure, direction):
|
||||
references = procedure.getAllCallees() if direction == "outgoing" else procedure.getAllCallers()
|
||||
kinds = {0: "none", 1: "unknown", 2: "direct", 3: "objective_c"}
|
||||
calls = {}
|
||||
unresolved = []
|
||||
for reference in references:
|
||||
source, target, kind = reference.fromAddress(), reference.toAddress(), reference.type()
|
||||
if source in BAD_ADDRESSES:
|
||||
yield None, None, None
|
||||
continue
|
||||
if target in BAD_ADDRESSES:
|
||||
yield source, None, {"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind}
|
||||
unresolved.append({"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind})
|
||||
continue
|
||||
yield source, target, {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
|
||||
|
||||
|
||||
def _native_calls(procedure, direction):
|
||||
calls = {}
|
||||
unresolved = []
|
||||
for source, target, detail in _iter_native_calls(procedure, direction):
|
||||
if detail is None:
|
||||
continue
|
||||
if target is None:
|
||||
unresolved.append(detail)
|
||||
else:
|
||||
calls[(source, target)] = detail
|
||||
calls[(source, target)] = {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
|
||||
return calls, sorted(unresolved, key=lambda item: int(item["address"], 16))
|
||||
|
||||
|
||||
@@ -564,136 +547,6 @@ def _procedure_map(document):
|
||||
return result
|
||||
|
||||
|
||||
def _caller_target_data(procedure):
|
||||
addresses = []
|
||||
seen = set()
|
||||
size = 0
|
||||
for block in procedure.basicBlockIterator():
|
||||
for address in _block_instruction_addresses(procedure, block):
|
||||
if address in seen:
|
||||
continue
|
||||
encoded = _hex(address)
|
||||
size += len(encoded) + 4
|
||||
if size > CALLER_SCAN_ALLOWANCE_BYTES:
|
||||
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": False}
|
||||
addresses.append(encoded)
|
||||
seen.add(address)
|
||||
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": True}
|
||||
|
||||
|
||||
def _caller_scan_target(document, params):
|
||||
return _caller_target_data(_procedure(document, params.get("procedure")))
|
||||
|
||||
|
||||
def _caller_base(document, params):
|
||||
procedure = _procedure(document, params.get("procedure"))
|
||||
callers = []
|
||||
size = 0
|
||||
complete = True
|
||||
for item in procedure.getAllCallerProcedures():
|
||||
address = _hex(item.getEntryPoint())
|
||||
size += len(address) + 4
|
||||
if size > CALLER_SCAN_ALLOWANCE_BYTES:
|
||||
complete = False
|
||||
break
|
||||
callers.append(address)
|
||||
return {
|
||||
**_caller_target_data(procedure),
|
||||
"callers": sorted(callers, key=lambda value: int(value, 16)),
|
||||
"callers_complete": complete,
|
||||
}
|
||||
|
||||
|
||||
def _caller_scan_page(document, params):
|
||||
"""Observe forward relationships in one bounded, stateless bridge request."""
|
||||
start = params.get("start")
|
||||
if isinstance(start, bool) or not isinstance(start, int) or start < 0:
|
||||
raise InvalidRequestError("Caller scan start must be a non-negative integer")
|
||||
segments = document.getSegmentsList()
|
||||
counts = [segment.getProcedureCount() for segment in segments]
|
||||
total = sum(counts)
|
||||
end = min(start + 64, total)
|
||||
if start > total:
|
||||
raise InvalidRequestError("Caller scan start exceeds the procedure inventory")
|
||||
items = []
|
||||
item_bytes = 0
|
||||
page_limit = CALLER_SCAN_ALLOWANCE_BYTES
|
||||
oversized = False
|
||||
identities = {}
|
||||
containing = {}
|
||||
|
||||
def identity(procedure):
|
||||
entry = procedure.getEntryPoint()
|
||||
if entry not in identities:
|
||||
identities[entry] = _procedure_identity(procedure)
|
||||
return identities[entry]
|
||||
|
||||
def owner(address):
|
||||
if address not in containing:
|
||||
procedure, _ = _containing_procedure(document, address)
|
||||
containing[address] = identity(procedure) if procedure is not None else None
|
||||
return containing[address]
|
||||
|
||||
segment_start = 0
|
||||
for segment, count in zip(segments, counts):
|
||||
begin, stop = max(start - segment_start, 0), min(end - segment_start, count)
|
||||
segment_start += count
|
||||
for index in range(begin, max(begin, stop)):
|
||||
candidate = segment.getProcedureAtIndex(index)
|
||||
item = {"caller": identity(candidate), "callees": [], "references": []}
|
||||
size = len(json.dumps(item, ensure_ascii=True).encode("utf-8")) + 1
|
||||
remaining = page_limit - item_bytes
|
||||
too_large = size > remaining
|
||||
if not too_large:
|
||||
for callee in candidate.getAllCalleeProcedures():
|
||||
address = _hex(callee.getEntryPoint())
|
||||
size += len(address) + 4
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
item["callees"].append(address)
|
||||
if not too_large:
|
||||
callee_entries = set(item["callees"])
|
||||
for source, destination, call in _iter_native_calls(candidate, "outgoing"):
|
||||
# Account for every native record, even when its target cannot
|
||||
# be associated with an observed procedure.
|
||||
size += 32
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
if destination is None:
|
||||
continue
|
||||
source_procedure = item["caller"] if _hex(source) == item["caller"]["address"] else owner(source)
|
||||
target_address = _hex(destination)
|
||||
if target_address in callee_entries:
|
||||
target_entry = target_address
|
||||
else:
|
||||
target_procedure = owner(destination)
|
||||
if target_procedure is None:
|
||||
continue
|
||||
target_entry = target_procedure["address"]
|
||||
edge = {
|
||||
"source_address": _hex(source), "target_address": _hex(destination),
|
||||
"source_procedure": "caller" if source_procedure is not None and source_procedure["address"] == item["caller"]["address"] else source_procedure,
|
||||
"target_procedure": target_entry,
|
||||
"call": call,
|
||||
}
|
||||
size += len(json.dumps(edge, ensure_ascii=True).encode("utf-8")) + 1
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
item["references"].append(edge)
|
||||
if too_large:
|
||||
oversized = not items
|
||||
break
|
||||
items.append(item)
|
||||
item_bytes += size
|
||||
else:
|
||||
continue
|
||||
break
|
||||
return {"start": start, "next": start + len(items), "total": total, "items": items, "oversized": oversized}
|
||||
|
||||
|
||||
def _string_record(segment, address, display):
|
||||
"""Read one native typed object, retaining its shortened display separately."""
|
||||
length = segment.getObjectLength(address)
|
||||
@@ -1034,15 +887,6 @@ def _dispatch(method, params):
|
||||
if method in EXHAUSTIVE_ANALYSIS_METHODS:
|
||||
_api().require_analysis_complete(document, method)
|
||||
|
||||
if method == "rea_caller_scan_target":
|
||||
return _caller_scan_target(document, params)
|
||||
|
||||
if method == "rea_caller_base":
|
||||
return _caller_base(document, params)
|
||||
|
||||
if method == "rea_caller_scan_page":
|
||||
return _caller_scan_page(document, params)
|
||||
|
||||
if method == "analyze_function":
|
||||
return _analyze_function(document, params)
|
||||
if method == "read_function_instructions":
|
||||
|
||||
@@ -65,10 +65,7 @@ func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard SHA256.hash(data: executableBytes).map({ String(format: "%02x", $0) }).joined() == request.sha256 else {
|
||||
try fail("target-mismatch", "Running executable bytes differ from the active target digest")
|
||||
}
|
||||
// .optionIncludingWindow omits a selected window while it is off screen, which
|
||||
// is not absence: select from every window, and let capture report what it can.
|
||||
guard let windows = (CGWindowListCopyWindowInfo([.optionAll], kCGNullWindowID) as? [[String: Any]])?
|
||||
.filter({ ($0[kCGWindowNumber as String] as? NSNumber)?.uint32Value == request.window_id }),
|
||||
guard let windows = CGWindowListCopyWindowInfo([.optionIncludingWindow], request.window_id) as? [[String: Any]],
|
||||
windows.count == 1, let window = windows.first,
|
||||
(window[kCGWindowOwnerPID as String] as? NSNumber)?.int32Value == request.pid,
|
||||
(window[kCGWindowLayer as String] as? NSNumber)?.intValue == 0,
|
||||
|
||||
@@ -15,8 +15,6 @@ struct ProcessIdentityObservation: Encodable {
|
||||
let effective_uid: UInt32?
|
||||
let lifecycle: String?
|
||||
let relation: ProcessRelationObservation?
|
||||
/** Display-only kernel accounting name (p_comm, at most MAXCOMLEN bytes). */
|
||||
let name: String?
|
||||
let reason: String?
|
||||
}
|
||||
|
||||
@@ -33,7 +31,6 @@ struct ProcessMetadata {
|
||||
let parentPid: Int32
|
||||
let processGroupId: Int32
|
||||
let pointerSize: Int
|
||||
let name: String
|
||||
}
|
||||
|
||||
struct ProcessSnapshot {
|
||||
@@ -95,8 +92,7 @@ func readProcessSnapshot(
|
||||
lifecycle: process.kp_proc.p_stat == SZOMB ? "zombie" : "non-zombie",
|
||||
parentPid: process.kp_eproc.e_ppid,
|
||||
processGroupId: process.kp_eproc.e_pgid,
|
||||
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4,
|
||||
name: processName(process)
|
||||
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4
|
||||
)
|
||||
)
|
||||
},
|
||||
@@ -107,13 +103,6 @@ func readProcessSnapshot(
|
||||
return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(ENOMEM)")
|
||||
}
|
||||
|
||||
/** Decode the NUL-terminated accounting name; invalid UTF-8 is replaced for display. */
|
||||
private func processName(_ process: kinfo_proc) -> String {
|
||||
withUnsafeBytes(of: process.kp_proc.p_comm) { bytes in
|
||||
String(decoding: bytes.prefix { $0 != 0 }, as: UTF8.self)
|
||||
}
|
||||
}
|
||||
|
||||
func readProcessIdentities(pids: [Int32]) -> [ProcessIdentityObservation] {
|
||||
readProcessIdentities(pids: pids, snapshot: readProcessSnapshot())
|
||||
}
|
||||
@@ -141,7 +130,6 @@ func readProcessIdentities(
|
||||
effective_uid: nil,
|
||||
lifecycle: nil,
|
||||
relation: nil,
|
||||
name: nil,
|
||||
reason: "not-in-snapshot"
|
||||
)
|
||||
}
|
||||
@@ -156,7 +144,6 @@ func readProcessIdentities(
|
||||
parent_pid: metadata.parentPid,
|
||||
process_group_id: metadata.processGroupId
|
||||
),
|
||||
name: metadata.name,
|
||||
reason: nil
|
||||
)
|
||||
}
|
||||
@@ -170,25 +157,12 @@ private func unavailableIdentity(pid: Int32, reason: String) -> ProcessIdentityO
|
||||
effective_uid: nil,
|
||||
lifecycle: nil,
|
||||
relation: nil,
|
||||
name: nil,
|
||||
reason: reason
|
||||
)
|
||||
}
|
||||
|
||||
enum ProcargsMalformedStage: String, CaseIterable {
|
||||
case header
|
||||
case argumentCount = "argument_count"
|
||||
case executablePath = "executable_path"
|
||||
case pointerSize = "pointer_size"
|
||||
case pathPadding = "path_padding"
|
||||
case arguments
|
||||
case terminator
|
||||
case trailingEntries = "trailing_entries"
|
||||
}
|
||||
|
||||
enum RunTokenReadError: Error {
|
||||
case invalidBuffer
|
||||
case invalidBufferAt(ProcargsMalformedStage)
|
||||
case duplicateToken
|
||||
case invalidTokenEncoding
|
||||
case environmentUnavailable
|
||||
@@ -228,22 +202,6 @@ func processRunTokenObservation(
|
||||
parse: Result<ParsedRunToken, RunTokenReadError>,
|
||||
codeSigningFlags: () -> UInt32?
|
||||
) -> RunTokenObservation {
|
||||
// Native inspection restrictions apply even when argv is incomplete or
|
||||
// overwritten, before parsing reaches the environment. Keep this fact at
|
||||
// the native boundary; cleanup still decides whether the process is related.
|
||||
func withheldEnvironment() -> RunTokenObservation? {
|
||||
guard let flags = codeSigningFlags() else { return nil }
|
||||
if (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func unavailableEnvironment(_ reason: String) -> RunTokenObservation {
|
||||
withheldEnvironment() ?? unavailable(pid: pid, reason: reason)
|
||||
}
|
||||
switch parse {
|
||||
case .success(let parsed):
|
||||
if let runId = parsed.runId {
|
||||
@@ -254,25 +212,38 @@ func processRunTokenObservation(
|
||||
if parsed.environmentEntryCount > 0 {
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
}
|
||||
if let withheld = withheldEnvironment() { return withheld }
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
case .failure(let failure):
|
||||
switch failure {
|
||||
case .environmentUnavailable:
|
||||
return unavailableEnvironment("environment_unavailable")
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "environment_unavailable")
|
||||
case .ambiguousEnvironmentBoundary:
|
||||
return unavailableEnvironment("ambiguous_environment_boundary")
|
||||
// These failures already establish that token bytes were available.
|
||||
if let flags = codeSigningFlags(), (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "ambiguous_environment_boundary")
|
||||
case .duplicateToken:
|
||||
return unavailable(pid: pid, reason: "duplicate_run_token")
|
||||
case .invalidTokenEncoding:
|
||||
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
|
||||
case .appleVectorUnavailable:
|
||||
return unavailableEnvironment("apple_vector_unavailable")
|
||||
return unavailable(pid: pid, reason: "apple_vector_unavailable")
|
||||
case .invalidBuffer:
|
||||
return unavailableEnvironment("malformed_procargs")
|
||||
case .invalidBufferAt(let stage):
|
||||
return unavailableEnvironment("malformed_procargs_\(stage.rawValue)")
|
||||
return unavailable(pid: pid, reason: "malformed_procargs")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -331,30 +302,27 @@ func parseProcessEnvironment(
|
||||
in bytes: [UInt8],
|
||||
pointerSize: Int = MemoryLayout<UnsafeRawPointer>.size
|
||||
) throws -> ParsedRunToken {
|
||||
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBufferAt(.header) }
|
||||
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBuffer }
|
||||
let argc = bytes.withUnsafeBytes { $0.loadUnaligned(as: Int32.self).littleEndian }
|
||||
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBufferAt(.argumentCount) }
|
||||
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBuffer }
|
||||
var offset = MemoryLayout<Int32>.size
|
||||
guard let executablePath = readString(in: bytes, offset: &offset) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.executablePath)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBufferAt(.pointerSize) }
|
||||
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBuffer }
|
||||
let pathRecordSize = Array("executable_path=".utf8).count + executablePath.count + 1
|
||||
let padding = (pointerSize - pathRecordSize % pointerSize) % pointerSize
|
||||
guard offset + padding <= bytes.count,
|
||||
bytes[offset..<(offset + padding)].allSatisfy({ $0 == 0 }) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.pathPadding)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
offset += padding
|
||||
let tokenPrefix = Array("REA_PROCESS_RUN_ID=".utf8)
|
||||
var tokenInArgument = false
|
||||
for _ in 0..<argc {
|
||||
guard let argument = readString(in: bytes, offset: &offset, allowEmpty: true) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.arguments)
|
||||
guard readString(in: bytes, offset: &offset, allowEmpty: true) != nil else {
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
tokenInArgument = tokenInArgument || argument.starts(with: tokenPrefix)
|
||||
}
|
||||
guard bytes.last == 0 else { throw RunTokenReadError.invalidBufferAt(.terminator) }
|
||||
guard bytes.last == 0 else { throw RunTokenReadError.invalidBuffer }
|
||||
if offset == bytes.count {
|
||||
throw RunTokenReadError.environmentUnavailable
|
||||
}
|
||||
@@ -372,7 +340,7 @@ func parseProcessEnvironment(
|
||||
continue
|
||||
}
|
||||
guard let entry = readString(in: bytes, offset: &offset) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.trailingEntries)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
trailingEntries.append(entry)
|
||||
}
|
||||
@@ -393,28 +361,22 @@ func parseProcessEnvironment(
|
||||
throw RunTokenReadError.appleVectorUnavailable
|
||||
}
|
||||
if trailingEntries[appleVectorStart...].contains(where: {
|
||||
$0.starts(with: tokenPrefix)
|
||||
$0.starts(with: Array("REA_PROCESS_RUN_ID=".utf8))
|
||||
}) {
|
||||
throw RunTokenReadError.ambiguousEnvironmentBoundary
|
||||
}
|
||||
let environmentEntries = trailingEntries[..<appleVectorStart]
|
||||
var found: String?
|
||||
for entry in environmentEntries {
|
||||
if entry.starts(with: tokenPrefix) {
|
||||
if entry.starts(with: Array("REA_PROCESS_RUN_ID=".utf8)) {
|
||||
guard found == nil else { throw RunTokenReadError.duplicateToken }
|
||||
let value = entry.dropFirst(tokenPrefix.count)
|
||||
let value = entry.dropFirst("REA_PROCESS_RUN_ID=".utf8.count)
|
||||
guard let decoded = String(bytes: value, encoding: .utf8) else {
|
||||
throw RunTokenReadError.invalidTokenEncoding
|
||||
}
|
||||
found = decoded
|
||||
}
|
||||
}
|
||||
// A changed process title can overwrite the original argument boundaries,
|
||||
// causing a real environment entry to be consumed as an argument. A real
|
||||
// argument can have this form too, so it proves neither ownership nor absence.
|
||||
if tokenInArgument {
|
||||
throw RunTokenReadError.ambiguousEnvironmentBoundary
|
||||
}
|
||||
return ParsedRunToken(runId: found, environmentEntryCount: environmentEntries.count)
|
||||
}
|
||||
|
||||
|
||||
@@ -16,10 +16,10 @@ PROFILE = "[email protected];[email protected];[email protected]"
|
||||
PACKAGES = {"pwntools": "4.15.0", "pyelftools": "0.33", "unicorn": "2.1.2"}
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
# (e_machine, ELF class bits, byte order) -> architecture pwntools must also report.
|
||||
PROFILES = {(62, 64, "little"): "amd64", (183, 64, "little"): "aarch64", (3, 32, "little"): "i386",
|
||||
(40, 32, "little"): "arm", (8, 32, "little"): "mips", (8, 32, "big"): "mips"}
|
||||
PROFILE_SCOPE = ("Layout profile requires ELF64 x86-64, ELF64 AArch64 little-endian, ELF32 i386, "
|
||||
"ELF32 ARM little-endian or ELF32 MIPS little- or big-endian objects.")
|
||||
PROFILES = {(62, 64, "little"): "amd64", (3, 32, "little"): "i386", (40, 32, "little"): "arm",
|
||||
(8, 32, "little"): "mips", (8, 32, "big"): "mips"}
|
||||
PROFILE_SCOPE = ("Layout profile requires ELF64 x86-64, ELF32 i386, ELF32 ARM little-endian "
|
||||
"or ELF32 MIPS little- or big-endian objects.")
|
||||
|
||||
|
||||
class LayoutFailure(Exception):
|
||||
|
||||
@@ -70,7 +70,6 @@ export default defineConfig({
|
||||
{ text: "Apple applications", link: "/apple-application-analysis" },
|
||||
{ text: "Desktop installations", link: "/desktop-applications" },
|
||||
{ text: "Firmware", link: "/firmware-analysis" },
|
||||
{ text: "iNES containers", link: "/ines-containers" },
|
||||
{ text: "SQLite databases", link: "/sqlite-databases" },
|
||||
{ text: "Packet captures", link: "/packet-captures" },
|
||||
{ text: "Signal recordings", link: "/signal-recordings" },
|
||||
|
||||
@@ -103,10 +103,6 @@ existing analysis families directly, for example
|
||||
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
|
||||
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
|
||||
|
||||
Screen capture validates the PNG signature, IHDR framing, dimensions, and IHDR
|
||||
checksum before publishing. It does not validate encoding fields or decode and
|
||||
validate image contents; that limitation is retained in Evidence.
|
||||
|
||||
When `push_adb_file` receives an existing directory, adb places the local file
|
||||
inside it under the local basename; Evidence reports that effective file path
|
||||
while retaining the caller's directory path in the input parameters. The
|
||||
@@ -257,12 +253,12 @@ beyond already-connected devices (`adb connect`) are not exposed yet.
|
||||
|
||||
## Real verification
|
||||
|
||||
`npm run verify:adb` exercises observation and transfer operations against a live device. It
|
||||
`npm run verify:adb` exercises every operation against a live device. It
|
||||
requires `adb` (or `REA_ADB_PATH`) and any attached emulator or phone;
|
||||
`--serial` selects one when several are attached, `--pull PACKAGE` pulls a
|
||||
real package and verifies the returned digests against the files on disk.
|
||||
The optional `--install-apk` lifecycle is unavailable until the selected APK
|
||||
identity and device cleanup ownership can be established safely.
|
||||
real package and verifies the returned digests against the files on disk,
|
||||
and `--install-apk PATH` drives a full install → resolve → uninstall
|
||||
lifecycle with a lane-owned probe.
|
||||
|
||||
Record: verified against adb 34.0.5 on Linux against an Android 14
|
||||
(API 34) x86_64 emulator — all operations above, a 224-package inventory,
|
||||
|
||||
@@ -158,12 +158,12 @@ assigned by the app.
|
||||
|
||||
Use the report's hints to decide what to record next:
|
||||
|
||||
| Reason | Next step |
|
||||
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `unconfirmed` | Read the item and its list after the change. |
|
||||
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
|
||||
| `no-observation` | Read back a created item. For an edit or delete, also read it before changing it. |
|
||||
| `contradicted` or `side-effect` | Inspect the reported disagreement. The operation may need behavior REA cannot model yet. |
|
||||
| Reason | Next step |
|
||||
| ------------------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| `unconfirmed` | Read the item and its list after the change. |
|
||||
| `no-state-target` | Record reads of the collection or item the operation changes. |
|
||||
| `no-observation` | Read back a created item. For an edit or delete, also read it before changing it. |
|
||||
| `contradicted` or `side-effect` | Inspect the reported disagreement. The operation may need behavior REA cannot model yet. |
|
||||
|
||||
See the [full reason table](agent-environments.md#build-a-bundle-in-one-step)
|
||||
for the other cases. Rebuild from the new recording and check the report
|
||||
|
||||
+17
-40
@@ -174,7 +174,7 @@ what to record next; hints never change what is served:
|
||||
|
||||
| Reason | Hint |
|
||||
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `no-state-target` | The write could not be linked to a modelled target. Check the path template and recorded views; record missing reads. Some route forms may not be supported. |
|
||||
| `no-state-target` | The write's path names no collection or item the capture reads. Read the collection or item it changes. |
|
||||
| `no-call` | No recorded call of this tool was modelled, for example because every call failed. Record a successful call. |
|
||||
| `body-unusable` | The retained request or response cannot be modelled. Record a successful call with a complete supported request and reproducible response. |
|
||||
| `no-observation` | Nothing showed the entity or document this write changed. Read a created entity back by id right after the write; for an update or delete the state must already hold the entity, so read it before the write or record the create that made it. |
|
||||
@@ -183,7 +183,6 @@ what to record next; hints never change what is served:
|
||||
| `side-effect` | The write changed something besides its own entity, which the environment does not model. |
|
||||
| `unconfirmed` | No later read showed the effect. Read the entity and its list after the write. |
|
||||
| `incomplete-entities` | The write is confirmed, but the views that show its entity cannot render it in full. Read the written entity through every view that lists it. |
|
||||
| `unknown-clock` | The learned rules generate timestamps, but the capture establishes no global served clock start. Record usable start times and completed durations for every request. |
|
||||
|
||||
The bundle ships `evidence.json`, the whole capture Evidence, response bodies
|
||||
included. Inspection excludes the values of the standard authentication headers
|
||||
@@ -205,10 +204,7 @@ attached and a transaction no record represents, and each transaction URL's
|
||||
query once. Bodies larger than 4 MiB of UTF-8 text are not
|
||||
scanned; `credential-scan-incomplete` counts them and lists up to 20 record or
|
||||
event pointers, including base64 bodies whose decoded scan was cut. An empty name
|
||||
warning does not establish that these bodies were checked. A browser scenario's
|
||||
saved values (`save` and `save_response`) named like credentials but not marked
|
||||
`secret: true` are recorded unredacted; `credential-named-saved-values` lists up
|
||||
to 20 of them by parameter pointer. Names do not establish sensitivity,
|
||||
warning does not establish that these bodies were checked. Names do not establish sensitivity,
|
||||
so the build never refuses on them. Capture
|
||||
|
||||
Evidence also records the capture file's absolute path (`subject.local_path`,
|
||||
@@ -406,8 +402,7 @@ returns that record's recorded response. Selection compares the supplied path
|
||||
parameters against the record's path segments at the parameter indexes, then
|
||||
the supplied query values against the record's query string, and keeps the
|
||||
record with the most matches; ties keep manifest order. A query argument accepts
|
||||
one string, a number or boolean sent as its JSON text, or an array of strings for
|
||||
repeated values; the array is matched
|
||||
one string or an array of strings for repeated values; the array is matched
|
||||
against every occurrence in captured order. The raw recorded query remains the
|
||||
replay evidence. The result reports
|
||||
where the answer came from:
|
||||
@@ -614,9 +609,8 @@ shape and carries no annotations.
|
||||
Path parameters are required strings whose
|
||||
description names the segment index, the templating rule and the observed
|
||||
values (sampled within the list budget above; the manifest keeps them all);
|
||||
query parameter names accept an optional string, number or boolean (a number or
|
||||
boolean is sent as its JSON text) or array of strings (for repeated values in
|
||||
captured order); write methods accept an
|
||||
query parameter names accept an optional string or array of strings (for
|
||||
repeated values in captured order); write methods accept an
|
||||
optional `body` object described by the recorded top-level key types. Each
|
||||
input keeps its own argument: a query name equal to a path parameter's name is
|
||||
advertised as `query_<name>`, a query name longer than 64 characters as
|
||||
@@ -723,8 +717,6 @@ and `POST` tools the capture shows only reading (a lookup answering as a `GET`
|
||||
did, an RPC tunnel naming read operations, a session renewal), stay served and
|
||||
answer from their recordings with the reason `query-body`; they are not write
|
||||
tools.
|
||||
`HEAD`, `OPTIONS` and `TRACE` tools also replay their captured responses; their
|
||||
state views report `method-replay` and coverage counts them as recorded reads.
|
||||
|
||||
A line establishes whole-collection membership only from the beginning:
|
||||
`offset`, `start`, `_start` and `page` must be zero when present. Positive or
|
||||
@@ -739,14 +731,14 @@ tools are not served, so no call claims a change it does not make; each one's
|
||||
plan states why in its `reason`. A served write changes the live state, which
|
||||
every MCP session and `data/get` then see:
|
||||
|
||||
| Effect | Call |
|
||||
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
|
||||
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, nested clock, increment, slug, stamp, derived, composed, reference, unique and renewed changes the plan models. |
|
||||
| `update` (action) | A `POST`, `PUT` or `PATCH` to an item path plus an action segment (`/{type}/{id}/archive`) sets, on the entity before the segment, each scalar the recorded calls showed the action set (`set`), or flips each boolean they showed it flipping both ways (`toggle`), with any sent members; it is served only when a recorded call found what it sets already set, or flipped back what it toggles. |
|
||||
| `delete` | Removes the entity the path names. |
|
||||
| bulk `update` or `delete` | A `DELETE`, `PUT` or `PATCH` at a collection, or a `POST` to it plus `delete`, `remove` or `destroy`, applies its effect to every entity the body names, as recorded: an array of ids or of objects carrying the id field, at the root or as the body's one array member (`{"IDs": […], "Read": true}`). |
|
||||
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
|
||||
| Effect | Call |
|
||||
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
|
||||
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, nested clock, increment, slug, stamp, derived, composed, reference, unique and renewed changes the plan models. |
|
||||
| `update` (action) | A `POST`, `PUT` or `PATCH` to an item path plus an action segment (`/{type}/{id}/archive`) sets, on the entity before the segment, each scalar the recorded calls showed the action set (`set`), with any sent members; it is served only when a recorded call found those fields already set, so a toggle never is. |
|
||||
| `delete` | Removes the entity the path names. |
|
||||
| bulk `update` or `delete` | A `DELETE`, `PUT` or `PATCH` at a collection, or a `POST` to it plus `delete`, `remove` or `destroy`, applies its effect to every entity the body names, as recorded: an array of ids or of objects carrying the id field, at the root or as the body's one array member (`{"IDs": […], "Read": true}`). |
|
||||
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
|
||||
|
||||
An update's unsent changes follow the plan's rules. A `held-clock` is a clock
|
||||
the client sends back as it read it and the application stamps anew (Ghost's
|
||||
@@ -757,9 +749,7 @@ the entity already names under its `created…by` counterpart into a
|
||||
`modified…by` or `updated…by` field. A `derived` field takes a value showing
|
||||
the one sent member's text that the entity stores nowhere (a field Ghost
|
||||
derives from `html`), only with that member's recorded value. An action's
|
||||
`set` rule sets a scalar to the value the recorded calls left it at, and its
|
||||
`toggle` rule flips a boolean, refusing a call on an entity that holds none
|
||||
there (Miniflux's `PUT /v1/entries/{id}/bookmark` stars and unstars).
|
||||
`set` rule sets a scalar to the value the recorded calls left it at.
|
||||
|
||||
Its answer has the status and content type of the plan's recorded response,
|
||||
with the written entity or its identity rendered in, no body, or the recorded
|
||||
@@ -774,19 +764,11 @@ sending a `derived` rule's member to an entity that does not hold the value the
|
||||
recorded call found, a nested clock on an entity holding no such object, and a
|
||||
rendered answer missing a field the recorded response carried, and an update or delete of an entity the state does not hold answers
|
||||
the recorded 400, 404 or 410 of the tool or of an item view of the type, if
|
||||
any. A served delete of a type whose item view has a `deleted_flag` (the
|
||||
member, `deleted`, `isDeleted` or `is_deleted`, its reads set `true` on a
|
||||
record the application soft-deleted, as EspoCRM does) keeps the entity until
|
||||
`data/reset` or `data/add`, within the size `data/add` loads, so a read of it by
|
||||
id answers its last fields with that flag `true`. A refused call leaves the
|
||||
state unchanged. Values a write generates are
|
||||
any. A refused call leaves the state unchanged. Values a write generates are
|
||||
deterministic: the clock starts at `write_clock` and advances one second per
|
||||
served write, sequences take one past both the largest held value and the
|
||||
largest they generated, and UUIDs and tokens derive from the state Evidence
|
||||
id, tool, field and write count, skipping held values. If `write_clock` is null,
|
||||
plans needing `clock`, `held-clock`, or `nested-clock` retain their learned rules
|
||||
but are withheld with `unknown-clock`; other confirmed plans can still be served.
|
||||
`data/reset` and
|
||||
id, tool, field and write count, skipping held values. `data/reset` and
|
||||
`data/add` restart the clock, counter and sequences, so the same calls after a
|
||||
reset give the same state. Answers carry `state: {"source": "state", "view":
|
||||
"create" | "update" | "delete", …}` or `"document-write"`; a bulk answer adds
|
||||
@@ -1149,12 +1131,7 @@ global conflict involving an equal pair is not used to normalize either pair; th
|
||||
|
||||
A step agrees when the statuses and the normalised bodies are equal. The report gives agreement overall, per probe and
|
||||
per tool, `agreement_ignoring_order` for steps that differ only in item order, and each step's differences, at most 20,
|
||||
with a JSON pointer and both values cut short. Each step reports `differences_total` before that cap,
|
||||
`differences_omitted`, and `differences_truncated` (true when entries were omitted). Agreement and categories use the
|
||||
complete comparison. Omitted entries are not retained, so `--full-output` cannot recover them; inspect the original
|
||||
capture and probe Evidence to investigate beyond the returned pointers.
|
||||
|
||||
The difference categories are:
|
||||
with a JSON pointer and both values cut short:
|
||||
|
||||
| Category | Meaning |
|
||||
| --------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
|
||||
+3
-289
@@ -30,18 +30,15 @@ code units; follow `coverage.next_offset`. The page preserves `source_status` an
|
||||
`reported_total_bytes`: exhausting retained text does not make a partial decoded
|
||||
manifest complete. Complete source Evidence remains exportable under the existing
|
||||
session lifetime. Servers without retention reject summary before starting JADX.
|
||||
One-shot CLI output remains complete by default. Select `--detail summary` with
|
||||
`--evidence_output /local/capture.json` to retain complete Evidence before returning
|
||||
a linked view. Older servers require the existing complete request after schema discovery. APK authenticity and runtime behavior remain
|
||||
One-shot CLI output remains complete. Older servers require the existing complete
|
||||
request after schema discovery. APK authenticity and runtime behavior remain
|
||||
unverified.
|
||||
|
||||
## Manifest component projection
|
||||
|
||||
`inspect_android_package` also returns a static inventory of declared
|
||||
activities, aliases, services, receivers and providers from the decoded
|
||||
manifest. The binary XML decoder preserves component class spelling, including
|
||||
`$` in nested classes, without applying JADX class aliases. `declared_name`
|
||||
preserves the decoded source attribute; `name` resolves a
|
||||
manifest. `declared_name` preserves the source attribute; `name` resolves a
|
||||
relative name only when the manifest package is known and the resolved copy
|
||||
fits the UTF-8 byte budget. An unresolved package or exhausted name budget
|
||||
retains the declared name and marks the inventory partial. Empty component
|
||||
@@ -79,61 +76,6 @@ capacity limitation appears only when an actual collection limit was reached.
|
||||
The projection does not merge manifests, expand resource references, or
|
||||
observe runtime enabled/export state.
|
||||
|
||||
`enabled` and `grant_uri_permissions` report only literal component-level
|
||||
`android:enabled` and provider-level `android:grantUriPermissions` declarations.
|
||||
They are `null` when absent or not a literal boolean; missing attributes are not
|
||||
filled from application defaults. A nonliteral declaration gets a limitation.
|
||||
`process` preserves the exact component-level `android:process` spelling (including
|
||||
empty or symbolic values), or `null` when absent or omitted under the inventory-wide
|
||||
1 MiB copy budget. Neither field resolves resources, merges manifest values or
|
||||
establishes the effective runtime process or enabled state. The exact boolean
|
||||
attributes are retained in `enabled_declaration` and
|
||||
`grant_uri_permissions_declaration`, including resource references.
|
||||
|
||||
`intent_filters` preserves each filter's source ordinal, exact priority spelling,
|
||||
actions, categories and individual data declarations (scheme, host, port, path,
|
||||
path prefix/pattern/advanced pattern/suffix and MIME type). These are declarations,
|
||||
not computed Android matching rules. `meta_data` lists names; values remain in the
|
||||
complete decoded manifest. Component, filter, data and metadata `location` values
|
||||
use one-based lines/columns and UTF-16 offsets into that manifest. A component's
|
||||
`declaration_index` preserves its original declaration ordinal, including gaps.
|
||||
|
||||
`declaration_coverage` distinguishes complete, partial and unknown observations.
|
||||
Historical Evidence omits newly introduced fields and defaults coverage to
|
||||
`unknown`; omission never establishes absence. Current records contain the fields,
|
||||
with null for absent or unresolved scalar declarations. The inventory retains at
|
||||
most 32 filters and 64 nested action/category/data/metadata records per component,
|
||||
and shares a 1 MiB structured-value copy budget across components. Omitted values
|
||||
or unsupported filter constructs mark coverage partial and include a limitation;
|
||||
complete retained manifest text remains accessible.
|
||||
|
||||
For a small answer, select `detail: "summary"` with an explicit view:
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": "android",
|
||||
"facet": "components",
|
||||
"actions": ["android.provider.Telephony.SMS_RECEIVED"],
|
||||
"fields": ["name", "permission", "enabled", "process", "intent_filters"],
|
||||
"offset": 0,
|
||||
"limit": 16
|
||||
}
|
||||
```
|
||||
|
||||
The same JSON works with CLI `--view` or retained `inspect_analysis_view`.
|
||||
`kinds`, `names` and `actions` use exact matching; names accept either the recorded
|
||||
declared or resolved name, actions match any requested action, and different
|
||||
filters combine with AND. `exported` matches only explicit declarations, keeping
|
||||
`exported_default` separate. Empty selectors match nothing. Every selected row
|
||||
retains its parent-array `source_index` and `declaration_coverage`; source coverage,
|
||||
parent identity and limitations remain alongside the page. Follow
|
||||
`coverage.next_offset`. The 1 MiB projection budget can shorten a page; a single
|
||||
oversized row requires fewer selected fields. Exhausting a partial source does
|
||||
not establish complete manifest coverage. `facet: "component-summary"` applies
|
||||
the same filters and counts kinds, declaration coverage and explicit/default
|
||||
exported values separately. Both adapters use the same retained projection;
|
||||
repeated pages do not reload JADX.
|
||||
|
||||
The service `foreground_service_type` field preserves the declared attribute,
|
||||
including an empty value or surrounding spaces; `null` means the attribute is
|
||||
absent or the component is not a service. This field does not validate service
|
||||
@@ -252,35 +194,6 @@ queries accept the same `overload_index`; the bridge resolves the selected
|
||||
method node and the result reports its index and displayed signature. These
|
||||
indices are tied to the current artifact and engine, not stable across builds.
|
||||
|
||||
Method reference queries also accept an external class, such as
|
||||
`android.app.NotificationManager --method-name notify`, when the APK directly
|
||||
invokes that exact DEX class and method name. If multiple prototypes are observed,
|
||||
the input error lists their exact DEX descriptors and local overload indices;
|
||||
repeat with an explicit `--overload-index`. These are the prototypes observed in
|
||||
the APK, not the complete method inventory of an Android framework or library.
|
||||
|
||||
Such results report `target_definition: "external"` and
|
||||
`reference_scope: "direct_dex_invokes"`. Each using method has `source_selector`
|
||||
with `class_name`, `method_name` and `overload_index`; pass those fields with the
|
||||
same APK path to `inspect_android_method` to inspect the caller. Repeated direct
|
||||
invokes in one method remain one using node with multiple instruction offsets.
|
||||
This scans raw invoke instructions in JADX's admitted APK methods without
|
||||
generating their source. It does not resolve reflection, JNI, method handles,
|
||||
inherited method lookup or possible runtime dispatch. Class-only queries and
|
||||
class/method inspection still require an APK definition; an external target has
|
||||
no invented body. Unknown class/method names remain input errors.
|
||||
|
||||
External prototype and caller metadata use a bridge-frame-derived representation
|
||||
allowance that reserves space for retained raw calls and CLI/MCP copies. Exhausting
|
||||
it returns a typed `resource_constraint`, rather than a truncated caller list
|
||||
reported as complete. Direct instruction offsets have their separate explicit
|
||||
`direct_invokes_exhaustive` coverage. Older retained Evidence may omit the new
|
||||
definition, scope and caller-selector fields; omission does not establish them.
|
||||
The bridge checks prototype expansion before copying descriptors or joining
|
||||
displayed signatures. It reuses one bounded prototype inventory for the exact
|
||||
class/method selection in the current APK session; replacing the session or
|
||||
selection invalidates that retained result.
|
||||
|
||||
`inspect_android_class` also reports `dex_static_value` for each `static final`
|
||||
field whose class definition encodes a value in its DEX static values, so
|
||||
compile-time constants are readable even when the class has no `<clinit>`.
|
||||
@@ -317,162 +230,6 @@ instructions into field declarations, leaving an empty or incomplete method
|
||||
body. The selected method's disassembly preserves those instructions and retains
|
||||
the same source byte limit and explicit truncation reporting.
|
||||
|
||||
## Apply an R8/ProGuard mapping
|
||||
|
||||
Release builds shrunk by R8 or ProGuard rename classes and members. Select the
|
||||
build's `mapping.txt` with `mapping_path` on `inspect_android_class` or
|
||||
`inspect_android_method`, or `--mapping-path` on the matching CLI command:
|
||||
|
||||
```sh
|
||||
rea inspect-android-class /path/Example.apk a.b --mapping-path /path/mapping.txt \
|
||||
--detail summary --evidence_output /tmp/class-evidence.json
|
||||
```
|
||||
|
||||
For agent workflows, select `detail: "summary"` on class and method inspection.
|
||||
It preserves the complete normalized member inventory or selected method body,
|
||||
including mapping annotations, while retaining duplicate raw provider calls in
|
||||
the complete parent Evidence. The reply links that parent's identity and digest;
|
||||
export it with `export_evidence_bundle` or read its summary again through
|
||||
`inspect_analysis_view`. CLI summaries require `--evidence_output` and return
|
||||
the portable bundle path. Omitted or `complete` detail preserves complete inline
|
||||
Evidence. Search and reference operations retain their existing contracts.
|
||||
Symbol summary views admit at most **1 MiB**, including view context. An admitted
|
||||
method's source can already consume that amount; a larger combined summary
|
||||
fails explicitly instead of silently truncating facts. MCP refusal includes
|
||||
the retained parent ID. Select complete detail, or inspect one method rather
|
||||
than a whole class, when the requested summary does not fit.
|
||||
|
||||
The mapping path must be absolute. Selectors still take the observed names, such as
|
||||
`a.b`, and observed rows are unchanged. The result adds one `mapping` block with
|
||||
the file's path, SHA-256 and size, the header values the file reports
|
||||
(`compiler`, `compiler_version`, `min_api`, `map_version`, `pg_map_id`,
|
||||
`pg_map_hash`), valid record counts and up to 16 malformed line samples. Its
|
||||
`symbols` list has one entry per observed class, inner class, method and field;
|
||||
`inspect_android_method` lists the class and the selected method. `coverage`
|
||||
counts each resolution status. With summary delivery, these unchanged facts are
|
||||
under the view's `summary` field; the view separately reports source coverage.
|
||||
|
||||
Lookups use the bridge's raw DEX names, not JADX display aliases such as a
|
||||
renamed reserved word. Each symbol resolves as:
|
||||
|
||||
- `mapped`: exactly one record matches. A class needs one record for its raw
|
||||
name (`verified_by: "class"`). A method must also match its DEX prototype
|
||||
(`"descriptor"`) and a field its DEX type (`"type"`). R8 residual signature
|
||||
metadata is used when present; otherwise original types are translated
|
||||
through the file's class records.
|
||||
- `ambiguous`: `multiple-records`, `descriptor-mismatch`, `type-mismatch`,
|
||||
`ambiguous-class` or `residual-signature-conflict`, with `candidate_count` and
|
||||
up to 8 candidates. Conflicting residual metadata never selects a winner. A name match
|
||||
whose prototype or type does not verify is never reported as mapped.
|
||||
- `unmapped`: no valid record matched the raw identity.
|
||||
- `unknown` with reason `projection-budget`: this claim did not fit the combined
|
||||
reply budget. The observed raw symbol remains present; `coverage.unknown`
|
||||
counts these omissions. Inspect that specific class or method separately,
|
||||
or consult the selected local mapping file for complete source records.
|
||||
|
||||
A line continues an inline frame stack when it repeats the previous line's
|
||||
minified range and obfuscated name and gives a single call-site line, such as
|
||||
`:7`. Only the outermost original of a stack becomes a candidate. Original
|
||||
line numbers and source files are not projected. Malformed records are counted
|
||||
and skipped, so an `unmapped` symbol might be described by one of them. A file
|
||||
that is missing, not a regular file, not UTF-8, larger than the byte limit or
|
||||
without any class record, such as `usage.txt` or `seeds.txt`, is refused before
|
||||
JADX is used. Requests read their mappings one at a time, in the same queue as
|
||||
the JADX work and under the same operation deadline. A mapping failure therefore never starts or retires the JADX
|
||||
session, and a different mapping reuses the loaded APK.
|
||||
|
||||
## Link native declarations to JNI candidates
|
||||
|
||||
When advertised, `inspect_android_jni_bindings` links original `ACC_NATIVE`
|
||||
declarations from standard root `classes*.dex` members to bundled ELF libraries.
|
||||
The supplied JAR's DEX reader preserves duplicate declarations before decompiler
|
||||
deduplication. This operation does not load the APK into the decompiler.
|
||||
MCP defaults to retained summary delivery when `detail` is omitted; select
|
||||
`complete` explicitly to receive the full report inline.
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/absolute/Example.apk",
|
||||
"class_name": "org.conscrypt.NativeCrypto",
|
||||
"method_names": ["EVP_PKEY_new_RSA"],
|
||||
"abis": ["arm64-v8a"],
|
||||
"detail": "summary"
|
||||
}
|
||||
```
|
||||
|
||||
Class/method names and optional `descriptors` select raw DEX identities.
|
||||
`library_paths` selects exact logical APK paths such as
|
||||
`lib/arm64-v8a/libconscrypt_jni.so`; each library's `source_path` preserves its
|
||||
original ZIP spelling. Omitted filters include all scoped facts; `[]` matches
|
||||
none. Class selection never narrows the global inventory used for table-class
|
||||
attribution.
|
||||
|
||||
Independent `declarations`, `libraries`, `candidates` and `bindings` collections
|
||||
have stable IDs and source ordinals. Candidates retain APK/library digests,
|
||||
ABI, function addresses and file offsets. Table candidates also retain section,
|
||||
table and entry identity plus all three name/signature/function pointer slots
|
||||
and their relocation sources. JNI exports follow the
|
||||
[UTF-16 escaping and short-before-long rules](https://docs.oracle.com/en/java/javase/21/docs/specs/jni/design.html#resolving-native-method-names);
|
||||
failed escaping does not acquire an invented export match. Tables require
|
||||
NUL-terminated [Modified UTF-8 strings](https://docs.oracle.com/en/java/javase/21/docs/specs/jni/types.html#modified-utf-8-strings),
|
||||
valid descriptors and executable load mappings. REL, RELA and RELR retain
|
||||
separate provenance; unsupported packed relocations and unresolved pointer
|
||||
spans stay explicit unknowns.
|
||||
|
||||
A static `JNINativeMethod` entry names no class. Attribute it uniquely only
|
||||
when exactly one declaration in the complete recorded native inventory has
|
||||
that `(name, descriptor)` pair. Multiple classes remain ambiguous after a
|
||||
class filter; a partial inventory cannot establish unique attribution.
|
||||
|
||||
Each selected declaration has a row per selected/observed ABI. `matched` means
|
||||
candidate presence was observed; `ambiguous` means multiple observed targets
|
||||
or class associations remain. Binding `coverage` is separate: a matched
|
||||
partial row does not establish complete uniqueness, and an empty partial scan
|
||||
is `unknown`, never `unmatched`. ELF/ABI label mismatches preserve facts without
|
||||
asserting a binding. No result establishes library loading, runtime registration
|
||||
or method invocation.
|
||||
|
||||
Summary returns the first 64 selected binding rows, their declaration/candidate
|
||||
closure and every selected library's identity, coverage and failure reason;
|
||||
counts and page coverage describe the full selected binding set. Selected
|
||||
declaration IDs have separate preview coverage, so native declarations remain
|
||||
visible even when the APK has no bundled ABI libraries. Complete
|
||||
Evidence stays retained. Read exact fields or further pages without reopening
|
||||
files or rerunning analysis:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": { "kind": "retained-evidence", "evidence_id": "ev_<digest>" },
|
||||
"view": {
|
||||
"kind": "android-jni",
|
||||
"collection": "bindings",
|
||||
"offset": 64,
|
||||
"limit": 64,
|
||||
"fields": ["declaration_id", "abi", "status", "coverage", "candidate_ids"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Collections also include `declarations`, `candidates` and `libraries`, with
|
||||
exact filters and batch lookup by declaration/candidate/library IDs. `row_index`
|
||||
is the original parent collection index; DEX and
|
||||
ZIP source ordinals stay separate. Complete detail returns the full report.
|
||||
CLI uses the same selection and delivery contracts. Its common one-shot
|
||||
default remains complete; summary delivery requires an explicit Evidence path:
|
||||
|
||||
```sh
|
||||
rea inspect-android-jni-bindings /absolute/Example.apk \
|
||||
--query '{"class_name":"org.conscrypt.NativeCrypto","method_names":["EVP_PKEY_new_RSA"],"abis":["arm64-v8a"]}' \
|
||||
--detail summary --evidence_output /absolute/fresh-jni-evidence.json
|
||||
```
|
||||
|
||||
Optional `--view` accepts the same JSON page and requires summary delivery.
|
||||
Evidence bundles never overwrite files. Complete declarations and one library
|
||||
selection are cached within the immutable APK session. Changing ABI/library
|
||||
selection replaces the scan; class/method selection reuses it. Fresh analysis
|
||||
still validates current caller APK bytes. Idle/close releases the caches;
|
||||
retained Evidence remains historical and independently readable.
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- Package inspection retains decoded manifest XML, summary declarations,
|
||||
@@ -509,11 +266,6 @@ retained Evidence remains historical and independently readable.
|
||||
`dex_descriptor`, for example `(I)V`. Display names can be JADX aliases;
|
||||
the descriptor alone is not a complete raw DEX method ID. Overload indices
|
||||
are tied to this artifact and engine, not stable identities across builds.
|
||||
- A `mapping` block appears only when the caller selected `mapping_path`. Its
|
||||
names are claims from that file, not observations of the APK. REA does not
|
||||
check that the mapping belongs to this build; `pg_map_id` is reported, not
|
||||
compared. Evidence keeps the selected path in its parameters and the file's
|
||||
SHA-256 in the result.
|
||||
- Method inspection reports Java or smali, fallback markers and complete/partial
|
||||
returned text. Decompiled source is a derived representation. A complete text
|
||||
result does not establish complete semantics or a byte-identical reconstruction.
|
||||
@@ -603,19 +355,6 @@ target/configuration changes, failure, timeout, cancellation, MCP disconnect or
|
||||
server shutdown. One-shot CLI commands always join cleanup before returning.
|
||||
REA never writes to the original APK or launches target code.
|
||||
|
||||
JNI scans reuse the analyzer process without loading the APK into the
|
||||
decompiler. They admit at most 64 MiB per DEX member and 256 MiB of DEX bytes,
|
||||
with a 4 MiB native-declaration JSON allowance before MCP string escaping.
|
||||
Native libraries are processed sequentially: one 64 MiB buffer, 256 MiB total
|
||||
selected read work, and a 16 MiB parsed-library-facts allowance. ZIP inventory
|
||||
and library counts also have explicit resource bounds. Unexamined occurrences
|
||||
remain unknown. A complete relationship report has a separate 16 MiB serialized
|
||||
allowance; summaries/pages allow 1 MiB including their provenance envelope.
|
||||
These are resource admissions, not claims about maximum valid Android formats.
|
||||
Select fewer ABI/library occurrences or smaller retained pages when needed.
|
||||
Completed parsed library facts survive failed reader finalization in typed
|
||||
partial observations; the exact reader owner remains available for cleanup retry.
|
||||
|
||||
On Windows, stdin writes preserve stream backpressure and run outside the Node
|
||||
event loop. Cancellation terminates the retained job, including descendants,
|
||||
before workspace cleanup. On Windows, a detached REA helper owns the private
|
||||
@@ -701,27 +440,6 @@ complete, case-sensitive search contract. A protocol frame above **8 MiB**, or c
|
||||
**32 MiB per operation**, fails with a diagnostic instead of returning an incomplete inventory
|
||||
as complete. These are memory-safety boundaries, not undocumented result caps.
|
||||
|
||||
A selected R8/ProGuard mapping is re-read and SHA-256 checked on every call, up
|
||||
to **64 MiB**. The serialized provider keeps one successful parsed index, keyed
|
||||
by byte length and digest with fixed parser options. Identical bytes reuse that
|
||||
index; changed bytes are reparsed, missing files still fail, and each response
|
||||
preserves the currently selected path. The index is released on provider idle
|
||||
retirement or close. Omitting `mapping_path` never applies the cached mapping.
|
||||
During replacement, the old index can coexist with the new parse until it
|
||||
succeeds; these transient allocations count toward process memory.
|
||||
|
||||
Before retaining records, the parser admits an estimated **128 MiB** index
|
||||
representation and **8 MiB** of expanded mapped claims per residual class,
|
||||
including shared header facts. The index allowance accounts for strings and
|
||||
containers; it is not a measured V8 heap cap. Parameter and metadata expansion
|
||||
is admitted before splitting or JSON decoding. These fixed resource bounds
|
||||
have no runtime override; refusal reports `resource_constraint`, the source
|
||||
line and mapping SHA-256. Ambiguous lookups count all candidates while keeping
|
||||
only the first eight in stable file order. A separate **8 MiB** combined claim
|
||||
budget applies to the actual query set, including outer and inner classes,
|
||||
before cloning the report. Claims that do not fit become explicit `unknown`
|
||||
entries; other admitted claims and all observed identities remain available.
|
||||
|
||||
## Real verification
|
||||
|
||||
Download the explicit public fixtures once; this installs no tools:
|
||||
@@ -765,7 +483,3 @@ containing node's unknown `dex_offset`. `direct_invokes_exhaustive: false`
|
||||
means instruction rows were bounded by the local bridge frame budget; `null`
|
||||
means the producer did not establish instruction coverage. Using-node totals
|
||||
and coverage do not imply complete instruction evidence.
|
||||
`direct_invokes_coverage` reports counts of using nodes with complete, partial,
|
||||
or unknown positions, plus the number of retained instruction sites. This
|
||||
aggregate makes position truncation visible without scanning every caller;
|
||||
older retained Evidence may omit it, which means the aggregate was unexamined.
|
||||
+12
-24
@@ -179,31 +179,26 @@ share the application workflow, projection, retention and publication owners.
|
||||
- `schemes_agree_on_signer` (observed): every present block scheme presents
|
||||
the same _set_ of signer certificates. An extra signer in one scheme is
|
||||
disagreement; a signer without a certificate makes it `null` (unknown).
|
||||
A single present block scheme with known signers agrees with itself;
|
||||
v1 is compared separately below.
|
||||
Build-tools 33+ rotate through v3.1 by default, so a rotated app usually
|
||||
reads `false` here and `true` below.
|
||||
- `signer_change_declared_by_rotation` (declared): the signers differ, but
|
||||
each appears in a lineage the APK declares. Read the two together: block
|
||||
schemes with different signers covered by a declared lineage show `false`
|
||||
/ `true`; differing signers without such a lineage show `false` / `false`.
|
||||
A lineage alone does not establish a change between block schemes. It is
|
||||
`null` when a signer's identity is unknown, or when the signers differ and
|
||||
a declared lineage could not be decoded.
|
||||
each appears in a lineage the APK declares. Read the two together: a
|
||||
rotated APK shows `false` / `true`; signers that differ with no declared
|
||||
lineage containing them show `false` / `false`. It is `null` when a
|
||||
signer's identity is unknown, or when the signers differ and a declared
|
||||
lineage could not be decoded.
|
||||
- `v1_signers_match_block_signers` (observed): the v1 signer certificates
|
||||
equal what the lowest block scheme that has signers (v2, else v3, else
|
||||
v3.1) expects v1 to carry. That is its signer set, except that a v3/v3.1
|
||||
signer declaring a rotation lineage expects the lineage's first
|
||||
(original) certificate, as apksig requires. `false` is notable: the JAR
|
||||
signature names a different key than the block signature. When v1 uses
|
||||
the lineage's original key and v3 its newer key, this comparison reads
|
||||
`true`. If v3 is the only present block scheme, agreement is `true` and
|
||||
declared inter-block signer change is `false` even though rotation is
|
||||
present. If v2 also uses the original key, agreement is `false` and the
|
||||
declared inter-block change is `true`. `null` when
|
||||
v1 or a populated block scheme is absent, any signer's identity is unknown (a
|
||||
signature names a different key than the block signature. A key rotation
|
||||
keeps v1 (and v2) on the original key, so it reads `true` while
|
||||
`schemes_agree_on_signer` reads `false`, with or without v2. `null` when
|
||||
v1 or a block scheme is absent, any signer's identity is unknown (a
|
||||
`decode_error`, a `null` certificate or a `subjectKeyIdentifier`
|
||||
`SignerInfo`), or the lineage's original certificate cannot be
|
||||
identified. When identities or a lineage origin are unknown, limitations
|
||||
explain why; absence is shown in the scheme records.
|
||||
identified; a limitation then says why.
|
||||
- `strongest_scheme`, `rotation_present`, `source_stamp_present`,
|
||||
`self_consistent`.
|
||||
|
||||
@@ -244,13 +239,6 @@ serialized size of the report together with its limitations (which repeat
|
||||
archive names) is then checked against an 8 MiB output budget (the exact check
|
||||
is authoritative); an over-budget report is refused, not truncated.
|
||||
|
||||
Signing blocks admit at most 4096 ID/value pairs; at most 4096 signing-relevant
|
||||
META-INF entries are retained. These fixed counts, signer/lineage counts and the
|
||||
report byte budget return nonretryable `resource_constraint` errors naming the
|
||||
effective allowance. This release has no runtime override for them. Waiting for
|
||||
an active scan cannot raise a fixed count, and complete CLI JSON cannot raise the
|
||||
report budget. Occupied scan-queue capacity remains retryable after a scan finishes.
|
||||
|
||||
The whole-file reader allocates the selected snapshot and bounded read scratch;
|
||||
total process memory exceeds the selected file's size. It also computes the complete
|
||||
file SHA-256, so inspection is not header-only. Content digests are recomputed
|
||||
|
||||
@@ -100,13 +100,12 @@ still suppresses all string projection, including in this mode.
|
||||
- **Metadata comes from apktool.yml as apktool printed it.** Version codes
|
||||
keep their own spelling (`'7'` versus `7`); `package_name` is read from
|
||||
the decoded manifest, not inferred.
|
||||
The `version: 2.9.0`-or-later header that Apktool 2.9+ and 3 emit selects
|
||||
their single-line string dialect: literal hashes, single quotes, empty
|
||||
strings, and the text `null` stay part of the value; producer backslashes
|
||||
and Unicode escapes decode once. Older/tagged documents retain YAML scalar
|
||||
semantics; double-quoted values also retain YAML string semantics.
|
||||
Apktool 3's emitted `version: 3.x.y` header selects its single-line string
|
||||
dialect: literal hashes, single quotes, empty strings, and the text `null`
|
||||
stay part of the value; producer backslashes and Unicode escapes decode
|
||||
once. Older/tagged documents retain YAML scalar semantics; double-quoted
|
||||
values also retain YAML string semantics.
|
||||
This distinction follows the released
|
||||
[2.9.0 metadata writer](https://github.com/iBotPeaches/Apktool/blob/b64e257c9b689eb9630ad0241f2fe4f4a749955b/brut.apktool/apktool-lib/src/main/java/brut/androlib/apk/VersionInfo.java#L38-L42),
|
||||
[3.0.3 metadata writer](https://github.com/iBotPeaches/Apktool/blob/18b5e99cb56ff9451e8aa55b065dcf5bbd616975/brut.apktool/apktool-lib/src/main/java/brut/androlib/meta/VersionInfo.java#L49-L55)
|
||||
and [string escape owner](https://github.com/iBotPeaches/Apktool/blob/18b5e99cb56ff9451e8aa55b065dcf5bbd616975/brut.j.yaml/src/main/java/brut/yaml/YamlStringEscapeUtils.java#L55-L116).
|
||||
- **The manifest is the decoded text**, bounded at 512 KiB and reported as
|
||||
|
||||
@@ -101,8 +101,8 @@ limitation reports how many inventoried entries were left out. IPA projection
|
||||
still lists every archive component, including `SwiftSupport/`.
|
||||
|
||||
`platforms` lists `macos` for application roots with `Contents/`. It lists `ios`
|
||||
for shallow application roots that have an `Info.plist` at their root. An
|
||||
application root with neither, such as an empty inventory, has no platform.
|
||||
for shallow application roots that have an `Info.plist`. An application root
|
||||
with neither, such as an empty inventory, has no platform.
|
||||
|
||||
## macOS bundle anatomy
|
||||
|
||||
@@ -149,11 +149,6 @@ directories rather than `Versions/Current`. A framework's `info_plist_path` is
|
||||
because `Versions/Current` decides which one applies) or when its single
|
||||
version directory has no `Resources/Info.plist`.
|
||||
|
||||
A `shallow` bundle's `info_plist_path` is its root `Info.plist`. Without one,
|
||||
it is `Resources/Info.plist`: a flattened macOS framework keeps its binary,
|
||||
`_CodeSignature/`, and `Resources/` directly in the `.framework`, like a single
|
||||
`Versions/<version>/` directory without the `Versions/` wrapper.
|
||||
|
||||
AppleDouble sidecar files (`._*` and `__MACOSX/`) are inventory facts, but they
|
||||
describe neighbouring files. They are excluded from roots, bundle roles, and the
|
||||
anatomy components.
|
||||
|
||||
@@ -8,7 +8,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Production Composition"
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>typed capability bindings: construction, availability and retryable lifetimes<br/>pure capability declarations feed provider catalogs"]
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>static WASM/EVM descriptors: named contracts, factories, retryable lifetimes"]
|
||||
end
|
||||
|
||||
subgraph "Adapters"
|
||||
@@ -21,15 +21,14 @@ graph TD
|
||||
subgraph "Application Layer"
|
||||
APP["src/application/<br/>Shared CLI/MCP workflows<br/>enhanced analysis and investigations"]
|
||||
SESSION["SessionProviderRouter + BinarySession<br/>one immutable deep binding per target<br/>disjoint auxiliary routing"]
|
||||
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback<br/>ProviderInspectionScope: prerequisite observation lifetime"]
|
||||
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback"]
|
||||
RECORDS["src/application/investigation/<br/>composed InvestigationRecords + narrow ports<br/>EvidenceLedger and Unknown ownership"]
|
||||
LIFETIMES["CapabilityScope<br/>host-owned service close capabilities<br/>released peers removed; failed cleanup retained for retry"]
|
||||
SNAPSHOT["BinarySessionRecords + AnalysisSnapshotCache<br/>binary target/profile snapshots and session file ownership<br/>record coordination and post-commit notifications"]
|
||||
JS_RECON["src/application/javascript/ + src/artifacts/javascript/<br/>JavaScript artifact reconstruction<br/>directory/ASAR reads<br/>AST-only bundle projection"]
|
||||
end
|
||||
|
||||
subgraph "Provider Layers (selected adapters)"
|
||||
PROVIDER_OWNERS["src/providers/<br/>shared deferred client owners<br/>exact cleanup retry and attributed failures"]
|
||||
PROVIDER_OWNERS["src/providers/<br/>shared deferred client and inspection owners<br/>exact cleanup retry and attributed failures"]
|
||||
PWNTOOLS["src/native/pwntools/ + src/native/pwndbg/<br/>BYO offline ELF/core adapters<br/>owned snapshots/Python and optional core-only GDB<br/>unchanged upstream parsers, ABI structures and mapping API"]
|
||||
WASM["src/wasm/<br/>caller-supplied WABT; immutable artifact inspection<br/>owned process supervision and retryable workspace cleanup"]
|
||||
APKSIG["src/apkSignature/<br/>in-process APK Signing Block + v1 reader<br/>bounded single read, no device or execution"]
|
||||
@@ -76,7 +75,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Domain Layer"
|
||||
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>format declarations, provider selection, Evidence, comparisons, application graphs"]
|
||||
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>format declarations, Evidence, comparisons, application graphs"]
|
||||
end
|
||||
|
||||
subgraph "Configuration"
|
||||
@@ -100,7 +99,6 @@ graph TD
|
||||
MAIN --> COMPOSITION
|
||||
CLI --> COMPOSITION
|
||||
SERVER --> COMPOSITION
|
||||
COMPOSITION --> LIFETIMES
|
||||
COMPOSITION --> SESSION
|
||||
APP --> FIRMWARE
|
||||
FIRMWARE --> PROCESS
|
||||
@@ -151,7 +149,8 @@ graph TD
|
||||
OBJDUMP --> DOMAIN
|
||||
COMPOSITION --> RIZIN
|
||||
RIZIN --> DOMAIN
|
||||
COMPOSITION --> GDB
|
||||
CLI --> GDB
|
||||
SERVER --> GDB
|
||||
GDB --> PROCESS
|
||||
GDB --> DOMAIN
|
||||
HOPPER --> HOPPER_BRIDGE
|
||||
@@ -207,7 +206,8 @@ graph TD
|
||||
NATIVE_READERS --> DOMAIN
|
||||
COMPOSITION --> FORMAT_READERS
|
||||
FORMAT_READERS --> DOMAIN
|
||||
COMPOSITION --> RUNTIME
|
||||
SERVER --> RUNTIME
|
||||
CLI --> RUNTIME
|
||||
RUNTIME --> DOMAIN
|
||||
PROCESS --> FILESYSTEM
|
||||
AUX --> FILESYSTEM
|
||||
|
||||
@@ -9,11 +9,9 @@ ordinary binary inspection use the same modular capability.
|
||||
## Bring your own engine
|
||||
|
||||
Real verification covers **Linux x64** hosts and **EXEC/DYN/REL** objects in
|
||||
these profiles: ELF64 x86-64, ELF64 little-endian AArch64, ELF32 i386, ELF32
|
||||
little-endian ARM and ELF32 little- or big-endian MIPS. Other machines, classes
|
||||
and byte orders, including big-endian ARM, return `unsupported_target`. Other
|
||||
hosts, including Linux arm64, return `capability_unavailable` naming the
|
||||
detected host before any target is read. Provide an absolute
|
||||
these profiles: ELF64 x86-64, ELF32 i386, ELF32 little-endian ARM and ELF32
|
||||
little- or big-endian MIPS. Other machines, classes and byte orders, including
|
||||
AArch64 and big-endian ARM, return `unsupported_target`. Provide an absolute
|
||||
Python executable whose environment already contains unchanged pwntools 4.15.0,
|
||||
pyelftools 0.33 and Unicorn 2.1.2:
|
||||
|
||||
@@ -426,10 +424,8 @@ rea inspect-native-exports ./libexample.so --detail summary --evidence_output ./
|
||||
without an export directory is a complete empty inventory.
|
||||
- ELF rows list defined `.dynsym` symbols with global, weak or GNU-unique
|
||||
binding and default or protected visibility. Each row keeps `value`
|
||||
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`,
|
||||
`binding` and `visibility` (`"default"` or `"protected"`; `"unknown"` in older
|
||||
historical Evidence where visibility was not recorded). ELF string tables do
|
||||
not declare a text encoding. Names that
|
||||
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`
|
||||
and `binding`. ELF string tables do not declare a text encoding. Names that
|
||||
contain non-ASCII bytes are decoded as UTF-8 when those bytes form valid
|
||||
UTF-8; `name_utf8_decodable` records whether the non-ASCII name bytes are
|
||||
valid UTF-8 and `name_bytes_base64` retains those exact bytes (`null` for
|
||||
|
||||
@@ -52,11 +52,7 @@ observations may omit the coverage field; treat omission as `unknown` rather
|
||||
than inferring completeness from the array.
|
||||
|
||||
Passive DOM comparison checks complete node inventories and complete frame
|
||||
parent/child structure independently. Node value lengths are comparable only
|
||||
when both nonempty snapshots declare the same `node_value_length_unit`.
|
||||
Missing or different units leave length comparison unknown; independent node
|
||||
or frame structural differences can still prove change. Empty inventories have
|
||||
no value lengths to compare. Frame IDs and sibling enumeration order
|
||||
parent/child structure independently. Frame IDs and sibling enumeration order
|
||||
are capture-local and do not establish a change; duplicate IDs, missing parents,
|
||||
and cyclic ancestry remain unknown. A complete frame difference can still prove
|
||||
change when DOM nodes are incomplete, and vice versa. Partial DOM or metadata
|
||||
@@ -404,19 +400,7 @@ contextual evidence with `timestamp: null`, accompanied by a limitation stating
|
||||
that it does not establish activity in the selected window. Main-frame scope
|
||||
enforcement still applies to contextual navigation events. Network redirect
|
||||
events retain their paired prior-hop response context when the timestamped
|
||||
next-request event falls inside the window. A redirect row's optional
|
||||
`redirect_response` contains the prior response `url`, CDP-reported `status`,
|
||||
and `scope`. The URL has transport credentials removed; URL and status are
|
||||
withheld unless its origin is approved. Missing or malformed response URLs have
|
||||
null scope and add `invalid_protocol_value` timeline coverage; a valid URL with an
|
||||
unsupported protocol has `unsupported` scope. Outside-policy and unsupported
|
||||
responses mark the timeline as policy-filtered with `disallowed_origin` or
|
||||
`unsupported_url`. Missing status remains null rather than zero. A supplied
|
||||
non-numeric status is also null and records `invalid_protocol_value` coverage;
|
||||
the other valid response facts remain available. Historical rows may omit this
|
||||
field; omission does not establish that no prior response existed. Response
|
||||
headers and bodies are discarded. The row timestamp belongs to the next request,
|
||||
not an exact prior response time. The pre-arm timeline buffer is bounded
|
||||
next-request event falls inside the window. The pre-arm timeline buffer is bounded
|
||||
by original CDP message bytes. If clock sampling completes after that budget is
|
||||
exhausted, the operation returns `payload_limit` instead of a partial window.
|
||||
Cancellation and clock-request failures retain their own outcomes. Session
|
||||
@@ -431,8 +415,7 @@ allowance; exhaustion fails the observation explicitly instead of guessing scope
|
||||
Response metadata parses repeated CDP header field lines independently. The
|
||||
Link projection retains their declarations in order. `csp.directives` lists the
|
||||
directives of every CSP policy in order, across field lines and comma-separated
|
||||
policies, without recording policy boundaries, so a directive name can repeat
|
||||
across policies. As in CSP3, a repeated name within one policy is ignored.
|
||||
policies, without recording policy boundaries, so a directive name can repeat.
|
||||
Single-valued policy fields combine their lines as one comma-separated value,
|
||||
as RFC 9110 specifies. Each link keeps its target in `href` and
|
||||
its relationship context in `context_url` with a separate `context_scope`;
|
||||
@@ -465,7 +448,7 @@ not bound the whole process heap.
|
||||
REA retains local observation data and removes only URL userinfo credentials, explicitly declared scenario secrets, and values from structured credential/storage fields:
|
||||
|
||||
- URL observations preserve the complete local URL, including query values, fragments, duplicate parameters, and their order. Only username/password userinfo is removed; malformed original URL strings remain available with `origin: null`.
|
||||
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values. `dom.node_value_length_unit` is `utf16-code-units` for web observations and applies to every node's `node_value_length`; it counts JavaScript string code units, not Unicode code points or UTF-8 bytes. Electron observations declare `utf8-bytes` instead. Historical captures without the unit leave it unknown; reading them does not infer a unit or convert their stored lengths.
|
||||
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values.
|
||||
- Supported DOM URL attributes are projected separately as resolved destinations. Empty `action` on HTML or XHTML forms and empty `formaction` on HTML or XHTML submit controls resolve to the document URL; other relative URL attributes resolve against the captured document base.
|
||||
- Accessibility structure and reported roles are retained by default, while names and descriptions require `include_accessibility_text: true`. The browser protocol may omit an individual role; that role remains unknown for comparison. Other observed accessibility differences can still establish a change.
|
||||
- Network observations retain method, status, MIME type, size, type, initiator stack location, and complete URLs with userinfo credentials removed. Redirects reported by CDP are attached as ordered `redirects` on the one request ID, with the prior request URL, CDP response URL, method/type, response status/MIME/encoded length, request timestamp, and the next `requestWillBeSent` event timestamp. CDP delivers `redirectResponse` with that next request event, so the latter is an event boundary rather than an exact response timestamp. Redirect response headers and bodies are discarded; hops outside the selected origins are discarded with the request chain. Headers are discarded after an allowlisted projection of length/encoding, structured CSP/Link/policy fields, and untrusted agent hints. Cookies and authorization headers are never retained.
|
||||
@@ -500,7 +483,7 @@ REA retains local observation data and removes only URL userinfo credentials, ex
|
||||
connection/frame inventories; matching direction/opcode sequences alone do
|
||||
not prove alignment after drops. Redirect resource types use only shared known metadata:
|
||||
an omitted type leaves equality unknown rather than proving a modification.
|
||||
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. Source locations are CDP's zero-based lines and UTF-16 columns. A negative column, which Chrome reports on the first line of some inline scripts, is kept as reported and does not locate a column in the source URL; an invalid line or a non-integer column is `null` (WebMCP `registration_source` is `null`) and is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
|
||||
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
|
||||
- WebSocket observations retain direction, opcode, and payload byte length. A missing or malformed CDP opcode is returned as `null`, with opcode-dependent payload byte length and shape also `null`, and marks `websocket_frames` incomplete. Missing or wrong-typed payload data is not treated as empty; its byte length is `null`, and a text payload shape is unknown. Numeric opcode `0` and an observed empty payload remain distinct from missing data. Opcode is nullable in the public inspection schema, so clients must handle unknown protocol values. When shape capture is selected, text messages are classified as text or complete value-free JSON shape; binary bytes, hashes, prefixes, and raw payloads are never retained.
|
||||
- Storage observations always redact values. Key names, IndexedDB names, and cache names require `include_storage_keys`; stable content fingerprints require the additional `include_storage_fingerprints` selection. Cookie fingerprints cover cookies applicable to the current main-frame URL, including its path, rather than every cookie path on that origin. Other storage remains scoped to the origin. Cache fingerprints hash all decoded response bytes, including bodies above 64 KiB, subject to the [64 MiB per-message CDP limit](#request-boundary). Missing or undecodable cache bodies and partial IndexedDB remote objects make fingerprint coverage incomplete, so identical observations remain `unknown`.
|
||||
- Unavailable or malformed storage inventories mark `storage_keys` unavailable, even without fingerprints. Comparisons suppress inferred key additions and removals while preserving modifications proven by complete fingerprints on both sides. Previously exported captures are not retroactively corrected; recapture them to obtain these coverage markers.
|
||||
|
||||
@@ -51,30 +51,6 @@ covers endpoint discovery only. The following WebSocket handshake and
|
||||
`Target.attachToTarget` each use `command_timeout_ms`. `capture.limits` controls retained event and network-content
|
||||
budgets; it does not change those liveness deadlines.
|
||||
|
||||
## Validate a scenario before running it
|
||||
|
||||
`validate-browser-scenario INPUT_JSON` checks one scenario against the same
|
||||
`browserScenarioSchema` refinement that `capture_browser_scenario` admits, and
|
||||
stops there. `INPUT_JSON` may be inline JSON, a JSON file path, or `-` for
|
||||
standard input.
|
||||
|
||||
```bash
|
||||
rea validate-browser-scenario ./browser-scenario.json --json
|
||||
```
|
||||
|
||||
A valid scenario reports `{"valid":true,"actions":<count>,"secrets":<count>}`
|
||||
and exits 0. An invalid one exits 1 and reports the same field paths capture
|
||||
would report, so a correction made here is the correction capture needs:
|
||||
duplicate or reserved step IDs, duplicate secret identities and
|
||||
environment-variable names, undeclared secret references, request-step
|
||||
violations, and unknown fields.
|
||||
|
||||
Validation performs no browser discovery, launch, attachment, or navigation and
|
||||
constructs no provider. It does not read secret environment variables, so a
|
||||
scenario whose declared variables are unset still validates; the missing-value
|
||||
failure belongs to capture. A schema-valid scenario does not prove that the
|
||||
executable or target is available on the host.
|
||||
|
||||
## Minimal launch example
|
||||
|
||||
Save this as `browser-scenario.json`, replacing the executable path with an
|
||||
@@ -237,8 +213,7 @@ HTTP(S) URLs may include ordinary query values and fragments directly. Use
|
||||
ordered structured query entries when a value needs to reference a declared
|
||||
environment-backed secret. Form, storage, and cookie values are either literal
|
||||
strings or declared secret references; request step headers can also read a
|
||||
cookie or local storage value from the page, and request steps and `goto`
|
||||
destinations can send [saved values](#saved-values). Raw URL userinfo credentials are rejected.
|
||||
cookie or local storage value from the page. Raw URL userinfo credentials are rejected.
|
||||
Captured credential-header
|
||||
values are not retained, and declared secret values are redacted automatically.
|
||||
A secret may be declared solely to
|
||||
@@ -411,10 +386,8 @@ data you may change. This scenario creates one item:
|
||||
- Request steps follow no redirects: a redirect answer fails the step, and
|
||||
nothing is sent to the location it names, though a write the redirect
|
||||
answered may already have been applied. `expected_status` cannot name a 3xx
|
||||
status. Manual redirects can be opaque to the page and may not produce a
|
||||
response event; their status and `Location` can therefore be unavailable even
|
||||
when network events and header values were selected. Such captures report
|
||||
incomplete event coverage.
|
||||
status. Give the path the redirect names; with header values selected, the
|
||||
capture records its `Location`.
|
||||
- Header values are literals, declared secrets, a cookie the browser holds for
|
||||
the request URL (`{"source": "cookie", "name"}`, HttpOnly cookies included,
|
||||
whose value the page's scripts can then see while the request runs), or a
|
||||
@@ -433,17 +406,6 @@ data you may change. This scenario creates one item:
|
||||
by default) get smaller: after a 32-character `X-CSRFToken` value, a body over
|
||||
about 47 KiB is kept as `unavailable`. Raise `capture.limits` when the
|
||||
application's responses are larger.
|
||||
- A header can send Basic credentials, `{"source": "basic_credentials", "parts":
|
||||
[username, password]}`: the base64 of `username:password`, after a `prefix`
|
||||
such as `"Basic "` or with none, as some APIs take encoded credentials in
|
||||
their own header. Each part is a literal, a declared secret, a saved value, a
|
||||
cookie or a local storage entry, so a header a frontend builds from a login
|
||||
token can be sent too; the password is never a literal other than empty, since
|
||||
Evidence records literals as written, and a username with a colon fails the
|
||||
step, since Basic authentication ends the username at its first colon. The
|
||||
encoded value is redacted however short, as
|
||||
`[REDACTED:page:<step_id>.<header>]`, and parts read from the page are
|
||||
redacted as page values are.
|
||||
- Validation refuses a header name given twice in a step, header names a
|
||||
page's `fetch` cannot set, such as `Cookie`, `Origin`, `Host` or `Sec-*`, and
|
||||
`User-Agent`, which Chrome drops; it refuses line breaks and other ASCII
|
||||
@@ -469,143 +431,6 @@ Captures with request steps state in `limitations` that the steps sent their
|
||||
declared requests through the page's fetch. A scenario without request steps
|
||||
is unchanged.
|
||||
|
||||
## Saved values
|
||||
|
||||
A step can keep a string or number from a JSON response, by name, and a later
|
||||
step can send it, so a scenario can read back, edit and delete an object whose
|
||||
id the application assigns:
|
||||
|
||||
```json
|
||||
{
|
||||
"browser": {
|
||||
"mode": "launch",
|
||||
"executable_path": "/absolute/path/to/chromium"
|
||||
},
|
||||
"start_url": { "url": "http://127.0.0.1:3000/items" },
|
||||
"actions": [
|
||||
{
|
||||
"step_id": "save-item-1",
|
||||
"action": "click",
|
||||
"timeout_ms": 10000,
|
||||
"locator": { "kind": "role", "role": "button", "name": "Save" },
|
||||
"save_response": {
|
||||
"method": "POST",
|
||||
"path": "/api/items",
|
||||
"save": [{ "name": "item_1", "json_pointer": "/id" }]
|
||||
}
|
||||
},
|
||||
{
|
||||
"step_id": "open-item-1",
|
||||
"action": "goto",
|
||||
"timeout_ms": 10000,
|
||||
"destination": { "url": "http://127.0.0.1:3000/items/{saved:item_1}" },
|
||||
"wait_until": "load"
|
||||
},
|
||||
{
|
||||
"step_id": "copy-item-1",
|
||||
"action": "request",
|
||||
"timeout_ms": 10000,
|
||||
"method": "POST",
|
||||
"path": "/api/items/{saved:item_1}/copies",
|
||||
"json": {
|
||||
"source_id": "{saved:item_1}",
|
||||
"title": "Copy of {saved:item_1}"
|
||||
},
|
||||
"expected_status": [201]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- A `request` step saves with `save: [{name, json_pointer, secret}]`, from its
|
||||
own response. A `click` or `press` step saves with `save_response: {method,
|
||||
path, save}`, from a newly observed matching response while the step waits.
|
||||
The recorder watches before the action and excludes requests already running,
|
||||
such as an earlier autosave. Method/path matching does not prove that the UI
|
||||
caused the request: use a path that distinguishes the intended operation from
|
||||
background polling. Multiple observed matches before binding fail as ambiguous;
|
||||
requests starting after this observation window cannot be ruled out. It matches the method and a path on the
|
||||
start origin as the browser spells it, so a path may name `My List` or
|
||||
`My%20List` (`{*}` matches one segment, and the query is ignored), and fails
|
||||
the step unless a 2xx JSON response arrives before its `timeout_ms`; one still
|
||||
being read when the step times out or is cancelled saves nothing. A pointer
|
||||
that names no string or number fails the step, so later steps never send a
|
||||
missing value, as does a response over 4,000,000 characters: a request step
|
||||
stops reading and aborts its fetch there. A UI step reads a body only when
|
||||
its decoded size is bounded at 12,000,000 bytes or less: its received size,
|
||||
or the declared Content-Length of one served from cache, times 1032 for each
|
||||
gzip or deflate coding, the most DEFLATE expands. Other codings and unknown
|
||||
or zero sizes are refused before materializing the body; decoded text over
|
||||
the character limit also fails. The UI action may already have applied.
|
||||
- `{saved:name}` sends a saved value in a request step's `path`, including its
|
||||
query, and in a `goto` destination's `url`, percent-encoded; a destination's
|
||||
structured `query` values are sent as declared. A saved value that makes a
|
||||
path segment `.` or `..`, alone or with the text beside it (including `%2e`),
|
||||
fails the step, since the URL would drop it. In a request step's `json` body,
|
||||
a string that is only `{saved:name}` becomes the saved value with the JSON
|
||||
type it was saved with, so a saved number id is sent as a number, and
|
||||
`{saved:name}` inside a longer string is filled as text. A header value or
|
||||
Basic credentials part sends one as `{"source": "saved", "name"}`. Other
|
||||
braces, such as `{id}`, are sent as written, and so is a percent-encoded
|
||||
`%7Bsaved:name%7D` anywhere but a `goto` URL, whose parsed form spells the
|
||||
braces that way.
|
||||
- Validation refuses a saved value that no earlier step saves. A later save of
|
||||
the same name replaces the value. An integer past 2^53, which JSON parsing
|
||||
would round to another number, and a number that is not finite, such as
|
||||
`1e400`, negative zero, or a value that underflows to zero, such as `1e-400`, fail the step
|
||||
that saves it.
|
||||
- A scenario keeps at most 4,000,000 characters of saved values at once,
|
||||
counting distinct retained redaction spellings, including trimmed variants,
|
||||
of values saved as secret. Replacing a binding releases its live value
|
||||
charge, while secret spellings remain protected for the scenario lifetime.
|
||||
A save past the budget fails before changing the binding.
|
||||
- A value saved with `secret: true` is redacted however short, as a declared
|
||||
secret is, as `[REDACTED:page:<step_id>.<name>]`. The selected source
|
||||
response is protected before retaining its body or header values, including
|
||||
when only headers were requested. If that body cannot be read safely, its
|
||||
body and header values remain unavailable. Before publishing the capture,
|
||||
the selected source transaction's earlier URL, request headers and request
|
||||
body are also protected using the admitted values. Unresolved source values
|
||||
withhold that transaction's affected metadata/content with an explicit
|
||||
limitation. Before publication, retained events are rechecked
|
||||
with the final marked-response values, including immediate follow-up requests
|
||||
and their response bodies, console/page errors, WebSocket text and event URL/name fields. Learned redaction facts are independent of source-identity readiness. Unknown source values withhold only the selected
|
||||
source transaction. Other saved values, such as ids, stay visible, since
|
||||
environments model them. The environment build warns about saved values
|
||||
named like credentials that are not marked secret
|
||||
(`credential-named-saved-values`).
|
||||
- Page-derived cookie/local-storage values and constructed Basic credentials
|
||||
remain redacted only from observations made after they were read/built.
|
||||
They do not retroactively rewrite earlier facts. The final event projection
|
||||
borrows only declared and response-marked patterns; promotion of an already
|
||||
held page value to an explicit response secret preserves its first marker.
|
||||
Projection state is session-local, references admitted pattern/byte owners,
|
||||
and is invalidated on admission or response classification changes.
|
||||
- Screenshots are withheld as `missing` from the start when the scenario plans
|
||||
any response save marked `secret: true`, page-derived or Basic authentication
|
||||
header values, or whenever nonempty sensitive values
|
||||
are already known. Raw pixels cannot be safely text-redacted, so the provider
|
||||
does not request pixels in those cases. This also protects earlier steps and
|
||||
partial failures without retaining an image to scrub later. Other requested
|
||||
artifacts remain available; affected screenshot/aggregate completeness is
|
||||
partial and is not eligible for equality.
|
||||
- Saved URLs and expanded request JSON are admitted before building their expanded strings or
|
||||
serializing. Its allowance is the smaller of Node's string limit and
|
||||
one sixteenth of current V8 available heap bytes, allowing for the expanded
|
||||
tree, serialization, Playwright transport and browser-side copies. Size
|
||||
accounting includes JSON escaping and repeated references; each distinct
|
||||
saved value's size is measured once per request. An oversized request fails
|
||||
before browser dispatch; the complete request envelope, including repeated
|
||||
header values, is checked as well. URL encoding uses a conservative
|
||||
nine-characters-per-UTF-16-unit allowance before encoding a saved value.
|
||||
Reduce the payload/repetitions, or raise the parent
|
||||
Node heap and restart; this is a conservative allocation estimate, not a
|
||||
measurement or guarantee of browser/process-family memory.
|
||||
- Evidence records a request step's empty `save` list as absent, so a scenario
|
||||
that saves nothing records the same parameters and `scenario_sha256` as
|
||||
before saved values existed, and REA versions without saved values read its
|
||||
captures.
|
||||
|
||||
## Network evidence
|
||||
|
||||
`capture.events: ["network"]` retains request, response, completion, failure,
|
||||
@@ -653,9 +478,8 @@ the browser's wall-clock start, and `elapsed_ms`, its time from that start to
|
||||
the end of the response, when the browser reports them. The environment views
|
||||
time scenario records from these, as HAR entries are timed: the browser's start
|
||||
where it reported one, else `observed_at`, which trails it by up to tens of
|
||||
milliseconds while REA reads other responses. A duration is retained only from a
|
||||
unique terminal event that reports it; missing or ambiguous durations stay
|
||||
omitted, while an explicitly reported zero remains zero. Captures made before these fields existed lack them and
|
||||
milliseconds while REA reads other responses, and a duration of 0 where the
|
||||
browser reported none. Captures made before these fields existed lack them and
|
||||
are read in record order. REA versions before them refuse captures that carry
|
||||
them. Scenario comparisons ignore all three (`ignore_network_request_timing`).
|
||||
Requests sharing a URL remain distinct. Redirects link to the preceding
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
# Bun standalone executables
|
||||
|
||||
`inspect-artifact` inventories embedded Bun module graphs in Mach-O `__BUN,__bun`,
|
||||
PE `.bun`, ELF `.bun`, and appended trailers. `extract-artifact` writes their stored
|
||||
module contents through the existing safe output tree. Neither operation executes
|
||||
the executable or decompiles bytecode.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./hello --json
|
||||
rea extract-artifact ./hello --json
|
||||
```
|
||||
|
||||
In MCP, open the executable with `open_binary`, then call `inspect_artifact`.
|
||||
Use `detail: "summary"` for large graphs and `inspect_analysis_view` to page retained
|
||||
Evidence. The root metadata's `entry_point_logical_path` is the reader-owned
|
||||
location of its entry module. Use it as an inventory `logical_paths` filter,
|
||||
then read selected node fields with the returned content identity. This also
|
||||
works for names mapped to indexed extraction paths. The complete Evidence stays
|
||||
retained; no source-name normalization is needed in the caller.
|
||||
|
||||
The root reports the carrier, layout, module count, entry point, raw flags and
|
||||
compile exec-argv. Each module retains its source name, loader and encoding values,
|
||||
entry-point status, sizes and SHA-256. Sourcemaps, bytecode and module-info regions
|
||||
are separate children. Source names such as `/$bunfs/root/main.js` remain in
|
||||
metadata; extraction uses a relative path. Unsafe or non-UTF-8 names receive an
|
||||
indexed path and retain their source spelling or bytes. Virtual names deeper
|
||||
than 256 logical path parts also use an indexed path with an explicit limitation,
|
||||
preventing unbounded path-tree work while retaining the source fact. Duplicate names remain
|
||||
separate occurrences; ordinary extraction collision rules apply.
|
||||
|
||||
The recognized record shapes are the 36-byte Bun 1.3.3 layout and the 52-byte
|
||||
Bun 1.3.9 layout. These labels identify structural layouts, not a detected compiler
|
||||
version. Bun does not store an explicit revision identifier: unknown table sizes
|
||||
and ambiguous known layouts are unsupported. A future layout that is byte-for-byte
|
||||
compatible cannot be distinguished. Invalid pointers and truncated claimed sections
|
||||
are malformed. An absent trailer without a named section does not establish a Bun
|
||||
graph. Unknown enum values remain raw, with a null name.
|
||||
|
||||
Sourcemaps are stored Bun regions, not necessarily standalone JSON source maps.
|
||||
Extracted contents retain their original encoding. Use JavaScript analysis only on
|
||||
compatible source files; bytecode is retained as evidence, not decoded.
|
||||
|
||||
Module parsing reads exact metadata ranges rather than loading stored asset
|
||||
bodies or whole pages around small pointers. A cache retains at most 256 KiB
|
||||
and 1,024 ranges for one reader preparation. Cumulative uncached metadata reads
|
||||
are limited to 256 MiB, checked before allocation or I/O. The module table,
|
||||
decoded strings and combined logical path tree have separate admission checks;
|
||||
at most 250,000 distinct path prefixes are admitted before inventory construction.
|
||||
Stored bodies are range-streamed for hashing and extraction. Header/suffix observations are reused
|
||||
only for the same admitted file state, with selected-path checks before publication.
|
||||
|
||||
See [limits and recovery](design/limits-and-recovery.md) for admission ceilings.
|
||||
Synthetic tests cover the carriers and malformed declarations. Run
|
||||
`node scripts/verify-bun-standalone.mjs` after `npm run build:cached` for local Bun
|
||||
verification, and add `--docker` to compile a Linux fixture with `oven/bun:1.4.0`.
|
||||
Bun and Docker are fixture producers only and are not installed by REA.
|
||||
|
||||
Layout references: [Bun 1.3.3 serializer](https://github.com/oven-sh/bun/blob/bun-v1.3.3/src/StandaloneModuleGraph.zig)
|
||||
and [Bun 1.3.9 serializer](https://github.com/oven-sh/bun/blob/bun-v1.3.9/src/StandaloneModuleGraph.zig).
|
||||
|
||||
The current 52-byte record order can also be checked in the
|
||||
[Bun 1.4.4 Rust serializer](https://github.com/oven-sh/bun/blob/bun-v1.4.4/src/standalone_graph/StandaloneModuleGraph.rs).
|
||||
@@ -1,107 +0,0 @@
|
||||
# Capability composition and ownership
|
||||
|
||||
REA keeps domain contracts and shared workflows separate from provider adapters
|
||||
and CLI/MCP translation. A capability can have several provider identities; a
|
||||
provider can contribute operations to several capabilities. Those associations
|
||||
are source facts, while host availability is an observation.
|
||||
|
||||
## Source declarations and runtime bindings
|
||||
|
||||
`src/composition/capabilityDeclarations.ts` contains static provider identities,
|
||||
named contract-group references and precise operation subsets. Complete groups
|
||||
refer to `TOOL_CONTRACT_GROUPS` in the existing contract owner; family definitions
|
||||
and provider catalogs read the same arrays, so a new group operation is included
|
||||
without updating a copied provider list. Metadata imports only the group ID type
|
||||
and pure metadata leaves; it must not load factories, schemas, engines or
|
||||
generated catalogs. Catalog generation resolves groups and subsets against the
|
||||
canonical contracts and rejects unknown groups/operations and conflicting
|
||||
identities. New provider provenance must come from the implementation
|
||||
that actually produces the Evidence, including format-specific capture adapters
|
||||
and derived workflows.
|
||||
|
||||
`src/composition/capabilityServices.ts` binds typed service factories to those
|
||||
capability keys. Each runtime entry owns construction, cleanup and any available
|
||||
host/configuration facts or live probes. A service entry requires a close
|
||||
capability; entries adapting managers retain their existing `closeAll` operation.
|
||||
The host maps entries without erasing their individual service/port types. There
|
||||
is no generic JSON operation dispatcher.
|
||||
|
||||
These two declarations have different loading needs. Static facts can be read
|
||||
without loading a schema or provider graph. The full runtime composition is
|
||||
materialized only when the MCP runtime needs it. One-shot CLI adapters import the
|
||||
selected factory rather than constructing a complete capability host. CLI and MCP
|
||||
reuse the same provider/service factories and application workflows, while
|
||||
keeping their respective argument translation and lifetime.
|
||||
|
||||
## Resource lifetime
|
||||
|
||||
`CapabilityScope` registers a module's actual close capability when its instance
|
||||
is composed. There is no independent server provider-close list to update.
|
||||
Concurrent shutdown calls join the same cleanup attempt. A successful peer is
|
||||
removed from pending cleanup; failed peers remain owned and are retried on the
|
||||
next close. Synchronous throws and rejections without a reason remain failures,
|
||||
and cannot prevent other peers from attempting cleanup. Shutdown seals further
|
||||
ownership registration.
|
||||
|
||||
The host distinguishes these lifetimes:
|
||||
|
||||
- Host-owned service instances are released by that host.
|
||||
- Caller-owned optional observations remain borrowed by SDK server instances;
|
||||
their process-level owner closes them, including discarded discovery probes.
|
||||
- CLI invocations use their existing finalizers to join cleanup and preserve the
|
||||
original observation, cancellation and cleanup errors.
|
||||
- Binary engines, bundle readers and investigation records retain their existing
|
||||
owners. The `BinarySession` transition/drain barrier still coordinates target
|
||||
changes, reset and disposal. A module refactor must not create independent
|
||||
barriers that allow late work to repopulate reset records.
|
||||
|
||||
An application workflow may borrow a shared resource scope, such as byte entropy
|
||||
using the host's artifact scope. Its wrapper does not acquire a second ownership
|
||||
claim over the borrowed scope.
|
||||
|
||||
## Availability and registration
|
||||
|
||||
Module bindings provide host support, selected configuration and precisely typed
|
||||
live availability probes through `CapabilityAvailabilityFacts`. Constructors do
|
||||
not run these probes or launch engines. Session discovery invokes the existing
|
||||
provider-owned probes only for advertised families; live failures remain fresh
|
||||
observations. Caller policy, selected environments and injected providers retain
|
||||
their existing precedence.
|
||||
|
||||
MCP registration remains in the MCP adapter. It consumes composed services,
|
||||
record ports, admission and logging; it must not add fallback constructors that
|
||||
can escape the host owner. Named contracts bind handlers and schemas. The complete
|
||||
tool-family registrar map is checked by TypeScript, and catalog order never
|
||||
selects a handler or schema.
|
||||
|
||||
## Extending a capability
|
||||
|
||||
1. Add or reuse the domain contract and application workflow, with explicit
|
||||
Evidence provenance, coverage, unknowns and caller choices.
|
||||
2. Put concrete construction in `src/composition/` and keep its default factory
|
||||
lazy. Reference adapter-owned pure identity metadata from both the provider
|
||||
and its static capability declaration.
|
||||
3. Add the runtime binding if a host-owned service is needed. Bind cleanup and
|
||||
availability/probes there, preserving borrowed versus owned resources.
|
||||
4. Add the CLI/MCP argument adapters and their named registrations. A one-shot
|
||||
command uses the shared selected factory and existing finalization helpers.
|
||||
5. Run source-boundary, discovery, SDK-schema and relevant public workflow checks.
|
||||
Exercise partial cleanup, retry, cancellation and caller-owned peers at the
|
||||
applicable production seam; use actual applications/providers for routing and
|
||||
complete Evidence verification. Recording ports do not prove real providers.
|
||||
|
||||
The constructor boundary guard covers the centralized Frida, Cutter, GDB and
|
||||
Rizin constructors without rejecting legitimate type-only injected interfaces.
|
||||
Other settled pure/provider boundaries continue checking type imports too. The
|
||||
shared parser records module-edge erasure once. Whole-declaration type imports
|
||||
are erased; inline type bindings retain their runtime module edge under
|
||||
`verbatimModuleSyntax`. Static dynamic imports remain runtime edges too.
|
||||
Tests belong with their semantic owner and must be
|
||||
found exactly once; provider roots need the corresponding adapter discovery lane.
|
||||
|
||||
This is an internal composition contract. External plugin distribution, automatic
|
||||
provider installation and a universal provider interface are separate concerns.
|
||||
|
||||
The [initial verification report](verification/capability-composition-2026-11-01.md)
|
||||
records the executed public workflows, cleanup boundaries, resource measurements
|
||||
and remaining provider qualification gaps for this composition increment.
|
||||
+13
-280
@@ -51,15 +51,6 @@ module page. See
|
||||
[JavaScript application workflows](javascript-application-workflows.md) for
|
||||
views, feature traces and comparisons that take the same saved Evidence.
|
||||
|
||||
Add `--timings` to a command to write one JSON timing record to stderr. For
|
||||
example, `rea inspect-elf-notes program --json --timings > result.json` keeps
|
||||
stdout as a parseable result and writes
|
||||
`{"rea_timings":{"version":1,"command_ms":42}}` to stderr. `command_ms`
|
||||
measures CLI command dispatch and result delivery; it is not provider-only or
|
||||
analysis-only time. The flag is off by default, and the record omits stages
|
||||
that REA did not measure. Root version reporting does not dispatch a command
|
||||
and has no command timing record.
|
||||
|
||||
## JSON Lines record output
|
||||
|
||||
Use `--format jsonl` to consume row-oriented results one record per line:
|
||||
@@ -165,15 +156,6 @@ into the current shell:
|
||||
eval "$(rea completions bash)"
|
||||
```
|
||||
|
||||
For zsh, initialize completion before loading the hook. If your shell already
|
||||
runs `compinit`, place the hook after that existing initialization:
|
||||
|
||||
```zsh
|
||||
autoload -Uz compinit
|
||||
compinit
|
||||
eval "$(rea completions zsh)"
|
||||
```
|
||||
|
||||
The Bash hook supports macOS system Bash 3.2. It uses optional completion
|
||||
features when the shell provides them and falls back to portable registration.
|
||||
Completion includes standard global options and the active command's options;
|
||||
@@ -259,40 +241,6 @@ Formats 2, 3 and 4 retain header, directory and integrity facts. Compact deliver
|
||||
keeps complete Evidence in the requested bundle. See [Godot PCK](godot-pck.md)
|
||||
for embedded packs, batch queries and unknown encrypted coverage.
|
||||
|
||||
Inspect a UnityFS AssetBundle without running Unity:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact content.bundle --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact content.bundle --json
|
||||
```
|
||||
|
||||
Format versions 6 through 8 retain header, block and directory facts with
|
||||
none/LZ4/LZ4HC block decompression. See [UnityFS](unityfs.md) for encryption
|
||||
and LZMA coverage boundaries.
|
||||
|
||||
Inspect an ESP32-family app or bootloader image without esptool:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact firmware.bin --json
|
||||
rea extract-artifact firmware.bin --json
|
||||
```
|
||||
|
||||
The inventory lists each segment with its ESP-IDF load classification, the
|
||||
checksum and SHA-256 results, the descriptor and any Secure Boot v2 signature
|
||||
blocks. See [ESP images](esp-image.md) for recognition, coverage and refusals.
|
||||
|
||||
Inspect an ESP flash dump, or one read from a flash address other than 0:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact flash.bin --json
|
||||
rea inspect-artifact dump.bin --target-format esp-flash-image --flash-offset 0x1000 --json
|
||||
rea extract-artifact flash.bin --json
|
||||
```
|
||||
|
||||
The inventory lists the bootloader, the partition table and each partition,
|
||||
with ESP image facts for app partitions and the OTA records. See
|
||||
[ESP flash images](esp-flash-image.md) for table location, inputs and limits.
|
||||
|
||||
Inspect a Node.js SEA carrier without running it:
|
||||
|
||||
```sh
|
||||
@@ -303,52 +251,6 @@ rea extract-artifact ./application --json
|
||||
See [Node.js SEA](node-sea.md) for supported layouts, script and asset paths,
|
||||
fuse observations, and the real-fixture verification lane.
|
||||
|
||||
Inventory and extract Intel HEX or S-record regions:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./firmware.hex --json
|
||||
rea extract-artifact ./firmware.hex --json
|
||||
```
|
||||
|
||||
See [Intel HEX and S-record images](hex-records.md) for recognition, region
|
||||
names, metadata and limits.
|
||||
|
||||
Inspect and extract stored U-Boot legacy uImage payloads and components:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./firmware.bin --detail summary --evidence_output firmware.json --json
|
||||
rea extract-artifact ./firmware.bin --json
|
||||
```
|
||||
|
||||
See [U-Boot uImage](uimage.md) for verified CRCs, physical component ranges,
|
||||
unknown names, resource limits and retained views. Compressed bytes stay stored.
|
||||
|
||||
`inspect-artifact` and `extract-artifact` also read ZIP members compressed with
|
||||
Zstandard (standard method 93). They retain the shared ZIP CRC, declared-size,
|
||||
overlap and archive expansion checks. The native streaming decoder keeps the
|
||||
existing 128 MiB Zstandard window cap; frames requiring a larger window are
|
||||
refused. Deprecated method 20 and other unregistered ZIP codecs remain unsupported.
|
||||
|
||||
Inventory and extract UF2 or DfuSe regions:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./firmware.uf2 --json
|
||||
rea extract-artifact ./firmware.dfu --json
|
||||
```
|
||||
|
||||
See [UF2 and DfuSe images](uf2-dfuse.md) for recognition, families, numbering,
|
||||
the DfuSe suffix CRC and limits.
|
||||
|
||||
Inventory and extract U-Boot FIT images:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./boot.itb --json
|
||||
rea extract-artifact ./boot.itb --json
|
||||
```
|
||||
|
||||
See [U-Boot FIT images](fit-images.md) for recognition, data sources, hashes
|
||||
and limits.
|
||||
|
||||
Inspect local model declarations without loading weights:
|
||||
|
||||
```bash
|
||||
@@ -561,25 +463,6 @@ be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and
|
||||
[Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
|
||||
for import, client timeouts and recovery.
|
||||
|
||||
A one-shot command releases its provider session and REA runtime before it
|
||||
prints a failure, so it omits the same-session reuse and
|
||||
`update_analysis_settings` guidance an MCP connection receives. Its
|
||||
`details.limit` reports `scope: "rea-invocation"` and
|
||||
`applies_at: "next_invocation"`; set the named `environment` (or the per-call
|
||||
`parameter`) for a new invocation and rerun. For example, a Ghidra decompile
|
||||
deadline names `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS`.
|
||||
|
||||
`rea debug-with-gdb` and `rea debug-with-rizin` hold their debugger only for the
|
||||
command's own process. If cleanup cannot confirm the debugger stopped, they print
|
||||
`cleanup_incomplete` and exit nonzero without stopping it. `remediation.action`
|
||||
names the process, as does `retained_debugger_pids` in `details.diagnostics`, or
|
||||
in `details.diagnostics.cleanup_error.details.diagnostics` when the command also
|
||||
failed or was interrupted. Exiting closes the debugger's command and output pipes.
|
||||
REA does not establish what the debugger or its target does next, so check that
|
||||
process. An MCP server that shuts down likewise exits with status 1 instead of
|
||||
waiting on a debugger it could not stop. It names the process on stderr and as
|
||||
`retained_debugger_pids` in its error log.
|
||||
|
||||
If one-shot analysis fails and closing its session also fails, the original
|
||||
error details remain inline and `details.cleanup_error` describes the later
|
||||
close failure. Each diagnostic retains its own reported resources; the later
|
||||
@@ -608,14 +491,6 @@ Dispatching a provider metadata write invalidates the active session's snapshot
|
||||
even if its reply fails: the provider may already have applied the write.
|
||||
Close and reopen the target before importing or saving another snapshot.
|
||||
|
||||
Saved snapshots include only cache entries whose subject identifies the selected
|
||||
target. Unbound observations and observations of child artifacts remain available
|
||||
in the open session and its Evidence bundle; they are excluded from the target's
|
||||
saved replay entries without changing their subjects.
|
||||
The snapshot also retains available Evidence linked by selected records, including
|
||||
transitive sources with an unbound or different subject. Those sources are not
|
||||
replay entries for the selected target. Unrelated records remain excluded.
|
||||
|
||||
Snapshots retain eligible target-scoped question histories in full, including
|
||||
mutation Evidence recorded while another target was active. Observations and
|
||||
related questions must still belong to the saved target. If those dependencies
|
||||
@@ -637,30 +512,6 @@ cached result. `close_binary` accepts `snapshot_path` and optional
|
||||
`overwrite: true` to save before releasing provider resources. A failed save
|
||||
leaves the session open so the caller can resolve the output failure.
|
||||
|
||||
When a validated snapshot does not match the selected binding, the error's
|
||||
`details.binding_mismatch` reports confirmed target identities, provider
|
||||
identities, or profile digests that differ. `snapshot` is the saved value and
|
||||
`selected` is the requested value. Multiple differences can be reported. A
|
||||
`profile_comparison: "not_compared"` means the profile was not available at
|
||||
that boundary; it does not establish a match. A profileless snapshot can report
|
||||
`workflow_scope: "incompatible"` even when its target and profile match.
|
||||
Choose a matching target, provider, and profile, or analyze without the snapshot
|
||||
and save a new one for the intended binding.
|
||||
|
||||
For a profile mismatch, the saved snapshot contains
|
||||
`binding.analysis_profile.parameters`. If `binary_session` reports an active
|
||||
profile whose digest matches `binding_mismatch.profile_digest.selected`, its
|
||||
`analysis_provider_binding.analysis_profile.parameters` can be compared with
|
||||
those saved parameters. A rejected switch leaves the previous session active,
|
||||
so its profile may be different from the attempted selection; check the digest
|
||||
before using it for this comparison. A configuration difference does not by
|
||||
itself establish why analysis results differ.
|
||||
|
||||
If no target is open and you intend to inspect the selected binding, open that
|
||||
target without the snapshot, then check its profile digest in `binary_session`
|
||||
before comparing parameters. This may start the selected provider; the saved
|
||||
snapshot remains a separate file.
|
||||
|
||||
If a selected snapshot's file descriptor cannot close, `open_binary` reports
|
||||
`cleanup_incomplete`. `binary_session` reports `cleanup_pending` even when no
|
||||
target opened. Retry `close_binary` to release the retained descriptor before
|
||||
@@ -695,14 +546,6 @@ rea compare /absolute/path/to/evidence/left.json /absolute/path/to/evidence/righ
|
||||
rea compare left.json right.json --record_pairs '[{"left_evidence_id":"ev_<left-sha256>","right_evidence_id":"ev_<right-sha256>"}]' --json
|
||||
```
|
||||
|
||||
Migration: replace `rea compare-bundles` with `rea compare`; the former command
|
||||
has been removed. The MCP tool remains `compare_bundles`.
|
||||
|
||||
`rea compare` compares records by Evidence identity and explicit pairs. It does
|
||||
not read artifact contents: the inventories of two builds are unresolved
|
||||
records, or one opaque change when you pair them. Use `compare-artifacts`,
|
||||
described at the end of this section, to compare their contents.
|
||||
|
||||
`--record_pairs` supplies an explicit JSON array of observation pairs, matching
|
||||
MCP `compare_bundles.record_pairs`. Pair IDs must exist on their named sides and
|
||||
be one-to-one. Without pairs, one-sided membership remains unknown; REA does
|
||||
@@ -719,63 +562,12 @@ with `details.reason: path-limit` and `boundary: filesystem-path`.
|
||||
For application traces and version comparisons, see
|
||||
[JavaScript application workflows](javascript-application-workflows.md).
|
||||
MCP [retained Evidence references](mcp-contracts.md#retained-application-evidence-inputs)
|
||||
belong to one connection. Separate CLI calls can consume full saved records or
|
||||
resolve exact retained IDs from an explicitly selected local bundle:
|
||||
|
||||
```bash
|
||||
rea trace-application-feature request.json --evidence-bundle /absolute/path/evidence.json --json
|
||||
```
|
||||
|
||||
Here `request.json` uses the same logical Evidence input as MCP, for example
|
||||
`"application": {"kind":"retained-evidence","evidence_id":"ev_<64 lowercase hex characters>"}`.
|
||||
Inline records and references may be mixed in the request's declared source
|
||||
fields, including arrays and nested inputs. The bundle is read and authenticated
|
||||
once for that command; no target is reopened, no producer is rerun, and no
|
||||
record is selected by file order. Source identity, provenance, coverage and
|
||||
partial or unknown facts agree with the complete inline form. When starting
|
||||
from summary or inspection Evidence, select its
|
||||
`normalized_result.parent_evidence_id` for a workflow requiring complete source
|
||||
facts. Omit `--evidence-bundle` when all sources are inline.
|
||||
|
||||
The JSON workflow commands supporting this option are:
|
||||
|
||||
- `trace-application-feature`, `trace-javascript-semantics`,
|
||||
`compare-application-versions`, `compare-javascript-export-shapes`,
|
||||
`compare-source-to-bundle`, and `reconcile-javascript-runtime`.
|
||||
- `import-managed-reconstruction`, `verify-managed-native-boundaries`, and
|
||||
`project-managed-application-graph`.
|
||||
- `project-android-application-graph`, `project-apple-application-graph`, and
|
||||
`project-harmony-application-graph`.
|
||||
- `compare-wasm-interfaces` and `inspect-analysis-view`.
|
||||
|
||||
For `inspect-analysis-view`, the request's source reference selects a parent
|
||||
from the bundle for every supported view kind. Bundles exported from MCP work
|
||||
for these commands. Its inline source form cannot also select a bundle.
|
||||
A missing ID is refused with `details.reason: "missing"`;
|
||||
supplying a bundle does not make incompatible Evidence valid for a workflow.
|
||||
`compare-managed-members` keeps its two assembly paths, and
|
||||
`compare-process-captures` keeps its two capture Evidence JSON paths. The bundle
|
||||
option above selects an input; producer options that write a new bundle, such
|
||||
as `inspect-managed-members --evidence-bundle`, retain their existing meaning.
|
||||
File selectors remain available only on contracts that explicitly support them.
|
||||
|
||||
Compare two artifact inventories with `compare-artifacts`. It takes the same
|
||||
`left` and `right` input as MCP `compare_artifacts`: complete Evidence, or an
|
||||
`evidence-file` source with an optional `evidence_id` that selects one record
|
||||
from a bundle. Relative paths resolve from the working directory:
|
||||
|
||||
```bash
|
||||
rea inspect-artifact old.zip --json > old.json
|
||||
rea inspect-artifact new.zip --json > new.json
|
||||
rea compare-artifacts '{"left":{"kind":"evidence-file","path":"old.json"},"right":{"kind":"evidence-file","path":"new.json"}}' --json
|
||||
```
|
||||
|
||||
A one-shot `compare-artifacts` command keeps no session Evidence, so it refuses a
|
||||
`retained-evidence` source with `details.reason: "missing"`. Select the record
|
||||
from a saved bundle instead.
|
||||
For an artifact summary selected from a bundle, include its complete parent
|
||||
inventory in the same bundle. The command resolves that parent from the
|
||||
authenticated file records; a summary-only file cannot supply missing facts.
|
||||
belong to one connection. Separate CLI calls can consume full saved records.
|
||||
`inspect-analysis-view REQUEST.json --evidence-bundle BUNDLE.json` resolves
|
||||
the request's `retained-evidence` source ID from that authenticated bundle,
|
||||
including bundles exported from an MCP session. The same view JSON works for
|
||||
every supported parent kind; no target is reopened and no record is selected
|
||||
by file order. Omit `--evidence-bundle` when the request supplies inline Evidence.
|
||||
|
||||
## Import historical source
|
||||
|
||||
@@ -954,13 +746,7 @@ rea mcp --http --port 8765
|
||||
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the environment,
|
||||
then to `127.0.0.1` and an ephemeral port. A bind outside loopback requires
|
||||
`MCP_ALLOWED_HOSTS` or `--allowed-hosts`, a comma-separated list of client host
|
||||
names or IP addresses without URLs or ports.
|
||||
|
||||
Repeat `--allowed-hosts` to append names, or put multiple names in one
|
||||
comma-separated value. All occurrences are validated together and deduplicated
|
||||
after hostname normalization; the explicit list replaces `MCP_ALLOWED_HOSTS`.
|
||||
|
||||
For example:
|
||||
names or IP addresses without URLs or ports. For example:
|
||||
|
||||
```sh
|
||||
rea mcp --http --host 0.0.0.0 --allowed-hosts 127.0.0.1,rea.lan
|
||||
@@ -1165,8 +951,8 @@ Malformed format values and token windows fail with `INVALID_OUTPUT_VALUE` in th
|
||||
selected structured format. Token limits and offsets must be nonnegative safe
|
||||
integers; zero selects an empty window. These checks run before command execution. `--json` is indented on a terminal and compact when piped or
|
||||
redirected, which keeps large results smaller for agents and scripts; pipe
|
||||
through `jq .` for indented files. A formatting refusal exits nonzero even when
|
||||
the underlying operation succeeded.
|
||||
through `jq .` for indented files. Output selection and formatting do not change
|
||||
operation status.
|
||||
|
||||
`--format md` escapes backslashes, pipes, and inline Markdown syntax in table
|
||||
headers and cells so source text is displayed literally, and renders cell line
|
||||
@@ -1178,42 +964,6 @@ values instead of replacing them with an object label. Table widths are measured
|
||||
iteratively, so an admitted row page is not limited by the JavaScript function
|
||||
argument count.
|
||||
|
||||
Markdown refuses selected NUL characters and unpaired UTF-16 surrogates because
|
||||
renderers or UTF-8 output would substitute them. TOON refuses selected unpaired
|
||||
surrogates for the same UTF-8 reason. Use JSON or YAML to retain those values.
|
||||
An authored replacement character and valid supplementary characters are not
|
||||
treated as malformed.
|
||||
|
||||
`--format csv` and `--format tsv` export one selected row collection. Select an
|
||||
exact array path with `--filter-output` and provide comma-separated direct
|
||||
scalar columns with `--columns`; column order is the output column order:
|
||||
|
||||
```bash
|
||||
rea inspect-analysis-view request.json --format csv \
|
||||
--filter-output normalized_result.rows --columns name,address
|
||||
```
|
||||
|
||||
The first line is a REA metadata record,
|
||||
`# rea-delimited-v1 <JSON>`. It carries the selected path, view request, limitations and unknowns,
|
||||
coverage, source coverage and completeness, command, and duration. A metadata
|
||||
value of `null` means that the selected row collection did not provide that
|
||||
fact; it does not mean complete coverage. The following line is the header.
|
||||
CSV records use CRLF line endings; TSV records use LF. Cells containing the
|
||||
delimiter, quotes, or line breaks are quoted and embedded quotes are doubled.
|
||||
Values are emitted as UTF-8 without Unicode normalization. Missing and `null`
|
||||
cells are empty; booleans and finite numbers use their ordinary text forms.
|
||||
String cells and headers that begin with spreadsheet formula syntax are refused
|
||||
with a structured error; unpaired surrogates in selected cells or headers are
|
||||
also refused. JSON retains those values unchanged.
|
||||
|
||||
Delimited output does not flatten nested Evidence or nested field values.
|
||||
It returns a structured error and a nonzero status if the filter does not
|
||||
select exactly one row array, a row is not an object, or a selected cell is
|
||||
nested or formula-leading. JSON remains the complete canonical output; use it
|
||||
to retain fields outside the selected columns or preserve exact formula-leading
|
||||
text. Token counting and token windows are not
|
||||
available for CSV/TSV because slicing would corrupt records or metadata.
|
||||
|
||||
`--filter-output` selects fields from successful command data. Classified failures
|
||||
retain their complete typed diagnostic projection even when the selector matches
|
||||
a field such as `code` or `message`, or names a missing field. `--full-output`
|
||||
@@ -1232,10 +982,10 @@ Cutter when available. This is separate from `--format json`, which selects the
|
||||
CLI result representation; the global `--json` flag selects that same CLI
|
||||
representation.
|
||||
|
||||
Cutter no longer accepts command-local `--json=true` or `--json=false`.
|
||||
Replace these with `--json-output=true` or `--json-output=false`, respectively.
|
||||
Combine `--json-output` with bare `--json` when both Cutter command output and
|
||||
the CLI result should use JSON; either argument order is accepted.
|
||||
Cutter still accepts deprecated `--json=true` and `--json=false` compatibility
|
||||
values. When both forms are present, `--json-output` takes precedence in either
|
||||
argument order. Bare `--json` retains its global meaning of selecting JSON CLI
|
||||
output formatting.
|
||||
|
||||
| Status | Meaning |
|
||||
| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
@@ -1308,18 +1058,6 @@ new file. A publication that has committed can win a later cancellation; the
|
||||
after commit, the `cleanup_incomplete` error reports the same file as
|
||||
`details.published_path` and `details.bytes`.
|
||||
|
||||
When a failure retains a completed observation and no bundle committed,
|
||||
`details.source_closure` states what that observation cites. `publication` is
|
||||
`not_published`. `records` holds authentic source Evidence this invocation
|
||||
captured, within 64 KiB. `omitted` lists up to 64 other identities and
|
||||
`omitted_count` the total: `not_captured` means the invocation holds no such
|
||||
record, so nothing is shown for it or its ancestors; `not_inlined` means it was
|
||||
captured but is not in this response. `complete` is true only when every cited
|
||||
record is in `records` or embedded in the observation. If examining the closure
|
||||
itself fails, `collection_failure` names the error type and the closure state is
|
||||
unknown; the original failure is still returned. Nothing is retained after exit,
|
||||
and running the command again is a new observation.
|
||||
|
||||
When the reader of piped output closes early, as `rea ... | head` does, the
|
||||
CLI stops writing, prints no error and exits `141` (`128 + 13`, the status of a
|
||||
SIGPIPE-terminated writer). The operation itself may have completed; only its
|
||||
@@ -1375,8 +1113,3 @@ rea inspect-pe-load-config /absolute/path/to/application.exe --json
|
||||
Reports TLS callbacks and size-covered load-config fields without execution.
|
||||
Use `--max-callbacks` and `--max-file-bytes` to set read budgets. See
|
||||
[PE TLS and load configuration](pe-load-config.md).
|
||||
|
||||
Bun standalone executables use the existing `inspect-artifact` and
|
||||
`extract-artifact` commands to inventory and materialize embedded source,
|
||||
sourcemaps and bytecode without running the executable. See
|
||||
[Bun standalone executables](bun-standalone.md) for layout coverage and limits.
|
||||
@@ -14,17 +14,12 @@ interpreter: the npm launcher's `#!/usr/bin/env node` still searches `PATH` for
|
||||
Node. Running that command successfully in a terminal does not establish that
|
||||
the client can launch it.
|
||||
|
||||
For JavaScript installations, setup registers the selected runtime
|
||||
For direct JavaScript installations, setup registers the selected runtime
|
||||
executable and REA script by absolute path. On POSIX, it prefers executable
|
||||
`node` or `bun` and REA aliases on setup's `PATH` only when they resolve to the
|
||||
same selected files. This preserves manager-owned aliases across version updates;
|
||||
otherwise setup uses the selected paths directly. This also applies when setup
|
||||
runs through a package runner: retain the selected cache entry and runtime while
|
||||
the registration uses them. Clearing either makes startup fail without resolving
|
||||
another package from the client's project. Restore the same installation before
|
||||
repairing it; an unidentified registration remains untouched. See
|
||||
[installation and recovery](installation.md#package-runners-and-global-installation).
|
||||
Setup does not copy your ambient environment into client
|
||||
otherwise setup uses the selected paths directly. Package-runner registrations keep
|
||||
their selected runner. Setup does not copy your ambient environment into client
|
||||
settings. To refresh an older registration, preview
|
||||
`rea setup --client <client-id> --skill=false --dry-run`, then run the same command
|
||||
without `--dry-run` and approve the displayed changes. Setup preserves unrelated
|
||||
@@ -45,10 +40,6 @@ the client's configured command or reproduce the client's environment. If doctor
|
||||
passes but the client disconnects, inspect the client's startup error and registered
|
||||
launcher; see [verification scope](client-verification.md#launch-environment-verification).
|
||||
|
||||
`rea doctor` has the same limit for provider checks, and reports a provider setting
|
||||
the registration's file does not carry as a
|
||||
[non-blocking unknown](installation.md#check-readiness-for-your-task).
|
||||
|
||||
## Runtime settings and client deadlines
|
||||
|
||||
`update_analysis_settings` changes curated defaults inside the running REA
|
||||
@@ -83,17 +74,6 @@ Connect Fiddler Everywhere's MCP server separately for HTTP traffic debugging,
|
||||
preserving your REA client entry. See the
|
||||
[setup instructions and public skills](web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
|
||||
|
||||
## VS Code
|
||||
|
||||
`rea setup --client vscode` configures the default VS Code user profile's
|
||||
`mcp.json`. Setup honors `VSCODE_PORTABLE` and `VSCODE_APPDATA` when they are
|
||||
present in its environment. It cannot infer a running window's named profile
|
||||
or `--user-data-dir`; configure that profile separately when needed.
|
||||
|
||||
`rea setup --client copilot_cli` instead configures Copilot's user
|
||||
`mcp-config.json`. VS Code can forward eligible profile servers to Copilot
|
||||
sessions; see the [VS Code MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration).
|
||||
|
||||
## GitHub Copilot CLI
|
||||
|
||||
Setup preserves an existing `tools` selection in the REA registration, including
|
||||
@@ -277,7 +257,7 @@ This headroom is a practical default, not a guarantee for every analysis
|
||||
duration. Setup preserves positive timeout values already in the `rea` entry.
|
||||
|
||||
Setup also forwards explicitly selected input/output schema profiles,
|
||||
`REA_MCP_MAX_RESPONSE_BYTES`, `REA_MCP_MAX_INPUT_ELEMENTS`, provider paths and supported Ghidra, JADX and Kuna
|
||||
`REA_MCP_MAX_RESPONSE_BYTES`, provider paths and supported Ghidra, JADX and Kuna
|
||||
settings. It also forwards explicit executable and resource selections for ADB,
|
||||
apktool, packet capture, GDB/Rizin/objdump, Cutter, WABT, Wakaru, firmware and
|
||||
signal tools, browser inspection, and pwntools/pwndbg. This includes their
|
||||
@@ -287,9 +267,8 @@ patterns. A provider being unavailable does not discard its selected paths;
|
||||
doctor reports its readiness separately. Explicit selections override discovered
|
||||
defaults, including when setup runs again with a changed selection. Unrelated
|
||||
environment variables, process ownership tokens and one-shot target paths are not
|
||||
copied. For an explicitly configured startup target, see the
|
||||
[initial-target settings](mcp-contracts.md#initial-target-at-startup). Existing
|
||||
client environment overrides remain intact unless setup supplies the same key.
|
||||
copied. Existing client environment overrides remain intact unless setup supplies
|
||||
the same key.
|
||||
You can edit `startupTimeoutMs` and `toolTimeoutMs` there; doctor keeps positive
|
||||
caller-selected values aligned and reports a non-blocking advisory when a tool
|
||||
timeout is shorter than the known provider allowance.
|
||||
@@ -507,51 +486,17 @@ comments and unrelated settings. It resolves that file from `HERMES_HOME`, else
|
||||
from the platform default Hermes itself uses — `%LOCALAPPDATA%\hermes` on
|
||||
Windows, `~/.hermes` elsewhere — including any `HERMES_DATA_DIR_SUFFIX`. Hermes
|
||||
connects an entry unless `enabled` is false, so setup states `enabled: true`.
|
||||
Aliases whose anchor is defined in the `rea` entry retain their original resolved
|
||||
values when setup refreshes or uninstall removes that entry. Setup does not
|
||||
redirect another server's alias to REA's new launcher or policy.
|
||||
This also covers an anchor on the removed registration key. References to
|
||||
definitions outside the entry keep sharing the original value, including when
|
||||
another definition later reuses the same anchor name.
|
||||
|
||||
Hermes personal skills install under the active Hermes home's `skills` directory.
|
||||
Setup, doctor and uninstall share REA's sticky-profile resolver.
|
||||
A custom `HERMES_HOME` outside the platform default home
|
||||
reads its own `active_profile`. An override physically inside the default home
|
||||
uses that default home's sticky selection and profile root together, including
|
||||
when the final override directory does not yet exist.
|
||||
|
||||
For example, with `HERMES_HOME=~/.hermes/sub` and `~/.hermes/active_profile` set
|
||||
to `coder`, setup selects `~/.hermes/profiles/coder/config.yaml` and that
|
||||
profile's skills directory. It does not read `~/.hermes/sub/active_profile`.
|
||||
Without an outer named selection, this override uses `~/.hermes/sub/config.yaml`.
|
||||
Use a home outside the platform default tree for an independent sticky selection.
|
||||
|
||||
This describes REA's current routing, not full Hermes CLI equivalence.
|
||||
At Hermes commit `748c79a1`, the
|
||||
[sticky lookup](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_cli/main.py#L612-L632)
|
||||
also reads the outer selection, but
|
||||
[`resolve_profile_env`](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_cli/profiles.py#L2431-L2461)
|
||||
resolves it beneath the configured `HERMES_HOME`. For nested overrides, confirm
|
||||
the profile Hermes actually uses and point `HERMES_HOME` directly at that
|
||||
`profiles/<name>` directory before setup. Compatibility is tracked in
|
||||
[#3939](https://github.com/morluto/rea/issues/3939).
|
||||
|
||||
REA resolves existing ancestor directories before checking physical containment,
|
||||
including symlinks and junctions, even when the override directory does not exist.
|
||||
An override beneath a symlinked default home therefore uses the default home's
|
||||
sticky selection. If `~/.hermes/link` points outside the default home,
|
||||
`HERMES_HOME=~/.hermes/link/new` instead uses that override root, whether or not
|
||||
`new` exists. Previewed destinations retain the configured path spelling.
|
||||
An unreadable path or dangling alias leaves the home unresolved; repair that path
|
||||
before configuring Hermes. Inspect setup's previewed destinations before applying
|
||||
changes.
|
||||
|
||||
An explicit `HERMES_HOME` pointing to `profiles/<name>` selects that named
|
||||
profile directly, bypassing sticky selection. An invalid, missing or deleted
|
||||
sticky profile is reported instead of writing into the default home; repair the
|
||||
profile or run `hermes profile use default` before retrying. It also remains an
|
||||
unresolved skill selection, so setup cannot report a successful empty installation.
|
||||
Setup, doctor and uninstall follow the root's `active_profile` selection, as
|
||||
plain Hermes CLI commands do. An explicit `HERMES_HOME` pointing to
|
||||
`profiles/<name>` selects that named profile directly. An invalid, missing or
|
||||
deleted sticky profile is reported instead of writing into the default home;
|
||||
it also remains an unresolved skill selection, so setup cannot report a
|
||||
successful empty installation. Overrides nested beneath the native Hermes home
|
||||
use that native home's sticky selection and profile root together, matching
|
||||
[Hermes's root rules](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_constants.py#L83).
|
||||
Repair the profile or run `hermes profile use default` before retrying.
|
||||
REA follows `HERMES_HOME` whitespace trimming, environment-variable expansion,
|
||||
and home expansion before resolving that profile. Setup previews the resolved
|
||||
configuration and skill destinations; doctor and uninstall use those same paths.
|
||||
@@ -568,12 +513,10 @@ installs its skill copy there.
|
||||
## Factory Droid
|
||||
|
||||
For [Factory Droid](https://docs.factory.com/harness/mcp), setup writes
|
||||
`mcpServers.rea` into the user-level `~/.factory/mcp.json` (the
|
||||
`FACTORY_HOME_OVERRIDE` override replaces the home that contains `.factory`).
|
||||
Entries carry `command`, `args`, and `env`. Droid discovers personal skills
|
||||
from `~/.factory/skills` ([skills](https://docs.factory.com/harness/skills),
|
||||
following `FACTORY_HOME_OVERRIDE` when set), so setup installs its skill copy
|
||||
there.
|
||||
`mcpServers.rea` into the user-level `~/.factory/mcp.json`. Entries carry
|
||||
`command`, `args`, and `env`. Droid discovers personal skills from
|
||||
`~/.factory/skills` ([skills](https://docs.factory.com/harness/skills)), so
|
||||
setup installs its skill copy there.
|
||||
|
||||
## Trae
|
||||
|
||||
@@ -621,10 +564,9 @@ the fallback file untouched. Existing REA environment overrides are preserved
|
||||
when refreshing its command. A malformed or unreadable fallback file fails setup instead of
|
||||
being silently suppressed; repair or remove it and rerun. Uninstall removes an
|
||||
REA-owned `rea` entry from both the native table and the fallback, backing up
|
||||
each changed file. If removing the last native server would activate a retained
|
||||
fallback server, uninstall refuses before changing either file; review the
|
||||
fallback configuration before retrying. Other fallback servers, settings and a
|
||||
foreign `rea` entry are kept. ZCode disables a
|
||||
each changed file, so an emptied native table cannot load a leftover fallback
|
||||
registration; other fallback servers, settings and a foreign `rea` entry are
|
||||
kept. ZCode disables a
|
||||
server with `enable: false` on its entry; doctor reports such a registration
|
||||
as stale, and rerunning setup removes the flag to repair it. Native configuration
|
||||
uses strict JSON; setup refuses comments, trailing commas and a BOM rather than
|
||||
@@ -647,12 +589,6 @@ boundaries. FreeBuff MCP loading and skill discovery have not been verified
|
||||
with an installed native FreeBuff client/version; filesystem tests cover REA's
|
||||
editing, discovery, and ownership behavior only. Automatic applicability
|
||||
requires the FreeBuff profile directory even when the shared MCP file exists.
|
||||
When uninstall cannot establish FreeBuff or another shared-storage reader from
|
||||
product discovery, a malformed or unreadable shared MCP file is retained
|
||||
unchanged and does not stop other REA removals, including `--purge-data`. When
|
||||
FreeBuff or ZCode product applicability is established or unresolved, the same
|
||||
malformed shared file stops uninstall before removals. See
|
||||
[Diagnose, update, and remove](installation.md#diagnose-update-and-remove).
|
||||
|
||||
FreeBuff setup support requires `[email protected]` or later. Older versions
|
||||
reject `rea setup --client freebuff` as an unsupported client. See the
|
||||
@@ -756,6 +692,10 @@ can override an inherited disable. Lower files and other server entries remain
|
||||
unchanged; a legacy server named `servers` that conflicts with the native table
|
||||
requires manual repair before setup can write.
|
||||
|
||||
When Hermes aliases refer to an anchored REA entry, refreshing or removing REA
|
||||
retains the aliases' original resolved values. Setup does not redirect another
|
||||
server's alias to REA's new launcher or policy.
|
||||
|
||||
With no client selected, the interactive skill choice targets the shared
|
||||
`~/.agents/skills` copy. Selecting a client targets its personal destination
|
||||
when supported. Use `rea setup --skill-client shared --skill` for the shared
|
||||
|
||||
@@ -222,25 +222,6 @@ Error copying, attribution, redaction, and delivery must preserve these facts.
|
||||
Human-readable remediation explains the same failure; clients should not need
|
||||
to parse that sentence to identify an editable setting.
|
||||
|
||||
`scope` and `applies_at` describe a continuing caller, such as an MCP
|
||||
connection. A one-shot invocation, such as a CLI command, has released its
|
||||
provider session and exited its runtime before its caller reads the failure,
|
||||
so its adapter projects with an explicit `"invocation"` caller lifetime.
|
||||
`details.limit` then keeps every other field and reports
|
||||
`scope: "rea-invocation"`, `applies_at: "next_invocation"` and the launch
|
||||
`environment` seed when one exists. `agent_editable: true` still holds, because
|
||||
the caller chooses that environment value or per-call `parameter` for the new
|
||||
invocation; `rea_restart_required: false` means no running REA process needs a
|
||||
restart. A producer keeps guidance that holds only while its caller retains the
|
||||
session, such as "the Ghidra session remains usable" or a same-session runtime
|
||||
update, in `continuingCallerGuidance`; the shared presentation adds it only for
|
||||
a continuing caller. The producer's own message and remediation must stay true
|
||||
after the session is released. A composed error records each embedded child
|
||||
failure through `childFailureDiagnostics`, which snapshots it at composition
|
||||
for both lifetimes when they differ, so cleanup history keeps its state and
|
||||
the final projection selects the caller's form. Failures retained inside
|
||||
Evidence keep the continuing form, so CLI and MCP Evidence stay identical.
|
||||
|
||||
Cache policy is part of enforcement. Successful reusable analysis can survive
|
||||
a budget update. Transient Ghidra decompile failures remain uncached so a raised
|
||||
deadline can recover on the same Program. Admission checks still apply to reused
|
||||
@@ -305,9 +286,7 @@ whether live repair is possible.
|
||||
## What agents do
|
||||
|
||||
Use `details.limit` for a direct runtime update when it names an editable
|
||||
setting with `scope: "rea-runtime"`. For `scope: "rea-invocation"`, set its
|
||||
`environment` or `parameter` for a new invocation instead.
|
||||
Follow `remediation.action` for per-call, fixed and external controls.
|
||||
setting. Follow `remediation.action` for per-call, fixed and external controls.
|
||||
Inspect settings only when bounds or application details are missing. Restart only when it says the value is
|
||||
read at startup. A sentence that a control does not raise the limit is not a
|
||||
request to change it.
|
||||
@@ -337,12 +316,8 @@ request to change it.
|
||||
`runtime-configuration:` check. Re-run `rea setup` for that client when the
|
||||
setting is on the registration allowlist, then reconnect so the new process
|
||||
sees it.
|
||||
- `invalid_request`: follow the specific recovery and reported issue. Inspect
|
||||
the schema or CLI help when the input violates the contract or the response
|
||||
asks for `inspect_input_contract`. A valid selector can instead require a
|
||||
different choice, such as selecting a reported address; an inventory miss
|
||||
does not establish that the target lacks the item. Do not repeat the
|
||||
unchanged call when the response says it cannot succeed unchanged.
|
||||
- `invalid_request`: correct the reported fields from the current schema or
|
||||
CLI help. Do not repeat the unchanged call.
|
||||
|
||||
A restart is required to adopt a changed launch environment or registration.
|
||||
The curated runtime settings above require no REA restart. It is not the first step for a transient
|
||||
@@ -403,17 +378,3 @@ The [verification report](../verification/runtime-analysis-settings.md) separate
|
||||
public MCP checks, simulated provider coverage, real Ghidra execution, and
|
||||
measured task costs. It does not establish real-provider behavior for every
|
||||
editable setting or active worker telemetry.
|
||||
|
||||
## Bun standalone module graphs
|
||||
|
||||
`MAX_BUN_MODULE_COUNT` admits at most 100,000 module records.
|
||||
`MAX_BUN_RETAINED_BYTES` admits at most 256 MiB of serialized graph payload and
|
||||
separately bounds retained entry metadata. Physical metadata read work is also
|
||||
limited to 256 MiB per preparation, including repeated uncached ranges. A
|
||||
256 KiB / 1,024-entry exact-range cache avoids full-page amplification. The
|
||||
combined logical path tree admits at most 250,000 distinct prefixes; this is
|
||||
checked before adding nodes, with shared prefixes counted once. The shared
|
||||
artifact expansion budget also bounds the sum of referenced member lengths. Refusals name the boundary.
|
||||
There is no runtime override; additional host memory or `NODE_OPTIONS` does not
|
||||
raise these ceilings. Use a smaller producer fixture or a separately reviewed
|
||||
reader for larger graphs. No partial extraction is claimed on admission failure.
|
||||
@@ -135,13 +135,11 @@ observations, not an atomic directory snapshot. Unread bytes remain unknown.
|
||||
## Resource bounds and reuse
|
||||
|
||||
Defaults are 20,000 examined entries and 8 MiB of total probe reads. Callers may
|
||||
select up to 100,000 entries and 64 MiB. A separate 32 MiB representation
|
||||
budget bounds retained rows, paths and facts before growth. Each retained path
|
||||
is charged at the larger of its UTF-16 heap size and its escaped JSON size, so
|
||||
typical macOS bundle paths fit about 20,000 rows, and control-character paths
|
||||
fewer. Limits and failed probes produce partial coverage. Traversal and reads
|
||||
are serial. Each reader admits one inspection at a time; overlapping requests
|
||||
receive a retryable capacity refusal without cancelling the admitted caller.
|
||||
select up to 100,000 entries and 64 MiB. A separate 32 MiB conservative
|
||||
representation budget bounds retained paths and facts before growth. Limits
|
||||
and failed probes produce partial coverage. Traversal and reads are serial.
|
||||
Each reader admits one inspection at a time; overlapping requests receive a
|
||||
retryable capacity refusal without cancelling the admitted caller.
|
||||
|
||||
Unity reference decoding stops after a witness in each `Data/Managed` directory;
|
||||
other assemblies' references remain unexamined, while their headers are still
|
||||
|
||||
@@ -1,110 +0,0 @@
|
||||
# .NET single-file bundles
|
||||
|
||||
Use `rea inspect-artifact /absolute/path/host --json` to inventory an ordinary
|
||||
.NET single-file host and `rea extract-artifact /absolute/path/host --json` to
|
||||
materialize its members into a fresh, REA-owned output directory. With a host
|
||||
selected through `open_binary`, the corresponding MCP tools are
|
||||
`inspect_artifact` and `extract_artifact`. No .NET installation is needed for
|
||||
inspection or extraction.
|
||||
|
||||
For a compact inventory, call `inspect_artifact` with `detail: "summary"`,
|
||||
then use `inspect_analysis_view` with the returned `parent_evidence_id`.
|
||||
Inventory occurrence pages accept `logical_paths`, `occurrence_ids`, selected
|
||||
`fields`, `offset`, and `limit`; node pages accept batches of `artifact_ids`
|
||||
to retrieve expanded-byte digests. Include `format_metadata` and
|
||||
`source_location` when member declarations and stored extents matter. Identity
|
||||
fields survive projection, and the complete inspection Evidence stays retained.
|
||||
|
||||
REA detects the apphost marker and its preceding header offset from PE, ELF,
|
||||
and thin Mach-O bytes, regardless of filename. An unbundled apphost's zero
|
||||
offset remains an ordinary native artifact. Universal Mach-O slice inventory
|
||||
keeps its existing behavior; bundled contents inside slices are not expanded.
|
||||
When a native file has a recognized PyInstaller CArchive, apphost-marker
|
||||
scanning stops before its payload. A .NET helper stored inside that archive
|
||||
does not replace the outer container's format; an outer .NET host can still
|
||||
contain a PyInstaller helper as a member.
|
||||
|
||||
## Supported layout
|
||||
|
||||
| Header | Producer generation | Member encoding |
|
||||
| ------ | ---------------------------------- | --------------------------- |
|
||||
| 1.0 | .NET Core 3.x | Uncompressed |
|
||||
| 2.0 | .NET 5 | Uncompressed |
|
||||
| 6.0 | .NET 6 and later using this layout | Uncompressed or raw DEFLATE |
|
||||
|
||||
Versions, minor versions, and flags outside this set are refused explicitly;
|
||||
they do not silently become an ordinary PE or a successful empty inventory.
|
||||
Flags 0 and the v2+ compatibility flag 1 are admitted. Unknown file-type bytes
|
||||
are retained with a limitation, rather than treated as executable authority.
|
||||
Encrypted/protected wrappers and recovering their payloads are outside this
|
||||
reader's scope.
|
||||
|
||||
The host keeps its whole-file SHA-256. Each embedded file gets its expanded-byte
|
||||
SHA-256, path, declared size, compressed size, and `format_metadata` declaration
|
||||
(`major_version`, `minor_version`, `file_type`). Its `source_location` describes
|
||||
the stored host extent, including the compressed length when applicable.
|
||||
The root manifest's `root_format` is `dotnet-bundle`; the root content node
|
||||
still records its native PE/ELF/Mach-O format. A file-type declaration or `.dll`
|
||||
suffix does not establish a valid assembly. Inspect an extracted component
|
||||
explicitly with the managed tools to establish PE/CLI facts.
|
||||
|
||||
## Admission and containment
|
||||
|
||||
Metadata is range-read with a 64 KiB cache. Policy budgets are 16 MiB of
|
||||
manifest metadata, 100,000 files, 64 KiB per UTF-8 string, 512 MiB of stored or
|
||||
expanded bytes per file, 256 path parts per member, 250,000 distinct in-memory
|
||||
path-tree nodes, and 2 GiB of aggregate expanded bytes. The node budget counts
|
||||
shared directory prefixes only once. These are REA
|
||||
resource policies, not limits imposed by Microsoft's format. Marker scanning
|
||||
streams 64 KiB chunks and reuses completed probe facts during extraction;
|
||||
ordinary native files without a marker incur one extra
|
||||
read pass before host hashing.
|
||||
|
||||
Extraction hashes the whole host again through the same descriptor used to read
|
||||
members and compares it with the inventory's root digest before creating an
|
||||
output directory. This adds a full-host read pass to extraction, including when
|
||||
the embedded members themselves are small.
|
||||
|
||||
Malformed/truncated headers, unsafe or conflicting paths, overlapping member
|
||||
ranges, members crossing the manifest, contradictory configuration extents,
|
||||
invalid UTF-8, corrupt/trailing compressed data, and incorrect expanded sizes
|
||||
fail before output publication. Source descriptor/path identity is checked
|
||||
during reading; extraction verifies the host digest and member bytes against
|
||||
the inventory. Existing output roots are never overwritten. Cancellation and
|
||||
failures release owned streams/descriptors through the shared extraction
|
||||
rollback scope.
|
||||
|
||||
## Reproduce producer fixtures
|
||||
|
||||
The source-owned generator uses the installed .NET 8 SDK's **Microsoft
|
||||
HostModel** producer to emit x64 v1, v2, plain v6, and compressed v6 bundles
|
||||
for Windows, Linux, or macOS. It does not run generated hosts or download runtime packs. Supply an
|
||||
SDK apphost template from
|
||||
`packs/Microsoft.NETCore.App.Host.<rid>/<version>/runtimes/<rid>/native/apphost`
|
||||
(`apphost.exe` on Windows). Pass `windows` (the default), `linux`, or `osx`
|
||||
as the optional third generator argument to match that template.
|
||||
|
||||
```bash
|
||||
dotnet build scripts/fixtures/dotnet-bundle/BundleFixture.csproj -c Release --configfile scripts/fixtures/dotnet-bundle/NuGet.Config -o /absolute/path/producer -p:BaseIntermediateOutputPath=/absolute/path/producer-obj/
|
||||
dotnet /absolute/path/producer/BundleFixture.dll /absolute/path/apphost.exe /absolute/path/absent-bundles-directory
|
||||
rea inspect-artifact /absolute/path/absent-bundles-directory/net8-compressed/Probe.exe --json
|
||||
rea extract-artifact /absolute/path/absent-bundles-directory/net8-compressed/Probe.exe --json
|
||||
```
|
||||
|
||||
The generator writes `oracle.json` beside each host. An independent C#
|
||||
`BinaryReader`/`DeflateStream` pass records version, member types and stored
|
||||
extents, then compares every expanded member digest with its source file.
|
||||
Compare those records with inspection Evidence and the extracted bytes.
|
||||
This verifies Microsoft's compression writer separately from synthetic parser
|
||||
fixtures; it is not evidence that the bundled host executes on a target runtime.
|
||||
|
||||
Deterministic boundary and public-entrypoint coverage:
|
||||
|
||||
```bash
|
||||
npm run test:focused -- tests/boundary/filesystem/dotnetBundleArtifacts.test.ts tests/acceptance/dotnetBundleJourney.test.ts
|
||||
```
|
||||
|
||||
Layout references: Microsoft's [manifest writer](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/Bundle/Manifest.cs),
|
||||
[file entry writer](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/Bundle/FileEntry.cs),
|
||||
[apphost marker handling](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/AppHost/HostWriter.cs),
|
||||
and [compression producer](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/Bundle/Bundler.cs).
|
||||
@@ -75,13 +75,6 @@ DOM node indices retain their snapshot positions. Invalid rows are omitted with
|
||||
completeness diagnostics, so `total_nodes` can exceed the retained node count
|
||||
and a parent index can refer to an omitted source node.
|
||||
|
||||
`dom.node_value_length_unit` is `utf8-bytes`: each node's `node_value_length`
|
||||
counts the bytes needed to encode its value as UTF-8, not characters. Web
|
||||
observations instead declare `utf16-code-units`. These existing measurements
|
||||
are preserved; the unit is reported once for the whole DOM, without retaining
|
||||
text values. Historical captures without the unit leave it unknown; reading
|
||||
them does not infer a unit or convert their stored lengths.
|
||||
|
||||
Script metadata retains its execution-context frame ID when CDP supplies one and
|
||||
records admission separately in `frame_scope`. An `authorized` scope refers to
|
||||
the observed document generation; `unknown` retains the known frame ID without
|
||||
@@ -147,14 +140,6 @@ window-targeted renderer reload/crash actions, and synthetic `open-url` or
|
||||
WebContents identities, process metrics, and IPC channel and value-shape metadata.
|
||||
Payload values are not retained.
|
||||
|
||||
For the exact Playwright launch sequence, the active hook removes only REA's
|
||||
bootstrap arguments from the application-visible `process.argv`. The vector keeps
|
||||
the selected entry point and every caller argument, in order, including repeated
|
||||
hook paths, `.asar` filenames, and debug-like options. The private launch handoff
|
||||
is consumed before application code runs. If the observed sequence cannot be
|
||||
matched exactly, argv stays unchanged and capture coverage reports `hook_conflict`;
|
||||
REA does not guess an entry point or remove arguments by filename patterns.
|
||||
|
||||
`window_index` selects from Playwright's current Page list at the start of each action. REA resolves that Page to its corresponding BrowserWindow once; the action's `target` uses the BrowserWindow's capture-scoped `window_id`, matching the window inventory. Reordering either enumeration after selection does not retarget the action. Handle cleanup failures retain the completed action records and remain owned for provider cleanup retry.
|
||||
|
||||
Run the real fixture verifier with an operator-provided Electron runtime:
|
||||
@@ -201,9 +186,6 @@ neither result means the requested host operation occurred. The existing
|
||||
`openExternal`, `openPath`, and `trashItem` blocks reject their promises.
|
||||
The `trashItem` block prevents that call from reaching the OS trash operation;
|
||||
it does not intercept other ways the application can remove files.
|
||||
Process rejection and uncaught-exception events are observed at dispatch without
|
||||
adding exception handlers. The runtime and application retain their warning,
|
||||
handler, and fatal-exit policies, including for rejected shell-block promises.
|
||||
CLI SIGINT/SIGTERM requests cancellation and awaits owned-process cleanup.
|
||||
Window IDs, URLs, and cached titles come from one main-process Electron snapshot,
|
||||
so a crashed renderer does not require a successful renderer-side title query.
|
||||
|
||||
+1
-7
@@ -38,12 +38,6 @@ were examined; an unavailable section name does not erase those note facts.
|
||||
For GNU notes, `gnu.build_id_hex`, `gnu.abi_tag`, or `gnu.gold_version` are
|
||||
filled when the descriptor matches that type. Other owners keep `gnu` null.
|
||||
|
||||
For Linux `CORE` notes, `core.prpsinfo`, `core.file`, `core.siginfo`, or
|
||||
`core.auxv` are decoded for process arguments and status (`NT_PRPSINFO`), mapped
|
||||
file memory regions and paths (`NT_FILE`), fault signal and code (`NT_SIGINFO`), and
|
||||
auxiliary vectors (`NT_AUXV`) across 32-bit and 64-bit images. Other owners or unparsed
|
||||
notes keep `core` null.
|
||||
|
||||
`PT_NOTE` and `SHT_NOTE` are examined independently. The same file range may
|
||||
appear twice when both tables declare it.
|
||||
|
||||
@@ -54,7 +48,7 @@ the image never carried notes in another container or overlay.
|
||||
|
||||
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
|
||||
32 note entries are examined by default across every table. Retained note
|
||||
names, section names, GNU descriptor text, and core dump string representations share a scan budget based on
|
||||
names, section names, and GNU descriptor text share a scan budget based on
|
||||
the source bytes plus any decoded section-name table, capped at
|
||||
`max_file_bytes`. The raw source and decoded table must also fit together
|
||||
within that ceiling. If admission fails, the error reports the required
|
||||
|
||||
@@ -52,15 +52,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -124,15 +115,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -196,15 +178,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -268,15 +241,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -340,15 +304,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -412,15 +367,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -484,15 +430,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -556,15 +493,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -628,15 +556,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -700,15 +619,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -772,15 +682,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -800,7 +701,7 @@
|
||||
"calculated_sha256",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": {}
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
@@ -826,33 +727,7 @@
|
||||
"calculated_md5",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": {}
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"logical_path": { "type": "string" },
|
||||
"algorithm": { "type": "string", "const": "dfu-crc32" },
|
||||
"declared_crc32": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-f0-9]{8}$"
|
||||
},
|
||||
"calculated_crc32": {
|
||||
"anyOf": [
|
||||
{ "type": "string", "pattern": "^[a-f0-9]{8}$" },
|
||||
{ "type": "null" }
|
||||
]
|
||||
},
|
||||
"unpacked": { "type": "boolean" }
|
||||
},
|
||||
"required": [
|
||||
"logical_path",
|
||||
"algorithm",
|
||||
"declared_crc32",
|
||||
"calculated_crc32",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": {}
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -918,15 +793,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -990,15 +856,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1062,15 +919,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1134,15 +982,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1206,15 +1045,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1278,15 +1108,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1350,15 +1171,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1422,15 +1234,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1494,15 +1297,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -1,348 +0,0 @@
|
||||
# ESP flash images
|
||||
|
||||
REA reads a dump of an ESP32-family SPI flash, such as the output of
|
||||
`esptool read-flash 0 ALL dump.bin` or `esptool merge-bin`, through
|
||||
`inspect_artifact` and `extract_artifact`. The reader is static: no device,
|
||||
esptool or ESP-IDF is needed, and no code is run.
|
||||
|
||||
The container format is `esp-flash-image`. Entries are the bootloader region,
|
||||
the partition table sector and one file per partition. The bootloader and
|
||||
each app partition are also decoded as [ESP images](esp-image.md), so their
|
||||
headers, segments, integrity and descriptors appear in the root
|
||||
`format_metadata`. Layout facts are cited to ESP-IDF v6.1
|
||||
(`esp_flash_partitions.h`, `flash_partitions.c`, `gen_esp32part.py`,
|
||||
`bootloader_common_loader.c`, `bootloader_utility.c` and each chip's
|
||||
`esp_rom_caps.h`).
|
||||
|
||||
## Recognition
|
||||
|
||||
A file is a flash image when REA finds a partition table in the entry
|
||||
structure ESP-IDF's bootloader verifies (`esp_partition_table_verify`):
|
||||
32-byte `0x50AA` entries, at most one `0xEBEB` MD5 entry over the entries
|
||||
before it, and a terminating entry within `0xC00` bytes. ESP-IDF accepts a
|
||||
table with no partition entries, but REA requires one, so that neither
|
||||
erased flash nor a lone MD5 entry is read as a table. A table whose MD5 does not match, or that has a second MD5 entry, is
|
||||
still read, with `md5.status` `invalid` and a limitation, because the
|
||||
bootloader would refuse it. The table is located in this order:
|
||||
|
||||
1. at the caller's `partition_table_offset`, when given; no scan follows;
|
||||
2. at the default `0x8000`;
|
||||
3. by scanning every `0x1000`-aligned flash sector for a table whose MD5
|
||||
verifies. This location is derived, and the inventory says so.
|
||||
|
||||
A table never lies inside the located bootloader image, so the default
|
||||
offset and the scan are tried only after the image's last segment. Without
|
||||
a caller selection, the scan runs only when an ESP image sits at a ROM
|
||||
bootloader offset, and a single app or bootloader image ends at its last
|
||||
segment, so neither ordinary files nor single images are scanned. A C3 or
|
||||
S3 dump begins with its bootloader, so flash layouts are detected before a
|
||||
file is read as one ESP image.
|
||||
|
||||
Unless the caller names the table with `partition_table_offset`, REA
|
||||
refuses two kinds of dump before looking for a table:
|
||||
|
||||
- an image of a known chip at another chip's ROM offset, such as an ESP32
|
||||
bootloader at the start of a dump read from `0x1000`. The dump's origin
|
||||
is not the flash address REA assumed, given or default, and a table found
|
||||
there would place every partition at the wrong file offset; pass the
|
||||
`flash_offset` where the dump starts. When a partition table sits where
|
||||
the image's own ROM offset puts `0x8000`, after the image's segments, the
|
||||
refusal names that offset.
|
||||
Without a selected format such a dump opens as its bootloader image, whose
|
||||
[ESP image](esp-image.md) facts say how to reopen it.
|
||||
- an ESP8266 image at a ROM offset. ESP8266 dumps can hold the same table
|
||||
format, and they keep the [ESP image](esp-image.md) reader's ESP8266
|
||||
refusal.
|
||||
|
||||
When no table is found the layout is not recognized. This is an unknown
|
||||
outcome, not a malformed file: flash encryption, which encrypts the
|
||||
bootloader and the table, is one cause. When a table is found without a
|
||||
located bootloader and `flash_offset` is omitted, a limitation says that
|
||||
each partition's file offset assumes the dump starts at flash `0x0`.
|
||||
|
||||
## Caller inputs
|
||||
|
||||
| Input | Meaning |
|
||||
| ------------------------- | ----------------------------------------------------------------------------- |
|
||||
| `format: esp-flash-image` | Read the file as a flash dump; with no table found, refuse it |
|
||||
| `flash_offset` | Flash address (`0x…`) of the dump's first byte; default 0 |
|
||||
| `partition_table_offset` | Flash address (`0x…`, a multiple of `0x1000`) of the table; disables the scan |
|
||||
|
||||
MCP takes these on `open_binary`; the CLI takes `--target-format
|
||||
esp-flash-image`, `--flash-offset` and `--partition-table-offset` on
|
||||
`inspect-artifact` and `extract-artifact`. The offsets require the format, and
|
||||
the format takes no `architecture` or `entry_point`. The inputs are part of
|
||||
the target's identity, so reopening the same bytes with other inputs reads
|
||||
them again instead of reusing the earlier reading. The `open_binary` summary
|
||||
reports them as `flash_offset` and `partition_table_offset`, null when
|
||||
omitted.
|
||||
|
||||
## Layout facts
|
||||
|
||||
The root `format_metadata` (`esp-flash-image`) records:
|
||||
|
||||
- `flash_offset`: its `value` and whether the `caller` gave it or it is the
|
||||
`default`;
|
||||
- `chip`: from the bootloader's `chip_id`, or null without a located
|
||||
bootloader;
|
||||
- `bootloader`: the region from the chip's ROM bootloader offset to the
|
||||
table, with its ESP image facts;
|
||||
- `partition_table`: its flash address, the `method` that found it
|
||||
(`default`, `caller` or `scan`), the partition count and the MD5 `status`
|
||||
(`valid`, `invalid` or `absent`) with the stored and computed digests;
|
||||
- `apps`: ESP image facts for each app partition the dump holds; app
|
||||
partitions declared over the same bytes share one decode;
|
||||
- `otadata`: the otadata partition's records, or null without one;
|
||||
- `boot_selection`: the app the bootloader would select, for every table;
|
||||
- `partitions_outside_dump`: declared partitions that begin before the
|
||||
dump's first byte, which have no entry;
|
||||
- `undescribed`: ranges outside every region, each from a byte that is not
|
||||
erased flash to the end of its gap.
|
||||
|
||||
The ROM loads the bootloader from `ESP_ROM_BOOTLOADER_OFFSET_FLASH`:
|
||||
`0x1000` for ESP32 and ESP32-S2; `0x0` for ESP32-S3, C2, C3, C6, C61, H2 and
|
||||
H21; and `0x2000` for ESP32-C5, H4, P4 and S31. A bootloader is located only
|
||||
where its own `chip_id` says the ROM loads it.
|
||||
|
||||
### Partitions
|
||||
|
||||
Each partition entry is `partitions/<label>.bin` when the label is a safe
|
||||
name of up to 16 letters, digits, `.`, `_` and `-` that starts with a
|
||||
letter, digit or underscore and has at least one letter or underscore, is
|
||||
not a Windows device name such as `NUL` or `COM1`, alone or before a dot,
|
||||
and no other label repeats it, ignoring case as some file systems do; it is
|
||||
`partitions/<index>.bin` otherwise. `source_path` keeps the label, the
|
||||
table's own name for the partition. Its `format_metadata`
|
||||
(`esp-flash-partition`) records the table `index`, `label` (null when not
|
||||
UTF-8), `label_hex` (the label's bytes when it is not UTF-8, else null),
|
||||
`type`, `subtype`, ESP-IDF's `type_name` and `subtype_name`,
|
||||
`flash_address`, `size`, the `encrypted` and `readonly` flags and any
|
||||
`unknown_flags`. The `encrypted` flag says how a device with flash
|
||||
encryption stores the partition; it does not establish that the dump's
|
||||
bytes are ciphertext.
|
||||
|
||||
Each partition entry also records where the bytes the dump holds are
|
||||
written, relative to the partition's start: `written_offset`, the first
|
||||
byte that is not erased flash (`0xFF`), `written_end`, just past the last
|
||||
such byte, and `written_bytes`, how many bytes are not `0xFF`. A partition
|
||||
erased throughout has null offsets and `written_bytes` `0`; a partition the
|
||||
file holds none of has all three null. The range bounds the written
|
||||
bytes and does not say whether erased flash lies between them; extract the
|
||||
partition to read them. REA reads the bytes the partitions cover once, so
|
||||
overlapping partitions share that read however many the table declares.
|
||||
|
||||
ESP-IDF 5.3 and later may declare primary `bootloader` and `partition_table`
|
||||
entries over the bootloader region and the table sector. They are
|
||||
partitions like any other, whose bytes are also `bootloader.bin` and
|
||||
`partition-table.bin`, and they are exempt from the rule that partitions
|
||||
follow the table, as in `gen_esp32part.py`.
|
||||
|
||||
Every partition from the dump's first byte on is an entry holding the bytes
|
||||
the file has. One that the file holds only in part, as when an `esptool
|
||||
merge-bin` dump ends inside its last partition, has an entry limitation that
|
||||
gives the size the table declares; one that lies past the end of the file is
|
||||
an entry with no bytes and a limitation, and has no image facts. A partition
|
||||
that begins before the dump's first byte has no entry and is listed in
|
||||
`partitions_outside_dump`. One unreadable partition does not hide the
|
||||
others: partitions outside the dump or overlapping another are each reported
|
||||
while the rest stay readable.
|
||||
|
||||
### App partitions and the bootloader
|
||||
|
||||
Each app partition and the bootloader region carry their entry's `path`
|
||||
and ESP image `facts`:
|
||||
`status` is `esp-image`, `erased` (the image header bytes are erased flash,
|
||||
as in an empty OTA slot) or `unrecognized` (ciphertext or other data). For an
|
||||
image they hold the [ESP image](esp-image.md) root facts, `segments`,
|
||||
`signature_blocks`, `coverage` and `limitations`, with offsets relative to
|
||||
the region. Extract a partition and inspect it as an ESP image for its
|
||||
segments as entries.
|
||||
|
||||
The summary view (`inspect_artifact` with `detail: "summary"`, or the
|
||||
inventory summary of `inspect_analysis_view`) reports the root facts under
|
||||
their own kind, `esp-flash-image-summary`, and gives each of these images a
|
||||
`summary` instead of its `facts`: `status`, `chip`, `entry_addr`,
|
||||
`segment_count`, `load_ranges` (the load address, declared length and
|
||||
classification of each segment that is mapped, loaded or unclassified),
|
||||
`signature_block_count`, the `descriptor` kind with the fields that tell
|
||||
builds apart (an app's `project_name`, `version`, `idf_ver`,
|
||||
`secure_version` and `app_elf_sha256`; a bootloader's `version`, `idf_ver`
|
||||
and `secure_version`), the `checksum` and `appended_sha256` statuses,
|
||||
`coverage` and `limitations`. The default complete detail keeps the full
|
||||
facts, every segment included, and the summary's `inventory_evidence_ids`
|
||||
name the Evidence that holds them.
|
||||
|
||||
`erased` is the bootloader's view: with no image header at the start of the
|
||||
slot, it finds no app there. An erased slot can still hold data further in,
|
||||
even a complete app image. When it does, its entry's `written_offset` and
|
||||
`written_end` say where, and a limitation says that the slot holds bytes the
|
||||
bootloader does not read; REA does not look for images in them.
|
||||
|
||||
### OTA data
|
||||
|
||||
The otadata partition holds two `esp_ota_select_entry_t` records, one at the
|
||||
start of each 4 KiB sector. Like the bootloader, REA reads the last otadata
|
||||
partition the table declares, and the last factory app and OTA app of each
|
||||
subtype; a limitation names any repeated declaration. REA reports each
|
||||
record the partition and the dump hold: its `ota_seq`, `seq_label`,
|
||||
`ota_state` and its name, `crc`, whether the CRC matches (CRC-32 of
|
||||
`ota_seq` seeded with `0xFFFFFFFF`) and whether the bootloader would treat
|
||||
it as valid.
|
||||
|
||||
### Boot selection
|
||||
|
||||
`boot_selection` is present for every table. It is derived as
|
||||
`bootloader_utility_get_selected_boot_partition` does without app rollback
|
||||
or anti-rollback, and `basis` names its branch:
|
||||
|
||||
- `otadata`: the valid record with the higher `ota_seq` (record 0 on a tie)
|
||||
selects slot `(ota_seq - 1) % ota_app_count`, indexed by OTA subtype;
|
||||
- `factory-fallback`: no record is usable, or the table has no OTA app, and
|
||||
a factory app is declared; or records are set but none is valid, as after
|
||||
a CRC mismatch, which selects the factory index whether or not a factory
|
||||
app is declared;
|
||||
- `ota_0-fallback`: no record is usable, or the table has no OTA app, and no
|
||||
factory app is declared, so the bootloader selects boot index 0;
|
||||
- `no-otadata`: the table declares no otadata partition, or one at offset 0,
|
||||
which the bootloader takes for none, so it selects the factory index, as in
|
||||
the common factory-only layout;
|
||||
- `none`: the bootloader rejects the table, for an MD5 mismatch or a second
|
||||
MD5 entry, or cannot read an otadata partition smaller than two 4 KiB
|
||||
sectors, and resets without selecting an app;
|
||||
- `unknown`: the dump lacks an otadata record the bootloader reads.
|
||||
|
||||
Without an OTA app the records do not matter, so a dump that lacks them
|
||||
still selects a fallback. REA does not compare the partitions with the
|
||||
flash size in the bootloader header, which the bootloader also checks.
|
||||
|
||||
`active_record` and `slot` are set for `otadata` and `slot` for
|
||||
`ota_0-fallback`. `partition_index` is null when the table declares no
|
||||
partition for the selected slot or fallback; the bootloader then tries the
|
||||
other apps. `assumptions` list what the derivation leaves out: records used
|
||||
as written (for the branches that read them), anti-rollback, load fallback,
|
||||
and factory-reset and test-app GPIO holds and a wake from deep sleep; they
|
||||
are empty for `none` and `unknown`. `running_partition` is always
|
||||
`unknown`: at boot the bootloader may first mark a pending-verify record
|
||||
aborted, check the app's secure version against eFuse, and fall back to
|
||||
another partition when an app fails to load.
|
||||
|
||||
## Coverage and extraction
|
||||
|
||||
Erased flash (`0xFF`) between regions is described. In a gap between
|
||||
regions, the first byte that is not erased starts an `undescribed` range to
|
||||
the end of the gap, and makes coverage `unknown`; the rest of that range is
|
||||
not examined. Coverage is also `unknown` when no bootloader is located, or
|
||||
when a declared partition begins before the dump's first byte, since it has
|
||||
no entry. A partition cut short or left out by the end of the file does not
|
||||
make it `unknown`: its entry holds the bytes the file has, none for one past
|
||||
the end, and its limitation says what the file lacks. Coverage describes the
|
||||
inventory of the file, not the observation of every declared flash byte.
|
||||
|
||||
So artifact comparison classifies every partition entry of two dumps. Where
|
||||
both lack the same bytes, as two `merge-bin` dumps of the same layout do,
|
||||
the entries are equal; a dump against a longer one reports each partition
|
||||
it cuts short or lacks as `changed`, and that entry's limitation says the
|
||||
file did not hold the bytes.
|
||||
|
||||
`extract_artifact` writes every entry as an ordinary file, `nvs_keys`
|
||||
partitions included. On a device without flash encryption an `nvs_keys`
|
||||
partition holds the NVS encryption keys in plain text, and a dump of it
|
||||
contains them. REA decrypts nothing, recovers no keys and does not interpret
|
||||
NVS.
|
||||
|
||||
## Comparison
|
||||
|
||||
Artifact comparison uses each partition's `label`, `label_hex`, `type`,
|
||||
`subtype`, `flash_address`, `size` and flags as its declaration, so a
|
||||
partition that moves, changes type or flags, or is renamed with unchanged
|
||||
bytes is `changed` in `metadata`. Partitions pair by path. The root facts
|
||||
are not compared as a declaration: they derive from the dump's bytes, which
|
||||
content comparison covers, and from the caller's `flash_offset` and
|
||||
`partition_table_offset`, which are part of the target's identity and
|
||||
decide each partition's byte range.
|
||||
|
||||
## Limits
|
||||
|
||||
- A flash dump inside another container, such as a ZIP, is classified from
|
||||
its first bytes and so reads as an ESP image, not as a flash layout; extract
|
||||
it first.
|
||||
- ESP8266 dumps are refused; Secure Boot v1 bootloader digests and
|
||||
flash-encrypted layouts are not read.
|
||||
- Partitions are not checked against the flash chip size, which the
|
||||
bootloader also verifies.
|
||||
- NVS, FAT, SPIFFS and other data partitions are extracted, not decoded.
|
||||
- An image nested in a partition is decoded for its facts; its segments are
|
||||
not inventory entries.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic fixtures with computed MD5s and CRCs, and one CRC otatool wrote,
|
||||
cover the table rules, every way the table is located, the scan's chunk and
|
||||
sector edges, the misplaced-origin guard with the origin omitted or given,
|
||||
the ESP8266 refusal, the `0x0`, `0x1000` and `0x2000` ROM bootloader
|
||||
offsets, single images that are not scanned, overlaps, primary entries,
|
||||
truncation and partitions outside the dump, index names for unsafe labels,
|
||||
repeated declarations, the otadata states and records, and each branch of
|
||||
the bootloader's selection. CLI and stdio MCP tests exercise the inputs and
|
||||
their identity and extraction. MCP `compare_artifacts` tests compare
|
||||
partition declarations, labels that are not UTF-8 included, two dumps that
|
||||
end inside their last partition, and such a dump against a full one.
|
||||
|
||||
The optional real lane builds its dumps with esptool and ESP-IDF's own tools:
|
||||
|
||||
```sh
|
||||
REA_RISCV_CLANG=/path/to/clang REA_RISCV_LLD=/path/to/ld.lld \
|
||||
REA_ESPTOOL_PYTHON=/path/to/python-with-esptool \
|
||||
REA_ESP_IDF_PATH=/path/to/esp-idf \
|
||||
npm run verify:esp-flash
|
||||
```
|
||||
|
||||
It converts the conformance ELFs with esptool, builds a default, a
|
||||
relocated and a primary-entry partition table with `gen_esp32part.py`,
|
||||
assembles dumps with `esptool merge-bin`, one of them written from flash
|
||||
`0x1000` with `--target-offset`, and writes OTA records with `otatool`
|
||||
through `tests/conformance/esp/flash_file_tool.py`. That script replaces
|
||||
parttool's esptool calls with reads and writes of the dump file, so
|
||||
otatool's own OTA logic runs unchanged. The lane then checks:
|
||||
|
||||
- the bootloader and app facts against `esptool image-info`;
|
||||
- every partition against `gen_esp32part.py`'s decoding of the table REA
|
||||
extracted;
|
||||
- both OTA records against `otatool read_otadata` after switching to slot 0,
|
||||
then to slot 1, then erasing the otadata, and the derived selection after
|
||||
each: slot 0, slot 1, then the factory fallback;
|
||||
- that each extracted file is the dump's bytes at its range and that the
|
||||
source is unchanged;
|
||||
- a table with the primary `bootloader` and `partition_table` entries that
|
||||
`--primary-bootloader-offset` fills, read with complete coverage and no
|
||||
overlap or misplacement limitation; it has no otadata, so its boot
|
||||
selection is `no-otadata` with the factory app;
|
||||
- the relocated table, found by scanning and named with
|
||||
`partition_table_offset`, and the dump written from `0x1000`, read with
|
||||
`flash_offset` and, without it, left an ESP image whose `flash_dump`
|
||||
names `flash_offset` `0x1000`;
|
||||
- an app with a flipped byte beside valid partitions, and a truncated dump,
|
||||
read with complete coverage and its missing partitions as entries with no
|
||||
bytes.
|
||||
|
||||
The lane ran on macOS arm64 with Node 24.22.1, Homebrew Clang and LLD 21.1.8
|
||||
and ESP-IDF v5.5.2's `gen_esp32part.py`, `otatool.py` and `parttool.py`,
|
||||
once with esptool v5.6.0 and once with 4.12.0. Its `gen_esp32part.py` is
|
||||
identical to v6.1's, and its otatool and parttool differ only in import style
|
||||
and esptool command spelling. All checks passed in both runs. With v5.6.0 and
|
||||
the build already cached, the run took 24.1 s elapsed, 16.9 s user CPU and
|
||||
6.3 s system CPU under `/usr/bin/time -l`; the largest single process peaked
|
||||
at a 278 MiB resident set, and the combined peak of all processes was not
|
||||
measured.
|
||||
|
||||
The lane does not establish:
|
||||
|
||||
- dumps read from a device: its dumps come from `merge-bin`;
|
||||
- chips other than ESP32-C3 and ESP32;
|
||||
- `ota_state` and record validity, which `read_otadata` does not print, or
|
||||
OTA states the bootloader writes itself, app rollback or anti-rollback;
|
||||
- selection with gaps or repeated OTA subtypes, which otatool does not
|
||||
write;
|
||||
- flash-encrypted dumps, which are not recognized by design.
|
||||
|
||||
It does not run in CI.
|
||||
@@ -1,333 +0,0 @@
|
||||
# ESP images
|
||||
|
||||
REA reads ESP32-family firmware images through `inspect-artifact` /
|
||||
`inspect_artifact` and `extract-artifact` / `extract_artifact`. OTA downloads,
|
||||
app partitions and second-stage bootloaders are all ESP images. Reading is
|
||||
static: no esptool, device or flash access is needed, and no code is run or
|
||||
disassembled.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./firmware.bin --json
|
||||
rea inspect-artifact ./ota.bin --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact ./firmware.bin --json
|
||||
```
|
||||
|
||||
For MCP, open the file with `open_binary`, then call `inspect_artifact`. The
|
||||
summary's `root_format_metadata` holds the image facts. Use its
|
||||
`parent_evidence_id` with `inspect_analysis_view` to select segment rows.
|
||||
`extract_artifact` writes one file per segment and per signature block.
|
||||
|
||||
For agent sessions, advertise the `session,artifact,application` tool families
|
||||
and consider setting both `REA_MCP_INPUT_SCHEMA_PROFILE` and
|
||||
`REA_MCP_OUTPUT_SCHEMA_PROFILE` to `compact`. The output profile bounds each
|
||||
advertised output schema to 4,096 bytes by relaxing nested client validation;
|
||||
REA still validates complete results against the canonical contracts. Pair
|
||||
this with retained summaries and selected inventory views to limit the context
|
||||
needed for one question. See [MCP contracts](mcp-contracts.md) for the profiles
|
||||
and their client compatibility tradeoffs.
|
||||
|
||||
The layout facts follow ESP-IDF v6.1 (`esp_app_format.h`, `esp_app_desc.h`,
|
||||
`esp_bootloader_desc.h`, `esp_image_format.c`, the ROM `secure_boot.h`
|
||||
headers and `docs/en/security/secure-boot-v2.rst`) and esptool v5.6.0
|
||||
(`bin_image.py`, `cmds.py`, `loader.py`, espsecure).
|
||||
|
||||
## Recognition
|
||||
|
||||
The container format is `esp-image`. A file is an ESP image when its bytes
|
||||
show all of these; the file name does not matter:
|
||||
|
||||
- magic `0xE9` at offset 0 and 1 to 16 segments. A count of 0 is valid but
|
||||
loads nothing, and accepting it would admit x86 programs that begin
|
||||
`E9 00`;
|
||||
- a flash mode and flash speed code that ESP-IDF's enums define;
|
||||
- a `hash_appended` byte of 0 or 1, as esptool requires;
|
||||
- when the file holds the first segment header, a segment length that is a
|
||||
multiple of 4.
|
||||
|
||||
Later segments, the checksum and the digest are integrity facts. A truncated
|
||||
download is still an ESP image, with its integrity reported as unavailable.
|
||||
|
||||
| Input | Outcome |
|
||||
| --------------------------------------------------- | ---------------------------------------------------------------- |
|
||||
| ESP8266 image (v1 or v3): same magic, 8-byte header | Refused: "ESP8266 firmware images are not supported…" |
|
||||
| Flash-encrypted image (ciphertext) | Not recognized: the bytes carry no magic. Unknown, not malformed |
|
||||
| ESP8266 v2 image (magic `0xEA`) | Not recognized |
|
||||
|
||||
An ESP8266 image has its first segment header at offset 8, and that segment
|
||||
loads into ESP8266 DRAM, IRAM (Arduino's eboot links at `0x4010F000`) or the
|
||||
IROM flash window, where v3 and Arduino app images begin. In an ESP32-family
|
||||
header the same bytes hold `wp_pin` and the drive settings, which esptool and
|
||||
ESP-IDF write as `0xEE` and zeros. Inside another container, an ESP8266 image
|
||||
is classified as a plain file.
|
||||
|
||||
Recognition cannot exclude every file that starts with an x86 near jump
|
||||
(`0xE9`), such as a DOS `.com` program. An explicit `--target-format dos-com`
|
||||
or `raw-binary` selection skips ESP recognition.
|
||||
|
||||
## Image facts
|
||||
|
||||
Root `format_metadata` has `kind: "esp-image"`:
|
||||
|
||||
| Field | Label | Meaning |
|
||||
| ------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
|
||||
| `header` | observed | Every `esp_image_header_t` field, with ESP-IDF's names, for example `entry_addr`, `chip_id`, `min_chip_rev_full` |
|
||||
| `chip` | derived | `target`, `isa` and the table's `source` (`ESP-IDF v6.1`) from `chip_id`; `null` for an ID the table lacks |
|
||||
| `checksum` | derived | `offset`, `stored`, `computed` and `status`: `valid`, `invalid` or `unavailable` |
|
||||
| `appended_sha256` | derived | The digest `hash_appended` adds: `offset`, `stored`, `computed` and `status`, including `not-appended` |
|
||||
| `image_end` | derived | End of the checksum block and any appended digest; `null` when the file ends before it |
|
||||
| `signature_sector_offset` | derived | The Secure Boot v2 signature sector, when it holds at least one valid block |
|
||||
| `undescribed_trailing` | derived | Byte ranges after the image that nothing describes; empty when every byte is described |
|
||||
| `descriptor` | observed | `esp_app_desc_t` or `esp_bootloader_desc_t` fields, with ESP-IDF's names, and the descriptor's file `offset` |
|
||||
|
||||
The descriptor's `kind`, derived from its magic, classifies the image as an
|
||||
`app` or a `bootloader`. Without a descriptor the role is unknown.
|
||||
|
||||
The ISA comes from `chip_id`, never from the series letter: ESP32-S31 is
|
||||
RISC-V. The chip table covers every `esp_chip_id_t` in ESP-IDF v6.1:
|
||||
|
||||
| `chip_id` | Target | ISA | `chip_id` | Target | ISA |
|
||||
| --------- | --------- | ------ | --------- | ---------- | ----- |
|
||||
| `0x0000` | `esp32` | xtensa | `0x0012` | `esp32p4` | riscv |
|
||||
| `0x0002` | `esp32s2` | xtensa | `0x0014` | `esp32c61` | riscv |
|
||||
| `0x0005` | `esp32c3` | riscv | `0x0017` | `esp32c5` | riscv |
|
||||
| `0x0009` | `esp32s3` | xtensa | `0x0019` | `esp32h21` | riscv |
|
||||
| `0x000C` | `esp32c2` | riscv | `0x001C` | `esp32h4` | riscv |
|
||||
| `0x000D` | `esp32c6` | riscv | `0x0020` | `esp32s31` | riscv |
|
||||
| `0x0010` | `esp32h2` | riscv | | | |
|
||||
|
||||
Other IDs, for example esptool v5.6.0's ESP32-E22 (`0x1F`), have `chip: null`
|
||||
and unclassified segments.
|
||||
|
||||
**Integrity.** The checksum is the XOR of every segment data byte, seeded with
|
||||
`0xEF`, in the last byte of the next 16-byte block. The appended SHA-256 covers
|
||||
every byte through the checksum. Both are facts, not admission rules: a
|
||||
mismatch is reported with a limitation and the segments stay readable.
|
||||
|
||||
**Descriptors.** Both descriptors begin segment 0: ESP-IDF's bootloader reads
|
||||
the app descriptor there, and esptool places the bootloader descriptor there.
|
||||
Strings stop at the first NUL and are omitted, with a limitation, when they
|
||||
are not valid UTF-8. The bootloader descriptor's magic is a single byte
|
||||
(`0x50`), so its reserved bytes, which ESP-IDF zero-fills, must also be zero.
|
||||
A `--ram-only-header` image puts RAM segments first, so its app descriptor is
|
||||
not in segment 0 and its role is unknown. `mmu_page_size` keeps ESP-IDF's
|
||||
name and value: the MMU page size as a base-2 logarithm, as
|
||||
`esp_app_desc_t` declares it, so `16` means 64 KiB. It is `0` in images built
|
||||
before ESP-IDF v5.4. `spi_flash_mode` is the next byte, which ESP-IDF v6.1
|
||||
names, kept as stored and not interpreted; releases that reserve it leave it
|
||||
`0`. A file can end inside the app descriptor: every field it holds in full
|
||||
is kept, a field it cuts off is `null`, and `observed_length` gives the
|
||||
descriptor bytes the file holds. The segment's truncation is reported as usual.
|
||||
|
||||
## Segments
|
||||
|
||||
Each header-declared segment is one entry named `segment-<index>.bin`, for
|
||||
example `segment-00.bin`, so two builds pair segments by position. Its
|
||||
`source_location` is the segment data in the image file. `format_metadata`
|
||||
(`esp-image-segment`) records `index`, `load_addr`, `data_len`,
|
||||
`header_offset` and `classification`.
|
||||
|
||||
`classification` follows ESP-IDF's bootloader (`should_map` and
|
||||
`should_load` in `esp_image_format.c`) on a cold boot:
|
||||
|
||||
| Value | Meaning |
|
||||
| ------------ | ------------------------------------------------------------------------------------------------- |
|
||||
| `mapped` | Inside the chip's IROM, DROM or, where the MMU is per memory type, PSRAM range (`soc.h`) |
|
||||
| `loaded` | `should_load` selects it: not mapped, at or above `0x10000000`. RAM ranges are not checked |
|
||||
| `not-loaded` | Below `0x10000000`: the padding segments esptool writes at address 0, including `--secure-pad-v2` |
|
||||
| `unknown` | The chip is unknown, or the mapping depends on the bootloader's MMU page size |
|
||||
|
||||
`loaded` does not mean the bootloader accepts the segment. Before loading,
|
||||
ESP-IDF's `verify_load_addresses` rejects the image when a loaded segment is
|
||||
not entirely in the chip's DRAM, IRAM, RTC memory or SPM, or overlaps the
|
||||
bootloader's own stack, data or loader code. For example, an ESP32-C6
|
||||
segment at `0x43000000` is `loaded` here and rejected there. REA models no
|
||||
RAM ranges.
|
||||
|
||||
Padding segments are inventoried and extractable, but are never load regions.
|
||||
When waking from deep sleep, ESP-IDF also skips RTC-memory segments; that
|
||||
runtime case is not modelled. ESP32-C6 and ESP32-H2 map flash from
|
||||
`0x42000000` through 256 pages of the bootloader's configured MMU page size,
|
||||
which no image records. The app descriptor's page size describes the app's
|
||||
build, not the bootloader's. With page sizes from 8 KiB (which the hardware
|
||||
supports and esptool builds for) to 64 KiB, segments from `0x42200000` to
|
||||
`0x43000000` are `unknown`. Ranges follow ESP-IDF's `should_map`, not esptool's
|
||||
wider placement ranges.
|
||||
|
||||
A segment cut short by the end of the file is the bytes the file holds, with
|
||||
an entry limitation that gives the declared length.
|
||||
|
||||
Artifact comparison uses each segment's `load_addr` and `data_len` as its
|
||||
declaration, so a segment that moves with unchanged bytes is `changed` in
|
||||
`metadata`. Segments pair by index: inserting or removing one shifts the
|
||||
paths of the segments after it. Header offsets and `classification` are not
|
||||
compared; the image's own change covers the header and chip. The image's and
|
||||
the signature blocks' `format_metadata` carry no declaration: their fields
|
||||
come from their own bytes, which content comparison covers, and a block's
|
||||
`image_digest_matches` follows the image's content.
|
||||
|
||||
## Secure Boot v2 signatures
|
||||
|
||||
A signature sector starts at the next 4 KiB boundary after the image and
|
||||
holds up to three 1216-byte slots. A slot holds a block when it has the `0xE7`
|
||||
magic and a matching CRC32 at offset 1196, the rule ESP-IDF and espsecure
|
||||
apply. Each slot is examined independently, so an erased or damaged slot does
|
||||
not hide a later block. Each block is an entry, `signature-block-<slot>.bin`,
|
||||
with `esp-image-signature-block` metadata:
|
||||
|
||||
| Field | Meaning |
|
||||
| ---------------------------------- | ------------------------------------------------------------------------- |
|
||||
| `index`, `version`, `scheme` | Slot 0 to 2, version byte, and `rsa` (`0x02`) or `ecdsa` (`0x03`) |
|
||||
| `sha_version`, `curve_id` | For ECDSA, the third byte (0 is SHA-256, 1 SHA-384) and the curve ID byte |
|
||||
| `digest_algorithm`, `image_digest` | The digest the block records |
|
||||
| `image_digest_matches` | Whether it equals the digest of every byte before the sector |
|
||||
| `public_key_sha256` | SHA-256 of the block's public key, the value eFuse key digests hold |
|
||||
| `signature_status` | Always `unverified` |
|
||||
|
||||
Signatures are observed, not verified: REA checks no public key, signature or
|
||||
eFuse. `public_key_sha256` is the digest espsecure `signature-info-v2`
|
||||
prints, so it can be compared with `espefuse summary`. An unknown version or
|
||||
`sha_version` leaves the layout-dependent fields `null`. A slot with the magic
|
||||
but no matching CRC32 is not a block: it is reported in a limitation, whether
|
||||
or not another slot holds a block, and its bytes are undescribed. Secure Boot
|
||||
v1 signatures are not decoded.
|
||||
|
||||
## Coverage and limitations
|
||||
|
||||
Bytes after the image are described when they are erased flash (`0xFF`) or a
|
||||
signature sector's blocks; erased slots and slack in that sector count as
|
||||
erased flash. Every other byte is in `undescribed_trailing`.
|
||||
|
||||
| Situation | Outcome |
|
||||
| ---------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
|
||||
| Segment data or a segment header beyond the end of the file | Truncation limitation; checksum and SHA-256 `unavailable`; coverage `unknown` |
|
||||
| File ends before the checksum or the appended digest | That status is `unavailable`, with a limitation; coverage `unknown`, since a signature sector could follow |
|
||||
| Checksum or SHA-256 mismatch | `invalid`, with a limitation; coverage is unchanged |
|
||||
| Erased flash after the image that runs past the three slots | Described; no limitation |
|
||||
| Erased flash that ends before the three slots are complete | Limitation; coverage `unknown`, since a signed image cut there would have lost its sector |
|
||||
| A signature sector the file cuts short | Limitation; coverage `unknown` |
|
||||
| Any other bytes after the image | Ranges in `undescribed_trailing`, a limitation, and coverage `unknown` |
|
||||
| Segment length that is not a multiple of 4 | Limitation: ESP-IDF rejects such an image |
|
||||
| Segment of 16 MiB or more, except on chips with 32-bit cache addresses | Limitation: ESP-IDF rejects such an image |
|
||||
|
||||
ESP32-S3, ESP32-P4, ESP32-C5, ESP32-C61, ESP32-H4 and ESP32-S31 map flash
|
||||
through 32-bit cache addresses (`SOC_SPI_MEM_SUPPORT_CACHE_32BIT_ADDR_MAP`), so
|
||||
ESP-IDF sets no 16 MiB segment limit for them. For an unknown chip the limit
|
||||
is not applied.
|
||||
|
||||
Undescribed bytes include other flash contents such as an app partition's
|
||||
stale data, a Secure Boot v1 signature, and the flash segments that esptool
|
||||
`--ram-only-header` writes after the checksum without counting them. REA
|
||||
reports their ranges and does not guess their structure. In the padding, in a
|
||||
sector without a valid block, in the sector's slack and after the sector, the
|
||||
first byte that is not erased starts a range to the end of that region. A
|
||||
slot that is neither a block nor erased is reported whole. Adjacent ranges
|
||||
are merged.
|
||||
|
||||
A whole-flash dump whose partition table REA finds is read as an
|
||||
[ESP flash image](esp-flash-image.md). Without one, a dump of a chip whose
|
||||
bootloader is at offset 0, such as ESP32-C3 or ESP32-S3, is recognized as that
|
||||
bootloader image, and the partition table, apps and data follow as
|
||||
undescribed bytes.
|
||||
|
||||
A dump read from a later flash address, such as an ESP32 dump read from
|
||||
`0x1000` with `read-flash 0x1000 …` or `merge-bin --target-offset 0x1000`,
|
||||
opens as its bootloader image. REA says so when the file shows it: the image
|
||||
at file offset 0 is not an app, its chip's ROM loads the bootloader from a
|
||||
later address (`0x1000` for ESP32 and S2, `0x2000` for C5, H4, P4 and S31),
|
||||
and a partition table sits where that origin puts `0x8000` (file offset
|
||||
`0x7000` for `0x1000`), after the image ends. Then `flash_dump` records
|
||||
`flash_offset` and
|
||||
`table_file_offset`, a limitation says to reopen the file with
|
||||
`format: "esp-flash-image"` and that `flash_offset`, and `inspect_artifact`
|
||||
lists the same `open_binary` call as a next probe. `flash_dump` is null for
|
||||
every other image. Evidence that REA recorded before it reported the field
|
||||
omits it; REA still reads and compares such Evidence, and the omission says
|
||||
nothing about where the file was read from.
|
||||
|
||||
## Limits
|
||||
|
||||
An image has at most 16 segments and 3 signature blocks. Decoding reads each
|
||||
byte once and retains only fixed-size fields. It reads through erased flash to
|
||||
find the first byte that is not erased, and stops there. There is no
|
||||
file-size limit.
|
||||
|
||||
## Analyzing a segment
|
||||
|
||||
Inventory does not disassemble code or build a memory map. `open_binary`
|
||||
admits neither RISC-V nor Xtensa as a raw-binary architecture, so a segment
|
||||
cannot yet be opened as a raw image. Each segment's `source_location` and
|
||||
`load_addr` give the file offset and address to map it at by hand, for
|
||||
example with the Rizin tool's `om` command.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic fixtures with computed checksums, digests and CRCs cover
|
||||
recognition, ESP8266 layouts, the ESP32, ESP32-C3, ESP32-C6 and ESP32-P4
|
||||
memory maps, truncation at each structure, integrity mismatches, erased and
|
||||
undescribed trailing bytes, signature slots including damaged and unknown
|
||||
layouts, and the refusals. CLI and stdio MCP tests exercise the same artifact
|
||||
provider and advertised schemas.
|
||||
|
||||
The optional real lane builds ELF files from source, converts them with
|
||||
esptool and compares REA's facts with `esptool image-info`:
|
||||
|
||||
```sh
|
||||
REA_RISCV_CLANG=/path/to/clang REA_RISCV_LLD=/path/to/ld.lld \
|
||||
REA_ESPTOOL_PYTHON=/path/to/python-with-esptool \
|
||||
npm run verify:esp-images
|
||||
```
|
||||
|
||||
It needs a Clang with the RISC-V target, `ld.lld` and a Python that can run
|
||||
`esptool` and `espsecure`. It checks each tool before building anything and
|
||||
names the variable to set when one is missing, and it warns when esptool is
|
||||
not v5.6.0, the version whose sources the layout facts cite. It builds an
|
||||
ESP32-C3 app, the same app with `--dont-append-digest`, `--secure-pad-v2`,
|
||||
`--ram-only-header` and `--pad-to-size`, a bootloader, and ESP8266 v1 and v3
|
||||
images. It then checks:
|
||||
|
||||
- entry point, chip ID, each segment's length, address and header offset,
|
||||
checksum, digest and descriptor fields against `image-info`;
|
||||
- each segment's classification in the app, its variants and the
|
||||
bootloader, whose linker scripts fix every section's region, including the
|
||||
`not-loaded` padding that `--secure-pad-v2` adds;
|
||||
- after signing with RSA-3072 and appending ECDSA P-256 and P-384 blocks,
|
||||
each block's `public_key_sha256` against espsecure `signature-info-v2`,
|
||||
and that its recorded digest matches;
|
||||
- extraction of the app and the signed image: each segment file equals the
|
||||
bytes after esptool's segment header offset, each signature block equals
|
||||
its slot in the sector espsecure appended, and the source's SHA-256 is
|
||||
unchanged and equals the reported `root_sha256`;
|
||||
- the `--pad-to-size` erased tail is described; the `--ram-only-header`
|
||||
flash segments are undescribed;
|
||||
- a flipped checksum, a flipped digest byte, a truncated segment (where
|
||||
`image-info` stops with "End of file reading segment"), and both ESP8266
|
||||
refusals.
|
||||
|
||||
`REA_XTENSA_ESP32_GCC` adds an ESP32 (Xtensa) app built with Espressif's
|
||||
`xtensa-esp32-elf-gcc`. Without it the report marks that image as skipped.
|
||||
|
||||
Each command runs as a process tree owned by the shared verifier command
|
||||
owner, so a timeout also stops the linker or assembler a compiler driver
|
||||
starts. The lane removes its workspace only after every tree has stopped and
|
||||
keeps it when that cleanup cannot be confirmed.
|
||||
|
||||
The lane ran on macOS arm64 with Node 24.22.1, Homebrew Clang and LLD 21.1.8,
|
||||
and `xtensa-esp32-elf-gcc` 14.2.0 (crosstool-NG esp-14.2.0_20251107), once
|
||||
with esptool and espsecure v5.6.0 and once with 4.12.0. All checks passed for
|
||||
all 13 images in both runs. With v5.6.0 the complete run, including the builds
|
||||
and every esptool call, took 21.7 s elapsed, 16.9 s user CPU and 6.1 s system
|
||||
CPU under `/usr/bin/time -l` with warm caches. The largest single process
|
||||
peaked at a 253 MiB resident set; the combined peak of all processes was not
|
||||
measured.
|
||||
|
||||
The lane does not establish:
|
||||
|
||||
- the other chips: only ESP32-C3 (RISC-V) and ESP32 (Xtensa) images come
|
||||
from a real producer; the other rows of the chip table are checked against
|
||||
the ESP-IDF v6.1 headers by reading them;
|
||||
- images from a real ESP-IDF project build, an OTA download or a device's
|
||||
flash, none of which it inspects;
|
||||
- that a signature is valid: it compares digests, and nothing verifies a
|
||||
signature.
|
||||
|
||||
It does not run in CI.
|
||||
+2
-13
@@ -28,15 +28,6 @@ The host must already provide util-linux `prlimit`, normally `/usr/bin/prlimit`;
|
||||
no system tools and changes no user configuration. [Upstream provenance and
|
||||
license](https://github.com/morluto/rea/blob/main/third_party/evmole/README.md) are packaged.
|
||||
|
||||
Limiter discovery validates the selected path without launching it. A relative
|
||||
path or directory is reported as `not_configured`, a missing file as
|
||||
`executable_missing`, and denied access or another filesystem-probe failure as
|
||||
`runtime_unavailable`. Recovery names the selected path and
|
||||
`REA_EVM_PRLIMIT_COMMAND`; denied access asks for execute permission and
|
||||
filesystem access. The original admission or system code stays in diagnostics.
|
||||
An available filesystem probe does not verify the selected tool's version or
|
||||
util-linux compatibility, and REA does not substitute another command.
|
||||
|
||||
## Evidence semantics
|
||||
|
||||
- `encoding` is required: raw bytes or UTF-8 hex, with optional `0x` prefix and
|
||||
@@ -89,10 +80,8 @@ without bytecode text or duplicated raw stdout:
|
||||
}
|
||||
```
|
||||
|
||||
The result is distinct view Evidence linked to its complete parent. Like the
|
||||
parent, it has `confidence: inferred`; projection never strengthens inferred
|
||||
candidates. Its `normalized_result.summary` preserves every producer-returned
|
||||
selector, decoded
|
||||
The result is distinct derived Evidence linked to its complete parent. Its
|
||||
`normalized_result.summary` preserves every producer-returned selector, decoded
|
||||
byte offset, dispatch, argument/mutability string including null, carrier/decoded
|
||||
identities, original limitations and explicit unknowns. Payload `evidence_kind`
|
||||
remains `inferred`. It preserves stderr/truncation and records stdout text length.
|
||||
|
||||
@@ -26,9 +26,6 @@ REA does not install these tools, their dependencies or system packages. It
|
||||
checks the tool's `--version` banner when executing a request. Binwalk must
|
||||
print `binwalk <version>` and Unblob must print the version alone. Inspection and
|
||||
extraction have independent prerequisites; one does not require the other.
|
||||
Capability discovery recommends `REA_BINWALK_COMMAND` for missing inspection
|
||||
and `REA_UNBLOB_COMMAND` for missing extraction. Configuring the other engine
|
||||
does not enable the unavailable operation.
|
||||
Both require util-linux `prlimit`, normally `/usr/bin/prlimit`; override its
|
||||
absolute path with `REA_FIRMWARE_PRLIMIT_COMMAND` if needed. `rea doctor` does
|
||||
not currently inspect firmware readiness. Engine selection/version checks run
|
||||
@@ -85,11 +82,6 @@ Published file `relative_path` preserves producer spelling for lineage; `path`
|
||||
uses the writer's actual normalized filename. An extracted filename that cannot
|
||||
be represented losslessly as UTF-8 fails publication explicitly with its raw
|
||||
name bytes in hexadecimal instead of silently omitting the entry.
|
||||
Unpublished symlinks retain the extracted tree's target bytes in lowercase
|
||||
`link_target_hex`; `link_target` is null when those bytes are not valid UTF-8.
|
||||
Older retained Evidence may omit the hexadecimal field. Unblob may have rewritten
|
||||
the target before REA observed it, so these bytes are not necessarily those stored
|
||||
in the firmware image.
|
||||
|
||||
## CLI and MCP
|
||||
|
||||
@@ -294,68 +286,9 @@ REA_FIRMWARE_VERIFY_GHIDRA=1 npm run verify:firmware
|
||||
generated gzip/USTAR/ext4 with a Linux x64 ELF child; SquashFS, UBIFS, proprietary
|
||||
firmware, other hosts and architectures remain unverified.
|
||||
|
||||
## Inspect ESP images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on an ESP32-family app or
|
||||
bootloader image, such as an OTA download or app partition, for its chip,
|
||||
classified segments, checksum and SHA-256 results, descriptor and Secure Boot
|
||||
v2 signature blocks. `extract-artifact` / `extract_artifact` writes one file
|
||||
per segment and signature block. Reading is portable and needs no external
|
||||
tool. See [ESP images](esp-image.md).
|
||||
|
||||
## Inspect ESP flash dumps
|
||||
|
||||
Use the same tools on a whole-flash dump, such as `esptool read-flash 0 ALL`
|
||||
output. The inventory reports the bootloader, the partition table and how it
|
||||
was found, each partition with its declared type, flags and digest, ESP image
|
||||
facts for the bootloader and each app partition, and the OTA records with a
|
||||
derived selection; the running partition stays unknown. Extraction writes
|
||||
each partition, `nvs_keys` included, as an ordinary file. For a dump that
|
||||
does not start at flash address 0, select `esp-flash-image` with
|
||||
`flash_offset`. See [ESP flash images](esp-flash-image.md).
|
||||
|
||||
## Inspect extracted devicetrees
|
||||
|
||||
Use [`inspect_devicetree`](devicetree.md) / `rea inspect-devicetree` on an
|
||||
extracted DTB for its node/property tree, reservations, hardware summary and
|
||||
selected phandle relationships. FDT v16/v17 inspection is portable and requires
|
||||
no external tool or disassembler.
|
||||
|
||||
## Inspect Intel HEX and S-record images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on an Intel HEX or Motorola S-record
|
||||
file for its address regions, declared start address and record facts, then
|
||||
`extract-artifact` / `extract_artifact` for one raw file per region. Decoding is
|
||||
portable and requires no external tool. See
|
||||
[Intel HEX and S-record images](hex-records.md).
|
||||
|
||||
## Inspect U-Boot legacy uImages
|
||||
|
||||
`inspect-artifact` / `extract-artifact` recognize a legacy uImage by its
|
||||
`0x27051956` magic and inventory it without running or decompressing it.
|
||||
The root facts record the 64-byte header (architecture, OS, type, and
|
||||
compression as raw values and names, load and entry addresses, image name,
|
||||
and timestamp) with the header and data CRC32 verified; the stored payload
|
||||
is extractable as `payload.bin`, and multi-file and script images additionally
|
||||
expose each sub-image as `image-<n>.bin` from the stored size table. Declared
|
||||
addresses are facts from the header, not proven entry points, and compressed
|
||||
payloads stay as stored. Intel HEX/S-record and FIT images are separate
|
||||
readers and are not covered here. See [U-Boot legacy uImage](uimage.md) for
|
||||
component padding, preserved names, resource policies and verification scope.
|
||||
|
||||
## Inspect UF2 and DfuSe images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on a UF2 file for each familyID's
|
||||
address regions and its block numbering, or on a DfuSe file for its targets,
|
||||
elements and DFU suffix, then `extract-artifact` / `extract_artifact` for one
|
||||
raw file per region or element. A DfuSe suffix CRC mismatch follows
|
||||
`integrity_policy`. Decoding is portable and requires no external tool. See
|
||||
[UF2 and DfuSe images](uf2-dfuse.md).
|
||||
|
||||
## Inspect U-Boot FIT images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on a U-Boot FIT (`.itb`) for its
|
||||
images, declared load and entry addresses, configurations and verified hashes,
|
||||
then `extract-artifact` / `extract_artifact` for one raw file per image.
|
||||
Decoding is portable and requires no external tool. See
|
||||
[U-Boot FIT images](fit-images.md).
|
||||
@@ -1,172 +0,0 @@
|
||||
# U-Boot FIT images
|
||||
|
||||
REA inventories U-Boot Flattened Image Tree (FIT, `.itb`) files through
|
||||
`inspect-artifact` / `inspect_artifact`, and materializes the stored bytes of
|
||||
each image through `extract-artifact` / `extract_artifact`. Decoding is static
|
||||
and needs no external tool.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./boot.itb --json
|
||||
rea inspect-artifact ./boot.itb --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact ./boot.itb --json
|
||||
```
|
||||
|
||||
For MCP, open the file with `open_binary`, then call `inspect_artifact`. Summary
|
||||
delivery retains the complete Evidence; use `parent_evidence_id` with
|
||||
`inspect_analysis_view` to read selected images and fields. `extract_artifact`
|
||||
returns a fresh `output_root` containing one raw file per image. FIT summaries
|
||||
use the current projection identity; there is no pre-coverage FIT summary variant.
|
||||
|
||||
## Recognition
|
||||
|
||||
A FIT is a flattened devicetree (FDT v16 or v17) with a top-level `/images`
|
||||
node. Recognition walks the structure block within the first 4 KiB of the file
|
||||
only; the file name does not matter. A devicetree whose `/images` node begins
|
||||
after 4 KiB, a devicetree without `/images`, and any other FDT version stay on
|
||||
the plain devicetree route: `open_binary` refuses them and
|
||||
[`inspect_devicetree`](devicetree.md) still parses them. A U-Boot control DTB
|
||||
that contains `/images` is treated as a FIT. The container format is `fit`.
|
||||
|
||||
Recognition is a probe. The complete devicetree parse decides, so a file whose
|
||||
prefix declares `/images` but whose structure is malformed fails as a
|
||||
malformed FIT rather than falling back to plain classification.
|
||||
|
||||
## Images and metadata
|
||||
|
||||
Each child of `/images` is one inventory entry named `images/<unit-name>.bin`,
|
||||
listed in devicetree order. Its bytes come from exactly one data source:
|
||||
|
||||
| Source | Stored bytes |
|
||||
| --------------- | --------------------------------------------------------------------------------------- |
|
||||
| `data` | The embedded property value |
|
||||
| `data-offset` | `data-size` bytes at `data-offset` past the devicetree end, aligned to 4 (`mkimage -E`) |
|
||||
| `data-position` | `data-size` bytes at `data-position` from the start of the file (`mkimage -p`) |
|
||||
|
||||
`data-position` is read as U-Boot reads it: an offset from the start of the
|
||||
FIT, not a machine address.
|
||||
|
||||
Image `format_metadata` (`fit-image`) records `index`, `name`, `description`,
|
||||
`type`, `arch`, `os`, `compression`, `load` and `entry`, the `data_source`,
|
||||
the raw `declared_offset` for external data, `image_data` (the shared-data
|
||||
link), `cipher_present`, `cipher_algo`, `hash_algorithms`, `hashes_verified`,
|
||||
`hashes_unverified`, `signature_count`, `signatures_unreadable` and
|
||||
`signature_algorithms`. Algorithm lists are arrays. Nullable declaration fields
|
||||
are absent unless named in `unreadable_fields`, which records parsing failures
|
||||
per field. Hash or signature warnings do not make unrelated declarations unknown.
|
||||
`load` and `entry` fold one or two 32-bit cells into a lowercase
|
||||
`0x…` value, as U-Boot does, independent of `#address-cells`.
|
||||
|
||||
Root `format_metadata` (`fit`) records `fdt_version`, `fdt_totalsize`,
|
||||
`external_data_base` (the aligned devicetree end), the root `description`,
|
||||
`timestamp` (seconds since 1970) and `address_cells`, plus
|
||||
`default_configuration` and `configurations`. A missing description or
|
||||
timestamp is `null`.
|
||||
|
||||
Image bytes are reported as stored. Compressed images are not decompressed and
|
||||
ciphered images are not decrypted: entries report `encrypted: false` and the
|
||||
cipher declaration in `cipher_present` and algorithm in `cipher_algo`. A missing
|
||||
or malformed algorithm stays `null`; it does not establish unciphered bytes.
|
||||
Embedded `image-data` links require byte-identical stored properties; external
|
||||
links require matching physical ranges.
|
||||
|
||||
The extracted file holds the stored bytes. Direct-analysis suggestions require
|
||||
`compression: "none"` and `cipher_present: false`.
|
||||
|
||||
## Hashes and signatures
|
||||
|
||||
Hash subnodes are children whose name starts with `hash`. Inventory verifies
|
||||
`crc32` (4 bytes, big-endian), `md5`, `sha1`, `sha256`, `sha384` and `sha512`
|
||||
over the stored bytes, the same bytes mkimage hashed after any ciphering. All
|
||||
hashes of an image are computed in one streamed pass while the inventory is
|
||||
prepared, so a mismatch fails the whole inventory, naming the image, hash
|
||||
node, algorithm, and the declared and computed values. A supported algorithm
|
||||
whose value has the wrong length also fails.
|
||||
|
||||
Other algorithms such as `crc16-ccitt`, `crc8` or `sm3_256`, a hash without a
|
||||
readable `algo`, and a hash without a `value` are not verified; each is
|
||||
counted in `hashes_unverified` and reported as an image limitation.
|
||||
|
||||
Signature subnodes (names starting with `signature`) on images and
|
||||
configurations retain the total declaration count, readable algorithms and
|
||||
`signatures_unreadable`. Missing or malformed algorithms carry source limitations
|
||||
so unknown declarations remain distinguishable from absence. Signatures are never
|
||||
verified.
|
||||
|
||||
## Configurations
|
||||
|
||||
`/configurations` is optional. Each child records its `name`, `description`,
|
||||
and the image references `kernel`, `firmware`, `fdt` (base plus overlays),
|
||||
`ramdisk` and `loadables`, always as lists, plus its signature algorithms.
|
||||
`compatible`, `fpga`, `script`, `cmdline` and `load-only` are not reported. A
|
||||
reference to a name that is not an `/images` child, and a `default` that names
|
||||
no configuration, are reported as limitations and kept as declared. Each
|
||||
configuration carries `unresolved_reference_count`; dangling references share one
|
||||
configuration-level diagnostic while every original reference remains available.
|
||||
|
||||
## Facts, limitations and failures
|
||||
|
||||
| Situation | Outcome |
|
||||
| ---------------------------------------------------------------------------------------------- | ------------------------------------ |
|
||||
| Malformed devicetree, truncated header, `totalsize` past the end of the file | Fails as malformed |
|
||||
| No data source, more than one data source, external data without `data-size` | Fails as malformed |
|
||||
| `data-offset`, `data-position` or `data-size` not one 32-bit cell, `image-data` not one string | Fails as malformed |
|
||||
| External data outside the file, overlapping the devicetree or another image | Fails as malformed |
|
||||
| Two images with an identical external range linked by `image-data` | Accepted as shared data |
|
||||
| A dangling, chained or mismatched `image-data` link | Fails as malformed |
|
||||
| Hash mismatch, or a supported hash value with the wrong length | Fails integrity |
|
||||
| Unsupported, algorithm-less or value-less hashes | Unverified, reported as a limitation |
|
||||
| Malformed `description`, `type`, `arch`, `os`, `compression`, `load`, `entry` or `timestamp` | `null`, reported as a limitation |
|
||||
| Dangling configuration reference, missing default configuration | Reported as a limitation |
|
||||
|
||||
Failures apply to the whole file and return no partial inventory.
|
||||
|
||||
## Limits
|
||||
|
||||
The devicetree, including any embedded image data, must be at most 16 MiB; a
|
||||
larger `totalsize` fails as a memory limit. Store large images externally
|
||||
(`mkimage -E`): external data of any size is streamed from the file and never
|
||||
loaded into memory. A FIT may declare at most 1,024 images and 1,024
|
||||
configurations, and an image at most 8 hash nodes; the devicetree parser's
|
||||
node, property and depth bounds also apply. The sum of declared image sizes
|
||||
is bounded by the shared archive expansion budget. These are fixed resource
|
||||
bounds with no runtime override. Configurable ceilings are tracked in
|
||||
[#6176](https://github.com/morluto/rea/issues/6176).
|
||||
|
||||
## Analyzing an image
|
||||
|
||||
Inventory does not execute anything, and `load` and `entry` are declarations.
|
||||
After extraction:
|
||||
|
||||
- inspect `flat_dt` images with [`inspect_devicetree`](devicetree.md);
|
||||
- open an uncompressed image with `open_binary` using format `raw-binary`, an
|
||||
architecture matching its declared `arch`, and `entry_point` set to its
|
||||
declared `load` address; raw-binary maps file offset 0 and seeds analysis at
|
||||
`entry_point`. Automatic suggestions require nonempty bytes, an admitted
|
||||
architecture/address/extent, and a declared entry that is absent or equal to
|
||||
load. Distinct or unreadable entry declarations remain in Evidence without
|
||||
a raw-loader suggestion, since this loader cannot represent separate mapping
|
||||
and analysis entry addresses.
|
||||
|
||||
## Comparison
|
||||
|
||||
Artifact comparison reports changes to an image's payload interpretation,
|
||||
including load and entry declarations. Equivalent external offset forms and
|
||||
inventory positions leave those declarations unchanged. Unavailable
|
||||
interpretations remain unknown, while independently known changes survive.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic fixtures with computed hashes cover embedded, `data-offset` and
|
||||
`data-position` layouts, shared data, overlap and bounds failures, every
|
||||
verified hash algorithm, configurations, resource limits, inventory,
|
||||
extraction and reader lifecycle. CLI and stdio MCP tests exercise the same
|
||||
artifact provider and advertised schemas. Real U-Boot `mkimage` 2023.01 output
|
||||
has also been verified for embedded, `-E -B 0x200`, and
|
||||
`-E -p 0x2000 -B 0x200` layouts, with byte-exact extraction checked against
|
||||
`dumpimage`. The OpenWrt 24.10.3 ABT ASR3000 recovery FIT was checked against
|
||||
`mkimage`, `fdtget` and `dumpimage`, including its kernel, initrd and devicetree.
|
||||
No image was executed.
|
||||
|
||||
For small containers, complete delivery avoids an additional view request.
|
||||
Choose summary and selected views when the omitted facts justify that extra
|
||||
request; summary delivery does not reduce the underlying inventory work.
|
||||
@@ -109,13 +109,9 @@ server; older servers keep the complete request after schema discovery.
|
||||
header check does not establish valid table contents or exported symbols.
|
||||
- **`libflutter` facts** are per-ABI observations: byte size, SHA-256
|
||||
digest, GNU build IDs from ELF-declared `PT_NOTE` / `SHT_NOTE` ranges,
|
||||
and Dart/toolchain strings the build carries. `dart_version` accepts the
|
||||
labeled `Dart VM version:` form and the unlabeled Dart
|
||||
`Version::String()` form (`<version> (<channel>) (<commit time>)` with an
|
||||
optional ` on "<os>_<arch>"` suffix) that release engines embed. An
|
||||
unlabeled match is a candidate based on its shape; it is not bound to a
|
||||
Dart symbol and another string can match. `null` means no supported pattern
|
||||
was found, not proof that no version exists. Android clang/LLVM toolchain
|
||||
and any _labeled_ strings the build
|
||||
carries. Release engine builds often strip the `Dart VM version` string —
|
||||
absence is reported as `null`, not guessed. Android clang/LLVM toolchain
|
||||
lines are reported verbatim as build provenance.
|
||||
`build_id_candidates` preserves distinct IDs. `build_id` is populated only
|
||||
when all declared note ranges were examined and one ID was found;
|
||||
@@ -126,13 +122,10 @@ server; older servers keep the complete request after schema discovery.
|
||||
Compressed `SHT_NOTE` sections are skipped with an unsupported-compression
|
||||
limitation and partial coverage; IDs from other note ranges remain candidates.
|
||||
Encoded note storage is never interpreted as note headers or inflated.
|
||||
- Version and toolchain strings must fit their complete grammar.
|
||||
`dart_version_capped` and `toolchain_capped` independently report labels
|
||||
omitted by grammar or projection bounds; either makes identification
|
||||
coverage partial. Values split across read chunks are retained without
|
||||
publishing truncated prefixes. Unlabeled runs that miss the
|
||||
`Version::String()` grammar are ignored and do not set
|
||||
`dart_version_capped`.
|
||||
- Labeled strings must fit their complete grammar. `dart_version_capped`
|
||||
and `toolchain_capped` independently report labels omitted by grammar or
|
||||
projection bounds; either makes identification coverage partial. Values
|
||||
split across read chunks are retained without publishing truncated prefixes.
|
||||
- **`coverage: "partial"`** also appears when an ABI carries `libapp.so`
|
||||
without `libflutter.so` (or vice versa).
|
||||
- The APK itself is digested before parsing, so `target.sha256` binds the
|
||||
|
||||
+1
-1
@@ -119,7 +119,7 @@ installation's language catalog, the same catalog REA uses to admit
|
||||
and still succeeds on the stock checks.
|
||||
|
||||
The ELF lane (`npm run verify:ghidra:ppc`) is separate. See
|
||||
[PowerPC ELF analysis through Ghidra](ghidra-ppc.md).
|
||||
[PowerPC ELF analysis through Ghidra](/ghidra-ppc).
|
||||
|
||||
## Not supported yet
|
||||
|
||||
|
||||
+15
-20
@@ -39,9 +39,9 @@ bridge-reported `language_id` after import. A caller-selected
|
||||
pin; combining the pinned profile with an environment override is a typed
|
||||
refusal. A header that declares an ABI field but carries no record is still
|
||||
refused. Two further boundaries apply: an inspection that could not examine
|
||||
every ABI declaration source does not establish the recordless admission
|
||||
(target opening already refuses a carved ELF whose section table extends past
|
||||
the captured bytes, as described below), and an `EF_MIPS_ARCH_1` or
|
||||
every ABI declaration source (for example a carved ELF whose section table
|
||||
extends past the captured bytes without a program-header record) does not
|
||||
establish the recordless admission, and an `EF_MIPS_ARCH_1` or
|
||||
`EF_MIPS_ARCH_32` header that _declares_ an ABI is refused rather than
|
||||
relabeled, because the declared-ABI lane is verified for
|
||||
`EF_MIPS_ARCH_32R2` headers only. The recordless MIPS-I lane pins the same
|
||||
@@ -77,16 +77,13 @@ argument ABI and unknown nonzero MSA values remain unsupported. Without a
|
||||
separate ABI flags record, FP attribute values are admitted only when absent,
|
||||
`ANY` (0), double precision (1), or FPXX (5), the values covered by the
|
||||
recordless profile. Parsed values are retained in the MIPS analysis profile.
|
||||
A carved ELF whose section header table extends beyond the captured file is
|
||||
refused when the target is opened, before provider selection, as is every ELF
|
||||
declaration beyond EOF ([installation](installation.md#ghidra)). The refusal
|
||||
names the declared range and the file size. It applies even when a
|
||||
program-header ABI record was captured: an uncaptured section header could
|
||||
carry an unsupported or conflicting declaration, so the captured bytes are not
|
||||
analyzed as a complete smaller program. The provider also refuses any ABI
|
||||
declaration inspection that reports unexamined headers. A missing
|
||||
program-header record does not make ABI declarations in uncaptured sections
|
||||
absent or supply a default ABI.
|
||||
For carved ELFs whose section table extends
|
||||
beyond the captured file, REA examines every complete captured section header
|
||||
and records the unavailable headers as a limitation. The provider refuses an
|
||||
incomplete ABI declaration inspection even when a program-header ABI record is
|
||||
valid, because an unavailable section header could carry an unsupported or
|
||||
conflicting declaration. A missing program-header record does not make ABI
|
||||
declarations in uncaptured sections absent or supply a default ABI.
|
||||
The field layout follows
|
||||
[`Elf_MIPS_ABIFlags_v0`](https://github.com/bminor/glibc/blob/master/elf/elf.h).
|
||||
|
||||
@@ -260,13 +257,11 @@ observations and does not relabel the upstream result as verified.
|
||||
|
||||
CLI/MCP discovery and decompilation, direct calls, target/profile/Evidence
|
||||
identity, snapshot round-trip and final process cleanup remain covered.
|
||||
For both byte orders, the lane also cuts the image one byte before its section
|
||||
header table, after checking that every program header and payload stays
|
||||
captured. `open_binary` and the CLI must both refuse that copy before provider
|
||||
selection as `target_unavailable` with constraint `file_format_rejected`,
|
||||
naming the declared section-table range and the captured size, and leave it
|
||||
unchanged. Caller-selected Ghidra heap limits are passed to the lane's CLI and
|
||||
MCP child workflows.
|
||||
For both byte orders, the lane also carves away the section table while retaining
|
||||
all program-header payloads. It checks the actual Ghidra language, entry-address
|
||||
decompilation, distinct artifact digest and missing-section limitations through
|
||||
CLI Evidence and an MCP-produced snapshot. Caller-selected Ghidra heap limits
|
||||
are passed to these child workflows.
|
||||
For soft-float fixtures the lane also checks, through `inspect_native_api`, the
|
||||
exact parameter and return types and storage of float, double, mixed (pair
|
||||
alignment and stack spill) and struct-returning functions; that a caller's
|
||||
|
||||
@@ -31,8 +31,7 @@ operation-scoped JVM `OutOfMemoryError` observations retain their separate resou
|
||||
classification. Automatic kernel or cgroup OOM attribution is not implemented.
|
||||
|
||||
After the authenticated startup handshake, rejected inputs (`invalid_request`,
|
||||
`not_found`, `ambiguous`, an invalid function name, or an address without one
|
||||
exact file offset) use
|
||||
`not_found`, `ambiguous`, or an invalid function name) use
|
||||
`diagnostics.diagnostic_scope: "request"` when their active output checkpoint is
|
||||
available. They preserve the active request's
|
||||
`failure_output`, target/profile identity and runtime/log coordinates, while
|
||||
@@ -172,11 +171,6 @@ have no file backing. ELF mappings are observed
|
||||
without independent format-specific load-image verification, as the result's
|
||||
status and limitations state.
|
||||
|
||||
Relocation rows retain Ghidra's `symbol` value, including an empty string or
|
||||
null when returned by Ghidra. It is a provider observation, not proof of the
|
||||
runtime-resolved address. Older Evidence may not have recorded this field;
|
||||
a relocation page requesting it reports that uncertainty.
|
||||
|
||||
For large relocation inventories, select MCP `inspect_native_load_image` with
|
||||
`detail: "summary"`. It returns verification state, observed identity and counts,
|
||||
while retaining complete Evidence. Use `normalized_result.parent_evidence_id`
|
||||
@@ -305,7 +299,7 @@ host/provider acceptance run with only the host compiler.
|
||||
`--dll`. These static fixtures contain source-owned encoded instructions; they
|
||||
are not compiler-produced programs. See the [Windows verification scope](windows-ghidra-p0.md#native-authority-and-target-identity)
|
||||
for real-provider coverage and exclusions.
|
||||
These lanes check all 27 read-only operations, target/snapshot/import SHA-256 linkage,
|
||||
These lanes check all 26 read-only operations, target/snapshot/import SHA-256 linkage,
|
||||
and project, endpoint, process, and runtime cleanup. The independent native
|
||||
lane checks DACLs, handle admission, cancellation, and Job Object lifecycle.
|
||||
`npm run verify:ghidra:windows:package` packs and installs REA into an isolated
|
||||
|
||||
+3
-3
@@ -85,9 +85,9 @@ node scripts/verify-real-ghidra-mips.mjs --psp-missing-extension
|
||||
npm run verify:ghidra:mips
|
||||
```
|
||||
|
||||
The generic verifier also checks that a section-truncated firmware copy is
|
||||
refused when the target is opened. That check runs only in generic mode; the
|
||||
PSP report explicitly records carved-PSP coverage as not run.
|
||||
The generic verifier also exercises a section-truncated firmware fixture. That
|
||||
check uses generic MIPS symbols and language expectations and runs only in generic
|
||||
mode; the PSP report explicitly records carved-PSP coverage as not run.
|
||||
|
||||
The maintained PSP fixture builds from `tests/conformance/psp/`, never executes,
|
||||
and is independent of commercial game data. PSP GNU tools supply ELF/ABI,
|
||||
|
||||
+40
-264
@@ -9,43 +9,29 @@ lane shape as [generic MIPS support](ghidra-mips.md).
|
||||
|
||||
ELF `EM_RISCV` (243) is retained as the provider-neutral `riscv` family. The
|
||||
target also retains the raw ELF declarations that decide a lane: class, byte
|
||||
order, `e_type`, `e_flags`, and the ISA string that `.riscv.attributes`
|
||||
declares. Recognition does not mean every provider or RISC-V variant is
|
||||
supported. Existing PE, managed and Mach-O architecture sets are unchanged.
|
||||
Headerless inputs use the separate [raw-binary admission
|
||||
order, `e_type`, and `e_flags`. Recognition does not mean every provider or
|
||||
RISC-V variant is supported. Existing PE, managed and Mach-O architecture sets
|
||||
are unchanged. Headerless inputs use the separate [raw-binary admission
|
||||
boundary](raw-binary.md); raw RISC-V images are not supported.
|
||||
|
||||
The Ghidra lane admits little-endian `ET_EXEC` images with no `EF_RISCV_RVE`,
|
||||
`EF_RISCV_TSO`, or any bit outside `EF_RISCV_RVC | EF_RISCV_FLOAT_ABI`, in one
|
||||
of three lanes:
|
||||
|
||||
- ELF64 or ELF32 images that declare `EF_RISCV_FLOAT_ABI_DOUBLE` use Ghidra's
|
||||
stock double-float model unchanged. Their `.riscv.attributes` is not read for
|
||||
admission.
|
||||
- ELF32 images that declare the soft-float ABI (`e_flags` float-ABI bits `0`,
|
||||
ilp32) use REA's [ilp32 calling-convention policy](#ilp32). They must also
|
||||
declare an RV32I base and only [decoded extensions](#isa-declarations) in
|
||||
`Tag_RISCV_arch`.
|
||||
|
||||
These are header declarations, not proof that every instruction conforms to
|
||||
that ISA.
|
||||
The Ghidra lane admits little-endian ELF32 and ELF64 `ET_EXEC` images that
|
||||
declare `EF_RISCV_FLOAT_ABI_DOUBLE` in `e_flags`, with no `EF_RISCV_RVE`,
|
||||
`EF_RISCV_TSO`, or any bit outside `EF_RISCV_RVC | EF_RISCV_FLOAT_ABI`. These are
|
||||
header declarations, not proof that every instruction conforms to that ISA.
|
||||
|
||||
Provider selection returns the specific unsupported constraint rather than
|
||||
guessing a compatible profile:
|
||||
|
||||
| Declaration | Outcome |
|
||||
| -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
|
||||
| Little-endian ELF64 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:64:default` + `gcc`, lane `elf64-exec-le-double-float-generic-v1` |
|
||||
| Little-endian ELF32 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:32:default` + `gcc`, lane `elf32-exec-le-double-float-generic-v1` |
|
||||
| Little-endian ELF32 `ET_EXEC`, soft-float ABI, decoded extensions only | admitted, `RISCV:LE:32:default` + `gcc` with the `__rea_ilp32` prototype, lane `elf32-exec-le-soft-float-generic-v1` |
|
||||
| ELF32 soft-float ABI without `Tag_RISCV_arch`, or with any other extension | refused; the reason names the extensions outside the decoded set |
|
||||
| ELF32 single-float ABI (ilp32f) | refused until its [written policy](#ilp32f) is implemented and verified |
|
||||
| ELF64 soft- or single-float ABI (lp64, lp64f), or the quad-float ABI | refused; no verified calling-convention policy |
|
||||
| Big-endian | refused; Ghidra 12.1.4 ships no big-endian RISC-V language |
|
||||
| `ET_REL`, `ET_DYN`, other `e_type` | refused with the declared value |
|
||||
| `EF_RISCV_RVE` | refused; the embedded 16-register profile (x0–x15) needs its own processor specification |
|
||||
| `EF_RISCV_TSO` | refused; the total-store-order memory model is unverified here |
|
||||
| Reserved or newer `e_flags` bits | refused with the offending bits named |
|
||||
| Declaration | Outcome |
|
||||
| ----------------------------------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| Little-endian ELF64 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:64:default` + `gcc` |
|
||||
| Little-endian ELF32 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:32:default` + `gcc` |
|
||||
| Big-endian | refused; Ghidra 12.1.4 ships no big-endian RISC-V language |
|
||||
| `ET_REL`, `ET_DYN`, other `e_type` | refused with the declared value |
|
||||
| `EF_RISCV_RVE` | refused; the embedded 16-register profile (x0–x15) needs its own processor specification |
|
||||
| `EF_RISCV_TSO` | refused; the total-store-order memory model is unverified here |
|
||||
| Soft, single, or quad float ABI | refused; the pinned generic compiler specification models double-float |
|
||||
| Reserved or newer `e_flags` bits | refused with the offending bits named |
|
||||
|
||||
`e_type` is the only load-shape declaration this lane reads. A fixed-address
|
||||
`ET_EXEC` declaration does **not** establish static linkage: `PT_INTERP` and
|
||||
@@ -72,193 +58,15 @@ with the `gcc` compiler specification per class, records
|
||||
pin without changing the requested override. An override that matches the pin is
|
||||
accepted and recorded as `configured-v1`.
|
||||
|
||||
The soft-float lane pins the same `RISCV:LE:32:default` language. Every RV32
|
||||
language in Ghidra 12.1.4, including the deprecated soft-float `RV32I`, `RV32IC`
|
||||
and `RV32IMC` variants in `old/riscv_deprecated.ldefs`, uses the same
|
||||
`riscv.ilp32d.sla` decoder; only the compiler specification differs. REA keeps
|
||||
the generic language and corrects the calling convention instead.
|
||||
|
||||
## Calling conventions
|
||||
|
||||
Ghidra's generic RV32 compiler specification, `riscv32-fp.cspec`, models the
|
||||
ilp32d ABI. Its default prototype passes `float` and `double` values in
|
||||
`fa0`–`fa7` and returns them in `fa0`, and it passes a structure of one or two
|
||||
floating-point members in floating-point registers. An ilp32 image passes all
|
||||
of these in integer registers. Measured on Ghidra 12.1.4 with an ilp32 build of
|
||||
the fixture in `tests/conformance/c/riscv-float-abi.c`, the stock model
|
||||
decompiles `rea_riscv_abi_pick(1.5f, 2.5f, sel)` as
|
||||
`rea_riscv_abi_pick(in_fa0,in_fa1,0x3fc00000)`.
|
||||
|
||||
### Decision record: D3, RISC-V float ABIs
|
||||
|
||||
Decided in [#7477](https://github.com/morluto/rea/issues/7477).
|
||||
|
||||
- **Choice.**
|
||||
- Admit little-endian ELF32 `ET_EXEC` images that declare the soft-float ABI
|
||||
(ilp32) through a new lane, `elf32-exec-le-soft-float-generic-v1`. Lane ids
|
||||
are built from their fields: `elf<class>-exec-le-<float ABI>-float-generic-v1`.
|
||||
- Keep the pinned `RISCV:LE:32:default` language and `gcc` compiler
|
||||
specification. After analysis, `ReaGhidraRiscvCallingConvention.java`
|
||||
derives the program-specific `__rea_ilp32` prototype from the program's
|
||||
default prototype as the [ilp32 policy](#ilp32) states. It installs the
|
||||
prototype through a Ghidra specification extension and assigns it, with
|
||||
dynamic storage, to every non-thunk function, every external function and
|
||||
every function definition. Thunks follow their targets. The script fails,
|
||||
and writes no report, if the derived prototype or any function still
|
||||
places a value in a floating-point register, or if the prototype does not
|
||||
pass a 16-byte value by reference.
|
||||
- One derivation serves every policy. A policy row says how it changes the
|
||||
default prototype's floating-point entries and preserved registers; every
|
||||
policy passes values of 9 bytes or more by reference.
|
||||
- The profile commits `riscv_calling_convention: ilp32-prototype-v1`. The
|
||||
bridge reports the policy and its function and definition counts, and REA
|
||||
requires that report exactly when the profile commits the policy.
|
||||
- Admission requires a `Tag_RISCV_arch` declaration with an RV32I base and
|
||||
only extensions that Ghidra 12.1.4's generic RV32 language implements with
|
||||
p-code ([ISA declarations](#isa-declarations)).
|
||||
- Soft-float profiles add `float_abi` and `arch` to `riscv_elf`. Double-float
|
||||
profiles keep their exact parameters, so their digests, sessions and
|
||||
snapshots are unchanged.
|
||||
- ilp32f keeps its [written policy](#ilp32f) and stays refused until its
|
||||
prototype is implemented and verified. Quad-float, RVE, TSO, and ELF64
|
||||
soft- or single-float images stay refused.
|
||||
- **Alternatives** (and what switching to each would take):
|
||||
- Pin the deprecated soft-float languages (`RISCV:LE:32:RV32IMC` and others,
|
||||
`riscv32.cspec`): REA's language catalog would have to read `old/`, the
|
||||
variants cover only I, IC and IMC, Ghidra marks them deprecated, and no
|
||||
stock specification models ilp32f.
|
||||
- Assign `__rea_ilp32` only to callables whose types use floating-point
|
||||
values, as the MIPS soft-float lane does: typed storage is the same, but
|
||||
prototypes Ghidra infers for untyped code would keep the ilp32d model. In
|
||||
an ilp32 image that contains F or D instructions, that model can still
|
||||
claim `fa0`. Switching is one predicate in the Java script and a new
|
||||
policy id.
|
||||
- Admit with a limitation and no correction: a reduced-capability contract
|
||||
that needs its own review, because arguments, returns and aggregates
|
||||
become unverified.
|
||||
- Keep refusing: the lane rows become refusals again.
|
||||
- Admit images without `Tag_RISCV_arch`, or refuse a list of known
|
||||
extensions instead of admitting a list: a declaration that REA cannot
|
||||
check would be treated as compatible. Widening the decoded set later is
|
||||
additive.
|
||||
- Add `float_abi` and `arch` to double-float profiles as well: one shape for
|
||||
every lane, but every existing RISC-V session and snapshot would be
|
||||
invalidated.
|
||||
- Ship ilp32 and ilp32f together (the question in #7477): ilp32f needs a
|
||||
float-and-integer structure rule that Ghidra's model lacks, so it ships
|
||||
separately with its own fixtures.
|
||||
- **Semantic owner and consumers.**
|
||||
- Owners:
|
||||
- `src/domain/riscvIsaDeclaration.ts` owns the float-ABI vocabulary of
|
||||
`e_flags` and the parsing of `.riscv.attributes` and `Tag_RISCV_arch`.
|
||||
- `GHIDRA_RISCV_LANES`, `GHIDRA_RISCV_CALLING_CONVENTIONS` and
|
||||
`GHIDRA_RISCV32_DECODED_EXTENSIONS` (`src/ghidra/GhidraRiscvProfile.ts`)
|
||||
own admission, lane ids and the policy table.
|
||||
- `bridge/ghidra/ReaGhidraRiscvCallingConvention.java` owns how each policy
|
||||
becomes a Ghidra prototype.
|
||||
- Consumers: target resolution, Ghidra provider admission, the analysis
|
||||
profile, the provider client and launcher, the session handshake, project
|
||||
history, the real-Ghidra lane and this page. Later work reuses the table:
|
||||
ilp32f adds a row, and RISC-V ESP images (F6) select a row through D5.
|
||||
- **Costs of changing it.**
|
||||
- **Code:** S. A policy or lane is one table row plus the Java derivation.
|
||||
- **Compatibility:** the soft-float lane id, `riscv_elf.float_abi`,
|
||||
`riscv_elf.arch` and `riscv_calling_convention` are persisted in sessions,
|
||||
snapshots and Evidence once released. A changed derivation ships as a new
|
||||
policy id, such as `ilp32-prototype-v2`, which invalidates soft-float
|
||||
snapshots only.
|
||||
- **Migration:** none for admitted targets; double-float profiles are pinned
|
||||
by tests. Previously refused soft-float targets become admitted.
|
||||
- **Verification:** rerun `npm run verify:ghidra:riscv`.
|
||||
|
||||
### ilp32
|
||||
|
||||
Policy `ilp32-prototype-v1`, prototype `__rea_ilp32`. XLEN is 32 bits and no
|
||||
floating-point register passes a value.
|
||||
|
||||
| Topic | psABI rule | `__rea_ilp32` |
|
||||
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Integer and pointer values | `a0`–`a7`, then the stack in 4-byte-aligned slots | The default integer and stack entries, unchanged |
|
||||
| `float` | One integer register, or a stack slot | The `fa0`–`fa7` entries and the `float` rule are removed, so a `float` takes the next integer register |
|
||||
| `double` and 8-byte values | A register pair with the low word in the lower register. Named arguments need no aligned pair. With one register left, the low word goes in `a7` and the high word on the stack | The default `join` rule with stack spill: Ghidra writes `a2:4,a1:4` for a pair and `Stack[0x0]:4,a7:4` for the split |
|
||||
| Aggregates up to 8 bytes | One or two integer registers, laid out as in memory, also when every member is floating-point | The homogeneous-float-aggregate rule is removed; the `join` rule applies |
|
||||
| Aggregates over 8 bytes | By reference: the caller passes the address of a copy | The default rule that converts structures and unions of 9 bytes or more to pointers |
|
||||
| Return values | As the first argument of the same type: `a0`, or `a0` and `a1`. Larger values go to a caller buffer whose address is an implicit first argument in `a0` | The `fa0`/`fa1` return entries and floating-point rules are removed; Ghidra's hidden-return fallback supplies the pointer in `a0` |
|
||||
| Variadic arguments | The integer convention. A variadic argument with 8-byte alignment and size takes an aligned (even-numbered) register pair, or the stack; after the first stack argument, the rest go on the stack | The default variadic rule (`join` with alignment), unchanged |
|
||||
| Preserved registers | `s0`–`s11` and `sp`; `gp` and `tp` are not allocated. No floating-point register is preserved | The default preserved list without `fs0`–`fs11`, which join the killed-by-call list |
|
||||
| Scalars over 8 bytes | By reference (`long double` is 16 bytes), and returned through a caller buffer | A rule before the final join converts any value of 9 bytes or more to a pointer; a return that size already goes to a caller buffer |
|
||||
|
||||
Ghidra recovers the arguments of a variadic call from the registers the caller
|
||||
writes, not from their types. A padding register that still holds an earlier
|
||||
value can appear as an argument, and arguments after an unwritten register are
|
||||
omitted.
|
||||
|
||||
### ilp32f
|
||||
|
||||
Written policy for `EF_RISCV_FLOAT_ABI_SINGLE` on ELF32. REA does not admit it
|
||||
yet. ABI_FLEN is 32 bits.
|
||||
|
||||
| Topic | psABI rule |
|
||||
| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
|
||||
| `float` | `fa0`–`fa7` while one is free, then the integer convention |
|
||||
| `double` and other values wider than 32 bits | The integer convention, as in ilp32 |
|
||||
| Structure of one or two floats (after flattening) | One floating-point register per member when enough are free, otherwise the integer convention |
|
||||
| Structure of one float and one integer, in either order | One floating-point and one integer register when both are free, otherwise the integer convention |
|
||||
| Unions | Never flattened; the integer convention |
|
||||
| Return values | As the first argument: `fa0`, `fa0` and `fa1`, or `fa0` and `a0` |
|
||||
| Variadic arguments | The integer convention, as in ilp32 |
|
||||
| Preserved registers | `s0`–`s11`, `sp`, and `fs0`–`fs11` for values of 32 bits or fewer |
|
||||
|
||||
The planned prototype limits the default floating-point entries to 4 bytes and
|
||||
keeps the shared rule that passes values of 9 bytes or more by reference.
|
||||
Observed on Ghidra 12.1.4, a `double` then takes an integer pair and a structure
|
||||
of two floats takes `fa0` and `fa1`. Ghidra's model has no rule for a structure
|
||||
of one float and one integer: it joins `struct { float; int; }` into `a1:a0`
|
||||
instead of `fa0` and `a0`. The ilp32f policy needs that rule and its own
|
||||
fixtures before it is admitted.
|
||||
|
||||
### ilp32d and lp64d
|
||||
|
||||
The double-float lanes use Ghidra's stock `riscv32-fp.cspec` and
|
||||
`riscv64-fp.cspec` models unchanged. This page does not verify them. They share
|
||||
the float-and-integer structure gap above.
|
||||
|
||||
## ISA declarations
|
||||
|
||||
The target reader finds the one `SHT_RISCV_ATTRIBUTES` section and reads
|
||||
`Tag_RISCV_arch`, such as `rv32i2p1_m2p0_c2p0_zicsr2p0_zifencei2p0`. It
|
||||
retains the string, the base and the extension names, or an unavailable reason
|
||||
when the section is malformed, declares other vendors or scopes, or uses a
|
||||
non-canonical string. A missing section or tag is recorded as absent.
|
||||
|
||||
Calling-convention lanes admit an image only when it declares an RV32I base
|
||||
and only extensions that Ghidra 12.1.4's generic RV32 language implements with
|
||||
p-code: `m`, `a`, `f`, `d`, `c`, `zicsr`, `zifencei`, and the subset names
|
||||
toolchains record, `zmmul`, `zaamo`, `zalrsc`, `zca`, `zcf` and `zcd`. Any
|
||||
other extension is refused. In particular:
|
||||
|
||||
- Ghidra 12.1.4 decodes no Zcb, Zcmp or Zcmt instruction.
|
||||
- Its bit-manipulation, cryptography and packed-SIMD files follow draft
|
||||
encodings and mark nearly every instruction unimplemented.
|
||||
- Vendor extensions, such as Espressif's `xesploop`, `xespv` and `xespdsp`,
|
||||
decode only as custom-opcode placeholders.
|
||||
|
||||
ESP-IDF builds ESP32-C2 and C3 images as `rv32imc_zicsr_zifencei` and ESP32-C5,
|
||||
C6, C61, H2 and H21 images as `rv32imac_zicsr_zifencei_zaamo_zalrsc`, all ilp32.
|
||||
Those base strings are admitted. Builds that add `_zba_zbb_zbs`,
|
||||
`_zcb_zcmp_zcmt` or an `xesp` extension are refused.
|
||||
|
||||
## Evidence
|
||||
|
||||
The lane identity is one of the lane ids above, and `riscv_elf` retains
|
||||
`elf_class`, `byte_order`, `type`, and the raw `flags` alongside the read-only
|
||||
booleans `float_abi_double`, `rvc_declared`, `rve_declared`, and
|
||||
`tso_declared`. Soft-float profiles also retain `float_abi: soft`, the declared
|
||||
`arch` string, and `riscv_calling_convention: ilp32-prototype-v1`. Those fields
|
||||
report declarations the header and attributes make, and the policy REA applied;
|
||||
none of them is evidence of the original compiler or of instruction
|
||||
conformance. Session Evidence states the prototype and how many functions and
|
||||
definitions use it.
|
||||
The lane identity is `elf32-exec-le-double-float-generic-v1` or
|
||||
`elf64-exec-le-double-float-generic-v1`, and `riscv_elf` retains `elf_class`,
|
||||
`byte_order`, `type`, and the raw `flags` alongside the read-only booleans
|
||||
`float_abi_double`, `rvc_declared`, `rve_declared`, and `tso_declared`. Those
|
||||
fields report declarations the header makes; none of them is evidence of the
|
||||
original compiler, of instruction conformance, or of a floating-point calling
|
||||
convention beyond the ABI the pinned language models.
|
||||
|
||||
## Verification and limits
|
||||
|
||||
@@ -268,9 +76,9 @@ tree ships no host decompiler binary, so `Decompiler/os/<platform>/decompile` an
|
||||
[installation](installation.md#ghidra).
|
||||
|
||||
Run `npm run verify:ghidra:riscv` with an existing Ghidra 12.1.4 installation,
|
||||
JDK 21+, and a RISC-V-capable Clang, LLD, LLVM readelf and LLVM objdump. The
|
||||
verifier acquires no tools and never executes target code. Select installed
|
||||
tools explicitly when they are not on `PATH`:
|
||||
JDK 21+, and a RISC-V-capable Clang, LLD and LLVM readelf. The verifier acquires
|
||||
no tools and never executes target code. Select installed tools explicitly when
|
||||
they are not on `PATH`:
|
||||
|
||||
```sh
|
||||
GHIDRA_INSTALL_DIR=/path/to/ghidra_12.1.4_PUBLIC \
|
||||
@@ -278,52 +86,20 @@ JAVA_HOME=/path/to/jdk-21 \
|
||||
REA_RISCV_CLANG=/path/to/llvm/bin/clang \
|
||||
REA_RISCV_LLD=/path/to/lld/bin/ld.lld \
|
||||
REA_RISCV_READELF=/path/to/llvm/bin/llvm-readelf \
|
||||
REA_RISCV_OBJDUMP=/path/to/llvm/bin/llvm-objdump \
|
||||
npm run verify:ghidra:riscv
|
||||
```
|
||||
|
||||
The lane builds four `ET_EXEC` fixtures from committed freestanding C and
|
||||
assembly sources:
|
||||
The lane builds the committed freestanding C and assembly sources into RV32 and
|
||||
RV64 `ET_EXEC` fixtures. Independent LLVM readelf observations establish ELF
|
||||
declarations and symbol addresses. Public CLI and stdio MCP workflows verify
|
||||
function enumeration, decompilation, pinned language/compiler identity, decoded
|
||||
probe bytes, global data, and direct calls. It validates advertised MCP output
|
||||
schemas and both JSON representations, saves and reopens snapshots, rejects
|
||||
header-mutated soft-float and unsupported high-bit declarations, and checks
|
||||
unchanged source bytes and owned cleanup. Failures retain the diagnostic
|
||||
workspace; successful runs remove it and report the observed process lineage.
|
||||
|
||||
- RV32 and RV64 double-float builds (`rv{32,64}imafdc`) of `riscv.c`.
|
||||
- An RV32 soft-float build of `riscv-float-abi.c` with ESP32-C3's
|
||||
`-march=rv32imc_zicsr_zifencei -mabi=ilp32`, typed with DWARF.
|
||||
- The same objects linked with `--strip-all`. Every allocated section keeps
|
||||
its address and bytes, so the typed build's symbols locate its functions.
|
||||
|
||||
Independent LLVM readelf observations establish ELF declarations,
|
||||
`Tag_RISCV_arch` and symbol addresses. Public CLI and stdio MCP workflows verify
|
||||
function enumeration, decompilation, pinned language/compiler identity, the
|
||||
committed lane and policy, decoded probe bytes, global data, and direct calls.
|
||||
They validate advertised MCP output schemas and both JSON representations, save
|
||||
and reopen snapshots, and check unchanged source bytes and owned cleanup.
|
||||
|
||||
For the soft-float builds the lane also checks:
|
||||
|
||||
- `fence.i` and `csrr` decode, as ESP32-C3's Zifencei and Zicsr declare.
|
||||
- An LLVM objdump reading of `rea_riscv_abi_caller` finds each call-site word
|
||||
where the [ilp32 policy](#ilp32) puts it: an unaligned `double` in `a1`/`a2`,
|
||||
the `a7`/stack split, two floats in an integer pair, a large structure and a
|
||||
`long double` passed by reference (the pointed-to copies hold the source
|
||||
values), aligned variadic pairs, and an indirect call.
|
||||
- `inspect_native_api` reports `__rea_ilp32` with exactly that storage for the
|
||||
typed build. The stripped build has one integer word per parameter location
|
||||
and returns in the typed build's registers, or in none when the result goes
|
||||
through caller memory.
|
||||
- The caller's pseudocode passes the source constants through that storage and
|
||||
names no floating-point register.
|
||||
- Retained Evidence states the prototype outcome; the stripped build has no
|
||||
function definitions.
|
||||
|
||||
Header-mutated copies check admission. The RV32 double-float image relabelled as
|
||||
soft-float (`e_flags` `0x1`), refused before this lane, is admitted through the
|
||||
soft-float lane over CLI and MCP. The soft-float image relabelled as
|
||||
single-float, RV64 soft-float, and unsupported high bits are refused with their
|
||||
reasons. Failures retain the diagnostic workspace; successful runs remove it and
|
||||
report the observed process lineage.
|
||||
|
||||
Module tests cover admission, malformed headers and attribute sections, the
|
||||
handshake, and the conformance checks against recorded LLVM and Ghidra output.
|
||||
Call-graph completeness, string inventories, dynamic linkage, Andes or RVE/TSO
|
||||
images, big-endian targets, ilp32f images, and decoding beyond the probes are
|
||||
**not** established by this lane.
|
||||
Module tests cover admission and malformed headers. Call-graph completeness,
|
||||
string inventories, dynamic linkage, Andes or RVE/TSO images, and big-endian
|
||||
targets are **not** established by this lane. `.riscv.attributes` ISA extensions
|
||||
are not inspected; the generic language pin remains an analysis convention.
|
||||
+3
-12
@@ -1,18 +1,9 @@
|
||||
# Godot PCK archives
|
||||
|
||||
`inspect-artifact` and `extract-artifact` read ordinary Godot PCK formats 1, 2, 3
|
||||
`inspect-artifact` and `extract-artifact` read ordinary Godot PCK formats 2, 3
|
||||
and 4 without running the game or requiring Godot. Official Godot 4.4.1, 4.6.1
|
||||
and 4.7.1 produced formats 2, 3 and 4 respectively in verification; other engine
|
||||
releases have not been verified. Format 1 compatibility follows the
|
||||
[Godot 3.5 loader](https://github.com/godotengine/godot/blob/3.5/core/io/file_access_pack.cpp#L201-L242)
|
||||
and [exporter](https://github.com/godotengine/godot/blob/3.5/editor/editor_export.cpp#L1094-L1164)
|
||||
layout and is covered by synthetic standalone and embedded fixtures; an actual
|
||||
Godot 3 export has not been exercised.
|
||||
|
||||
Format 1 uses an 84-byte header, no pack or member flags, and absolute member
|
||||
offsets in the complete source file, including an embedded executable carrier.
|
||||
Its reported flags and file base are zero because those fields are absent.
|
||||
Zero-offset removal records remain unsupported; REA does not apply patches.
|
||||
and 4.7.1 produced these respective formats in verification; other engine
|
||||
releases have not been verified. Format 1 is unsupported.
|
||||
|
||||
Inspection retains the original header and directory facts, including engine
|
||||
version, flags, declared file count, source paths, offsets and declared MD5.
|
||||
|
||||
@@ -36,7 +36,7 @@ The MCP tool `project_harmony_application_graph` takes the same
|
||||
|
||||
- exact component paths, artifact IDs, and hashes (manifests, Ark bytecode,
|
||||
resources, native libraries, JavaScript, signing)
|
||||
- a packaging-model hint: Stage is claimed only from an exactly cased `module.json` entry
|
||||
- a packaging-model hint: Stage is claimed only from a `module.json` entry
|
||||
outside a classified resource path, FA only from `config.json` outside a
|
||||
classified resource path, otherwise `unknown`
|
||||
- FA resources under `assets/<module>/resources/` as well as Stage `resources/`
|
||||
@@ -46,9 +46,6 @@ The MCP tool `project_harmony_application_graph` takes the same
|
||||
- path-based N-API bridge hypotheses between bytecode and `libs/<abi>` native
|
||||
libraries
|
||||
|
||||
An unavailable manifest can still supply a path-based model hint; its content
|
||||
identity remains unknown. Manifest names follow case-sensitive ZIP member names.
|
||||
|
||||
File and slice occurrences supply component and child-package paths; directory
|
||||
trees and symlinks retain their identities in the source inventory without
|
||||
being classified as code files. Ark and JavaScript runtime hints summarize the
|
||||
|
||||
@@ -1,129 +0,0 @@
|
||||
# Intel HEX and S-record images
|
||||
|
||||
REA inventories Intel HEX (I8HEX, I16HEX, I32HEX) and Motorola S-record (S19,
|
||||
S28, S37) programming files through `inspect-artifact` / `inspect_artifact`, and
|
||||
materializes their decoded bytes through `extract-artifact` /
|
||||
`extract_artifact`. Decoding is static and needs no external tool.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./firmware.hex --json
|
||||
rea inspect-artifact ./firmware.s37 --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact ./firmware.hex --json
|
||||
```
|
||||
|
||||
For MCP, open the file with `open_binary`, then call `inspect_artifact`. Summary
|
||||
delivery retains the complete Evidence; use `parent_evidence_id` with
|
||||
`inspect_analysis_view` to read selected regions and fields. `extract_artifact`
|
||||
returns a fresh `output_root` containing one raw file per region.
|
||||
|
||||
## Recognition
|
||||
|
||||
A file is recognized only when its first line, starting at byte 0, is a fully
|
||||
valid record: correct start character, hex digits, byte count, checksum, known
|
||||
record type and fixed length. The file name does not matter. A byte-order mark,
|
||||
a leading blank line or an invalid first record leaves the file unrecognized,
|
||||
so it falls through to plain `file` classification. Recognition never searches
|
||||
for a record later in the file. The container formats are `intel-hex` and
|
||||
`srec`.
|
||||
|
||||
## Regions and metadata
|
||||
|
||||
Each inventory entry is one address-contiguous region of decoded bytes, named
|
||||
`0x<start>-0x<end>.bin` with an inclusive end address of eight lowercase hex
|
||||
digits, for example `0x08000000-0x0800001f.bin`. Regions are listed in ascending
|
||||
address order. Out-of-order records are sorted into regions; gaps separate
|
||||
regions and are never filled.
|
||||
|
||||
Region `format_metadata` (`hex-record-region`) records `start_address`,
|
||||
`end_address`, `length`, `record_count`, and `first_line`/`last_line`: the
|
||||
1-based line span of the contributing data records.
|
||||
|
||||
Root `format_metadata` (`hex-records`) records:
|
||||
|
||||
| Field | Meaning |
|
||||
| ------------------------------------------------- | --------------------------------------------------------------------------------- |
|
||||
| `encoding`, `variant` | `intel-hex` or `srec`; the variant derived from the record types seen, or `mixed` |
|
||||
| `line_count`, `record_count`, `data_record_count` | Lines read, non-blank records, and data records |
|
||||
| `data_bytes`, `duplicate_bytes`, `region_count` | Unique decoded bytes, bytes rewritten with identical values, and regions |
|
||||
| `lowest_address`, `highest_address` | Address bounds of the decoded bytes, or `null` without data |
|
||||
| `termination` | Whether Intel `01` or S-record `S7`/`S8`/`S9` was present |
|
||||
| `start_address`, `start_address_record` | The Intel `05` EIP or the `S7`/`S8`/`S9` address, with its record type |
|
||||
| `start_segment_cs`, `start_segment_ip` | The Intel `03` CS:IP pair |
|
||||
| `header_hex`, `header_text` | S-record `S0` data as hex, and as text only when every byte is printable ASCII |
|
||||
| `declared_data_record_count` | The last `S5`/`S6` count, validated against the data records before it |
|
||||
|
||||
`variant` is `null` for an S-record file in which no data or termination
|
||||
record establishes an address width. Start addresses are declared facts, not
|
||||
proven entry points.
|
||||
|
||||
## Facts, limitations and failures
|
||||
|
||||
| Situation | Outcome |
|
||||
| ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------- |
|
||||
| Gaps between regions | Separate regions; never zero-filled |
|
||||
| Out-of-order records | Sorted into regions |
|
||||
| An address written twice with the same value | Accepted and counted in `duplicate_bytes` |
|
||||
| An address written twice with different values | Fails, naming the address and the line spans of both writes |
|
||||
| Checksum mismatch, odd hex length, non-hex character, byte count disagreeing with the line | Fails with the line, declared and computed values, and record type |
|
||||
| Line longer than 521 (Intel) or 514 (S-record) characters, excluding the line ending | Fails before the line is decoded |
|
||||
| Intel type `06` or higher, `S4`, wrong fixed length for Intel `01`–`05` or `S5`–`S9` | Fails |
|
||||
| Missing Intel `01` or `S7`–`S9` | Regions are returned; coverage is unknown and the file may be truncated |
|
||||
| A non-blank record after termination, or a second `S0` | Fails; blank lines are allowed anywhere |
|
||||
| `S5`/`S6` count different from the data records before it | Fails |
|
||||
| Termination type not matching the data address width | Reported as a limitation; the declared fact is kept |
|
||||
| Conflicting repeated `03`/`05` start records | Fails; identical repeats are allowed |
|
||||
| An Intel data record whose 16-bit offset wraps past `0xFFFF` | Fails: producers disagree on whether the address wraps or continues linearly |
|
||||
| I16HEX data above `0xFFFFF`, or data crossing the S1/S2/S3 address width | Fails |
|
||||
|
||||
Lowercase hex, LF and CRLF line endings, and trailing spaces or tabs are
|
||||
accepted. A CR not followed by LF is an invalid character. Failures apply to
|
||||
the whole file and return no partial regions.
|
||||
|
||||
## Limits
|
||||
|
||||
The file must be at most 128 MiB. Every data byte needs two hex digits, so the
|
||||
decoder reserves at most half the file size for decoded bytes before reading,
|
||||
and stores only written addresses. The decoder retains at most 65,536
|
||||
address-discontiguous record runs before merging them into regions. A run is a
|
||||
sequence of consecutive records whose addresses continue each other. Both are
|
||||
fixed resource bounds, not format limits, and fail as limits (`file-size` and
|
||||
`capacity`) with no runtime override. Configurable ceilings are tracked in
|
||||
[#6176](https://github.com/morluto/rea/issues/6176). The line-length limits are
|
||||
the one-byte record count: Intel HEX allows 255 data bytes, while S-record
|
||||
counts include address and checksum bytes.
|
||||
|
||||
Decoded regions are cached only for the reader's lifetime. Each open checks
|
||||
its admitted carrier identity again. Region streams copy at most 64 KiB per
|
||||
chunk, so consumers cannot mutate cached bytes and opening a large region does
|
||||
not copy it in full. Caller cancellation keeps its exact reason, and closing
|
||||
the reader waits for active region streams to close.
|
||||
|
||||
## Analyzing a region
|
||||
|
||||
Inventory does not establish the CPU, byte order or memory map, or that a
|
||||
declared start address is executable. To disassemble a region, extract it and
|
||||
open the region file with `open_binary` using format `raw-binary`, an
|
||||
architecture you choose, and `entry_point` set to the region start address;
|
||||
raw-binary maps file offset 0 at `entry_point`. Each region is opened
|
||||
separately, so references between regions are not resolved. A caller-selected
|
||||
Ghidra language and base address for raw-binary imports is tracked in
|
||||
[#4246](https://github.com/morluto/rea/issues/4246); Thumb-only Cortex-M images
|
||||
may be refused or decoded in ARM state until
|
||||
[#5744](https://github.com/morluto/rea/issues/5744) is resolved.
|
||||
|
||||
## Verification
|
||||
|
||||
Fixtures with computed checksums cover supported record types, checksums,
|
||||
termination, address gaps and overlaps, resource refusals, inventory, extraction
|
||||
and reader lifecycle. CLI and stdio MCP tests exercise the same artifact
|
||||
provider and advertised schemas, including retained Evidence views.
|
||||
|
||||
GNU objcopy 2.40 output was also checked on Linux: the system `true` ELF was
|
||||
converted to Intel HEX and S-record, with additional address-rebased I32HEX
|
||||
and S37 cases. Each yielded 13 regions. A 19,124,817-byte Intel HEX conversion
|
||||
of the system Python 3.11 ELF yielded 11 regions containing 6,799,261 decoded
|
||||
bytes. Public CLI inspection and extraction matched GNU BFD address ranges
|
||||
and independently exported section bytes. Adjacent GNU sections were joined
|
||||
without filling address gaps. These are real producer outputs from system
|
||||
applications, not vendor-firmware or target-execution verification; srec_cat
|
||||
and other producers remain unverified. No general speedup is claimed.
|
||||
@@ -40,12 +40,7 @@ Record the image digest, REA source revision and these dependency versions:
|
||||
- OpenSSL and cURL advertising HTTP3 and HTTP2 in `curl --version`.
|
||||
|
||||
The verified guest used Python 3.14.7, cURL 8.22.0, ngtcp2 1.25.0 and nghttp3
|
||||
1.18.0. The current requirements update certifi from 2026.7.22 to 2026.10.19
|
||||
to remove the compromised emSign trust root described in
|
||||
[GHSA-6q48-6c7w-9gw2](https://github.com/certifi/python-certifi/security/advisories/GHSA-6q48-6c7w-9gw2).
|
||||
The revised dependency graph has not been rerun in that sealed guest; the
|
||||
historical verification does not cover this update. Other dependency profiles
|
||||
need verification. Back up the guest's
|
||||
1.18.0. Other dependency profiles need verification. Back up the guest's
|
||||
`/etc/hosts` before adding `127.0.0.1 lab.test`. Reserve guest TCP/UDP loopback
|
||||
ports 18443 and 19443. These changes belong only to the disposable guest.
|
||||
|
||||
|
||||
+2
-74
@@ -43,8 +43,7 @@ rea decompile /absolute/path/to/program main --json
|
||||
|
||||
If the registration cannot be read or validated, provider diagnostics retain its
|
||||
selected `registration_path` and the configuration owner's setting/constraint
|
||||
facts in `configuration_details.settings`. The availability's `remediation`, and
|
||||
each operation it makes unavailable, carry the configuration owner's repair.
|
||||
facts in `configuration_details.settings`, plus `configuration_remediation`.
|
||||
`configured` means a path was selected, not that it is valid or an IDA connection
|
||||
was established; failed registrations remain unavailable. Correct
|
||||
`REA_IDA_MCP_CONFIG` in the next CLI invocation's environment or the MCP server's
|
||||
@@ -81,71 +80,6 @@ Native MCP 1.1.2 can navigate in the previously focused pseudocode tab when swit
|
||||
|
||||
Native tool observations retain actual tool names and complete JSON text. Upstream `structuredContent` can be a shortened preview even while text contains the full result. REA reads the full native representation and rejects missing/malformed text instead of treating previews as complete observations.
|
||||
|
||||
### Migrate a legacy attached registration
|
||||
|
||||
Both attached profiles remain supported. Migrating replaces the upstream server;
|
||||
adding `"protocol": "native"` to a legacy Python proxy registration is insufficient.
|
||||
The native plugin is separate from mrexodia/ida-pro-mcp, and upgrading mrexodia to
|
||||
its modern tool catalog does not select this profile.
|
||||
|
||||
1. Back up the working registration file. Install and start the native plugin
|
||||
using its [upstream instructions](https://github.com/0xshlomil/ida-free-mcp).
|
||||
IDA and the plugin are bring-your-own; REA does not install or build them.
|
||||
Use a build with the navigation correction described above, and restrict its
|
||||
listener to loopback or apply the documented host firewall restriction.
|
||||
Stop the legacy endpoint first if both servers would use the same port.
|
||||
2. Open the same original input binary in IDA and wait for analysis. Keep that
|
||||
file readable by REA. The native metadata resource must report its recorded
|
||||
input SHA-256; the resource's `path` identifies the database, not an input
|
||||
file to substitute. Universal Mach-O inputs still require a thin extraction.
|
||||
3. Replace the registration, using the native plugin's actual local endpoint.
|
||||
For example, this legacy registration:
|
||||
|
||||
```json
|
||||
{
|
||||
"mode": "attached",
|
||||
"command": "python",
|
||||
"args": [
|
||||
"/absolute/path/to/ida_pro_mcp/server.py",
|
||||
"--ida-rpc",
|
||||
"http://127.0.0.1:13337"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
becomes:
|
||||
|
||||
```json
|
||||
{
|
||||
"mode": "attached",
|
||||
"protocol": "native",
|
||||
"url": "http://127.0.0.1:13337/mcp"
|
||||
}
|
||||
```
|
||||
|
||||
Keep `REA_IDA_MCP_CONFIG` pointing at the replacement file and close/reopen
|
||||
the REA binary session to connect with it. If changing the environment's
|
||||
configuration path, restart the MCP server as described above. Do not reuse
|
||||
the legacy RPC URL as an MCP HTTP endpoint.
|
||||
|
||||
4. Check `binary_session` availability before resuming workflows. Direct
|
||||
`procedure_callers` becomes unavailable and dossier callers remain unknown;
|
||||
incoming address xrefs do not establish direct call edges. Native strings
|
||||
use C-string rendering. Assembly and instruction reads no longer navigate
|
||||
the GUI; native pseudocode still changes focus and closes existing
|
||||
pseudocode tabs. IDA Free decompilation may require cloud access and consent.
|
||||
5. Run the [real-provider verification](#verification) with two distinct
|
||||
functions and the required second-function marker. Supply independently
|
||||
known string and xref oracles for those checks. Existing Windows evidence
|
||||
covers IDA Pro 9.3 with the documented source build and listener change;
|
||||
it does not establish stock-plugin, Windows Free, or native macOS behavior.
|
||||
A valid registration or successful doctor check alone does not verify the
|
||||
connection, target identity, or function switching.
|
||||
|
||||
REA continues to verify identity and image base around requests and closes only
|
||||
its connection, preserving the attached database. Restore the backed-up legacy
|
||||
registration and its upstream endpoint to return to the legacy profile.
|
||||
|
||||
The legacy producer moves the active GUI as a side effect: assembly and raw function instructions jump to the function, and successful pseudocode opens or reuses a pseudocode view. A function dossier does both. Native GUI decompilation also changes the active view. `binary_session` reports these operations with `effects.may_show_ui: true`; headless operations and the other attached operations report `false`.
|
||||
|
||||
## Give REA a target for headless analysis
|
||||
@@ -221,12 +155,6 @@ For guided registration, set `REA_IDA_MCP_CONFIG` before `rea setup --client <cl
|
||||
|
||||
## Coverage and evidence
|
||||
|
||||
REA admits x86, x86_64, arm and arm64 executables through IDA. Every other CPU
|
||||
family, including families that REA recognizes later, is refused until REA
|
||||
admits it explicitly. Raw-binary and GameCube DOL targets are refused; open
|
||||
them with Ghidra. Admission is not verification: the real workflows under
|
||||
[Verification](#verification) cover x86_64 and arm64 only.
|
||||
|
||||
All profiles support procedure/string inventory and literal or regex search, procedure resolution, pseudocode, assembly, raw function instructions, resolved callees, incoming address references, and a function dossier. Inventories and modern pseudocode/assembly follow upstream pagination. Upstream limits on modern xrefs and callees are retained as explicit truncation limitations. In the modern headless profile, targets that are not proven x86-only use processor-neutral `callgraph` edges instead of the x86 instruction checks in upstream `callees` (mrexodia/ida-pro-mcp#549). REA requests depth zero: the pinned upstream emits the root's outgoing edges without scanning child functions, then REA batches function-name lookup for those addresses. The graph covers recognized function destinations only; imports and other non-function targets remain unknown. A server without `callgraph` returns an empty relationship list with an explicit coverage limitation. Graph and path workflows retain those limitations alongside observed edges. A failed or inconsistent graph reply remains an error. A transport-level truncated preview is rejected rather than reported as complete evidence; the error preserves upstream retrieval guidance.
|
||||
|
||||
All profiles expose `list_documents` as the single target admitted for the session. Attached mode verifies its digest against IDA metadata before returning it; headless mode binds the digest-checked private input. `inspect_address_context`, `trace_feature`, and `binary_overview` can therefore resolve that identity without upstream document-listing tools. Segment enumeration is optional: when the selected profile cannot provide it, `binary_overview` still returns document and procedure/string counts and omits `segments` and `segment_count`; omission means unobserved, not empty. The modern headless profile also supports `resolve_containing_procedure`. An address resolves through upstream `lookup_funcs` to the canonical entry of the function that contains it; the body stays unavailable like other IDA identities. When no function contains it, REA reads the database-scoped minimal `survey_binary` and reports `not_in_procedure` for an address inside one of its segments (ends are exclusive) and `outside_segments` otherwise. A missing, truncated or inconsistent survey fails the read instead of guessing a reason. `trace_feature` marks a trace truncated, with a residual unknown for that address, when upstream caps the references it returned. Attached profiles do not declare address containment resolution.
|
||||
@@ -247,7 +175,7 @@ The verifier requires the resolved procedure in both inventory and procedure sea
|
||||
|
||||
The initial real workflows use stdio registrations for the legacy Windows GUI profile and a Windows x64 headless supervisor with IDA 9.3. The headless workflow has also passed on macOS arm64 with IDA Professional 9.1 (build 9.1.250226) against the same upstream commit `c133c3853faa111a9b00ee615c013b720d0c4acd`, using a source-built arm64 Mach-O fixture; Mach-O C symbols carry a leading underscore, so the procedure name is `_rea_fixture_add`. Streamable HTTP is tested against a loopback MCP fixture with the pinned client SDK; a real IDA HTTP-supervisor workflow remains unverified. Linux headless engine operation, other IDA releases, architectures, and modern attached GUI tools require their corresponding real workflows before support is claimed.
|
||||
|
||||
Native Windows verification uses IDA Pro 9.3 with the public two-function fixture and Windows `whoami.exe`, using upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c` with a loopback-listener change. It covers production CLI/MCP parity, distinct-function switching, identity and connection cleanup. See the [correctness and performance measurements](https://github.com/morluto/rea/blob/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency/ida-native-windows.md). This follow-up did not run Windows IDA Free or native macOS.
|
||||
Native Windows verification uses IDA Pro 9.3 with the public two-function fixture and Windows `whoami.exe`, using upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c` with a loopback-listener change. It covers production CLI/MCP parity, distinct-function switching, identity and connection cleanup. See the [correctness and performance measurements](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). This follow-up did not run Windows IDA Free or native macOS.
|
||||
|
||||
Headless readiness distinguishes an explicit `auto_analysis_ready: false` from
|
||||
unknown readiness (`null`). The former returns a retryable capability refusal
|
||||
|
||||
@@ -528,12 +528,6 @@ message names one reason:
|
||||
| `image-range-overlap` | Two images claim the same type definitions. |
|
||||
| `invalid-string` | A name index is outside the strings section, its string is unterminated, or it is not valid UTF-8. |
|
||||
|
||||
Record extents are checked for the decoded images, types, methods, fields,
|
||||
properties, events, parameters, string literals, nested types, generic
|
||||
containers, generic parameters and interface offsets. Other header sections
|
||||
are checked for file bounds and overlap; their internal record layouts are not
|
||||
decoded or validated by this inventory.
|
||||
|
||||
A version field that disagrees with the file's layout, for example a version
|
||||
39 file relabeled as 29, usually surfaces as `unexpected-header-size` or
|
||||
`record-size-mismatch`.
|
||||
|
||||
+14
-13
@@ -47,19 +47,20 @@ For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
|
||||
|
||||
## Choose a guide
|
||||
|
||||
| Task | Guides |
|
||||
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
|
||||
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [PE imports](pe-imports.md), [PE TLS/load config](pe-load-config.md), [ELF notes](elf-notes.md), [Linux kernel modules](kernel-modules.md), [Apple IPS](apple-ips.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
|
||||
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [GameCube DOL](ghidra-dol.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
|
||||
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
|
||||
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
|
||||
| Inspect assemblies, packages, or firmware | [Godot PCK](godot-pck.md), [UnityFS](unityfs.md), [U-Boot uImage](uimage.md), [.NET](managed-code-analysis.md), [Python packages](python-packaged-applications.md), [Android](android-analysis.md), [APK signatures](apk-signature.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [ESP images](esp-image.md), [ESP flash images](esp-flash-image.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
|
||||
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
|
||||
| Serve recorded behavior as an agent environment | [Agent environments](agent-environments.md) |
|
||||
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
|
||||
| Capture and compare a command | [Process capture](process-capture.md) |
|
||||
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
|
||||
| Task | Guides |
|
||||
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
|
||||
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [PE imports](pe-imports.md), [PE TLS/load config](pe-load-config.md), [ELF notes](elf-notes.md), [Linux kernel modules](kernel-modules.md), [Apple IPS](apple-ips.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
|
||||
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [PE imports](pe-imports.md), [PE TLS/load config](pe-load-config.md), [ELF notes](elf-notes.md), [Apple IPS](apple-ips.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
|
||||
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [GameCube DOL](ghidra-dol.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
|
||||
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
|
||||
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
|
||||
| Inspect assemblies, packages, or firmware | [Godot PCK](godot-pck.md), [.NET](managed-code-analysis.md), [Python packages](python-packaged-applications.md), [Android](android-analysis.md), [APK signatures](apk-signature.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
|
||||
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
|
||||
| Serve recorded behavior as an agent environment | [Agent environments](agent-environments.md) |
|
||||
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
|
||||
| Capture and compare a command | [Process capture](process-capture.md) |
|
||||
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
|
||||
|
||||
For development, use [testing](testing.md), [client verification](client-verification.md),
|
||||
[tool design](tool-design.md), [admission limits and recovery](design/limits-and-recovery.md),
|
||||
|
||||
@@ -1,72 +0,0 @@
|
||||
# Classic iNES container inspection
|
||||
|
||||
REA recognizes classic iNES from the `NES` + `0x1a` byte signature, independently
|
||||
of the file extension. This read-only container inspection uses portable
|
||||
Node.js file reads; it needs neither Ghidra nor an emulator.
|
||||
|
||||
```bash
|
||||
rea inspect-artifact ./cartridge.nes --json
|
||||
rea extract-artifact ./cartridge.nes --json
|
||||
```
|
||||
|
||||
Through MCP, call `open_binary` with the local path, then `inspect_artifact`.
|
||||
Use `extract_artifact` to materialize the regions in the fresh directory named
|
||||
by its result. The caller owns that successful extraction directory.
|
||||
|
||||
## Header and file regions
|
||||
|
||||
Inspection returns `container-header` and `reconstruction` observations inline.
|
||||
The single inventory Evidence under `substeps[].evidence.normalized_result`
|
||||
contains the complete graph and an `ines` object:
|
||||
|
||||
- `ines.header`: original 16-byte `header_hex`, PRG/CHR file-unit counts, mapper,
|
||||
raw flags 6/7, trainer and battery flags, and the declared mirroring mode.
|
||||
- `occurrences[]`: each region's path, `source_location.offset` and
|
||||
`source_location.length` in the original container. Follow `artifact_id` to
|
||||
the node's observed SHA-256 and byte count.
|
||||
- `ines.reconstruction`: ordered region paths, concatenated byte count,
|
||||
SHA-256 and `verified: true`. REA accumulates the concatenation digest during
|
||||
the same physical-order region scan that produces node hashes, then compares
|
||||
it with the original whole-file digest. Final source freshness is also checked.
|
||||
No reconstructed file is written during inspection.
|
||||
|
||||
Regions are `header.bin`, optional `trainer.bin` (512 bytes),
|
||||
`prg/000.bin` onward (16 KiB each), `chr/000.bin` onward (8 KiB each), and
|
||||
optional `trailing.bin`. Every input byte is retained, including unused header
|
||||
fields and uninterpreted trailing data. Zero CHR-ROM units create no CHR region;
|
||||
REA does not fabricate CHR-RAM contents.
|
||||
|
||||
After extraction, concatenate the materialized files in
|
||||
`ines.reconstruction.region_order` from the inspection inventory. Their bytes
|
||||
reproduce the input exactly. Graph path order is lexical and is not physical
|
||||
file order; use the explicit region order or source offsets.
|
||||
|
||||
## Supported boundary
|
||||
|
||||
This increment supports classic iNES with a complete 16-byte header, nonzero
|
||||
PRG count, complete declared trainer/PRG/CHR payload, and zero reserved bytes
|
||||
12–15. Dirty legacy headers are refused because their upper mapper bits are
|
||||
ambiguous; REA does not silently repair them. NES 2.0 and UNIF signatures,
|
||||
unsupported header variants and truncated payloads are explicitly refused.
|
||||
An arbitrary file named `.nes` is not admitted as iNES.
|
||||
|
||||
The PRG/CHR units describe file storage, not mapper bank sizes or CPU windows.
|
||||
The mapper number and flags are header declarations, not proof of cartridge
|
||||
hardware or runtime behavior. Bank switching, 6502/65816 disassembly, emulator
|
||||
sessions and interpretation of console-specific trailing data remain outside
|
||||
this lane. Raw-binary Ghidra analysis is a separate workflow. This container
|
||||
increment addresses the first milestone of [#2072](https://github.com/morluto/rea/issues/2072);
|
||||
the related [#1641](https://github.com/morluto/rea/issues/1641) tracks 6502/65816
|
||||
native analysis with explicit load addresses and provider limitations. Extracted
|
||||
PRG units still need caller-selected CPU windows before static code analysis;
|
||||
their physical file offsets do not establish those addresses.
|
||||
|
||||
Source-owned byte fixtures verify original offsets and hashes, extraction and
|
||||
byte-exact reconstruction, CLI/stdio MCP parity, malformed/unsupported inputs,
|
||||
failed target-switch recovery, cancellation and source identity. These checks
|
||||
do not establish emulator behavior or compatibility with every historical ROM.
|
||||
The portable CLI/stdio MCP acceptance and source-identity checks run in Linux
|
||||
and macOS CI. The same workflows have also been executed on native Windows x64.
|
||||
|
||||
Format references: [iNES](https://www.nesdev.org/wiki/INES) and
|
||||
[NES 2.0](https://www.nesdev.org/wiki/NES_2.0).
|
||||
+74
-299
@@ -3,14 +3,13 @@
|
||||
Run REA, connect your selected agents, and check the tools needed for your task.
|
||||
Setup previews configuration changes and asks for approval before applying them.
|
||||
|
||||
| Task | Section |
|
||||
| ---------------------------------- | --------------------------------------------------------------------------------------------------- |
|
||||
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
|
||||
| Install a Claude Code/Codex plugin | [Agent plugins and marketplaces](#agent-plugins-and-marketplaces) |
|
||||
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
|
||||
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), the [IDA guide](ida-provider.md), or [Kuna](kuna-provider.md) |
|
||||
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
|
||||
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
|
||||
| Task | Section |
|
||||
| --------------------------------- | --------------------------------------------------------------------------------------------------- |
|
||||
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
|
||||
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
|
||||
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), the [IDA guide](ida-provider.md), or [Kuna](kuna-provider.md) |
|
||||
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
|
||||
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
|
||||
|
||||
## Start setup
|
||||
|
||||
@@ -85,13 +84,6 @@ For an existing configuration, setup and uninstall recovery refuse a detected
|
||||
replacement or edit and name the recovery backup. Quit clients before editing their configuration;
|
||||
these checks do not lock out another writer modifying the same file in place,
|
||||
or exclude pathname changes between the final check and publication.
|
||||
Setup refreshes an existing `rea` registration only when its launcher proves REA
|
||||
ownership. Foreign or unresolved launcher identity blocks setup before any
|
||||
backup or write, including an OpenCode legacy entry that migration would remove.
|
||||
Review the existing launcher and manually rename or remove that registration to
|
||||
approve replacing it. A different existing installation path is foreign to the
|
||||
current entrypoint; a path that cannot be resolved remains unknown. Cordis uses
|
||||
its stable REA entry ID instead of launcher identity.
|
||||
Setup also retains valid environment overrides in REA's existing server entry,
|
||||
such as `REA_MCP_INPUT_SCHEMA_PROFILE`, `REA_MCP_OUTPUT_SCHEMA_PROFILE`, and
|
||||
`REA_MCP_MAX_RESPONSE_BYTES`, and `REA_MCP_TOOL_FAMILIES`.
|
||||
@@ -170,9 +162,7 @@ Setup leaves existing skill copies in other locations untouched.
|
||||
Setup refuses symbolic links at the selected skill directory or its managed
|
||||
`references` directory, matching uninstall and update's ownership rules. Use a
|
||||
regular skill directory, or select `--skill=false` to configure the client
|
||||
registration without installing the bundled skill. A non-directory ancestor is
|
||||
reported as `ENOTDIR`; preserve the blocking file and correct the directory path
|
||||
before rerunning setup.
|
||||
registration without installing the bundled skill.
|
||||
The final selected directories are checked before approval and again before
|
||||
setup changes any client or Hopper installation. Unselected client destinations
|
||||
do not block other setup choices. Doctor reports linked or non-directory skill
|
||||
@@ -275,7 +265,7 @@ After reviewing an update plan, rerun that same prefixed setup command without
|
||||
`--dry-run` and approve its exact changes. Profile removal likewise needs the
|
||||
prefix on `claude mcp remove --scope user rea` and any `rea uninstall` command;
|
||||
keep the broader client scope of uninstall in mind. Restart the selected
|
||||
profile after changing its registration.
|
||||
profile after changing its pinned registration.
|
||||
|
||||
`claude mcp remove` removes only the registration. To remove REA from this
|
||||
profile without running the broader uninstall flow, first back up the profile's
|
||||
@@ -325,26 +315,30 @@ commands. Record an existing selection first if you intend to restore it after
|
||||
finishing. The configuration path below is the default profile's
|
||||
`~/.claude.json`; a different active profile uses its own config root instead.
|
||||
|
||||
Register with the variable unset, substituting the absolute runtime and entry
|
||||
script from your existing installation or setup dry-run:
|
||||
Register with the variable unset:
|
||||
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```bash
|
||||
claude mcp add --scope local rea -- "/absolute/path/to/node" "/absolute/path/to/rea/scripts/rea.mjs" mcp
|
||||
claude mcp add --scope local rea -- npx -y rea-agents@6.8.0 mcp
|
||||
```
|
||||
|
||||
On native Windows, use the same project directory and absolute paths:
|
||||
On native Windows, use the same project directory and wrap the package-runner
|
||||
shim so the stdio client can launch it:
|
||||
|
||||
```powershell
|
||||
claude mcp add --scope local rea -- "C:\Program Files\nodejs\node.exe" "C:\tools\rea\scripts\rea.mjs" mcp
|
||||
claude mcp add --scope local rea -- cmd.exe /d /c npx -y rea-agents@6.8.0 mcp
|
||||
```
|
||||
|
||||
<!-- x-release-please-end -->
|
||||
|
||||
Local scope stores the entry under that directory's path in `~/.claude.json`,
|
||||
so other projects don't load it. This route doesn't install the bundled skill.
|
||||
Avoid `--scope project`, which writes `.mcp.json` into the repository where it
|
||||
can be committed or copied. `rea doctor` and `rea uninstall` read only the
|
||||
user-level entry, so they don't report or remove a local-scope registration;
|
||||
remove it with `claude mcp remove --scope local rea` from the same directory.
|
||||
`rea update` doesn't refresh it either, so it continues using the selected installation.
|
||||
`rea update` doesn't refresh it either, so it stays on the version it names.
|
||||
To move it to a newer release, first back up the project entry in
|
||||
`~/.claude.json` and record its environment and other metadata. In particular,
|
||||
preserve caller-selected overrides such as `REA_MCP_INPUT_SCHEMA_PROFILE` and
|
||||
@@ -381,7 +375,7 @@ Select the part you want to check:
|
||||
| Task | Readiness check |
|
||||
| ----------------------------------- | ------------------------------------------------------------------------ |
|
||||
| Static JavaScript/Electron analysis | Run `rea analyze-javascript-application PATH --json` directly. |
|
||||
| One analysis engine | `rea doctor --provider ghidra --json` (or `hopper`, `ida`, `kuna`) |
|
||||
| One analysis engine | `rea doctor --provider ghidra --json` (or `hopper`, `ida`) |
|
||||
| One agent registration | `rea doctor --client codex --json` (see [client IDs](#supported-agents)) |
|
||||
| Installed workflow instructions | `rea doctor --skill --json` |
|
||||
|
||||
@@ -390,20 +384,6 @@ A scoped report has `scope.mode: "explicit"`. Its `scope_checks` determine
|
||||
`environment_healthy` summarizes the full audit. `--target` adds a target check,
|
||||
but the report remains audit-wide unless a scope option is also supplied.
|
||||
|
||||
Provider checks read the terminal running doctor. A client-started server may
|
||||
receive a different environment, and doctor cannot observe it. When an available
|
||||
provider relies on a setting that a registration's file does not record, or
|
||||
records with other text, doctor adds a non-blocking
|
||||
`registration_environment:<client>` check with `classification: "unknown"`. The
|
||||
setting names appear in that registration's `provider_settings`. Selecting the
|
||||
client or an affected provider places the check in `scope_checks`; it never
|
||||
changes `healthy`, because the client may still pass the setting on. Preview
|
||||
`rea setup --client <client-id> --skill=false --dry-run` from that terminal and
|
||||
approve it to record the settings, then restart the client. To see what the
|
||||
running server can use, call `binary_session` in the client and read
|
||||
`result.tool_availability`. For Hopper, only an explicit `HOPPER_LAUNCHER_PATH`
|
||||
is compared.
|
||||
|
||||
An invalid `REA_LOG_LEVEL` fails every scope, including one client or the
|
||||
installed skill, because the executable rejects it before any command. A
|
||||
selected client also fails for every other setting that MCP startup rejects,
|
||||
@@ -457,19 +437,11 @@ and other MCP servers. Purging removes REA's legacy cache and state under
|
||||
archives under `${XDG_CACHE_HOME:-~/.cache}/rea/hopper-releases`. Other cache
|
||||
namespaces remain intact. A caller-selected `REA_KUNA_CACHE_DIR` is retained
|
||||
and reported because REA cannot establish ownership of its entire contents.
|
||||
Lexically equivalent spellings of that default directory, including a trailing
|
||||
separator, a `.` segment, or a repeated separator, share its purge result.
|
||||
The comparison preserves POSIX `..` traversal and literal backslashes.
|
||||
A client configuration that is malformed, unreadable, or at an unsafe
|
||||
path stops the operation before anything is removed when that client's
|
||||
product applicability is established or unresolved. Shared registration
|
||||
storage such as `~/.agents/mcp.json` whose product cannot be established
|
||||
(for example when FreeBuff and ZCode discovery directories are absent) is
|
||||
retained unchanged and does not block other REA removals, including cache
|
||||
and state deletion with `--purge-data`. A client that fails while being
|
||||
updated stops the remaining removals. Uninstall reports each candidate
|
||||
`configuration_path` with its outcome. A client with multiple files keeps
|
||||
their individual `actions` alongside its aggregate status and detail,
|
||||
path stops the operation before anything is removed, and a client that fails
|
||||
while being updated stops the remaining removals. Uninstall reports each
|
||||
candidate `configuration_path` with its outcome. A client with multiple files
|
||||
keeps their individual `actions` alongside its aggregate status and detail,
|
||||
including earlier removals when a later file fails. The paths name
|
||||
the discovered configuration candidates, not resolved physical file identities.
|
||||
For JSON clients, uninstall removes every repeated `rea` member in the effective
|
||||
@@ -565,8 +537,8 @@ non-TTY, and JSON modes without applying configuration changes.
|
||||
## Released package and main
|
||||
|
||||
Repository main documents its current code and generated catalog. `@latest`
|
||||
selects the npm release, and managed MCP registrations bind the installation
|
||||
that performed setup. Installing newer instructions does not update a
|
||||
selects the npm release, and persistent MCP registrations are pinned to the
|
||||
version that performed setup. Installing newer instructions does not update a
|
||||
running server or its registration.
|
||||
|
||||
The npm release checked on **2026-10-11** was **6.4.0**, published from
|
||||
@@ -680,8 +652,7 @@ Hopper is separate commercial software with its own license. Its free demo has
|
||||
vendor-defined limits, and a paid license is optional. REA reuses any detected
|
||||
installation and preserves Hopper during uninstall.
|
||||
|
||||
The automated Hopper installer supports macOS 12+, subject to the minimum
|
||||
macOS version of the current Hopper release, and x86_64 hosts running
|
||||
The automated Hopper installer supports macOS 12+ and x86_64 hosts running
|
||||
Ubuntu 24.04+, Fedora 41+, Nobara 44+, Arch Linux, CachyOS, or Omarchy. The
|
||||
official Linux packages are x86_64-only, so setup does not offer Hopper on other
|
||||
Linux architectures; configure [Ghidra](#ghidra) there instead. REA itself runs on any
|
||||
@@ -690,33 +661,12 @@ an existing Hopper launcher is selected through `HOPPER_LAUNCHER_PATH`. Ghidra
|
||||
and IDA have their own provider-specific host requirements; Windows Ghidra uses
|
||||
the [experimental P0 boundary](windows-ghidra-p0.md).
|
||||
|
||||
Bazzite uses RPM packages, but REA does not install Hopper automatically on it.
|
||||
Setup skips the Hopper offer because Bazzite is outside the supported host list,
|
||||
regardless of whether Xvfb is installed. Bazzite documents
|
||||
[containers for installing Linux packages](https://docs.bazzite.gg/Installing_and_Managing_Software/Containers/).
|
||||
If using a container, check its distribution and architecture against the list
|
||||
above. Install REA and Hopper in that container, give it access to the files you
|
||||
want to analyze, and configure your agent to launch REA there. Verify installation
|
||||
and analysis before relying on that workflow.
|
||||
|
||||
`rea doctor` checks health; it does not offer to install Hopper. Run `rea setup`
|
||||
for installation. Setup cannot prompt when its output is redirected or during a
|
||||
container image build. To approve Hopper installation without prompts, run
|
||||
`rea setup --yes --install-hopper`.
|
||||
|
||||
If a Hopper request fails on macOS or Linux, the setup result's `download`
|
||||
field names the request (`release_feed`, `package` or `public_key`), its `url`,
|
||||
and the observed outcome `kind`: `http_status` with `httpStatus`, `timeout`,
|
||||
`network`, `response_too_large`, or `url_refused`. The remediation names the
|
||||
same URL and outcome. HTTP 404 or 410 means the file is missing, so an
|
||||
immediate retry is unlikely to help; 5xx, 408 and 429 indicate a temporary
|
||||
service failure. The status does not establish the underlying cause or guarantee
|
||||
that retrying will succeed. Doctor and Linux launches report the same request
|
||||
and status when publisher verification cannot download the current package.
|
||||
If the Linux package download receives an HTTP failure, setup includes the
|
||||
observed HTTP status in its `download_failed` diagnostic. The status does not
|
||||
establish the underlying cause or guarantee that retrying will succeed.
|
||||
|
||||
Hopper metadata and package downloads require HTTPS throughout the redirect
|
||||
chain; a redirect to HTTP or to a URL carrying credentials is refused before
|
||||
that destination is requested, and those credentials are not reported.
|
||||
chain; a redirect to HTTP is refused before that destination is requested.
|
||||
|
||||
On macOS, approved setup downloads the official DMG, checks its published size
|
||||
and digest, validates the application bundle, and atomically installs it to
|
||||
@@ -724,15 +674,6 @@ and digest, validates the application bundle, and atomically installs it to
|
||||
operator can choose its demo mode or activate an existing license. Homebrew and
|
||||
administrator access are not used.
|
||||
|
||||
Each Hopper release declares its own minimum macOS version, so the setup plan
|
||||
cannot confirm support before approval. Setup compares this Mac's version with
|
||||
the release metadata's `deployment_target` before downloading the DMG, and with
|
||||
the signed application's `LSMinimumSystemVersion` before copying it. If this Mac
|
||||
is older, setup fails with `incompatible_host_version`, names both versions, and
|
||||
installs nothing. Update macOS, or install a Hopper release that supports this
|
||||
macOS version yourself and set `HOPPER_LAUNCHER_PATH`. An existing Hopper
|
||||
installation is not checked against the current release's minimum.
|
||||
|
||||
The Hopper launcher action used by REA creates a document from an executable;
|
||||
its supported command-line interface does not attach to an already-open
|
||||
document. While another live REA session owns the same target and loader
|
||||
@@ -773,12 +714,6 @@ Rotating signing subkeys remain bound to that primary identity.
|
||||
The feed row itself is not PGP-signed; doctor labels its display version as
|
||||
`reported_release_version`, separately from the authenticated launcher identity.
|
||||
|
||||
Even a correctly signed package can contain a launcher whose hash disagrees with
|
||||
the feed. REA stops before installing that package. When reporting this error,
|
||||
include the release version, package format and both hashes from the diagnostic.
|
||||
Each format needs its own check: a matching Ubuntu DEB does not establish that
|
||||
the Fedora RPM matches.
|
||||
|
||||
The approval plan includes the caller-selected cache location. Approved setup
|
||||
retains the signed archive, public key and release metadata under
|
||||
`${XDG_CACHE_HOME:-$HOME/.cache}/rea/hopper-releases/<launcher-sha256>`.
|
||||
@@ -812,16 +747,6 @@ host mount and the rest of `/tmp` remain unchanged; this fallback never invokes
|
||||
`sudo`. `rea doctor --provider hopper --json` reports the selected
|
||||
strategy and both host and effective mount facts.
|
||||
|
||||
You do not need to start a separate `Xvfb :99` server. REA starts and manages its
|
||||
own display. After installation, run `rea doctor --provider hopper --json` and
|
||||
check `hopper-demo-runtime` for display and Python problems. Setup also runs these
|
||||
health checks: installing the package is only one step toward a working provider.
|
||||
|
||||
If an image build ends with `exit status 1`, read the setup output immediately
|
||||
before that summary to find the failing step. On minimal Ubuntu images, run
|
||||
`apt-get update` before setup if an earlier layer deleted the package indexes.
|
||||
REA's package installation commands do not refresh those indexes.
|
||||
|
||||
Hopper also embeds Python; the Python executable and standard library alone
|
||||
are insufficient on Ubuntu. Approved setup checks the system interpreter
|
||||
at `/usr/bin/python3` and, when necessary, installs its matching
|
||||
@@ -916,13 +841,9 @@ stage `connection`.
|
||||
If macOS denies Apple Events, allow the application that starts REA under
|
||||
**System Settings → Privacy & Security → Automation**, then retry. REA releases
|
||||
the target reservation when the native helper reports its complete authorization
|
||||
rejection after literal argument admission. Before application handoff, REA refuses
|
||||
double quotes, backslashes, or line breaks in the final launcher path or arguments,
|
||||
with `details.launch_rejection.boundary: "argument_encoding"` and the affected source/index.
|
||||
Select literal-safe launcher/runtime paths and loader arguments, or another
|
||||
compatible provider. REA preserves original file names and evidence: a quoted
|
||||
executable name can still work when its prepared image path is literal-safe;
|
||||
database paths must themselves be representable. Other failed handoffs
|
||||
rejection and its arguments cannot alter the helper's generated AppleScript.
|
||||
Arguments containing quotes, backslashes, or line breaks remain unchanged but
|
||||
cannot establish that a failed handoff opened no document. Other failed handoffs
|
||||
retain ownership until document closure is confirmed. If a timed-out bridge
|
||||
becomes ready later, `close_binary` connects to that run, authenticates its
|
||||
identity, and requests document shutdown. Once its listener is ready, the bridge
|
||||
@@ -989,12 +910,7 @@ selected only when it is the arm64 family's only candidate. REA checks the
|
||||
entire source's SHA-256 while copying the slice; source identity and reported
|
||||
offsets still refer to the original container. Any other ambiguous architecture
|
||||
subtype requires an explicitly extracted thin image; REA does not guess.
|
||||
Set `HOPPER_LOADER_ARGS_JSON` in REA's launch environment to a JSON array of
|
||||
strings to override the derived Hopper loader arguments, for example
|
||||
`["-l","Mach-O"]`. Empty string elements are preserved; NUL bytes are not
|
||||
accepted. An omitted setting or empty array uses the derived arguments. `rea
|
||||
setup` forwards an explicitly selected setting to supported registrations;
|
||||
restart the MCP server to apply a change. These arguments are Hopper-specific.
|
||||
Configured loader arguments remain explicit overrides.
|
||||
FAT64 is distinct from the CPU architecture: FAT32 containers can contain 64-bit
|
||||
executable images. This preparation avoids the Raw Binary loader dialog observed
|
||||
with native FAT64 loading on Hopper 6.1.0-demo.
|
||||
@@ -1032,9 +948,7 @@ including while a shutdown request is in flight. A helper exit alone is
|
||||
insufficient, and an application that never authenticated has no verified
|
||||
process identity. If no matching bridge becomes available, inspect and close
|
||||
the reported document in Hopper before ending the owning REA connection.
|
||||
`close_binary` preserves Evidence and complete Unknown history in memory on the
|
||||
same connection. Export the bundle before ending the REA process to resume later;
|
||||
`reset_investigation(save_path)` checkpoints and explicitly clears those records.
|
||||
Export the Evidence bundle before `close_binary` clears session records.
|
||||
`binary_session.cleanup_pending` remains true while provider cleanup is retained,
|
||||
and provider health remains visible even with `open: false`; that flag alone
|
||||
does not establish successful cleanup.
|
||||
@@ -1056,14 +970,6 @@ When Hopper cannot start in CI, run
|
||||
`rea doctor --provider hopper --json` in the failing runner.
|
||||
Structured failures distinguish private-display dependencies, an unsupported
|
||||
demo dialog or build, process-ownership conflicts, and an early lifecycle exit.
|
||||
Once launcher authentication succeeds, an absent first-run dialog is reported as
|
||||
`demo_dialog_absent` rather than a build or publisher rejection. The watcher
|
||||
waits 10 seconds by default. Set `REA_HOPPER_DEMO_DIALOG_TIMEOUT_MS` in the
|
||||
environment that starts REA to a positive decimal integer of milliseconds from
|
||||
1 to 2147483647 when a slower host needs a longer window. An invalid value is
|
||||
`invalid_dialog_timeout`: the error names that setting, shows the rejected
|
||||
value and its source, lists the allowed range, and asks you to correct the
|
||||
setting and restart REA.
|
||||
If a private-display probe times out or is cancelled and its process cleanup
|
||||
fails, REA retains that exact probe owner and reports `cleanup_incomplete`.
|
||||
Session close retries the probe before releasing its cleanup marker; doctor
|
||||
@@ -1096,7 +1002,7 @@ reported as incompatible instead of being admitted from its directory name, and
|
||||
a header that cannot be read or recognized is reported as unknown. REA does not
|
||||
build or install native tools, or change Gatekeeper quarantine settings.
|
||||
|
||||
The adapter exposes 27 read-only operations: fifteen inventory/name/search
|
||||
The adapter exposes 26 read-only operations: fourteen inventory/name/search
|
||||
operations and twelve function-analysis operations. These cover metadata,
|
||||
decompilation, assembly, resolved calls, typed references, xrefs, function
|
||||
dossiers, instructions, recovered data types, measured load mappings, loaded
|
||||
@@ -1175,15 +1081,14 @@ rea providers --json
|
||||
```
|
||||
|
||||
`rea setup` can configure supported Windows agents and install the bundled
|
||||
REA skill after approval. Managed registrations use the absolute selected Node or
|
||||
Bun runtime and REA entry script, including when setup ran through a package runner. Hopper
|
||||
REA skill after approval. Direct registrations use Node to launch REA's entry
|
||||
script; package-runner registrations use the pinned `npx` or `bunx` command. Hopper
|
||||
installation remains unavailable on Windows. Setup never installs Ghidra,
|
||||
Java, or Python. It preserves valid detected Ghidra/JDK settings in agent
|
||||
registrations. See [Windows Ghidra P0](windows-ghidra-p0.md) for provider diagnostics.
|
||||
|
||||
The separate `rea mcp add` builtin delegates most clients to Incur's package-runner
|
||||
child launcher. Run that acquisition from a trusted directory; prefer managed
|
||||
`rea setup` when supported. That child launcher can fail on Windows with `MCP_ADD_FAILED: spawn npx ENOENT`
|
||||
The separate `rea mcp add` builtin currently uses Incur's package-runner child
|
||||
launcher, which can fail on Windows with `MCP_ADD_FAILED: spawn npx ENOENT`
|
||||
before registering a client. REA's setup path is unaffected. For a client supported
|
||||
by setup, review `rea setup --client opencode --skill=false --dry-run --json`
|
||||
(substitute its client ID), then use the same scoped command without `--dry-run`
|
||||
@@ -1288,10 +1193,8 @@ bridge, and health readiness. Large binaries can need more: set
|
||||
milliseconds in the server environment. An absent setting uses the default;
|
||||
empty, invalid or out-of-range supplied values fail configuration validation.
|
||||
When the deadline elapses, the `provider_timeout` error names
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS` with the deadline that was applied. In
|
||||
fresh-import mode it also suggests `REA_GHIDRA_DISABLED_ANALYZERS` for analyzers
|
||||
that dominate the time and `REA_GHIDRA_PROJECT_PATH` for reusing a completed
|
||||
analysis; in existing-project mode it omits that import and analyzer advice.
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS` with the deadline that was applied and suggests
|
||||
`REA_GHIDRA_DISABLED_ANALYZERS` for analyzers that dominate the time.
|
||||
REA parses the supplied configuration environment and
|
||||
passes the deadline to each provider client; a startup failure is returned by the query that triggered it,
|
||||
without exposing partial analysis. This deadline is separate from MCP transport
|
||||
@@ -1345,8 +1248,7 @@ fails with `provider_timeout` and that message; other Ghidra tools stay
|
||||
available. If the caller also cancels while the timeout settles, the session
|
||||
still closes and the message says so. Session reuse concerns an active MCP or
|
||||
library session; a standalone CLI command still closes its own session when it
|
||||
finishes, so its timeout names `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS` for a new
|
||||
invocation instead. In MCP, `ghidra.decompile_timeout_seconds` can override the launch
|
||||
finishes. In MCP, `ghidra.decompile_timeout_seconds` can override the launch
|
||||
value for newly accepted operations on the existing Program; explicit `null`
|
||||
removes the deadline. A transient timeout remains uncached so the same function
|
||||
can be retried after an update. See the
|
||||
@@ -1483,33 +1385,6 @@ REA passes `-processor`/`-cspec`, checks both against the installation's `.ldefs
|
||||
|
||||
Language definitions must be well-formed XML encoded as UTF-8 or BOM-marked UTF-16, with any encoding declaration matching the bytes. Literal Unicode replacement characters (U+FFFD) are accepted; malformed source bytes are rejected.
|
||||
|
||||
Ghidra imports little-endian ARM ELF files, including Cortex-M
|
||||
(`arm-none-eabi`) firmware, as `ARM:LE:32:v8`. That language defaults to ARM
|
||||
state, and its ELF loader marks code as Thumb only where `$t` mapping symbols
|
||||
or odd function-symbol addresses say so. Cortex-M cores run only Thumb code, so
|
||||
the gap matters mainly for code without those markers, such as stripped images
|
||||
and handlers reached only through the vector table. REA has no M-profile ARM
|
||||
lane. As an unverified interim, select an M-profile language, which decodes in
|
||||
Thumb state by default. For ARMv6-M, ARMv7-M and ARMv7E-M cores:
|
||||
|
||||
```bash
|
||||
export REA_GHIDRA_LANGUAGE_ID="ARM:LE:32:Cortex"
|
||||
```
|
||||
|
||||
For ARMv8-M cores, use `ARM:LE:32:v8-m` instead. REA has not verified either
|
||||
language for Cortex-M analysis. While the variable is set, it applies to every
|
||||
Ghidra import in the REA server:
|
||||
|
||||
- raw-binary, DOS, GameCube DOL and PSP opens are refused, as are opens in
|
||||
lanes that reject a different language, such as RISC-V;
|
||||
- REA asks Ghidra to import everything else in the selected ARM language,
|
||||
including x86, x86-64, arm64, PowerPC and Hexagon ELF files and PE and
|
||||
Mach-O files, because it does not check that the override's processor
|
||||
matches the target.
|
||||
|
||||
Restart REA without the variable before you analyze other targets. A future
|
||||
M-profile lane may refuse it; remove the setting then.
|
||||
|
||||
To seed code that auto-analysis cannot reach (computed jumps, pointers in data):
|
||||
|
||||
```bash
|
||||
@@ -1551,21 +1426,6 @@ current setting. For Windows PE resource analysis, the exact name is
|
||||
`WindowsResourceReference`. Disabling an analyzer skips the facts it would have
|
||||
produced.
|
||||
|
||||
Ghidra's Objective-C Message Analyzer can log
|
||||
`Required Objective-C data type not found in data type manager` /
|
||||
`Try adding libobjc.dylib` when a Mach-O links Objective-C runtime symbols but
|
||||
defines no class metadata (thin Cocoa glue in a C++ binary is a common case).
|
||||
That text is non-fatal Ghidra analyzer output: auto-analysis continues, and the
|
||||
analyzer simply skips `objc_msgSend` call-target recovery. REA's headless import
|
||||
does not accept a "add this library" option, and on modern macOS
|
||||
`/usr/lib/libobjc.A.dylib` exists only in the dyld shared cache, so there is no
|
||||
on-disk library to add. Treat the line as a coverage limitation, not an REA
|
||||
failure; check the session's `failure_kind` (and related provider errors) for
|
||||
any real abort. To silence that analyzer, set
|
||||
`REA_GHIDRA_DISABLED_ANALYZERS="Objective-C Message Analyzer"`. For large
|
||||
executables that miss the startup deadline instead, raise
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS` and the [Ghidra heap](#ghidra-heap-and-cpu-controls).
|
||||
|
||||
### Ghidra heap and CPU controls
|
||||
|
||||
Set resource controls in the environment that launches the CLI or MCP server,
|
||||
@@ -1661,93 +1521,12 @@ On a controlled Windows x64 runner, use
|
||||
`npm run verify:ghidra:windows`. The verifier generates a deterministic native
|
||||
PE fixture from source bytes and requires the Windows native authority before
|
||||
opening the provider. The controls are implemented on main; see the
|
||||
[release boundary](#released-package-and-main). The lane verifies all 27 admitted
|
||||
[release boundary](#released-package-and-main). The lane verifies all 26 admitted
|
||||
read-only operations, digest identity, and runtime cleanup. The separate
|
||||
`verify:ghidra:windows:package` lane checks the packed CLI/MCP boundary with an
|
||||
ordinary user. It requires the built Windows native bundle and an existing
|
||||
Ghidra/JDK installation; neither is inferred from startup alone.
|
||||
|
||||
## Agent plugins and marketplaces
|
||||
|
||||
REA's npm package is the shared Claude Code and Codex plugin artifact. It
|
||||
contains `.claude-plugin/plugin.json`, `.codex-plugin/plugin.json`, and the
|
||||
canonical generated `skills/` tree. Both manifests select `./skills` and run a plain Node bootstrap pinned to
|
||||
that exact package version. The client must have a supported standalone `node`
|
||||
on its PATH. Claude expands its documented `CLAUDE_PLUGIN_ROOT` in the script
|
||||
argument; Codex resolves `cwd: "."` from the installed plugin root before
|
||||
launching the relative script. The bootstrap verifies the package version and runs its bundled runtime in
|
||||
the same process, preserving stdio and REA's provider/worker cleanup. It does
|
||||
not start another package runner or download a different runtime. The existing release skill
|
||||
generator also pins CLI recovery commands to the same version.
|
||||
|
||||
The repository's single-plugin marketplaces are declared in
|
||||
`.claude-plugin/marketplace.json` and `.agents/plugins/marketplace.json`.
|
||||
Both select an exact `rea-agents` npm version instead of mutable source-checkout
|
||||
contents. Release Please updates their selectors alongside the plugin manifests.
|
||||
No plugin-only publication or generated skill files committed to Git are needed.
|
||||
The qualified release archive contains a runtime-only manifest and
|
||||
`npm-shrinkwrap.json` plus bundled runtime dependencies, including PTY packages
|
||||
for each supported OS and CPU; development dependencies and development-only overrides
|
||||
stay in the source manifest and lockfile. Both clients receive this same archive.
|
||||
|
||||
This distribution becomes installable after the first qualified npm release
|
||||
containing these plugin manifests. Earlier published packages are not changed
|
||||
by this source proposal. For local source development, run `npm ci && npm run build:cached` to generate
|
||||
the runtime and packaged skill before loading the repository as a plugin.
|
||||
A source plugin uses the locally built runtime and generated skill; rebuild after
|
||||
source changes. Missing builds and mismatched version pins fail with an actionable
|
||||
error. The authored `.agents/skills/` tree is not the plugin's shipped skill layout.
|
||||
Windows source workflows that need native controls still require the
|
||||
[Windows source-checkout build](https://github.com/morluto/rea/blob/main/native/windows/README.md#build).
|
||||
|
||||
In Claude Code:
|
||||
|
||||
```text
|
||||
/plugin marketplace add morluto/rea
|
||||
/plugin install rea@rea
|
||||
```
|
||||
|
||||
In Codex CLI, add the marketplace and install REA:
|
||||
|
||||
```bash
|
||||
codex plugin marketplace add morluto/rea
|
||||
codex plugin add rea@rea
|
||||
```
|
||||
|
||||
Each release updates both manifest versions and their pinned server version
|
||||
together, which is when installed copies refresh the bundled skill and move to
|
||||
the new server. A plugin does not run `rea setup`, so provider
|
||||
configuration still follows the sections above. Use the plugin or a
|
||||
`rea setup` registration for a client, not both, to avoid two `rea` servers.
|
||||
|
||||
`rea doctor` does not see plugin-owned state. Its client and skill checks read
|
||||
only the registrations and skill locations that `rea setup` manages, so for a
|
||||
plugin-only install `rea doctor --client <id>` and `rea doctor --skill` report
|
||||
`missing` and recommend `rea setup`. Do not follow that remediation. Check the
|
||||
plugin with the client instead (`claude plugin list` and `/mcp` in Claude
|
||||
Code, `codex plugin list` in Codex).
|
||||
|
||||
For analysis engines, ask the connected server: have the agent call
|
||||
`binary_session` and read `result.tool_availability`, which reports what the
|
||||
server the plugin started can use, with reasons and remediation. See
|
||||
[MCP contracts](mcp-contracts.md).
|
||||
|
||||
A plugin does not put `rea` on your `PATH`. To check an engine from a
|
||||
terminal, use the package runner at the version the plugin pins:
|
||||
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@6.8.0 doctor --provider ghidra --json
|
||||
```
|
||||
|
||||
<!-- x-release-please-end -->
|
||||
|
||||
If your installed plugin is older, use the version in its manifest instead.
|
||||
That report describes the terminal's environment, not the plugin server's.
|
||||
The server inherits the client's environment, so a setting such as
|
||||
`GHIDRA_INSTALL_DIR` that exists only in your terminal does not reach it.
|
||||
|
||||
## MCP Registry
|
||||
|
||||
REA is published in the official MCP Registry as `io.github.morluto/rea`. Registry
|
||||
@@ -1758,45 +1537,40 @@ For a client that supports Registry discovery, search for `io.github.morluto/rea
|
||||
and select the npm package. The published metadata launches the existing stdio
|
||||
server through `npx` with the `mcp` command.
|
||||
|
||||
For supported clients, prefer `rea setup`: it records the absolute JavaScript
|
||||
runtime and entry script of the REA installation performing setup. Client starts
|
||||
then use that installation directly, keeping the client's working directory
|
||||
without running npm or Bun package resolution there. Registry clients control
|
||||
their own package acquisition; Registry metadata cannot name local absolute paths.
|
||||
For a client that requires manual configuration, use:
|
||||
|
||||
For manual stdio configuration, substitute the absolute paths of an existing
|
||||
installation (use the command shown in a setup dry-run):
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "/absolute/path/to/node",
|
||||
"args": ["/absolute/path/to/rea/scripts/rea.mjs", "mcp"]
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@6.8.0", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Run current setup to migrate recognized older `npx`/`bunx` registrations, with
|
||||
backups, then restart the client. Unrelated or unidentified registrations remain
|
||||
untouched. Setup does not prewarm a separate package-runner command.
|
||||
<!-- x-release-please-end -->
|
||||
|
||||
If setup itself runs from a package-runner cache, the registration depends on
|
||||
that cache entry and the selected runtime remaining installed. Removing either
|
||||
makes startup fail; it does not download or resolve another package. Restore the
|
||||
same installation and rerun setup. Changing package requests, cache roots, or
|
||||
runtimes can create a different installation identity. If the old entry cannot be
|
||||
identified, restore it first or manually remove the confirmed REA registration
|
||||
before running setup again. A global installation provides a more durable path.
|
||||
Persistent registrations should use one exact package version. `rea setup`
|
||||
writes the same package-runner shape, or `bunx rea-agents@<version> mcp` when
|
||||
setup runs through `bunx`, and pins it to the exact version that performed
|
||||
setup. Run current setup to refresh an older registration, then restart the
|
||||
client.
|
||||
|
||||
Package runners key their package cache by the requested spec, so the entry
|
||||
installed for the `@latest` invocation that ran setup cannot serve the pinned
|
||||
registration. Setup therefore materializes the pinned entry once, best-effort
|
||||
with a bounded timeout, before finishing. When that step cannot run, for
|
||||
example offline, the first client start performs the one-time install inside
|
||||
the client's MCP startup deadline and may need one more restart.
|
||||
|
||||
## Package runners and global installation
|
||||
|
||||
Package runners can select project-local packages and read project configuration,
|
||||
even for explicit version requests. Acquire REA from a trusted working directory,
|
||||
outside the target project, and pass absolute target paths to analysis commands.
|
||||
REA cannot validate package acquisition that happens before its entrypoint starts.
|
||||
To request the latest release explicitly, use:
|
||||
A package-runner invocation can use a REA version installed in the current project. To
|
||||
request the latest release explicitly, use:
|
||||
|
||||
```bash
|
||||
npx rea-agents@latest setup
|
||||
@@ -1813,14 +1587,16 @@ For an intentional rollback, make the package request explicit:
|
||||
npm exec --yes --package=rea-agents@2.4.0 -- rea setup
|
||||
```
|
||||
|
||||
Setup binds persistent MCP registrations to the installation that performed setup.
|
||||
Running current setup later migrates recognized legacy package-runner registrations
|
||||
through the normal reviewed setup transaction.
|
||||
Setup continues to pin persistent MCP registrations to the exact version that
|
||||
performed setup. Running current setup later migrates unversioned or older
|
||||
managed registrations through the normal reviewed setup transaction.
|
||||
|
||||
With Bun, `bunx rea-agents setup` (or `bunx rea-agents@latest setup`) is the
|
||||
equivalent acquisition entry point. Managed registrations use the selected Bun
|
||||
executable and REA entry script directly. Windows registrations likewise use
|
||||
absolute runtime and script paths, without a package-runner command shim.
|
||||
equivalent entry point. Setup run through `npx` registers
|
||||
`npx -y rea-agents@<version> mcp`; setup run through `bunx` or `bun x` registers
|
||||
`bunx rea-agents@<version> mcp`. On Windows, setup wraps either command in
|
||||
`cmd.exe /d /c` so clients can start the runner's command shim. Neither
|
||||
registration points into the runner's temporary package cache.
|
||||
|
||||
REA supports Node.js 22.x (>=22.23.4), 24.x (>=24.22.1), and 26.12.1+. Node.js 23, 25, earlier 26.x releases, and prereleases are unsupported. It uses the npm already paired with that runtime and never upgrades Node.js, npm, or Homebrew.
|
||||
|
||||
@@ -1854,9 +1630,8 @@ Review the plan, approve the selected client changes, then restart those clients
|
||||
This registers the installed entrypoint directly; installing the global package
|
||||
alone does not replace existing `npx` registrations. Each client still has its own
|
||||
REA process and analysis state. Direct registrations follow the installed CLI;
|
||||
use `rea update` to update it. Running setup through `npx` again selects the
|
||||
installation that executes setup, potentially in npm's temporary package cache;
|
||||
retain that cache entry for as long as the registration uses it.
|
||||
use `rea update` to update it. Running setup through `npx` again selects the pinned
|
||||
package-runner command. Do not register paths inside npm's temporary package cache.
|
||||
|
||||
### npm workspace errors before REA starts
|
||||
|
||||
|
||||
@@ -156,31 +156,9 @@ Select one literal seed kind: node ID, route, string, API, IPC channel, module,
|
||||
or native export. Matching is exact or literal substring matching; regular
|
||||
expressions and executable predicates are not accepted. The trace includes all
|
||||
matching seeds and the complete graph reachable in the selected direction.
|
||||
`both` follows relationships in either direction, except that a container
|
||||
reached through `contains` from one of its children only leads further up its
|
||||
containment chain, so a shared asset or artifact root does not pull in its
|
||||
other children.
|
||||
|
||||
When a string seed has no application match and the supplied Evidence has a
|
||||
semantic graph, `semantic_discovery` lists matching retained semantic node
|
||||
labels and direct string call arguments using the same matching mode and case
|
||||
rules. These are candidate starting
|
||||
points, not application trace matches. Its graph coverage and total describe the
|
||||
retained graph; missing or partial source coverage cannot establish absence.
|
||||
Results follow retained node order and default to offset 0 and limit 64. Supply
|
||||
`semantic_page` and follow `next_offset` for further matches. Use a returned
|
||||
`trace_seed` with the same application Evidence and `trace_javascript_semantics`
|
||||
to examine relationships from that node. Other typed seeds do not search labels.
|
||||
|
||||
The result contains the matching basis, complete reachable subgraph, terminal
|
||||
paths, authority summaries, and native handoffs. A bidirectional trace of a
|
||||
large application can exceed one MCP response. Over MCP, pass
|
||||
`view: {"kind":"summary"}` or a `feature-trace` page of `paths`, `nodes`,
|
||||
`edges`, `seed_matches` or `native_handoffs` to retain the complete trace in
|
||||
the session and return only selected rows with the trace's identity and
|
||||
coverage. Page the retained trace with `inspect_analysis_view`; an oversized
|
||||
complete reply offers its first `paths` page as a ready-to-call continuation.
|
||||
See [MCP contracts](mcp-contracts.md) for filters. A handoff binds
|
||||
paths, authority summaries, and native handoffs. A handoff binds
|
||||
the exact native artifact digest and requested exports from the application
|
||||
graph. Trace direction selects the reachable addons; requested
|
||||
exports for each selected addon come from the complete supplied graph, so an
|
||||
@@ -205,10 +183,6 @@ rereading mutable files. A seed may identify a semantic or application node,
|
||||
literal, function fingerprint, property, endpoint, event, or boundary field.
|
||||
For a recovered parse, coercion, or validation boundary, its field value is the
|
||||
static operation's method name (for example, `Number`).
|
||||
When a recovered boundary call directly supplies a scalar binding, the graph
|
||||
connects the boundary to that binding; unresolved boundary calls remain
|
||||
candidates. A boundary nested inside another expression or compound assignment
|
||||
does not establish a direct result binding.
|
||||
The query declares backward provenance, forward influence, callers, or
|
||||
ownership and can restrict admitted relation kinds. REA returns every matching
|
||||
seed and the complete reachable subgraph; callers do not choose node, relation,
|
||||
@@ -231,25 +205,6 @@ resulting value. Deferred callback bodies do not become inputs to the enclosing
|
||||
assignment. Computed method keys belong to the enclosing callable; parameter
|
||||
defaults and method bodies belong to the method. Uncertain later property reads
|
||||
remain opt-in through `include_ambiguous_dynamic_edges`.
|
||||
|
||||
Binary, logical, unary, template, conditional, and sequence expressions retain
|
||||
`influences` edges from supported immediate inputs to the result, then to a
|
||||
direct declaration, assignment, call argument, or return. These are static
|
||||
dependencies, not aliases or proof that the expression executed; conditional
|
||||
operands and logical-assignment writes carry `conditional: true`. Unsupported
|
||||
operand dependencies, unmodeled receiving containers, dynamic call results,
|
||||
and sequence side effects remain unknown. An applicable
|
||||
unknown makes the trace status `partial` even when a seed was found. Literal
|
||||
operator operands, direct call arguments and directly returned scalar literals
|
||||
can also be seeds. Older graphs that only recorded a call-site literal remain
|
||||
searchable. The offered per-file node capacity also bounds the producer's
|
||||
expression and operand collections before graph projection.
|
||||
Expression nodes and unknowns share the existing per-file graph admission
|
||||
budgets; omitted facts make graph coverage truncated and the trace partial.
|
||||
Consumers that previously treated `found` as compatible with a relevant
|
||||
unknown should handle `partial` and inspect the returned unknowns. Retained
|
||||
graphs without `influences` edges remain readable but do not gain edges until
|
||||
the application is analyzed again.
|
||||
Global timer, `fetch`, `WebSocket`, `JSON.parse`, and coercion calls receive
|
||||
these built-in effects only when the file has no visible possible write to that
|
||||
global, including a computed write through a global object. The check does not
|
||||
@@ -314,14 +269,7 @@ mapped candidate, but structural reachability, semantic influence, runtime
|
||||
observation, and causal proof remain separate claims.
|
||||
|
||||
Semantic literal nodes keep their complete value in `properties.value`; literal
|
||||
queries match that field. Call-site nodes also retain direct scalar arguments in
|
||||
`properties.arguments`, with source argument indexes; literal queries can start
|
||||
at those call sites. These are syntactic facts, not runtime values, and an index
|
||||
after a spread is not necessarily the runtime argument position. Nonliteral
|
||||
arguments and values not representable in JSON are omitted. The primitive byte
|
||||
bound and the worker transfer record ceiling can also omit direct values; the
|
||||
call-site unknown reports the applicable limit. String labels use
|
||||
`string literal` when that is
|
||||
queries match that field. String labels use `string literal` when that is
|
||||
shorter than the complete JSON value. A literal's `identity.role_key` commits
|
||||
its canonical JSON value as `value-sha256:<digest>` when the digest key is
|
||||
shorter; smaller values keep their existing JSON key. This keeps large values
|
||||
@@ -629,10 +577,8 @@ or process workflows.
|
||||
## CLI and verification
|
||||
|
||||
All six CLI commands accept inline JSON or a path to a JSON file. The CLI
|
||||
returns an Evidence record directly. Later workflow inputs can contain full
|
||||
records, or exact retained references when `--evidence-bundle PATH` selects an
|
||||
authenticated exported bundle. A separate CLI process has no retained MCP
|
||||
connection state. See [CLI Evidence reuse](cli.md#import-export-and-compare-evidence).
|
||||
returns an Evidence record directly. Put the full records in a later CLI input;
|
||||
a separate CLI process has no retained MCP connection state.
|
||||
|
||||
File inputs must be regular files; symlinks to regular files are accepted.
|
||||
Directories, named pipes and device files produce an input error before JSON
|
||||
@@ -655,12 +601,8 @@ rea trace-application-feature ./trace-input.json --json
|
||||
MCP analysis returns the complete Evidence record, with the operation result
|
||||
in `normalized_result` and its identity in `evidence_id`. If the connected server advertises retained references, reuse its
|
||||
exact returned ID as `{"kind":"retained-evidence","evidence_id":"RETURNED_ID"}`
|
||||
in `application`, or `left`/`right` for comparisons. For summary or inspection
|
||||
replies, use `normalized_result.parent_evidence_id` to select the complete source.
|
||||
`RETURNED_ID` is a template, not a literal valid ID. Native-observation arrays
|
||||
and `reconcile_javascript_runtime`'s nested static layers and runtime-observation
|
||||
array also accept references where advertised, independently mixed with inline
|
||||
records. Exact identity, coverage and partial or unknown facts are preserved.
|
||||
in `application`, or `left`/`right` for comparisons. `RETURNED_ID` is a template,
|
||||
not a literal valid ID. Native Evidence arrays still use complete records.
|
||||
References are scoped to one connection; `close_binary` clears them. Export a
|
||||
bundle before closing and import it on another connection, or supply the full
|
||||
inline Evidence there. Versions before 4.1.0 accept full inline Evidence only; installing
|
||||
|
||||
@@ -72,18 +72,6 @@ code-point order. Dependency manifests remain package observations but do not
|
||||
declare Electron main or renderer entries. If only dependency manifests are
|
||||
present, the inventory artifact remains the graph root.
|
||||
|
||||
Package observations use `renderer_declarations` to record the actual present
|
||||
fields: strings, including empty strings, remain exact; non-string values use `{ "status": "non-string" }`
|
||||
without copying nested browser mappings. An empty record means no such fields
|
||||
were present; `null` means package metadata was invalid or unavailable. Renderer
|
||||
role mechanisms and package load edges name the field that supplied the candidate,
|
||||
selected as the first nonempty string from `renderer`, `browser`, then `module`.
|
||||
The package load edge retains `declared_path`, `source_field`, and resolution
|
||||
facts for that selection. New analyses no longer emit the derived package
|
||||
`renderer` alias; consumers must read declarations for source metadata and load
|
||||
edges for the selected candidate. Previously retained Evidence keeps its original
|
||||
properties and remains readable without migration.
|
||||
|
||||
## Select sources before analysis
|
||||
|
||||
For a question about known entry sources, supply exact artifact-relative file
|
||||
@@ -251,12 +239,10 @@ An unpacked entry whose companion bytes were not supplied remains
|
||||
`unavailable`. REA continues analyzing embedded JavaScript and records the
|
||||
missing native/resource bytes as unknown. The occurrence keeps
|
||||
`hash_status: unavailable`; no content-addressed child artifact is created for
|
||||
those missing bytes. New ASAR file occurrences record `unpacked: true` for
|
||||
companion files and `unpacked: false` for packed entries, including when the
|
||||
file bytes are unavailable. An omitted flag in an older inventory remains
|
||||
unknown; it does not establish either storage mode. See
|
||||
[MCP integrity handling](mcp-contracts.md#integrity-record-and-continue) for the
|
||||
tool-result contract.
|
||||
those missing bytes. New inventories preserve an ASAR companion declaration as
|
||||
`unpacked: true`; an omitted flag in an older inventory does not establish that
|
||||
unavailable bytes belonged to a companion. See [MCP integrity handling](mcp-contracts.md#integrity-record-and-continue)
|
||||
for the tool-result contract.
|
||||
|
||||
JavaScript whose syntax cannot be parsed retains unavailable semantic coverage
|
||||
with `incomplete-module` unknowns for each semantic family and unknown omitted
|
||||
@@ -364,13 +350,6 @@ Selected bounded text is then parsed as inert data to recover:
|
||||
relative to the calling module; string worker and service-worker paths use
|
||||
the document base URL or Node's working directory, which static analysis
|
||||
does not establish, so they keep their expression and stay unresolved);
|
||||
- Vite preload dependency tables (`__vite__mapDeps`): `./` entries resolve
|
||||
relative to the importing chunk; base-relative entries such as
|
||||
`assets/chunk.js` carry a `vite-output:` specifier and resolve under the one
|
||||
ancestor directory of the importer that holds them, the build output
|
||||
directory. A file found there but not analyzed, such as a stylesheet, stays
|
||||
`unavailable` with its path. Selecting only the assets directory leaves them
|
||||
`not-found`;
|
||||
- route, network endpoint, storage, and vendor-marker observations;
|
||||
- local source-map declarations and original
|
||||
source names and content digests;
|
||||
@@ -560,8 +539,6 @@ alternative object. A traced invocation whose key, API object or world stays
|
||||
unresolved or guarded keeps the call's own unresolved observation.
|
||||
Unknown spreads, reassigned bindings, accessors,
|
||||
dynamic channels and unsupported syntax do not become exact implementations.
|
||||
Literal switch cases retain their selection conditions. Possible fallthrough
|
||||
remains partial; dynamic case expressions remain unknown.
|
||||
This pass neither executes the bundle nor proves that runtime guards pass.
|
||||
An ambiguous or unresolved handler retains its channel registration but does
|
||||
not acquire a path to one arbitrarily selected callback. Unknown factory
|
||||
@@ -644,13 +621,7 @@ identity. This static check does
|
||||
not evaluate arbitrary call side effects or runtime registration. Missing findings
|
||||
do not establish that an application has no Electron boundary usage.
|
||||
|
||||
IPC pairing is an inference, not an observation. A main handler registered on an
|
||||
unresolved channel may receive any literal channel, so a compatible renderer send
|
||||
or `invoke` beside it is ambiguous, not unpaired. A wrapper's registration is
|
||||
exempt only when every invocation is a traced direct call supplying a literal
|
||||
channel, each recorded as its own finding. A guarded invocation keeps its
|
||||
predicate on a call path instead, so its channel still competes without pairing.
|
||||
A renderer `invoke` pairs only
|
||||
IPC pairing is an inference, not an observation. A renderer `invoke` pairs only
|
||||
with one unique `ipcMain.handle`/`handleOnce` candidate on the same exact literal
|
||||
channel; a renderer send pairs only with one unique `ipcMain.on`/`once`/`addListener`
|
||||
candidate. Dynamic channels are never paired, and multiple compatible handlers
|
||||
@@ -876,22 +847,8 @@ and Evidence. If that expansion does not fit, the file node retains
|
||||
`static_analysis` and any admitted `semantic_module_analysis` observations,
|
||||
including their exact values and source locations. Use the public modules item
|
||||
view to read them. The graph marks those unexpanded relationships as unknown.
|
||||
|
||||
The parent charges everything it retains against the heap that was free when
|
||||
analysis started, less a transfer reserve. Semantic facts and expanded
|
||||
relationships stop at half of that capacity, and a source refused there keeps
|
||||
its static facts with the refusal reported. Static facts can continue to three
|
||||
quarters of it while the live heap also has room for another copy of what it
|
||||
holds, so results charged for earlier sources do not cost later sources their
|
||||
static facts when the heap can still hold them. When that share runs short,
|
||||
relationships that were reserved but not yet expanded are given up before any
|
||||
static facts, and each affected file reports the omission.
|
||||
|
||||
If static facts themselves exhaust that share, or the parent's live heap cannot
|
||||
hold the next checkpoint, analysis stops; inventoried but unexamined files remain
|
||||
present with explicit unknown scopes. The failure reports the limit that applied
|
||||
in bytes, what was still available, the live heap in use and what the checkpoint
|
||||
needed.
|
||||
If cumulative result capacity is exhausted, analysis stops; inventoried but
|
||||
unexamined files remain present with explicit unknown scopes.
|
||||
|
||||
### Large local JSON inputs
|
||||
|
||||
@@ -1016,12 +973,7 @@ reason in the renderer observations.
|
||||
HTML script source ranges follow the HTML parser across LF, CRLF, and bare CR
|
||||
line endings, preserving UTF-16 columns.
|
||||
Inline JavaScript in local HTML, meaning classic or module `<script>` bodies
|
||||
without a `src` attribute, is not analyzed. Embedded SVG scripts use `href`
|
||||
(or `xlink:href` when `href` is absent) instead: an existing reference, even
|
||||
an empty one, suppresses the inline body. SVG's `src` and HTML-only `language`
|
||||
attributes do not suppress its JavaScript body. SVG references and inline
|
||||
bodies share the same inert parsing and source coordinates as HTML scripts;
|
||||
template content stays inert. Each HTML file that has an inline body gets an
|
||||
without a `src` attribute, is not analyzed. Each HTML file that has one gets an
|
||||
unknown `parse-html-inline-scripts` scope with the exact body count and the
|
||||
script type and source range of the first 100 bodies; beyond that, coverage is
|
||||
truncated under the `html_inline_script_locations_per_file` limit, and the
|
||||
|
||||
@@ -65,9 +65,8 @@ confidence and basis, and does not prove a backend endpoint or reachability.
|
||||
Pages obey the shared 1 MiB projection budget; MCP transport applies its own
|
||||
wire-size guard. A byte-limited page returns a continuation; an indivisible
|
||||
oversized record or metadata returns a typed resource error. Select fewer
|
||||
fields or retrieve the complete retained parent through `get_evidence_bundle`
|
||||
(`detail: "records"` with its `evidence_id`), `export_evidence_bundle`, or the
|
||||
CLI's exported bundle. Full Evidence is never
|
||||
fields or retrieve the complete retained parent through `get_evidence_bundle`,
|
||||
`export_evidence_bundle`, or the CLI's exported bundle. Full Evidence is never
|
||||
silently truncated.
|
||||
|
||||
Collections are `summary`, `routes`, `occurrences`, `sources`, and `unknowns`.
|
||||
|
||||
@@ -93,19 +93,13 @@ coverage. It does not print captured source text.
|
||||
Runtime targets, frames, scripts, and workers become capture-scoped graph
|
||||
nodes. REA evaluates compatible static candidates conservatively:
|
||||
|
||||
V8 Inspector targets whose protocol type is `node` use JavaScript asset/module
|
||||
candidates for their verified entry location. A target-only correspondence does
|
||||
not mark that script loaded; load-state evidence requires a script observation.
|
||||
|
||||
1. A captured-source SHA-256 plus one matching mapped location yields
|
||||
a content-and-location match.
|
||||
2. When no static candidate exists at a mapped location, one unique exact
|
||||
content digest can match. Whole-asset digests and reconstructed module-factory source
|
||||
digests remain distinguishable in the reported basis.
|
||||
3. Without captured bytes, one unique mapped artifact path can produce a
|
||||
medium-confidence location match. A proven factory module contained within
|
||||
the located asset is not a competing whole-file location; separate assets
|
||||
and independently plausible modules remain candidates.
|
||||
medium-confidence location match.
|
||||
4. Multiple candidates remain `ambiguous`; no `observed_as` edge is emitted.
|
||||
With captured source and mapped candidates, only mapped content matches
|
||||
qualify; unrelated content matches elsewhere are excluded.
|
||||
|
||||
@@ -63,7 +63,7 @@ All four preserve target-file bytes. `open_jeb_project` selects or creates an en
|
||||
- The engine caps `list_units` pages at 100; a full page is reported as `coverage: "partial"` because more units may exist. An empty remainder is `complete` only when fewer units than requested were returned.
|
||||
- When `decompile_jeb_item` omits `unit_path`, the engine implicitly selects the project's first code unit; REA reports `unit_path: null` and a limitation instead of guessing the effective unit.
|
||||
- `item_address` is a JEB code-item reference, such as a value returned by the engine's `list_code_items`. Preserve that representation and select its code unit explicitly. For the audited CE SQLite unit, the exported method reference was `sqlite3_libversion_number`; using its raw ELF symbol address did not produce a successful decompilation. File offsets, ELF virtual addresses and engine item references have different semantics. REA currently exposes unit listing, while code-item discovery remains an upstream JEB operation.
|
||||
- Valid engine refusals (`success: false` envelopes) return `execution_failure` with the engine's message when available. Diagnostics preserve the original envelope, including any recovery advice, and the inputs of a refused primary request. If client metadata fails after a primary reply, that reply and the caller's inputs remain in partial Evidence. REA does not infer an unsupported target from a refusal, invoke a recovery tool, or retry the request.
|
||||
- Engine refusals (`success: false` envelopes) are preserved as unsupported-target errors carrying the engine's own message, for example a nonexistent item address.
|
||||
- Successfully parsed client information is reused for the lifetime of the MCP connection, including when the engine omits its version. Explicit client inspection and availability probes refresh it.
|
||||
- An unreachable endpoint is a provider-availability failure with recovery guidance; REA does not retry or spawn the engine.
|
||||
|
||||
|
||||
+7
-50
@@ -1,27 +1,9 @@
|
||||
# Kuna provider
|
||||
|
||||
[Kuna](https://github.com/Noelo-Lab/kuna) is an Apache-2.0 decompiler distributed as one `kuna` executable. For a new installation, use Kuna 1.990, REA's recommended verified release. REA uses a `kuna` you supply; it does not download, install, update or bundle Kuna. REA accepts other releases; `rea doctor` and analysis Evidence report them as unverified. Verification compares the reported version; it does not authenticate the executable. Analysis profiles retain the observed version and executable digest, independently of REA's current verified-release policy. The 1.990 real-provider checks ran on macOS arm64; Linux and other hosts have not been verified with that release.
|
||||
[Kuna](https://github.com/Noelo-Lab/kuna) is an Apache-2.0 decompiler distributed as one `kuna` executable. REA uses a `kuna` you supply. It does not download, install, update or bundle Kuna. REA verified this provider with Kuna 1.833; it accepts other releases and reports them as unverified in `rea doctor` and in each analysis profile.
|
||||
|
||||
Kuna is static: it reads the selected image and never executes it.
|
||||
|
||||
This profile change makes earlier Kuna analysis snapshots and resumable directory
|
||||
exports incompatible, even with the same Kuna executable. Keep the old files:
|
||||
their recorded Evidence remains historical. To continue analysis, open the
|
||||
binary without the old snapshot and save a new one, or choose a new export
|
||||
directory. Existing snapshot Evidence can be extracted from its
|
||||
`evidence_bundle` JSON member and imported as a separate Evidence bundle; that
|
||||
does not restore cached workflow entries or resume the old directory export.
|
||||
For example, using `jq` and a new output path:
|
||||
|
||||
```bash
|
||||
jq -e '.evidence_bundle | select(type == "object")' old-snapshot.json > history.json
|
||||
rea evidence-import history.json --json
|
||||
```
|
||||
|
||||
The CLI command validates the bundle in a one-shot session. To use the records
|
||||
in an MCP session, call `import_evidence_bundle` with the extracted file path.
|
||||
This validates the embedded bundle, not the rest of the snapshot container.
|
||||
|
||||
## Configure
|
||||
|
||||
```bash
|
||||
@@ -48,10 +30,6 @@ completes. Cleanup retries keep both the observations and the exact file owner.
|
||||
| `REA_KUNA_TIMEOUT_MS` | 120 s | Deadline for one single-function decompile, and for the inventory run REA makes when the export ends without one. |
|
||||
| `REA_KUNA_JOBS` | 1 | `decompile-project --jobs`. 1 is Kuna's reference answer; more jobs finish sooner but lose callee-first prototypes, and change about 40–60% of function bodies on the measured binaries. |
|
||||
|
||||
Pseudocode read from a parallel project export includes a prototype-context
|
||||
limitation in its Evidence, including in function dossiers and composed
|
||||
workflows. Early single-function results have their own source limitation.
|
||||
|
||||
A single-function decompile, inventory, string, graph or xref run that exceeds its deadline fails as `provider_timeout`, naming the setting that owns the deadline and that raising it takes a restart. A project export that exceeds `REA_KUNA_PROJECT_TIMEOUT_MS` is an export failure instead: requests that need the export report `capability_unavailable` with the export's own retry guidance.
|
||||
|
||||
Windows is not supported yet. Supported targets are ELF and thin Mach-O executables for x86-64, AArch64 and ARM; PE and universal Mach-O are rejected with a typed reason.
|
||||
@@ -60,7 +38,7 @@ Windows is not supported yet. Supported targets are ELF and thin Mach-O executab
|
||||
|
||||
| Tool | Source |
|
||||
| ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `list_procedures`, `search_procedures`, `procedure_address` | Kuna's callable inventory (`kuna functions --json`), including import slots but not [Mach-O data](#mach-o-data-entries) |
|
||||
| `list_procedures`, `search_procedures`, `procedure_address` | Kuna's callable inventory (`kuna functions --json`), including import slots it does not decompile |
|
||||
| `address_name` | Function entries, then the data labels Kuna's export assigns |
|
||||
| `procedure_pseudo_code`, `batch_decompile` | The project export, or a single-function run while the export builds |
|
||||
| `procedure_assembly` | The project export's labeled listing |
|
||||
@@ -69,8 +47,6 @@ Windows is not supported yet. Supported targets are ELF and thin Mach-O executab
|
||||
| `xrefs` | Exact-address incoming reference sites, with forwarding aliases filtered explicitly |
|
||||
| `analyze_function` | Pseudocode and assembly from the export, plus call relationships; typed references, comments, strings and CFG remain unavailable |
|
||||
|
||||
Kuna's inventory gives some entries aliases, such as ELF versioned-symbol names and generated `sub_` names. Name selectors accept an alias that names one entry and list every entry a shared alias names. `search_procedures` matches aliases too: each entry appears once under its primary name, with the matching aliases in `matched_aliases`. `list_procedures` shows primary names only.
|
||||
|
||||
Typed procedure references, basic blocks, raw-memory tools, annotations and type recovery are not available from this provider. Use another provider for those operations.
|
||||
|
||||
## How answers are produced
|
||||
@@ -86,7 +62,7 @@ The first text or assembly request in a session looks for this target's export i
|
||||
- Assembly and data names wait for the export; progress notifications report the wait every ten seconds. A session does not rebuild a failed export: requests that need it fail with `retryable: false`, the export's failure, its captured Kuna output and what to change before reopening the target. A setting such as `REA_KUNA_PROJECT_TIMEOUT_MS` or `REA_KUNA_JOBS` is read when REA starts, so changing it also needs a restart. Pseudocode keeps using single-function runs.
|
||||
- A single-function answer says so in its limitations. It lacks the export's whole-program prototypes, so a later request can return different text for the same function.
|
||||
|
||||
Pseudocode, assembly, `analyze_function` and data-name answers record where their text came from in `raw_result.source`: `single-function`, or `project-export` with the cache directory. `procedure_pseudo_code` and `procedure_assembly` answers from the export also name the file, byte span and the file's SHA-256. Combined `analyze_function` answers retain both facet records in `raw_result.source.files`, each with its file, digest and byte span. Both texts come from one export; losing that project retries the complete report. Listings and procedure lookups carry no `raw_result`; omitted [Mach-O data entries](#mach-o-data-entries) are reported in their limitations. Agents can open or search the export's `.c` and `.asm` there directly when that is cheaper than calling a tool per function. Text read that way is target-derived, like any decompiler output, but it bypasses the untrusted-content marker that REA's tool results carry.
|
||||
Pseudocode, assembly, `analyze_function` and data-name answers record where their text came from in `raw_result.source`: `single-function`, or `project-export` with the cache directory. `procedure_pseudo_code` and `procedure_assembly` answers from the export also name the file, byte span and the file's SHA-256. Combined `analyze_function` answers retain both facet records in `raw_result.source.files`, each with its file, digest and byte span. Both texts come from one export; losing that project retries the complete report. Listings and procedure lookups carry no `raw_result`. Agents can open or search the export's `.c` and `.asm` there directly when that is cheaper than calling a tool per function. Text read that way is target-derived, like any decompiler output, but it bypasses the untrusted-content marker that REA's tool results carry.
|
||||
|
||||
The background export belongs to the session. `close_binary`, or the end of a CLI command, cancels it unless another session still waits for it. A one-shot CLI decompile therefore answers from a single-function run; any request that needs the export, such as `procedure_assembly` or `analyze_function`, keeps it.
|
||||
|
||||
@@ -96,17 +72,6 @@ Regex procedure and string search uses ECMAScript Unicode matching in a cancella
|
||||
|
||||
Single-function subprocess output and each cached source facet share a 16 MiB buffered-text capacity. Cached spans are checked before allocation; larger text remains in the local export and the error identifies the file and range. `analyze_function` with `detail: "complete"` (the CLI default) returns the complete dossier inline, which can still be large; later retained views do not remove that first transfer. The MCP default retains the dossier and returns its summary, so page the facets you need through `inspect_analysis_view`, or prefer the individual text tools for a focused question.
|
||||
|
||||
## Mach-O data entries
|
||||
|
||||
Kuna's inventory also lists Mach-O exports that are data, such as globals in `__DATA,__data`, constants in `__TEXT,__const` and `__mh_execute_header`. REA classifies each entry with the target's own section table, the same load-command facts `inspect_macho` reports, read from the leading bytes kept while the session's private copy is hashed:
|
||||
|
||||
- A section with `S_ATTR_PURE_INSTRUCTIONS` or `S_ATTR_SOME_INSTRUCTIONS`, or of type `S_SYMBOL_STUBS`, holds code; the entry stays.
|
||||
- A lazy or non-lazy symbol-pointer section (`__got`, `__la_symbol_ptr`) holds import slots; the entry stays.
|
||||
- Any other section, or the header and load commands before the first section, holds data. Listings and searches omit the entry and say how many they omitted in each region. Pseudocode, assembly, `analyze_function` and `procedure_address` refuse it by name or address, and call queries refuse its name, naming its region; call queries by exact address stay validated by Kuna's graph. `address_name` and `xrefs` still accept it.
|
||||
- An address outside every section and the header stays, with a limitation saying REA cannot classify it. If the section table cannot be read, every entry stays and listings say why. Answers about one such entry, including pseudocode, assembly, `analyze_function`, `procedure_address`, call queries by name and `address_name`, carry the same limitation; code and import slots carry none.
|
||||
|
||||
A section's attributes are the format's declaration, not proof of what runs there, and a kept entry is not proven to be a function. ELF inventories are kept as Kuna reports them. On a Mach-O target, the first listing or lookup establishes the private copy that Kuna commands also use, once per session, even when a warm export answers it.
|
||||
|
||||
## Strings
|
||||
|
||||
`list_strings` and `search_strings` share one lazy `kuna strings --json --encoding all --min-length 5 --termination any --no-xrefs` scan of the digest-checked private input. Skipping references avoids program lifting, a call graph and a C export. REA validates the input, scan policy, every row and the total before caching any facts. Encoding, character count, byte length and terminator facts must agree; supplementary UTF-8 characters count as Unicode scalars. Malformed output is an error, not an empty search. REA parses the JSON as Kuna writes it instead of buffering it, so the scan has no fixed size: Kuna writes about 290 bytes per row, 31 MB for a 20 MB Ghostty binary and 179 MB (611k rows) for an arm64 Firefox `XUL`. Rows are validated one at a time, and the first contradiction stops Kuna. Text REA does not keep is dropped unassembled; kept text and retained rows are admitted against the REA process's live heap headroom as they arrive; a refusal is a `memory` resource constraint with `retryable: false`, and closing other targets or starting REA with a larger Node.js heap makes room. String scans use the whole-program `REA_KUNA_PROJECT_TIMEOUT_MS` deadline; `REA_KUNA_TIMEOUT_MS` applies to single-function decompilation. The parsed inventory belongs to one selected target/executable profile and expires on session close; Kuna diagnostics beyond their 8 MiB buffer are retained as a failure until close, while other failures are not cached.
|
||||
@@ -160,19 +125,11 @@ REA writes Kuna's exports only inside `REA_KUNA_CACHE_DIR`:
|
||||
- `trash/` holds retired builds and evicted exports awaiting removal.
|
||||
- `home/` is Kuna's HOME.
|
||||
|
||||
The cache key includes the selected launcher and resolved executable paths, plus
|
||||
the target's native basename: that name determines the export filenames and
|
||||
appears in generated source. REA accepts one current manifest layout. Older
|
||||
exports are rebuilt on demand and retired through normal owned-cache cleanup;
|
||||
no manual deletion or migration is needed. Manifests must name the producer's
|
||||
exact flat file layout, with matching basenames and producing inputs. Replaced
|
||||
entry, project or manifest symlinks are refused before their targets are read;
|
||||
an unverified `projects/` parent grants no removal authority. Warm reuse checks
|
||||
the manifest's file identity as well as the export files. These refusal and
|
||||
migration cases are verified with controlled Kuna-format exports; they do not
|
||||
establish behavior of an independently installed Kuna release.
|
||||
The cache key includes the selected launcher and resolved executable paths.
|
||||
Older exports without that identity are rebuilt automatically and retired through
|
||||
normal cache cleanup; no manual migration is needed.
|
||||
|
||||
Builds publish by one rename, so a reader sees a complete export or none. If later staging retirement fails, the typed cleanup error retains the completed export's key, directory and manifest in `published_export`. The session adopts that verified project; later queries reuse its immutable files while `project_export.cleanup_incomplete` reports the retained owner's live state. `close_binary` retries only the remaining cleanup, and a successful retry resolves prior cleanup errors. A failed export followed by failed staging cleanup retains both failures; it does not invent a publication receipt. A session keeps the export it uses from eviction only within its own REA process. If another REA process sharing the cache evicts it, the session rebuilds it; meanwhile pseudocode falls back to single-function runs. A damaged or edited export is removed and rebuilt. REA retains the file state verified with each digest, checks it before warm-cache reuse, and checks open descriptors before and after reading text. Assembly indexes are checked against the same state; a changed file makes the whole report retry rather than mixing facets from different exports. Staging moves to `trash/` for removal only after REA proves its Kuna process stopped. Staging of a process it cannot prove stopped is never swept; REA keeps it owned, retries the stop on `close_binary`, and reports it until the stop is proven; that MCP error names the same-connection retry as `recovery.kind=retry_cleanup` ([retained cleanup retry](mcp-contracts.md#retained-cleanup-retry)). Failed file removal also stays bound to its export and is retried by `close_binary`; closing does not report success while that cleanup remains. `rea doctor --provider kuna` reports the cache directory without creating it.
|
||||
Builds publish by one rename, so a reader sees a complete export or none. If later staging retirement fails, the typed cleanup error retains the completed export's key, directory and manifest in `published_export`. The session adopts that verified project; later queries reuse its immutable files while `project_export.cleanup_incomplete` reports the retained owner's live state. `close_binary` retries only the remaining cleanup, and a successful retry resolves prior cleanup errors. A failed export followed by failed staging cleanup retains both failures; it does not invent a publication receipt. A session keeps the export it uses from eviction only within its own REA process. If another REA process sharing the cache evicts it, the session rebuilds it; meanwhile pseudocode falls back to single-function runs. A damaged or edited export is removed and rebuilt. REA retains the file state verified with each digest, checks it before warm-cache reuse, and checks open descriptors before and after reading text. Assembly indexes are checked against the same state; a changed file makes the whole report retry rather than mixing facets from different exports. Staging moves to `trash/` for removal only after REA proves its Kuna process stopped. Staging of a process it cannot prove stopped is never swept; REA keeps it owned, retries the stop on `close_binary`, and reports it until the stop is proven. Failed file removal also stays bound to its export and is retried by `close_binary`; closing does not report success while that cleanup remains. `rea doctor --provider kuna` reports the cache directory without creating it.
|
||||
|
||||
## Verify with a real Kuna
|
||||
|
||||
|
||||
+11
-108
@@ -40,38 +40,6 @@ inspection is needed; see [selected member inspection](#selected-member-inspecti
|
||||
Compare separate builds with [member comparison](#member-comparison), preserving
|
||||
their paths, digests, and selection scope.
|
||||
|
||||
### Field constant defaults
|
||||
|
||||
`inspect_managed_members` reads field-owned values from the ECMA-335 `Constant`
|
||||
table without loading the assembly. Each selected field's `constant` contains
|
||||
the declaration's row offset, element-type code, blob index and exact `raw_hex`,
|
||||
plus its decoded `element_type` and `value`. For example, an `i4` declaration
|
||||
with bytes `865d0000` returns `23942`.
|
||||
|
||||
Boolean, character, signed/unsigned integers, floating-point, UTF-16 strings
|
||||
and class-null declarations are supported. `i8` and `u8` values are decimal
|
||||
strings to avoid JSON integer precision loss. Floating-point negative zero,
|
||||
NaN and infinities are the strings `-0`, `NaN`, `Infinity` or `-Infinity`;
|
||||
`raw_hex` preserves their original bits. UTF-16 code units, including embedded
|
||||
NULs and unpaired surrogates, are preserved.
|
||||
|
||||
`constant: null` means neither a Constant declaration nor `HasDefault` was
|
||||
observed. Historical Evidence can omit the field entirely; omission does not
|
||||
prove that the artifact has no default. Missing or duplicate declarations,
|
||||
inconsistent flags, malformed values and unsupported element types return
|
||||
`state: "unavailable"`, a limitation and partial coverage while retaining other
|
||||
member facts. This decoding shares the existing metadata representation budget
|
||||
and only decodes blobs for the selected fields.
|
||||
|
||||
Retained `managed-members` field pages can select `fields: ["constant"]` while
|
||||
keeping original field identity and source offsets. These values are metadata
|
||||
declarations, including enum members' encoded underlying constants, not observed
|
||||
runtime state or inferred enum-name mappings. Parameter/property defaults are
|
||||
not exposed by this field-focused reader. Member comparison still covers field
|
||||
names, flags and signatures; it does not compare constant defaults.
|
||||
|
||||
The encoding follows [ECMA-335, Partition II, sections 22.9 and 23.1.16](https://ecma-international.org/publications-and-standards/standards/ecma-335/).
|
||||
|
||||
### Metadata overview and retained rows
|
||||
|
||||
The existing CLI and omitted/complete MCP detail return complete metadata
|
||||
@@ -152,50 +120,6 @@ literal pool's own coverage. Older Evidence that never examined exported types
|
||||
or user strings reports a null count and refuses those pages rather than
|
||||
returning an empty one.
|
||||
|
||||
To find specific retained members without returning their IL anchors, select
|
||||
exact names and defining types, then project columns:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "<parent_evidence_id>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "managed-members",
|
||||
"collection": "methods",
|
||||
"names": ["ParseNumber", "ParseReadNumber"],
|
||||
"declaring_types": ["Newtonsoft.Json.JsonTextReader"],
|
||||
"fields": ["name", "declaring_type", "signature"],
|
||||
"offset": 0,
|
||||
"limit": 64
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Filters are exact and case-sensitive; lists are alternatives and filters
|
||||
intersect. An empty list matches nothing. Names select `name` in types,
|
||||
exported types, fields, methods and member references; in call edges they select
|
||||
`target_name`, and in field accesses `field_name`. Declaring-type filters select
|
||||
`declaring_type` in fields/methods, `parent_name` in member references,
|
||||
`target_declaring_type` in call edges and `field_declaring_type` in field accesses.
|
||||
Types, exported types and user strings do not accept a declaring-type filter;
|
||||
user strings do not accept a name filter. No display strings are reparsed.
|
||||
|
||||
`fields` accepts only columns of that collection. Omitting it preserves full
|
||||
rows, including method bodies. Selecting `body` explicitly returns the full
|
||||
recorded body and anchors; selecting no columns (`[]`) returns identity fields.
|
||||
Projected or filtered rows add their original `source_index`; tokens and source
|
||||
byte offsets remain, with defining/parent tokens or IL-site endpoints where
|
||||
applicable. The page retains module/MVID, artifact digest, parent Evidence,
|
||||
producer selection and source coverage. Full facts remain in the parent.
|
||||
|
||||
Paging offsets and totals count known matches in original order. `source_count`
|
||||
counts the producer-returned collection; `unresolved_filter_rows` counts rows
|
||||
that could match but lack a requested name/type. Those candidates remain unknown
|
||||
and are excluded from the matched total. Exhausting the known matches does not
|
||||
prove absence among unresolved or unexamined source rows.
|
||||
|
||||
#### CLI summary and saved member rows
|
||||
|
||||
The CLI accepts the same `complete` and `summary` detail choices. Omitted or
|
||||
@@ -243,17 +167,13 @@ authenticates the saved bundle; the full source is retained on disk.
|
||||
|
||||
The canonical parser admits PE/CLI bytes and their metadata/CIL without loading
|
||||
an assembly. It reports observed implementation markers and unavailable facts;
|
||||
it does not itself unpack .NET single-file hosts or infer a NativeAOT identity
|
||||
from an ordinary native PE. It does not decode IL2CPP; use
|
||||
it does not unpack .NET single-file hosts or infer a NativeAOT identity from an
|
||||
ordinary native PE. It does not decode IL2CPP; use
|
||||
[IL2CPP metadata](il2cpp-metadata.md) for the image, type and method names in a
|
||||
Unity `global-metadata.dat`. Inputs without admitted CLI metadata do not
|
||||
become managed assemblies through naming or routing guesses.
|
||||
Inspect separately obtained components explicitly and preserve their identities.
|
||||
|
||||
For ordinary .NET single-file hosts, use the existing artifact tools to
|
||||
[inventory and extract bundle members](dotnet-bundles.md), then inspect the
|
||||
extracted assembly explicitly with the managed tools.
|
||||
|
||||
A valid implementation marker establishes a candidate native boundary, not
|
||||
recovered native semantics or runtime behavior.
|
||||
|
||||
@@ -512,18 +432,8 @@ raw digest stands in. Missing or unreadable TypeDef and TypeRef operands are
|
||||
reported as coverage issues. Invalid NestedClass rows retain their failure
|
||||
reasons and withhold TypeDef signature identities; type names and nesting are
|
||||
parsed once by their shared owner. Each current inspection records
|
||||
`signature_resolution: "type-names-v2"`, even when none of its operands can be
|
||||
resolved. Version 2 also appends readable names to signature operands, for
|
||||
example `class:7772 (AnalysisJob)`. Coded indexes remain build-local references;
|
||||
names are display text and do not change raw or resolved signature digests.
|
||||
Nested names use `Outer/Inner`. Generic instantiations keep their decoded
|
||||
arguments, for example ``class:9 (System.Collections.Generic.List`1)<string>``;
|
||||
custom modifiers are named too. Valid but unresolved TypeSpecs, invalid scopes,
|
||||
unreadable type names, cycles and nesting beyond the cap keep their raw operand
|
||||
spelling. Invalid or missing metadata coordinates retain the existing malformed
|
||||
signature result and source issue; rendering does not bypass validation.
|
||||
Version 1 Evidence remains readable and compares using the same digest policy.
|
||||
Historical Evidence defaults this attestation to `null`. When either
|
||||
`signature_resolution: "type-names-v1"`, even when none of its operands can be
|
||||
resolved. Historical Evidence defaults this attestation to `null`. When either
|
||||
observation predates resolution, both sides compare raw digests, which pair only where type rows agree,
|
||||
and the comparison reports that limitation. The signature is exact whether or
|
||||
not REA decoded it, so undecoded signatures pair by that tuple too, but they
|
||||
@@ -750,15 +660,13 @@ name and raw signature digest available while marking that signature malformed
|
||||
and coverage partial.
|
||||
A fat method header's nonzero LocalVarSigTok must name an existing StandAloneSig
|
||||
row whose blob is a structurally valid LocalVarSig. REA validates its local
|
||||
types and exact blob coverage, retaining `local_var_signature` with the raw blob
|
||||
digest and whether named types, modifiers, or generic variables require metadata
|
||||
or declaring context. Historical Evidence defaults this field to null. An invalid
|
||||
types and exact blob coverage but does not project their identities. An invalid
|
||||
token or malformed signature leaves decoded CIL anchors intact but marks the
|
||||
body malformed and coverage partial. Unsupported local type encodings retain
|
||||
the readable body with partial coverage. Zero continues to mean no locals
|
||||
signature. Raw table tokens are not cross-build identities. Comparison uses
|
||||
validated context-free signature digests; signatures requiring metadata or
|
||||
declaring context remain unknown across artifacts.
|
||||
signature. Comparison detects whether a locals declaration is present, but raw
|
||||
table tokens are not cross-build identities and local type contents are not
|
||||
compared.
|
||||
|
||||
## Decoded-CIL fingerprint
|
||||
|
||||
@@ -960,14 +868,9 @@ MethodDef flags or implementation flags value differs, including accessibility
|
||||
or synchronization changes. Observed method-body `init_locals` and `max_stack`
|
||||
differences also contribute to `metadata`, even when CIL decoding is incomplete.
|
||||
For complete bodies, presence or absence of a locals signature also contributes;
|
||||
the locals table token is build-local and is not compared. Different validated
|
||||
context-free LocalVarSig digests report a metadata encoding difference without
|
||||
asserting a semantic type change. Equal digests establish unchanged bytes only
|
||||
when neither signature requires metadata or declaring context. Historical
|
||||
unrecorded signatures and context-dependent signatures keep local identity
|
||||
unknown, preserving any independently observed changes. The same method token
|
||||
in an artifact with the same authenticated SHA-256 identifies the same local
|
||||
signature bytes, including for retained historical Evidence.
|
||||
the locals table token is build-local and is not compared. Local type contents
|
||||
are not currently compared, so this facet does not establish equivalent locals
|
||||
declarations.
|
||||
An unavailable header value keeps body coverage unknown rather than establishing
|
||||
equality or a difference. These comparisons are separate from the CIL/signature
|
||||
matching tiers: identical instructions do not establish unchanged method metadata.
|
||||
|
||||
+55
-436
@@ -26,23 +26,6 @@ can be passed directly to `compare_functions`, and `inspect_artifact` Evidence
|
||||
to `compare_artifacts`. Use `get_evidence_bundle` when the task needs broader
|
||||
retained session history or an explicit bundle for transfer.
|
||||
|
||||
`compare_artifacts` does not need that Evidence sent back. Name each side once
|
||||
with a source that `inspect_analysis_view` also accepts:
|
||||
`{"kind": "retained-evidence", "evidence_id": "<id>"}` for a record retained in
|
||||
this session, or `{"kind": "evidence-file", "path": "/absolute/evidence.json"}`
|
||||
for a saved record or bundle. Add `evidence_id` to select one record from a
|
||||
bundle that holds more than one. Complete inline Evidence is still accepted as
|
||||
the third form. The comparison cites the records that the sources resolve to,
|
||||
so all three forms give the same comparison Evidence. A retained reference that
|
||||
the session cannot resolve is refused with `evidence_integrity_mismatch` and
|
||||
`details.reason: "missing"`.
|
||||
|
||||
`compare_artifacts` can also resolve an artifact summary to its exact retained
|
||||
parent in the session, or from the authenticated bundle containing a file-selected
|
||||
summary. File-local records take precedence over the session lookup. Unrelated
|
||||
records are not retained by the file resolver. Supported historical summaries keep their original
|
||||
Evidence identity and omissions; a missing or mismatched parent is rejected.
|
||||
|
||||
`analyze_function` does not deliver its complete record by default. Without
|
||||
`detail`, it retains the complete dossier in the session and returns
|
||||
`inspect_analysis_view` summary Evidence whose
|
||||
@@ -53,9 +36,8 @@ reject the summary record. Give `compare_functions` the retained dossier as
|
||||
`{"kind": "retained-evidence", "evidence_id": "<parent_evidence_id>"}`; give
|
||||
`match_native_functions` the same reference inside `left_observations` or
|
||||
`right_observations`, since its `left` and `right` take `list_procedures`
|
||||
Evidence. `build_call_path.functions` also accepts retained references to the
|
||||
complete dossiers. Request `detail: "complete"` only when the record must travel
|
||||
inline.
|
||||
Evidence. Request `detail: "complete"` for `build_call_path`, which accepts
|
||||
only inline complete dossiers.
|
||||
|
||||
Saved records from the pre-6.6 identity projection that omitted
|
||||
`relationship_coverage: null` from its digest remain incompatible, even when
|
||||
@@ -70,16 +52,13 @@ the same Evidence authentication failures: an admitted source that fails
|
||||
authentication is `evidence_integrity_mismatch`, while an authenticated source with an unsupported
|
||||
workflow role or invalid selector is `invalid_request`.
|
||||
|
||||
Ghidra inventories, measured load-image observations and function replies use
|
||||
`raw_result: null` when their useful facts are already preserved in
|
||||
`normalized_result`, limitations, locations and relationship coverage. Distinct
|
||||
overview details, file-offset selection roles and database types supplemented
|
||||
with derived metadata keep their upstream representation. When a Ghidra naming
|
||||
edit is not applied, `raw_result` keeps the bridge's `failures` and shared
|
||||
`conflicts` table without a second copy of the result booleans. Native load-image
|
||||
attestation continues to use the bridge reply locally before Evidence
|
||||
projection. Read function results from `normalized_result`; older retained
|
||||
Evidence keeps its original representation and identity.
|
||||
Ghidra inventories and function replies use `raw_result: null` when their facts
|
||||
are already preserved in `normalized_result`, limitations, locations and
|
||||
relationship coverage. Distinct overview details, measured load-image
|
||||
observations, file-offset selection roles and database types supplemented with
|
||||
derived metadata keep their upstream representation. Read function results from
|
||||
`normalized_result`; older retained Evidence keeps its original representation
|
||||
and identity.
|
||||
|
||||
Artifact source-map filename relationships require a unique, identified regular
|
||||
file at the sibling path. A known directory cannot be the mapped file; the map
|
||||
@@ -161,9 +140,7 @@ for the error code, target details, partial observations, remediation, and the
|
||||
optional bounded `recovery` next step. `recovery.kind=inspect_input_contract`
|
||||
names the rejected operation: re-read that operation's advertised input schema
|
||||
(call `tools/list` if the catalog may be stale), correct the listed fields, and
|
||||
do not repeat the unchanged call. A well-formed request the selected target
|
||||
cannot answer, such as an address without one exact file offset, omits this
|
||||
hint; follow its `remediation.action`. `recovery.kind=inspect_timeout_state` names
|
||||
do not repeat the unchanged call. `recovery.kind=inspect_timeout_state` names
|
||||
the operation and the REA server/provider deadline. Inspect partial Evidence
|
||||
and reported cleanup state, then follow the returned remediation. Change a
|
||||
request field or REA_* setting only when the returned remediation explicitly
|
||||
@@ -173,9 +150,6 @@ recommendation to change it. When it names no setting, reduce supported work;
|
||||
do not invent one. Raising only the MCP
|
||||
client's deadline cannot extend this budget. These hints never include the
|
||||
complete catalog or caller arguments.
|
||||
`recovery.kind=retry_cleanup` appears only on a `close_binary`
|
||||
`cleanup_incomplete` error whose retained resources the session will retry; see
|
||||
[retained cleanup](#retained-cleanup-retry).
|
||||
Failures omit `structuredContent`, since each advertised `outputSchema`
|
||||
describes successful results. Successful replies retain their schema-validated
|
||||
structured data and matching text projection. Oversized errors are retained
|
||||
@@ -185,13 +159,8 @@ if retention is unavailable or fails, the diagnostic reports that reason.
|
||||
SDK-generated input validation failures are normalized into the same canonical
|
||||
`invalid_request` text when the bounded diagnostic fits the response budget.
|
||||
The complete field/type diagnostic is preserved when normalization fits the
|
||||
response budget, including request-specific rejected fields and reasons. Zod
|
||||
input issues retain their field paths and available type and range constraints.
|
||||
SDK diagnostic parts may be distributed between field issues and the main
|
||||
message so each retained explanation appears once.
|
||||
If their conservative projection allowance is exhausted, the response states
|
||||
that field-level detail was omitted and retains the original SDK diagnostic.
|
||||
Oversized SDK diagnostics keep the existing retained-Evidence recovery path. An unknown
|
||||
response budget, including request-specific rejected fields and reasons. Oversized
|
||||
SDK diagnostics keep the existing retained-Evidence recovery path. An unknown
|
||||
tool remains an MCP JSON-RPC `-32602` error, but its message directs the caller
|
||||
to `tools/list` and an advertised operation instead of repeating the unknown
|
||||
name.
|
||||
@@ -210,19 +179,8 @@ unresolved relationship reads. Cancellation remains a tool error.
|
||||
|
||||
`trace_feature` validates each reference's containing-procedure resolution using
|
||||
the same contract as navigation context. Explicit absence remains a successful
|
||||
negative observation. If the provider reports a non-retryable capability refusal
|
||||
attributed to the owner lookup without incomplete cleanup, the reference remains visible
|
||||
with `containing_procedure.status: "unavailable"` and a residual unknown. That
|
||||
unknown does not imply references were truncated.
|
||||
Malformed provider replies return `unreadable_output` for
|
||||
`resolve_containing_procedure` before successful-result projection;
|
||||
cancellation and incomplete cleanup remain errors.
|
||||
|
||||
For a procedure match, `procedure_references` with `direction: "incoming"` can
|
||||
provide the selected provider's reference kinds when supported. Its scope covers
|
||||
the procedure body, so compare target addresses when selecting references to the
|
||||
exact entry. It is a separate observation and does not establish runtime use or
|
||||
complete reference discovery.
|
||||
negative observation; malformed provider replies return `unreadable_output`
|
||||
for `resolve_containing_procedure` before successful-result projection.
|
||||
|
||||
REA prepares complete MCP results within the pinned stdio client's 10 MiB
|
||||
receive-buffer budget, including both text and structured representations and
|
||||
@@ -298,12 +256,6 @@ limitation and reason IDs index that page's `items.messages` dictionary. Expand
|
||||
those references to recover exact text, repeated messages, and absent versus
|
||||
empty optional fields. Failed nodes retain their original error.
|
||||
|
||||
Retained `xrefs` Evidence also supports `native-observation` address pages and
|
||||
`native-context` source caveats. `xrefs` still returns complete Evidence by
|
||||
default and does not accept `detail`; when its MCP response exceeds the budget,
|
||||
follow the returned continuation to page the retained result. These addresses
|
||||
remain untyped; paging does not add reference kinds or establish completeness.
|
||||
|
||||
These views reference complete source context through `source_context.digest`
|
||||
and caveat counts. Read it using the same parent with
|
||||
`view: {"kind":"native-context","facet":"overview"}`. The other facets are
|
||||
@@ -330,10 +282,6 @@ offered for completed `analyze_function` results only after session recording is
|
||||
acknowledged; the complete result and its retention reference remain intact.
|
||||
Oversized native inventories likewise include a ready-to-call first
|
||||
`native-inventory` page. Reduce its limit or select fields when a row is large.
|
||||
An oversized `trace_application_feature` result includes a first `feature-trace`
|
||||
page of its retained terminal `paths`; page `nodes` or `edges` from there.
|
||||
An oversized `batch_decompile` result includes a first body-free
|
||||
`batch-decompile` page of its retained `items`; read bodies from there.
|
||||
|
||||
A broad follow-up or `get_evidence_bundle` can also exceed the response budget; exporting
|
||||
preserves the complete session without sending it through a single MCP frame.
|
||||
@@ -347,35 +295,7 @@ a provider. `retention: "complete-record"` describes stored Evidence, while
|
||||
`native_dossier.value_flow` independently reports truncation, unavailable or
|
||||
unknown metadata. Missing omission counts remain null. Summary output carries
|
||||
every matching record; actual transport admission still applies. Omitted
|
||||
`detail` preserves complete canonical bundle delivery; filters require summary
|
||||
or records.
|
||||
|
||||
To read a discovered record without transferring the whole bundle, call
|
||||
`get_evidence_bundle` with `detail: "records"` and the same exact `filters`.
|
||||
The result (`kind: "evidence-record-selection"`) pages the matching complete
|
||||
records in canonical bundle (Evidence ID) order with `offset` (default 0) and
|
||||
`limit` (default 8), and reports `matching_records`, `total_retained_records`
|
||||
and `next_offset`. Each record is the retained Evidence itself, with an
|
||||
unchanged ID, subject digest, provider, profile and readback, so it works for
|
||||
any operation, including records that `inspect_analysis_view` does not support
|
||||
such as annotations. `missing_evidence_links` lists direct `evidence_links` of
|
||||
the returned records that this session or bundle does not retain; links are not
|
||||
traversed. A selection is not an Evidence bundle: it omits manifests and
|
||||
residual unknowns, cannot be imported, and describes historical observations
|
||||
rather than current provider or database state. `offset` and `limit` require
|
||||
`detail: "records"`. If one page exceeds the response budget, select fewer
|
||||
records or one `evidence_id`; a single record larger than one response still
|
||||
needs `export_evidence_bundle`, or `inspect_analysis_view` pages where its parent
|
||||
supports views.
|
||||
|
||||
Residual-unknown mutations reject missing Evidence or unknown references and
|
||||
unproven verified resolutions with nonretryable `invalid_request` diagnostics.
|
||||
`details.constraint` carries the validator's `kind`, owning `unknown_id`, and
|
||||
missing `evidence_id` or `related_unknown_id` where applicable. Correct the
|
||||
reported input or supply the required retained Evidence; unchanged retries do
|
||||
not repair these constraints. Rejected mutations leave retained state unchanged.
|
||||
Revision conflicts remain retryable `revision_conflict`; broken revision histories
|
||||
remain `evidence_integrity_mismatch`.
|
||||
`detail` preserves complete canonical bundle delivery; filters require summary.
|
||||
|
||||
`list_unknowns` returns every matching current head by default. For a smaller
|
||||
inline response, supply `offset` and `limit`, then follow `next_offset` until it
|
||||
@@ -383,12 +303,6 @@ is null. `total` counts all matching heads; each page preserves complete records
|
||||
in stable unknown ID order. Filters apply before paging. Pages reflect current
|
||||
session state, so keep the unknown ledger unchanged while traversing it.
|
||||
|
||||
`list_unknowns` only lists registered heads. Import restores the Unknown records
|
||||
present in a bundle; it does not turn residual question text embedded in Evidence
|
||||
into registry entries. An empty registry therefore does not establish that
|
||||
retained Evidence has no open questions. Inspect the relevant Evidence or its
|
||||
analysis view for its residual questions.
|
||||
|
||||
Automatically retained workflow questions include their immutable result
|
||||
Evidence ID. Distinct results retain independent questions and source links;
|
||||
repeating the same result does not create another unknown. Workflow Evidence
|
||||
@@ -402,32 +316,11 @@ When `record_unknown` or `update_unknown` commits but its complete reply exceeds
|
||||
the response budget, `details.reported_limits.committed_unknown` identifies the
|
||||
exact unknown ID, committed revision and mutation Evidence IDs. Its retained
|
||||
Evidence reference remains available on the same connection. Export the bundle
|
||||
to recover the result; do not repeat the committed mutation. A single unknown
|
||||
may be too large for a `list_unknowns` page. A delivery failure does not undo
|
||||
that revision. If a list response overflows, retry on the same connection with
|
||||
the same filters and offset and a smaller limit; export if one head still cannot
|
||||
fit.
|
||||
|
||||
To find missing sources before a downstream analysis refuses them, read
|
||||
`missing_direct_links` from the summary, or call `get_evidence_bundle` with
|
||||
`detail: "links"`. The result (`kind: "evidence-link-inspection"`) transfers no
|
||||
record payload. `filters` select the starting records, while membership is
|
||||
checked against every record in the session or file, so a filtered-out source is
|
||||
`retained`, not missing. Rows (`evidence_id`, `linked_evidence_id`, `status`) are
|
||||
in canonical order, default to `link_status: "missing"` and page with `offset` and
|
||||
`limit` (default 64); `links` counts every inspected link regardless of that
|
||||
selection. `traversal: "direct"` (the default) reads the selected records' own
|
||||
links and sets `retained_closure` to null, because it makes no closure claim.
|
||||
`traversal: "retained-closure"` also follows retained links, as composed workflows
|
||||
such as `build_call_path` do, and reports `complete` or `incomplete`; anything
|
||||
beyond a missing link remains unknown. Missing means absent from this session or
|
||||
file, not nonexistent, and neither status describes analysis coverage. Importing
|
||||
a missing source changes this membership without changing any Evidence ID.
|
||||
or page `list_unknowns` to recover the result; do not repeat the committed
|
||||
mutation. A delivery failure does not undo that revision.
|
||||
|
||||
For a portable exported bundle, the CLI shares this projection:
|
||||
`rea inspect-evidence-bundle '{"path":"/tmp/evidence.json","detail":"summary","filters":{"operation":"analyze_function"}}' --format json`.
|
||||
`"detail":"records"` with an `evidence_id` filter returns that complete record,
|
||||
and `"detail":"links"` reads link membership.
|
||||
The CLI validates and materializes that local bundle under its existing memory
|
||||
budget; summary delivery does not make arbitrary large files cheap to import.
|
||||
|
||||
@@ -445,38 +338,6 @@ that buffer. It does not raise REA's reserved single-string limit, which is
|
||||
raise that Node ceiling. It does not change analysis coverage or retained
|
||||
content. Ordinary responses keep their existing complete result contract.
|
||||
|
||||
### Input validation resource allowance
|
||||
|
||||
Both HTTP and stdio admit at most 1,024 combined array elements and object
|
||||
members in one tool's arguments by default. The pinned SDK counts them before
|
||||
canonical schema validation or lazy runtime materialization. Oversized calls
|
||||
return a bounded, nonretryable `resource_constraint` with resource `transport`
|
||||
and `reported_limits.max_input_elements`; the full element count is not claimed.
|
||||
No provider or tool handler starts for a refused call. Subsequent calls remain
|
||||
usable.
|
||||
|
||||
Set `REA_MCP_MAX_INPUT_ELEMENTS` to a positive decimal safe integer and restart
|
||||
REA to select another allowance for a larger complete observation or batch.
|
||||
Where a tool supports retained-Evidence or Evidence-file selectors, prefer
|
||||
those selectors to resending large inline records. Admission never truncates
|
||||
the caller's selected input or removes retained Evidence.
|
||||
This is a configurable transport admission policy, not a per-array `maxItems`
|
||||
contract or an exact heap guarantee. Missing required fields can amplify one
|
||||
input element into many issues: on the pinned Evidence array schema,
|
||||
1,024 empty objects produced 16,384 issues and about 26 MiB heapUsed delta
|
||||
before SDK formatting. The default therefore admits at most 1,023 such objects
|
||||
plus the root member, rather than extrapolating the scalar case to larger batches;
|
||||
actual memory also depends on schema branches, runtime state and concurrent
|
||||
work. HTTP's separate request-byte limit and the client's receive limit still
|
||||
apply.
|
||||
|
||||
Canonical schemas and CLI/application analysis semantics remain shared and
|
||||
unchanged. Admitted arguments retain the same values and ordinary validation
|
||||
diagnostics. SDK embedders can select `maxToolInputElements` in the server
|
||||
options; the common REA MCP server supplies the finite default to eager and
|
||||
deferred registrations alike. This does not impose the separate semantic
|
||||
batch-workload limit tracked in issue #5930.
|
||||
|
||||
### Error recovery and deadline ownership
|
||||
|
||||
Initialization instructions and the static JavaScript tool description teach
|
||||
@@ -512,12 +373,10 @@ result's `_meta`, whether the call succeeded or failed:
|
||||
```
|
||||
|
||||
The marker is declared once per tool contract (`returnsTargetContent`) and is
|
||||
on by default: `export_evidence_bundle`, `record_unknown`, `search_guidance`,
|
||||
and `read_guidance_section` omit it. The first two contain only REA session
|
||||
state or caller-supplied values; the guidance tools return REA's own installed
|
||||
documentation, not text from an analysis target. Tools whose results or errors
|
||||
can include provider, launcher, or device output stay marked, including
|
||||
`binary_session` (provider failure
|
||||
on by default: only `export_evidence_bundle` and `record_unknown`, whose
|
||||
results and errors contain only REA session state or caller-supplied values,
|
||||
omit it. Tools whose results or errors can include provider, launcher, or
|
||||
device output stay marked, including `binary_session` (provider failure
|
||||
reasons and availability diagnostics), `close_binary` (provider cleanup
|
||||
diagnostics with captured launcher output), `inspect_adb_client` and
|
||||
`inspect_apktool_client` (version output and stderr excerpts), and
|
||||
@@ -554,9 +413,6 @@ session retention refuses summary detail with `capability_unavailable`;
|
||||
For `inspect_artifact`, omission also keeps the original request parameters and
|
||||
complete output shape. Delivery selection never reaches the inventory engine;
|
||||
the complete inspection and its atomic inventory Evidence remain retained.
|
||||
A view is never more certain than its parent: a summary or other view of
|
||||
`inferred` Evidence is `inferred`. Other views are `derived`, unless the
|
||||
view kind itself infers, as described below.
|
||||
|
||||
`inspect_analysis_view` also accepts `api-surface` for retained HAR capture Evidence,
|
||||
grouping request lines by method, origin and format-rule path template. Templated
|
||||
@@ -588,11 +444,7 @@ names, the native API record, and high-pcode facets without starting a provider.
|
||||
and limit default to 0 and 64; pseudocode uses UTF-16 code units and never
|
||||
splits surrogate pairs, while other pages count rows. Unavailable facts and
|
||||
provider limitations remain explicit. Source is
|
||||
an exact same-session retained reference, portable inline Evidence, or an
|
||||
Evidence record or bundle file. Native function, inventory, and observation
|
||||
views require the `rea.analysis` predicate; function-matching views require
|
||||
`rea.native-function-matching`. A mismatched predicate returns an
|
||||
`invalid_request` identifying `source.predicate_type`. Views are
|
||||
an exact same-session retained reference or portable inline Evidence. Views are
|
||||
a summary, a layout mitigations or linkage facet, one section/symbol/module, or
|
||||
a stable page with a caller-selected positive `limit`. Module pages include
|
||||
JavaScript assets, bundled modules, and source modules. Select an exact
|
||||
@@ -631,69 +483,6 @@ the working directory). Save its complete `--json` Evidence, then use
|
||||
`inspect-analysis-view` with an `evidence-file` source and the same view. Neither
|
||||
adapter executes the target or repeats analysis for retained queries.
|
||||
|
||||
`trace_application_feature` follows the same delivery contract. Omitting `view`
|
||||
returns the complete trace. `view: {"kind":"summary"}` or a `feature-trace` page
|
||||
retains the complete trace in the session first and returns only the selected
|
||||
facts; query further pages with `inspect_analysis_view` and the returned
|
||||
`parent_evidence_id`. The summary carries the trace identity (`trace_id`,
|
||||
`source_evidence_id`, `source_graph_id`, seed and direction), the producer's
|
||||
`summary` counts, `trace_coverage`, the traced subgraph's `graph_coverage`,
|
||||
`evidence_links` and any no-match `semantic_discovery`. A `feature-trace` page
|
||||
selects one `collection` in retained order and repeats that context:
|
||||
|
||||
- `paths`: exact terminal paths; `where.node_ids` selects paths visiting any of
|
||||
those nodes and `where.end_kinds` selects their terminal node kinds.
|
||||
- `nodes`: compact rows with identity, `seed`, `observation_count` and distinct
|
||||
authority/state/confidence `basis`; `where.node_ids` and `where.kinds` filter
|
||||
exactly, and `include_observations: true` adds each node's complete retained
|
||||
observations.
|
||||
- `edges`: exact relationships; `where.node_ids` matches either endpoint and
|
||||
`where.relations` the relation.
|
||||
- `seed_matches` and `native_handoffs`: the producer's exact rows.
|
||||
|
||||
Omitted filters select all; empty arrays select none. `offset` and `limit`
|
||||
default to 0 and 64. Page `coverage` counts the selected collection only; it
|
||||
never upgrades `trace_coverage`, and graph connectivity does not prove runtime
|
||||
reachability. The CLI keeps complete trace JSON; save it and use
|
||||
`inspect-analysis-view` with an `evidence-file` source.
|
||||
|
||||
### Selected batch decompile results
|
||||
|
||||
`batch_decompile` returns every pseudocode body by default. `detail: "summary"`
|
||||
retains the complete batch in the session first and returns its producer counts,
|
||||
`source_provider` (the composing workflow), `source_analysis_profile_digest`,
|
||||
`decompiling_provider`, procedure identity counts, total and largest
|
||||
`pseudocode` size, and `limitation_count`. A server that retains no Evidence
|
||||
refuses the summary before analysis. Further views use the returned
|
||||
`parent_evidence_id` through `inspect_analysis_view`, or a saved Evidence file or
|
||||
portable bundle, and never repeat decompilation.
|
||||
|
||||
A `batch-decompile` view selects one `collection` and repeats the batch counts
|
||||
and provider identities:
|
||||
|
||||
- `items`: rows in retained order with their original `index`, so duplicate
|
||||
selectors stay distinct. A row keeps the selector in `address`, its `procedure`
|
||||
identity and `status`, plus `pseudocode_code_units` or its exact `error` and
|
||||
`limitations`. `where.indices`, `where.addresses`, `where.procedure_addresses`
|
||||
and `where.statuses` filter exactly and combine; an item with unknown identity
|
||||
never matches `procedure_addresses`. Omitted filters select all, and empty
|
||||
arrays select none. `offset` and `limit` default to 0 and 64.
|
||||
`include_pseudocode: true` adds each selected body.
|
||||
- `pseudocode`: one `index`'s text in UTF-16 code-unit pages. `offset` and `limit`
|
||||
default to 0 and 16,384, and `coverage` measures that body. An offset inside a
|
||||
surrogate pair is refused, and a limit ending inside one stops before it. An
|
||||
error item has no text and is refused with a pointer to `items`.
|
||||
|
||||
Page `coverage` counts the selection only: exhausting it establishes nothing
|
||||
about functions outside the batch. Evidence `limitations` aggregate every item's
|
||||
provider observations and are not attributed to one successful item; only error
|
||||
items keep their own. A page over the response budget offers a smaller selection
|
||||
from the same `source`: fewer rows, or one body's `pseudocode` pages when a single
|
||||
row does not fit. An inline source offers none, because the record cannot be
|
||||
repeated in that reply; resend it with a smaller selection.
|
||||
`export_evidence_bundle` remains the complete alternative. The CLI has no batch command; save the Evidence and use
|
||||
`inspect-analysis-view` with an `evidence-file` source or `--evidence-bundle`.
|
||||
|
||||
Completed `inspect_artifact` and atomic `inventory_artifact` Evidence also
|
||||
support summary and inventory views. A summary reports the manifest's expected
|
||||
counts separately from available record counts, observed entry kinds, inventory
|
||||
@@ -739,12 +528,10 @@ reach WABT or change the original source Evidence ID.
|
||||
Completed `inspect_macho` Evidence supports a summary and
|
||||
`view: {kind: "macho", facet: "exports", offset: 0, limit: 64}`. Facets are
|
||||
`entrypoints`, `architectures`, `build_metadata`, `load_commands`,
|
||||
`dependencies`, `imports`, `exports`, `segments`, `encryption`, and
|
||||
`provenance`; pages return the original rows. Evidence retained before
|
||||
`encryption` was reported pages no encryption rows and reports that collection
|
||||
as non-exhaustive with an unknown status. The summary keeps the selected
|
||||
slice, header fields and UUID, and replaces each collection with its returned
|
||||
count, reported total and `exhaustive` flag. Each page repeats the paged collection's `source_total` and
|
||||
`dependencies`, `imports`, `exports`, `segments`, and `provenance`; pages return
|
||||
the original rows. The summary keeps the selected slice, header fields and UUID,
|
||||
and replaces each collection with its returned count, reported total and
|
||||
`exhaustive` flag. Each page repeats the paged collection's `source_total` and
|
||||
`source_exhaustive`; exhausting a non-exhaustive collection does not establish
|
||||
that unreturned symbols are absent. Large frameworks return megabytes of
|
||||
symbols (Electron Framework: 2.5 MB of imports and exports, 3.3 KB as a
|
||||
@@ -762,24 +549,10 @@ counts, and whether the decoded graph is incomplete. Each page repeats
|
||||
budget (XLD.app: 6.7 MB), so select `detail: "summary"` on the first call.
|
||||
Delivery detail never reaches the decoder or changes the source Evidence ID.
|
||||
|
||||
Logical Evidence source fields share one input contract across domains:
|
||||
complete inline Evidence or an exact reference to a record already retained by
|
||||
the current connection. This applies to individual sources, optional sources,
|
||||
arrays and nested sources declared by the tool's input schema:
|
||||
|
||||
| Workflows | Retained-reference inputs |
|
||||
| ----------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
|
||||
| `compare_functions`, `compare_process_captures`, `compare_wasm_interfaces` | `left`, `right` |
|
||||
| `match_native_functions` | `left`, `right`, `left_observations[]`, `right_observations[]` |
|
||||
| `trace_application_feature`, `trace_javascript_semantics`, `compare_source_to_bundle` | `application`; feature tracing also accepts `native_observations[]` |
|
||||
| `compare_application_versions`, `compare_javascript_export_shapes` | `left`, `right`; version comparison also accepts both native-observation arrays |
|
||||
| `reconcile_javascript_runtime` | `static_layers[].analysis`, `runtime_observations[]` |
|
||||
| `compare_managed_members`, `verify_managed_native_boundaries`, `import_managed_reconstruction` | Comparison `left`/`right`; verification `managed_boundaries`/`native_observations[]`; reconstruction `static_members` |
|
||||
| `project_managed_application_graph` | `managed_artifact`, `managed_members`, `managed_native_boundaries` |
|
||||
| `project_android_application_graph`, `project_apple_application_graph`, `project_harmony_application_graph` | `inventory_evidence[]` |
|
||||
| `build_call_path`, `find_changed_behavior`, `correlate_static_and_runtime` | `functions[]`, `comparisons[]`, or `static_comparisons[]`/`runtime_comparisons[]`, respectively |
|
||||
|
||||
For example:
|
||||
`trace_application_feature`, `trace_javascript_semantics`,
|
||||
`compare_application_versions`, `compare_source_to_bundle`, and
|
||||
`compare_javascript_export_shapes` accept complete inline application Evidence
|
||||
or an exact reference to a record already retained by the current connection:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -795,46 +568,27 @@ For example:
|
||||
```
|
||||
|
||||
Use the `evidence_id` returned by `analyze_javascript_application` (or another
|
||||
compatible application-graph producer). When the producer returned a summary
|
||||
or inspection page, use its `normalized_result.parent_evidence_id` to select
|
||||
the complete source, rather than the view's own `evidence_id`. Each source can
|
||||
independently be inline or retained, including within one array. Both forms use
|
||||
the same application workflow and owner validation, preserve exact source
|
||||
identity, provenance, authority, coverage and partial or unknown facts, and do
|
||||
not run the producer again. A reference cannot supply facts absent from that
|
||||
record. Workflows that follow source links also require those linked records
|
||||
in the ledger. File references are accepted only where the tool explicitly
|
||||
advertises a file selector; a retained ID does not authorize a file read.
|
||||
compatible application-graph producer). Comparisons accept this form in `left`
|
||||
and `right`; each side can independently be inline or retained. Native
|
||||
observation arrays continue to take complete inline Evidence. Results and their
|
||||
Evidence remain complete inline, and both input forms pass the same semantic,
|
||||
identity, authority, and provenance checks without running the producer again.
|
||||
|
||||
References belong to the current connection's Evidence ledger. Opening another
|
||||
target or restarting the current target preserves retained records, including
|
||||
when provider cleanup fails; records remain available for export and retry.
|
||||
`close_binary` releases the engine and replay cache while preserving these records,
|
||||
even when no binary is active or cleanup fails. `reset_investigation` explicitly
|
||||
clears all records and Unknown revisions after draining earlier work; a healthy
|
||||
active engine remains open. Its optional absolute `save_path` saves a complete
|
||||
canonical bundle before clearing. Failed save, cancellation observed before clearing or incomplete cleanup
|
||||
preserves records, and a published file is reported even if later cleanup fails.
|
||||
A fresh connection has its own in-memory ledger; import the saved bundle to restore
|
||||
original Evidence IDs and complete Unknown history. A missing reference reports its exact ID and
|
||||
`close_binary` clears them, even when no binary is active. A fresh connection
|
||||
has its own ledger. A missing reference reports its exact ID and
|
||||
`details.reason: "missing"`; the server cannot infer
|
||||
whether it was never recorded, cleared, or retained by another connection.
|
||||
Supply complete inline Evidence, repeat its producer, or import an exported
|
||||
Evidence bundle before referencing that imported record. Export a bundle or use
|
||||
`reset_investigation(save_path)` before ending the process if the investigation
|
||||
needs it later. Retention has no automatic persistence or silent eviction.
|
||||
`binary_session.investigation` reports record/revision/head counts, estimated
|
||||
retained representation bytes and the fixed allowance derived from initial V8
|
||||
heap headroom. A capacity refusal preserves the existing investigation; checkpoint
|
||||
and reset it, or start with a larger Node heap. These byte estimates are not RSS.
|
||||
Evidence bundle before referencing that imported record. Export a bundle before
|
||||
closing if the investigation needs it later.
|
||||
|
||||
CLI workflows that advertise `--evidence-bundle` can resolve the same reference
|
||||
JSON from one explicitly selected local bundle, including an MCP export. They
|
||||
also accept portable inline Evidence and use the same application workflows.
|
||||
A standalone CLI invocation cannot resolve another MCP connection's ledger
|
||||
without that exported bundle. See [CLI Evidence reuse](cli.md#import-export-and-compare-evidence)
|
||||
for the supported commands. No additional lookup call, provider selection, or
|
||||
approval step is required for a same-session follow-up.
|
||||
CLI application workflows continue to read portable inline Evidence from files
|
||||
and run the same analysis workflows; a standalone CLI invocation cannot resolve
|
||||
another MCP connection's retained records. No additional lookup call, provider
|
||||
selection, or approval step is required for a same-session follow-up.
|
||||
CLI `inspect-artifact` continues to return complete portable Evidence. Save that
|
||||
JSON to a file and use `inspect-analysis-view` with inline source Evidence for
|
||||
later inventory summaries or pages; a one-shot CLI session cannot retain a
|
||||
@@ -864,27 +618,13 @@ requirements. Compare the union of group names with `tools/list` when checking
|
||||
that a client received the full catalog. An operation absent from all groups is
|
||||
missing from the complete inventory; it is not an implicit unavailable entry.
|
||||
|
||||
Availability also reflects the active provider client's dispatched requests. A
|
||||
refusal or provider failure observed for an operation keeps that operation
|
||||
unavailable until a request for it dispatched later succeeds. A provider adapter
|
||||
or process failure also marks that provider's other declared-available
|
||||
operations unavailable until any request to that provider dispatched later
|
||||
succeeds; their remediation states that the status was inferred rather than
|
||||
observed for them. Recovery is not proof that each operation works, and failures
|
||||
observed on other operations or providers remain. Outcomes are ordered by
|
||||
dispatch to the provider client rather than reply completion, so a late reply
|
||||
from an older request does not override a newer outcome. Request deadlines and
|
||||
cancellations neither mark nor clear availability. A new provider client starts
|
||||
without these outcomes.
|
||||
|
||||
`detail` selects the projection and the result echoes it. The default
|
||||
`summary` keeps open state, target identity, the binding, current-route
|
||||
`capabilities`, run, activity and health status, and each provider candidate's
|
||||
identity, selection, and availability and target-support `status`, `code`, and
|
||||
`reason`. It lists the fields it leaves out in `omitted_fields`: an unavailable
|
||||
candidate's `remediation`, provider `diagnostics`, candidate `capabilities`, and
|
||||
each tool group's static `surface`, `effects`, and `annotations`. Omitted fields
|
||||
are not empty or false.
|
||||
`reason`. It lists the fields it leaves out in `omitted_fields`: provider
|
||||
`diagnostics`, candidate `capabilities`, and each tool group's static `surface`,
|
||||
`effects`, and `annotations`. Omitted fields are not empty or false.
|
||||
`detail: "complete"` returns them inline, groups tools by those flags too, and
|
||||
has no `omitted_fields`. The output schema keys each result's candidate,
|
||||
tool-group and omission fields by its `detail`. CLI `capabilities` and
|
||||
@@ -901,13 +641,6 @@ finish without reading relationships, even when neither traversal mode is
|
||||
available. Discovery does not block that direct invocation or replace its typed
|
||||
partial/error outcomes.
|
||||
|
||||
`trace_native_ui_action` reports authored-UI and native-seed modes separately.
|
||||
Apple dispatch metadata requires a Mach-O target; another provider may supply
|
||||
the symbol-only alternative on other formats. If neither mode is available,
|
||||
read each mode's missing operations and remediation: a remedy for an authored
|
||||
app-bundle route does not by itself describe the native-seed route. Availability
|
||||
does not guarantee that every selector or provider operation will succeed.
|
||||
|
||||
With complete detail, the `effects` and `annotations` arrays list the true
|
||||
members of their fixed vocabularies. A known member omitted from either array is false. This applies
|
||||
only to these closed flag sets; it does not make an omitted tool, provider, mode,
|
||||
@@ -937,9 +670,7 @@ set, `shared_limitations` is the ordered common prefix; append a group's ordered
|
||||
`effects` arrays use the same true-only closed-vocabulary rule as tool
|
||||
availability. Expand every group operation while retaining its provider
|
||||
identity, availability, reason, rejection code, effects, and ordered
|
||||
limitations. An unavailable group carries `remediation` when its readiness
|
||||
owner supplied a repair instruction; `tool_availability` uses that instruction
|
||||
rather than the failure description.
|
||||
limitations.
|
||||
|
||||
`binary_session.analysis_provider_candidates` is authoritative for deep-engine
|
||||
discovery. Each candidate's capability sets, returned with complete detail,
|
||||
@@ -956,37 +687,6 @@ concrete version, selection source, and complete analysis profile through
|
||||
choices return typed selection details. A selected provider is never replaced
|
||||
automatically after a runtime failure.
|
||||
|
||||
`open_binary` resolves an Apple bundle directory (`.app`, `.appex`,
|
||||
`.framework`, `.xpc`, `.systemextension`, `.bundle` or `.plugin`, compared
|
||||
case-insensitively) to the program file its `Info.plist` names in
|
||||
`CFBundleExecutable`; the Evidence subject is that program file. It reads the
|
||||
macOS `Contents/` layout or the flat root layout. A framework with `Versions/`
|
||||
may instead keep `Info.plist` in `Versions/Current/Resources/` with its program
|
||||
file in `Versions/Current/`; a flattened framework without `Versions/`, as
|
||||
Electron and Gecko apps ship it, keeps `Resources/Info.plist` beside its
|
||||
program file. An app keeps its iOS-on-Mac `Wrapper` resolution. For the
|
||||
other types, exactly one layout may declare the bundle, and its `Info.plist`,
|
||||
program directory and program file must stay inside the bundle. Resolution
|
||||
never guesses a program file. Each refusal reports one `details.constraint`:
|
||||
|
||||
- `bundle_info_plist_missing`, with `probed_paths` listing every layout's
|
||||
`Info.plist`.
|
||||
- `bundle_layout_ambiguous`, with `probed_paths` listing the `Info.plist`
|
||||
files found.
|
||||
- `bundle_current_version_missing`, with `probed_paths` naming
|
||||
`Versions/Current`.
|
||||
- `bundle_path_escapes`, with `probed_paths` naming the path that resolved
|
||||
outside the bundle.
|
||||
- `bundle_executable_undeclared`, with `info_plist`.
|
||||
- `bundle_program_missing`, with `info_plist` and `probed_paths` naming the
|
||||
declared program file.
|
||||
|
||||
An unreadable or malformed `Info.plist` keeps its reason without a bundle
|
||||
constraint, and `details.path` names that file. A `.dext`, `.kext`,
|
||||
`.qlgenerator` or `.mdimporter` directory reports `bundle_type_unsupported`. A
|
||||
resource root reports its `layout` as `contents`, `root`,
|
||||
`versioned-framework` or `flattened-framework`.
|
||||
|
||||
`open_bundle_resources` selects a separate read-only app resource session
|
||||
without resolving `CFBundleExecutable`, including when it is absent or
|
||||
unresolvable. `binary_session` reports `session_kind: "bundle_resources"` and the
|
||||
@@ -1014,14 +714,6 @@ is `unavailable` with a reason when ownership could not be revalidated, or
|
||||
at that bounded check. These are historical snapshots, not live process
|
||||
inventories, and do not claim that no short-lived descendant existed.
|
||||
|
||||
After a failed close retires the target, `binary_session.cleanup_retry` names
|
||||
`close_binary` only while that session still retains the cleanup bound to the
|
||||
close error. Target-dependent availability then gives the same conditional
|
||||
cleanup guidance before suggesting `open_binary`. The hint disappears after
|
||||
cleanup succeeds. A pending flag alone does not establish this retry: active
|
||||
targets, an in-flight initial close, and external-mutation blockers keep their
|
||||
own recovery and do not receive this hint.
|
||||
|
||||
`binary_session.cleanup_pending` reports whether the session retains a provider
|
||||
client, session file resource, or provider-candidate resource whose cleanup
|
||||
has not succeeded. It can be true with
|
||||
@@ -1037,36 +729,6 @@ inspection fails or is cancelled before a provider client exists. If no client
|
||||
remains, no provider run or health observation is available; follow the cleanup
|
||||
error's reported resources.
|
||||
|
||||
### Retained cleanup retry
|
||||
|
||||
A `close_binary` that retires the target clears engine replay state while
|
||||
retaining investigation Evidence and Unknown history, including when cleanup fails. When the session keeps every reported owner for the next
|
||||
`close_binary` on the same connection, its `cleanup_incomplete` error carries
|
||||
`recovery: {"kind": "retry_cleanup", "operation": "close_binary"}`.
|
||||
Keep the connection open, let the reported condition settle or resolve it, call
|
||||
`close_binary` without `snapshot_path`, and check the result. The retry does not
|
||||
rerun the failed analysis and can report `cleanup_incomplete` again.
|
||||
`retryable` still describes the failed operation and stays `false`. A failed
|
||||
`snapshot_path` save returns before retirement, keeps the target open and has
|
||||
no `retry_cleanup` hint. `open_binary`, analysis tools and one-shot CLI commands
|
||||
do not report this hint either, even when their cleanup remains pending.
|
||||
|
||||
## Initial target at startup
|
||||
|
||||
MCP normally starts without a target; use `open_binary` to select one during the
|
||||
session. To open a target before serving MCP, set `HOPPER_TARGET_PATH` to the
|
||||
selected path in REA's launch environment. `HOPPER_TARGET_KIND` accepts
|
||||
`executable` (the default) or `database`. Despite the names, these are initial
|
||||
session settings; the configured provider still determines how the target is
|
||||
opened. If the initial open fails, REA reports the failure and exits with code 1
|
||||
without serving MCP. Shutdown during startup cancels and cleans up the open as
|
||||
described below.
|
||||
|
||||
The source-checkout helper `node scripts/print-mcp-config.mjs /path/to/target`
|
||||
prints a registration containing these settings; `--kind database` selects the
|
||||
database kind. The helper is not part of the installed npm package. `rea setup`
|
||||
does not copy these one-shot target settings into generated registrations.
|
||||
|
||||
## Transports
|
||||
|
||||
The eager stdio runtime owns SIGINT, SIGTERM and stdin EOF before opening an
|
||||
@@ -1311,15 +973,6 @@ mapping. A successful start-address mapping does not establish that an entire
|
||||
range is contiguous in the original file, or that its bytes equal analysis
|
||||
memory after loader fixups. Uninitialized, missing, and ambiguous mappings fail
|
||||
explicitly; preserve the reported failure rather than guessing an offset.
|
||||
For Ghidra, such a well-formed address returns `invalid_request` whose message
|
||||
states the bridge's reason and whose remediation points to
|
||||
`inspect_native_load_image`. It carries no `inspect_input_contract` recovery.
|
||||
`details.issues[0].observed` keeps the caller's exact `address` and a
|
||||
`file_mapping` outcome: `unmapped` (outside every memory block),
|
||||
`uninitialized`, `absent` (no authoritative target-file source), `ambiguous`,
|
||||
or `unrepresentable` (beyond the exact JSON integer range). Initialization, not
|
||||
the loaded flag, decides whether source bytes can exist, so an initialized
|
||||
file-backed overlay still maps.
|
||||
|
||||
For constructor-populated pointer or resource slots, a static zero is not proof
|
||||
of a later null value. Inspect the slot's `xrefs` and the writer's assembly or
|
||||
@@ -1442,11 +1095,11 @@ If the client timed out or cancelled during startup, the pending query may have
|
||||
been interrupted. Reissuing `open_binary` for the same active target can reuse
|
||||
the current client, so it is not a fresh-start recovery. On the same connection:
|
||||
|
||||
1. Keep useful inline Evidence or use the retained investigation references;
|
||||
export a bundle before ending the process to resume later. `binary_session` can show the selected binding and its
|
||||
1. Keep any useful inline Evidence, or export a bundle before closing if retained
|
||||
records are needed. `binary_session` can show the selected binding and its
|
||||
recorded state; target selection alone does not prove engine readiness.
|
||||
2. Call `close_binary` and check its result. It drains owned work, closes the
|
||||
provider and engine replay state while preserving investigation Evidence and Unknown history. A `cleanup_incomplete` result
|
||||
provider, and clears retained session records. A `cleanup_incomplete` result
|
||||
must be addressed according to its reported owned resources before retrying.
|
||||
3. Call `open_binary` with the same caller-selected path and
|
||||
`provider_id: "ghidra"`, then retry the first query with an appropriate client
|
||||
@@ -1497,9 +1150,7 @@ cursor outside a procedure is represented as
|
||||
procedure resolution returns typed unreadable output; provider errors never
|
||||
become absence from their message text. `inspect_address_context` requires an explicit address and
|
||||
returns name, procedure, comment, inline-comment, and bookmark facets;
|
||||
unsupported facets are local `unavailable` outcomes. A facet whose input is
|
||||
rejected includes its canonical `error` projection, including any structured
|
||||
issues, alongside the other facet outcomes. Use `current_document`,
|
||||
unsupported facets are local `unavailable` outcomes. Use `current_document`,
|
||||
`current_address`, and `current_procedure` for direct single-field lookups; use
|
||||
the aggregate tools when you need related context together.
|
||||
|
||||
@@ -1545,36 +1196,17 @@ bytes and returns the `integrity_contradictions` records for the extracted
|
||||
files. An active ASAR unpacked entry whose companion file is absent cannot be
|
||||
materialized; extraction fails as `unavailable` with that logical path.
|
||||
|
||||
A container can also declare a checksum over its own bytes; today that is the
|
||||
DFU suffix CRC of a DfuSe file. A mismatch is a contradiction of the root
|
||||
occurrence (logical path `.`), recorded as a `dfu-crc32` pair. Under
|
||||
`record-and-continue` the root becomes `mismatched` while its elements stay
|
||||
listed and extractable as observed-untrusted bytes, and `extract_artifact`
|
||||
returns the root record with the extracted files' records.
|
||||
|
||||
If native DMG child traversal is unavailable after carrier identification,
|
||||
the typed failure retains a revalidated root-only inventory under
|
||||
`details.partial_observation.inventory`, with unknown child coverage and the
|
||||
producer's diagnostics/provenance. It does not borrow or detach an existing
|
||||
attachment. Native checksum verification failures still refuse traversal;
|
||||
`record-and-continue` recovery applies to structured entry and container
|
||||
contradictions.
|
||||
`record-and-continue` recovery applies to structured entry contradictions.
|
||||
|
||||
Occurrences use normalized logical paths for identity and lookup. Optional
|
||||
`source_path` retains the original path within the owning reader when Unicode
|
||||
NFC or reader admission changes its spelling; it is source evidence, not an
|
||||
output destination. New ASAR file occurrences explicitly record the boolean
|
||||
`unpacked`: `true` identifies a companion file and `false` identifies a packed
|
||||
entry. This field describes archive storage, independently of whether the bytes
|
||||
were available or passed integrity verification. Older occurrences may omit it;
|
||||
omission remains unknown and must not be defaulted to `false`. Consumers that
|
||||
previously accepted only literal `true` must also accept `false`, while retaining
|
||||
support for omitted historical values.
|
||||
`compare_artifacts` reports a metadata change when both storage values are
|
||||
known and differ. If only one value is known, storage comparison is unknown;
|
||||
when both are omitted, storage is not compared.
|
||||
|
||||
POSIX letter-colon names such as `a:b` remain valid logical
|
||||
output destination. POSIX letter-colon names such as `a:b` remain valid logical
|
||||
names; Windows extraction refuses names its filesystem cannot materialize.
|
||||
|
||||
Extraction materializes regular files exposed by the active archive reader.
|
||||
@@ -1601,22 +1233,11 @@ failures for such entries name this cause and the operation to rerun with
|
||||
Advertised input and output schemas contain no reachable recursive references.
|
||||
Input schemas inline repeated definitions before applying the selected
|
||||
presentation profile. Both `full` and `compact` start from this self-contained
|
||||
projection; output schemas can share schema-local definitions. Analyst-authored
|
||||
full inputs retain nested fields, descriptions and valid literal examples. The compatibility
|
||||
projection; output schemas can share schema-local definitions. Full inputs
|
||||
retain nested fields, descriptions and valid literal examples. The compatibility
|
||||
projections below can relax advertised constraints; canonical runtime validation
|
||||
still enforces the complete input contract.
|
||||
|
||||
`compare_web_captures` declares its complete producer observations as object
|
||||
operands instead of repeating their full output schemas in the input catalog.
|
||||
Pass the original `inspect_web_page` normalized result in `before.inspection`
|
||||
and `after.inspection`, optional complete `discover_webmcp_tools` results in
|
||||
`webmcp`, or complete `capture_browser_scenario` results in the scenario fields.
|
||||
Do not construct partial observations from the abbreviated declaration: the
|
||||
server still validates every canonical field before comparison. Input groups
|
||||
and analyst-authored normalization rules remain explicit. This presentation
|
||||
changes no capture bytes, identity, completeness, or comparison semantics and
|
||||
does not reduce `inspect_analysis_view` selectors or other tools' declarations.
|
||||
|
||||
When a host forwards input schemas to the model, their examples consume context.
|
||||
REA's advertised examples therefore never copy complete producer Evidence.
|
||||
An embedded Evidence record is shown as its exact `{"kind":"retained-evidence","evidence_id":"ev_..."}`
|
||||
@@ -1855,8 +1476,6 @@ validates every result before delivery. On catalogs large enough that full
|
||||
input schemas alone approach the frame limit, keep the input profile at its
|
||||
`compact` default as well; with full input schemas the advertisement is
|
||||
dominated by the input side.
|
||||
The full input profile has no one-megabyte catalog-frame guarantee, including
|
||||
when output schemas are omitted.
|
||||
|
||||
Only client-side schema discovery and validation are reduced or omitted, so
|
||||
hosts using these profiles should rely on REA's validation and retain their
|
||||
|
||||
+3
-51
@@ -4,52 +4,6 @@ REA exposes six provider-neutral investigation workflows through the MCP
|
||||
`prompts` capability. They complement the generated tool catalog; they do not add, remove, or
|
||||
invoke tools by themselves.
|
||||
|
||||
Clients that expose tools but not prompts can use the target-free
|
||||
`guided_workflows` tool with `action: "list"` or `action: "render"`. CLI callers
|
||||
can use `rea guided-workflows` to list workflows, or pass `--action render`,
|
||||
`--workflow-id ID`, and `--arguments-json '{"feature":"offline search"}'` to
|
||||
render one. Both paths use the same contracts and renderer as native prompts;
|
||||
rendering returns guidance only and does not inspect a target, start analysis,
|
||||
or change session state.
|
||||
|
||||
The list contains workflow identifiers, descriptions, and argument requirements;
|
||||
render returns only the selected workflow's text. Unknown argument names and
|
||||
non-string values are rejected with field paths. `action: "list"` accepts no
|
||||
workflow identifier or arguments. The MCP adapter respects the same
|
||||
`REA_MCP_TOOL_FAMILIES` scope as native prompts, including render requests.
|
||||
|
||||
To find a section in the installed guidance bundle by analyst question, use
|
||||
`search_guidance` and pass a returned locator to `read_guidance_section`. The
|
||||
CLI equivalent is `rea guidance --query "exact JavaScript module"`, followed
|
||||
by `rea guidance --action read --locator LOCATOR`. Search is limited to the
|
||||
packaged managed skill files, works without providers or a target, and returns
|
||||
deterministically paged matches. Every non-generic query word must occur in the
|
||||
section or its subsections; generic-only queries return no matches. Results are documentation; locators
|
||||
are bound to the installed bundle and report stale or unavailable guidance without
|
||||
returning cached section content. A no-match result says nothing about analysis
|
||||
capability availability.
|
||||
|
||||
Each match reports its `match_scope`. A `section` match has every term in its
|
||||
own text, meaning its heading and body before its first subsection. A
|
||||
`subsections` match has the terms only spread across subsections, with no
|
||||
single subsection holding them all. Its `own_text_terms` lists the terms in its
|
||||
own text, and `subsections` lists up to 8 subsection locators, chosen greedily,
|
||||
whose own text holds the rest; `omitted_subsections` counts any more that were
|
||||
needed. Section matches rank first. A parent is not listed when one subsection
|
||||
holds every term, because that subsection is already a match, unless the
|
||||
parent's own text also holds them all. `content_length`
|
||||
is the size of the section that read returns, so callers can decide before
|
||||
reading a broad parent.
|
||||
|
||||
Search returns at most 20 matches per page with 500-code-unit context excerpts.
|
||||
Read retrieves the complete selected section, including its subsections. Admission permits 1 MiB per file,
|
||||
4 MiB per bundle, 4,096 sections, and 25,165,824 expanded UTF-16 code units before
|
||||
section text is materialized. Unreadable or oversized bundles return
|
||||
`bundle_unavailable`, with no partial index or inferred absence. Cleanup failures
|
||||
remain errors with their retained retry owner. The bundle and complete section
|
||||
are bound by fixed-size hashes in locators, so a long heading does not make its own
|
||||
locator unreadable.
|
||||
|
||||
## Prompt inventory
|
||||
|
||||
| Prompt | Purpose |
|
||||
@@ -83,11 +37,9 @@ directly to `analyze_javascript_application` with `input_path`; it does not
|
||||
require `open_binary`. When that Evidence, or `inspect_binary_layout` Evidence,
|
||||
is already retained, it suggests `inspect_analysis_view` before tracing a
|
||||
feature seed. The CLI equivalent is `rea analyze <directory>`.
|
||||
`open_binary` and the doctor target check admit files and Apple bundle
|
||||
directories (`.app`, `.appex`, `.framework`, `.xpc`, `.systemextension`,
|
||||
`.bundle` and `.plugin`). Another Apple bundle type is refused as unsupported,
|
||||
and another directory is reported with an available analysis action; this does
|
||||
not establish support for every directory format.
|
||||
`open_binary` and the doctor target check admit files and macOS app bundles.
|
||||
A directory outside that opening route is reported with an available analysis
|
||||
action; this does not establish support for every directory format.
|
||||
|
||||
For target paths in `compare_application_versions`, the prompt suggests
|
||||
opening each target before calling `inspect_artifact`; inspection operates on
|
||||
|
||||
+22
-228
@@ -68,10 +68,7 @@ function renamed to `dead` remains selectable by name. Use the names returned
|
||||
by the inventory, including their namespaces and any platform symbol prefix.
|
||||
Ghidra also accepts secondary symbols at a function entry, such as an imported
|
||||
`_main` label. A label inside a function is not a procedure-name selector; use
|
||||
its address when selecting the containing function. `search_procedures` matches
|
||||
the same names, in leaf or qualified spelling: the function appears once with
|
||||
its primary name as `value`, and `matched_aliases` lists the qualified names of
|
||||
the secondary symbols that matched.
|
||||
its address when selecting the containing function.
|
||||
Overloads can share a fully qualified name; ambiguity errors return every
|
||||
matching entry address so the caller can select the intended function directly.
|
||||
|
||||
@@ -130,11 +127,7 @@ rea inspect-native-data-type <binary> --address <typed-data-address> --provider
|
||||
|
||||
Instruction facts include bytes, length, decoder text, ordered register/scalar/
|
||||
address tokens, typed references and flow destinations. Mid-instruction, data,
|
||||
outside-memory and unlisted addresses have separate outcomes. For Ghidra,
|
||||
`not-in-listing` means the address is in memory without a listed instruction or
|
||||
defined data. REA does not attempt a decode there: the memory may be uninitialized
|
||||
or contain valid instructions. Older Evidence may report `undecodable` for this
|
||||
same ambiguous case; it does not establish that a decode failed. Effective
|
||||
outside-memory and undecodable addresses have separate outcomes. Effective
|
||||
memory base/index/displacement roles remain unavailable. Ghidra reports its program
|
||||
language variant in `mode`; other providers retain their own mode label. Ghidra
|
||||
MIPS results additionally expose `decoder_context`
|
||||
@@ -145,7 +138,7 @@ that do not expose this facet. The context is an analysis observation, not proof
|
||||
of a runtime processor state.
|
||||
|
||||
`inspect_native_instruction` also returns `incoming_references` to the exact
|
||||
queried address, including data, interior-instruction and unlisted addresses.
|
||||
queried address, including data, interior-instruction and undecodable addresses.
|
||||
Ghidra enumerates its ReferenceManager, excludes synthetic entry-point
|
||||
references, and returns deterministic source-address ordering while retaining
|
||||
distinct operand references from the same source. `[]` means that enumeration
|
||||
@@ -188,24 +181,12 @@ observed entry label. An interior address therefore remains visible without
|
||||
being mistaken for the function entry. Identity failures are reported as
|
||||
`procedure.status: "unknown"`; label failures leave a null name and `name_error`.
|
||||
Neither failure discards usable pseudocode, and names are not parsed from it.
|
||||
A provider answer without pseudocode is an error item that keeps that
|
||||
observation's own `limitations`, so a reason the provider reported for that
|
||||
entry, such as a Kuna export failure or a missing body, stays with the item.
|
||||
Null pseudocode is `capability_unavailable`; empty or non-text pseudocode is
|
||||
`unreadable_output`. Provider failures keep their typed error. Successful items
|
||||
do not repeat limitations; the batch Evidence aggregates every observation's
|
||||
limitations by provider and operation.
|
||||
Resolved entries also appear in the composed Evidence locations. New batch
|
||||
Evidence commits the revised workflow profile; retained historical Evidence
|
||||
remains valid. This tool
|
||||
has no dedicated CLI command; the existing `decompile` command remains the
|
||||
single-procedure CLI operation.
|
||||
|
||||
`detail: "summary"` retains the complete batch and returns only its counts,
|
||||
identities and sizes. `inspect_analysis_view` then reads rows or one body from
|
||||
the retained record, an Evidence file or a portable bundle without repeating
|
||||
decompilation: see [selected batch results](mcp-contracts.md#selected-batch-decompile-results).
|
||||
|
||||
Each batch schedules one provider read at a time, but has no shared input-count
|
||||
or aggregate retained-output allowance. It retains every requested row and full
|
||||
pseudocode body through Evidence construction. The MCP response budget checks
|
||||
@@ -300,20 +281,6 @@ relationships but reports their completeness as unknown. In particular, a resolv
|
||||
tail call observed in one direction does not prove that the reverse inventory is
|
||||
complete. An empty Hopper inventory cannot establish that no such call exists.
|
||||
No whole-binary reverse scan is performed.
|
||||
|
||||
For large native traces, `trace_feature` accepts `detail: "summary"`. It retains
|
||||
the complete trace and returns counts and its parent Evidence ID. Use
|
||||
`inspect_analysis_view` with that ID and `view: {kind: "native-feature-trace",
|
||||
collection: "references", offset: 0, limit: 64}` to page the original rows;
|
||||
`matches` and `unknowns` are separate collections. Matches accept an exact
|
||||
`address`; references accept exact `source_address` and `target_address`
|
||||
filters. Row indices refer to the retained collection, and page coverage does not establish provider discovery
|
||||
completeness. `list_unknowns` pages the independently managed question heads.
|
||||
In the one-shot CLI, `rea trace ... --detail summary --evidence_output PATH`
|
||||
stores the trace and its linked source Evidence for later
|
||||
`inspect-analysis-view` calls. This bundle does not include the CLI session's
|
||||
Unknown registry; its original question texts remain available in the trace's
|
||||
`unknowns` pages.
|
||||
Regex searches use ECMAScript Unicode syntax in a cancellable worker with a
|
||||
matching deadline and a worker old-generation heap. Deadline or cancellation
|
||||
stops matching while leaving the Hopper API available. The deadline is
|
||||
@@ -365,34 +332,10 @@ back. The effective heap stays unestablished when either flag is present,
|
||||
including when an absolute `--max-old-space-size` is also on the command
|
||||
line or in the recovered `NODE_OPTIONS`. Node accepts the config path as
|
||||
`=` or the next argument.
|
||||
Literal mode retains Python Unicode full case folding, which can expand a
|
||||
character into a sequence: for example, `ss` matches `ß`. This differs from
|
||||
REA's Unicode simple case folding used by other native search providers and
|
||||
Hopper regex mode.
|
||||
Literal mode retains Unicode casefold matching.
|
||||
|
||||
## Bounded static traces
|
||||
|
||||
Hopper caller queries, incoming procedure references and function dossiers
|
||||
reconcile reported reverse relationships with forward call observations. Each
|
||||
standalone query or composed workflow scans the current document in pages of at
|
||||
most 64 procedures and retains at most 4 MiB of encoded scan results. A composed
|
||||
workflow reuses its own index and clears it when the workflow finishes. The
|
||||
default cooperative scan budget is
|
||||
2,000 ms; set `REA_HOPPER_CALLER_SCAN_BUDGET_MS` to a positive integer number of
|
||||
milliseconds before starting REA to change it. The default limits additional
|
||||
work on the serial Hopper bridge; larger documents may need a higher setting.
|
||||
Results report the examined procedure count and any budget interruption.
|
||||
An individual page, native caller list, and target instruction list each have
|
||||
a 512 KiB encoded allowance. Hitting one reports the incomplete observation.
|
||||
Cancellation is checked between pages, but cannot interrupt a Hopper API call
|
||||
already executing; Hopper's native API itself materializes its reference arrays
|
||||
before REA can bound their copies. Hopper exposes no document revision token, so scans are not
|
||||
reused across workflows and coverage and consistency during a live scan remain
|
||||
unknown even when every procedure was examined.
|
||||
Hopper observations are re-observed rather than replayed from snapshots because
|
||||
the provider has no document revision token. Safe observations may still be
|
||||
retained as Evidence.
|
||||
|
||||
`trace_native_ui_action` / `rea trace-native-ui-action <target> <seed>` accepts
|
||||
one authored selector/object ID, native symbol or exact function address. It
|
||||
joins UI wiring to uniquely matched class/selector implementations and follows
|
||||
@@ -402,11 +345,6 @@ sites remain distinct. A static route does not establish runtime reachability.
|
||||
Swift/closure dispatch requires resolved static references; unavailable ABI
|
||||
forms remain unknown.
|
||||
|
||||
Encoded Objective-C method records retain their own locations in dispatch
|
||||
Evidence. A handler node uses the implementation address with an unknown file
|
||||
offset unless that function already has an independently located graph node;
|
||||
the method-record offset does not locate the handler's code.
|
||||
|
||||
Incomplete reference kinds do not discard known typed calls or partial
|
||||
provider call references. A `provider_call` with `observed` resolution retains
|
||||
the provider-reported call site and endpoint without claiming generic direct,
|
||||
@@ -452,31 +390,10 @@ Within one value trace, each exact procedure selector is attempted at most once,
|
||||
including failed analyses. A failed seed returns the provider's original typed
|
||||
failure and corrective guidance; it cannot establish a trace or target identity.
|
||||
Failures while expanding a validated seed remain unknown observations and do not
|
||||
establish canonical function identity. A cancelled expansion read returns the
|
||||
cancellation. An expansion read whose provider cannot confirm cleanup of
|
||||
resources it owns stops the trace: the result is that provider's own
|
||||
`cleanup_incomplete` failure with its resources and any bound cleanup retry, and the work
|
||||
admitted before the stop is retained as partial Evidence with a located unknown
|
||||
at the stopping read. A successful seed without a native def-use
|
||||
establish canonical function identity. A successful seed without a native def-use
|
||||
model still returns its procedure and an explicit coverage unknown. A new trace
|
||||
can retry failed selectors.
|
||||
|
||||
Paging and traversal completeness are separate facts. `page_truncated` means the
|
||||
page omits retained nodes, and the edges they source, before `offset` or after
|
||||
`next_offset`. Every page repeats `traversal`: `complete`, or `truncated` with
|
||||
`reasons` naming each observed cause: the `max_depth`, `max_functions`,
|
||||
`max_call_sites`, `max_nodes` or `max_edges` budget, the fixed
|
||||
`node_payload_bytes` allowance, `provider_pcode_truncated`,
|
||||
`ambiguous_call_candidates`, or, only in a stopped trace's partial Evidence,
|
||||
`cleanup_incomplete`. Each cause also has a located unknown. A budget is
|
||||
reported only after it rejects work, so an exactly filled budget is not
|
||||
truncation; a depth or candidate limit is reported only when no other path
|
||||
analyzes or queues that destination. `total_nodes` and `total_edges` count
|
||||
retained rows. `truncated` remains true for either kind of omission. Failed
|
||||
analyses and unresolved calls stay in `unknowns` and do not truncate the
|
||||
traversal. Each page reruns analysis; pages are not pinned to one snapshot
|
||||
across session edits.
|
||||
|
||||
The raw Java `is_dead` flag can remain set after Ghidra decodes a live block;
|
||||
`block_membership` reports actual syntax-tree membership separately.
|
||||
|
||||
@@ -567,10 +484,8 @@ output-budget omissions remain omissions.
|
||||
|
||||
Source input is limited to 64 MiB. Binary and direct XML decoding reserve an
|
||||
estimated 64 MiB decode budget before parsing; this is not a hard process-memory
|
||||
bound. The XML estimate charges every element and every text, comment or CDATA
|
||||
node of the decoder's DOM at a measured upper bound, so a plist made of many
|
||||
small elements reaches it at a few MiB of source. JSON decode memory has not
|
||||
been measured and has no equivalent decode estimate. A refused input reports non-retryable `resource_constraint` with
|
||||
bound. JSON decode memory has not been measured and has no equivalent decode
|
||||
estimate. A refused input reports non-retryable `resource_constraint` with
|
||||
`reported_limits.maximum_input_bytes` or `maximum_decoded_bytes` as applicable.
|
||||
Ordinary direct-source decoding has a ceiling of 998 source nesting levels,
|
||||
derived from the existing 1000-level JSON result capacity by reserving one level
|
||||
@@ -615,21 +530,7 @@ support for core-dump analysis, historical FVMLIB layouts, or unobserved native
|
||||
diagnostic spellings. Unknown or contradictory header values and signalled
|
||||
commands remain failures.
|
||||
|
||||
`inspect_macho` reads load-command framing and fields from the selected Mach-O
|
||||
bytes. It does not use line-delimited tool output to decide where a command,
|
||||
section, or embedded string begins. Names whose bytes are not valid UTF-8 have
|
||||
a null text value and retain their original bytes as lowercase hexadecimal in
|
||||
`name_bytes_hex`, `segment_bytes_hex`, or `path_bytes_hex`; fixed-width names
|
||||
stop at the first NUL, and command paths exclude their NUL terminator. Older
|
||||
retained Evidence may omit the byte fields. Large binary sizes and offsets that
|
||||
cannot be represented as safe JSON integers retain an exact hexadecimal field;
|
||||
the corresponding numeric field is null. A section alignment too large to
|
||||
represent in bytes retains `alignment_exponent` instead.
|
||||
|
||||
Unknown or partially decoded commands retain their selected-slice-relative
|
||||
offset and size. Their unexamined fields are described in load-command coverage
|
||||
limitations; an exhaustive command inventory does not by itself establish that
|
||||
every payload field was decoded. `inspect_macho.selected_architecture` identifies the exact selected lipo slice
|
||||
`inspect_macho.selected_architecture` identifies the exact selected lipo slice
|
||||
and its `container_file_offset` in the input file. A thin Mach-O has a known
|
||||
offset of zero; an unknown universal-slice offset stays null. The Evidence
|
||||
subject format describes the input container: `mach-o-universal` for FAT files,
|
||||
@@ -637,49 +538,15 @@ including those with one slice. The normalized result also reports the container
|
||||
in `format`; `selected_architecture` identifies the selected Mach-O image.
|
||||
Architecture `file_offset` values are container-relative. Entrypoint, segment
|
||||
and section `file_offset` values retain their source meaning and are relative
|
||||
to the selected Mach-O slice. Decoded load-command fields retain their original
|
||||
values. Repeated fields, such as each `LC_BUILD_VERSION` tool, are keyed
|
||||
`name #n` in order. Each decoded `LC_LINKER_OPTION` string is keyed `string #n`
|
||||
and checked against the declared `count`. Evidence locations map the LC_MAIN entrypoint and nonempty file-backed segment
|
||||
to the selected Mach-O slice. Raw load-command fields retain their original values.
|
||||
A field otool prints more than once, such as each `LC_BUILD_VERSION` tool, is
|
||||
keyed `name #n` in print order. Each `LC_LINKER_OPTION` string is keyed
|
||||
`string #n` and checked against the declared `count`. Evidence locations map the LC_MAIN entrypoint and nonempty file-backed segment
|
||||
ranges to container offsets by adding the observed slice offset. Zero-fill
|
||||
segments do not produce file ranges. When the selected slice's container offset
|
||||
is unknown, those mapped locations are omitted with an explicit limitation;
|
||||
architecture inventory locations remain available.
|
||||
|
||||
`inspect_macho.encryption` reports the `LC_ENCRYPTION_INFO(_64)` state of every
|
||||
slice, not only the selected one. It decodes only the FAT table, slice headers
|
||||
and load commands. Like every Mach-O container read, a `0xcafebabe` input of at
|
||||
most 16 MiB is first read whole to rule out a JVM class file, so that read
|
||||
includes code bytes; larger inputs and thin images read only the structures
|
||||
above. A slice is `encrypted` when its single encryption command has a nonzero
|
||||
`cryptid`, `not-encrypted` when `cryptid` is zero, and `absent` only when every
|
||||
load command was read and none was an encryption command. A command whose width
|
||||
or `cmdsize` contradicts the image, more than one such command, or a range past
|
||||
the slice is `malformed`; a slice whose commands could not be fully read is
|
||||
`malformed` or `unknown` with a reason, keeping any commands seen before the
|
||||
failure. A slice with more than two encryption commands keeps the first two
|
||||
and states the count in its reason. Range mapping in all slices shares one
|
||||
budget of segment and section records (16,384, a sixteenth of the native
|
||||
output budget); a slice past it keeps its status and reports its mapping as
|
||||
`unknown` with that reason. Framing caps retained segment commands before decoding.
|
||||
Only a successful mapping consumes the shared allowance; failed mappings leave
|
||||
it available to later slices. Each slice's `range` is end-exclusive in slice and input-file
|
||||
coordinates and maps to file-backed segment fragments, VM addresses and the
|
||||
overlapping nonempty sections; bytes outside every segment remain unmapped
|
||||
fragments. A segment with no VM size keeps its file bytes as fragments with
|
||||
null VM addresses, and a segment whose file size exceeds its VM size or whose
|
||||
VM range leaves the address space makes the mapping `unknown` with a reason
|
||||
while the slice keeps its encryption status. `architecture` is the lipo name of the record at the same offset,
|
||||
size and CPU identity, and is null when no single record matches. A ciphertext
|
||||
range adds a limitation naming its input-file bytes, because disassembly,
|
||||
decompilation, strings and references derived from them do not describe
|
||||
program code. Encryption is read through the same regular-file descriptor
|
||||
that `inspect_macho` binds for its load-command reads, whose SHA-256 must match
|
||||
the session target before and after the native tools run, or the call reports
|
||||
`artifact_changed`. A target that cannot be opened, or a symlinked, non-regular
|
||||
or changing target, fails the call instead of reporting tool facts without an
|
||||
encryption status.
|
||||
|
||||
Hopper's `address_to_file_offset` combines the native image-relative mapping with
|
||||
original-file coordinates only after checking the source against the session's
|
||||
SHA-256. FAT tables and embedded Mach-O headers come from that same hashing pass.
|
||||
@@ -716,9 +583,6 @@ complete observation.
|
||||
undecodable filenames and skipped symlinks, as partial coverage. Known
|
||||
executable roots remain available; unexamined names and symlink targets
|
||||
cannot establish an empty root set or the absence of other roots.
|
||||
A dependency symlink whose target is not valid UTF-8 is undetermined, with
|
||||
its stored target bytes in the limitation and partial coverage; a selected
|
||||
root crossing such a symlink is refused with the same bytes in its diagnostic.
|
||||
- `inspect_asset_catalog` / `rea inspect-asset-catalog <app>` reads compiled
|
||||
`Assets.car` metadata through macOS `assetutil --info`. Catalog digests, raw
|
||||
rendition fields, pagination and exact UI resource-name matches are returned.
|
||||
@@ -836,16 +700,10 @@ complete observation.
|
||||
release their representation reservation for later documents.
|
||||
Repeated NIBArchive fields retain all serialized values in order. Array
|
||||
outlet connections resolve each serialized member and retain the collection
|
||||
object ID and member position on the outlet observation. NIB hierarchy uses
|
||||
recognized child fields, reverse `NSSuperview` links and window content links.
|
||||
AppKit registry pairs with a resolvable `NSRoot` owner establish top-level
|
||||
membership; other registry associations alone do not establish visual
|
||||
parentage and leave hierarchy coverage partial. Known non-view cell ownership
|
||||
and control-view backreferences remain raw facts. Conflicting explicit parents
|
||||
remain unknown rather than choosing an archive-order winner. Retained objects
|
||||
keep their decoded fields; uncertain registry diagnostics retain the source
|
||||
registry, child and value IDs. Missing, malformed or repeated registry
|
||||
declarations also qualify hierarchy coverage.
|
||||
object ID and member position on the outlet observation. AppKit object-parent
|
||||
registries use paired keys and parents; objects paired with `NSRoot` are
|
||||
top-level, while nested objects keep their recorded parent. Conflicting or
|
||||
unreadable registry pairs remain incomplete hierarchy evidence.
|
||||
`NSString` objects with the [recognized `NS.bytes` representation](https://github.com/swiftlang/swift-corelibs-foundation/blob/main/Sources/Foundation/NSString.swift#L201-L215) are decoded
|
||||
as UTF-8 without substituting malformed bytes. A failed conversion retains
|
||||
the source string object ID and `nibValues.NS.bytes` base64 data; dependent
|
||||
@@ -872,11 +730,6 @@ complete observation.
|
||||
files and universal slices before metadata decoding. CPU subtypes and
|
||||
capability bits remain source facts; an `arm64` request may select an
|
||||
`arm64e` image under the existing slice preference policy.
|
||||
Objective-C class, protocol and category lists require eight-byte entries;
|
||||
Swift type and conformance sections require four-byte entries. Misaligned
|
||||
section lengths and unsupported non-PowerPC big-endian CPU declarations return
|
||||
`unsupported_target` with the selected path and precise reason. Recovery
|
||||
selects a well-formed or supported artifact and reopens that target.
|
||||
The byte reader holds the whole selected file in memory, so it admits files up
|
||||
to 64 MiB / 67108864 bytes by default. Select a larger budget with
|
||||
`max_input_bytes` (CLI `--max-input-bytes`), at most 1 GiB / 1073741824
|
||||
@@ -971,11 +824,7 @@ complete observation.
|
||||
and `__objc_catlist` categories are also decoded. Categories record the
|
||||
extended class, local or external, plus
|
||||
their methods, protocols and properties; category methods appear as
|
||||
implementations with a `category`. A category whose class reference is null,
|
||||
or is bound to a symbol other than `_OBJC_CLASS_$_`, such as the `_$s…CN`
|
||||
metadata symbol of another module's Swift class, keeps a null class, and
|
||||
`objc_properties_categories` coverage names that reference. Swift
|
||||
field-offset globals that are only
|
||||
implementations with a `category`. Swift field-offset globals that are only
|
||||
initialized at runtime stay unresolved. Generic, resilient, async and
|
||||
coroutine tables, and inherited overrides, have explicit unsupported or
|
||||
partial coverage. Other providers retain the
|
||||
@@ -1023,9 +872,7 @@ with a path-specific gap and incomplete capture coverage. Null alone does not
|
||||
establish that an accessibility value was successfully read or absent.
|
||||
|
||||
An initial PID or window mismatch is `invalid_request` on `pid` or
|
||||
`window_id` and leaves provider availability intact. A selected window that
|
||||
exists off screen is not a mismatch: selection succeeds and the permission and
|
||||
capture checks report their own outcome. An unavailable executable
|
||||
`window_id` and leaves provider availability intact. An unavailable executable
|
||||
URL or unresolved executable path leaves identity unknown and reports
|
||||
`capability_unavailable`. Identity or window loss
|
||||
after the initial capture remains an observation failure with collected facts.
|
||||
@@ -1174,55 +1021,11 @@ unpaired Unicode surrogate; a rejection identifies the field and UTF-16 index
|
||||
and leaves every annotation unchanged. CRLF, supplementary Unicode characters,
|
||||
and combining characters are preserved. Windows P0 remains read-only. Ghidra has no
|
||||
GUI authority, and REA never falls back automatically to Hopper.
|
||||
For a before-and-after `compare_functions` in MCP, call `analyze_function`
|
||||
before and after the edit while the session remains open; the annotation Evidence
|
||||
itself is not an `analyze_function` input. If comparing after close, export the
|
||||
needed Evidence before closing and import it into the later session. A one-shot
|
||||
CLI annotation returns its edited dossier before its temporary session is
|
||||
discarded; a later CLI invocation with the same fresh-import settings starts from
|
||||
the original executable. Comparison preserves correspondence across renames
|
||||
when authenticated artifact identity, architecture and provider-owned mapping
|
||||
and slice commitments establish the same canonical entry. The result reports
|
||||
`matched`, `method: "same-entry"` and `entry_address`; name and exact-text changes
|
||||
remain separate dimensions. Equal numeric addresses without applicable coordinate
|
||||
commitments do not establish same-entry correspondence. Inspect the retained
|
||||
source Evidence and coverage when correspondence remains ambiguous.
|
||||
To aggregate already-retained comparison results, pass their exact IDs to
|
||||
`find_changed_behavior`, for example:
|
||||
|
||||
```javascript
|
||||
await call("find_changed_behavior", {
|
||||
comparisons: comparisons.map(({ evidence_id }) => ({
|
||||
kind: "retained-evidence",
|
||||
evidence_id,
|
||||
})),
|
||||
});
|
||||
```
|
||||
|
||||
These IDs select comparison Evidence, not function dossiers or summary views.
|
||||
Complete inline comparisons remain accepted and can be mixed with references.
|
||||
The current server must retain the comparisons and their original source closure;
|
||||
import the complete source bundle after a session handoff. Missing or damaged
|
||||
records are explicit failures. A missing required retained record reports its exact
|
||||
`evidence_id` with `details.reference_scope: "retained-closure"`. Import the
|
||||
originating bundle with its linked sources, or re-create those observations in the
|
||||
current investigation, before retrying; resending the inline comparison alone
|
||||
does not restore its closure. Aggregating static differences does not establish
|
||||
observed runtime change.
|
||||
|
||||
`set_address_name` and `set_addresses_names` give Ghidra sessions the same
|
||||
compact naming results as Hopper: a local function entry renames that function,
|
||||
and any other mapped address, such as a global, receives a new or renamed primary
|
||||
label. A batch decides every address from the names held before it: addresses
|
||||
that request the same name in one namespace, or a name another selected address
|
||||
holds, are rejected without applying them, so JSON member order never selects a
|
||||
winner. Every other address applies separately, and each address reports one
|
||||
success boolean. `raw_result.failures` gives each failed address's canonical
|
||||
address and reason. A contested name appears once in `raw_result.conflicts`,
|
||||
with its namespace, requesting addresses and holding addresses, and conflict
|
||||
failures refer to it by index. To move a label between
|
||||
addresses, rename its holder first and assign the name in a later call. Use
|
||||
them to name many globals and helpers cheaply; use
|
||||
label. A batch applies each address separately and reports one success boolean
|
||||
per address. Use them to name many globals and helpers cheaply; use
|
||||
`annotate_native_function` when you also want comments and a refreshed dossier.
|
||||
Ghidra name edits preserve the existing namespace. Supply either a leaf name
|
||||
such as `renamed` or a fully qualified name in that namespace, such as
|
||||
@@ -1265,17 +1068,8 @@ counted as known archive omissions. Recognized symlinked `.nib` entries are
|
||||
not followed; they retain their paths and reader limitations as counted archive
|
||||
omissions. Either gap makes `archive_decode` partial and the graph truncated
|
||||
while preserving decoded documents. Its `examined` count remains actual decode
|
||||
attempts. `archive_object_decode` separately reports whether archive selection
|
||||
and decoding examined the object sources; hierarchy-only and connection-only
|
||||
unknowns do not make that facet partial. Asset-catalog resource-key joins use
|
||||
this source coverage together with document and object projection coverage.
|
||||
Incomplete hierarchy or synthetic hierarchy-node limits alone do not qualify
|
||||
otherwise complete resource-key joins; omitted archives or resource objects do.
|
||||
Archive omission reasons are counted independently of the bounded path samples.
|
||||
A UID outside the exact supported range reports `unsupported_uid_exact_range`,
|
||||
not budget exhaustion. Mixed source failures report `multiple_archive_decode_causes`;
|
||||
limitations retain omission counts by cause even when individual paths exceed
|
||||
the reporting allowance.
|
||||
attempts, and asset-catalog UI joins retain their existing incomplete-graph
|
||||
qualification.
|
||||
|
||||
UI dispatch joins use the authored receiver's class independently of its
|
||||
presentation kind (controller, view, control or another object). An unresolved
|
||||
|
||||
@@ -53,14 +53,6 @@ An unset fuse with a present blob is reported as a limitation. These are static
|
||||
sentinel observations, not proof of runtime activation: sentinel-like text in
|
||||
script or asset bytes also participates in the scan.
|
||||
|
||||
Artifact comparison uses each member's runtime role and exact source key as
|
||||
semantic metadata. Unicode-normalized graph paths do not erase differences in
|
||||
runtime asset keys. For members paired at the same logical path, a directory
|
||||
index change alone does not change the member. Ordinal-based fallback paths can
|
||||
change when entries are reordered. When only one observation records member
|
||||
metadata, the complete runtime member declaration remains unavailable. Original
|
||||
source fields remain available in the retained Evidence.
|
||||
|
||||
## Supported layout and limits
|
||||
|
||||
The decoder follows the 64-bit little-endian layout in Node
|
||||
|
||||
+3
-18
@@ -81,13 +81,9 @@ Its separate inferred truncation lengths do not replace that limit; an unset
|
||||
file-header limit remains `null`.
|
||||
|
||||
Addresses count once per packet, including loopback and nested IP occurrences.
|
||||
DNS address attribution requires one question in one response and complete,
|
||||
consistent RR owner/type and section counts. Only answer-section A/AAAA records
|
||||
matching that question (or its unambiguous answer-section CNAME chain) are
|
||||
associated, with the requested address type respected. Authority, additional,
|
||||
unrelated, malformed, and ambiguous addresses remain in the unattributed
|
||||
inventory and make coverage partial. These packet-local relationships do not
|
||||
prove final resolution or destination authenticity. Protocol distribution counts
|
||||
DNS addresses are associated only with one question in the same response packet;
|
||||
additional records may be included, so this does not prove final resolution.
|
||||
Ambiguous addresses remain in a separate inventory. Protocol distribution counts
|
||||
exact nonempty `frame.protocols` stack strings. Frames for which TShark reports
|
||||
an empty stack, including zero-length packets, still contribute to the packet
|
||||
count without adding a protocol row. An all-empty stack inventory can have
|
||||
@@ -106,17 +102,6 @@ URI leaves coverage partial. The HTTP URI profile covers HTTP/1 and HTTP/2;
|
||||
HTTP/3 URI semantics are unexamined. All admitted URI fields share the same
|
||||
credential exclusion and source-value retention rules.
|
||||
Counts preserve distinct observed source URIs before omission or coalescing.
|
||||
|
||||
Failed capture-summary TShark commands apply the same HTTP URI policy to
|
||||
retained JSON diagnostics, including cancellation, output-limit and cleanup
|
||||
failures. JSON spelling outside the excluded credential spans is preserved.
|
||||
An unfinished or malformed JSON token and its following suffix are withheld,
|
||||
with `captured_output.truncated: true`; complete preceding fields remain
|
||||
available. Diagnostic byte counts and output allowances describe original
|
||||
producer input, before credential exclusion. Caller-selected packet-field tools
|
||||
continue to return their selected recorded values.
|
||||
Origin-form request targets and HTTP/2 `:path` values retain path semantics,
|
||||
including leading double slashes and userinfo-like text within the path.
|
||||
`coverage: "partial"` records unresolved DNS attribution or URI item loss;
|
||||
component credential exclusion is also counted explicitly. Diagnostics retain
|
||||
both tool stderr streams. Recorded fields do not establish decrypted payloads,
|
||||
|
||||
+1
-11
@@ -40,12 +40,6 @@ signer identifier facts. A signer identified by subject key identifier reports
|
||||
that value as `subject_key_identifier_hex`. Other certificate types keep
|
||||
`pkcs7` null.
|
||||
|
||||
Each signer reports `unsigned_attribute_oids`: null means the field was absent,
|
||||
and an array lists the types when present. Older retained Evidence may omit the
|
||||
field. Unsigned attribute contents, including nested signatures and timestamp
|
||||
tokens, are not decoded; their presence makes coverage partial and adds a
|
||||
limitation. Certificate and signer counts refer to the outer SignedData.
|
||||
|
||||
A missing table is observed absence of that directory. It is not proof the
|
||||
image is unsigned in every catalog or overlay sense.
|
||||
|
||||
@@ -60,11 +54,7 @@ fails the same way. SignedData must contain at least one `SignerInfo`, and
|
||||
each `SignerInfo` must carry a non-empty signature value. Each embedded X.509
|
||||
certificate must include `subjectPublicKeyInfo` and `signatureValue`. Decoded
|
||||
certificate name text, serials, validity spellings, and subject key identifiers
|
||||
share a 64 KiB budget per PKCS#7 payload, and each payload decodes at most
|
||||
16,384 ASN.1 values. Exhausting either fixed allowance returns a nonretryable
|
||||
`resource_constraint` error naming it, not an invalid-input error; this release
|
||||
has no runtime override, and `max_file_bytes` and `max_entries` do not raise
|
||||
it. Unsupported certificate name
|
||||
share a 64 KiB budget per PKCS#7 payload. Unsupported certificate name
|
||||
encodings, including GeneralString, fail explicitly. TeletexString (T.61)
|
||||
values are rendered with the ISO-8859-1 compatibility mapping used by OpenSSL;
|
||||
this is not a full T.61 decoder, so the result includes a limitation whenever
|
||||
|
||||
+6
-75
@@ -224,30 +224,7 @@ child output. Child output remains in the capture Evidence on stdout.
|
||||
resource limits, and normalization settings have defaults; see
|
||||
`processScenarioSchema` for the exact contract. Event times are milliseconds
|
||||
from launch, must be ordered, and must fall within `timeout_ms`. Inputs marked
|
||||
`sensitive` are sent to the process but persisted with empty `data`,
|
||||
`sensitive: true`, and `input_bytes`. Exact nonempty CR/LF-separated input
|
||||
literals are removed from retained terminal text, including cross-chunk and
|
||||
VT-styled echoes. Rendering uses that redacted stream, so intermediate terminal
|
||||
states and serialized state cannot restore removed bytes. Normalized text is
|
||||
checked again. Encoded, transformed, or arbitrarily edited values are not a
|
||||
secret-detection contract; their privacy remains unknown. Unmarked inputs and
|
||||
caller-selected environment values remain evidence.
|
||||
|
||||
The capture-local matcher admits at most 65,536 prefix states and 8,388,608
|
||||
construction work units. Projection uses at most 32 MiB of reusable position
|
||||
indexes and 67,108,864 work units across one capture. Capacity failures return a
|
||||
non-retryable resource error and withhold terminal text in partial observations.
|
||||
If output admission leaves gaps, the sensitive capture withholds retained
|
||||
terminal text rather than interpreting separated fragments as a complete stream.
|
||||
`raw_terminal.redacted_bytes` reports bytes removed from admitted original
|
||||
chunks; these bytes still consume `output_bytes` alongside retained safe bytes.
|
||||
Private display effects remain unknown, with frame order and provenance retained.
|
||||
|
||||
This changes the sensitive-input Evidence representation: consumers of the old
|
||||
`<redacted-input:N-bytes>` placeholder must read `input_bytes` instead. Existing
|
||||
saved captures remain readable and are not rewritten; old files may still
|
||||
contain previously echoed input. A private-input comparison policy version
|
||||
separates old and new captures; reacquire both sides for comparable observations.
|
||||
`sensitive` are sent to the process but persisted as a byte-count placeholder.
|
||||
Environment overrides are recorded in the scenario commitment; inherited
|
||||
values are not copied into Evidence.
|
||||
|
||||
@@ -257,9 +234,7 @@ A bare executable name uses the scenario's `PATH` override, then the inherited
|
||||
An explicitly empty `PATH` searches only the working directory. Executable,
|
||||
working-directory, and observation paths preserve native `symlink/..` traversal.
|
||||
|
||||
Terminal width must be at least two columns: the pinned xterm renderer cannot
|
||||
represent a one-column screen. This applies to the initial size and every resize.
|
||||
Terminal dimensions must also satisfy `columns × (rows + scrollback) ≤ 1,000,000`.
|
||||
Terminal dimensions must satisfy `columns × (rows + scrollback) ≤ 1,000,000`.
|
||||
Every scheduled resize uses the same selected scrollback and must fit this
|
||||
working-buffer budget. REA rejects oversized scenarios before launching the
|
||||
command; it never clamps the requested dimensions. This limit is independent
|
||||
@@ -319,8 +294,7 @@ own classification until its exact resources are released.
|
||||
|
||||
Terminal frame `data` is the comparison text after the selected normalization.
|
||||
New captures also preserve `raw_data` when normalization changes a PTY chunk;
|
||||
otherwise `data` is already that text. For sensitive captures, both fields
|
||||
contain only the safe projection; no unredacted backup is retained. Older captures may lack the
|
||||
otherwise `data` is already the original text. Older captures may lack the
|
||||
original text for changed chunks. Both values participate in Evidence identity,
|
||||
while comparisons and trace assertions use the normalized text. Retaining the
|
||||
original adds at most `limits.output_bytes` of text, because it comes from the
|
||||
@@ -355,12 +329,9 @@ recapture both sides with the same observation version before comparison.
|
||||
When path normalization is enabled, aliases apply to selected path identities
|
||||
at clear text delimiters. A selected executable path does not alias a longer
|
||||
sibling filename such as `tool-helper`; selected parent directories still
|
||||
normalize within descendant paths, including when the working directory or an
|
||||
observation path is spelled with a trailing `/` or `/.`. A selected filesystem
|
||||
root `/` also aliases descendant paths, without rewriting URL separators or
|
||||
HTML closing tags. The comparison contract
|
||||
records these rules as `path-component-boundaries-v3`, so captures made with the
|
||||
earlier rules cannot be compared as if the text had the same meaning.
|
||||
normalize within descendant paths. The comparison contract records these rules as
|
||||
`path-component-boundaries-v2`, so captures made with the earlier prefix rules
|
||||
cannot be compared as if the text had the same meaning.
|
||||
|
||||
Rendered frame `lines` hold the visible rows after normalization, with trailing
|
||||
U+0020 spaces removed. Blank rows and internal spaces remain. `columns` records
|
||||
@@ -529,11 +500,6 @@ Every capture requires `truncation_details`, with separate accounting for:
|
||||
An unreadable directory is recorded in `enumeration_omissions` with its
|
||||
root-aliased path and operating-system error code. Entries observed before
|
||||
that directory remain available, while descendants beneath it stay unknown.
|
||||
A basename that is not valid UTF-8 is never looked up through replacement
|
||||
characters. It consumes entry capacity and records an `unrepresentable_name`
|
||||
omission at its containing directory, with exact basename bytes in
|
||||
`raw_name_hex` and a null `system_code`. That root's enumeration remains
|
||||
incomplete, so missing paths beneath it remain unknown.
|
||||
Each retained regular file without a digest has an aliased path, size,
|
||||
remaining budget and reason: `file_bytes_budget`,
|
||||
`file_changed_or_short_read`, or `file_unavailable`. Its `system_code` is
|
||||
@@ -652,33 +618,6 @@ that directory: it outlives each run, so the first compilation into a new
|
||||
directory pays a cold-cache build and later runs reuse it. An absent or empty
|
||||
value keeps the shared default.
|
||||
|
||||
With the default `xcrun` compiler and module cache, REA also keeps one verified
|
||||
copy of the compiled helper in `~/Library/Caches/rea/process-helper/`, so later
|
||||
REA processes skip both the compile and macOS's first-launch assessment of a
|
||||
new executable. Each entry is keyed by the helper source, the `swiftc` and SDK
|
||||
that `xcrun --find swiftc` and `xcrun --show-sdk-path` resolve (their real
|
||||
paths, sizes and modification times, so `DEVELOPER_DIR`, `TOOLCHAINS`,
|
||||
`xcode-select` and in-place toolchain updates all select a new entry), the CPU
|
||||
architecture and the Darwin release. Before running an entry, REA requires the
|
||||
cache directory, the entry and its files to be owned by the current user and
|
||||
not writable by group or others, every real parent directory to be owned by
|
||||
the current user or root and not writable by others (sticky root-owned
|
||||
directories excepted), no ACL on any of them to allow another principal a
|
||||
write-class right (the default `deny delete` entries are fine), the helper and
|
||||
manifest to be regular files rather than
|
||||
links or FIFOs, and the helper's SHA-256 to match the manifest. Anything else is
|
||||
never executed and never removed, because another REA process may still be
|
||||
verifying or running that path; REA then compiles a per-process helper as
|
||||
before. A published entry is never replaced. If the cache is unavailable, for
|
||||
example under a read-only home directory, the ownership operation proceeds
|
||||
with the per-process compile. These checks reject corruption, loosened
|
||||
permissions and other users, but they do not defend against another process
|
||||
running as the same user, which could already modify REA itself. Setting
|
||||
`REA_DARWIN_SWIFTC_PATH` or `REA_DARWIN_MODULE_CACHE_PATH` keeps the per-process
|
||||
compile and writes nothing to this cache. REA does not prune old entries; each
|
||||
is about 240 KiB. Delete the directory while no REA process is running, or run
|
||||
`rea uninstall --purge-data`, to remove them.
|
||||
|
||||
`xcrun` also maintains a separate tool-lookup cache. Its `--no-cache` option
|
||||
refreshes entries and does not guarantee a write-free lookup. To avoid that
|
||||
lookup while compiling REA's helper, set `REA_DARWIN_SWIFTC_PATH` to the absolute
|
||||
@@ -724,11 +663,3 @@ REA then preserves ownership uncertainty and can report `cleanup_incomplete`;
|
||||
it does not disable verification to continue. A sandbox that explicitly allows
|
||||
this read-only process inspector is a separate prerequisite from the compiler
|
||||
and cache settings.
|
||||
|
||||
Symlink observations preserve their stored target bytes in `symlink_target_hex`
|
||||
(lowercase hexadecimal). `symlink_target` retains valid UTF-8 text and is `null`
|
||||
when the target bytes cannot be decoded as UTF-8. Filesystem effects compare
|
||||
these exact bytes without resolving the link. Older captures without the hex
|
||||
field still parse; a legacy target containing U+FFFD has ambiguous byte identity,
|
||||
so matching target text alone produces an `unknown` effect. Symlink descendants
|
||||
remain outside filesystem observation regardless of target encoding.
|
||||
+5
-9
@@ -10,28 +10,24 @@ inferred from a `.bin` suffix or from unrecognized bytes.
|
||||
## Open and inspect
|
||||
|
||||
```bash
|
||||
rea inspect /absolute/path/to/kernel.img --target-format raw-binary --architecture arm --entry-point 0x8000 --provider ghidra --json
|
||||
rea function /absolute/path/to/kernel.img 0x8000 --target-format raw-binary --architecture arm --entry-point 0x8000 --provider ghidra --json
|
||||
rea inspect-native-load-image /absolute/path/to/kernel.img --target-format raw-binary --architecture arm --entry-point 0x8000 --provider ghidra --json
|
||||
rea inspect /absolute/path/to/ecu.bin --target-format raw-binary --architecture arm --entry-point 0x8000000 --provider ghidra --json
|
||||
rea function /absolute/path/to/ecu.bin 0x8000000 --target-format raw-binary --architecture arm --entry-point 0x8000000 --provider ghidra --json
|
||||
rea inspect-native-load-image /absolute/path/to/ecu.bin --target-format raw-binary --architecture arm --entry-point 0x8000000 --provider ghidra --json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "open_binary",
|
||||
"arguments": {
|
||||
"path": "/absolute/path/to/kernel.img",
|
||||
"path": "/absolute/path/to/ecu.bin",
|
||||
"format": "raw-binary",
|
||||
"architecture": "arm",
|
||||
"entry_point": "0x8000",
|
||||
"entry_point": "0x8000000",
|
||||
"provider_id": "ghidra"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The example maps a 32-bit ARM-state image at `0x8000`. For `arm`, decoding
|
||||
starts at the entry in ARM (A32) state, so Thumb-only images, including Cortex-M
|
||||
firmware, are not supported. See [Limitations](#limitations).
|
||||
|
||||
`architecture` is one of `x86`, `x86_64`, `arm`, `arm64`, `6502`, or `m68k`. `entry_point` is a
|
||||
hexadecimal address of the form `0x…`. The complete file is mapped starting at
|
||||
that address, and Ghidra seeds an `entry` function there before auto-analysis.
|
||||
|
||||
Loaded 100 of 3527 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user