mirror of
https://github.com/morluto/rea.git
synced 2026-11-08 08:11:31 -10:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8465514782 | ||
|
|
08e8501d7b | ||
|
|
95eb175457 | ||
|
|
1cd471487b | ||
|
|
473bcdd204 | ||
|
|
48a1312237 |
No files matched your search
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"name": "rea",
|
||||
"interface": {
|
||||
"displayName": "REA"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "rea",
|
||||
"source": {
|
||||
"source": "npm",
|
||||
"package": "rea-agents",
|
||||
"version": "6.8.0"
|
||||
},
|
||||
"policy": {
|
||||
"installation": "AVAILABLE",
|
||||
"authentication": "ON_INSTALL"
|
||||
},
|
||||
"category": "Developer Tools"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -15,11 +15,6 @@ Follow the checkout's [performance priorities](../../../AGENTS.md#performance),
|
||||
workflow, not additional permission to execute targets, publish, or merge.
|
||||
Use the current contracts, provider prerequisites, and user authorization.
|
||||
|
||||
Reserve performance measurements for changes that target performance or have a
|
||||
concrete regression concern. Routine cleanup uses correctness checks and does
|
||||
not require this optimization workflow or synthetic benchmarks merely because
|
||||
helper calls, imports, or representations change.
|
||||
|
||||
Read [the Kuna case study](references/kuna-case-study.md) when selecting a
|
||||
reuse strategy or interpreting a benchmark. Its pinned observations illustrate
|
||||
tradeoffs; they are not current-provider guarantees or universal speedups.
|
||||
|
||||
@@ -68,10 +68,9 @@ closing-issue reporters. Grouping preserves all those credits in one line.
|
||||
When Highlights, detailed entries or migrations reference the same work,
|
||||
put its thanks in one of those entries, not all of them. Include issue links.
|
||||
Direct-commit bullets carry the commit link and known human credit.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic, Cursor,
|
||||
Codex, and Copilot co-author trailers resolve to `@claude`, `@cursoragent`,
|
||||
`@codex`, and `@Copilot` with GitHub type `User`; the credit policy excludes
|
||||
them while retaining their metadata.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic, Cursor
|
||||
and Codex co-author trailers resolve to `@claude`, `@cursoragent` and `@codex`
|
||||
with GitHub type `User`; the credit policy excludes them while retaining their metadata.
|
||||
At the REA maintainer's request, omit `@morluto` self-thanks while preserving
|
||||
authorship in the inventory. Other human contributors remain eligible.
|
||||
Use explicit GitHub links for cross-repository references to avoid ambiguous
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: rea-tool-design
|
||||
description: Audit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
|
||||
description: Design or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
|
||||
---
|
||||
|
||||
# REA tool design
|
||||
@@ -10,20 +10,6 @@ contract semantics, provider ownership, and discoverability. Inspect the current
|
||||
contract and nearest existing tool relevant to the analyst question; source,
|
||||
documentation, and live catalogs may describe different versions.
|
||||
|
||||
For provider-backed work, exercise the analyst's task through REA and compare
|
||||
relevant native provider and independent artifact observations where available.
|
||||
Check applicable partial results, recovery and session boundaries; report what
|
||||
was not exercised. For failure guidance, follow [design recovery around the
|
||||
failure](../../../docs/tool-design.md#design-recovery-around-the-failure).
|
||||
Use [workflow verification](../../../docs/testing.md#provider-backed-workflow-verification)
|
||||
to select evidence for the proposed contract instead of seeking provider API
|
||||
parity.
|
||||
|
||||
For an audit, follow [audit existing tools](../../../docs/tool-design.md#audit-existing-tools):
|
||||
scope representative tasks, exercise the live workflow, and distinguish
|
||||
reproduced bugs, supported limitations, design opportunities and unverified
|
||||
claims. Report working alternatives and evidence limits as well as findings.
|
||||
|
||||
For a design request, return the proposed boundary, input/result shape, Evidence
|
||||
semantics, nearest alternative, and verification approach. Implement when
|
||||
requested, preserving the user's scope and existing authorization.
|
||||
|
||||
@@ -1,14 +1,14 @@
|
||||
---
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "59"
|
||||
version: "45"
|
||||
---
|
||||
|
||||
# REA
|
||||
|
||||
Use REA when a claim depends on a shipped binary or package, decompilation,
|
||||
requested runtime evidence, or comparison with
|
||||
passive application runtime evidence, or comparison with
|
||||
behavior not established by available source. For ordinary analysis of a
|
||||
complete source repository, use normal repository tools and do not run REA
|
||||
readiness or provider commands.
|
||||
@@ -17,9 +17,6 @@ readiness or provider commands.
|
||||
|
||||
Installing this skill supplies instructions; it does not register the REA MCP
|
||||
server, install analysis engines, or add tools to an already-running agent.
|
||||
The exception is a Claude Code or Codex plugin install: the plugin bundles
|
||||
this skill and registers its own `rea` server, outside the state that
|
||||
`rea setup` and `rea doctor` manage.
|
||||
|
||||
If REA tools are available and their registration is not known to be stale,
|
||||
proceed directly to the target. Do not run diagnostics or setup before every
|
||||
@@ -40,23 +37,43 @@ repeat or known not to have started. If it may already have changed the
|
||||
target, or it fails again, tell the user and ask them to restart the MCP
|
||||
client. If REA could not start its connection, retry once, then tell the user
|
||||
and ask them to restart the MCP client. Do not retry a startup failure that
|
||||
returns the same unchanged result. For plugin installs, follow
|
||||
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin).
|
||||
For setup-owned registrations, run `rea doctor` when REA could not start
|
||||
returns the same unchanged result. Run `rea doctor` when REA could not start
|
||||
or tools stay missing after that restart. For `cleanup_incomplete`, a retryable
|
||||
capacity limit, or `configuration_invalid`, follow the recovery guide's ladder
|
||||
before asking for a restart. Read that guide before a second repair attempt.
|
||||
|
||||
Choose the installation owner before repairing registration. Plugin installs use
|
||||
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin);
|
||||
only setup-owned registrations use the guide's
|
||||
[registration repair plan](references/connection-and-recovery.md#repair-registration-when-needed).
|
||||
A plugin-only doctor result of `missing` does not call for `rea setup`.
|
||||
1. Diagnose without changing REA or client configuration. `npx` may download
|
||||
the package and write to npm's cache. For Codex, run
|
||||
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
|
||||
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
|
||||
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
|
||||
`commandcode`, `vscode`, `grok_build`, `omp`, or `grok_bot`. `grok_bot` has no local
|
||||
file registration: doctor reports the Grok Bot chat step, and setup does not
|
||||
write `mcp.json`. If the client is unknown, use `doctor --json` and
|
||||
inspect its registration results before choosing a setup scope.
|
||||
2. Distinguish the reason. Missing, malformed, or stale registration needs a
|
||||
scoped configuration repair. An aligned registration with no tools in the
|
||||
active session needs a restart/reconnection; doctor cannot prove that the
|
||||
current agent has connected. A missing provider affects only tasks requiring
|
||||
that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
|
||||
and Android inspection has separate bring-your-own JADX/Java prerequisites.
|
||||
3. For a configuration repair, prepare the read-only plan:
|
||||
`npx -y rea-agents@latest setup --client codex --dry-run --json`.
|
||||
Use the current client's ID, show its exact proposed paths, backups, and
|
||||
changes, and obtain approval before setup writes configuration or installs
|
||||
Hopper. Setup normally installs the matching bundled skill too; include that
|
||||
replacement in the reviewed plan. After approval, apply the same scope with
|
||||
`npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
|
||||
only if that separate installation was needed and explicitly approved.
|
||||
4. Restart/reconnect the affected agent. Verify that REA tools actually appear
|
||||
in the session, then resume the original investigation. If they remain
|
||||
absent, inspect the client's MCP launch error rather than repeating setup.
|
||||
|
||||
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
|
||||
JADX, Kuna, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
|
||||
software to repair MCP registration. For an unsupported client, use manual stdio
|
||||
registration with an existing installation’s absolute runtime and entry script, or continue through the CLI.
|
||||
software to repair
|
||||
MCP registration. For an unsupported client, use manual stdio registration
|
||||
with a version-pinned `rea-agents` package or continue through the CLI.
|
||||
|
||||
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
|
||||
|
||||
@@ -69,13 +86,11 @@ graph, limitations, and unknowns with the same care as an MCP result; the CLI
|
||||
returns the Evidence record directly. This fallback does not establish that
|
||||
MCP is configured. Native CLI tasks still require their selected engine.
|
||||
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
|
||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place from a
|
||||
trusted directory outside the target project, with absolute target paths. Managed
|
||||
setup binds the current installation directly; retain its runtime and package
|
||||
cache entry, or restore that installation before repairing its registration.
|
||||
For setup-owned registration, configuration repair starts with a read-only setup plan; show paths, backups and
|
||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
|
||||
Configuration repair starts with a read-only setup plan; show paths, backups and
|
||||
skill replacement before applying authorized changes. Preserve existing approval
|
||||
and target-execution boundaries.
|
||||
and target-execution boundaries. REA setup never installs or upgrades unrelated
|
||||
software, including Node.js, npm, Homebrew, Java, Ghidra, IDA, JADX, Kuna, Binwalk or Unblob.
|
||||
|
||||
## Route the target first
|
||||
|
||||
@@ -94,9 +109,6 @@ explicit path or endpoint and do not need it.
|
||||
than extra calls and schemas. Runtime options are metadata, not files.
|
||||
Compact delivery still inventories and hashes members. See the
|
||||
[Python packaged applications guide](https://github.com/morluto/rea/blob/main/docs/python-packaged-applications.md).
|
||||
- Bun `--compile` executable: `open_binary`, then `inspect_artifact`; the Bun
|
||||
module graph is the application layer. Extract stored modules and analyze the
|
||||
JavaScript. See [Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
|
||||
`open_bundle_resources`, then the selected resource tool. This does not
|
||||
require `CFBundleExecutable` or a deep provider. Resource sessions have no
|
||||
@@ -109,25 +121,14 @@ explicit path or endpoint and do not need it.
|
||||
`extract_firmware` when extraction is requested, with a caller-selected new
|
||||
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
|
||||
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
|
||||
- ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact`
|
||||
without a provider; see the ESP section of `references/native-and-artifacts.md`.
|
||||
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
|
||||
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
|
||||
optional metadata recovery and its supported host/layout boundary.
|
||||
- Native program behavior, live calls, instrumentation, or command-run effects:
|
||||
choose the matching [native runtime workflow](references/runtime-observation.md#native-runtime-questions).
|
||||
`open_binary` and `trace_native_values` establish static facts; they do not
|
||||
observe execution. Runtime tools have distinct launch, attach, mutation and
|
||||
cleanup effects, and do not require a preliminary static session unless their
|
||||
own contract says so.
|
||||
- User-owned browser page already open: `list_browser_targets`.
|
||||
- Browser action the agent must perform (click, type, navigate):
|
||||
`capture_browser_scenario` with declared actions. See
|
||||
[runtime observation](references/runtime-observation.md) for connect/launch
|
||||
and the focused follow-up browser tools.
|
||||
- Agent environment (AgentEnv bundle) of a web application: read
|
||||
[environments](references/environments.md) before recording; it records with
|
||||
`capture_browser_scenario` and writes the bundle with `export_environment_bundle`.
|
||||
- Fiddler Everywhere traffic or a native `.saz` capture: read the
|
||||
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
|
||||
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
|
||||
@@ -150,9 +151,7 @@ explicit path or endpoint and do not need it.
|
||||
and field names/types before choosing fields for `inspect_packet_capture`.
|
||||
Query/page the relevant records; registrations do not establish observed traffic.
|
||||
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and
|
||||
TShark 4.x on an absolute `PATH` entry or via the optional absolute
|
||||
`REA_TSHARK_COMMAND` override; see
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
|
||||
Packet limits count input packets before filtering. Output defaults to 50 matching
|
||||
frames; use `match_offset` / `match_limit` for later pages, or explicit null
|
||||
@@ -248,46 +247,12 @@ explicit path or endpoint and do not need it.
|
||||
absence of that table, not proof the image is unsigned in every catalog sense.
|
||||
Signatures are not verified. See the
|
||||
[PE certificate guide](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
- PE import and delay-load tables: `inspect_pe_imports` with the explicit
|
||||
local PE32/PE32+ file. This built-in target-free inspection requires no
|
||||
Windows SDK. Preserve DLL names, named and ordinal functions, ILT/IAT RVAs,
|
||||
delay-load descriptors, and the pefile import hash of the standard import
|
||||
table. A missing import directory is observed absence of that table, not
|
||||
proof the image never imported APIs through another table. Bound snapshots
|
||||
and loaded IAT contents remain unobserved. See the
|
||||
[PE import guide](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
|
||||
- PE TLS callbacks and load configuration: `inspect_pe_load_config` with the
|
||||
explicit local PE32/PE32+ file. This built-in target-free inspection requires
|
||||
no Windows SDK. Preserve callback VAs as strings with their RVA, file offset
|
||||
and section, and the size-covered security cookie, SafeSEH, CFG,
|
||||
EH-continuation and XFG fields. Guard-flag mitigation claims are inferred
|
||||
linker declarations, not proof of enforcement. A missing directory is observed
|
||||
absence of that declaration. See the
|
||||
[PE load configuration guide](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
|
||||
- ELF note tables: `inspect_elf_notes` with the explicit local ELF32/ELF64
|
||||
file. This built-in target-free inspection requires no disassembler. Preserve
|
||||
PT_NOTE and SHT_NOTE ranges, owners, types, descriptor SHA-256, and GNU
|
||||
build-id or ABI-tag fields. A missing table is observed absence of those
|
||||
declarations, not proof the image never carried notes. See the
|
||||
[ELF notes guide](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
- Apple `.ips` crash reports: `inspect_apple_ips` with the explicit local
|
||||
file. This built-in target-free inspection requires no Xcode, atos, or
|
||||
pwntools. Preserve first-line metadata, `bug_type` as a string, process and
|
||||
exception facts, lossless integer registers, usedImages UUIDs, and
|
||||
documented classification matches. Only `bug_type` `"309"` yields a crash
|
||||
body; other types, including `"288"` stackshots, return no crash result.
|
||||
Opened Mach-O and dSYM UUID joins remain unobserved. See the
|
||||
[Apple IPS guide](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
|
||||
- Linux kernel module metadata: `inspect_kernel_module` with the explicit local
|
||||
ELF32/ELF64 relocatable file. This built-in target-free inspection requires
|
||||
no Linux host or `modinfo`. Preserve `.modinfo` key order and duplicate keys,
|
||||
`__versions` CRC hex, section file ranges, and the appended signature
|
||||
trailer's PKCS#7 SHA-256. A missing section is observed absence when section
|
||||
names are known, not proof the module never carried that metadata.
|
||||
`struct module` is not decoded. The signature is not validated. gzip, xz, and
|
||||
zstd containers are unsupported; decompress into a separate file with an
|
||||
external codec utility, then inspect the decompressed `ET_REL` image. See the
|
||||
[kernel module guide](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
|
||||
- CPython 3.7–3.13 `.pyc` header, nested marshal code objects and bytecode
|
||||
identity: `inspect_python_pyc` with the explicit local file. This built-in
|
||||
target-free inspection requires no Python installation. Preserve magic,
|
||||
@@ -298,10 +263,8 @@ explicit path or endpoint and do not need it.
|
||||
- Erlang BEAM IFF container: `inspect_erlang_beam` with the explicit local
|
||||
`.beam` file. This built-in target-free inspection requires no Erlang
|
||||
installation. Preserve chunk names, atom order, export/import/lambda tables,
|
||||
and the Code header. For selected facts, prefer `detail: "summary"` plus a
|
||||
`beam` view (collection, names/indices, fields and paging), retaining the full
|
||||
Evidence. CLI summary requires `--evidence_output`. Missing table counts stay
|
||||
null; decoded empty tables count zero. Bytecode is not executed. See the
|
||||
and the Code header. Missing optional tables stay null. Bytecode is not
|
||||
executed. See the
|
||||
[Erlang BEAM guide](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
|
||||
|
||||
- Explicit local SQLite database snapshot: `inspect_sqlite_database`. Inspect
|
||||
@@ -338,11 +301,6 @@ leaves its database open. The modern headless profile creates and closes an
|
||||
owned private database without saving. Installation and registration reuse
|
||||
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
|
||||
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
|
||||
For migration to the separate native IDA MCP plugin, follow the guide's
|
||||
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/ida-provider.md#migrate-a-legacy-attached-registration).
|
||||
Replace the upstream registration explicitly; modern mrexodia tools are not the
|
||||
native profile. Native attachment preserves the database but does not provide
|
||||
direct callers; check session availability and the guide's verification limits.
|
||||
Proceed directly to `analyze_function`, pseudocode, or function/string searches
|
||||
after opening; IDA does not supply `binary_overview`. Consult session availability
|
||||
when composing broader workflows. Modern direct callers and unsupported dossier
|
||||
@@ -373,24 +331,18 @@ native target whose analysis tool operates on the active target. Do not call an
|
||||
unadvertised tool; inspect `binary_session` with `{}` and its
|
||||
`result.tool_availability` for availability reasons and remediation. Expand each
|
||||
group's `names` into entries, inheriting the inventory-level client requirements
|
||||
unless the group overrides them. The grouped name union is the complete tool
|
||||
inventory; use `tools/list` for canonical tool schemas. The default summary keeps
|
||||
each provider candidate's availability and target-support status, code and
|
||||
reason; `omitted_fields` lists what it leaves out. Request `{"detail": "complete"}`
|
||||
only when you need an unavailable candidate's remediation, provider
|
||||
diagnostics, candidate capability sets, or tool surface, effect and annotation
|
||||
flags; in those closed flag vocabularies, listed
|
||||
flags are true and omitted known flags are false. Provider candidate facts
|
||||
describe possible providers, while top-level session capabilities describe the
|
||||
current route and `analysis_provider_binding` names the selected provider. Do
|
||||
not treat those scopes as interchangeable.
|
||||
unless the group overrides them. In its closed effect and annotation vocabularies,
|
||||
listed flags are true and omitted known flags are false. The grouped name union
|
||||
is the complete tool inventory; use `tools/list` for canonical tool schemas.
|
||||
Provider candidate facts describe possible providers, while top-level session
|
||||
capabilities describe the current route and `analysis_provider_binding` names the
|
||||
selected provider. Do not treat those scopes as interchangeable.
|
||||
|
||||
For a Ghidra startup timeout on a large native target, follow
|
||||
[deadline recovery](references/native-and-artifacts.md#ghidra-startup-deadline-recovery)
|
||||
before abandoning deep analysis or repeating the same cold import. The default
|
||||
330 seconds can be changed for new startup attempts through
|
||||
`update_analysis_settings` using `ghidra.startup_timeout_ms`. Changing the
|
||||
server's launch environment instead requires a new REA process.
|
||||
330 seconds is configurable; changing the server's environment requires a new
|
||||
REA process.
|
||||
|
||||
## Work summary-first
|
||||
|
||||
@@ -398,22 +350,7 @@ Start with the default result and use its inline Evidence and graph context.
|
||||
For a JavaScript application or ELF layout whose size is unknown or large,
|
||||
call `analyze_javascript_application` or `inspect_binary_layout` with
|
||||
`"detail": "summary"`, then inspect the retained `parent_evidence_id` with
|
||||
`inspect_analysis_view`. Over MCP, `analyze_function` is summary-first without
|
||||
a flag: its default reply is the summary, and the complete dossier stays
|
||||
retained under `parent_evidence_id` for facet paging. Pass that ID as a
|
||||
`retained-evidence` reference to `compare_functions`, or inside
|
||||
`match_native_functions` `left_observations`/`right_observations` or
|
||||
`build_call_path.functions`; request `"detail": "complete"` only when the
|
||||
dossier must travel inline.
|
||||
Use the same exact-reference form in other advertised logical Evidence inputs,
|
||||
including optional sources, arrays and nested sources across domains. Each
|
||||
source can independently be inline or retained; preserve its source identity,
|
||||
coverage, partial facts and unknowns. A summary or inspection page has its own
|
||||
Evidence ID: use `normalized_result.parent_evidence_id` when the consumer needs
|
||||
the complete source. For portable CLI follow-ups, commands advertising
|
||||
`--evidence-bundle` resolve references from that selected exported bundle.
|
||||
See [Evidence composition](references/evidence-workflows.md) for scope and
|
||||
recovery; file selectors are supported only where explicitly advertised.
|
||||
`inspect_analysis_view`.
|
||||
Page module identities and select the returned `node_id` before requesting an
|
||||
item; a single file path can represent several graph roles. Do not repeat an
|
||||
identical analysis or rejected call without a changed input or resolved blocker.
|
||||
@@ -432,10 +369,7 @@ output also streams without a single MCP frame.
|
||||
When the retained parent ID is unknown, use `get_evidence_bundle` with
|
||||
`detail: "summary"` and exact filters for operation, target SHA-256,
|
||||
analysis-profile digest or canonical procedure address. Select its returned
|
||||
Evidence ID for the next view instead of requesting the complete bundle. When
|
||||
that record has no supported view (for example an annotation), read it with
|
||||
`detail: "records"` and an `evidence_id` filter; `missing_evidence_links`
|
||||
names linked sources this session does not retain.
|
||||
Evidence ID for the next view instead of requesting the complete bundle.
|
||||
`retention: "complete-record"` means the whole record is stored; native
|
||||
value-flow truncation and unknown metadata remain separate. For an exported
|
||||
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
|
||||
@@ -504,22 +438,15 @@ as complete while required questions remain open.
|
||||
[references/javascript-applications.md](references/javascript-applications.md)
|
||||
- Android APK declarations, classes, methods, and static references:
|
||||
[references/android-applications.md](references/android-applications.md)
|
||||
- Native instrumentation, debugger and process capture; browser/Electron
|
||||
observation; static/runtime comparison:
|
||||
- Browser/Electron observation (passive and scenario) and static/runtime
|
||||
reconciliation:
|
||||
[references/runtime-observation.md](references/runtime-observation.md)
|
||||
- Evidence paging, comparisons, residual unknowns, and verification:
|
||||
[references/evidence-workflows.md](references/evidence-workflows.md)
|
||||
- Agent environments (AgentEnv bundles) of a web application, recorded from
|
||||
a live instance or a HAR:
|
||||
[references/environments.md](references/environments.md)
|
||||
|
||||
## Finish the task
|
||||
|
||||
Explain findings in plain language and tie them to returned evidence. When the
|
||||
user asks to build something, use normal coding tools and separate observed
|
||||
behavior from design choices. Close an opened native session with
|
||||
`close_binary` when its engine is no longer needed. Evidence references and Unknown
|
||||
history remain in memory on the same MCP connection. End the investigation with
|
||||
`reset_investigation`; supply `save_path` to checkpoint the complete bundle before
|
||||
clearing. Import the saved bundle to restore original references and Unknown
|
||||
history in a new process. Retention is bounded and is not automatic persistence.
|
||||
`close_binary` when the investigation is complete.
|
||||
@@ -13,34 +13,11 @@ own APK path and explicit selectors; do not open a native session first.
|
||||
Choose an `overload_index` from the class inventory when methods are ambiguous
|
||||
instead of guessing.
|
||||
|
||||
For JNI implementations, discover `inspect_android_jni_bindings`. Select raw
|
||||
DEX class/method identities and optional descriptors, ABI or logical library
|
||||
paths. Prefer `detail: "summary"`; complete Evidence remains retained and
|
||||
`android-jni` views page declarations, libraries, candidates or binding rows
|
||||
with exact fields. Global native-pair ambiguity survives class selection.
|
||||
Preserve slot/relocation provenance and source coverage: matched means observed
|
||||
candidate presence; partial scans do not establish uniqueness or absence.
|
||||
This reads DEX/ELF bytes without loading the APK into the decompiler or executing
|
||||
target code. Runtime registration and loading remain unknown. See the
|
||||
[JNI guide](https://github.com/morluto/rea/blob/main/docs/android-analysis.md#link-native-declarations-to-jni-candidates).
|
||||
|
||||
For an R8/ProGuard release build with its `mapping.txt`, pass the absolute
|
||||
`mapping_path` to `inspect_android_class` or `inspect_android_method`. Prefer
|
||||
`detail: "summary"` to receive normalized symbols or the selected body without
|
||||
duplicate raw class metadata. Complete Evidence stays retained under the linked
|
||||
parent ID and can be exported; CLI summary delivery requires `--evidence_output`.
|
||||
Omitted or complete detail preserves the full inline response. Keep
|
||||
selecting by observed names; the separate `mapping` block reports each symbol
|
||||
as `mapped`, `ambiguous` or `unmapped` after checking DEX types and prototypes.
|
||||
REA does not verify that the mapping belongs to the APK build.
|
||||
|
||||
The provider requires a separately supplied headless JADX MCP JAR 0.7.1 and
|
||||
Java 17 or newer. `REA_JADX_MCP_JAR` selects the absolute JAR path; `JAVA_HOME`
|
||||
can select Java. Use `rea doctor --provider jadx --json` to check the JADX
|
||||
configuration and obtain remediation for missing prerequisites. Java and the
|
||||
headless JADX MCP JAR remain caller-supplied; REA setup does not install or
|
||||
configure them. Configuration readiness does not establish successful analysis
|
||||
of a particular APK.
|
||||
can select Java. REA setup does not install or configure that JAR, and doctor
|
||||
provider scopes cover Hopper/Ghidra rather than JADX. Use existing tools and
|
||||
report missing prerequisites for the Android task specifically.
|
||||
|
||||
Linux and macOS use the POSIX owned-process boundary. Windows x64 requires the
|
||||
matching bundled native controls and uses an owned Job Object with bidirectional
|
||||
@@ -49,11 +26,8 @@ stdio. A full JDK, including `jdk.compiler`, is required on every host; Windows
|
||||
verification coverage. Calls use one worker and provider deadlines; optional
|
||||
`REA_JADX_HEAP_MIB` selects the JVM heap and `REA_JADX_OPERATION_TIMEOUT_MS`
|
||||
the per-operation deadline (integer milliseconds from 1 to 2147483647, default
|
||||
120000). To change that deadline on a running MCP server, set
|
||||
`jadx.operation_timeout_ms` through `update_analysis_settings`, then retry after
|
||||
work and cleanup settle. Newly accepted operations use the override; accepted
|
||||
work keeps its budget. Changing the launch environment requires a new process.
|
||||
The MCP client's tool timeout must also allow the selected deadline. The APK is never launched,
|
||||
120000; a change applies on the next process start; the MCP client's tool
|
||||
timeout must also allow it). The APK is never launched,
|
||||
and no emulator is required.
|
||||
|
||||
Manifest entries and decompiled text are static observations, not execution or
|
||||
|
||||
@@ -7,22 +7,18 @@ operation and the next useful step; after repair, continue the investigation.
|
||||
Commands below use an installed `rea`. If none is available, use the package
|
||||
invocation in `SKILL.md` when acquiring REA is authorized. Follow the connected
|
||||
server's schemas and the installed CLI's help rather than assuming repository
|
||||
main and an older release expose the same arguments. Tool-name prefixes belong
|
||||
to the client: find the canonical REA operation in its actual tool listing and
|
||||
server association before concluding that it is absent. A shared prefix alone
|
||||
does not establish which server an approval wildcard covers. See
|
||||
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/client-configuration.md#client-tool-names-and-discovery).
|
||||
main and an older release expose the same arguments.
|
||||
|
||||
## Choose the remedy from the observed failure
|
||||
|
||||
| Observation | Meaning and next step |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| No REA tools in the active session | If REA was installed as a client plugin, follow the plugin section below. Otherwise inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| MCP connection dropped or REA could not start | If the connection drops, retry that call once only when it is safe to repeat or known not to have started. If it may already have changed the target, or the same failure returns, tell the user REA's connection failed and ask them to restart the MCP client. If REA could not start its connection, retry once, then tell the user and ask them to restart the MCP client. Do not retry a startup failure that returns the same unchanged result. Run `rea doctor` when REA could not start or tools stay missing after restart. Doctor cannot restore a live connection by itself. |
|
||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. Request `{"detail": "complete"}` only for provider diagnostics. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. |
|
||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
||||
|
||||
If the client cannot launch its shell/sandbox helper or read the installed skill,
|
||||
@@ -116,32 +112,6 @@ is not a recommendation to change it. Do not invent a setting. Host memory and
|
||||
- `invalid_request`: correct the reported fields from the current schema. Do
|
||||
not repeat the unchanged call.
|
||||
|
||||
## REA installed as a client plugin
|
||||
|
||||
A Claude Code or Codex plugin owns its `rea` server and this skill. Doctor's
|
||||
`--client` and `--skill` checks and `rea setup` read and write only
|
||||
setup-owned state, so for a plugin-only install they report `missing`, and
|
||||
applying setup would add a second `rea` server. If this skill was loaded from
|
||||
a plugin, or the client lists a `rea` plugin, do not prepare a setup plan for
|
||||
missing tools. Instead:
|
||||
|
||||
- Check that the plugin is installed and enabled with the client's own
|
||||
listing: `claude plugin list` and `/mcp` in Claude Code, `codex plugin list`
|
||||
in Codex.
|
||||
- Restart or reconnect the client, then read its launch error for the `rea`
|
||||
server if tools are still absent. The plugin starts its bundled runtime; a missing build or version mismatch
|
||||
requires a reinstall, or a fresh build when developing from source.
|
||||
- Check analysis engines through the connected server's
|
||||
`binary_session.result.tool_availability`. `rea doctor --provider <id> --json`
|
||||
from a shell reports that shell's environment, which can differ from the
|
||||
environment the client gave the plugin server. If you still run it through
|
||||
the package runner, use the version pinned in the plugin manifest's server
|
||||
entry (`rea-agents@<version>`), not `@latest`, so it matches the server the
|
||||
plugin starts.
|
||||
|
||||
Use the setup repair below only when the user wants a setup-owned registration
|
||||
instead of the plugin, and remove or disable the plugin first.
|
||||
|
||||
## Repair registration when needed
|
||||
|
||||
For Codex:
|
||||
@@ -154,12 +124,9 @@ rea setup --client codex --dry-run --json
|
||||
Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
|
||||
`cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`,
|
||||
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `trae`, `grok_build`, `omp`, `pi`,
|
||||
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, `freebuff`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
||||
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
||||
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
|
||||
For unsupported clients, use manual stdio registration with the selected
|
||||
installation’s absolute runtime and entry script, or continue through the CLI.
|
||||
Acquire packages from a trusted directory outside the target project. Keep a
|
||||
package-runner installation and its runtime available while registrations use it.
|
||||
For unsupported clients, use version-pinned manual stdio registration or CLI.
|
||||
|
||||
Show the plan's exact paths, backups, changes and bundled-skill replacement.
|
||||
Obtain approval for writes when that concrete scope is not already authorized,
|
||||
@@ -204,17 +171,6 @@ writing. Managed policy can constrain client settings. Consult the
|
||||
for the installed client's supported settings. This is an optional recovery
|
||||
choice, not a prerequisite for every REA task.
|
||||
|
||||
## Repair an editable REA limit
|
||||
|
||||
If `details.limit` names an editable setting and its bounds, call
|
||||
`update_analysis_settings` directly with a `changes` entry containing `setting`,
|
||||
`action: "set"` and `value`. `inspect_analysis_settings` is optional when those
|
||||
facts are already present. Retry the same input after work and cleanup settle.
|
||||
Runtime defaults affect new work only; explicit per-call arguments take
|
||||
precedence and must be corrected directly when selected. Reset restores launch
|
||||
values. Updates survive target close/open but disappear on REA restart. They do
|
||||
not edit host deadlines, response buffers, registrations or fixed limits.
|
||||
|
||||
## Continue through an installed CLI
|
||||
|
||||
Check the command environment only if the launcher fails:
|
||||
|
||||
@@ -1,287 +0,0 @@
|
||||
# Agent environments from a live application
|
||||
|
||||
Use this when the user asks for an agent environment (an AgentEnv bundle) of a
|
||||
web application: a folder `agent-env` deploys, whose MCP tools answer from what
|
||||
one recording of the application observed and serve the writes that recording
|
||||
confirmed. The workflow records the application once with
|
||||
`capture_browser_scenario`, checks what the environment would serve, records
|
||||
again if a hint names a gap, then writes the bundle with
|
||||
`export_environment_bundle`. For a HAR the user already has, skip to
|
||||
[export](#export-and-hand-off) through `inspect_web_network_capture`, or use
|
||||
`rea environment-build <capture.har> <dir>`.
|
||||
|
||||
## Before recording
|
||||
|
||||
- Record only against an instance the user says is disposable or seeded for
|
||||
this purpose: every write step creates, edits or deletes its data. If the
|
||||
user has not said so, ask before any scenario that writes. REA does not
|
||||
start, seed or reset applications.
|
||||
- Credentials come from environment variables the user names. Declare each as a
|
||||
scenario secret, `{secret_id, environment_variable}`; the variable must be set
|
||||
in the environment REA's MCP server started with. Never ask for a password in
|
||||
chat. Declared secrets are redacted as the capture records.
|
||||
- Map the application first with `inspect_web_page` (and `capture_web_screenshot`
|
||||
when layout matters): the login form's locators, and the API calls behind the
|
||||
pages that list and edit the entities the user asked about.
|
||||
- `inspect_web_page` cannot sign in. To see pages behind sign-in, run a short
|
||||
`capture_browser_scenario` that signs in and opens them, with
|
||||
`"capture": {"after_each_step": ["accessibility"]}` (`"dom"` for markup).
|
||||
Read a step's tree with `inspect_analysis_view`, `view: {kind:
|
||||
"browser-scenario", collection: "text", step_index, artifact:
|
||||
"accessibility", offset: 0, limit: 2048}`, paging with `offset`. Capture
|
||||
options are only `after_each_step`, `at_end`, `events`, `network` and
|
||||
`limits`.
|
||||
|
||||
## Record a session an environment can learn writes from
|
||||
|
||||
One `capture_browser_scenario` records the whole session. Start from this
|
||||
shape, with the application's own locators, paths and headers:
|
||||
|
||||
```json
|
||||
{
|
||||
"browser": { "mode": "launch", "executable_path": "/usr/bin/chromium" },
|
||||
"start_url": { "url": "http://127.0.0.1:3000/login" },
|
||||
"secrets": [
|
||||
{ "secret_id": "admin_password", "environment_variable": "APP_PASSWORD" }
|
||||
],
|
||||
"capture": {
|
||||
"network": {
|
||||
"request_body": true,
|
||||
"response_body": true,
|
||||
"header_values": true
|
||||
}
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"step_id": "user",
|
||||
"action": "fill",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=username]" },
|
||||
"value": { "source": "literal", "value": "admin" }
|
||||
},
|
||||
{
|
||||
"step_id": "password",
|
||||
"action": "fill",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=password]" },
|
||||
"value": { "source": "secret", "secret_id": "admin_password" }
|
||||
},
|
||||
{
|
||||
"step_id": "sign-in",
|
||||
"action": "press",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=password]" },
|
||||
"key": "Enter"
|
||||
},
|
||||
{
|
||||
"step_id": "signed-in",
|
||||
"action": "wait_for",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "nav a[href='/settings']" },
|
||||
"state": "visible"
|
||||
},
|
||||
{
|
||||
"step_id": "folders-1",
|
||||
"action": "request",
|
||||
"timeout_ms": 15000,
|
||||
"method": "GET",
|
||||
"path": "/api/folders",
|
||||
"expected_status": [200]
|
||||
},
|
||||
{
|
||||
"step_id": "new-folder-1",
|
||||
"action": "click",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "role", "role": "button", "name": "New folder" }
|
||||
},
|
||||
{
|
||||
"step_id": "folder-title-1",
|
||||
"action": "fill",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=title]" },
|
||||
"value": { "source": "literal", "value": "Scratch folder 1" }
|
||||
},
|
||||
{
|
||||
"step_id": "create-folder-1",
|
||||
"action": "click",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "role", "role": "button", "name": "Create" },
|
||||
"save_response": {
|
||||
"method": "POST",
|
||||
"path": "/api/folders",
|
||||
"save": [{ "name": "folder_1", "json_pointer": "/id" }]
|
||||
}
|
||||
},
|
||||
{
|
||||
"step_id": "read-folder-1",
|
||||
"action": "request",
|
||||
"timeout_ms": 15000,
|
||||
"method": "GET",
|
||||
"path": "/api/folders/{saved:folder_1}",
|
||||
"expected_status": [200]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Give each step that accepts `timeout_ms` a deadline. Without one, a step waits
|
||||
until it completes or the scenario is cancelled, so a locator that never
|
||||
matches hangs the capture and the client with it. The `wait_for_timeout` pause
|
||||
instead uses `duration_ms` and does not accept `timeout_ms`. Match `wait_for`
|
||||
locators to exactly one element.
|
||||
|
||||
1. Sign in through the UI: `fill` the user and password fields (the password
|
||||
as a secret reference), submit, then `wait_for` an element only a signed-in
|
||||
user sees.
|
||||
2. Read with `request` steps: each collection the environment needs, twice
|
||||
before any write, so fields that change on their own show up; then each
|
||||
object a list shows by id, up to ten per collection. An object known only
|
||||
from a list that leaves out fields its own read shows cannot be read by id
|
||||
or updated in the environment.
|
||||
3. Write the way the application's UI does: fill its forms and click its
|
||||
buttons, so each request carries the fields the UI sends, including ones
|
||||
the server would otherwise derive. Give the click or key press that submits
|
||||
a create `save_response` to keep the new object's ID, then `goto` its page
|
||||
(`/items/{saved:item_1}`) or read it with a `request` step. Never guess an
|
||||
ID, and never create a probe object to learn the next one. Write with
|
||||
`request` steps instead only when the UI cannot reach the operation (an
|
||||
API-only resource, or a UI that does not call the JSON API), or when coverage
|
||||
withholds the UI-shaped write as `unexplained-change` or `contradicted`;
|
||||
that counts as one of the two re-recordings. A `request` step write that
|
||||
coverage withholds that way usually lacks fields the UI sends: record it
|
||||
through the UI instead. If a journey cannot be scripted, leave the write
|
||||
unserved and report the limitation.
|
||||
Record two numbered scratch objects per write type,
|
||||
with varied free text. After each write, and before the next one, read the
|
||||
object back by id and then its list: create 1, read 1, list, create 2,
|
||||
read 2, list; never several writes before their reads.
|
||||
4. Leave more than a second between two creates and between a create and its
|
||||
first edit (`wait_for_timeout` with `duration_ms: 1100`), so lists sorted
|
||||
by creation time show them in order.
|
||||
5. Update both objects and read each back, changing the fields a person
|
||||
would change (a name, an email address, a status, a due date) rather than
|
||||
only a description: a served update accepts only the members a recorded
|
||||
update stored. Run an action that turns something on or off (star,
|
||||
bookmark, archive) twice in a row on one object, reading it back after
|
||||
each: the second call shows whether it sets or flips the field, and the
|
||||
environment serves the action only once a call has shown it. Delete both
|
||||
and read each back (expect 404, or the object marked deleted) and the list.
|
||||
6. Vary what the two creates send, however you write them: the first every
|
||||
field the application takes, the second only what a person must give (its
|
||||
title or name), with the other fields left out or empty. A served create
|
||||
requires each member both creates filled in and lets a caller leave out one
|
||||
a create left out or sent empty, so an agent that names only a title is not
|
||||
refused for fields it had no reason to set.
|
||||
|
||||
About `request` steps:
|
||||
|
||||
- `path` is on the start URL's origin; the page must be on that origin.
|
||||
- Header values are literals, declared secrets, a cookie the browser holds
|
||||
(`{source: "cookie", name}`), a local storage entry
|
||||
(`{source: "local_storage", key, json_pointer}`), a saved value
|
||||
(`{source: "saved", name}`), or Basic credentials
|
||||
(`{source: "basic_credentials", parts: [username, password]}`, sent as the
|
||||
base64 of `username:password` and redacted), with an optional `prefix` such as
|
||||
`"Bearer "` or `"Basic "`. Each credential part is a literal, secret, saved
|
||||
value, cookie or local storage entry, so a header the frontend builds from a
|
||||
login token (the username and the token) can be copied too; the password is
|
||||
a secret or saved value, never a literal. Copy the
|
||||
headers the application's own requests send (a CSRF header such as
|
||||
`X-CSRFToken` from the `csrftoken` cookie, a bearer token). Values read from the page are redacted from records made after they
|
||||
are read, not from the login response that returned them, and values
|
||||
shorter than 8 characters are not redacted at all.
|
||||
- `save: [{name, json_pointer, secret}]` keeps a string or number from the
|
||||
step's JSON response, such as the created object's `/id`, for later steps;
|
||||
`secret: true` redacts it, as for a token the API returns, so mark every
|
||||
saved token, key or session value secret. This also protects the selected
|
||||
source response before its body or headers are retained. Compressed or
|
||||
unknown-size UI responses cannot supply saved values. On a `click` or `press`,
|
||||
`save_response: {method, path, save}` keeps values from the response to the
|
||||
new request observed while the step waits (`{*}` in `path` matches one segment).
|
||||
Matching does not prove UI causality: choose a path distinct from polling;
|
||||
multiple observed matches fail as ambiguous. The step
|
||||
fails unless a 2xx JSON response arrives before its `timeout_ms`. Later
|
||||
steps send a saved value as `{saved:name}` in a request `path` (and its
|
||||
query) or a `goto` URL; in the `json` body, where a string that is only
|
||||
`{saved:name}` keeps the saved JSON type; or as `{source: "saved", name}` in
|
||||
a header or credential part.
|
||||
- Reading a page value makes later bodies costlier to keep: after a CSRF
|
||||
token, bodies over about 47 KiB are kept as `unavailable`. When the API
|
||||
returns larger responses, raise `capture.limits`
|
||||
(`max_network_body_allocation_bytes`, `max_total_event_allocation_bytes`).
|
||||
- Request steps follow no redirects: a redirect fails the step, and
|
||||
`expected_status` cannot name a 3xx status. Use the path the redirect names,
|
||||
such as the one with the trailing slash.
|
||||
- Set `expected_status` on every step, so a refused call fails its step and
|
||||
the later steps report not having run, instead of recording read-backs of
|
||||
nothing.
|
||||
|
||||
Keep it to about 60 to 160 requests. A scenario with no `request` step is
|
||||
recorded exactly as before.
|
||||
|
||||
The capture returns its whole Evidence, every recorded body included, which is
|
||||
often more than a client shows inline. Before checking coverage, confirm that
|
||||
every entry of `normalized_result.steps` has `status: "completed"`; if the
|
||||
client saved the result to a file, search it for `"status":"failed"` and read
|
||||
that step's `error`. A failed sign-in or write cancels every step after it.
|
||||
Use the result's `evidence_id` for coverage and export; do not paste the
|
||||
Evidence back into another tool.
|
||||
|
||||
## Check coverage
|
||||
|
||||
`inspect_analysis_view` with `source: {kind: "retained-evidence",
|
||||
evidence_id}` (the capture's id) and `view: {kind: "environment-coverage",
|
||||
template_rules: ["after-uid-literal", "opaque-token", "sibling-variance",
|
||||
"prefix-variance"]}`. It reports read tools
|
||||
answered from the state or replayed (`reads.recorded_reasons`), the served write
|
||||
tools, and each withheld write with its `reason` and `hint`. It does not apply
|
||||
the serve-time catalog and state size checks; the export reports those.
|
||||
|
||||
Re-record a withheld write at most twice, and only for a recording gap. Stop at
|
||||
once when the reason is application behaviour, and report the write as
|
||||
withheld with its hint:
|
||||
|
||||
| Reason | What to do |
|
||||
| --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
|
||||
| `no-observation` | Re-record: read the entity back by id after the write; read it first for an update or delete. |
|
||||
| `unconfirmed` | Re-record: read the entity and its list after the write. |
|
||||
| `incomplete-entities` | Re-record: read the entity through every view that lists it. |
|
||||
| `unexplained-change` | Re-record once with a second, different call of the same write. |
|
||||
| `no-call` | Re-record once with a write that succeeds. |
|
||||
| `side-effect` | Stop: the application changes other entities. |
|
||||
| `contradicted` | Stop: a later read disagrees with the modelled effect. |
|
||||
| `body-unusable` | Re-record a UI write once with `request` steps (its body was not exposed); else stop. |
|
||||
|
||||
Use new scratch names on each attempt, and delete what an earlier attempt
|
||||
created before recording again (through the UI, or a `request` step with the
|
||||
id that capture shows): the environment starts from what the recording first
|
||||
reads, so a leftover object becomes seeded data that write tasks then find.
|
||||
New browser scenario captures include request times, so a fully timed capture
|
||||
can learn a write clock and serve updates that stamp modification times. Older
|
||||
captures or ones missing a request start keep record order and learn no clock.
|
||||
When coverage reports that limitation, record again with the current version
|
||||
before concluding that the timestamp change cannot be served.
|
||||
|
||||
## Export and hand off
|
||||
|
||||
`export_environment_bundle` with `source: {kind: "retained-evidence",
|
||||
evidence_id}` (or `{kind: "evidence-file", path}`), an absolute
|
||||
`output_directory` that does not exist yet and whose parent does, and a `name`
|
||||
(required when the application runs on `localhost` or an IP address). It returns
|
||||
the coverage again, `tool_count`, the written files and warnings; `dry_run:
|
||||
true` reports without writing. The bundle installs `rea-agents` of this REA's
|
||||
version from the registry; when the user runs an unreleased REA build, pass the
|
||||
`package_tarball` they give you (a packed `rea-agents` tarball) instead.
|
||||
|
||||
Tell the user how many tools answer from the state, which writes are served and
|
||||
which are withheld and why, and the run command (`agent-env run <folder>`). Say
|
||||
that the bundle ships the recorded responses and the capture's local paths
|
||||
(for a browser scenario, the browser executable's path), and
|
||||
list any `credential-named-members` warning pointers for review. Never publish
|
||||
or share the bundle yourself.
|
||||
|
||||
See
|
||||
[agent environments](https://github.com/morluto/rea/blob/main/docs/agent-environments.md)
|
||||
for what the state, replay and the bundle establish, and
|
||||
[evidence workflows](evidence-workflows.md) for the manifest and state views.
|
||||
@@ -11,44 +11,6 @@ returned inline. When complete `inspect_binary_layout` or
|
||||
`inspect_analysis_view` for a summary, one section/symbol/module, or a stable
|
||||
page instead of repeating the producer.
|
||||
|
||||
For any logical Evidence source that advertises retained references, pass
|
||||
`{kind: "retained-evidence", evidence_id}` instead of copying the complete
|
||||
record back into a call. The same contract covers individual, optional, array
|
||||
and nested sources across native, process, JavaScript, managed, mobile and WASM
|
||||
workflows. Sources can independently mix inline records and exact references.
|
||||
Resolution preserves the source's identity, provenance, coverage, partial facts
|
||||
and unknowns; it neither reruns analysis nor makes incomplete facts complete.
|
||||
Keep domain role and identity requirements: a native dossier cannot substitute
|
||||
for a managed member observation, for example.
|
||||
|
||||
When starting from a summary or inspection page, use its
|
||||
`normalized_result.parent_evidence_id` to reference the complete source.
|
||||
The view's own `evidence_id` names
|
||||
only that view. `compare_artifacts` explicitly supports artifact summary inputs
|
||||
and resolves their parents; do not assume every consumer does. When a workflow
|
||||
follows source links, keep those linked records available too.
|
||||
|
||||
MCP references belong to the current connection's ledger. If the ID is known,
|
||||
pass it directly; no discovery call or provider setup is needed. If it is missing,
|
||||
import the exported bundle or supply complete inline Evidence. Do not retry an
|
||||
unchanged reference or rerun a producer merely to move an existing record.
|
||||
Export before closing the session when the source must survive it. For CLI JSON
|
||||
workflows that advertise `--evidence-bundle`, the same reference JSON resolves
|
||||
against one explicitly selected authenticated bundle:
|
||||
|
||||
```bash
|
||||
rea trace-application-feature request.json --evidence-bundle /absolute/path/evidence.json --json
|
||||
```
|
||||
|
||||
This includes JavaScript traces/comparisons and runtime reconciliation, managed
|
||||
reconstruction/native verification/graph projection, Android/Apple/Harmony graph
|
||||
projection, WASM interface comparison and retained analysis views. The managed
|
||||
member comparison CLI still takes two assembly paths; process comparison takes
|
||||
two capture Evidence JSON paths. Check command help for the installed version.
|
||||
An input bundle grants no implicit file lookup: `evidence-file` and historical
|
||||
graph file selectors are accepted only by contracts advertising them. See the
|
||||
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/cli.md#import-export-and-compare-evidence).
|
||||
|
||||
Completed `inspect_artifact` inventory also supports a summary and
|
||||
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
|
||||
Omit `logical_paths` to page all available occurrences; `[]` selects none.
|
||||
@@ -73,14 +35,6 @@ Parent Evidence ID plus source index preserves same-address
|
||||
alternatives. Follow `coverage.next_offset` only as far as the question needs;
|
||||
exhausting retained rows does not establish complete program discovery.
|
||||
|
||||
`batch_decompile` returns every body unless you select `detail: "summary"` when
|
||||
the connected tool advertises it. Use its parent Evidence ID with
|
||||
`view: {kind: "batch-decompile", collection: "items"}` for body-free rows and
|
||||
exact `where.indices`, `addresses`, `procedure_addresses` or `statuses`. Add
|
||||
`include_pseudocode: true` for several chosen bodies, or use
|
||||
`collection: "pseudocode"` with one row's `index` to page a large body. Batch
|
||||
limitations aggregate every item, so do not attribute them to one successful row.
|
||||
|
||||
Use `record_unknown` to track unresolved questions in the user's investigation
|
||||
and name the authority or environment still required. Supply supporting and
|
||||
contradicting evidence IDs. It updates the session registry directly.
|
||||
@@ -92,22 +46,16 @@ Use comparisons with complete, compatible page sets when claiming equivalence
|
||||
or absence:
|
||||
|
||||
- `compare_artifacts` compares inventory pages by occurrence path, content,
|
||||
metadata, and graph relations. Name each side as
|
||||
`{kind: "retained-evidence", evidence_id}` (an `inspect_artifact` result,
|
||||
its nested inventory or its summary) or
|
||||
`{kind: "evidence-file", path, evidence_id?}` instead of resending Evidence.
|
||||
metadata, and graph relations.
|
||||
- `compare_functions` compares explicit function Evidence, not fuzzy
|
||||
whole-binary matches. It rejects an `analyze_function` summary reply; pass
|
||||
the summary's `parent_evidence_id` as a `retained-evidence` reference or
|
||||
the complete dossier.
|
||||
whole-binary matches.
|
||||
- `compare_bundles` compares canonical bundle membership and unknown history.
|
||||
- `find_changed_behavior` combines existing comparisons; static differences
|
||||
remain candidates, not causal proof. Its `comparisons` accept exact references.
|
||||
remain candidates, not causal proof.
|
||||
- `build_call_path` needs explicit function dossiers from one artifact/provider;
|
||||
pass complete parent IDs in `functions`. An incomplete frontier makes absence
|
||||
unknown.
|
||||
an incomplete frontier makes absence unknown.
|
||||
- `correlate_static_and_runtime` uses explicit mappings; matching patterns are
|
||||
hypotheses, not causality. Both comparison arrays accept exact references.
|
||||
hypotheses, not causality.
|
||||
- `verify_reconstruction` evaluates a finite typed specification. A pass covers
|
||||
only declared comparable claims, not global equivalence.
|
||||
|
||||
@@ -156,21 +104,3 @@ each plan states why in its `reason`. It replays the capture instead with
|
||||
`--replay-only` or when the state is unavailable (not derivable, or a catalog
|
||||
over the discovery budget), and its stopped document's `state` says which
|
||||
applied.
|
||||
Use `view: {kind: "environment-coverage"}` with the same rules to see what an
|
||||
environment built from a capture would serve: read tools answered from the
|
||||
state or replayed with their reason, and each write tool served or withheld
|
||||
with a hint about what to record next; it does not apply the serve-time
|
||||
catalog and state size checks. `rea environment-build <capture.har> <dir>`
|
||||
inspects, projects and exports in one step and prints the same coverage, or
|
||||
`null` when the bundle serves replay.
|
||||
|
||||
For a PE/ELF export question, `inspect_native_exports` accepts `detail: "summary"`
|
||||
and retains the complete table. Query `inspect_analysis_view` with
|
||||
`view: {kind: "native-exports", collection: "exports", names: ["selected_name"],
|
||||
fields: ["name"], offset: 0, limit: 32}` and the reported parent Evidence ID.
|
||||
All same-name versions and aliases survive; use `source_index` with the parent ID
|
||||
for stable identity. `collection: "issues"` pages interpretation failures. Page
|
||||
coverage describes selected retained rows; `summary.coverage` or `exports.coverage`
|
||||
still describes the source table, so an empty partial page does not prove absence.
|
||||
CLI `inspect-native-exports --detail summary --evidence_output PATH` saves the
|
||||
complete bundle for later `inspect-analysis-view --evidence-bundle PATH` requests.
|
||||
@@ -17,20 +17,15 @@ the complete artifact inventory and identity but reads source text and analyzes
|
||||
only selected files. The result and summary record `source_selection`; files
|
||||
outside scope and their relationships remain unknown, so partial application
|
||||
coverage does not mean the selected source failed. Use repeated `--source-path`
|
||||
on either static application CLI route, or `--source-paths-json` with a JSON
|
||||
file for selections too long for one command line. Omit the selection when the
|
||||
question requires complete application analysis; `source_paths: []` is invalid,
|
||||
not an unfiltered request. `detail: "summary"` reduces delivery
|
||||
on either static application CLI route. Omit the selection when the question
|
||||
requires complete application analysis; `source_paths: []` is invalid, not an
|
||||
unfiltered request. `detail: "summary"` reduces delivery
|
||||
and does not by itself reduce analysis work.
|
||||
|
||||
ASAR header/index admission defaults to 16 MiB / 16777216 bytes;
|
||||
an admission error names `asar.max_header_bytes` and its bounds (at most
|
||||
256 MiB). Call `update_analysis_settings` with that setting, `action: "set"`
|
||||
and enough bytes, then retry the same full-archive request. No inspection or
|
||||
REA restart is required. `REA_ASAR_MAX_HEADER_BYTES` seeds future REA processes.
|
||||
`source_paths`, `detail`, and `max_heap_mb` do not bypass or raise header admission.
|
||||
Runtime JavaScript timeout/heap defaults are also editable; explicit per-call
|
||||
`analysis_timeout_ms` and `max_heap_mb` take precedence.
|
||||
`REA_ASAR_MAX_HEADER_BYTES` (at most 256 MiB, set where REA starts) changes it.
|
||||
`source_paths`, `detail`, and `max_heap_mb` do not bypass or raise it. Or
|
||||
analyze an extracted directory.
|
||||
|
||||
Exact `#alias` specifiers resolve through the importer's
|
||||
enclosing package `imports` map to package-local files; nested `node_modules`
|
||||
@@ -78,15 +73,6 @@ classification, and unavailable sources never establish route reachability or
|
||||
the complete application route denominator. See the
|
||||
[route inventory guide](https://github.com/morluto/rea/blob/main/docs/javascript-route-inventory.md).
|
||||
|
||||
For captured endpoint observations and inference facts, `analyze_web_bundle`
|
||||
also accepts `view: {kind: "web-bundle", collection: "endpoints", limit: 32}`
|
||||
with `detail: "summary"`. Read later `endpoints`, `inferences`, or `summary`
|
||||
pages from the retained parent. Filter before requesting detail; retain
|
||||
`source_index` together with parent ID and collection, and request `location`
|
||||
or `basis` when tracing sources. Page exhaustion does not mean producer
|
||||
completeness: check `completeness`, structured `source_unknowns`, and limitations.
|
||||
Endpoint candidates remain inferences rather than confirmed network behavior.
|
||||
|
||||
## Read a module without repeating analysis
|
||||
|
||||
Use the actual returned parent ID, on the same connection, as `source.evidence_id`.
|
||||
@@ -169,10 +155,7 @@ leaving it, and `both` when the question needs both. Trace traversal includes
|
||||
containment relationships, so even a narrow seed can expand many nodes; a
|
||||
bidirectional trace can reach unrelated sibling modules. A module item or page
|
||||
is preferable when its observations already answer the question. Do not treat
|
||||
all reached nodes as participants in the feature. For a broad trace, pass
|
||||
`view: {"kind": "feature-trace", "collection": "paths"}` (or `summary`) to
|
||||
retain the complete trace and read terminal paths first; page `nodes` or `edges`
|
||||
with exact `where.node_ids` through `inspect_analysis_view` as needed.
|
||||
all reached nodes as participants in the feature.
|
||||
|
||||
Reuse retained parent Evidence when the server advertises it, or supply complete
|
||||
Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest
|
||||
@@ -239,19 +222,10 @@ This notation is
|
||||
a template: replace it with the actual returned ID. Versions before 4.1.0 accept only full
|
||||
inline Evidence. Use the exact ID
|
||||
returned by the producer on the same MCP connection. Resolution does not run
|
||||
analysis or select a provider; findings remain inline. For summary or inspection
|
||||
replies, select the complete `normalized_result.parent_evidence_id`. Native
|
||||
observation arrays and runtime reconciliation's `static_layers[].analysis` and
|
||||
`runtime_observations[]` also accept exact references where advertised. Each
|
||||
source can independently be inline or retained, preserving identity, coverage
|
||||
and unknowns. CLI workflows advertising `--evidence-bundle` resolve these inputs
|
||||
from the explicitly selected bundle. `close_binary` releases the engine while
|
||||
retaining references and complete Unknown history on the same MCP connection.
|
||||
`reset_investigation` explicitly clears them; optional `save_path` checkpoints the
|
||||
complete bundle before clearing. Import that bundle in another connection to
|
||||
restore its original references. Retention is bounded, in-memory and never silently
|
||||
evicted; `binary_session.investigation` reports the budget. A missing reference
|
||||
includes its ID and recovery guidance.
|
||||
analysis or select a provider; findings remain inline. Native observation
|
||||
arrays still take full Evidence. `close_binary` clears retained references;
|
||||
export a bundle before closing, or supply portable inline Evidence in another
|
||||
connection. A missing reference includes its ID and recovery guidance.
|
||||
|
||||
For exact file/native metadata or processing gaps, use
|
||||
`view: {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"], limit: 32}`.
|
||||
|
||||
@@ -1,35 +1,16 @@
|
||||
# Native, managed, and packaged artifacts
|
||||
|
||||
For release comparisons on one MCP connection, retain `list_procedures` Evidence
|
||||
from each target and reuse its exact IDs after target switches or `close_binary`.
|
||||
For a handoff or new process, export the complete bundle and import it there,
|
||||
including its source closure. `reset_investigation` explicitly clears retained
|
||||
records; optional `save_path` checkpoints them before clearing.
|
||||
`match_native_functions` pairs unique exact names and, when supplied,
|
||||
For release comparisons, export `list_procedures` Evidence from each target before
|
||||
closing it. Close clears session records: import both bundles after collecting the
|
||||
two targets, or supply portable inline Evidence. `match_native_functions` pairs unique exact names and, when supplied,
|
||||
complete observed body bytes. Reuse optional same-context `list_names`,
|
||||
`procedure_info`, `read_bytes` and existing `analyze_function` dossiers as
|
||||
`left_observations`/`right_observations`; an MCP `analyze_function` summary
|
||||
reply contributes its `parent_evidence_id` as a `retained-evidence` reference,
|
||||
not the summary itself. Avoid decompiling every function merely to match it. Its summary retains a parent for
|
||||
`procedure_info`, `read_bytes` and existing `analyze_function` Evidence; avoid
|
||||
decompiling every function merely to match it. Its summary retains a parent for
|
||||
`inspect_analysis_view` `function-matches` collections `matches` and `functions`.
|
||||
Select columns, batch IDs and page individual facts; aliases remain unresolved.
|
||||
Unpaired observations do not prove additions/removals. Existing matched dossier
|
||||
pairs include inputs for `compare_functions`; otherwise analyze just chosen pairs.
|
||||
|
||||
Aggregate completed comparisons with `find_changed_behavior`, selecting each
|
||||
comparison's exact `retained-evidence` ID instead of resending full records.
|
||||
Inline comparison Evidence remains accepted. References belong to the current
|
||||
server: after handoff, import the original bundle including its source closure.
|
||||
Do not substitute dossier IDs or summary views. Static differences remain
|
||||
behavior candidates, with runtime behavior unknown unless separately observed.
|
||||
|
||||
For a matching representation refusal, inspect `details.limit` and raise
|
||||
`native.matching_representation_bytes` with `update_analysis_settings` before
|
||||
retrying the unchanged MCP request. CLI launch seed:
|
||||
`REA_NATIVE_MATCHING_REPRESENTATION_BYTES`. The default is 96 MiB; the policy
|
||||
range is 1 byte to 1 GiB and does not bound process RSS. Keep the complete
|
||||
inventories: splitting them changes global name/body uniqueness.
|
||||
|
||||
## Apple app resources
|
||||
|
||||
Use `open_bundle_resources` with the selected app path, then `inspect_plist`,
|
||||
@@ -129,32 +110,6 @@ not proof the image is unsigned in every catalog or overlay sense. Signatures,
|
||||
chains, timestamps, and Authenticode digest coverage remain unobserved. See
|
||||
[PE Authenticode certificate tables](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
|
||||
## PE import and delay-load tables
|
||||
|
||||
Use `inspect_pe_imports` for an explicit local PE32/PE32+ image when the
|
||||
question concerns imported DLL and function names, ordinals, delay-load
|
||||
descriptors, or the pefile import hash. No `open_binary`, Windows SDK, or
|
||||
target execution is required. The CLI equivalent is `inspect-pe-imports PATH`.
|
||||
Preserve artifact SHA-256, directory file ranges, original DLL spellings, and
|
||||
ILT/IAT RVAs. A missing `IMAGE_DIRECTORY_ENTRY_IMPORT` directory is observed
|
||||
absence of that table, not proof the image never imported APIs through another
|
||||
table or overlay. Bound snapshots, loaded IAT contents, and export-forward
|
||||
resolution remain unobserved. See
|
||||
[PE import and delay-load tables](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
|
||||
|
||||
## PE TLS callbacks and load configuration
|
||||
|
||||
Use `inspect_pe_load_config` for an explicit local PE32/PE32+ image when the
|
||||
question concerns code that runs before the entry point through TLS callbacks,
|
||||
or the security cookie, SafeSEH, CFG, EH-continuation and XFG declarations in
|
||||
the load configuration. No `open_binary`, Windows SDK, or target execution is
|
||||
required. The CLI equivalent is `inspect-pe-load-config PATH`. Preserve artifact
|
||||
SHA-256, directory and callback-array file ranges, and callback VAs as
|
||||
hexadecimal strings. Fields outside the declared `Size` are `null`. Guard-flag
|
||||
mitigations carry `evidence_kind: "inferred"`; enforcement remains unobserved.
|
||||
See
|
||||
[PE TLS callbacks and load configuration](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
|
||||
|
||||
## ELF note tables
|
||||
|
||||
Use `inspect_elf_notes` for an explicit local ELF32/ELF64 image when the
|
||||
@@ -166,37 +121,6 @@ absence of those declarations, not proof the image never carried notes in
|
||||
another container. See
|
||||
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
|
||||
## Apple IPS crash reports
|
||||
|
||||
Use `inspect_apple_ips` for an explicit local Apple `.ips` file when the
|
||||
question concerns crash metadata, exception and termination facts, threads,
|
||||
usedImages, lossless registers, or documented crash classes. No `open_binary`,
|
||||
Xcode, atos, or target execution is required. The CLI equivalent is
|
||||
`inspect-apple-ips PATH`. Preserve artifact SHA-256, `bug_type` as a string,
|
||||
uppercase image UUIDs, and integer literals that exceed IEEE-754 range. Only
|
||||
`bug_type` `"309"` yields a crash body; other DiagnosticReports types return
|
||||
no crash result. Opened Mach-O and dSYM UUID joins remain unobserved. See
|
||||
[Apple IPS crash reports](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
|
||||
|
||||
## Linux kernel modules
|
||||
|
||||
Use `inspect_kernel_module` for an explicit local ELF32/ELF64 relocatable
|
||||
object when the question concerns `.modinfo` license, vermagic, alias,
|
||||
depends, or parm strings, the `__versions` symbol CRCs, or an appended module
|
||||
signature trailer. No `open_binary`, Linux host, or `modinfo` program is
|
||||
required. The CLI equivalent is `inspect-kernel-module PATH`. Preserve
|
||||
artifact SHA-256, key order, duplicate keys, CRC hex, and section file ranges.
|
||||
A missing section is observed absence when section names are known, not proof
|
||||
the module never carried that metadata. `.gnu.linkonce.this_module` is reported
|
||||
as present or absent; `struct module` stays unknown because its layout changes
|
||||
with the kernel version. `__ksymtab` records are not decoded. The PKCS#7 range
|
||||
and its SHA-256 are observed; certificate, signer, and digest coverage remain
|
||||
unvalidated. An absent trailer is not proof the module is unsigned in another
|
||||
container. gzip, xz, and zstd whole-file containers are unsupported and name
|
||||
`external_decompression` as the next probe: use an external codec utility to
|
||||
decompress into a separate file, then call `inspect_kernel_module` on that file. See
|
||||
[Linux kernel modules](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
|
||||
|
||||
## CPython pyc files
|
||||
|
||||
Use `inspect_python_pyc` for an explicit local CPython 3.7–3.13 `.pyc` when the
|
||||
@@ -214,11 +138,7 @@ question concerns IFF chunks, atom names, exports, imports, lambdas, or the
|
||||
Code header. No `open_binary`, Erlang installation, or bytecode execution is
|
||||
required. The CLI equivalent is `inspect-erlang-beam PATH`. Preserve artifact
|
||||
SHA-256, chunk file ranges, 1-based atom indices, and original atom spellings.
|
||||
Missing optional tables stay null. For a few facts, use `detail: "summary"` with
|
||||
a `beam` view selecting a collection, exact names/indices and fields; read later
|
||||
pages from `parent_evidence_id` or the complete Evidence bundle. Same-name
|
||||
exports retain every arity and original atom index. CLI summary delivery requires
|
||||
`--evidence_output`. Null table counts remain distinct from decoded empty tables. See
|
||||
Missing optional tables stay null. See
|
||||
[Erlang BEAM files](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
|
||||
|
||||
## SQLite snapshots
|
||||
@@ -311,9 +231,6 @@ unknown rather than false.
|
||||
For a completed `analyze_function` dossier, use `inspect_analysis_view` with
|
||||
`view: {kind: "native", facet: "value_flow_operations", offset: 0, limit: 32}`
|
||||
and the exact retained Evidence ID to read selected facts without reanalysis.
|
||||
The default MCP reply is already a summary of that dossier: its
|
||||
`normalized_result.parent_evidence_id` is the retained ID to page, and its
|
||||
null counts mark facets the provider did not record or declared unavailable.
|
||||
Native facets also include procedure, pseudocode, assembly, basic blocks,
|
||||
comments, callers/callees, references, unresolved calls, referenced strings and
|
||||
names, the native API record, and value-flow summaries. Pseudocode offsets and limits count UTF-16
|
||||
@@ -333,62 +250,36 @@ run, use `observe_native_calls` with explicit breakpoints and a bounded
|
||||
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
|
||||
runs and may change files or use the network. Hardened-runtime targets need the
|
||||
`get-task-allow` entitlement.
|
||||
Function breakpoints use exact LLDB runtime names: pass an `inspect_macho` export
|
||||
or import `runtime_name` (`main`), not its symbol-table `name` (`_main`). REA does
|
||||
not translate breakpoint names. Set `module` to the image
|
||||
file name when selecting that image only; otherwise loaded system images also
|
||||
match. Inspect retained locations and module paths before attributing a call.
|
||||
|
||||
Use `binary_session` with no arguments to check the open target, selected
|
||||
provider, and alignment. Its default summary `result.tool_availability` carries
|
||||
the complete tool inventory grouped by current status. Expand each group's
|
||||
`names` and inherit the inventory-level client requirements unless a group
|
||||
overrides them; availability, reasons, remediation, and modes remain inline.
|
||||
Provider candidates keep availability and target-support status, code, and
|
||||
reason. `omitted_fields` names what the summary leaves out; `{"detail":
|
||||
"complete"}` adds each unavailable candidate's `remediation`, provider
|
||||
diagnostics, candidate capability sets, and tool surface, `effects`, and
|
||||
`annotations`. Those arrays name true flags from fixed
|
||||
vocabularies, so an omitted known flag is false. `tools/list` remains the
|
||||
canonical source for full tool schemas when target or provider state changes.
|
||||
provider, and alignment. Its default `result.tool_availability` carries a
|
||||
complete inventory grouped by shared facts. Expand each group's `names` and
|
||||
inherit the inventory-level client requirements unless a group overrides them;
|
||||
availability, reasons, remediation, modes, and flags remain inline. The
|
||||
`effects` and `annotations` arrays name true flags from fixed vocabularies, so an
|
||||
omitted known flag is false. `tools/list` remains the canonical source for full
|
||||
tool schemas when target or provider state changes.
|
||||
|
||||
Provider capability groups carry provider identity and ordered limitations.
|
||||
Prepend each set's `shared_limitations` to a group's `limitations` to recover the
|
||||
operation's full ordered list. Candidate capabilities, returned with complete
|
||||
detail, describe possible providers. Top-level session capabilities describe the current route, and
|
||||
operation's full ordered list. Candidate capabilities describe possible
|
||||
providers. Top-level session capabilities describe the current route, and
|
||||
`analysis_provider_binding` identifies the selected provider.
|
||||
|
||||
## Reuse caller-selected Ghidra analysis
|
||||
|
||||
When the caller supplies an expanded project, configure `REA_GHIDRA_PROJECT_PATH`
|
||||
(the `.gpr` marker) and optionally `REA_GHIDRA_PROGRAM_NAME` (one root program),
|
||||
then restart/reconnect before `open_binary`. CLI uses the same configuration.
|
||||
REA opens only a fingerprint-bound owned copy with `-process -noanalysis -readOnly`;
|
||||
original project files are not opened by the engine or saved. Database edits,
|
||||
import overrides and repair scripts are unavailable in this mode. Treat original
|
||||
analysis completeness/settings as unknown and prior edits as caller-project state.
|
||||
The program's recorded import digest must match the target; changed project content
|
||||
requires reopening for a fresh profile. See the
|
||||
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/installation.md#reuse-an-existing-ghidra-project).
|
||||
|
||||
## Ghidra startup deadline recovery
|
||||
|
||||
`provider_timeout` during the first Ghidra query can cover import, auto-analysis,
|
||||
bridge startup, and readiness. The default 330,000 ms is configurable through
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS`; a smaller query still needs the same cold import.
|
||||
|
||||
When the error names `ghidra.startup_timeout_ms`, call
|
||||
`update_analysis_settings` with `action: "set"` and a larger value within the
|
||||
reported bounds. Resolve retained cleanup before the next startup attempt.
|
||||
No REA restart is needed; an already accepted startup keeps its original
|
||||
budget. A new attempt repeats import. Preserve useful Evidence before closing
|
||||
and reopening the target when remediation requires that lifecycle step.
|
||||
|
||||
For a per-function timeout, update `ghidra.decompile_timeout_seconds` and retry
|
||||
the same function on the existing Program. Explicit `null` removes that
|
||||
per-function deadline. Inspection is optional when the error supplies the
|
||||
setting and bounds. Runtime updates disappear when REA exits; environment
|
||||
changes only seed future REA processes.
|
||||
Preserve useful Evidence before closing the session or restarting REA. Check
|
||||
`close_binary` and resolve reported cleanup failures. Set a larger startup budget
|
||||
in the environment that launches REA, then restart REA and reconnect the MCP
|
||||
client before reopening the same target. Closing and reopening a binary in the
|
||||
existing server keeps its original configuration and starts import again.
|
||||
Environment changes in an agent's shell do not update an already running MCP
|
||||
server. If its launch configuration cannot be changed, report that constraint
|
||||
and the exact retry setting instead of claiming a fixed technical limit.
|
||||
|
||||
The client's tool-call deadline must allow startup plus the requested query;
|
||||
server settings and progress notifications cannot override it. CLI analysis
|
||||
@@ -434,21 +325,6 @@ encoded in bounded batches.
|
||||
The 25000 ms Cutter request deadline and the 256-candidate discovery budget
|
||||
still have no runtime override.
|
||||
|
||||
## Selected native facts and call graphs
|
||||
|
||||
For a large `get_call_graph`, or repeated `procedure_address`,
|
||||
`procedure_callees`, `procedure_callers` and `resolve_containing_procedure`
|
||||
queries, select `detail: "summary"` when the first 64 recorded observations
|
||||
suffice. Read further `native-observation` pages from the returned parent Evidence
|
||||
ID. Graph message IDs index that page's exact-text dictionary; keep level,
|
||||
source index, failed nodes, and partial/unknown relationship coverage.
|
||||
Read referenced source context through `native-context` facets `overview`,
|
||||
`analysis-profile`, `limitations` and `result-limitations`. Reuse inspected
|
||||
context/profile values only for the same digest; until caveats are read,
|
||||
interpretation beyond the returned facts remains unknown. Complete Evidence
|
||||
remains retained and exportable. Small complete replies can cost fewer tokens
|
||||
than additional context/page calls; count the completed task and discovery.
|
||||
|
||||
## Static memory and original file bytes
|
||||
|
||||
For Ghidra, `read_bytes` reads initialized analysis memory, including loader
|
||||
@@ -476,14 +352,6 @@ loaded bytes, file mappings, relocations and the entry against the immutable
|
||||
snapshot. Keep `mismatch` and `unsupported` explicit. Verified import covers the
|
||||
reported static image, not runtime DOS or PC-98 hardware behavior.
|
||||
|
||||
For a large load-image report, select `detail: "summary"` and query its
|
||||
`parent_evidence_id` with `inspect_analysis_view` kind `native-load-image`.
|
||||
Choose a collection such as `relocations` or `mappings`, select only needed
|
||||
fields, and page with offset/limit. Original indices preserve same-address
|
||||
alternatives. Null collections remain unrecorded or inapplicable; paging does
|
||||
not upgrade unsupported verification. CLI requires `--evidence_output` for
|
||||
summary delivery so complete source Evidence remains accessible.
|
||||
|
||||
For Ghidra decompilation, `procedure_pseudo_code` also accepts `procedure: "file-offset:0x..."`; CLI `rea decompile <path> --file-offset=0x... --provider ghidra` resolves the source byte offset internally. An unresolved 0x-prefixed numeric procedure selector also falls back to a unique file mapping. Existing symbols and analysis addresses win; multiple mappings fail as ambiguous. Preserve the returned file-offset and address locations and raw selection observations. ELF virtual addresses are distinct from file offsets; never assume an arbitrary offset plus image base is an analysis address.
|
||||
|
||||
Complete function body ranges use inclusive ends and can contain gaps. Do not
|
||||
@@ -527,11 +395,6 @@ names), coverage and per-collection counts. Page rows with
|
||||
`view: {kind: "managed-members", collection: "call_edges", offset: 0, limit: 64}`
|
||||
over `types`, `exported_types`, `fields`, `methods`, `member_refs`, `call_edges`,
|
||||
`field_accesses` or `user_strings`.
|
||||
Use exact `names` and `declaring_types` filters before paging supported
|
||||
collections. For a compact method lookup, request `fields: ["name",
|
||||
"declaring_type", "signature"]`; omit `body` to avoid anchors. Identity tokens,
|
||||
original `source_index`, module/MVID and source coverage remain. Check
|
||||
`unresolved_filter_rows` before interpreting an empty match as absence.
|
||||
|
||||
### .NET NativeAOT
|
||||
|
||||
@@ -589,16 +452,6 @@ REA. Symlinks and encrypted entries are inventory facts, not extractable files.
|
||||
The result reports the materialized directory; callers do not choose the
|
||||
destination. No separate permission grant is needed.
|
||||
|
||||
For classic iNES cartridges, `open_binary` recognizes the byte signature without
|
||||
an executable architecture. `inspect_artifact` retains header facts and each
|
||||
PRG/CHR file unit's offset, length and SHA-256; its nested inventory's
|
||||
`ines.reconstruction` verifies the actual ordered region bytes against the
|
||||
whole-file digest. Extraction includes the original header, optional trainer
|
||||
and uninterpreted trailing bytes. These file units are not mapper/CPU bank
|
||||
maps. NES 2.0, UNIF, dirty legacy headers and truncated payloads are explicitly
|
||||
refused. No emulator or 6502 analysis is performed. See the
|
||||
[iNES guide](https://github.com/morluto/rea/blob/main/docs/ines-containers.md).
|
||||
|
||||
Native DMG traversal automatically uses a read-only mount on supported macOS
|
||||
hosts, with an owned temporary mount directory and cleanup. It needs no approval
|
||||
flag. Unsupported hosts or mount failures remain explicit limitations; do not
|
||||
@@ -627,27 +480,9 @@ semantics are unsupported. `m68k` requires an even entry and decodes the 68040
|
||||
superset of the 68000 family; Amiga/Atari containers and vector-table entry
|
||||
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
|
||||
|
||||
Intel HEX, S-record, UF2 and DfuSe files are containers, not raw images:
|
||||
`inspect_artifact` lists address regions (UF2 per familyID, DfuSe per element
|
||||
with its alternate setting), and `extract_artifact` writes each region. Open an
|
||||
extracted region as `raw-binary` with its start address as `entry_point`. UF2
|
||||
numbering gaps leave coverage unknown; a DfuSe suffix CRC mismatch follows
|
||||
`integrity_policy`. See [UF2 and DfuSe](https://github.com/morluto/rea/blob/main/docs/uf2-dfuse.md).
|
||||
|
||||
### Function annotations in Ghidra
|
||||
|
||||
Fresh-import sessions admit annotations; existing-project reuse is read-only.
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier. Ghidra rejects batch addresses that request the same name or a name another selected address holds; `raw_result.failures` gives each failure's reason.
|
||||
|
||||
For `compare_functions`, obtain `analyze_function` Evidence before and after the
|
||||
edit while the MCP session is open; annotation Evidence is not a comparison
|
||||
input. Export the needed Evidence before close if comparison will happen later.
|
||||
Authenticated same-artifact coordinates preserve renamed entries as
|
||||
`matched / same-entry`, with the canonical `entry_address` inline; name and
|
||||
exact-text changes remain separate dimensions. This requires compatible
|
||||
architecture, provider-owned mapping and selected-image commitments. Equal
|
||||
numeric addresses alone cannot establish correspondence; inspect retained
|
||||
source Evidence and coverage when the match remains ambiguous.
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier.
|
||||
|
||||
## Offline PE resource inventory
|
||||
|
||||
@@ -675,9 +510,7 @@ For exact core-module import/export differences, use `compare_wasm_interfaces`
|
||||
with two complete inspection Evidence records (or current-session retained
|
||||
references) and explicit scope. Preserve unknown types/duplicate names; static
|
||||
interface equality is not runtime compatibility. CLI accepts complete inline
|
||||
Evidence in `rea compare-wasm-interfaces input.json --json`, or exact references
|
||||
with `--evidence-bundle /absolute/path/evidence.json`. Use the complete parent
|
||||
ID when the inspection returned a summary.
|
||||
Evidence in `rea compare-wasm-interfaces input.json --json`.
|
||||
|
||||
## Godot PCK
|
||||
|
||||
@@ -697,84 +530,3 @@ page cannot prove absence in unknown coverage. Retained views are historical,
|
||||
not fresh filesystem checks. No decryption, Godot resource decoding, GDScript
|
||||
decompilation or target execution is provided. See
|
||||
[Godot PCK](https://github.com/morluto/rea/blob/main/docs/godot-pck.md).
|
||||
|
||||
UnityFS AssetBundle containers (`.bundle`, `.ab`, YooAsset packages and other
|
||||
byte-identified files) list `CAB-*` SerializedFiles and `.resS` members with
|
||||
header and node facts; none/LZ4/LZ4HC member blocks stream on demand. Bundles
|
||||
declaring asset-bundle encryption keep header facts with unknown directory
|
||||
coverage; LZMA member bytes retain named occurrences with unavailable hashes.
|
||||
Virtual member offsets are not physical carrier locations. Format 6–8 readers
|
||||
cache one decoded block and support retained filtered/paged views. No
|
||||
decryption or SerializedFile object decoding is provided. See
|
||||
[UnityFS](https://github.com/morluto/rea/blob/main/docs/unityfs.md).
|
||||
|
||||
U-Boot legacy uImages use the same inspection/extraction workflows. Prefer a
|
||||
retained summary and selected inventory views for header facts or component
|
||||
IDs. Header addresses/CPU/OS are declarations; payloads remain as stored.
|
||||
Multi-file components retain their aligned physical ranges; the full payload
|
||||
is also materialized. Unknown name text retains its original 32 bytes. See
|
||||
[U-Boot uImage](https://github.com/morluto/rea/blob/main/docs/uimage.md).
|
||||
|
||||
## ESP images
|
||||
|
||||
For an ESP32-family app or bootloader image, `open_binary` then
|
||||
`inspect_artifact` with `detail: "summary"`. `root_format_metadata` holds the
|
||||
observed `header` and `descriptor` (its `kind` is the app or bootloader role)
|
||||
and the derived `chip`, `checksum`, `appended_sha256` and
|
||||
`undescribed_trailing` ranges. Segment rows (`segment-NN.bin`) carry
|
||||
`load_addr`, `classification` (`mapped`, `loaded`, `not-loaded` or `unknown`,
|
||||
per ESP-IDF `should_map`/`should_load`; RAM ranges are not checked) and
|
||||
`source_location`; select them with `inspect_analysis_view` and
|
||||
`logical_paths`. Signature blocks are `unverified`; `public_key_sha256`
|
||||
matches eFuse key digests.
|
||||
|
||||
Unknown `source_coverage` means truncation or undescribed trailing bytes, such
|
||||
as the flash segments of a `--ram-only-header` image; do not infer absence
|
||||
from it. Erased flash is described. Integrity mismatches are facts, not
|
||||
refusals. ESP8266 images are refused, and flash-encrypted images are not
|
||||
recognized. Segments are not disassembled, and RISC-V or Xtensa segments
|
||||
cannot be opened as raw-binary. See
|
||||
[ESP images](https://github.com/morluto/rea/blob/main/docs/esp-image.md).
|
||||
|
||||
## ESP flash images
|
||||
|
||||
A whole-flash dump opens as `esp-flash-image`. `root_format_metadata` holds
|
||||
`partition_table` (its `method`: `default`, `caller` or a derived
|
||||
`scan`), `bootloader` and `apps` ESP image facts, the `otadata` records,
|
||||
`partitions_outside_dump` and `boot_selection`. Every table has a
|
||||
`boot_selection`: its `basis` names the bootloader's branch (`no-otadata`
|
||||
without otadata, `none` when the bootloader rejects the table, `unknown`
|
||||
when a record is missing), and it lists its `assumptions`;
|
||||
`running_partition` is always `unknown`. Each image names its entry
|
||||
`path`. With `detail: "summary"` the root kind is `esp-flash-image-summary`
|
||||
and each image is a compact `summary` (chip, entry point, segment count,
|
||||
classified load ranges, signature block count, descriptor identity,
|
||||
integrity); complete detail keeps every segment. Partition rows
|
||||
(`partitions/<label>.bin`) carry the declared `type`, `subtype`, names,
|
||||
`flash_address`, `size` and flags; `encrypted` is a storage flag, not
|
||||
proof of ciphertext. `written_offset`/`written_end` bound the bytes that
|
||||
are not erased: an app slot whose facts say `erased` can still hold an older
|
||||
image, and a limitation says so. For a dump read from another flash address, call
|
||||
`open_binary` with `format: "esp-flash-image"` and `flash_offset`; an
|
||||
`esp-image` whose `flash_dump` is set names that offset in a next probe; a
|
||||
relocated table can be named with `partition_table_offset`. No table found
|
||||
means unknown, often flash encryption. Extraction writes `nvs_keys` as
|
||||
ordinary data. See
|
||||
[ESP flash images](https://github.com/morluto/rea/blob/main/docs/esp-flash-image.md).
|
||||
|
||||
## Bun standalone executables
|
||||
|
||||
A `bun build --compile` executable stores its module graph in Mach-O
|
||||
`__BUN,__bun`, a PE or ELF `.bun` section, or an appended trailer.
|
||||
`open_binary` then `inspect_artifact` reports root format `bun-standalone` with
|
||||
the carrier, structural layout, module count, entry point, raw flags and
|
||||
compile exec-argv; each module keeps its `/$bunfs/...` source name, loader,
|
||||
encoding, sizes and SHA-256, with sourcemap, bytecode and module-info regions as
|
||||
separate children. Use summary root metadata `entry_point_logical_path` as the
|
||||
exact inventory `logical_paths` selector, then read the selected content node by
|
||||
its artifact ID; the reader owns virtual-name normalization and indexed fallback. `extract-artifact` writes stored contents through the safe
|
||||
output tree; route extracted JavaScript to `analyze_javascript_application`.
|
||||
Layout labels describe record shapes, not a detected compiler version; unknown
|
||||
or ambiguous layouts are unsupported. Bytecode is retained, not decoded, and the
|
||||
executable is never run. See
|
||||
[Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
@@ -1,52 +1,4 @@
|
||||
# Runtime observation
|
||||
|
||||
## Native runtime questions
|
||||
|
||||
Start from the requested observation. State the selected target, input, expected
|
||||
result, failure signal and stopping condition before a run. Reuse static Evidence
|
||||
to choose a function or hypothesis; `trace_native_values` follows recovered
|
||||
static value dependencies and does not measure runtime values. Prefer static
|
||||
inspection or existing capture comparison when it already answers the question.
|
||||
|
||||
| Question | Existing workflow | Host and effects |
|
||||
| -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Which selected native entry is reached, with which integer argument registers? | `observe_native_calls` on the active Mach-O, with exact function/Objective-C breakpoints and a duration/event bound | macOS with LLDB and required debugger access. Launches a new owned process, stops at entries, records and continues; target execution may write files, show UI or use the network. It never attaches to an existing process. Inspect termination and partial coverage after the run. |
|
||||
| What does caller-authored instrumentation observe? | `list_frida_devices` / `list_frida_processes` when discovery is needed; `start_frida_session`, `load_frida_script`, then status/unload/close | Requires compatible installed Frida bindings and selected device access; the real REA verifier covers Linux. Attach selects an existing PID; spawn selects a program and starts it paused until `resume_frida_session`. Scripts execute in the target and can change memory, files, network activity and behavior. This is not passive inspection. |
|
||||
| Which debugger state or controlled execution explains the behavior? | `start_gdb_session` and `gdb_console`, or `start_rizin_debug_session` and `rizin_debug_command` | Persistent debugger workflows are Linux-only. GDB starts a debugger; console commands choose/load/attach/run targets. Rizin debug start launches the selected target and requires a durable `inferior_tty`. Both consoles are unrestricted and can change target memory/registers, run shell commands, write files and use the network. |
|
||||
| How do terminal output, process lifetime and selected files differ between runs? | `capture_process_scenario`, then `compare_process_captures` with compatible recorded inputs | New captures require Linux or macOS and a working native PTY backend; macOS also needs `xcrun`/Swift for ownership inspection. Capture launches an owned command with ordinary user permissions. Observation paths select snapshots, not a filesystem sandbox. Comparing retained captures does not execute targets and is also available on Windows. |
|
||||
| What does an existing desktop window show, or what changes after a declared UI action? | `observe_native_ui`; use `capture_native_ui_scenario` only for requested actions | macOS 14+ with required Screen Recording/Accessibility access. Select an existing PID/window. Observation does not launch or own the app; scenario actions can change its data or trigger network activity, and leave its state as-is. |
|
||||
|
||||
Close each persistent session with its matching tool. `close_frida_session`
|
||||
unloads scripts and detaches; it does not kill the target. A still-paused spawn
|
||||
is resumed only after instrumentation cleanup is confirmed. GDB and Rizin close
|
||||
can detach a live inferior and let it continue only after verifying its durable
|
||||
output terminal. When disposition or cleanup is unresolved, preserve the owned
|
||||
session and reported recovery; do not assume that closing the debugger terminated
|
||||
the target. Process capture and LLDB call observation instead own their launched
|
||||
processes and report their termination/cleanup outcome. Cancellation is not proof
|
||||
that cleanup completed.
|
||||
|
||||
Unrestricted GDB/Rizin commands and relevant start/close operations retire an
|
||||
active binary-analysis session in the same MCP server. Close an active Ghidra
|
||||
session explicitly first; after target effects settle, reopen the artifact before
|
||||
new static analysis. A Cutter bridge command is likewise unrestricted and may
|
||||
affect its GUI/target; it is not a passive native observation substitute.
|
||||
|
||||
To test a static hypothesis across two builds, retain compatible static
|
||||
comparison Evidence and `compare_process_captures` Evidence. Supply explicit
|
||||
finding/dimension mappings to `correlate_static_and_runtime`. It does not ingest
|
||||
arbitrary Frida, LLDB or debugger traces, auto-match names, or prove causality
|
||||
from cochange. Preserve those traces as their own Evidence and leave unobserved
|
||||
behavior unknown. Record truncation, unmatched breakpoints, changed target
|
||||
identity and cleanup uncertainty as limitations, rather than treating no retained
|
||||
event as proof that the behavior never occurs.
|
||||
|
||||
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/frida-instrumentation.md),
|
||||
[GDB/Rizin/Cutter](https://github.com/morluto/rea/blob/main/docs/reverse-engineering-tools.md),
|
||||
[process capture](https://github.com/morluto/rea/blob/main/docs/process-capture.md),
|
||||
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/native-investigation.md#native-desktop-observation).
|
||||
|
||||
## Browser and Electron
|
||||
# Browser and Electron runtime observation
|
||||
|
||||
Supply the existing browser or Electron instance's literal-port loopback CDP
|
||||
endpoint in the request. No separate permission configuration or per-call
|
||||
@@ -86,8 +38,6 @@ Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
|
||||
browser and profile. Check `browser.cleanup` for the outcome.
|
||||
Request only the DOM, accessibility, or network details needed for the question.
|
||||
The default final capture contains the URL; step outcomes are reported separately.
|
||||
For multi-step flows with trees or events, select `detail: "summary"` and page
|
||||
exact steps, transactions, text slices or events with `inspect_analysis_view` kind `browser-scenario`.
|
||||
Add a wait action for the expected page state before capturing it. Report that
|
||||
REA performed the declared actions.
|
||||
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
|
||||
"name": "rea",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"owner": {
|
||||
"name": "morluto",
|
||||
"url": "https://github.com/morluto"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "rea",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"source": {
|
||||
"source": "npm",
|
||||
"package": "rea-agents",
|
||||
"version": "6.8.0"
|
||||
},
|
||||
"category": "development"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
{
|
||||
"name": "rea",
|
||||
"version": "6.8.0",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"author": {
|
||||
"name": "morluto",
|
||||
"url": "https://github.com/morluto"
|
||||
},
|
||||
"homepage": "https://rea.tools/",
|
||||
"repository": "https://github.com/morluto/rea",
|
||||
"license": "MIT",
|
||||
"keywords": [
|
||||
"reverse-engineering",
|
||||
"binary-analysis",
|
||||
"decompiler",
|
||||
"mcp-server"
|
||||
],
|
||||
"skills": "./skills",
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "node",
|
||||
"args": [
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/plugin-mcp-launcher.mjs",
|
||||
"6.8.0"
|
||||
]
|
||||
}
|
||||
},
|
||||
"icon": "./assets/rea-plugin.png"
|
||||
}
|
||||
@@ -1,13 +0,0 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "node",
|
||||
"args": [
|
||||
"./scripts/plugin-mcp-launcher.mjs",
|
||||
"6.8.0"
|
||||
],
|
||||
"cwd": ".",
|
||||
"startup_timeout_sec": 30
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,40 +0,0 @@
|
||||
{
|
||||
"name": "rea",
|
||||
"version": "6.8.0",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"author": {
|
||||
"name": "morluto",
|
||||
"url": "https://github.com/morluto"
|
||||
},
|
||||
"homepage": "https://rea.tools/",
|
||||
"repository": "https://github.com/morluto/rea",
|
||||
"license": "MIT",
|
||||
"keywords": [
|
||||
"reverse-engineering",
|
||||
"binary-analysis",
|
||||
"decompiler",
|
||||
"mcp-server"
|
||||
],
|
||||
"skills": "./skills",
|
||||
"mcpServers": "./.codex-plugin/mcp.json",
|
||||
"interface": {
|
||||
"displayName": "REA",
|
||||
"shortDescription": "Reverse engineer anything",
|
||||
"longDescription": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"developerName": "morluto",
|
||||
"category": "Developer Tools",
|
||||
"capabilities": [
|
||||
"Interactive",
|
||||
"Read",
|
||||
"Write"
|
||||
],
|
||||
"websiteURL": "https://rea.tools/",
|
||||
"defaultPrompt": [
|
||||
"Use REA to explain how this binary implements a feature.",
|
||||
"Use REA to compare two versions of this application.",
|
||||
"Use REA to recover the sources from this JavaScript bundle."
|
||||
],
|
||||
"composerIcon": "./assets/rea-plugin.png",
|
||||
"logo": "./assets/rea-plugin.png"
|
||||
}
|
||||
}
|
||||
@@ -1,12 +0,0 @@
|
||||
# Review changes to security-sensitive automation and dependency inputs.
|
||||
/.github/CODEOWNERS @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/dependabot.yml @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/workflows/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/actions/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/verify-tools/skills/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/verify-tools/skills/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/fixtures/http3-lab-requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/website/requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/third_party/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
@@ -1,21 +0,0 @@
|
||||
name: REA production code
|
||||
|
||||
# Match the existing default-setup scan's local-input threat model.
|
||||
threat-models: [local]
|
||||
|
||||
# Retain root traversal, including future production roots. Hidden directories
|
||||
# need their own root; use a directory so Python can share this configuration.
|
||||
paths:
|
||||
- .
|
||||
- docs/.vitepress
|
||||
|
||||
# Keep the default query suite and all production roots, including bridge/,
|
||||
# install.sh, runtime scripts, release controllers, and GitHub Actions.
|
||||
# Exclude test inputs at extraction time so they cannot taint production callers.
|
||||
paths-ignore:
|
||||
- tests
|
||||
- scripts/fixtures
|
||||
- "**/*.test.*"
|
||||
- "**/*.fixture.*"
|
||||
- "**/test_*.py"
|
||||
- "**/*_test.py"
|
||||
@@ -1,33 +0,0 @@
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: npm
|
||||
directories:
|
||||
- /
|
||||
- /scripts/verify-tools/skills
|
||||
schedule:
|
||||
interval: weekly
|
||||
cooldown:
|
||||
default-days: 7
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
npm-minor-and-patch:
|
||||
update-types:
|
||||
- minor
|
||||
- patch
|
||||
|
||||
- package-ecosystem: pip
|
||||
directory: /website
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
|
||||
- package-ecosystem: github-actions
|
||||
directory: /
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
actions-minor-and-patch:
|
||||
update-types:
|
||||
- minor
|
||||
- patch
|
||||
@@ -36,9 +36,6 @@ unchecked. Maintainers can help resolve open questions.
|
||||
Use "none" or "not applicable" only when established. -->
|
||||
|
||||
- Semantic owner and earliest changed stage:
|
||||
<!-- Name the producer of parsed facts or resources and its affected consumers.
|
||||
Summarize superseded paths removed and any distinct policy, lifecycle,
|
||||
lazy-loading, or compatibility responsibility retained. -->
|
||||
- Contribution scope fit and linked maintainer decision:
|
||||
- CLI and MCP/tool-catalog contract:
|
||||
- Provider, bridge, target-format, or platform compatibility:
|
||||
@@ -71,10 +68,6 @@ source, link, or rendered-document checks. -->
|
||||
- User-visible CLI/MCP output (if applicable):
|
||||
- Remaining proof gaps:
|
||||
|
||||
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/testing.md#boundary-regressions)
|
||||
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/testing.md#lifecycle-regressions)
|
||||
covered by the evidence above, and explain relevant gaps.
|
||||
|
||||
For evidence-bearing changes:
|
||||
|
||||
- [ ] Observed, derived, and inferred claims remain distinguishable.
|
||||
@@ -106,8 +99,7 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
|
||||
|
||||
## Author readiness
|
||||
|
||||
<!-- The human owner completes these declarations before requesting review,
|
||||
unless the PR author is exempt as listed in CONTRIBUTING.md.
|
||||
<!-- The human owner completes these declarations before requesting review.
|
||||
The PR author is the owner by default. A different owner must explicitly accept
|
||||
ownership on this PR.
|
||||
AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA.
|
||||
|
||||
+60
-96
@@ -57,13 +57,11 @@ jobs:
|
||||
contents: read
|
||||
actions: read
|
||||
outputs:
|
||||
dependency_security: ${{ steps.plan.outputs.dependency_security }}
|
||||
tier: ${{ steps.plan.outputs.tier }}
|
||||
qualification_run_id: ${{ steps.qualification.outputs.run_id }}
|
||||
code: ${{ steps.plan.outputs.code }}
|
||||
reverse_tools: ${{ steps.plan.outputs.reverse_tools }}
|
||||
python: ${{ steps.plan.outputs.python }}
|
||||
packet_capture: ${{ steps.plan.outputs.packet_capture }}
|
||||
signal_recording: ${{ steps.plan.outputs.signal_recording }}
|
||||
sqlite: ${{ steps.plan.outputs.sqlite }}
|
||||
runtime: ${{ steps.plan.outputs.runtime }}
|
||||
package: ${{ steps.plan.outputs.package }}
|
||||
@@ -147,9 +145,6 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Validate source architecture before dependency installation
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
run: node scripts/verify/architecture-guards.mjs
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
@@ -270,27 +265,6 @@ jobs:
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
python:
|
||||
name: Python producer tests
|
||||
needs: changes
|
||||
if: needs.changes.outputs.python == 'true' && needs.changes.outputs.tier == 'fast'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
# Analysis-efficiency checks import compiled process modules.
|
||||
- name: Reuse portable build outputs across runs
|
||||
uses: ./.github/actions/portable-build-cache
|
||||
- run: npm run test:prepare
|
||||
- run: npm run test:python
|
||||
|
||||
test-source:
|
||||
name: Source tests
|
||||
needs: changes
|
||||
@@ -418,9 +392,7 @@ jobs:
|
||||
needs.changes.outputs.web_network == 'true' ||
|
||||
needs.changes.outputs.web_runtime == 'true' ||
|
||||
needs.changes.outputs.web_source_map == 'true' ||
|
||||
needs.changes.outputs.agentenv == 'true' ||
|
||||
needs.changes.outputs.packet_capture == 'true' ||
|
||||
needs.changes.outputs.signal_recording == 'true'
|
||||
needs.changes.outputs.agentenv == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
@@ -779,23 +751,9 @@ jobs:
|
||||
npm run verify:legacy-simulator
|
||||
npm run verify:apple-dispatch
|
||||
npm run verify:interface-builder
|
||||
- name: Verify portable iNES CLI, MCP and source boundaries
|
||||
run: >-
|
||||
npm run test:focused --
|
||||
tests/acceptance/inesContainer.test.ts
|
||||
tests/boundary/filesystem/inesArtifact.test.ts
|
||||
tests/boundary/filesystem/inesCancellation.test.ts
|
||||
- name: Run native Apple and process boundary regressions
|
||||
run: >-
|
||||
npm run test:focused --
|
||||
tests/boundary/providers/hopper/hopperMacLaunchArguments.test.ts
|
||||
tests/acceptance/providers/hopperStartupRecovery.test.ts
|
||||
tests/boundary/providers/hopper/hopperClient.helperFailure.test.ts
|
||||
tests/boundary/providers/hopper/bridgeBootstrapCleanup.test.ts
|
||||
tests/boundary/providers/hopper/bridgeTargetLease.integration.test.ts
|
||||
tests/acceptance/artifactDmgAttachmentRecoveryCli.test.ts
|
||||
tests/boundary/providers/native/nativeDmgArtifactReader.test.ts
|
||||
tests/process-global/dmgCancellationReason.test.ts
|
||||
tests/boundary/filesystem/installScript.test.ts
|
||||
tests/boundary/filesystem/nibNativeHierarchy.test.ts
|
||||
tests/boundary/filesystem/interfaceBuilderBundle.test.ts
|
||||
@@ -805,13 +763,9 @@ jobs:
|
||||
tests/boundary/cli/nativeDemangleCli.test.ts
|
||||
tests/boundary/cli/nativeDispatchPropertyAttributes.test.ts
|
||||
tests/boundary/mcp/nativeDemangleMcp.test.ts
|
||||
tests/boundary/mcp/nativeMachoSummaryMcp.test.ts
|
||||
tests/boundary/providers/native/nativeSignatureTargetBinding.test.ts
|
||||
tests/boundary/providers/native/nativeUiOffscreenWindow.test.ts
|
||||
tests/boundary/process/nativeUiVerifierLane.test.ts
|
||||
tests/boundary/cli/nativeSignatureCli.test.ts
|
||||
tests/boundary/cli/nativeMachoArm64_32Cli.test.ts
|
||||
tests/boundary/cli/nativeMachoEncryptionCli.test.ts
|
||||
tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts
|
||||
src/process/capture/FilesystemSnapshot.test.ts
|
||||
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
|
||||
@@ -836,7 +790,7 @@ jobs:
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- name: Verify static CLI/MCP journeys and synthetic report conformance
|
||||
- name: Run the public CLI/MCP reconstruction journey
|
||||
env:
|
||||
REA_READINESS_REPORT_PATH: readiness-report.json
|
||||
run: |
|
||||
@@ -899,31 +853,12 @@ jobs:
|
||||
# The matching native artifact establishes real Job Object and private
|
||||
# runtime boundaries for these fixtures. Ghidra and Java are not needed;
|
||||
# real engine claims remain in the separate controlled-provider workflow.
|
||||
# Run native proofs before the broad curated suite so an unrelated
|
||||
# fixture failure cannot hide the real control and recovery checks.
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-process-boundary-fixture
|
||||
path: .cache/windows-process-boundary/
|
||||
# Passing the fixture adds the breakaway, caller/child token, environment
|
||||
# semantics and ancestor reparse observations, which the lane skips
|
||||
# entirely when it is absent.
|
||||
- run: >-
|
||||
npm run verify:windows-native -- .
|
||||
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-native-inspection-fixture
|
||||
path: .cache/windows-native-inspection-fixture/
|
||||
- name: Verify native inspection failure ownership and recovery
|
||||
run: node scripts/verify-windows-native-inspection.mjs
|
||||
- name: Run curated Windows capability tests
|
||||
run: >-
|
||||
npx vitest run --coverage.enabled=false
|
||||
src/process/capture/FilesystemSnapshot.test.ts
|
||||
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
|
||||
tests/boundary/filesystem/typescriptCompiler.test.ts
|
||||
tests/boundary/filesystem/releaseProposal.test.ts
|
||||
tests/boundary/filesystem/binaryTarget.test.ts
|
||||
src/contracts/sessionPathInputs.test.ts
|
||||
tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts
|
||||
@@ -933,7 +868,6 @@ jobs:
|
||||
tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts
|
||||
tests/boundary/filesystem/setupUnavailableProvider.test.ts
|
||||
tests/boundary/filesystem/windowsNpxRegistration.test.ts
|
||||
tests/boundary/filesystem/claudeDesktopSkillUninstallWindows.test.ts
|
||||
tests/boundary/filesystem/clientConfigurationEnvironment.test.ts
|
||||
tests/boundary/filesystem/clientConfigurationLayeredEnvironment.test.ts
|
||||
tests/acceptance/setup/registrationRuntimeEnvironmentCli.test.ts
|
||||
@@ -945,7 +879,6 @@ jobs:
|
||||
tests/boundary/process/packageNpmInvocation.test.ts
|
||||
tests/boundary/process/packedBridgeVerification.test.ts
|
||||
tests/boundary/process/windowsOutputCapture.test.ts
|
||||
tests/boundary/process/windowsExecutableSelection.test.ts
|
||||
tests/acceptance/webModuleCli.test.ts
|
||||
tests/acceptance/webSourceLocationCli.test.ts
|
||||
tests/boundary/cli/referenceSourceImportStatus.test.ts
|
||||
@@ -970,6 +903,22 @@ jobs:
|
||||
tests/process-global/runnerCancellation.test.ts
|
||||
tests/process-global/windowsOwnedCommand.test.ts
|
||||
tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-process-boundary-fixture
|
||||
path: .cache/windows-process-boundary/
|
||||
# Passing the fixture adds the breakaway, caller/child token, environment
|
||||
# semantics and ancestor reparse observations, which the lane skips
|
||||
# entirely when it is absent.
|
||||
- run: >-
|
||||
npm run verify:windows-native -- .
|
||||
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-native-inspection-fixture
|
||||
path: .cache/windows-native-inspection-fixture/
|
||||
- name: Verify native inspection failure ownership and recovery
|
||||
run: node scripts/verify-windows-native-inspection.mjs
|
||||
- name: Verify real Node Inspector discovery and observation
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
@@ -1001,7 +950,7 @@ jobs:
|
||||
retention-days: 30
|
||||
|
||||
docs:
|
||||
name: Documentation
|
||||
name: Docs Generation
|
||||
needs: changes
|
||||
if: needs.changes.outputs.docs == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
@@ -1016,8 +965,6 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- run: npm run docs:check
|
||||
- name: Render documentation and validate page links
|
||||
run: npm run docs:build
|
||||
- name: Generation leaves authored files unchanged
|
||||
run: git diff --exit-code
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
@@ -1130,26 +1077,6 @@ jobs:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
packet_capture:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.packet_capture == 'true'
|
||||
secrets:
|
||||
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
|
||||
uses: ./.github/workflows/real-packet-capture.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
signal_recording:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.signal_recording == 'true'
|
||||
secrets:
|
||||
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
|
||||
uses: ./.github/workflows/real-signal-recording.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
nativeaot:
|
||||
needs: [changes, static]
|
||||
if: needs.changes.outputs.nativeaot == 'true'
|
||||
@@ -1165,6 +1092,45 @@ jobs:
|
||||
if: needs.changes.outputs.website == 'true'
|
||||
uses: ./.github/workflows/website-check.yml
|
||||
|
||||
dependency-security:
|
||||
name: Dependency security
|
||||
needs: changes
|
||||
if: needs.changes.outputs.dependency_security == 'true' || needs.changes.outputs.tier == 'reused'
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
NODE_OPTIONS: --max-old-space-size=1024
|
||||
UV_THREADPOOL_SIZE: "1"
|
||||
TURBO_TOKEN: ""
|
||||
TURBO_CACHE: local:rw
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
with:
|
||||
ref: ${{ inputs.source_sha || github.sha }}
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Validate registry identities before installing dependencies
|
||||
run: node scripts/ci/dependency-security.mjs --mode=preflight
|
||||
- run: npm ci --ignore-scripts --no-audit --no-fund --maxsockets=2
|
||||
- name: Check source lock advisories, installed signatures and identities
|
||||
run: node scripts/ci/dependency-security.mjs --mode=source
|
||||
- name: Check a fresh published consumer daily
|
||||
if: github.event_name == 'schedule' && github.event.schedule == '23 3 * * *'
|
||||
run: node scripts/ci/dependency-security.mjs --mode=published
|
||||
- name: Retain dependency security evidence
|
||||
if: always()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: dependency-security-${{ github.run_attempt }}
|
||||
path: .cache/dependency-security/
|
||||
if-no-files-found: warn
|
||||
retention-days: 14
|
||||
|
||||
required:
|
||||
name: CI required
|
||||
# Report failed dependencies, but release superseded PR concurrency slots.
|
||||
@@ -1172,13 +1138,11 @@ jobs:
|
||||
needs:
|
||||
[
|
||||
changes,
|
||||
dependency-security,
|
||||
static,
|
||||
static-deep,
|
||||
build,
|
||||
reverse-tools-quality,
|
||||
python,
|
||||
packet_capture,
|
||||
signal_recording,
|
||||
test-shard,
|
||||
test-source,
|
||||
sqlite-runtime,
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
name: CodeQL production
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
schedule:
|
||||
- cron: "37 5 * * 1"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: codeql-production-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
# A repository administrator must disable default setup before enabling
|
||||
# this workflow; default setup rejects advanced CodeQL result uploads.
|
||||
if: vars.REA_CODEQL_ADVANCED == 'true'
|
||||
name: Analyze production (${{ matrix.language }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write
|
||||
strategy:
|
||||
fail-fast: false
|
||||
max-parallel: 1
|
||||
matrix:
|
||||
language: [javascript-typescript, python, actions]
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Initialize production analysis
|
||||
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
build-mode: none
|
||||
config-file: .github/codeql/production.yml
|
||||
threads: 2
|
||||
ram: 4096
|
||||
- name: Analyze production code
|
||||
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
|
||||
with:
|
||||
category: /language:${{ matrix.language }}
|
||||
output: ${{ runner.temp }}/codeql-production-results
|
||||
threads: 2
|
||||
ram: 4096
|
||||
- name: Retain complete analysis evidence
|
||||
if: always()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: codeql-production-${{ matrix.language }}
|
||||
path: ${{ runner.temp }}/codeql-production-results
|
||||
if-no-files-found: error
|
||||
retention-days: 7
|
||||
@@ -62,36 +62,3 @@ jobs:
|
||||
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
|
||||
cat "${probe_root}/output.log"
|
||||
grep -F 'REA_ARM_CONTEXT_OK arm=0 thumb=1 unavailable=null (injected provider value)' "${probe_root}/output.log"
|
||||
# Each probe imports source-owned bytes at 0x1000, away from the Cortex-M
|
||||
# vector symbols that ARMCortex.pspec defines at 0, and decodes them from
|
||||
# the language's default context, so M-profile Thumb state must hold at
|
||||
# the first byte. The bytes are never executed.
|
||||
- name: Decode M-profile ARM and Xtensa bytes in their default context
|
||||
shell: bash
|
||||
run: |
|
||||
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
|
||||
export JAVA_HOME="${!java_home_variable}"
|
||||
probe_root="${RUNNER_TEMP}/rea-decode"
|
||||
mkdir -p "${probe_root}"
|
||||
# mrs r0, msp; bx lr
|
||||
printf '\357\363\010\200\160\107' > "${probe_root}/cortex.bin"
|
||||
# mrs r0, msplim (ARMv8-M only); bx lr
|
||||
printf '\357\363\012\200\160\107' > "${probe_root}/v8m.bin"
|
||||
# entry a1, 32; retw.n
|
||||
printf '\066\101\000\035\360' > "${probe_root}/xtensa.bin"
|
||||
decode() {
|
||||
local name="$1" language="$2"
|
||||
shift 2
|
||||
mkdir -p "${probe_root}/${name}-project"
|
||||
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/${name}-project" Decode \
|
||||
-import "${probe_root}/${name}.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
|
||||
-processor "${language}" -noanalysis -readOnly -deleteProject \
|
||||
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra" \
|
||||
-postScript ReaDecodeProbe.java "${language}" "$@" \
|
||||
> "${probe_root}/${name}.log" 2>&1 || { cat "${probe_root}/${name}.log"; exit 1; }
|
||||
cat "${probe_root}/${name}.log"
|
||||
grep -F "REA_DECODE_OK ${language} $*" "${probe_root}/${name}.log"
|
||||
}
|
||||
decode cortex ARM:LE:32:Cortex mrs/4/msp bx/2/lr
|
||||
decode v8m ARM:LE:32:v8-m mrs/4/msplim bx/2/lr
|
||||
decode xtensa Xtensa:LE:32:default entry/3/a1 retw.n/2
|
||||
@@ -1,63 +0,0 @@
|
||||
name: Real Android metadata bridge
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ["**"]
|
||||
paths:
|
||||
- .github/workflows/real-android-metadata.yml
|
||||
- bridge/android/**
|
||||
- src/android/JadxRelease.ts
|
||||
- scripts/fixtures/android/**
|
||||
- tests/conformance/android/**
|
||||
- tests/fixtures/android/static-initializers/**
|
||||
- tests/fixtures/android/direct-invokes/**
|
||||
- tests/fixtures/android/external-invokes/**
|
||||
- .nvmrc
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: android-metadata-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
android-metadata:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Fetch checksum-pinned engine and public APK fixture
|
||||
run: node scripts/fixtures/android/fetch.mjs "$RUNNER_TEMP/rea-android"
|
||||
- name: Compile and exercise the real metadata bridge
|
||||
shell: bash
|
||||
run: |
|
||||
export JAVA_HOME="$JAVA_HOME_21_X64"
|
||||
android_root="$RUNNER_TEMP/rea-android"
|
||||
classes="$android_root/classes"
|
||||
jar="$android_root/jadx-headless-mcp-0.7.1-all.jar"
|
||||
mkdir -p "$classes"
|
||||
"$JAVA_HOME/bin/javac" -J-Xmx512m -J-XX:ActiveProcessorCount=2 \
|
||||
-classpath "$jar" -d "$classes" \
|
||||
bridge/android/ReaJadxBridge.java \
|
||||
tests/conformance/android/ReaJadxMetadataRegression.java \
|
||||
tests/conformance/android/ReaJadxExternalReferencesRegression.java
|
||||
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
|
||||
-classpath "$classes:$jar" ReaJadxMetadataRegression \
|
||||
"$android_root/ApiDemos-debug.apk" \
|
||||
"$GITHUB_WORKSPACE/tests/fixtures/android/static-initializers" \
|
||||
"$classes" "$GITHUB_WORKSPACE/tests/fixtures/android/direct-invokes" \
|
||||
| tee "$android_root/metadata.log"
|
||||
grep -F 'RAW_CLASS_SELECTOR_REGRESSION_PASS nested=true metadata=true references=true' \
|
||||
"$android_root/metadata.log"
|
||||
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
|
||||
-classpath "$classes:$jar" ReaJadxExternalReferencesRegression \
|
||||
"$GITHUB_WORKSPACE/tests/fixtures/android/external-invokes" "$classes" \
|
||||
| tee "$android_root/external-references.log"
|
||||
grep -F 'EXTERNAL_REFERENCE_REGRESSION_PASS prototypes=2 using_nodes=3 sites=4' \
|
||||
"$android_root/external-references.log"
|
||||
@@ -55,12 +55,6 @@ jobs:
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Install AArch64 and ELF32 cross toolchains for source-owned fixtures
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --yes --no-install-recommends \
|
||||
gcc-aarch64-linux-gnu gcc-i686-linux-gnu gcc-arm-linux-gnueabi gcc-mips-linux-gnu gcc-mipsel-linux-gnu \
|
||||
libc6-dev-arm64-cross libc6-dev-i386-cross libc6-dev-armel-cross libc6-dev-mips-cross libc6-dev-mipsel-cross
|
||||
- name: Acquire exact upstream libraries in an isolated runner environment
|
||||
shell: bash
|
||||
run: |
|
||||
|
||||
@@ -48,13 +48,7 @@ jobs:
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
# Own a fresh bus so Chromium never depends on an inherited session address.
|
||||
- run: dbus-run-session -- npm run verify:browser
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
# The preceding verifier builds the runtime used by this request journey.
|
||||
- run: dbus-run-session -- node scripts/verify-browser-request.mjs
|
||||
- run: npm run verify:browser
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
@@ -1,35 +1,39 @@
|
||||
name: Real packet capture
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/captures/**"
|
||||
- "tests/boundary/cli/captureCleanup.test.ts"
|
||||
- "tests/boundary/mcp/packetCaptureMcp.test.ts"
|
||||
- "src/captures/**"
|
||||
- "src/pcap/**"
|
||||
- "src/domain/pcap/**"
|
||||
- "src/domain/analysisView/pcapCaptureView.ts"
|
||||
- "src/application/pcap/**"
|
||||
- "src/contracts/pcap/**"
|
||||
- "src/server/registerPcapTools.ts"
|
||||
- "tests/boundary/pcap/**"
|
||||
- "src/domain/captures/**"
|
||||
- "src/application/captures/**"
|
||||
- "src/contracts/captures/**"
|
||||
- "src/composition/captures.ts"
|
||||
- "src/cli/captureCommands.ts"
|
||||
- "src/server/registerCaptureTools.ts"
|
||||
- "scripts/verify-packet-capture.mjs"
|
||||
- ".github/workflows/real-packet-capture.yml"
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
concurrency:
|
||||
group: real-packet-capture-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
tshark:
|
||||
runs-on: ubuntu-24.04
|
||||
@@ -44,18 +48,8 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run build:cached
|
||||
- name: Run focused tests and require Linux resource and cleanup cases
|
||||
run: |
|
||||
report=".cache/rea-packet-required-tests.json"
|
||||
mkdir -p .cache
|
||||
rm -f "$report"
|
||||
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
|
||||
tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts tests/acceptance/pcapVerifier.test.ts
|
||||
node scripts/ci/require-provider-tests.mjs packet "$report"
|
||||
- run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
|
||||
- run: node scripts/verify-packet-capture.mjs
|
||||
env:
|
||||
REA_TSHARK_COMMAND: /usr/bin/tshark
|
||||
|
||||
@@ -1,35 +1,32 @@
|
||||
name: Real signal recording
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/signals/**"
|
||||
- "tests/boundary/cli/signalCleanup.test.ts"
|
||||
- "tests/boundary/mcp/signalRecordingMcp.test.ts"
|
||||
- "src/signals/**"
|
||||
- "src/domain/signals/**"
|
||||
- "src/application/signals/**"
|
||||
- "src/contracts/signals/**"
|
||||
- "src/composition/signals.ts"
|
||||
- "src/cli/signalCommands.ts"
|
||||
- "src/server/registerSignalTools.ts"
|
||||
- "scripts/verify-signal-recording.mjs"
|
||||
- ".github/workflows/real-signal-recording.yml"
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
concurrency:
|
||||
group: real-signal-recording-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
sigrok:
|
||||
runs-on: ubuntu-24.04
|
||||
@@ -44,18 +41,8 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run build:cached
|
||||
- name: Run focused tests and require Linux resource and cleanup cases
|
||||
run: |
|
||||
report=".cache/rea-signal-required-tests.json"
|
||||
mkdir -p .cache
|
||||
rm -f "$report"
|
||||
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
|
||||
tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
|
||||
node scripts/ci/require-provider-tests.mjs signal "$report"
|
||||
- run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
|
||||
- run: node scripts/verify-signal-recording.mjs
|
||||
env:
|
||||
REA_SIGROK_COMMAND: /usr/bin/sigrok-cli
|
||||
@@ -44,8 +44,7 @@ jobs:
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
# Own a fresh bus so Chromium never depends on an inherited session address.
|
||||
- run: dbus-run-session -- npm run verify:browser:runtime
|
||||
- run: env -u DBUS_SESSION_BUS_ADDRESS npm run verify:browser:runtime
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
@@ -46,12 +46,8 @@ jobs:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Install the pinned compiler into an isolated fixture directory
|
||||
run: npm install --prefix "$RUNNER_TEMP/rea-web-source-map-compiler" --no-audit --no-fund --package-lock=false [email protected]
|
||||
# Own a fresh bus so Chromium never depends on an inherited session address.
|
||||
- run: dbus-run-session -- npm run verify:browser:source-maps
|
||||
- run: npm run verify:browser:source-maps
|
||||
env:
|
||||
# Retain browser launch argv/PID/stderr in the verifier's captured
|
||||
# CLI failure diagnostics without changing its deadline or retrying.
|
||||
DEBUG: pw:browser
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
REA_WEB_SOURCE_MAP_COMPILER: ${{ runner.temp }}/rea-web-source-map-compiler/node_modules/esbuild/lib/main.js
|
||||
@@ -77,15 +77,26 @@ jobs:
|
||||
- name: Rehearse the original qualified tarball
|
||||
env:
|
||||
RECOVERY_VERSION: ${{ steps.release.outputs.version }}
|
||||
run: node scripts/verify-release-artifact.mjs .cache/verified-package/package.tgz "$RECOVERY_VERSION"
|
||||
RECOVERY_SOURCE_SHA: ${{ steps.release.outputs.source_sha }}
|
||||
run: node scripts/verify-release-artifact.mjs .cache/verified-package/package.tgz "$RECOVERY_VERSION" --dependency-security=.cache/dependency-security --dependency-source=.cache/release-source --dependency-source-sha="$RECOVERY_SOURCE_SHA"
|
||||
- name: Verify published CLI, MCP and update
|
||||
env:
|
||||
RECOVERY_VERSION: ${{ steps.release.outputs.version }}
|
||||
run: node scripts/verify-published-package.mjs "$RECOVERY_VERSION"
|
||||
RECOVERY_SOURCE_SHA: ${{ steps.release.outputs.source_sha }}
|
||||
run: node scripts/verify-published-package.mjs "$RECOVERY_VERSION" --dependency-security=.cache/dependency-security --dependency-source=.cache/release-source --dependency-source-sha="$RECOVERY_SOURCE_SHA"
|
||||
- name: Validate released registry metadata
|
||||
working-directory: .cache/release-source
|
||||
run: npm run mcp-registry:check
|
||||
|
||||
- name: Retain recovery consumer dependency security evidence
|
||||
if: always()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: recovery-consumer-security-${{ github.run_attempt }}
|
||||
path: .cache/dependency-security/
|
||||
if-no-files-found: warn
|
||||
retention-days: 14
|
||||
|
||||
publish-mcp:
|
||||
name: Finish MCP Registry publication
|
||||
needs: verify
|
||||
|
||||
@@ -47,63 +47,19 @@ jobs:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
run: |
|
||||
# Filter at the repository connection, without a search-index dependency.
|
||||
pages=$(gh api graphql --paginate --slurp \
|
||||
-f owner="${GH_REPO%/*}" -f name="${GH_REPO#*/}" \
|
||||
-f query='query($owner:String!,$name:String!,$endCursor:String) {
|
||||
repository(owner:$owner,name:$name) {
|
||||
nameWithOwner
|
||||
pullRequests(first:100,after:$endCursor,states:OPEN,baseRefName:"main",labels:["rea:release-notes-finalized"]) {
|
||||
nodes { number headRefName headRepository { nameWithOwner } }
|
||||
pageInfo { hasNextPage endCursor }
|
||||
}
|
||||
}
|
||||
}')
|
||||
finalized=$(jq -er '
|
||||
if type != "array" or length == 0 then error("Missing GitHub pages") else . end |
|
||||
if .[-1].data.repository.pullRequests.pageInfo.hasNextPage != false
|
||||
then error("Incomplete GitHub pagination") else . end |
|
||||
map(
|
||||
if (.errors // [] | length) != 0 then error("GitHub query failed") else . end |
|
||||
.data.repository |
|
||||
if (.nameWithOwner | type) != "string" or (.pullRequests.nodes | type) != "array" or
|
||||
(.pullRequests.pageInfo.hasNextPage | type) != "boolean"
|
||||
then error("Invalid GitHub repository connection") else . end |
|
||||
.nameWithOwner as $repo | .pullRequests.nodes[] |
|
||||
if (.number | type) != "number" or (.headRefName | type) != "string" or (has("headRepository") | not) or
|
||||
(.headRepository != null and (.headRepository.nameWithOwner | type) != "string")
|
||||
then error("Invalid GitHub pull request") else . end |
|
||||
select(.headRepository.nameWithOwner == $repo) |
|
||||
select(.headRefName | startswith("release-please--branches--main--")) | .number
|
||||
) | map(tostring) | join(" ")' <<< "$pages")
|
||||
finalized=$(gh api --paginate "repos/${GH_REPO}/pulls?state=open&base=main&per_page=100" \
|
||||
--jq '.[] | select(.head.repo.full_name == .base.repo.full_name) | select(.head.ref | startswith("release-please--branches--main--")) | select(any(.labels[]; .name == "rea:release-notes-finalized")) | .number')
|
||||
if [[ -n "${finalized}" ]]; then
|
||||
echo "Finalized release notes on PR ${finalized}; automatic proposal refresh is paused." >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "frozen=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Check out proposal controller and complete Git history
|
||||
if: steps.editorial.outputs.frozen != 'true'
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
|
||||
if: steps.editorial.outputs.frozen != 'true'
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
|
||||
- name: Install proposal controller dependencies
|
||||
if: steps.editorial.outputs.frozen != 'true'
|
||||
run: npm ci
|
||||
|
||||
- name: Update release proposal
|
||||
if: steps.editorial.outputs.frozen != 'true'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
SOURCE_SHA: ${{ github.sha }}
|
||||
run: node scripts/release-proposal.mjs --target-branch main --source-sha "$SOURCE_SHA"
|
||||
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
target-branch: main
|
||||
skip-github-release: true
|
||||
|
||||
merged-release:
|
||||
name: Release reviewed main merge
|
||||
@@ -276,25 +232,15 @@ jobs:
|
||||
node scripts/release-notes.mjs publication --repository "$GH_REPO" \
|
||||
--source "$SOURCE_SHA"
|
||||
|
||||
- name: Prepare checkpoint release proposal
|
||||
if: inputs.phase == 'prepare'
|
||||
id: proposal
|
||||
env:
|
||||
# A dedicated credential starts PR CI automatically; the built-in
|
||||
# token creates approval-required PR runs for maintainers to approve.
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
RELEASE_BRANCH: ${{ inputs.release_branch }}
|
||||
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
|
||||
run: node scripts/release-proposal.mjs --target-branch "$RELEASE_BRANCH" --source-sha "$SOURCE_SHA"
|
||||
|
||||
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
|
||||
if: inputs.phase == 'publish'
|
||||
id: release
|
||||
with:
|
||||
# A dedicated credential starts PR CI automatically; the built-in
|
||||
# token creates approval-required PR runs for maintainers to approve.
|
||||
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
target-branch: ${{ inputs.release_branch }}
|
||||
skip-github-release: false
|
||||
skip-github-pull-request: true
|
||||
skip-github-release: ${{ inputs.phase == 'prepare' }}
|
||||
skip-github-pull-request: ${{ inputs.phase == 'publish' }}
|
||||
|
||||
- name: Require a merged release PR for publication
|
||||
if: inputs.phase == 'publish' && steps.release.outputs.release_created != 'true'
|
||||
@@ -314,26 +260,26 @@ jobs:
|
||||
fi
|
||||
|
||||
- name: Check out release pull request
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ fromJSON(steps.proposal.outputs.pr).headBranchName }}
|
||||
ref: ${{ fromJSON(steps.release.outputs.pr).headBranchName }}
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Node.js for generated documentation
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
|
||||
- name: Install dependencies
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm ci
|
||||
|
||||
- name: Validate prepared release version and migration notes
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
env:
|
||||
RELEASE_BRANCH: ${{ inputs.release_branch }}
|
||||
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
|
||||
@@ -343,11 +289,11 @@ jobs:
|
||||
--source-sha "$SOURCE_SHA"
|
||||
|
||||
- name: Regenerate release documentation
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm run docs:generate
|
||||
|
||||
- name: Validate generated release documentation
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm run docs:check && git diff --exit-code
|
||||
|
||||
qualification:
|
||||
@@ -396,7 +342,16 @@ jobs:
|
||||
env:
|
||||
VERIFIED_PACKAGE_TARBALL: ${{ steps.package.outputs.tarball }}
|
||||
VERIFIED_PACKAGE_VERSION: ${{ steps.package.outputs.version }}
|
||||
run: node scripts/verify-release-artifact.mjs "$VERIFIED_PACKAGE_TARBALL" "$VERIFIED_PACKAGE_VERSION"
|
||||
run: node scripts/verify-release-artifact.mjs "$VERIFIED_PACKAGE_TARBALL" "$VERIFIED_PACKAGE_VERSION" --dependency-security=.cache/dependency-security
|
||||
|
||||
- name: Retain consumer dependency security evidence
|
||||
if: always()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: qualified-consumer-security-${{ github.run_attempt }}
|
||||
path: .cache/dependency-security/
|
||||
if-no-files-found: warn
|
||||
retention-days: 14
|
||||
|
||||
publish:
|
||||
name: Publish to npm
|
||||
@@ -469,7 +424,16 @@ jobs:
|
||||
- name: Verify published CLI from npm
|
||||
env:
|
||||
VERIFIED_PACKAGE_VERSION: ${{ steps.package.outputs.version }}
|
||||
run: node scripts/verify-published-package.mjs "$VERIFIED_PACKAGE_VERSION"
|
||||
run: node scripts/verify-published-package.mjs "$VERIFIED_PACKAGE_VERSION" --dependency-security=.cache/dependency-security
|
||||
|
||||
- name: Retain consumer dependency security evidence
|
||||
if: always()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: published-consumer-security-${{ github.run_attempt }}
|
||||
path: .cache/dependency-security/
|
||||
if-no-files-found: warn
|
||||
retention-days: 14
|
||||
|
||||
publish-mcp:
|
||||
name: Publish to MCP Registry
|
||||
|
||||
@@ -17,10 +17,10 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
# Exercise unavailable-runtime diagnostics and rolling supported releases.
|
||||
# Security floors prevent fallback to an older hosted-toolcache patch.
|
||||
# Exercise unavailable-runtime diagnostics, the first supported release,
|
||||
# and new patches on both supported lines without hand-editing versions.
|
||||
# The repository-pinned Node 24 version also runs the complete test suite.
|
||||
node: ["^22.23.4", "24.22.1", "^24.22.1", "^26.12.1"]
|
||||
node: ["22.x", "24.22.1", "24.x", "26.x"]
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
|
||||
@@ -21,10 +21,6 @@ jobs:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Set up Node.js for the website Worker checks
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
|
||||
@@ -54,10 +54,6 @@ jobs:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Set up Node.js for the website Worker checks
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
@@ -79,21 +75,10 @@ jobs:
|
||||
PY
|
||||
- name: Configure Pages
|
||||
uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
|
||||
# upload-pages-artifact excludes dot directories, including .well-known.
|
||||
- name: Package the website including agent discovery
|
||||
run: |
|
||||
tar --dereference --hard-dereference --directory website/_site \
|
||||
--exclude=.git --exclude=.github -cf "$RUNNER_TEMP/artifact.tar" .
|
||||
tar -tf "$RUNNER_TEMP/artifact.tar" \
|
||||
./.well-known/ard.json ./.well-known/ai-catalog.json \
|
||||
./.well-known/agent-skills/index.json > /dev/null
|
||||
- name: Upload the website
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0
|
||||
with:
|
||||
name: github-pages
|
||||
path: ${{ runner.temp }}/artifact.tar
|
||||
retention-days: 1
|
||||
if-no-files-found: error
|
||||
path: website/_site
|
||||
|
||||
deploy:
|
||||
needs: prepare
|
||||
@@ -145,24 +130,14 @@ jobs:
|
||||
PUBLISH_TARGET: ${{ inputs.target }}
|
||||
run: |
|
||||
python3 -u - <<'PY'
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from urllib.error import URLError
|
||||
from urllib.parse import urljoin, urlsplit
|
||||
from urllib.parse import urljoin
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
def open_public(url, accept=None):
|
||||
headers = {
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
}
|
||||
if accept:
|
||||
headers["Accept"] = accept
|
||||
return urlopen(Request(url, headers=headers), timeout=10)
|
||||
|
||||
expected = os.environ["GITHUB_SHA"]
|
||||
pages_url = os.environ["PAGES_URL"]
|
||||
if not pages_url:
|
||||
@@ -178,7 +153,11 @@ jobs:
|
||||
for name, base in list(pending.items()):
|
||||
url = urljoin(base, "deployment.json") + "?commit=" + expected
|
||||
try:
|
||||
with open_public(url) as response:
|
||||
request = Request(url, headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
})
|
||||
with urlopen(request, timeout=10) as response:
|
||||
actual = json.load(response).get("commit")
|
||||
if actual == expected:
|
||||
print(f"{name} serves {expected}: {base}")
|
||||
@@ -193,27 +172,6 @@ jobs:
|
||||
time.sleep(5)
|
||||
if pending:
|
||||
sys.exit("Could not verify published revision on: " + ", ".join(pending))
|
||||
for name, base in hosts.items():
|
||||
with open_public(urljoin(base, ".well-known/agent-skills/index.json")) as response:
|
||||
skill, = json.load(response)["skills"]
|
||||
# Fetch from each selected host, including the /rea/ mirror.
|
||||
artifact_url = urljoin(base, urlsplit(skill["url"]).path.lstrip("/"))
|
||||
with open_public(artifact_url) as response:
|
||||
digest = "sha256:" + hashlib.sha256(response.read()).hexdigest()
|
||||
if digest != skill["digest"]:
|
||||
sys.exit(name + " serves a skill archive with the wrong digest")
|
||||
for manifest in ("ard.json", "ai-catalog.json"):
|
||||
with open_public(urljoin(base, ".well-known/" + manifest)) as response:
|
||||
if not json.load(response).get("entries"):
|
||||
sys.exit(name + " serves an empty discovery manifest")
|
||||
if name == "Cloudflare":
|
||||
with open_public(base, accept="text/markdown") as response:
|
||||
vary = {value.strip().lower() for value in response.headers.get("Vary", "").split(",")}
|
||||
if response.headers.get_content_type() != "text/markdown" or "accept" not in vary:
|
||||
sys.exit("Cloudflare did not negotiate Markdown with Vary: Accept")
|
||||
if 'rel="ard"' not in response.headers.get("Link", ""):
|
||||
sys.exit("Cloudflare is missing its discovery Link header")
|
||||
print(name + " serves the discovery manifests and verified skill archive")
|
||||
with open(os.environ["GITHUB_STEP_SUMMARY"], "a", encoding="utf-8") as summary:
|
||||
links = " and ".join(f"[{name}]({url})" for name, url in hosts.items())
|
||||
summary.write(f"Published commit `{expected}` to {links}.\n")
|
||||
|
||||
+1
-1
@@ -50,7 +50,7 @@ Thumbs.db
|
||||
|
||||
# Local reference sources and analysis artifacts
|
||||
/_reference/
|
||||
/experiments/analysis-efficiency/
|
||||
/experiments/analysis-efficiency/results/
|
||||
|
||||
# Generated website download
|
||||
/website/public/examples/notes-example.zip
|
||||
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"threshold": 4,
|
||||
"threshold": 3.5,
|
||||
"reporters": ["consoleFull"],
|
||||
"ignore": [
|
||||
"node_modules/**",
|
||||
|
||||
+1
-4
@@ -10,10 +10,7 @@
|
||||
"node_modules/**",
|
||||
"CHANGELOG.md",
|
||||
"package-lock.json",
|
||||
".claude-plugin/plugin.json",
|
||||
".codex-plugin/plugin.json",
|
||||
"_reference/**",
|
||||
"third_party/**",
|
||||
".codex-plugin/mcp.json"
|
||||
"third_party/**"
|
||||
]
|
||||
}
|
||||
+3
-95
@@ -1,13 +1,7 @@
|
||||
{
|
||||
"$schema": "./node_modules/oxlint/configuration_schema.json",
|
||||
"plugins": ["typescript", "import", "unicorn", "oxc", "vitest"],
|
||||
"plugins": ["typescript", "import", "unicorn", "oxc"],
|
||||
"rules": {
|
||||
"rea/no-buffer-deep-equality": "error",
|
||||
"rea/cancellation-cause": "error",
|
||||
"no-async-promise-executor": "error",
|
||||
"no-unreachable": "error",
|
||||
"no-unsafe-optional-chaining": "error",
|
||||
"no-useless-escape": ["warn", { "allowRegexCharacters": ["-"] }],
|
||||
"complexity": ["warn", 20],
|
||||
"max-depth": ["warn", 4],
|
||||
"max-lines": [
|
||||
@@ -29,38 +23,7 @@
|
||||
"max-params": ["warn", 4],
|
||||
"no-var": "warn",
|
||||
"no-var-requires": "error",
|
||||
"typescript/no-import-type-side-effects": "error",
|
||||
"vitest/no-standalone-expect": "off",
|
||||
"vitest/no-conditional-expect": "off",
|
||||
"vitest/expect-expect": "off",
|
||||
"vitest/no-disabled-tests": "off",
|
||||
"vitest/require-mock-type-parameters": "off",
|
||||
"vitest/require-to-throw-message": "off",
|
||||
"import/no-duplicates": [
|
||||
"warn",
|
||||
{
|
||||
"preferInline": false
|
||||
}
|
||||
],
|
||||
"vitest/valid-expect": [
|
||||
"warn",
|
||||
{
|
||||
"maxArgs": 2
|
||||
}
|
||||
],
|
||||
"rea/no-evidence-json-preadmission": "error",
|
||||
"rea/no-positional-tool-contract": "error",
|
||||
"rea/no-fixture-root-cleanup": "error",
|
||||
"typescript/no-floating-promises": "error",
|
||||
"typescript/no-misused-promises": "warn",
|
||||
"typescript/return-await": ["error", "error-handling-correctness-only"],
|
||||
"typescript/switch-exhaustiveness-check": [
|
||||
"error",
|
||||
{
|
||||
"considerDefaultExhaustiveForUnions": true
|
||||
}
|
||||
],
|
||||
"rea/no-shared-module-mock": "error"
|
||||
"typescript/no-import-type-side-effects": "error"
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
@@ -73,12 +36,6 @@
|
||||
"no-var-requires": "off"
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": ["src/**/*.ts"],
|
||||
"rules": {
|
||||
"typescript/no-misused-promises": "error"
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": ["tests/**/*.ts", "src/**/*.test.ts", "src/**/*.fixture.ts"],
|
||||
"rules": {
|
||||
@@ -90,8 +47,7 @@
|
||||
"skipBlankLines": true,
|
||||
"skipComments": true
|
||||
}
|
||||
],
|
||||
"typescript/no-misused-promises": "warn"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
@@ -135,45 +91,6 @@
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": ["tests/**/*.test.ts", "src/**/*.test.ts"],
|
||||
"rules": {
|
||||
"vitest/valid-expect": [
|
||||
"warn",
|
||||
{
|
||||
"maxArgs": 2
|
||||
}
|
||||
],
|
||||
"vitest/valid-expect-in-promise": "error",
|
||||
"vitest/no-focused-tests": "error"
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": [
|
||||
"scripts/lib/http3-capture-assertions.test.mjs",
|
||||
"scripts/lib/http3-verifier-lifecycle.test.mjs"
|
||||
],
|
||||
"rules": {
|
||||
"typescript/no-floating-promises": [
|
||||
"error",
|
||||
{
|
||||
"allowForKnownSafeCalls": [
|
||||
{
|
||||
"from": "package",
|
||||
"package": "@types/node",
|
||||
"name": "test"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": ["scripts/fixtures/javascript-recovery/code-split/entry.mjs"],
|
||||
"rules": {
|
||||
"typescript/no-floating-promises": "off"
|
||||
}
|
||||
}
|
||||
],
|
||||
"env": {
|
||||
@@ -192,14 +109,5 @@
|
||||
".devin",
|
||||
".factory",
|
||||
"third_party"
|
||||
],
|
||||
"options": {
|
||||
"reportUnusedDisableDirectives": "error"
|
||||
},
|
||||
"jsPlugins": [
|
||||
{
|
||||
"name": "rea",
|
||||
"specifier": "./scripts/lint/index.mjs"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "6.8.0"
|
||||
".": "6.7.0"
|
||||
}
|
||||
@@ -28,10 +28,6 @@ Assess brittleness in touched code during implementation, debugging, review, and
|
||||
|
||||
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
|
||||
|
||||
After confirming a bug, search sibling adapters, consumers and similar workflows
|
||||
for the same failure. Verify suspected matches, record working comparisons, and
|
||||
group findings by root cause instead of filing duplicate reports.
|
||||
|
||||
When addressing a review thread, inspect the producing boundary and affected consumers for brittleness. Complete directly related refactoring in the same change. Before resolving the thread, explain the root cause, structural fix, validation, and any remaining cleanup.
|
||||
|
||||
Parse unknown boundary values and model expected failures with the tagged error algebra and `Result`. Preserve partial facts and meaningful failure reasons. Keep identity and lookup values separate from display formatting; preserve source values and explicit unknowns when normalization loses information. Absolute paths, file URLs, and HTTP paths have different semantics.
|
||||
@@ -57,12 +53,9 @@ Optimize in this order:
|
||||
3. CPU and memory. Avoid unnecessary work and copies; bound concurrency,
|
||||
retained state and serialization growth.
|
||||
|
||||
Reserve performance measurements for changes that target performance or have a
|
||||
concrete regression concern. Routine cleanup uses correctness checks; changes
|
||||
to helper calls, imports, or representations alone do not require synthetic
|
||||
benchmarks. When measurement is warranted, measure representative completed
|
||||
tasks and compare equivalent workflows before and after where available. Report
|
||||
token usage, elapsed time, CPU time and peak process-family memory. Separate cold
|
||||
For changes affecting these costs, measure representative completed tasks and
|
||||
compare equivalent workflows before and after where available. Report token
|
||||
usage, elapsed time, CPU time and peak process-family memory. Separate cold
|
||||
and warm runs, label estimates, and state unmeasured costs. Fewer bytes alone
|
||||
do not establish fewer tokens or faster analysis.
|
||||
|
||||
@@ -79,24 +72,13 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
|
||||
|
||||
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
|
||||
|
||||
When changing installers or setup recovery, follow
|
||||
[Checking installers and publisher downloads](docs/testing.md#checking-installers-and-publisher-downloads).
|
||||
|
||||
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/testing.md#developer-commands).
|
||||
|
||||
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
|
||||
|
||||
## Tool and Test Changes
|
||||
|
||||
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
|
||||
|
||||
For user- or agent-facing tool problems, prioritize fixing the owning behavior,
|
||||
contract, and actionable result or recovery at the point of use. Document the
|
||||
result in product docs when explanation is needed. Add skill guidance only for
|
||||
workflow judgment or context that the tool cannot reasonably convey; do not use
|
||||
a skill as the primary repair for a discoverability or contract defect.
|
||||
|
||||
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Select applicable [boundary regressions](docs/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
|
||||
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
|
||||
|
||||
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
|
||||
|
||||
@@ -124,12 +106,11 @@ distinguish source-derived conclusions, proposed checks, and unavailable coverag
|
||||
Address actionable automated review findings and explain disagreements.
|
||||
|
||||
Leave the template's **Author readiness** declarations unchecked for the human
|
||||
owner to complete. Never attest to human review on their behalf. Draft status
|
||||
is optional; preserve the PR's current draft status unless the user asks to change
|
||||
it. Later pushes do not invalidate the readiness declaration or require another
|
||||
human sign-off. Keep the PR body and affected validation current as work changes.
|
||||
The readiness check validates the filled declaration or the author exemption
|
||||
listed in CONTRIBUTING.md; CI and automated review results do not establish that
|
||||
owner to complete. Never attest to human review on their behalf. Keep the PR in
|
||||
draft until the human declares the PR initially ready. Later pushes do not
|
||||
invalidate that declaration or require another human sign-off. Keep the PR body
|
||||
and affected validation current as work changes. The readiness check validates
|
||||
the filled declaration; CI and automated review results do not establish that
|
||||
human review occurred, maintainer scope approval, or merge permission.
|
||||
|
||||
<!-- BEGIN:turborepo-agent-rules -->
|
||||
|
||||
+3
-105
@@ -1,107 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## [6.8.0](https://github.com/morluto/rea/compare/rea-agents-6.7.0...rea-agents-6.8.0) (2026-10-25)
|
||||
|
||||
### Highlights
|
||||
|
||||
- Build local agent environments from network captures, browser scenarios, or a prompt, then serve derived reads and evidence-gated writes and compare them with fresh application captures.
|
||||
- Inspect more shipped formats, including GPU code objects, BEAM, Python bytecode, Unity IL2CPP metadata, Godot PCK, PyInstaller, Node.js SEA, devicetree, and additional PE/ELF tables.
|
||||
- Keep large investigations focused with retained summaries, selected fact queries, configurable recovery limits, and row-oriented CLI exports. Complete Evidence remains available.
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
- **Function analysis:** `analyze_function` now defaults to a retained summary. Set `detail: "complete"` for the previous inline dossier; the CLI retains its complete default and requires `--evidence_output` when selecting a summary. [#5660](https://github.com/morluto/rea/pull/5660). Thanks @CryptoJones, @gensokyo-dev, @RishTheFish21.
|
||||
- **Accessibility:** Accept JSON numbers and booleans in node `value`, alongside strings and null. Update consumers that previously assumed text-only values. [cc90771](https://github.com/morluto/rea/commit/cc90771af14f045538651615147d8bf86b576549), [#6759](https://github.com/morluto/rea/pull/6759), [#6467](https://github.com/morluto/rea/issues/6467). Thanks @coygeek, @gensokyo-dev, @mvanhorn.
|
||||
- **JSON Lines:** `--format jsonl` now emits object-collection rows followed by a `$rea: "jsonl/v1"`, `type: "metadata"` record; scalar selections and unsupported reports are refused. Parse control records separately, select a single supported collection when necessary, and use `--format json` for the previous whole-document output. See [JSON Lines framing](https://github.com/morluto/rea/blob/rea-agents-6.8.0/docs/cli.md#json-lines-record-output). [#5629](https://github.com/morluto/rea/pull/5629), [#5487](https://github.com/morluto/rea/issues/5487). Thanks @gensokyo-dev, @jayykernel, @vijaykiran06.
|
||||
- **Interface Builder:** Keyed-archive graph identities use decimal table UIDs instead of authored `objectID`/`object-id`. Treat authored IDs as attributes and update persisted graph lookups; ibtool and native NIBArchive identities keep their source semantics. [6fbe67d](https://github.com/morluto/rea/commit/6fbe67d66e3d716db69929503a3a232a7ada0860).
|
||||
- **Frida:** `instrument-with-frida` JSON returns the canonical Evidence directly. Read `normalized_result.cleanup_error` instead of the former top-level wrapper field. [48f700e](https://github.com/morluto/rea/commit/48f700ee35e0988ed973e5486dc9db33f2a5d944).
|
||||
- **Reconstruction proof:** ID-array proof records no longer establish verified or ready ledgers. Supply implementation/revision bindings and individual claim outcomes; handle inapplicable, unavailable, and failed proof diagnostics. [f8b65e7](https://github.com/morluto/rea/commit/f8b65e78723b1e2169f94dd1226198ea3139707e).
|
||||
- **Mach-O imports:** Handle a null import `name` when identity is ambiguous; use `raw_name` and `name_status` to distinguish observed expressions from exact symbols. [5d0d243](https://github.com/morluto/rea/commit/5d0d243a45c9132e1aff75d454ab53d86a74cc31), [169512a](https://github.com/morluto/rea/commit/169512a35bb015ff209168fb8bc1c9b31a2c678b).
|
||||
- **Node.js:** Use a runtime satisfying `^22.23.4 || ^24.22.1 || >=26.12.1`; Node 23 and 25 are unsupported. Upgrade Node before installing or updating REA; the installer does not upgrade it. [#6315](https://github.com/morluto/rea/pull/6315), [#6283](https://github.com/morluto/rea/issues/6283). Thanks @GamerDubz, @gensokyo-dev, @ShikamaruNaaraa.
|
||||
- **Browser storage:** Historical hashed storage remains readable, but comparisons stay unknown until both captures are recaptured in the new representation. [#6341](https://github.com/morluto/rea/pull/6341), [#6289](https://github.com/morluto/rea/issues/6289). Thanks @coygeek, @gongfpp.
|
||||
- **Environment serving:** `environment-serve` serves derived state by default, including write tools whose effects are confirmed by recorded evidence. Use `--replay-only` for stateless replay of manifest tools. Read stop-summary `state` instead of `state_evidence_id`. [#5351](https://github.com/morluto/rea/pull/5351), [#6158](https://github.com/morluto/rea/pull/6158). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **MCP schemas:** Input advertisements default to compact. Set `REA_MCP_INPUT_SCHEMA_PROFILE=full` for complete advertised guidance and nested constraints. [#5975](https://github.com/morluto/rea/pull/5975). Thanks @gensokyo-dev.
|
||||
- **Browser console arguments:** Accept null `argument_types` and `text_capture.status: "unavailable"` when the protocol argument inventory is omitted or malformed; preserve that uncertainty. [#6016](https://github.com/morluto/rea/pull/6016), [#5934](https://github.com/morluto/rea/issues/5934). Thanks @coygeek, @kaoru0822-kitauji.
|
||||
- **Reconstruction summaries:** `summary.by_evidence_authority` is keyed by Evidence authority; read the former source-state counts from required `summary.by_source_state`. Regenerate ledgers whose closure digests depended on the old projection. [#5875](https://github.com/morluto/rea/pull/5875), [#5838](https://github.com/morluto/rea/issues/5838). Thanks @coygeek, @kaoru0822-kitauji.
|
||||
- **Swift bindings:** Conformance `type_name` and `protocol_name` may be null; consume typed `type_binding`/`protocol_binding` evidence. Replace the former Objective-C-specific import-binding type/schema with `MachoImportBinding`. [d0067ba](https://github.com/morluto/rea/commit/d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f).
|
||||
- **WebSocket frames:** Accept null opcodes and preserve unknown frame facts; numeric opcodes are restricted to the four-bit protocol range. [0bc2b42](https://github.com/morluto/rea/commit/0bc2b42001f684beffcf47b70e507443b1a5d320).
|
||||
- **Resource-only inspection:** Evidence subjects and executable digests may be null. Use `resource_root` and each resource's reported provenance instead of requiring an executable target. [6a6adf7](https://github.com/morluto/rea/commit/6a6adf703fa44100643947d20fbc670f86260b1b).
|
||||
- **Browser timestamps:** Console and session-timeline timestamps may be null when the protocol supplies no finite/comparable time. Keep unknown time explicit in sorting and comparisons. [690fc51](https://github.com/morluto/rea/commit/690fc518540bae982038a74f47ef7781201785aa), [2a0410f](https://github.com/morluto/rea/commit/2a0410f1717b3126e8a95063d77dea66a597010a).
|
||||
- **Reference sources:** Read failed root enumeration from `rootFailure` rather than a synthetic failed child at `"."`; preserve the top-level source-graph limitation. [6296fdf](https://github.com/morluto/rea/commit/6296fdf560f1ac4aa55985558edaa98300aacdbc).
|
||||
- **Filesystem capture:** Accept `entry_unavailable` and `identity_changed` enumeration omission reasons in addition to the previous reason literals. [c85b44e](https://github.com/morluto/rea/commit/c85b44e1b14851c54e65327a4807f79ad5cd5a61). Thanks @rudrakshagarwal4200-ai.
|
||||
- **APK locales:** Replace numeric legacy selectors such as `es-r419` with `es-419` or `b+es+419`. Two-letter regions remain supported. [29910d1](https://github.com/morluto/rea/commit/29910d16b98fe4b607a7aa15928789a73614b0b9). Thanks @mertkayacs.
|
||||
- **XML plist scalars:** Recognize `$plist_type: "unknown"` and read `element`, `literal`, and `reason`; unresolved XML scalars use this wrapper instead of the former real/null unknown marker or malformed scalar prefixes. Supported extended UTC years preserve their timestamp. [3537e58](https://github.com/morluto/rea/commit/3537e58c1ab29309f06eabf50bc52adba9817a0c).
|
||||
|
||||
### Changes
|
||||
|
||||
- **Binary layouts:** Inspect ELF32 i386, little-endian ARM, and little- or big-endian MIPS `EXEC`/`DYN`/`REL` layouts on Linux x64 using a caller-supplied pwntools environment. [#7052](https://github.com/morluto/rea/pull/7052), [#6934](https://github.com/morluto/rea/issues/6934). Thanks @roblennon, @salmanalmutairi1.
|
||||
- **Agent environments:** Project network captures and Evidence files into manifests, preserve template path pairs and positions, export AgentEnv bundles, and serve manifests as replay environments. [#4954](https://github.com/morluto/rea/pull/4954), [#5293](https://github.com/morluto/rea/pull/5293), [#5664](https://github.com/morluto/rea/pull/5664), [#4999](https://github.com/morluto/rea/pull/4999), [#4975](https://github.com/morluto/rea/pull/4975). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Capture to environment:** Build environments from browser scenarios or a prompt, extract the initial observed state, and compare the environment with application probe captures. [#5182](https://github.com/morluto/rea/pull/5182), [#6946](https://github.com/morluto/rea/pull/6946), [#5319](https://github.com/morluto/rea/pull/5319), [#5699](https://github.com/morluto/rea/pull/5699). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **State-backed tools:** Render read filters, search, sorting and paging from state; apply confirmed write effects with broader state targets and recording-based write admission. [#5352](https://github.com/morluto/rea/pull/5352), [293832b](https://github.com/morluto/rea/commit/293832bafc9a054914ec514fe906dd4936c7a372), [#5800](https://github.com/morluto/rea/pull/5800), [#5787](https://github.com/morluto/rea/pull/5787), [#5600](https://github.com/morluto/rea/pull/5600), [5375ff0](https://github.com/morluto/rea/commit/5375ff0f9aadb4fd0d46333815aa0431e3df1010), [#6169](https://github.com/morluto/rea/pull/6169), [#6385](https://github.com/morluto/rea/pull/6385), [#6897](https://github.com/morluto/rea/pull/6897). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **State learning:** Model application-assigned values, preserve view-specific clock spellings, learn write clocks from scenario timing, derive fields, and learn creates across sorted arrays and referenced entities. [7ec9b6e](https://github.com/morluto/rea/commit/7ec9b6e56e1b3762285dca881db6d900a389ab9c), [#6136](https://github.com/morluto/rea/pull/6136), [957bcfa](https://github.com/morluto/rea/commit/957bcfa149c1a0593bb0aa2466cc56d1d4f62e7d), [#6138](https://github.com/morluto/rea/pull/6138), [#6967](https://github.com/morluto/rea/pull/6967), [#7025](https://github.com/morluto/rea/pull/7025), [#7034](https://github.com/morluto/rea/pull/7034). Thanks @earakely-scale.
|
||||
- **Recorded writes:** Serve creates whose compatible read-backs add omitted fields or express timestamps at a different precision, and whole-object updates that regenerate observed slugs or modification times. Conflicting observations still withhold writes. State views and bundles carrying the new read-back data need REA 6.8.0 or later to serve. [#7080](https://github.com/morluto/rea/pull/7080). Thanks @earakely-scale.
|
||||
- **Native investigations:** Page native inventory summaries, match functions across builds, export pseudocode to resumable directories, and select objdump disassembly by symbol. [#5378](https://github.com/morluto/rea/pull/5378), [#5390](https://github.com/morluto/rea/pull/5390), [#6170](https://github.com/morluto/rea/pull/6170), [#5171](https://github.com/morluto/rea/pull/5171), [#4249](https://github.com/morluto/rea/issues/4249), [#5343](https://github.com/morluto/rea/issues/5343), [#5159](https://github.com/morluto/rea/issues/5159). Thanks @coygeek, @CryptoJones, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Engine workflows:** Bound a single Ghidra decompilation without closing its session, analyze GameCube/Wii DOL executables, and inspect static CUDA/HIP code objects. [#5662](https://github.com/morluto/rea/pull/5662), [#6240](https://github.com/morluto/rea/pull/6240), [#5813](https://github.com/morluto/rea/pull/5813), [#5340](https://github.com/morluto/rea/issues/5340), [#5345](https://github.com/morluto/rea/issues/5345). Thanks @arielfikru, @coygeek, @gensokyo-dev, @RishTheFish21.
|
||||
- **PE/ELF:** Inspect export tables without an engine, PE imports/delay-loads, Authenticode certificates, TLS callbacks/load configuration, ELF notes, Linux kernel module metadata, and Zstandard section-name tables. [#6278](https://github.com/morluto/rea/pull/6278), [#6509](https://github.com/morluto/rea/pull/6509), [#4830](https://github.com/morluto/rea/pull/4830), [#7008](https://github.com/morluto/rea/pull/7008), [#4837](https://github.com/morluto/rea/pull/4837), [#7010](https://github.com/morluto/rea/pull/7010), [#6987](https://github.com/morluto/rea/pull/6987), [#4057](https://github.com/morluto/rea/issues/4057). Thanks @gensokyo-dev, @kaoru0822-kitauji, @mvanhorn, @Rinkia, @ShuaixinHuang.
|
||||
- **Apple/native metadata:** Inspect IPS crash reports, decode Swift nominal declarations, and select universal Mach-O slices in Go/Rust metadata readers. [#6511](https://github.com/morluto/rea/pull/6511), [#6257](https://github.com/morluto/rea/pull/6257), [#6219](https://github.com/morluto/rea/pull/6219), [#5341](https://github.com/morluto/rea/issues/5341), [#4102](https://github.com/morluto/rea/issues/4102). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @mvanhorn, @Rinkia.
|
||||
- **Packaged applications:** Triage desktop installation directories and inspect/extract Godot PCK, PyInstaller archives, and Node.js single-executable application blobs. [#5207](https://github.com/morluto/rea/pull/5207), [#3634](https://github.com/morluto/rea/pull/3634), [afa5da7](https://github.com/morluto/rea/commit/afa5da745e0b139e68029a96b776ae30e48ea2b2), [#2433](https://github.com/morluto/rea/pull/2433), [#7000](https://github.com/morluto/rea/pull/7000), [#4399](https://github.com/morluto/rea/issues/4399). Thanks @0xthearchitect, @coygeek, @gensokyo-dev, @mvanhorn.
|
||||
- **Unity and managed code:** Inventory IL2CPP images/types/methods and metadata versions 24/27; read member names and literals, link native methods and generic instances, and export Ghidra seeds. Project managed call/field-access edges and page managed member summaries. [#3807](https://github.com/morluto/rea/pull/3807), [#5326](https://github.com/morluto/rea/pull/5326), [52d1a72](https://github.com/morluto/rea/commit/52d1a72af047a73b787b8935b032786a34966036), [#5327](https://github.com/morluto/rea/pull/5327), [#5785](https://github.com/morluto/rea/pull/5785), [#6260](https://github.com/morluto/rea/pull/6260), [#5167](https://github.com/morluto/rea/pull/5167), [#3784](https://github.com/morluto/rea/issues/3784), [#5112](https://github.com/morluto/rea/issues/5112), [#4056](https://github.com/morluto/rea/issues/4056), [#5148](https://github.com/morluto/rea/issues/5148). Thanks @8rulerstar, @alexlanghart, @elricfd, @gensokyo-dev, @jaelliot, @kaoru0822-kitauji, @Rinkia.
|
||||
- **Android:** Inspect APK signing declarations and structured manifest components; report static-final DEX field values. [e93e3b1](https://github.com/morluto/rea/commit/e93e3b1e7d6e7e4f49c2de24b92452ec29e853f8), [#4158](https://github.com/morluto/rea/pull/4158), [#5025](https://github.com/morluto/rea/pull/5025), [#6981](https://github.com/morluto/rea/pull/6981), [#4537](https://github.com/morluto/rea/issues/4537), [#6397](https://github.com/morluto/rea/issues/6397). Thanks @Abelo9996, @BaaRai, @FerroLx, @gensokyo-dev, @sserrgo12.
|
||||
- **Bytecode and firmware:** Inspect and query CPython pyc and BEAM facts, inspect flattened devicetrees, and attribute MMIO peripheral regions in native disassembly. [#4826](https://github.com/morluto/rea/pull/4826), [#6928](https://github.com/morluto/rea/pull/6928), [#4833](https://github.com/morluto/rea/pull/4833), [#6943](https://github.com/morluto/rea/pull/6943), [#7009](https://github.com/morluto/rea/pull/7009), [#5348](https://github.com/morluto/rea/pull/5348), [#6922](https://github.com/morluto/rea/issues/6922), [#6938](https://github.com/morluto/rea/issues/6938). Thanks @gensokyo-dev, @liziye627-design, @mvanhorn.
|
||||
- **JavaScript:** Retain queryable static route inventories, link assignment values to writes, and accept source selections as a JSON array or file. [#6019](https://github.com/morluto/rea/pull/6019), [#6206](https://github.com/morluto/rea/pull/6206), [#5029](https://github.com/morluto/rea/pull/5029), [#4772](https://github.com/morluto/rea/issues/4772), [#6193](https://github.com/morluto/rea/issues/6193). Thanks @gensokyo-dev, @IvenKooLab, @jacobmalibiran, @JoshOsullivan-au, @kaoru0822-kitauji, @KJLavender.
|
||||
- **Data inspection:** Decode selected protobuf/gRPC bodies, inspect GGUF/safetensors declarations, and produce byte-entropy profiles. [#5031](https://github.com/morluto/rea/pull/5031), [#5603](https://github.com/morluto/rea/pull/5603), [#2331](https://github.com/morluto/rea/pull/2331), [#4986](https://github.com/morluto/rea/issues/4986), [#3323](https://github.com/morluto/rea/issues/3323). Thanks @AayushJB03, @gensokyo-dev, @kaoru0822-kitauji, @KJLavender, @michaelkillgta, @mvanhorn.
|
||||
- **CLI input/output:** Accept structured request documents on stdin and expose version/runtime output formats. Row-oriented JSONL exports use the migration above. [#7040](https://github.com/morluto/rea/pull/7040), [#5486](https://github.com/morluto/rea/issues/5486), [#6530](https://github.com/morluto/rea/pull/6530), [169512a](https://github.com/morluto/rea/commit/169512a35bb015ff209168fb8bc1c9b31a2c678b), [#5483](https://github.com/morluto/rea/issues/5483). Thanks @Abelo9996, @ahmed-husssain, @CassieFox23, @DeryFerd, @eco-null, @ezrabrilliant, @GamerDubz, @gensokyo-dev, @gongfpp, @hifzabuildsai, @jayykernel, @mhamzasakhi, @pablodiazjorge, @Plastic-time, @raidnouredden-ux, @SandstromPL, @ShuaixinHuang, @Sriram-PR, @XiaoBinGan.
|
||||
- **MCP and setup:** Add opt-in Streamable HTTP and FreeBuff setup registration. FreeBuff support first appears in 6.8.0; installed native FreeBuff loading and skill discovery remain unverified. [#4955](https://github.com/morluto/rea/pull/4955), [#2736](https://github.com/morluto/rea/pull/2736), [#3243](https://github.com/morluto/rea/issues/3243). Thanks @0xLLVEL, @codebuff-team, @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Recovery controls:** Adjust per-call WASM/SQLite/firmware deadlines, MCP startup budgets, regex matching deadlines/worker heap, capture decoder heap, ASAR header admission, Apple dispatch input, and the bridge request ceiling. MCP runtime limit recovery reports actionable settings. [#5798](https://github.com/morluto/rea/pull/5798), [#5818](https://github.com/morluto/rea/pull/5818), [aa53c06](https://github.com/morluto/rea/commit/aa53c065387c58b4189d82c47f1f3445fc505e30), [#5863](https://github.com/morluto/rea/pull/5863), [#6304](https://github.com/morluto/rea/pull/6304), [#6267](https://github.com/morluto/rea/pull/6267), [#6301](https://github.com/morluto/rea/pull/6301), [#6299](https://github.com/morluto/rea/pull/6299), [#6209](https://github.com/morluto/rea/pull/6209), [#6233](https://github.com/morluto/rea/pull/6233), [#6376](https://github.com/morluto/rea/pull/6376), [#7049](https://github.com/morluto/rea/pull/7049), [#5797](https://github.com/morluto/rea/issues/5797), [#5812](https://github.com/morluto/rea/issues/5812), [#5851](https://github.com/morluto/rea/issues/5851), [#6176](https://github.com/morluto/rea/issues/6176), [#3917](https://github.com/morluto/rea/issues/3917). Thanks @gensokyo-dev, @jaychou813, @kaoru0822-kitauji.
|
||||
- **APK output:** Set `string_tables_only` or CLI `--string-tables-only` to omit duplicate primary strings and avoid their output-capacity charge. [7c458f0](https://github.com/morluto/rea/commit/7c458f04b9155707c8ba406f8b455b10016f1b99), [#4990](https://github.com/morluto/rea/pull/4990), [#4982](https://github.com/morluto/rea/issues/4982). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Retained analysis:** Reuse successful validation of authenticated immutable native/imported-JavaScript parents and captures. Query retained Go build identity, Rust dependencies, JavaScript source-to-bundle mappings, and native export facts without repeating full output. [8f7d12e](https://github.com/morluto/rea/commit/8f7d12e30d63f3d3b4cfe2a90336362841438f56), [#4962](https://github.com/morluto/rea/pull/4962), [#5349](https://github.com/morluto/rea/pull/5349), [#5524](https://github.com/morluto/rea/pull/5524), [#5815](https://github.com/morluto/rea/pull/5815), [#6128](https://github.com/morluto/rea/pull/6128), [#6907](https://github.com/morluto/rea/pull/6907), [#4810](https://github.com/morluto/rea/issues/4810), [#5499](https://github.com/morluto/rea/issues/5499), [#5802](https://github.com/morluto/rea/issues/5802), [#6112](https://github.com/morluto/rea/issues/6112), [#6900](https://github.com/morluto/rea/issues/6900). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Compact results:** Use retained Frida/GDB/objdump summaries, omit redundant Ghidra raw results, and defer the analysis runtime during MCP catalog discovery. [#5661](https://github.com/morluto/rea/pull/5661), [#5614](https://github.com/morluto/rea/pull/5614), [#5355](https://github.com/morluto/rea/pull/5355), [#4424](https://github.com/morluto/rea/pull/4424), [#5640](https://github.com/morluto/rea/issues/5640), [#5584](https://github.com/morluto/rea/issues/5584), [#5337](https://github.com/morluto/rea/issues/5337). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @leeguooooo.
|
||||
- **Analysis work:** Reuse Hopper literal-string records, JavaScript workers and captured prefix selections, Interface Builder validation and plist scalar schemas; bound stable reads to one snapshot buffer and avoid decoding omitted GGUF strings. [#5017](https://github.com/morluto/rea/pull/5017), [#5819](https://github.com/morluto/rea/pull/5819), [31de177](https://github.com/morluto/rea/commit/31de177404461d122761d8243abe4fd24564f4da), [#5885](https://github.com/morluto/rea/pull/5885), [85e38ac](https://github.com/morluto/rea/commit/85e38ac199dff15e8c43343d9170db7e0e46d284), [#6966](https://github.com/morluto/rea/pull/6966), [#5781](https://github.com/morluto/rea/pull/5781), [#5010](https://github.com/morluto/rea/issues/5010), [#5855](https://github.com/morluto/rea/issues/5855), [#6954](https://github.com/morluto/rea/issues/6954), [#5774](https://github.com/morluto/rea/issues/5774). Thanks @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Native scheduling:** Batch modern IDA callee lookups, schedule decompilation reads on demand, index NativeAOT type closure, and retain call-path ancestry once per visited node. [0c6fed9](https://github.com/morluto/rea/commit/0c6fed90b74c75a9fd23d1ec1b510a7274f85887), [06fe7ee](https://github.com/morluto/rea/commit/06fe7eecbc8c387414b4c1aa0d88ad6314f54f0c), [ce61680](https://github.com/morluto/rea/commit/ce61680c04c95a9cdfad5335632a99542af26d37), [5e4ec0f](https://github.com/morluto/rea/commit/5e4ec0fa1b678aa3bf20704ec9110224ba8c1676).
|
||||
- **Selected processing:** Prepare CLI positional schemas only for the selected command, separate entropy reads from report resolution, prune typed-array reference escapes, index state-write cutoffs, decode only selected replay query values, and skip payload text during WAT framing. [315b234](https://github.com/morluto/rea/commit/315b23403f0819967878cee418c3f9cb4b1a69d9), [#6950](https://github.com/morluto/rea/pull/6950), [#5139](https://github.com/morluto/rea/pull/5139), [#5350](https://github.com/morluto/rea/pull/5350), [96ea32b](https://github.com/morluto/rea/commit/96ea32b7e4e28d3c9cd79a1de4b8e1ba3b4244e9), [70835d3](https://github.com/morluto/rea/commit/70835d3d5f030deadf606e1c79c70a60883e50c9), [#5766](https://github.com/morluto/rea/pull/5766), [#6947](https://github.com/morluto/rea/issues/6947), [#5760](https://github.com/morluto/rea/issues/5760). Thanks @galleonli, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
|
||||
### Fixes
|
||||
|
||||
- **IPS report fields:** Preserve the recorded terminating process and PID, termination flags, parent PID, and build/platform metadata. [#7079](https://github.com/morluto/rea/pull/7079), [#7078](https://github.com/morluto/rea/issues/7078). Thanks @rodricksz4h5.
|
||||
- **Ghidra language selection:** Accept stock language IDs containing `LEBE` or slash-separated processor names, and report conflicting DOL language overrides as configuration errors naming `REA_GHIDRA_LANGUAGE_ID`. [#7070](https://github.com/morluto/rea/pull/7070), [#7072](https://github.com/morluto/rea/issues/7072), [#7081](https://github.com/morluto/rea/pull/7081), [#7082](https://github.com/morluto/rea/issues/7082). Thanks @Adolanium.
|
||||
- **Setup and Hopper:** Preserve the Linux embedding runtime, supported x86_64 host restriction, readiness-based provider choices, orderly demo-helper shutdown, and installed apt dependency versions. Keep cleanup and skill publication ownership through retries. [#5041](https://github.com/morluto/rea/pull/5041), [#3422](https://github.com/morluto/rea/pull/3422), [f5d1012](https://github.com/morluto/rea/commit/f5d1012fabfe99af82523c3078ce9d93eb3780c1), [#4966](https://github.com/morluto/rea/pull/4966), [#6974](https://github.com/morluto/rea/pull/6974), [211e353](https://github.com/morluto/rea/commit/211e3533b34f81ae0910ad4f53f2a3d3c8587d3e), [#6920](https://github.com/morluto/rea/pull/6920), [#5015](https://github.com/morluto/rea/issues/5015), [#3419](https://github.com/morluto/rea/issues/3419), [#4921](https://github.com/morluto/rea/issues/4921), [#6975](https://github.com/morluto/rea/issues/6975). Thanks @Abel-I-Akhigbe, @Adolanium, @DavidFB-creator, @DeryFerd, @gensokyo-dev, @kaoru0822-kitauji, @sx4im.
|
||||
- **Client setup:** Treat empty APPDATA as unset, distinguish MSIX Claude Desktop process views, preserve aliased Hermes server tables, reuse client dialect/alias bindings, and materialize the pinned package-runner cache before the first start. [#5126](https://github.com/morluto/rea/pull/5126), [#4903](https://github.com/morluto/rea/pull/4903), [#3785](https://github.com/morluto/rea/pull/3785), [b7eeded](https://github.com/morluto/rea/commit/b7eededf5af5061a834f93f57f2c888acf870344), [#7032](https://github.com/morluto/rea/pull/7032), [#5104](https://github.com/morluto/rea/issues/5104), [#4278](https://github.com/morluto/rea/issues/4278), [#3320](https://github.com/morluto/rea/issues/3320), [#7027](https://github.com/morluto/rea/issues/7027). Thanks @CmsChase, @coygeek, @myso-kr, @pablodiazjorge, @vaidehi-snk, @wxj006007.
|
||||
- **Target admission:** Preserve thin arm64_32 Mach-O support, distinguish JVM identity before FAT admission, validate FAT format/CPU identity, preserve original DOS header padding, and reject NULs in local objdump/rizin paths. [#5788](https://github.com/morluto/rea/pull/5788), [6843dbe](https://github.com/morluto/rea/commit/6843dbea9bce4709ee2e8bf46c38f0bbf640716b), [497037e](https://github.com/morluto/rea/commit/497037e6e3975cbb37cfd89ea0ad300faa2a4769), [#7013](https://github.com/morluto/rea/pull/7013), [#3600](https://github.com/morluto/rea/pull/3600), [#5713](https://github.com/morluto/rea/issues/5713), [#6510](https://github.com/morluto/rea/issues/6510). Thanks @coygeek, @dhruvac29, @kaoru0822-kitauji, @Lew1sWong, @qorvr.
|
||||
- **Mach-O and dylibs:** Preserve container framing, facts when dyld rejects a missing UUID, verified-signature snapshot identity, unsigned diagnostics, absolute paths inside the selected root, legacy platform rules, lazy-load payloads, and reasons for rejecting present dylibs. [cf3288d](https://github.com/morluto/rea/commit/cf3288d8cf7a09685acf89103b36c32359be56fa), [2b263a7](https://github.com/morluto/rea/commit/2b263a7fe9e9d4c185c7d6af6b4e67b21a73af30), [eb7acc7](https://github.com/morluto/rea/commit/eb7acc70bf06f119e8639e30a78081d5ba82b0bf), [1502cfb](https://github.com/morluto/rea/commit/1502cfbc8ca0d1d1d86eebf334018dfb418bbac0), [90062c0](https://github.com/morluto/rea/commit/90062c092228dea9d33fadc0d7e1b968105b8b2c), [#5650](https://github.com/morluto/rea/pull/5650), [3b67b36](https://github.com/morluto/rea/commit/3b67b362aa2cd79eac5af57d0965e81a7d0b8c2b), [#5551](https://github.com/morluto/rea/issues/5551). Thanks @coygeek.
|
||||
- **Native facts:** Admit direct dispatch-metadata providers; preserve record-local Objective-C omissions, snapshot profiles, class-import binding identity, stripped class labels, version-like names, and qualified Swift protocol identities. [50b6975](https://github.com/morluto/rea/commit/50b6975d861069e2c6cfa202780621449f863802), [9eec57b](https://github.com/morluto/rea/commit/9eec57b469087fe9bff3d29dcc02f47074159e91), [0ba4d6f](https://github.com/morluto/rea/commit/0ba4d6f43ad610dacc861a1548be1a5c4712cd61), [4d7de00](https://github.com/morluto/rea/commit/4d7de00d4cc9a95ec2b7a3123b04e01db2daafa0), [22c84ce](https://github.com/morluto/rea/commit/22c84ce64aa6c7601aa81af86d4117b5a4c2feac), [4852c83](https://github.com/morluto/rea/commit/4852c830af1e3b9df437235b7a8fa8970a6d1c59), [aa9043f](https://github.com/morluto/rea/commit/aa9043f6b61c453e4735795cfe6a559bf6b6cc16).
|
||||
- **Apple resources:** Preserve independent plist scalars, processing instructions, nonstandard table widths, unsupported NIB connector coverage, and strict IPS JSON surrogate validation; classify Interface Builder objects by their declared superclass chains. [fd728f2](https://github.com/morluto/rea/commit/fd728f271e7de7bcfa63890b573715720d5f948c), [#5806](https://github.com/morluto/rea/pull/5806), [4526862](https://github.com/morluto/rea/commit/4526862728af01f1135714079c0d8a8c754eb364), [14cd78c](https://github.com/morluto/rea/commit/14cd78c3224edcf79fca152915077112f547b9ed), [#6990](https://github.com/morluto/rea/pull/6990), [#5805](https://github.com/morluto/rea/issues/5805), [#5228](https://github.com/morluto/rea/pull/5228), [#5218](https://github.com/morluto/rea/issues/5218). Thanks @coygeek, @DeryFerd, @kaoru0822-kitauji, @Mangolycheematcha, @mhamzasakhi, @Shreyash021104.
|
||||
- **Archives and files:** Complete positional ZIP reads, preserve bounded ASAR signatures, report stable symlink loops as resolution failures, retain target applicability, and report unknown traversal coverage. Retain reported integrity contradictions for members present in only one compared artifact. [#4414](https://github.com/morluto/rea/pull/4414), [e8da7e1](https://github.com/morluto/rea/commit/e8da7e1edd075feec0e6beedec93d5645fed8125), [#5571](https://github.com/morluto/rea/pull/5571), [#5763](https://github.com/morluto/rea/pull/5763), [#5867](https://github.com/morluto/rea/pull/5867), [#5521](https://github.com/morluto/rea/issues/5521), [#5731](https://github.com/morluto/rea/issues/5731), [#5837](https://github.com/morluto/rea/issues/5837), [#7076](https://github.com/morluto/rea/pull/7076). Thanks @Afloat16, @coygeek, @jdotc1, @kaoru0822-kitauji, @WenyuChiou.
|
||||
- **Extraction and file ownership:** Restrict destination-case recovery to owned collisions, retain physical-read failures through late cancellation/cleanup, preserve cleanup capabilities for retry, and retain independent firmware extraction failures. [932fbb7](https://github.com/morluto/rea/commit/932fbb7d84a9a995b9aee3c1be7da05a8c4d7bbb), [9499667](https://github.com/morluto/rea/commit/949966712e68d8857a50a165ecddf04c2d7e6d3b), [56679dd](https://github.com/morluto/rea/commit/56679dd2ae31dda58e181f8bf99775014f7bb82b), [b3c912a](https://github.com/morluto/rea/commit/b3c912ab79ad147a37b6cfbe1552a1a5eecec629), [7298bd0](https://github.com/morluto/rea/commit/7298bd08d134860b84b4073f6703021916a5a4c5).
|
||||
- **Processes and cancellation:** Retain partial samples and shutdown gaps, unresolved descendants and caller abort reasons; bind cleanup errors to their owners, propagate comparison contradictions, and report no normal exit code for signalled targets. [ba866a2](https://github.com/morluto/rea/commit/ba866a26ecbe179a717e746cbc0d1605ae616ad5), [#5270](https://github.com/morluto/rea/pull/5270), [#6144](https://github.com/morluto/rea/pull/6144), [a721c42](https://github.com/morluto/rea/commit/a721c42c07dc674c32ab939627355b8a1bee97a5), [#6298](https://github.com/morluto/rea/pull/6298), [#6486](https://github.com/morluto/rea/pull/6486), [#5935](https://github.com/morluto/rea/issues/5935), [#5944](https://github.com/morluto/rea/issues/5944), [#6100](https://github.com/morluto/rea/issues/6100), [#6101](https://github.com/morluto/rea/issues/6101), [#6110](https://github.com/morluto/rea/issues/6110), [#6113](https://github.com/morluto/rea/issues/6113), [#6121](https://github.com/morluto/rea/issues/6121), [#6131](https://github.com/morluto/rea/issues/6131), [#6132](https://github.com/morluto/rea/issues/6132), [#6218](https://github.com/morluto/rea/issues/6218), [#6445](https://github.com/morluto/rea/issues/6445). Thanks @agorevski, @coygeek, @gongfpp, @kwakhyun, @mhamzasakhi.
|
||||
- **Provider cleanup:** Preserve Windows native handles until close is confirmed, native/Inspector/Electron cleanup ownership, Kuna probe identity and cleanup, IDA registration configuration, and Hopper preparation ownership. [#5772](https://github.com/morluto/rea/pull/5772), [87ab054](https://github.com/morluto/rea/commit/87ab054ea384b3fcbae2998b1f94685edcd5f624), [ff78165](https://github.com/morluto/rea/commit/ff78165978de9dbcf9fa16c2acf74bf85a69dc44), [04ae07a](https://github.com/morluto/rea/commit/04ae07ab8c9ae1a6697b6ad15489bbde515da8b5), [dd0015a](https://github.com/morluto/rea/commit/dd0015aaf156c5701ef9598bcc6c885e6b8da803), [b231c28](https://github.com/morluto/rea/commit/b231c28cf64c2f799ada99343f8f21e0e6acb15e), [#4505](https://github.com/morluto/rea/issues/4505), [#5616](https://github.com/morluto/rea/issues/5616).
|
||||
- **Provider diagnostics:** Give Ghidra restart recovery, stable Homebrew aliases, unsupported-host results for Linux-only integrations, correct IDA regex/readiness handling, and clear diagnostics when Windows refuses a verified native addon. [#5356](https://github.com/morluto/rea/pull/5356), [7c6142e](https://github.com/morluto/rea/commit/7c6142efdc19a42c148d0e8c96b69d18e63c7f87), [#5793](https://github.com/morluto/rea/pull/5793), [#4862](https://github.com/morluto/rea/pull/4862), [99ccfb6](https://github.com/morluto/rea/commit/99ccfb6e709ac4bc0afe176315280abde9ebcb9f), [#6832](https://github.com/morluto/rea/pull/6832), [#5730](https://github.com/morluto/rea/issues/5730), [#4860](https://github.com/morluto/rea/issues/4860), [#5723](https://github.com/morluto/rea/issues/5723). Thanks @coygeek, @dark13b, @gensokyo-dev, @jgt87, @jpmorris41488, @Lew1sWong.
|
||||
- **MCP results:** Preserve protocol envelopes on oversized errors, enforce and advertise absolute Evidence paths, retain typed output-filter errors and selected JSON paths, and give client/provider timeout recovery. Compact advertisements now normalize reported enum and union shapes for strict gateway translation. [562dab0](https://github.com/morluto/rea/commit/562dab00c5cbf76a9b2695bad93792eccdc9a49d), [3cf6206](https://github.com/morluto/rea/commit/3cf6206614e65b5ebb6284a5709090dfb8f9aaee), [0e83d92](https://github.com/morluto/rea/commit/0e83d92c583ae36afe00b2c23dd2a0f1d025cbca), [4714c7e](https://github.com/morluto/rea/commit/4714c7eaa3d1d8b6a5df0aff586e844b9d9ab06d), [1bfa60a](https://github.com/morluto/rea/commit/1bfa60ac20e1a344bcf3132c08d15b6860b05c61), [#5608](https://github.com/morluto/rea/pull/5608), [#3806](https://github.com/morluto/rea/pull/3806), [#5598](https://github.com/morluto/rea/issues/5598), [#3800](https://github.com/morluto/rea/issues/3800). Thanks @gensokyo-dev, @Hahaa909, @wang1xiang.
|
||||
- **Browser lifecycles:** Retain historical outcomes through cleanup, stop late navigation before session release, stop expired work, observe aborted fetch settlement, and mark later scenario actions skipped after failure or cancellation. [2e5171e](https://github.com/morluto/rea/commit/2e5171e391b83f8a467d0861fd452c1f78621fd8), [599fdac](https://github.com/morluto/rea/commit/599fdac029912f96c511f859660483405d2de26f), [b2fad36](https://github.com/morluto/rea/commit/b2fad362f2b37f81dc9a6f784f4521e4ca24d41e), [13a8640](https://github.com/morluto/rea/commit/13a864020414577006dce58d7fdaa877ddd0cb52), [#6023](https://github.com/morluto/rea/pull/6023), [#5943](https://github.com/morluto/rea/issues/5943). Thanks @coygeek, @kaoru0822-kitauji.
|
||||
- **Browser comparisons:** Preserve frame identity, default origin scope, unknown body/shape coverage, correspondence and uncertainty, and independent retained-observation budgets. [#4040](https://github.com/morluto/rea/pull/4040), [#4413](https://github.com/morluto/rea/pull/4413), [#4476](https://github.com/morluto/rea/pull/4476), [a0c22ff](https://github.com/morluto/rea/commit/a0c22ff0125b8af7612db54d3708413e2bfecb25), [a0fcc60](https://github.com/morluto/rea/commit/a0fcc60d89eb0cc09b08181eb9f961acfd07d8e9), [#3878](https://github.com/morluto/rea/issues/3878), [#3910](https://github.com/morluto/rea/issues/3910), [#3941](https://github.com/morluto/rea/issues/3941), [#4385](https://github.com/morluto/rea/issues/4385), [#4477](https://github.com/morluto/rea/issues/4477). Thanks @Adolanium, @coygeek, @eco-null, @gensokyo-dev, @kaoru0822-kitauji, @XY-code1.
|
||||
- **Electron observations:** Preserve listener callback ownership and count registrations by source callsite; block trash calls in the active hook, retain window handles through cancelled selection, and preserve cancellation during finalization. [dfe8378](https://github.com/morluto/rea/commit/dfe8378cf5a727e3d93d47a5986233175c6ef898), [5064007](https://github.com/morluto/rea/commit/506400748eb1d6c674f0cf51330073929c45f939), [816a9dc](https://github.com/morluto/rea/commit/816a9dc98886e6d824cd8ae88b68731b161d5dda), [1707d17](https://github.com/morluto/rea/commit/1707d171d750955bf6387e028e378183b360ffe3), [4e2a566](https://github.com/morluto/rea/commit/4e2a566d6c7e053ecbbee1b684311ed31ea163bf).
|
||||
- **JavaScript source maps:** Keep unmapped and missing mappings unknown, preserve partial indexed-map facts and section paths, propagate preparation capability errors, and retain observed worker failures through cancellation. [a4dbb36](https://github.com/morluto/rea/commit/a4dbb36d8ea83e9f94dd289ac1a2d220d2c72c9b), [2b2e362](https://github.com/morluto/rea/commit/2b2e362a00fc5ba2f93fa6c62e2d8780072d1b4b), [498a667](https://github.com/morluto/rea/commit/498a667234b0140c1cfa43c819b35454ab11d14e), [#6149](https://github.com/morluto/rea/pull/6149), [#6214](https://github.com/morluto/rea/pull/6214), [f01dcca](https://github.com/morluto/rea/commit/f01dccab77f712fb4ac936d0a6e92a70d12f0789), [#6082](https://github.com/morluto/rea/issues/6082), [#6061](https://github.com/morluto/rea/issues/6061). Thanks @coygeek, @diyasharma05, @kaoru0822-kitauji.
|
||||
- **JavaScript semantics:** Preserve overflowing timer uncertainty, unavailable-scope comparisons, directory-form URL/ESM path intent, and artifact confinement. Replaced globals lose built-in effects; erased imports no longer become runtime bindings. Treat a known truthy `fork()` `silent` option as piped stdio unless a string/array `stdio` override takes precedence; unresolved options stay dynamic. [#3456](https://github.com/morluto/rea/pull/3456), [8d198d6](https://github.com/morluto/rea/commit/8d198d6ec707402d75276f516e0090d71767b7e4), [#4646](https://github.com/morluto/rea/pull/4646), [#2700](https://github.com/morluto/rea/pull/2700), [#4565](https://github.com/morluto/rea/pull/4565), [#6994](https://github.com/morluto/rea/pull/6994), [#3400](https://github.com/morluto/rea/issues/3400), [#3882](https://github.com/morluto/rea/issues/3882), [#3367](https://github.com/morluto/rea/issues/3367), [#7083](https://github.com/morluto/rea/pull/7083), [#7084](https://github.com/morluto/rea/issues/7084). Thanks @Adolanium, @amyanger, @coygeek, @gensokyo-dev, @HawkForge, @rohitg00, @viniciusdsandrade.
|
||||
- **JavaScript routes and imports:** Treat path data as route candidates, preserve route flag and lexical RPC identities, package-main trailing-slash selection, catch aliases, and factory parameter ownership. [#6360](https://github.com/morluto/rea/pull/6360), [38ecc02](https://github.com/morluto/rea/commit/38ecc0248e1993d25a2f7dd350024484cce797bb), [#4219](https://github.com/morluto/rea/pull/4219), [21097b1](https://github.com/morluto/rea/commit/21097b192b1ab94647e3446b1a2a4dd00e924d5e), [f774dc4](https://github.com/morluto/rea/commit/f774dc420b3befe2b8e68971b4116da2d62fc800), [#6349](https://github.com/morluto/rea/issues/6349), [#4189](https://github.com/morluto/rea/issues/4189). Thanks @coygeek, @hemv-857, @kaoru0822-kitauji, @Nikhi00718.
|
||||
- **Managed/IL2CPP:** Preserve extension-only scope uncertainty and type discovery after member limits, nested names, ReadyToRun/declaration facts, selected type scope, valid shared-suffix strings, and truncated IL2CPP metadata prefixes. [#5674](https://github.com/morluto/rea/pull/5674), [#5312](https://github.com/morluto/rea/pull/5312), [87fa1b6](https://github.com/morluto/rea/commit/87fa1b613d2b6bda1722855a00f1390f27731b64), [bc2d494](https://github.com/morluto/rea/commit/bc2d494030efd1ccabaeaa0bd49226aa87c378a2), [1f0eb22](https://github.com/morluto/rea/commit/1f0eb228e50908ee2a2aaa72d329178ae1a2cdbd), [4c5fae5](https://github.com/morluto/rea/commit/4c5fae5179dd54c4a0d427cd2a115b9c887e3841), [b3aadaf](https://github.com/morluto/rea/commit/b3aadaf474e59b33e5ff465236008a67eccefb87), [#5467](https://github.com/morluto/rea/issues/5467), [#5146](https://github.com/morluto/rea/issues/5146). Thanks @coygeek, @eco-null, @elricfd, @umi008.
|
||||
- **Format boundaries:** Preserve independent facts across malformed inputs in model, GPU, WASM, Rust, Android, and SQLite inspection; report optional metadata warnings without discarding core results. [eb41f35](https://github.com/morluto/rea/commit/eb41f358cf83d5626f09de68332f9718f9aa6482), [#5789](https://github.com/morluto/rea/pull/5789), [00a9e95](https://github.com/morluto/rea/commit/00a9e95de91a4bd4d2a41f8aee704a53fa575750), [fc63fe8](https://github.com/morluto/rea/commit/fc63fe8682654086261540dcf3c59afc36c7d58f), [39a91d4](https://github.com/morluto/rea/commit/39a91d4db96ca13224c25936330ea84fe78970e5), [6c6e2ad](https://github.com/morluto/rea/commit/6c6e2ad62b20f810b54f5a567e66fc55241514b2), [929a0a7](https://github.com/morluto/rea/commit/929a0a76c870d9991e015dde58308fd9df0280b8), [d137a4c](https://github.com/morluto/rea/commit/d137a4cdde0049de967f1d5cd429ed9e1f7d0a10), [ef475fc](https://github.com/morluto/rea/commit/ef475fcf8b0601c61052b7091568ba574546417d), [#5749](https://github.com/morluto/rea/issues/5749). Thanks @coygeek, @gensokyo-dev.
|
||||
- **Capture decoding:** Preserve malformed protobuf trailer facts and wire/schema identity, HTTP/2 URI and UTF-8 failure facts, WebMCP property coverage, and offline capture availability without a browser provider. Report invalid record ordinals as selection errors with actual bounds where available, and trim surrounding SP/HTAB from gRPC-web trailer values while preserving internal whitespace. [9cf7a7c](https://github.com/morluto/rea/commit/9cf7a7cb0d3c72a1121fde280cb11bf911af8055), [b3c3b21](https://github.com/morluto/rea/commit/b3c3b21e43cab4e71b460d08829d8c6e4e9b9c21), [4cb1f1d](https://github.com/morluto/rea/commit/4cb1f1d820df4498ee464bcf72a5f47d40cf8dab), [51e5938](https://github.com/morluto/rea/commit/51e593877642ab42fb5bf25812f4a4110ee7ac26), [d23be5c](https://github.com/morluto/rea/commit/d23be5c0898c39c9f0c54e2e0b43ee746d92d316), [#6725](https://github.com/morluto/rea/pull/6725), [#3947](https://github.com/morluto/rea/issues/3947), [#7067](https://github.com/morluto/rea/pull/7067), [#7068](https://github.com/morluto/rea/issues/7068), [#7069](https://github.com/morluto/rea/pull/7069), [#7071](https://github.com/morluto/rea/issues/7071). Thanks @Adolanium, @coygeek, @Sriram-PR.
|
||||
- **Evidence and errors:** Authenticate ledger Evidence before committing; preserve parent-schema diagnostics, canonical correction details and owned cleanup outcomes; identify the rejected function-Evidence side/kind. Fixed allowances now name their actual limits. [0407bbf](https://github.com/morluto/rea/commit/0407bbfbe65e8fa475e8bb0c78c30170e2c2f55e), [#6361](https://github.com/morluto/rea/pull/6361), [1629266](https://github.com/morluto/rea/commit/1629266e3c08c90fed6bba5383528af7986a12b7), [#6969](https://github.com/morluto/rea/pull/6969), [#5860](https://github.com/morluto/rea/pull/5860), [#6034](https://github.com/morluto/rea/issues/6034), [#5932](https://github.com/morluto/rea/issues/5932), [#5847](https://github.com/morluto/rea/issues/5847). Thanks @AyoubElleuch, @coygeek, @dhruvac29, @kaoru0822-kitauji.
|
||||
- **ADB and firmware recovery:** Report an unparsed push checksum as unverified, normalize absent installer-package sentinels, describe extraction-relative Unblob links, and avoid recommending a budget increase at its fixed ceiling. [#6010](https://github.com/morluto/rea/pull/6010), [#4073](https://github.com/morluto/rea/pull/4073), [#4573](https://github.com/morluto/rea/pull/4573), [#6971](https://github.com/morluto/rea/pull/6971), [#5924](https://github.com/morluto/rea/issues/5924), [#4175](https://github.com/morluto/rea/issues/4175). Thanks @coygeek, @Ganesh2325, @kaoru0822-kitauji, @Rinkia, @XiaoBinGan, @yankikucuk.
|
||||
- **CLI formatting:** Share Markdown report formatting, exclude inherited row values, and preserve arrays of empty objects as JSON. Render Markdown table headers and cells literally while keeping intraword underscores readable. [b784912](https://github.com/morluto/rea/commit/b784912811fd10ac9dc425696dc0e9af632afa34), [#6984](https://github.com/morluto/rea/pull/6984), [#7058](https://github.com/morluto/rea/pull/7058), [#7057](https://github.com/morluto/rea/issues/7057), [#7073](https://github.com/morluto/rea/pull/7073). Thanks @Plastic-time, @ShuaixinHuang, @viniciusdsandrade.
|
||||
- **Format identity:** Preserve Python marshal scalar/decode limits, empty optional BEAM chunks, PE resource branch identities and certificate alignment, unsupported compressed ELF notes, and codec-framed GPU bundle extents. [0603223](https://github.com/morluto/rea/commit/06032237c6755e3dd5f9a8db732c89958b238cc7), [c46064c](https://github.com/morluto/rea/commit/c46064c3f6eaba121c5de7b17390bfdecdd132ef), [66671e8](https://github.com/morluto/rea/commit/66671e874888e98f7e2b27e68a7c6fb320bbb827), [03077d5](https://github.com/morluto/rea/commit/03077d54afc49613db4e38f70c506bc60f5223aa), [#6721](https://github.com/morluto/rea/pull/6721), [9fb8c11](https://github.com/morluto/rea/commit/9fb8c114a03ad60e57c11586967c6df845a3f97e), [f6a1e2c](https://github.com/morluto/rea/commit/f6a1e2c892dc23c7583c3d03af9edb01632ff5a5), [#6694](https://github.com/morluto/rea/issues/6694). Thanks @coygeek, @Deepak-Kambala.
|
||||
- **WASM and APK metadata:** Preserve ambiguous WASM identity and partial facts, glue URL query/fragment text across line terminators, APK signer identity, and raw issuer identity when comparison rules are unknown. [af8f0ba](https://github.com/morluto/rea/commit/af8f0badf64fd3546392b6aa4625777904b2f051), [#6802](https://github.com/morluto/rea/pull/6802), [f783f0f](https://github.com/morluto/rea/commit/f783f0f1e82472cdddd4de2156d974f729e332a6), [115a78f](https://github.com/morluto/rea/commit/115a78fe241276c34917b496621954c5f29bfc15), [#6639](https://github.com/morluto/rea/issues/6639). Thanks @coygeek, @gensokyo-dev, @gongfpp.
|
||||
- **Source identity:** Preserve Next route spelling, module grammar and loader request identity, directory package-main behavior, accessor effects, exact source targets, and resolver-discovery versus module-loading distinctions. [ddd7ac7](https://github.com/morluto/rea/commit/ddd7ac7b846b48191ead1f7dcd0e3bd735468fa8), [9974512](https://github.com/morluto/rea/commit/99745126009b15060d71c38809418858a07d5afa), [f2e5fd1](https://github.com/morluto/rea/commit/f2e5fd148a85fa737302dd194a33172c4561093e), [baf1678](https://github.com/morluto/rea/commit/baf16784af3592534687052431c145daef55cfd1), [6e324b2](https://github.com/morluto/rea/commit/6e324b224405f2befd953234f6dc210cf72ad50a), [75daf0c](https://github.com/morluto/rea/commit/75daf0c7df108f04ab41f34aec1a31d4a7ceb322), [f91b7f8](https://github.com/morluto/rea/commit/f91b7f8009c9af27a5e0148e3b07d20bfe3b7a2a).
|
||||
- **Graphs and selected facts:** Preserve JavaScript graph omissions/version facts, Electron native-loader facts, web JSON identity/coverage, module-path selection, feature-trace qualifications, selected native paths, endpoint context, and mixed Swift demangler outcomes. [30db499](https://github.com/morluto/rea/commit/30db499db310f073d9bf3d3acfd77985e1cb7e2e), [5a7bddb](https://github.com/morluto/rea/commit/5a7bddb3aeb0345854e76b79d529aed1c40e2054), [79805f2](https://github.com/morluto/rea/commit/79805f244a1ec0850f3109929168a9c8ebb1b908), [f719dc4](https://github.com/morluto/rea/commit/f719dc46c51a55b3926eec9659d06e7dfb95132b), [7784fdb](https://github.com/morluto/rea/commit/7784fdbad78f8b384d513f0e7dae8e983cd48ef8), [76500f1](https://github.com/morluto/rea/commit/76500f12b54e07b1ea700a403b2593890d63b42a), [676d29f](https://github.com/morluto/rea/commit/676d29f9eef794cea668f5f09f0ff510c848a513), [edf7b3d](https://github.com/morluto/rea/commit/edf7b3d32c3248f221e27f7ce5a3d34e8590285b), [93de7bc](https://github.com/morluto/rea/commit/93de7bc7e324a761162a31625b838ba0d4e3824d).
|
||||
- **Managed boundaries:** Honor native platform symbol lookup, consume resolved IL2CPP ELF declarations, and attribute malformed comparison inputs to the supplied argument. [64fa1db](https://github.com/morluto/rea/commit/64fa1db1309218283df19f3ee4dba61f5fbd2a67), [87745af](https://github.com/morluto/rea/commit/87745af52ab0ccd3a172c2534c516d41b0095035), [96ad94f](https://github.com/morluto/rea/commit/96ad94f69eda5d68c7335ff775b6927b1376fd2b).
|
||||
- **Provider and worker recovery:** Preserve output loss after successful command exit, physical refusal diagnostics, bounded worker control framing, JavaScript worker-phase/budget attribution, admitted ambiguity choices, and selected EVM limiter availability. Scope objdump debug-link limitations to DWARF requests and keep process-capture preparation nonexecuting. [e639edf](https://github.com/morluto/rea/commit/e639edf6204c05285d96840478926592daf95876), [171b33b](https://github.com/morluto/rea/commit/171b33b2c4defc6cbea49545bedf4272b72d1ea2), [34563fc](https://github.com/morluto/rea/commit/34563fcc9bd92af30668820dcac5635e27181094), [3f9357b](https://github.com/morluto/rea/commit/3f9357b5990d166a38440f56f439452266650093), [cec8f1e](https://github.com/morluto/rea/commit/cec8f1e8f305d6332272457a6ee419d3c5fb6359), [a74d859](https://github.com/morluto/rea/commit/a74d859a19122fb8f1282b3632d320997b1caae0), [c284168](https://github.com/morluto/rea/commit/c284168c4b1cd214ce7f504d43da28528d82d603), [4fb936f](https://github.com/morluto/rea/commit/4fb936fe86bb7a07f94f819ae8510301f2ef69c9).
|
||||
- **Setup and limits:** Retain unknown environment-budget facts, selected-skill/maintenance uncertainty, Hopper URL admission and bookmark verification, and listener failure recovery that matches the system cause. [a718037](https://github.com/morluto/rea/commit/a7180370332350bc0f998fbd643888221759570e), [d1fbebb](https://github.com/morluto/rea/commit/d1fbebb578538dc94fd5e376e6e1959218efa835), [60accad](https://github.com/morluto/rea/commit/60accad2de8f2862fe91901502fe26cc5390fb36), [fd7da77](https://github.com/morluto/rea/commit/fd7da77fc02dcfaf7d1bc9f2fb240d71cdd27c59), [0c440bf](https://github.com/morluto/rea/commit/0c440bfc07f3cb343e2e836d27cccaed539cfe9c).
|
||||
- **State and Evidence:** Retain export tarball ownership and SQLite primary outcomes through cleanup retry, count emitted SQLite schema names across encodings, scope display refresh to matching snapshot bindings, and report retained-result validation failures consistently. Reconstruction uses one effective completion scope and kind-based labels when names are blank. [ab58edc](https://github.com/morluto/rea/commit/ab58edcdf50b4bd687900ee2dc362ecc5c3dce15), [1006714](https://github.com/morluto/rea/commit/100671427105791a5c5913a918b63a394f6bc83e), [a47788a](https://github.com/morluto/rea/commit/a47788a6a4abb067273f954275a9d0a269adf1e7), [8aa40e9](https://github.com/morluto/rea/commit/8aa40e9a61155cff8b6f3c138c601bf805b20653), [b8c1b45](https://github.com/morluto/rea/commit/b8c1b45316882beeab4a9de49381d94e131fab0f), [a35b933](https://github.com/morluto/rea/commit/a35b9335a0124674ee5e3abe8c075f1fff6b4fa6), [#6137](https://github.com/morluto/rea/pull/6137), [#6126](https://github.com/morluto/rea/issues/6126). Thanks @carlogiovannineri-rgb, @mohithhhh.
|
||||
- **Capture coverage:** Name browser bundle budgets and deadline recovery, separate domain readiness from worker attribution, attribute rendered-frame omissions to their producing stage, share offline-capture host admission with discovery, preserve selected signal-file semantics, and retain short Flutter GNU build IDs. [#6036](https://github.com/morluto/rea/pull/6036), [ba78133](https://github.com/morluto/rea/commit/ba781330d6d28b22d15498e63c144fb40e086156), [5934387](https://github.com/morluto/rea/commit/5934387f4185129efe9dc8b7338d8bc3fa5b25bd), [ad629b2](https://github.com/morluto/rea/commit/ad629b2e31c3a9495b27179360b508fa021fe86a), [928ba76](https://github.com/morluto/rea/commit/928ba76882bfd216dde8ed79118789b2c3aad4f9), [678c4cf](https://github.com/morluto/rea/commit/678c4cfa68e2fee12f620075e12dc87d9b1195b0).
|
||||
- **Additional boundary facts:** Retain parser format/query occurrence identity and independent malformed-input facts, unresolved nested Swift identifiers, and completed signing-capture coverage; describe crash documents without assuming a provider. [5f702dd](https://github.com/morluto/rea/commit/5f702dd6ccd9e8e505e224854b91678439c3b41f), [99d2d7a](https://github.com/morluto/rea/commit/99d2d7ad22bcbd780919760008aba6a88e7d1077), [90e433d](https://github.com/morluto/rea/commit/90e433df06b3f83417176b8db1b0a905f9df1d3a), [#5548](https://github.com/morluto/rea/pull/5548), [ee12d51](https://github.com/morluto/rea/commit/ee12d517a25d34853d40e31fc039c7d1576c60e1), [#5538](https://github.com/morluto/rea/issues/5538). Thanks @coygeek, @ShuaixinHuang.
|
||||
- **Documentation:** Add the Romanian README. [#7026](https://github.com/morluto/rea/pull/7026). Thanks @An3treiu.
|
||||
|
||||
Full contribution record: [1740 commits and 497 merged PRs](https://github.com/morluto/rea/blob/rea-agents-6.8.0/docs/releases/6.8.0.contributions.json).
|
||||
|
||||
## [6.7.0](https://github.com/morluto/rea/compare/rea-agents-6.6.0...rea-agents-6.7.0) (2026-10-18)
|
||||
|
||||
### Highlights
|
||||
@@ -180,7 +78,7 @@ Full contribution record: [4 commits and 1 merged PR](https://github.com/morluto
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
- **Saved Evidence:** Evidence and analysis snapshots now require explicit nullable `relationship_coverage` in their identity commitments. Earlier records whose digest omitted null coverage are incompatible even when the serialized field is explicitly null. Artifact inventories require scoped coverage and explicit `integrity_contradictions`; obsolete `coverage_status` and prose-based coverage inference are removed. Regenerate old Evidence/snapshots and inventory observations with this release: saved files are not migrated and omitted facts are not filled with guessed defaults. ([db8128d](https://github.com/morluto/rea/commit/db8128dc0d1c4f731f3d19877fa00d25cf17f657), [0405c9d](https://github.com/morluto/rea/commit/0405c9d361de6d904dfa16cc0b6d8350407f2c9b), [a9588be](https://github.com/morluto/rea/commit/a9588be4ebdd20eeda975cb63ffcdb1cd868b340)).
|
||||
- **Saved Evidence:** Evidence and analysis snapshots now require explicit nullable `relationship_coverage` in their identity commitments. Artifact inventories require scoped coverage and explicit `integrity_contradictions`; obsolete `coverage_status` and prose-based coverage inference are removed. Regenerate old Evidence/snapshots and inventory observations with this release: saved files are not migrated and omitted facts are not filled with guessed defaults. ([db8128d](https://github.com/morluto/rea/commit/db8128dc0d1c4f731f3d19877fa00d25cf17f657), [0405c9d](https://github.com/morluto/rea/commit/0405c9d361de6d904dfa16cc0b6d8350407f2c9b), [a9588be](https://github.com/morluto/rea/commit/a9588be4ebdd20eeda975cb63ffcdb1cd868b340)).
|
||||
- **Browser capture results:** Consumers must retain resolved capture limits, event families and omission facts. Set desired budgets through the supported capture options; omitted caller inputs still select defaults, but stored observations must report the resolved values. Admitted content reads queue behind bounded concurrency. ([57a23f0](https://github.com/morluto/rea/commit/57a23f0a2da0fdc3835e203718ded8acea5a34e6)).
|
||||
- **Android version codes:** `inspect_adb_package.version_code` is a decimal string rather than a JSON number. Read it as a string, or use `BigInt` for numeric comparison. ([#3485](https://github.com/morluto/rea/pull/3485), [morluto/rea#3368](https://github.com/morluto/rea/issues/3368), [7666a48](https://github.com/morluto/rea/commit/7666a482b69dc7cabdbac3bbaf89a43e32300192)). Thanks @coygeek, @kaoru0822-kitauji.
|
||||
- **Native UI failures:** Failed or cancelled native UI scenarios now return typed errors with the collected report in `partial_observation`. Handle the error path and read that report instead of expecting a successful result containing failed steps. ([d909919](https://github.com/morluto/rea/commit/d9099190ab3a819bd3a9fd62a8714cef398016d5)).
|
||||
@@ -2683,7 +2581,7 @@ Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. R
|
||||
* enforce conventional pull request titles ([3a21f16](https://github.com/morluto/rea/commit/3a21f16c1832430765083636511a8c12dbccf2dd))
|
||||
* verify installed package with Linux Hopper ([4f97f69](https://github.com/morluto/rea/commit/4f97f699e4d5471557d1ce0ff91e144065d06a61))
|
||||
|
||||
## [0.3.0](https://github.com/morluto/rea/compare/rea-0.2.1...rea-agents-0.3.0) (2026-07-12)
|
||||
## [0.3.0](https://github.com/morluto/rea/compare/rea-agents-0.2.1...rea-agents-0.3.0) (2026-07-12)
|
||||
|
||||
|
||||
### Features
|
||||
@@ -2736,7 +2634,7 @@ Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. R
|
||||
* redesign and localize README ([051b6db](https://github.com/morluto/rea/commit/051b6db65687fd14826b6462cd623feb2504f123))
|
||||
* redesign and localize README ([910eda6](https://github.com/morluto/rea/commit/910eda6f323ba1151b0771fe5de6c60a91e3d46d))
|
||||
|
||||
## [0.2.0](https://github.com/morluto/rea/releases/tag/rea-0.2.0) (2026-07-12)
|
||||
## [0.2.0](https://github.com/morluto/rea/compare/rea-0.1.0...rea-0.2.0) (2026-07-12)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
+13
-64
@@ -10,32 +10,13 @@ and maintenance cost. Keep cosmetic polish with useful fixes; confusing or
|
||||
incorrect user instructions warrant standalone corrections. Collect other
|
||||
cosmetic-only suggestions in an issue rather than separate PRs.
|
||||
|
||||
For selected architectural owners and dependency paths, use the
|
||||
[architecture map](docs/architecture.mermaid). It is not a complete directory or
|
||||
import inventory. For current provider wiring, start with
|
||||
[`src/composition/binary.ts`](src/composition/binary.ts) for deep candidates and
|
||||
the other `src/composition/` factories for tool services, including
|
||||
[`gpu.ts`](src/composition/gpu.ts) and [`model.ts`](src/composition/model.ts).
|
||||
Live Frida, Cutter, GDB and Rizin construction also uses shared composition
|
||||
factories. For service ownership, availability and catalog declarations, follow
|
||||
the [capability composition guide](docs/capability-composition.md).
|
||||
Production factories construct fresh providers without starting engines or
|
||||
acquiring targets; optional adapter failures must preserve successful peers. Run
|
||||
For capability organization and provider composition, use the
|
||||
[architecture map](docs/architecture.mermaid). Production factories in
|
||||
`src/composition/` construct fresh providers without starting engines or acquiring
|
||||
targets; optional adapter failures must preserve successful peers. Run
|
||||
`npm run verify:test-discovery` after adding or moving tests. Resolved import
|
||||
boundaries are checked by `npm run verify:module-boundaries`, included in lint.
|
||||
|
||||
Before retiring an alias or facade, trace runtime registration, dynamic imports,
|
||||
JavaScript verifiers, subprocess scripts, and packaged entrypoints as well as
|
||||
TypeScript references. Tests may be the last consumers of a superseded internal
|
||||
path; migrate their distinct assertions to the current production path before
|
||||
removing it. Preserve validation, lazy loading, caller policy, and resource
|
||||
acquisition and cleanup supplied by the old boundary. Published APIs, persisted
|
||||
formats, and independently versioned bridges require an explicit compatibility
|
||||
decision; an internal representation alone does not establish that obligation.
|
||||
See [interpreting boundary facts once](docs/tool-design.md#interpret-boundary-facts-once)
|
||||
for admission ownership and the distinction between repeated validation and
|
||||
independent checks.
|
||||
|
||||
For analysis workflows and agent interfaces, follow the
|
||||
[performance priorities](AGENTS.md#performance). Explain the selected output
|
||||
scope, reuse/cache decisions, and relevant before/after measurements when these
|
||||
@@ -46,9 +27,7 @@ compare equivalent real tasks before claiming an improvement.
|
||||
|
||||
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
|
||||
When adding or changing an admission ceiling, budget, or deadline, follow
|
||||
[analysis settings, limits, and recovery](docs/design/limits-and-recovery.md)
|
||||
for ownership, operation snapshots, provider integration, and the steps for
|
||||
adding a runtime setting.
|
||||
[admission limits and recovery](docs/design/limits-and-recovery.md).
|
||||
|
||||
## Reporting issues
|
||||
|
||||
@@ -93,8 +72,7 @@ Before requesting maintainer review, the human owner must:
|
||||
passing. Follow the [verification requirements](#development-feedback-and-pr-verification)
|
||||
for required checks and real-provider coverage.
|
||||
- Complete the **Author readiness** declaration in the PR template, explicitly
|
||||
stating that the PR is ready for initial review, unless the PR author is
|
||||
exempt from the declaration as listed below.
|
||||
stating that the PR is ready for initial review.
|
||||
|
||||
Keep the PR description concise and current. Explain the problem, observable
|
||||
outcome, approach, and useful evidence; link long logs or put them in optional
|
||||
@@ -103,11 +81,7 @@ Human ownership and review remain required when CI or an automated reviewer
|
||||
reports success. Readiness requests maintainer review; it does not grant scope
|
||||
approval or permission to merge.
|
||||
|
||||
Draft status is optional. Draft PRs may request early feedback with open questions
|
||||
and incomplete checks. Agents preserve the current draft status unless asked to
|
||||
change it; missing readiness declarations or later updates do not require a PR
|
||||
to become a draft.
|
||||
|
||||
Draft PRs may request early feedback with open questions and incomplete checks.
|
||||
Leave the readiness declaration unchecked until the human owner has completed
|
||||
it. The declaration records initial readiness; later pushes do not invalidate
|
||||
it or require another human sign-off. Keep the PR description and affected
|
||||
@@ -115,12 +89,9 @@ validation current as work changes. AI agents must leave human declarations to
|
||||
the human owner and must not attest to a human review on their behalf.
|
||||
|
||||
The `PR readiness` check reports pending for drafts and requires a named human
|
||||
owner and all three template declarations for non-drafts. PRs authored by
|
||||
@morluto, @kaoru0822-kitauji, @N0zoM1z0, or @gensokyo-dev are exempt from
|
||||
the declaration; their human ownership, review, and evidence responsibilities
|
||||
above still apply. The check identifies authors by GitHub account ID. It checks
|
||||
that the declaration is filled, not whether the code or evidence is correct.
|
||||
Existing PRs can copy the **Author readiness** section from the current template.
|
||||
owner and all three template declarations for non-drafts. It checks that the
|
||||
declaration is filled, not whether the code or evidence is correct. Existing PRs
|
||||
can copy the **Author readiness** section from the current template.
|
||||
Open PRs must have distinct head commits because GitHub shares commit statuses.
|
||||
|
||||
Maintainers enable merge enforcement after the workflow is merged: run
|
||||
@@ -235,8 +206,6 @@ Use the development toolchain pinned in `.nvmrc` and
|
||||
`package.json#packageManager`. Run `nvm use` before installing dependencies;
|
||||
installed-package runtime support is documented in
|
||||
[installation](docs/installation.md#start-setup).
|
||||
Use Node.js and npm for source builds and verifier scripts; the installed
|
||||
package's Bun runtime support does not extend to those development scripts.
|
||||
|
||||
```bash
|
||||
npm ci
|
||||
@@ -251,15 +220,6 @@ catalogs, documentation projections, or managed verification evidence. Turbo
|
||||
caches deterministic builds and static checks across Git worktrees. After a package, lockfile, or managed-skill version change, run
|
||||
`npm run metadata:generate` before building.
|
||||
|
||||
Dependabot proposes updates for the npm lockfiles, website Python requirements,
|
||||
and GitHub Actions. Review dependency changes and run the checks relevant to
|
||||
their consumers. For dependencies maintained outside Dependabot, including
|
||||
`third_party/` and the HTTP/3 lab's pinned requirements, the maintainer should
|
||||
review upstream security advisories when updating or using them. Preserve source
|
||||
and version provenance, update license notices when needed, and report which
|
||||
verification was actually repeated. The lab requirements describe a recorded
|
||||
environment; a dependency update does not verify the revised environment.
|
||||
|
||||
Repository invariants live in [AGENTS.md](AGENTS.md). Keep provider-specific
|
||||
protocols in adapters and shared CLI/MCP workflows in the application layer.
|
||||
|
||||
@@ -286,10 +246,7 @@ The general source suite (`npm run test:fast`, `npm run test:changed`, and
|
||||
On a Windows checkout, some source tests fail because fixtures use POSIX paths,
|
||||
`#!/bin/sh` tool stubs, or POSIX rename semantics. Compare failures against the
|
||||
base revision, or run the suite on Linux or WSL, before attributing them to a
|
||||
change. The embedded-artifact parent-swap tests exercise the selected-path
|
||||
replacement when the rename succeeds; if Windows refuses it while the admitted
|
||||
file is open, they instead verify the refusal and cleanup. See
|
||||
[testing](docs/testing.md#developer-commands) for command scopes
|
||||
change. See [testing](docs/testing.md#developer-commands) for command scopes
|
||||
and lane selection.
|
||||
|
||||
`git submodule update --init --recursive` fails at the pinned Unblob submodule
|
||||
@@ -403,17 +360,9 @@ Real-provider execution is uncached.
|
||||
### Temporary files and diagnostics
|
||||
|
||||
Tests use `createTestTempDirectory` from `tests/fixtures/temporaryDirectory.ts`
|
||||
for exact-path, awaited cleanup, including failures and timeouts. It registers
|
||||
deletion with Vitest's `onTestFinished`; do not also add its result to a roots
|
||||
array or an `afterEach` removal. `createTestWorkspace` from
|
||||
`tests/support/workspace/workspaceFixture.ts` leaves cleanup with its caller,
|
||||
which must await `removeTestWorkspace`. Keep explicit resource teardown before
|
||||
directory deletion, using the case's `finally` or `afterEach`; Vitest runs
|
||||
`afterEach` before `onTestFinished` and awaits finish hooks in reverse
|
||||
registration order.
|
||||
for exact-path, awaited cleanup, including failures and timeouts.
|
||||
`npm run verify:test-temp-hygiene` runs the suite under a fresh `TMPDIR` and
|
||||
rejects remaining REA-owned paths. It cannot detect duplicate deletion owners or
|
||||
prove resource-close ordering. Never glob-clean shared `/tmp/rea-*` content.
|
||||
rejects remaining REA-owned paths. Never glob-clean shared `/tmp/rea-*` content.
|
||||
|
||||
`REA_LOG_LEVEL` selects `trace`, `debug`, `info`, `warn`, `error`, `fatal`, or
|
||||
`silent`. Logs go to stderr: MCP defaults to `info`, and one-shot CLI logging
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md)
|
||||
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md) · [Română](docs/readme/README_ro.md)
|
||||
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md)
|
||||
|
||||
# REA: Reverse Engineer Anything
|
||||
|
||||
@@ -20,9 +20,7 @@
|
||||
|
||||
**[Website](https://rea.tools/) · [Guides](https://rea.tools/guides/) · [Showcases](https://rea.tools/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Join the REA community on Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [Build RL environments](#build-rl-environments) · [FAQ](#faq) · [Documentation](#documentation)
|
||||
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [FAQ](#faq) · [Documentation](#documentation)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -64,7 +62,7 @@ See a feature in an app that you want in your own product? Ask your agent to inv
|
||||
|
||||
REA connects your agent to tools for inspecting native binaries, JavaScript and Electron apps, .NET assemblies, and websites. You can also use the same tools from your terminal. Analysis runs locally, and results include the evidence and limitations behind each conclusion.
|
||||
|
||||
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, IDA or Kuna installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
|
||||
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, or IDA installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
|
||||
|
||||
> **[Visit the REA website](https://rea.tools/)** for setup instructions, illustrated guides, and real case studies.
|
||||
|
||||
@@ -85,15 +83,12 @@ npx rea-agents setup
|
||||
|
||||
Choose your agents, review the proposed changes, and approve them. Setup adds
|
||||
REA's MCP server and matching workflow instructions, with backups of existing
|
||||
configuration. Restart your agent afterward. Setup records the selected runtime and REA
|
||||
entry script directly. If acquired through a package runner, keep its cache entry
|
||||
available. Run package acquisition from a trusted directory outside the target
|
||||
project. Installation and recovery details are in the setup guide below.
|
||||
configuration. Restart your agent afterward.
|
||||
|
||||
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
|
||||
[other agents](docs/installation.md#supported-agents). See
|
||||
[installation and setup](docs/installation.md) for provider configuration and
|
||||
manual MCP registration and Claude Code/Codex plugin installation.
|
||||
manual MCP registration.
|
||||
|
||||
MCP uses compact input schemas by default to reduce context overhead. Set
|
||||
`REA_MCP_INPUT_SCHEMA_PROFILE=full` in your MCP registration when you need the
|
||||
@@ -101,12 +96,6 @@ complete advertised field guidance and nested validation details. Both profiles
|
||||
keep the full tool set and canonical server validation. See
|
||||
"Compact input schema profile" in the MCP contracts guide.
|
||||
|
||||
Complete output schemas are advertised by default. If your client caps the
|
||||
complete `tools/list` frame, set `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact` to
|
||||
bound each advertised output schema, or `off` to omit output schemas entirely.
|
||||
Both settings keep canonical server-side validation and complete structured
|
||||
results. See "Output schemas" in the MCP contracts guide.
|
||||
|
||||
### Ask your agent
|
||||
|
||||
```text
|
||||
@@ -194,7 +183,7 @@ Additional tools and host support depend on the target:
|
||||
| Target | What REA returns | Requirements and guide |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/native-investigation.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
|
||||
@@ -203,10 +192,7 @@ Additional tools and host support depend on the target:
|
||||
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
|
||||
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
|
||||
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
|
||||
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
|
||||
@@ -255,12 +241,7 @@ for features added since the latest npm release.
|
||||
|
||||
## Showcases
|
||||
|
||||
<a href="https://rea.tools/showcase/">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/rea-showcases-dark.svg" />
|
||||
<img alt="Illustrations of the DX-Ball sound-pan, Notion clipboard-bridge, and TH04 bullet-ring showcases" src="docs/assets/rea-showcases.svg" />
|
||||
</picture>
|
||||
</a>
|
||||
[](https://rea.tools/showcase/)
|
||||
|
||||
### DX-Ball: reconstruct a sound-pan calculation
|
||||
|
||||
@@ -287,53 +268,11 @@ historical compiler output.
|
||||
[Read the case study](https://rea.tools/showcase/th04/) ·
|
||||
[Reconstruction repository](https://github.com/N0zoM1z0/th04)
|
||||
|
||||
### Build RL environments with REA
|
||||
|
||||
Record a web app session, turn its API into tools, and export an environment an AI agent can use. Run it with AgentEnv, then add tasks and reward checks for RL training or evaluation.
|
||||
|
||||
[Read the guide](https://rea.tools/guides/rl-environments/) ·
|
||||
[Watch the video](https://x.com/reatools/status/2112574822584918075?s=20)
|
||||
|
||||
If you've used REA on something interesting, we'd love to see it. Share your
|
||||
case in an [issue](https://github.com/morluto/rea/issues) or a
|
||||
[pull request](https://github.com/morluto/rea/pulls), including the target,
|
||||
your question, how REA helped, and what you found.
|
||||
|
||||
## Build RL environments
|
||||
|
||||
Give an AI agent a local environment to practice tasks such as finding an issue or creating a folder, without running the original app. Use it for reinforcement learning (RL) or to test how well an agent completes tasks.
|
||||
|
||||
REA turns recorded API requests and responses into tools the agent can call. The environment uses the recorded data and supports changes only when the recording confirms their effects.
|
||||
|
||||
With REA set up in your agent, replace the example URL and folder tasks below with your own:
|
||||
|
||||
```text
|
||||
Use REA to build an RL environment from my test web app at
|
||||
http://127.0.0.1:3000.
|
||||
Cover listing, creating, renaming, and deleting folders. You may create and
|
||||
remove test folders. Leave existing data unchanged.
|
||||
Save it in a folder named my-env. Check which operations work and tell me
|
||||
how to run it.
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary>Use the command line instead</summary>
|
||||
|
||||
Use these commands if you already have a file containing the app’s recorded browser requests and responses. Running the result also requires AgentEnv and Docker.
|
||||
|
||||
```sh
|
||||
rea environment-build ./session.har ./my-env --name my-env --json
|
||||
agent-env run ./my-env
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
REA exports a runnable environment with a basic test to check that it starts and responds. Use it with your RL framework to define task goals, add reward checks, and run training.
|
||||
|
||||
[Build an environment](https://rea.tools/guides/rl-environments/) ·
|
||||
[Environment reference](docs/agent-environments.md) ·
|
||||
[Watch the demo on X](https://x.com/reatools/status/2112574822584918075?s=20)
|
||||
|
||||
## FAQ
|
||||
|
||||
<details>
|
||||
@@ -346,13 +285,11 @@ Any agent that supports local MCP servers. Setup configures the
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Do I need a native analysis engine?</strong></summary>
|
||||
<summary><strong>Do I need Hopper, Ghidra or IDA?</strong></summary>
|
||||
|
||||
Deep native analysis uses Hopper, Ghidra, IDA or Kuna, depending on the target
|
||||
and operation. Static JavaScript and .NET inspection work without a native
|
||||
analysis engine. Setup can install Hopper after approval; Ghidra, IDA and Kuna
|
||||
use your existing installations. See [provider setup](docs/installation.md#hopper)
|
||||
for the options and their supported targets.
|
||||
Deep native analysis uses one of them. Static JavaScript and .NET inspection
|
||||
work without a native analysis engine. Setup can install Hopper after approval;
|
||||
Ghidra and IDA use your existing installations. See [provider setup](docs/installation.md#hopper).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -432,7 +369,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Star history
|
||||
|
||||
🎉 **400,000 GitHub stars — thank you!**
|
||||
🎉 **300,000 GitHub stars — thank you!**
|
||||
|
||||
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
|
||||
|
||||
@@ -462,7 +399,7 @@ to improve the code or docs.
|
||||
|
||||
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
|
||||
[testing](docs/testing.md) for verification lanes, and the
|
||||
[architecture map](docs/architecture.mermaid) for selected layers and dependency paths.
|
||||
[architecture map](docs/architecture.mermaid) for the project structure.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
+1
-1
@@ -10,7 +10,7 @@ Do not disclose vulnerabilities, proof-of-concept exploits, credentials,
|
||||
private binaries, Hopper documents, or private Ghidra projects in a public
|
||||
issue.
|
||||
|
||||
Use **Report a vulnerability** in the repository's **Security** tab as the preferred way to submit the report privately. If GitHub private reporting is unavailable to you, email [[email protected]](mailto:[email protected]). Do not open a public issue to request a private contact channel.
|
||||
Use **Report a vulnerability** in the repository's **Security** tab to submit the report privately. Do not open a public issue to request a private contact channel.
|
||||
|
||||
Include the affected REA version, operating system/distribution, selected
|
||||
provider and version, impact, reproduction conditions, and any suggested
|
||||
|
||||
+21
-23
@@ -16,8 +16,6 @@ notices supplied with the software you actually obtain or redistribute.
|
||||
| Material | Use and licensing records |
|
||||
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| npm dependencies | [Complete inventory](licenses/npm-dependencies.txt) of every entry in `package-lock.json`, including transitive, optional platform, development, and website dependencies. [Runtime notices](licenses/npm-runtime-notices.txt) preserve the copyright, license, and NOTICE documents from integrity-verified release archives, plus the MIT notice for Rolldown's generated runtime helpers. Development dependencies are inventoried but are not all distributed in REA's npm package. |
|
||||
| pefile ordinal tables | Import-hash ordinal names are derived from [pefile 2024.8.26](https://github.com/erocarrera/pefile/tree/4b3b1e2e568a88d4f1897d694d684f23d9e270c4), copyright 2004–2024 Ero Carrera. Preserve the [MIT license](licenses/pefile-2024.8.26-LICENSE.txt). The Python package is not a runtime dependency. |
|
||||
| UF2 family names | UF2 family IDs and short names are copied from [microsoft/uf2 `utils/uf2families.json`](https://github.com/microsoft/uf2/blob/cf3bd7f9c274b949528f8cd6a19eef2502a687d2/utils/uf2families.json), copyright Microsoft Corporation. Preserve the [MIT license](licenses/microsoft-uf2-MIT.txt). |
|
||||
| NativeAOT parser | REA's derived parser uses the MIT-licensed [ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot/tree/effeb734fc570c32650f88b159608979dc7b423e) work of Washi and the NativeAOT layout definitions in [dotnet/runtime](https://github.com/dotnet/runtime/tree/a3b2d40328be0be3f8dd43f0e5cc925b8827b044). Preserve [Washi's MIT notice](licenses/nativeaot-parser-MIT.txt) and the [.NET Foundation MIT notice](licenses/dotnet-runtime-MIT.txt). |
|
||||
| EVM analysis | `[email protected]`, MIT, copyright 2023 Maxim Andreev. The published npm archive omits its license document; REA retains the [original upstream license](third_party/evmole/LICENSE) and [provenance](third_party/evmole/README.md). It is also included in the runtime notices. |
|
||||
| Windows native addon | The release addon is compiled with MinGW-w64 and statically links GCC runtime libraries. Preserve the [MinGW-w64 runtime notices](licenses/MinGW-w64-runtime.txt), [GPLv3 text](licenses/GPL-3.0.txt), and [GCC Runtime Library Exception 3.1](licenses/GCC-Runtime-Library-Exception-3.1.txt). The exception governs eligible compiled combinations; it does not relicense the runtime itself under MIT. The Node-API build headers are from Node 22.19.0; its [original license and component notices](licenses/Node-22.19.0-LICENSE.txt) are retained. Node itself is supplied by the user. Snapshot provenance and release-toolchain review requirements are in [licenses/README.md](licenses/README.md). |
|
||||
@@ -54,27 +52,27 @@ REA's adapters do not grant, replace, or waive these licenses. The upstream
|
||||
executables and their dependency graphs are supplied separately. Git submodules
|
||||
pin inspection sources; they are not included in the npm package.
|
||||
|
||||
| Project | License and source | REA relationship |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
|
||||
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
|
||||
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
|
||||
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
|
||||
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
|
||||
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
|
||||
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
|
||||
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
|
||||
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
|
||||
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
|
||||
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
|
||||
| Project | License and source | REA relationship |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra 12.1.4. Copies of its [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.833/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.833/NOTICE) | Bring your own `kuna`; REA verified 1.833 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
|
||||
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
|
||||
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
|
||||
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
|
||||
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
|
||||
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
|
||||
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
|
||||
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
|
||||
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
|
||||
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
|
||||
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
|
||||
|
||||
Ghidra's original notice credits software developed at the National Security
|
||||
Agency. It explains that US-government-created portions are not subject to US
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
# Plugin icon
|
||||
|
||||
`rea-plugin.png` is a square adaptation of the left panel of REA's GitHub
|
||||
social preview, retrieved on 2026-10-28:
|
||||
|
||||
https://repository-images.githubusercontent.com/1209966933/1a3fef9a-be4b-41f4-8830-a94c0e8288cc
|
||||
|
||||
The image was adapted with imagegen, preserving the cyan REA lettering,
|
||||
pixel typography, orange rule, and taglines. Both plugin manifests reference
|
||||
this shared asset; only the PNG is included in the npm package.
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 1.1 MiB |
@@ -1,35 +1,19 @@
|
||||
import com.atxx.jhmcp.*;
|
||||
import com.google.gson.*;
|
||||
import com.android.tools.smali.dexlib2.Opcodes;
|
||||
import com.android.tools.smali.dexlib2.dexbacked.DexBackedDexFile;
|
||||
import com.android.tools.smali.dexlib2.dexbacked.DexBackedClassDef;
|
||||
import com.android.tools.smali.dexlib2.dexbacked.DexBackedMethod;
|
||||
import io.modelcontextprotocol.kotlin.sdk.server.*;
|
||||
import io.modelcontextprotocol.kotlin.sdk.types.*;
|
||||
import jadx.api.*;
|
||||
import jadx.api.plugins.input.data.annotations.EncodedValue;
|
||||
import jadx.api.plugins.input.data.attributes.JadxAttrType;
|
||||
import jadx.api.plugins.input.data.ICodeReader;
|
||||
import jadx.api.plugins.input.data.IMethodRef;
|
||||
import jadx.api.plugins.input.insns.InsnIndexType;
|
||||
import jadx.api.plugins.input.insns.Opcode;
|
||||
import jadx.core.codegen.TypeGen;
|
||||
import jadx.core.dex.info.MethodInfo;
|
||||
import jadx.core.dex.instructions.args.ArgType;
|
||||
import jadx.core.dex.nodes.*;
|
||||
import jadx.core.xmlgen.BinaryXMLParser;
|
||||
import java.io.PrintStream;
|
||||
import java.io.StringReader;
|
||||
import java.io.InputStream;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.CountDownLatch;
|
||||
import java.util.stream.Collectors;
|
||||
import java.util.zip.CRC32;
|
||||
import java.util.zip.ZipEntry;
|
||||
import java.util.zip.ZipFile;
|
||||
import kotlin.Unit;
|
||||
import kotlin.ResultKt;
|
||||
import kotlin.coroutines.Continuation;
|
||||
@@ -39,10 +23,6 @@ import kotlin.coroutines.intrinsics.IntrinsicsKt;
|
||||
import kotlinx.coroutines.BuildersKt;
|
||||
import kotlinx.io.CoreKt;
|
||||
import kotlinx.io.JvmCoreKt;
|
||||
import javax.xml.parsers.DocumentBuilderFactory;
|
||||
import org.w3c.dom.Element;
|
||||
import org.w3c.dom.Node;
|
||||
import org.xml.sax.InputSource;
|
||||
|
||||
/** REA-owned metadata boundary for the unmodified jadx-headless-mcp 0.7.1 JAR. */
|
||||
class ReaJadxBridge {
|
||||
@@ -50,27 +30,15 @@ class ReaJadxBridge {
|
||||
private final SessionHolder holder;
|
||||
private JadxSession indexedSession;
|
||||
private Map<String, List<ClassSnapshot>> classes = Map.of();
|
||||
private JadxSession manifestSession;
|
||||
private ManifestSnapshot manifestSnapshot;
|
||||
private ExternalInventory externalInventory;
|
||||
|
||||
private record MethodSnapshot(String name, String rawName, String shortId, List<String> bodyAbsenceMarkers,
|
||||
Map<String, Object> summary) {}
|
||||
private record ClassSnapshot(JavaClass node, List<MethodSnapshot> methods, Map<String, Object> summary,
|
||||
Map<String, Object> summaryWithValues) {}
|
||||
private record ManifestSnapshot(String text, Map<String, Object> summary) {}
|
||||
private record ExternalMethod(MethodInfo info, Map<String, Object> summary) {}
|
||||
private record ExternalInventory(JadxSession session, String className, String methodName,
|
||||
List<ExternalMethod> methods) {}
|
||||
private record ClassSnapshot(JavaClass node, List<MethodSnapshot> methods, Map<String, Object> summary) {}
|
||||
|
||||
private static final class InputFailure extends IllegalArgumentException {
|
||||
final String field;
|
||||
InputFailure(String field, String message) { super(message); this.field = field; }
|
||||
}
|
||||
private static final class ReferenceLimit extends IllegalArgumentException {
|
||||
final String stage;
|
||||
ReferenceLimit(String stage) { super("External reference representation limit"); this.stage = stage; }
|
||||
}
|
||||
|
||||
private ReaJadxBridge(SessionHolder holder) { this.holder = holder; }
|
||||
|
||||
@@ -125,7 +93,6 @@ class ReaJadxBridge {
|
||||
args.has("threads") ? integer(args, "threads", 1) : null,
|
||||
args.has("code_cache_size") ? integer(args, "code_cache_size", 64) : null,
|
||||
optionalString(args, "resources"));
|
||||
clearManifest();
|
||||
return loadCall(next -> holder.load(path, options, next), continuation);
|
||||
}
|
||||
|
||||
@@ -166,14 +133,6 @@ class ReaJadxBridge {
|
||||
return value.getAsString();
|
||||
}
|
||||
|
||||
private static boolean flag(JsonObject args, String name) {
|
||||
JsonElement value = args.get(name);
|
||||
if (value == null) return false;
|
||||
if (!value.isJsonPrimitive() || !value.getAsJsonPrimitive().isBoolean())
|
||||
throw new IllegalArgumentException(name + " must be a boolean");
|
||||
return value.getAsBoolean();
|
||||
}
|
||||
|
||||
private static int integer(JsonObject args, String name, int fallback) {
|
||||
JsonElement value = args.get(name);
|
||||
if (value == null) return fallback;
|
||||
@@ -194,292 +153,7 @@ class ReaJadxBridge {
|
||||
return current;
|
||||
}
|
||||
|
||||
private void clearManifest() {
|
||||
manifestSession = null;
|
||||
manifestSnapshot = null;
|
||||
externalInventory = null;
|
||||
}
|
||||
|
||||
// One bounded DEX at a time, beside the already admitted analysis JVM. The
|
||||
// response allowance accounts for worst-case JSON escaping before retention.
|
||||
private static final int NATIVE_DEX_BYTES = 64 * 1024 * 1024;
|
||||
private static final long NATIVE_TOTAL_DEX_BYTES = 256L * 1024 * 1024;
|
||||
// This JSON is escaped again inside an 8 MiB MCP frame.
|
||||
private static final long NATIVE_JSON_BYTES = 4L * 1024 * 1024;
|
||||
|
||||
/** The bundled decoder allocates its declared UTF-16 length before validating bytes. */
|
||||
private static final class NativeDex extends DexBackedDexFile {
|
||||
private OptionalIndexedSection<String> boundedStrings;
|
||||
NativeDex(byte[] bytes) { super(Opcodes.getDefault(), bytes); }
|
||||
|
||||
@Override public OptionalIndexedSection<String> getStringSection() {
|
||||
if (boundedStrings == null) {
|
||||
var source = super.getStringSection();
|
||||
boundedStrings = new OptionalIndexedSection<>() {
|
||||
@Override public int size() { return source.size(); }
|
||||
@Override public int getOffset(int index) { return source.getOffset(index); }
|
||||
@Override public String getOptional(int index) { return index == -1 ? null : get(index); }
|
||||
@Override public String get(int index) {
|
||||
int offset = getBuffer().readSmallUint(source.getOffset(index));
|
||||
int units = getDataBuffer().readerAt(offset).readSmallUleb128();
|
||||
if (units > NATIVE_JSON_BYTES / 6
|
||||
|| units > getDataBuffer().getBuf().length - offset)
|
||||
throw new IllegalArgumentException("DEX string exceeds the native decoding allowance");
|
||||
return source.get(index);
|
||||
}
|
||||
};
|
||||
}
|
||||
return boundedStrings;
|
||||
}
|
||||
}
|
||||
|
||||
private static void checkNativeInterrupted() {
|
||||
if (Thread.currentThread().isInterrupted())
|
||||
throw new java.util.concurrent.CancellationException("Native declaration inventory interrupted");
|
||||
}
|
||||
|
||||
private static final class NativeInventory {
|
||||
final List<Map<String, Object>> methods = new ArrayList<>();
|
||||
final List<String> unknowns = new ArrayList<>();
|
||||
long retainedBytes = 1024;
|
||||
int dexEntries;
|
||||
int examinedDexEntries;
|
||||
int examinedClasses;
|
||||
int examinedMethods;
|
||||
boolean partial;
|
||||
boolean diagnosticLimit;
|
||||
|
||||
void unknown(String reason) {
|
||||
partial = true;
|
||||
long bytes = 64L + reason.length() * 6L;
|
||||
if (unknowns.size() < 16 && retainedBytes + bytes <= NATIVE_JSON_BYTES - 4096) {
|
||||
unknowns.add(reason);
|
||||
retainedBytes += bytes;
|
||||
} else if (!diagnosticLimit) {
|
||||
diagnosticLimit = true;
|
||||
unknowns.add("Additional native-inventory limitations exceed the retained diagnostic allowance.");
|
||||
}
|
||||
}
|
||||
|
||||
boolean retain(Map<String, Object> method, long bytes) {
|
||||
if (retainedBytes + bytes > NATIVE_JSON_BYTES - 4096) {
|
||||
unknown("Native declarations exceed the 4 MiB retained JSON allowance; remaining declarations are unknown.");
|
||||
return false;
|
||||
}
|
||||
retainedBytes += bytes;
|
||||
methods.add(method);
|
||||
return true;
|
||||
}
|
||||
|
||||
Map<String, Object> result() {
|
||||
return Map.of("scope", "apk-root-dex-declarations", "methods", List.copyOf(methods),
|
||||
"coverage", Map.of("status", partial ? "partial" : "complete",
|
||||
"dex_entries", dexEntries, "examined_dex_entries", examinedDexEntries,
|
||||
"examined_classes", examinedClasses, "examined_methods", examinedMethods),
|
||||
"unknowns", List.copyOf(unknowns));
|
||||
}
|
||||
}
|
||||
|
||||
/** Match Android's standard root multidex carriers, without a basename guess. */
|
||||
private static boolean nativeDexEntry(String name) {
|
||||
return name.equals("classes.dex") || name.matches("classes[0-9]+\\.dex");
|
||||
}
|
||||
|
||||
private static byte[] readNativeDex(ZipFile archive, ZipEntry entry) throws IOException {
|
||||
long size = entry.getSize();
|
||||
if (size < 0 || size > NATIVE_DEX_BYTES)
|
||||
throw new IllegalArgumentException("DEX exceeds the 64 MiB single-member read allowance");
|
||||
byte[] bytes = new byte[(int) size];
|
||||
CRC32 crc = new CRC32();
|
||||
try (InputStream input = archive.getInputStream(entry)) {
|
||||
int offset = 0;
|
||||
while (offset < bytes.length) {
|
||||
checkNativeInterrupted();
|
||||
int read = input.read(bytes, offset, Math.min(65536, bytes.length - offset));
|
||||
if (read < 0) throw new IOException("DEX is shorter than its ZIP declaration");
|
||||
if (read == 0) continue;
|
||||
crc.update(bytes, offset, read);
|
||||
offset += read;
|
||||
}
|
||||
if (input.read() != -1) throw new IOException("DEX exceeds its ZIP declaration");
|
||||
}
|
||||
if (entry.getCrc() < 0 || entry.getCrc() != crc.getValue())
|
||||
throw new IOException("DEX does not match its ZIP CRC32 declaration");
|
||||
return bytes;
|
||||
}
|
||||
|
||||
/** Raw DEX ownership precedes JADX's duplicate-class selection and aliases. */
|
||||
static Map<String, Object> nativeDeclarations(Path apk) {
|
||||
NativeInventory inventory = new NativeInventory();
|
||||
try (ZipFile archive = new ZipFile(apk.toFile())) {
|
||||
Map<String, Integer> names = new HashMap<>();
|
||||
var directory = archive.entries();
|
||||
while (directory.hasMoreElements()) {
|
||||
checkNativeInterrupted();
|
||||
ZipEntry entry = directory.nextElement();
|
||||
if (!entry.isDirectory() && nativeDexEntry(entry.getName())) {
|
||||
inventory.dexEntries++;
|
||||
names.merge(entry.getName(), 1, Integer::sum);
|
||||
}
|
||||
}
|
||||
long totalBytes = 0;
|
||||
int entryIndex = -1;
|
||||
directory = archive.entries();
|
||||
while (directory.hasMoreElements()) {
|
||||
checkNativeInterrupted();
|
||||
ZipEntry entry = directory.nextElement();
|
||||
entryIndex++;
|
||||
String path = entry.getName();
|
||||
if (entry.isDirectory() || !nativeDexEntry(path)) continue;
|
||||
if (names.get(path) != 1) {
|
||||
inventory.unknown("Duplicate root DEX entry name cannot be selected uniquely by the ZIP reader: " + path);
|
||||
continue;
|
||||
}
|
||||
if (entry.getSize() < 0 || entry.getSize() > NATIVE_DEX_BYTES
|
||||
|| entry.getSize() > NATIVE_TOTAL_DEX_BYTES - totalBytes) {
|
||||
inventory.unknown("Root DEX member exceeds the 64 MiB member or 256 MiB total read allowance: " + path);
|
||||
continue;
|
||||
}
|
||||
totalBytes += entry.getSize();
|
||||
try {
|
||||
DexBackedDexFile dex = new NativeDex(readNativeDex(archive, entry));
|
||||
var classes = dex.getClassSection();
|
||||
for (int classIndex = 0; classIndex < classes.size(); classIndex++) {
|
||||
checkNativeInterrupted();
|
||||
DexBackedClassDef cls = classes.get(classIndex);
|
||||
String type = cls.getType();
|
||||
if (!type.startsWith("L") || !type.endsWith(";"))
|
||||
throw new IllegalArgumentException("Class definition has no reference type");
|
||||
String className = type.substring(1, type.length() - 1).replace('/', '.');
|
||||
// Passing false retains duplicate encoded method references too.
|
||||
for (Iterable<? extends DexBackedMethod> group : List.of(
|
||||
cls.getDirectMethods(false), cls.getVirtualMethods(false))) {
|
||||
for (DexBackedMethod method : group) {
|
||||
checkNativeInterrupted();
|
||||
inventory.examinedMethods++;
|
||||
if ((method.getAccessFlags() & 0x100) == 0) continue;
|
||||
String methodName = method.getName();
|
||||
String returnType = method.getReturnType();
|
||||
long characters = 2L + returnType.length();
|
||||
for (CharSequence parameter : method.getParameterTypes()) {
|
||||
characters += parameter.length();
|
||||
if (characters * 6L > NATIVE_JSON_BYTES) break;
|
||||
}
|
||||
long bytes = 512L + 6L * (className.length() + methodName.length()
|
||||
+ path.length() + characters);
|
||||
if (inventory.retainedBytes + bytes > NATIVE_JSON_BYTES - 4096) {
|
||||
inventory.unknown("Native declarations exceed the 4 MiB retained JSON allowance; remaining declarations are unknown.");
|
||||
return inventory.result();
|
||||
}
|
||||
StringBuilder descriptor = new StringBuilder((int) characters).append('(');
|
||||
for (CharSequence parameter : method.getParameterTypes()) descriptor.append(parameter);
|
||||
descriptor.append(')').append(returnType);
|
||||
Map<String, Object> row = Map.of("source_index", inventory.methods.size(),
|
||||
"class_name", className, "method_name", methodName, "descriptor", descriptor.toString(),
|
||||
"dex_entry", path, "dex_entry_index", entryIndex, "class_index", classIndex,
|
||||
"method_index", method.getMethodIndex());
|
||||
if (!inventory.retain(row, bytes)) return inventory.result();
|
||||
}
|
||||
}
|
||||
inventory.examinedClasses++;
|
||||
}
|
||||
inventory.examinedDexEntries++;
|
||||
} catch (IOException | RuntimeException failure) {
|
||||
if (failure instanceof java.util.concurrent.CancellationException) throw (RuntimeException) failure;
|
||||
inventory.unknown("Root DEX native declarations could not be fully read from " + path + ": "
|
||||
+ failure.getClass().getSimpleName()
|
||||
+ (failure.getMessage() == null ? "" : " (" + failure.getMessage() + ")"));
|
||||
}
|
||||
}
|
||||
} catch (IOException failure) {
|
||||
// A failed archive close must fail the operation so the provider retires
|
||||
// its owned JVM; do not cache a result with lost file-resource ownership.
|
||||
throw new IllegalArgumentException("APK ZIP native declaration inventory could not finish its reader", failure);
|
||||
}
|
||||
return inventory.result();
|
||||
}
|
||||
|
||||
/** Original archive identity, never a deobfuscated basename or name suffix. */
|
||||
private ManifestSnapshot manifest(JadxSession current) {
|
||||
if (manifestSession != current || manifestSnapshot == null) {
|
||||
ResourceFile selected = null;
|
||||
for (ResourceFile resource : current.getResources()) {
|
||||
if (resource.getType() != ResourceType.MANIFEST
|
||||
|| !resource.getOriginalName().equals("AndroidManifest.xml")) continue;
|
||||
if (selected != null)
|
||||
throw new IllegalArgumentException("Root AndroidManifest.xml resource identity is ambiguous");
|
||||
selected = resource;
|
||||
}
|
||||
if (selected == null) throw new IllegalArgumentException("Root AndroidManifest.xml not found");
|
||||
String xml = manifestText(selected);
|
||||
ManifestSnapshot snapshot = new ManifestSnapshot(xml, manifestSummary(xml));
|
||||
manifestSnapshot = snapshot;
|
||||
manifestSession = current;
|
||||
}
|
||||
return manifestSnapshot;
|
||||
}
|
||||
|
||||
/** Decode with resource names but without JADX's class alias substitution. */
|
||||
static String manifestText(ResourceFile resource) {
|
||||
var decompiler = resource.getDecompiler();
|
||||
// BinaryXMLParser resolves android:name against its root's class identities
|
||||
// and replaces raw '$' and relative names with Java-style aliases. A fresh
|
||||
// root owns no classes, while its resource-name table preserves reference
|
||||
// rendering from the loaded target.
|
||||
RootNode root = new RootNode(decompiler.getArgs());
|
||||
root.getConstValues().setResourcesNames(decompiler.getRoot().getConstValues().getResourcesNames());
|
||||
try {
|
||||
return ResourcesLoader.decodeStream(resource,
|
||||
(size, input) -> new BinaryXMLParser(root).parse(input).getCodeStr());
|
||||
} catch (jadx.core.utils.exceptions.JadxException failure) {
|
||||
throw new IllegalArgumentException("Root AndroidManifest.xml did not decode as text", failure);
|
||||
}
|
||||
}
|
||||
|
||||
/** One parsed XML owner supplies namespace-aware values alongside the exact decoded text. */
|
||||
static Map<String, Object> manifestSummary(String xml) {
|
||||
Element root;
|
||||
try {
|
||||
// Match REA's existing JDK XML setup: document types are unsupported.
|
||||
// Parse an in-memory string; XInclude and external DTD/schema I/O stay disabled.
|
||||
DocumentBuilderFactory factory = DocumentBuilderFactory.newDefaultNSInstance();
|
||||
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
|
||||
factory.setXIncludeAware(false);
|
||||
factory.setAttribute(javax.xml.XMLConstants.ACCESS_EXTERNAL_DTD, "");
|
||||
factory.setAttribute(javax.xml.XMLConstants.ACCESS_EXTERNAL_SCHEMA, "");
|
||||
root = factory.newDocumentBuilder().parse(new InputSource(new StringReader(xml)))
|
||||
.getDocumentElement();
|
||||
} catch (javax.xml.parsers.ParserConfigurationException | org.xml.sax.SAXException
|
||||
| java.io.IOException failure) {
|
||||
throw new IllegalArgumentException("Root AndroidManifest.xml is not supported well-formed XML", failure);
|
||||
}
|
||||
if (!root.getTagName().equals("manifest") || root.getNamespaceURI() != null)
|
||||
throw new IllegalArgumentException("AndroidManifest.xml root must be an unnamespaced <manifest>");
|
||||
String android = "http://schemas.android.com/apk/res/android";
|
||||
String minSdk = "";
|
||||
String targetSdk = "";
|
||||
List<String> permissions = new ArrayList<>();
|
||||
for (Node child = root.getFirstChild(); child != null; child = child.getNextSibling()) {
|
||||
if (!(child instanceof Element element) || element.getNamespaceURI() != null) continue;
|
||||
if (element.getTagName().equals("uses-sdk")) {
|
||||
minSdk = element.getAttributeNS(android, "minSdkVersion");
|
||||
targetSdk = element.getAttributeNS(android, "targetSdkVersion");
|
||||
} else if (element.getTagName().equals("uses-permission")) {
|
||||
String permission = element.getAttributeNS(android, "name");
|
||||
if (!permission.isEmpty()) permissions.add(permission);
|
||||
}
|
||||
}
|
||||
Map<String, Object> summary = new LinkedHashMap<>();
|
||||
summary.put("package", root.getAttribute("package"));
|
||||
summary.put("version_name", root.getAttributeNS(android, "versionName"));
|
||||
summary.put("version_code", root.getAttributeNS(android, "versionCode"));
|
||||
summary.put("min_sdk", minSdk);
|
||||
summary.put("target_sdk", targetSdk);
|
||||
summary.put("permissions", List.copyOf(permissions));
|
||||
return Collections.unmodifiableMap(summary);
|
||||
}
|
||||
|
||||
private void indexClasses() {
|
||||
private ClassSnapshot selected(JsonObject args) {
|
||||
JadxSession current = session();
|
||||
if (indexedSession != current) {
|
||||
Map<String, List<ClassSnapshot>> index = new LinkedHashMap<>();
|
||||
@@ -505,60 +179,27 @@ class ReaJadxBridge {
|
||||
.map(Object::toString).collect(Collectors.joining(", "))
|
||||
+ "): " + method.getReturnType().toString();
|
||||
return new MethodSnapshot(name, rawName, shortId, List.copyOf(bodyAbsenceMarkers), Map.of(
|
||||
"name", name, "raw_name", rawName, "signature", signature,
|
||||
"name", name, "signature", signature,
|
||||
"dex_descriptor", descriptor,
|
||||
"is_constructor", method.isConstructor(), "def_pos", 0));
|
||||
}).toList();
|
||||
List<FieldNode> orderedFields = new ArrayList<>(node.getFields());
|
||||
orderedFields.sort(Comparator.comparing(FieldNode::getAlias));
|
||||
List<Map<String, Object>> fields = new ArrayList<>();
|
||||
List<Map<String, Object>> valuedFields = new ArrayList<>();
|
||||
boolean anyValue = false;
|
||||
for (FieldNode field : orderedFields) {
|
||||
// Raw DEX identity for mapping lookups; the display type can carry generics.
|
||||
Map<String, Object> entry = Map.of("name", field.getAlias(),
|
||||
"raw_name", field.getFieldInfo().getName(),
|
||||
"type", field.getType().toString(),
|
||||
"dex_type", TypeGen.signature(field.getFieldInfo().getType()), "def_pos", 0);
|
||||
fields.add(entry);
|
||||
Map<String, Object> value = dexStaticValue(field);
|
||||
if (value == null) valuedFields.add(entry);
|
||||
else {
|
||||
anyValue = true;
|
||||
Map<String, Object> valued = new LinkedHashMap<>(entry);
|
||||
valued.put("dex_static_value", value);
|
||||
valuedFields.add(Collections.unmodifiableMap(valued));
|
||||
}
|
||||
}
|
||||
List<Map<String, Object>> fields = node.getFields().stream()
|
||||
.sorted(Comparator.comparing(FieldNode::getAlias))
|
||||
.map(field -> Map.<String, Object>of("name", field.getAlias(),
|
||||
"type", field.getType().toString(), "def_pos", 0)).toList();
|
||||
List<String> inner = node.getInnerClasses().stream().map(ClassNode::getFullName).toList();
|
||||
List<String> innerRaw = node.getInnerClasses().stream().map(ClassNode::getRawName).toList();
|
||||
Map<String, Object> summary = Map.of(
|
||||
"full_name", cls.getFullName(), "raw_name", cls.getRawName(), "name", cls.getName(),
|
||||
"full_name", cls.getFullName(), "name", cls.getName(),
|
||||
"method_count", methods.size(), "field_count", fields.size(),
|
||||
"inner_class_count", inner.size(), "inner_classes", inner, "inner_class_raw_names", innerRaw,
|
||||
"methods", methods.stream().map(MethodSnapshot::summary).toList(), "fields", List.copyOf(fields));
|
||||
// DEX static values are opt-in, so method and reference lookups keep the member-only reply.
|
||||
Map<String, Object> summaryWithValues = summary;
|
||||
if (anyValue) {
|
||||
Map<String, Object> copy = new LinkedHashMap<>(summary);
|
||||
copy.put("fields", List.copyOf(valuedFields));
|
||||
summaryWithValues = Collections.unmodifiableMap(copy);
|
||||
}
|
||||
ClassSnapshot snapshot = new ClassSnapshot(cls, methods, summary, summaryWithValues);
|
||||
index.computeIfAbsent(cls.getFullName(), ignored -> new ArrayList<>()).add(snapshot);
|
||||
// Keep the binary identity supplied by JADX; '$' can also be a literal
|
||||
// name character, so replacing separators would invent lookup aliases.
|
||||
if (!cls.getRawName().equals(cls.getFullName()))
|
||||
index.computeIfAbsent(cls.getRawName(), ignored -> new ArrayList<>()).add(snapshot);
|
||||
"inner_class_count", inner.size(), "inner_classes", inner,
|
||||
"methods", methods.stream().map(MethodSnapshot::summary).toList(), "fields", fields);
|
||||
index.computeIfAbsent(cls.getFullName(), ignored -> new ArrayList<>())
|
||||
.add(new ClassSnapshot(cls, methods, summary));
|
||||
}
|
||||
}
|
||||
classes = Collections.unmodifiableMap(index);
|
||||
indexedSession = current;
|
||||
}
|
||||
}
|
||||
|
||||
private ClassSnapshot selected(JsonObject args) {
|
||||
indexClasses();
|
||||
String name = string(args, "class_name");
|
||||
List<ClassSnapshot> candidates = classes.get(name);
|
||||
if (candidates == null) throw new InputFailure("class_name", "Exact class not found: " + name
|
||||
@@ -568,28 +209,6 @@ class ReaJadxBridge {
|
||||
return candidates.get(0);
|
||||
}
|
||||
|
||||
/** A static final field's class-definition static value; a static initializer can still assign it. */
|
||||
private static Map<String, Object> dexStaticValue(FieldNode field) {
|
||||
if (!field.getAccessFlags().isStatic() || !field.getAccessFlags().isFinal()) return null;
|
||||
EncodedValue encoded = field.get(JadxAttrType.CONSTANT_VALUE);
|
||||
if (encoded == null) return null;
|
||||
Object value = encoded.getValue();
|
||||
// JADX also attaches the shared null to static finals without a value, so null is never reported.
|
||||
// Wide and floating values keep their Java text instead of a lossy JSON number.
|
||||
return switch (encoded.getType()) {
|
||||
case ENCODED_BOOLEAN -> Map.of("type", "boolean", "value", value);
|
||||
case ENCODED_BYTE -> Map.of("type", "byte", "value", value);
|
||||
case ENCODED_SHORT -> Map.of("type", "short", "value", value);
|
||||
case ENCODED_CHAR -> Map.of("type", "char", "value", (int) (Character) value);
|
||||
case ENCODED_INT -> Map.of("type", "int", "value", value);
|
||||
case ENCODED_LONG -> Map.of("type", "long", "value", value.toString());
|
||||
case ENCODED_FLOAT -> Map.of("type", "float", "value", value.toString());
|
||||
case ENCODED_DOUBLE -> Map.of("type", "double", "value", value.toString());
|
||||
case ENCODED_STRING -> Map.of("type", "string", "value", value);
|
||||
default -> null;
|
||||
};
|
||||
}
|
||||
|
||||
private List<MethodSnapshot> methods(ClassSnapshot cls, JsonObject args) {
|
||||
String name = string(args, "method_name");
|
||||
List<MethodSnapshot> result = cls.methods().stream().filter(method -> method.name().equals(name)).toList();
|
||||
@@ -658,18 +277,12 @@ class ReaJadxBridge {
|
||||
// JSON-RPC/CallToolResult envelope and generated numeric request id, and
|
||||
// account for JSON text being escaped again inside TextContent.
|
||||
private static final int MAX_XREF_TEXT_BYTES = 8 * 1024 * 1024 - 1024;
|
||||
// Leave room for the prototype reply, reference reply, retained raw calls,
|
||||
// normalized Evidence and their text/structured MCP copies, not just one frame.
|
||||
private static final int MAX_EXTERNAL_XREF_BYTES = MAX_XREF_TEXT_BYTES / 8;
|
||||
private static int escapedJsonBytes(Object value) {
|
||||
return JSON.toJson(JSON.toJson(value)).getBytes(StandardCharsets.UTF_8).length;
|
||||
}
|
||||
private static final class InvokeBudget {
|
||||
long remaining;
|
||||
InvokeBudget(long remaining) { this.remaining = Math.max(0, remaining); }
|
||||
boolean fitsText(long characters) {
|
||||
return remaining >= 256 && characters <= (remaining - 256) / 12;
|
||||
}
|
||||
boolean admit(Map<String, Object> site) {
|
||||
// Check a conservative two-layer escape bound before serialization.
|
||||
long bound = 256;
|
||||
@@ -684,55 +297,26 @@ class ReaJadxBridge {
|
||||
}
|
||||
private record DirectInvokes(List<Map<String, Object>> sites, boolean exhaustive) {}
|
||||
|
||||
@FunctionalInterface private interface InvokeVisitor {
|
||||
void accept(MethodInfo target, int offset, int index, Opcode opcode);
|
||||
}
|
||||
|
||||
/** Decode raw invoke identity once at the shared bridge boundary, without generating source. */
|
||||
private static void visitInvokes(MethodNode node, InvokeVisitor visitor) {
|
||||
checkNativeInterrupted();
|
||||
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
|
||||
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
|
||||
MethodNode node = source.getMethodNode();
|
||||
ICodeReader code = node.getCodeReader();
|
||||
if (code == null) return;
|
||||
if (code == null) return null;
|
||||
List<Map<String, Object>> sites = new ArrayList<>();
|
||||
boolean[] exhaustive = {true};
|
||||
code.visitInstructions(insn -> {
|
||||
checkNativeInterrupted();
|
||||
insn.decode();
|
||||
if (insn.getIndexType() != InsnIndexType.METHOD_REF) return;
|
||||
Opcode opcode = insn.getOpcode();
|
||||
if (!opcode.name().startsWith("INVOKE_")) return;
|
||||
IMethodRef ref = insn.getIndexAsMethod();
|
||||
ref.load();
|
||||
visitor.accept(MethodInfo.fromRef(node.root(), ref), insn.getOffset(), insn.getIndex(), opcode);
|
||||
});
|
||||
}
|
||||
|
||||
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
|
||||
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
|
||||
return directInvokes(source.getMethodNode(), target, budget);
|
||||
}
|
||||
|
||||
private static void visitDirectInvokes(MethodNode node, MethodInfo target, InvokeVisitor visitor) {
|
||||
visitInvokes(node, (referenced, offset, index, opcode) -> {
|
||||
if (referenced.getRawFullId().equals(target.getRawFullId()))
|
||||
visitor.accept(referenced, offset, index, opcode);
|
||||
});
|
||||
}
|
||||
|
||||
private static boolean referencesMethod(MethodNode node, MethodInfo target) {
|
||||
boolean[] found = {false};
|
||||
visitDirectInvokes(node, target, (referenced, offset, index, opcode) -> found[0] = true);
|
||||
return found[0];
|
||||
}
|
||||
|
||||
private static DirectInvokes directInvokes(MethodNode node, MethodInfo target, InvokeBudget budget) {
|
||||
if (node.getCodeReader() == null) return null;
|
||||
List<Map<String, Object>> sites = new ArrayList<>();
|
||||
boolean[] exhaustive = {true};
|
||||
visitDirectInvokes(node, target, (referenced, offset, index, opcode) -> {
|
||||
if (!MethodInfo.fromRef(node.root(), ref).getRawFullId().equals(target.getRawFullId())) return;
|
||||
Map<String, Object> site = Map.of(
|
||||
"dex_entry", node.getInputFileName(),
|
||||
"source_raw_method_id", node.getMethodInfo().getRawFullId(),
|
||||
"dex_code_unit_offset", offset,
|
||||
"dex_method_index", index,
|
||||
"dex_code_unit_offset", insn.getOffset(),
|
||||
"dex_method_index", insn.getIndex(),
|
||||
"opcode", opcode.name());
|
||||
if (budget.admit(site)) sites.add(site);
|
||||
else exhaustive[0] = false;
|
||||
@@ -740,143 +324,6 @@ class ReaJadxBridge {
|
||||
return new DirectInvokes(sites, exhaustive[0]);
|
||||
}
|
||||
|
||||
private List<ClassSnapshot> sourceClasses() {
|
||||
indexClasses();
|
||||
Set<ClassSnapshot> seen = Collections.newSetFromMap(new IdentityHashMap<>());
|
||||
List<ClassSnapshot> result = new ArrayList<>();
|
||||
for (List<ClassSnapshot> group : classes.values())
|
||||
for (ClassSnapshot source : group) if (seen.add(source)) result.add(source);
|
||||
result.sort(Comparator.comparing(source -> source.node().getRawName()));
|
||||
return result;
|
||||
}
|
||||
|
||||
private static long displayTypeCharacters(ArgType type) {
|
||||
if (type.isArray())
|
||||
return 2L * type.getArrayDimension() + displayTypeCharacters(type.getArrayRootElement());
|
||||
if (type.isPrimitive()) return type.getPrimitiveType().getLongName().length();
|
||||
if (type.isObject() && !type.isGeneric() && !type.isGenericType()) return type.getObject().length();
|
||||
throw new IllegalArgumentException("External prototype contains a non-DEX type");
|
||||
}
|
||||
|
||||
private static Map<String, Object> externalMethodSummary(MethodInfo info, InvokeBudget budget) {
|
||||
String name = info.getName();
|
||||
long signatureCharacters = name.length() + 4L + displayTypeCharacters(info.getReturnType())
|
||||
+ 2L * Math.max(0, info.getArgsCount() - 1);
|
||||
for (ArgType type : info.getArgumentsTypes()) signatureCharacters += displayTypeCharacters(type);
|
||||
long descriptorCharacters = info.getShortId().length() - (long) name.length();
|
||||
// Use parsed type facts and the existing short-id length before copying/joining strings.
|
||||
if (!budget.fitsText(2L * name.length() + descriptorCharacters + signatureCharacters))
|
||||
throw new ReferenceLimit("method_inventory");
|
||||
String descriptor = info.getShortId().substring(name.length());
|
||||
String signature = name + "(" + info.getArgumentsTypes().stream()
|
||||
.map(Object::toString).collect(Collectors.joining(", "))
|
||||
+ "): " + info.getReturnType();
|
||||
Map<String, Object> summary = Map.of("name", name, "raw_name", name, "signature", signature,
|
||||
"dex_descriptor", descriptor, "is_constructor", info.isConstructor(), "def_pos", 0);
|
||||
if (!budget.admit(summary)) throw new ReferenceLimit("method_inventory");
|
||||
return summary;
|
||||
}
|
||||
|
||||
/** Only directly referenced prototypes, not invented definitions or an external classpath. */
|
||||
private List<ExternalMethod> externalMethods(JsonObject args) {
|
||||
String className = string(args, "class_name");
|
||||
String methodName = string(args, "method_name");
|
||||
JadxSession current = session();
|
||||
if (externalInventory != null && externalInventory.session() == current
|
||||
&& externalInventory.className().equals(className) && externalInventory.methodName().equals(methodName))
|
||||
return externalInventory.methods();
|
||||
// Retain only the latest bounded selection, never results from a replaced APK/session.
|
||||
externalInventory = null;
|
||||
SortedMap<String, ExternalMethod> methods = new TreeMap<>();
|
||||
boolean[] observedClass = {false};
|
||||
InvokeBudget budget = new InvokeBudget(MAX_EXTERNAL_XREF_BYTES
|
||||
- 4096L - 12L * (className.length() + (long) methodName.length()));
|
||||
for (ClassSnapshot source : sourceClasses()) {
|
||||
for (MethodSnapshot selected : source.methods()) {
|
||||
visitInvokes(method(source, selected).getMethodNode(), (info, offset, index, opcode) -> {
|
||||
if (!info.getDeclClass().getRawName().equals(className)) return;
|
||||
observedClass[0] = true;
|
||||
if (!info.getName().equals(methodName) || methods.containsKey(info.getShortId())) return;
|
||||
methods.put(info.getShortId(), new ExternalMethod(info, externalMethodSummary(info, budget)));
|
||||
});
|
||||
}
|
||||
}
|
||||
if (!observedClass[0]) throw new InputFailure("class_name",
|
||||
"No APK definition or direct DEX invoke references the exact class: " + className);
|
||||
if (methods.isEmpty()) throw new InputFailure("method_name",
|
||||
"No direct DEX invoke references " + className + "." + methodName);
|
||||
List<ExternalMethod> result = List.copyOf(methods.values());
|
||||
externalInventory = new ExternalInventory(current, className, methodName, result);
|
||||
return result;
|
||||
}
|
||||
|
||||
private Map<String, Object> externalSummary(JsonObject args) {
|
||||
List<ExternalMethod> methods = externalMethods(args);
|
||||
return Map.of("class_name", string(args, "class_name"), "reference_target", "external",
|
||||
"observed_method_count", methods.size(),
|
||||
"prototypes", methods.stream().map(ExternalMethod::summary).toList());
|
||||
}
|
||||
|
||||
private record ExternalCaller(MethodNode node, Map<String, Object> row) {}
|
||||
|
||||
/** Reserve the complete using-node inventory before optional instruction positions. */
|
||||
private List<ExternalCaller> externalCallers(MethodInfo target, InvokeBudget budget) {
|
||||
List<ExternalCaller> callers = new ArrayList<>();
|
||||
for (ClassSnapshot source : sourceClasses()) {
|
||||
String className = String.valueOf(source.summary().get("full_name"));
|
||||
Map<String, Integer> overloads = new HashMap<>();
|
||||
for (MethodSnapshot candidate : source.methods()) {
|
||||
int overload = overloads.merge(candidate.name(), 1, Integer::sum) - 1;
|
||||
MethodNode node = method(source, candidate).getMethodNode();
|
||||
if (!referencesMethod(node, target)) continue;
|
||||
if (classes.get(className).size() != 1)
|
||||
throw new IllegalArgumentException("External caller has an ambiguous APK class identity: " + className);
|
||||
Map<String, Object> row = new LinkedHashMap<>(Map.of(
|
||||
"kind", "method", "name", candidate.name(), "full_name", className + "." + candidate.name(),
|
||||
"containing_class", className, "top_class", source.node().getTopParentClass().getFullName(),
|
||||
"def_pos", 0, "source_selector", Map.of("class_name", className,
|
||||
"method_name", candidate.name(), "overload_index", overload),
|
||||
"direct_invokes", List.of(), "direct_invokes_exhaustive", false));
|
||||
if (!budget.admit(row)) throw new ReferenceLimit("caller_inventory");
|
||||
callers.add(new ExternalCaller(node, row));
|
||||
}
|
||||
}
|
||||
return List.copyOf(callers);
|
||||
}
|
||||
|
||||
private Map<String, Object> externalReferences(JsonObject args) {
|
||||
List<ExternalMethod> methods = externalMethods(args);
|
||||
if (methods.size() != 1 && !args.has("overload_index"))
|
||||
throw new InputFailure("overload_index", "Select one of " + methods.size() + " observed external prototypes");
|
||||
int index = integer(args, "overload_index", 0);
|
||||
if (index >= methods.size()) throw new InputFailure("overload_index", "External prototype index out of range: " + index);
|
||||
ExternalMethod method = methods.get(index);
|
||||
MethodInfo target = method.info();
|
||||
Map<String, Object> selected = method.summary();
|
||||
InvokeBudget budget = new InvokeBudget(MAX_EXTERNAL_XREF_BYTES
|
||||
- 4096L - escapedJsonBytes(selected));
|
||||
List<ExternalCaller> callers = externalCallers(target, budget);
|
||||
List<Map<String, Object>> items = new ArrayList<>();
|
||||
for (ExternalCaller caller : callers) {
|
||||
DirectInvokes invokes = directInvokes(caller.node(), target, budget);
|
||||
caller.row().put("direct_invokes", invokes.sites());
|
||||
caller.row().put("direct_invokes_exhaustive", invokes.exhaustive());
|
||||
items.add(caller.row());
|
||||
}
|
||||
Map<String, Object> result = new LinkedHashMap<>();
|
||||
result.put("target", string(args, "class_name") + "." + target.getName());
|
||||
result.put("reference_target", "external");
|
||||
result.put("reference_scope", "direct_dex_invokes");
|
||||
result.put("dex_descriptor", selected.get("dex_descriptor"));
|
||||
result.put("reported_signature", selected.get("signature"));
|
||||
result.put("overload_index", index);
|
||||
result.put("total", items.size());
|
||||
result.put("count", items.size());
|
||||
result.put("resolve_line", false);
|
||||
result.put("items", items);
|
||||
return result;
|
||||
}
|
||||
|
||||
private CallToolResult handle(String name, kotlinx.serialization.json.JsonObject input) {
|
||||
JsonObject args = JsonParser.parseString(input.toString()).getAsJsonObject();
|
||||
if (name.equals("rea_jvm_status")) return json(Map.of(
|
||||
@@ -885,12 +332,7 @@ class ReaJadxBridge {
|
||||
"available_processors", Runtime.getRuntime().availableProcessors(),
|
||||
"java_version", System.getProperty("java.version"),
|
||||
"metadata_scope", "parsed_members"));
|
||||
if (name.equals("rea_native_declarations"))
|
||||
return json(nativeDeclarations(Path.of(string(args, "path"))));
|
||||
JadxSession current = session();
|
||||
if (name.equals("get_app_info")) return json(manifest(current).summary());
|
||||
if (name.equals("get_android_manifest"))
|
||||
return text(TextUtilKt.truncateToBytes(manifest(current).text(), current.getMaxSourceBytes()), false);
|
||||
if (name.equals("list_classes")) {
|
||||
List<String> names = current.getClassFqns();
|
||||
String query = args.has("query") ? string(args, "query") : "";
|
||||
@@ -913,15 +355,8 @@ class ReaJadxBridge {
|
||||
"scope", "top_level", "offset", offset, "limit", limit,
|
||||
"items", items));
|
||||
}
|
||||
indexClasses();
|
||||
if (flag(args, "reference_target") && !classes.containsKey(string(args, "class_name"))) {
|
||||
if (name.equals("get_class_summary")) return json(externalSummary(args));
|
||||
if (name.equals("get_xrefs_to_method")) return json(externalReferences(args));
|
||||
throw new InputFailure("class_name", "External prototypes are only available to method reference queries");
|
||||
}
|
||||
ClassSnapshot cls = selected(args);
|
||||
if (name.equals("get_class_summary"))
|
||||
return json(flag(args, "dex_static_values") ? cls.summaryWithValues() : cls.summary());
|
||||
if (name.equals("get_class_summary")) return json(cls.summary());
|
||||
if (name.equals("get_method_body")) {
|
||||
List<MethodSnapshot> overloads = methods(cls, args);
|
||||
int index = integer(args, "overload_index", 0);
|
||||
@@ -936,7 +371,7 @@ class ReaJadxBridge {
|
||||
"markers", code.getMarkers(), "body", code.getText()));
|
||||
}
|
||||
List<JavaNode> uses;
|
||||
String target = (String) cls.summary().get("full_name");
|
||||
String target = string(args, "class_name");
|
||||
MethodSnapshot selectedMethod = null;
|
||||
int overloadIndex = 0;
|
||||
if (name.equals("get_xrefs_to_class")) uses = cls.node().getUseIn();
|
||||
@@ -985,20 +420,6 @@ class ReaJadxBridge {
|
||||
return json(result);
|
||||
}
|
||||
|
||||
private static void registerBridgeTool(Server server, ReaJadxBridge bridge, String name,
|
||||
ToolSchema schema, String description) {
|
||||
Tool tool = new Tool(name, schema, description, null, null, null, null, null, null);
|
||||
server.addTool(tool, (connection, request, continuation) -> {
|
||||
try { return bridge.handle(name, request.getArguments()); }
|
||||
catch (InputFailure error) { return text(JSON.toJson(Map.of(
|
||||
"kind", "invalid-input", "field", error.field, "message", error.getMessage())), true); }
|
||||
catch (ReferenceLimit error) { return text(JSON.toJson(Map.of(
|
||||
"kind", "reference-output-limit", "stage", error.stage,
|
||||
"limit_bytes", MAX_EXTERNAL_XREF_BYTES)), true); }
|
||||
catch (IllegalArgumentException error) { return text(error.getMessage(), true); }
|
||||
});
|
||||
}
|
||||
|
||||
public static void main(String[] args) throws Exception {
|
||||
PrintStream protocolOut = System.out;
|
||||
System.setOut(System.err);
|
||||
@@ -1007,9 +428,10 @@ class ReaJadxBridge {
|
||||
SessionHolder holder = new SessionHolder(defaults);
|
||||
ServerCapabilities capabilities = new ServerCapabilities(
|
||||
new ServerCapabilities.Tools(null), null, null, null, null, null, null, Map.of());
|
||||
Server server = new Server(new Implementation("rea-jadx-bridge", "10", null, null, null),
|
||||
Server server = new Server(new Implementation("rea-jadx-bridge", "4", null, null, null),
|
||||
new ServerOptions(capabilities), "", ignored -> Unit.INSTANCE);
|
||||
SessionToolsKt.registerSessionTools(server, holder);
|
||||
ManifestToolsKt.registerManifestTools(server, holder);
|
||||
ClassToolsKt.registerClassTools(server, holder);
|
||||
MethodToolsKt.registerMethodTools(server, holder);
|
||||
XrefToolsKt.registerXrefTools(server, holder);
|
||||
@@ -1018,7 +440,7 @@ class ReaJadxBridge {
|
||||
server.removeTool("load_apk");
|
||||
server.addTool(loadTool, (connection, request, continuation) ->
|
||||
bridge.load(request.getArguments(), defaults, continuation));
|
||||
Set<String> consumed = Set.of("load_apk",
|
||||
Set<String> consumed = Set.of("load_apk", "get_app_info", "get_android_manifest",
|
||||
"list_classes", "get_class_summary", "get_method_body",
|
||||
"get_xrefs_to_class", "get_xrefs_to_method");
|
||||
for (String name : new ArrayList<>(server.getTools().keySet()))
|
||||
@@ -1027,18 +449,6 @@ class ReaJadxBridge {
|
||||
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of()), List.of(), null),
|
||||
"Observed worker JVM and metadata scope", null, null, null, null, null, null);
|
||||
server.addTool(diagnostics, (connection, request, continuation) -> bridge.handle("rea_jvm_status", request.getArguments()));
|
||||
registerBridgeTool(server, bridge, "rea_native_declarations",
|
||||
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of("path",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of("type",
|
||||
kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))))), List.of("path"), null),
|
||||
"Original ACC_NATIVE declarations from the admitted APK's standard root DEX members, without loading or decompiling the APK.");
|
||||
for (String name : List.of("get_app_info", "get_android_manifest")) {
|
||||
String description = name.equals("get_app_info")
|
||||
? "Namespace-aware summary of the exact root AndroidManifest.xml resource."
|
||||
: "Exact decoded root AndroidManifest.xml text, bounded by the session source byte limit.";
|
||||
registerBridgeTool(server, bridge, name,
|
||||
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of()), List.of(), null), description);
|
||||
}
|
||||
for (String name : List.of("list_classes", "get_class_summary",
|
||||
"get_method_body", "get_xrefs_to_class", "get_xrefs_to_method")) {
|
||||
Tool tool = server.getTools().get(name).getTool();
|
||||
@@ -1054,24 +464,20 @@ class ReaJadxBridge {
|
||||
if (name.equals("list_classes")) properties.put("query",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))));
|
||||
if (name.equals("get_class_summary")) properties.put("dex_static_values",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("boolean"))));
|
||||
if (name.equals("get_class_summary") || name.equals("get_xrefs_to_method"))
|
||||
properties.put("reference_target", new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("boolean"))));
|
||||
if (name.equals("get_class_summary")) properties.put("method_name",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))));
|
||||
String description = switch (name) {
|
||||
case "list_classes" -> "Pages of case-sensitive matching top-level class names; offset and next_offset are inventory positions, limit bounds matching rows. Omitted or empty query inventories all names.";
|
||||
case "get_class_summary" -> "Parsed member metadata for one exact class name; no source generation. Set dex_static_values to include encoded static final field values.";
|
||||
case "get_class_summary" -> "Parsed member metadata for one exact class name; no source generation.";
|
||||
case "get_method_body" -> "Source for an exact class and stable same-name overload index; static initializers use smali to preserve folded field initialization.";
|
||||
default -> "Complete static using-node inventory for an exact class or selected method overload; one node may contain multiple invoke instructions, and source lines are unresolved.";
|
||||
};
|
||||
registerBridgeTool(server, bridge, name,
|
||||
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(properties),
|
||||
tool.getInputSchema().getRequired(), null), description);
|
||||
Tool narrowed = new Tool(name, new ToolSchema(null, new kotlinx.serialization.json.JsonObject(properties),
|
||||
tool.getInputSchema().getRequired(), null), description, null, null, null, null, null, null);
|
||||
server.addTool(narrowed, (connection, request, continuation) -> {
|
||||
try { return bridge.handle(name, request.getArguments()); }
|
||||
catch (InputFailure error) { return text(JSON.toJson(Map.of(
|
||||
"kind", "invalid-input", "field", error.field, "message", error.getMessage())), true); }
|
||||
catch (IllegalArgumentException error) { return text(error.getMessage(), true); }
|
||||
});
|
||||
}
|
||||
CountDownLatch done = new CountDownLatch(1);
|
||||
server.onClose(() -> { done.countDown(); return Unit.INSTANCE; });
|
||||
@@ -1097,7 +503,6 @@ class ReaJadxBridge {
|
||||
BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE,
|
||||
(scope, continuation) -> server.close(continuation));
|
||||
} finally {
|
||||
bridge.clearManifest();
|
||||
BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE,
|
||||
(scope, continuation) -> holder.unload(continuation));
|
||||
}
|
||||
|
||||
+120
-389
@@ -40,7 +40,6 @@ import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.TreeMap;
|
||||
import java.util.TreeSet;
|
||||
import java.util.regex.Pattern;
|
||||
import java.util.regex.PatternSyntaxException;
|
||||
|
||||
@@ -179,10 +178,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private List<FunctionEntry> procedureInventory;
|
||||
private List<InventoryItem> stringInventory;
|
||||
private DecompInterface decompiler;
|
||||
/** Per-function native decompiler limit from the session; 0 leaves it unbounded. */
|
||||
private int decompileTimeoutSeconds;
|
||||
// DecompileProcess converts seconds to an int millisecond timer delay.
|
||||
private static final int MAX_DECOMPILE_TIMEOUT_SECONDS = Integer.MAX_VALUE / 1000;
|
||||
private final ReaGhidraExtensions analysisExtensions = new ReaGhidraExtensions();
|
||||
private static AddressSpace sessionDefaultAddressSpace;
|
||||
|
||||
@@ -194,7 +189,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
Path descriptorPath = Path.of(arguments[0]);
|
||||
SessionDescriptor descriptor = readDescriptor(descriptorPath);
|
||||
decompileTimeoutSeconds = descriptor.decompileTimeoutSeconds();
|
||||
// Windows retains a native immutable bearer descriptor lease until
|
||||
// runtime_close; POSIX removes it after this read. Normal cleanup removes
|
||||
// the Windows file, while abrupt owner death can leave private residue.
|
||||
@@ -405,8 +399,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
writeSuccess(writer, request.id, result);
|
||||
return;
|
||||
}
|
||||
decompileTimeoutSeconds = request.decompileTimeoutSeconds() == null
|
||||
? descriptor.decompileTimeoutSeconds() : request.decompileTimeoutSeconds();
|
||||
writeSuccess(writer, request.id, handleRequest(request, descriptor));
|
||||
}
|
||||
catch (RequestFailure failure) {
|
||||
@@ -435,8 +427,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
"Ghidra auto-analysis did not complete for this Program"
|
||||
);
|
||||
}
|
||||
if (descriptor.existingProject && isMutatingOperation(request.method))
|
||||
throw new RequestFailure("method_unavailable", "Existing Ghidra project sessions are read-only");
|
||||
return switch (request.method) {
|
||||
case "read_binary_inventory_summary" -> binaryInventorySummary(request.params);
|
||||
case "inspect_native_load_image" -> inspectNativeLoadImage(request.params);
|
||||
@@ -489,10 +479,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
};
|
||||
}
|
||||
|
||||
private static boolean isMutatingOperation(String method) {
|
||||
return method.equals("annotate_native_function") || method.equals("set_address_name") || method.equals("set_addresses_names");
|
||||
}
|
||||
|
||||
private JsonObject sessionInfo(SessionDescriptor descriptor) throws Exception {
|
||||
boolean timedOut = analysisTimeoutOccurred();
|
||||
JsonObject provider = new JsonObject();
|
||||
@@ -518,16 +504,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.addProperty("run_id", descriptor.runId);
|
||||
result.addProperty("profile_digest", descriptor.profileDigest);
|
||||
result.add("provider", provider);
|
||||
boolean readOnly = descriptor.existingProject || !descriptor.transport.equals("unix-socket");
|
||||
boolean readOnly = !descriptor.transport.equals("unix-socket");
|
||||
result.addProperty("read_only", readOnly);
|
||||
if (descriptor.existingProject) {
|
||||
result.add("analysis_complete", JsonNull.INSTANCE);
|
||||
result.addProperty("existing_project", true);
|
||||
} else result.addProperty("analysis_complete", !timedOut);
|
||||
result.addProperty("analysis_complete", !timedOut);
|
||||
result.addProperty("analysis_timed_out", timedOut);
|
||||
result.add("capabilities", GSON.toJsonTree(
|
||||
java.util.Arrays.stream(CAPABILITIES)
|
||||
.filter(value -> !readOnly || !isMutatingOperation(value))
|
||||
.filter(value -> !readOnly || !(value.equals("annotate_native_function")
|
||||
|| value.equals("set_address_name") || value.equals("set_addresses_names")))
|
||||
.toArray(String[]::new)
|
||||
));
|
||||
result.add("target", target);
|
||||
@@ -549,11 +533,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
String softFloat = currentProgram.getOptions(ghidra.program.model.listing.Program.PROGRAM_INFO)
|
||||
.getString("REA MIPS soft-float storage", null);
|
||||
if (softFloat != null) result.add("mips_soft_float_storage", JsonParser.parseString(softFloat));
|
||||
// Written by ReaGhidraRiscvCallingConvention after analysis; REA requires it
|
||||
// exactly when the committed profile declares a RISC-V calling-convention policy.
|
||||
String riscvCallingConvention = currentProgram.getOptions(ghidra.program.model.listing.Program.PROGRAM_INFO)
|
||||
.getString("REA RISC-V calling convention", null);
|
||||
if (riscvCallingConvention != null) result.add("riscv_calling_convention", JsonParser.parseString(riscvCallingConvention));
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -659,9 +638,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
mapping.addProperty("initialized", block.isInitialized());
|
||||
mapping.addProperty("loaded", block.isLoaded());
|
||||
mapping.addProperty("overlay", block.isOverlay());
|
||||
mapping.addProperty("readable", block.isRead());
|
||||
mapping.addProperty("writable", block.isWrite());
|
||||
mapping.addProperty("executable", block.isExecute());
|
||||
mapping.addProperty("length", source.getLength());
|
||||
FileBytes backing = source.getFileBytes().orElse(null);
|
||||
int sourceIndex = backing == null ? -1 : files.indexOf(backing);
|
||||
@@ -693,7 +669,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
row.addProperty("status", relocation.getStatus().toString());
|
||||
row.addProperty("type", relocation.getType());
|
||||
row.add("symbol", GSON.toJsonTree(relocation.getSymbolName()));
|
||||
row.add("values", GSON.toJsonTree(relocation.getValues() == null ? new long[0] : relocation.getValues()));
|
||||
byte[] original = relocation.getBytes();
|
||||
row.add("original_bytes_hex", original == null ? JsonNull.INSTANCE : GSON.toJsonTree(bytesHex(original)));
|
||||
@@ -929,33 +904,17 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireKeys(params, Set.of("document", "address"));
|
||||
requireDocument(params);
|
||||
Address address = requireAddress(params, "address");
|
||||
String at = canonicalAddress(address);
|
||||
// A well-formed address can still lack one exact source offset. Each
|
||||
// outcome has its own code, so REA reports the reason rather than
|
||||
// schema repair. Initialization, not the loaded flag, decides whether
|
||||
// source bytes can exist: file-backed overlays are not loaded.
|
||||
MemoryBlock block = currentProgram.getMemory().getBlock(address);
|
||||
if (block == null)
|
||||
throw new RequestFailure("address_unmapped",
|
||||
"Address " + at + " is outside every Ghidra memory block");
|
||||
if (!block.isInitialized())
|
||||
throw new RequestFailure("address_uninitialized",
|
||||
"Address " + at + " is in uninitialized Ghidra memory block " + block.getName());
|
||||
if (block == null || !block.isInitialized())
|
||||
throw new RequestFailure("invalid_request", "Address has no initialized file-backed memory: " + canonicalAddress(address));
|
||||
TargetFileOffset mapping = targetFileOffset(block, address);
|
||||
if (mapping.mappings() == 0)
|
||||
throw new RequestFailure("file_mapping_absent",
|
||||
"Address " + at + " in initialized Ghidra memory block " + block.getName() +
|
||||
" has no authoritative mapping to the admitted target file");
|
||||
if (mapping.mappings() > 1)
|
||||
throw new RequestFailure("file_mapping_ambiguous",
|
||||
"Address " + at + " has " + mapping.mappings() +
|
||||
" authoritative mappings to the admitted target file");
|
||||
if (mapping.mappings() != 1)
|
||||
throw new RequestFailure("invalid_request", "Address has " + mapping.mappings() + " authoritative target-file mappings: " + canonicalAddress(address));
|
||||
Long fileOffset = mapping.offset();
|
||||
if (fileOffset == null)
|
||||
throw new RequestFailure("file_offset_unrepresentable",
|
||||
"Address " + at + " maps to an admitted target-file offset beyond the exact JSON integer range");
|
||||
throw new RequestFailure("invalid_request", "Address lacks a safe file offset: " + canonicalAddress(address));
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("address", at);
|
||||
result.addProperty("address", canonicalAddress(address));
|
||||
result.addProperty("file_offset", fileOffset);
|
||||
return result;
|
||||
}
|
||||
@@ -1138,8 +1097,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
Function function = resolveProcedure(requireString(params, "procedure"));
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("name", procedureName(function));
|
||||
Symbol symbol = function.getSymbol();
|
||||
result.add("symbol", symbol == null ? JsonNull.INSTANCE : symbolMetadata(symbol));
|
||||
result.addProperty("entrypoint", canonicalAddress(function.getEntryPoint()));
|
||||
result.addProperty("basicblock_count", basicBlocks(function).size());
|
||||
JsonObject body = functionBody(function);
|
||||
@@ -1336,7 +1293,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
flow.addProperty("computed", false);
|
||||
if (instruction == null) {
|
||||
result.addProperty("status", !currentProgram.getMemory().contains(address) ? "outside-memory" : currentProgram.getListing().getInstructionContaining(address) != null
|
||||
? "not-instruction-boundary" : currentProgram.getListing().getDefinedDataContaining(address) != null ? "data" : "not-in-listing");
|
||||
? "not-instruction-boundary" : currentProgram.getListing().getDefinedDataContaining(address) != null ? "data" : "undecodable");
|
||||
for (String key : List.of("bytes", "length", "mnemonic", "raw_disassembly")) result.add(key, JsonNull.INSTANCE);
|
||||
} else {
|
||||
result.addProperty("status", "decoded");
|
||||
@@ -1408,7 +1365,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.add("incoming_references", incomingReferences);
|
||||
}
|
||||
List<String> limitations = new ArrayList<>(List.of("Operand components are Ghidra Listing decoder tokens; effective memory base/index/displacement roles and per-instruction context mode are unavailable. The mode field is the program language variant."));
|
||||
if (result.get("status").getAsString().equals("not-in-listing")) limitations.add("No listed instruction or defined data covers this in-memory address. REA did not attempt a decode; the memory may be uninitialized or contain valid instructions.");
|
||||
if (flow.get("kind").getAsString().equals("terminal")) limitations.add("No RETURN operation was observed in the provider's instruction p-code; terminal classification remains generic.");
|
||||
result.add("limitations", GSON.toJsonTree(limitations));
|
||||
return result;
|
||||
@@ -1599,8 +1555,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
} catch (RequestFailure failure) {
|
||||
// The listing, reference and CFG facts below do not depend on the
|
||||
// native decompiler. Cancellation still ends the request.
|
||||
if (!failure.code.equals("decompile_failed") &&
|
||||
!failure.code.equals("decompile_timed_out")) throw failure;
|
||||
if (!failure.code.equals("decompile_failed")) throw failure;
|
||||
decompilation = null;
|
||||
decompilerFailure = failure.getMessage().strip();
|
||||
}
|
||||
@@ -1954,62 +1909,41 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return symbol == null ? JsonNull.INSTANCE : GSON.toJsonTree(symbol.getName(true));
|
||||
}
|
||||
|
||||
// Analyst naming: a function entry renames the function; another mapped
|
||||
// address renames its non-dynamic primary label or receives a new global
|
||||
// label. Resolve that destination without editing the program.
|
||||
private AddressNameTarget addressNameTarget(Address address, String name)
|
||||
throws ghidra.util.exception.InvalidInputException {
|
||||
// Analyst naming: a function entry renames the function, other addresses get a primary label.
|
||||
private boolean applyAddressName(Address address, String name) throws Exception {
|
||||
if (!currentProgram.getMemory().contains(address))
|
||||
throw new RequestFailure("invalid_request", "Address is not mapped: " + canonicalAddress(address));
|
||||
validateAnnotationText(name, "name");
|
||||
Symbol edited = editedNameSymbol(address);
|
||||
return edited == null
|
||||
? new AddressNameTarget(address, currentProgram.getGlobalNamespace(), name)
|
||||
: new AddressNameTarget(address, edited.getParentNamespace(), symbolLeafName(edited, name));
|
||||
}
|
||||
|
||||
private Symbol editedNameSymbol(Address address) {
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
if (function != null) return function.getSymbol();
|
||||
Symbol primary = currentProgram.getSymbolTable().getPrimarySymbol(address);
|
||||
return primary == null || primary.isDynamic() ? null : primary;
|
||||
}
|
||||
|
||||
// Every address whose label or function symbol holds the destination.
|
||||
private List<Address> addressNameHolders(AddressNameTarget target) {
|
||||
Set<Address> holders = new TreeSet<>();
|
||||
for (Symbol symbol : currentProgram.getSymbolTable().getLabelOrFunctionSymbols(target.leaf(), target.namespace()))
|
||||
holders.add(symbol.getAddress());
|
||||
return List.copyOf(holders);
|
||||
}
|
||||
|
||||
// Apply one planned destination in its own transaction. Null means the
|
||||
// edit committed with matching primary-symbol readback.
|
||||
private String applyAddressName(AddressNameTarget target) {
|
||||
Address address = target.address();
|
||||
int transaction = currentProgram.startTransaction("REA address name");
|
||||
boolean commit = false;
|
||||
try {
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
Symbol edited = editedNameSymbol(address);
|
||||
SymbolTable table = currentProgram.getSymbolTable();
|
||||
Symbol primary = function == null ? table.getPrimarySymbol(address) : function.getSymbol();
|
||||
Namespace namespace = primary == null
|
||||
? currentProgram.getGlobalNamespace()
|
||||
: primary.getParentNamespace();
|
||||
String expected = primary == null ? name : symbolLeafName(primary, name);
|
||||
for (Symbol symbol : table.getLabelOrFunctionSymbols(expected, namespace)) {
|
||||
if (!address.equals(symbol.getAddress())) return false;
|
||||
}
|
||||
if (function != null) {
|
||||
function.setName(target.leaf(), SourceType.USER_DEFINED);
|
||||
} else if (edited != null) {
|
||||
edited.setName(target.leaf(), SourceType.USER_DEFINED);
|
||||
function.setName(expected, SourceType.USER_DEFINED);
|
||||
} else {
|
||||
Symbol created = currentProgram.getSymbolTable().createLabel(address, target.leaf(), SourceType.USER_DEFINED);
|
||||
created.setPrimary();
|
||||
if (primary != null && !primary.isDynamic()) {
|
||||
primary.setName(expected, SourceType.USER_DEFINED);
|
||||
} else {
|
||||
Symbol created = table.createLabel(address, expected, SourceType.USER_DEFINED);
|
||||
created.setPrimary();
|
||||
}
|
||||
}
|
||||
invalidateAnalysisCaches();
|
||||
Symbol readback = currentProgram.getSymbolTable().getPrimarySymbol(address);
|
||||
commit = readback != null && target.leaf().equals(readback.getName());
|
||||
return commit ? null : "readback_mismatch";
|
||||
commit = readback != null && expected.equals(readback.getName());
|
||||
return commit;
|
||||
}
|
||||
catch (ghidra.util.exception.InvalidInputException exception) {
|
||||
return "invalid_name";
|
||||
}
|
||||
catch (ghidra.util.exception.DuplicateNameException exception) {
|
||||
return "duplicate_name";
|
||||
catch (ghidra.util.exception.InvalidInputException | ghidra.util.exception.DuplicateNameException exception) {
|
||||
return false;
|
||||
}
|
||||
finally {
|
||||
currentProgram.endTransaction(transaction, commit);
|
||||
@@ -2019,17 +1953,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private JsonElement setAddressName(JsonObject params) throws Exception {
|
||||
requireKeys(params, Set.of("document", "address", "name"));
|
||||
requireDocument(params);
|
||||
String key = requireString(params, "address");
|
||||
Address address = requireAddress(params, "address");
|
||||
String name = requireString(params, "name");
|
||||
// A single edit keeps request errors for unmapped addresses and
|
||||
// unrepresentable text; other outcomes share the batch representation.
|
||||
if (!currentProgram.getMemory().contains(address))
|
||||
throw new RequestFailure("invalid_request", "Address is not mapped: " + canonicalAddress(address));
|
||||
validateAnnotationText(name, "name");
|
||||
Map<String, String> requested = new LinkedHashMap<>();
|
||||
requested.put(key, name);
|
||||
return applyAddressNames(requested);
|
||||
return GSON.toJsonTree(applyAddressName(requireAddress(params, "address"), requireString(params, "name")));
|
||||
}
|
||||
|
||||
private JsonElement setAddressesNames(JsonObject params) throws Exception {
|
||||
@@ -2037,129 +1961,34 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireDocument(params);
|
||||
if (!params.has("names") || !params.get("names").isJsonObject())
|
||||
throw new RequestFailure("invalid_request", "names must be an object of address -> name");
|
||||
Map<String, String> requested = new LinkedHashMap<>();
|
||||
for (Map.Entry<String, JsonElement> entry : params.getAsJsonObject("names").entrySet())
|
||||
requested.put(entry.getKey(), entry.getValue().getAsString());
|
||||
return applyAddressNames(requested);
|
||||
}
|
||||
|
||||
// Decide every request from the names held before any edit, so object
|
||||
// member order never selects which address receives a name. Reply with
|
||||
// per-key results and reasons; a contested destination is listed once in
|
||||
// conflicts and failures refer to it by index.
|
||||
private JsonObject applyAddressNames(Map<String, String> requested) {
|
||||
Map<String, Address> addressesByKey = new LinkedHashMap<>();
|
||||
Map<Address, List<AddressNameRequest>> requestsByAddress = new LinkedHashMap<>();
|
||||
for (Map.Entry<String, String> entry : requested.entrySet()) {
|
||||
Address address = null;
|
||||
JsonObject result = new JsonObject();
|
||||
for (Map.Entry<String, JsonElement> entry : params.getAsJsonObject("names").entrySet()) {
|
||||
try {
|
||||
address = parseReaAddress(entry.getKey());
|
||||
Address address = parseReaAddress(entry.getKey());
|
||||
requestsByAddress.computeIfAbsent(address, ignored -> new ArrayList<>())
|
||||
.add(new AddressNameRequest(entry.getKey(), address, entry.getValue()));
|
||||
.add(new AddressNameRequest(entry.getKey(), address, entry.getValue().getAsString()));
|
||||
}
|
||||
catch (RequestFailure failure) {
|
||||
// The key keeps its own invalid_address outcome below.
|
||||
}
|
||||
addressesByKey.put(entry.getKey(), address);
|
||||
}
|
||||
Map<Address, AddressNameFailure> failures = new HashMap<>();
|
||||
Map<AddressNameDestination, List<AddressNameTarget>> destinations = new LinkedHashMap<>();
|
||||
for (Map.Entry<Address, List<AddressNameRequest>> selected : requestsByAddress.entrySet()) {
|
||||
Address address = selected.getKey();
|
||||
String name = selected.getValue().get(0).name;
|
||||
if (selected.getValue().stream().anyMatch(request -> !request.name.equals(name))) {
|
||||
failures.put(address, new AddressNameFailure("conflicting_alias_names", null));
|
||||
continue;
|
||||
}
|
||||
if (!currentProgram.getMemory().contains(address)) {
|
||||
failures.put(address, new AddressNameFailure("unmapped_address", null));
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
AddressNameTarget target = addressNameTarget(address, name);
|
||||
destinations.computeIfAbsent(target.destination(), ignored -> new ArrayList<>()).add(target);
|
||||
}
|
||||
catch (RequestFailure | ghidra.util.exception.InvalidInputException exception) {
|
||||
failures.put(address, new AddressNameFailure("invalid_name", null));
|
||||
result.addProperty(entry.getKey(), false);
|
||||
}
|
||||
}
|
||||
// Reject a destination that several selected addresses request, or
|
||||
// that another address holds, instead of letting order pick a winner.
|
||||
List<AddressNameConflict> conflicts = new ArrayList<>();
|
||||
List<AddressNameTarget> independent = new ArrayList<>();
|
||||
for (List<AddressNameTarget> requesters : destinations.values()) {
|
||||
AddressNameTarget first = requesters.get(0);
|
||||
List<Address> holders = addressNameHolders(first);
|
||||
String reason;
|
||||
if (requesters.size() > 1) {
|
||||
reason = "duplicate_requested_name";
|
||||
} else if (holders.stream().allMatch(first.address()::equals)) {
|
||||
independent.add(first);
|
||||
continue;
|
||||
} else if (holders.stream().anyMatch(holder ->
|
||||
!holder.equals(first.address()) && !requestsByAddress.containsKey(holder))) {
|
||||
reason = "name_held_by_unselected_address";
|
||||
} else {
|
||||
reason = "name_held_by_selected_address";
|
||||
for (List<AddressNameRequest> requests : requestsByAddress.values()) {
|
||||
String name = requests.get(0).name;
|
||||
boolean conflictingAliasNames = requests.stream()
|
||||
.anyMatch(request -> !request.name.equals(name));
|
||||
boolean success = false;
|
||||
if (!conflictingAliasNames) {
|
||||
try {
|
||||
success = applyAddressName(requests.get(0).address, name);
|
||||
}
|
||||
catch (RequestFailure failure) {
|
||||
// Keep per-address failures in the existing batch result shape.
|
||||
}
|
||||
}
|
||||
AddressNameConflict conflict = new AddressNameConflict(
|
||||
first.namespace(),
|
||||
first.leaf(),
|
||||
requesters.stream().map(AddressNameTarget::address).sorted().toList(),
|
||||
holders
|
||||
);
|
||||
conflicts.add(conflict);
|
||||
for (AddressNameTarget target : requesters)
|
||||
failures.put(target.address(), new AddressNameFailure(reason, conflict));
|
||||
for (AddressNameRequest request : requests)
|
||||
result.addProperty(request.inputAddress, success);
|
||||
}
|
||||
// Independent edits have distinct destinations that no other address
|
||||
// holds, so their results do not depend on application order.
|
||||
for (AddressNameTarget target : independent) {
|
||||
String reason = applyAddressName(target);
|
||||
if (reason != null) failures.put(target.address(), new AddressNameFailure(reason, null));
|
||||
}
|
||||
// Requesters belong to exactly one destination, so ordering conflicts
|
||||
// by their first requester gives indexes independent of member order.
|
||||
conflicts.sort(Comparator.comparing((AddressNameConflict conflict) -> conflict.requesters().get(0)));
|
||||
Map<AddressNameConflict, Integer> indexes = new IdentityHashMap<>();
|
||||
JsonArray contested = new JsonArray();
|
||||
for (AddressNameConflict conflict : conflicts) {
|
||||
indexes.put(conflict, contested.size());
|
||||
JsonObject value = new JsonObject();
|
||||
if (conflict.namespace().isGlobal()) value.add("namespace", JsonNull.INSTANCE);
|
||||
else value.addProperty("namespace", conflict.namespace().getName(true));
|
||||
value.addProperty("name", conflict.leaf());
|
||||
value.add("requested_by", canonicalAddresses(conflict.requesters()));
|
||||
value.add("held_by", canonicalAddresses(conflict.holders()));
|
||||
contested.add(value);
|
||||
}
|
||||
JsonObject results = new JsonObject();
|
||||
JsonObject failed = new JsonObject();
|
||||
for (Map.Entry<String, Address> entry : addressesByKey.entrySet()) {
|
||||
Address address = entry.getValue();
|
||||
AddressNameFailure failure = address == null
|
||||
? new AddressNameFailure("invalid_address", null)
|
||||
: failures.get(address);
|
||||
results.addProperty(entry.getKey(), failure == null);
|
||||
if (failure == null) continue;
|
||||
JsonObject value = new JsonObject();
|
||||
if (address == null) value.add("address", JsonNull.INSTANCE);
|
||||
else value.addProperty("address", canonicalAddress(address));
|
||||
value.addProperty("reason", failure.reason());
|
||||
if (failure.conflict() == null) value.add("conflict", JsonNull.INSTANCE);
|
||||
else value.addProperty("conflict", indexes.get(failure.conflict()));
|
||||
failed.add(entry.getKey(), value);
|
||||
}
|
||||
JsonObject result = new JsonObject();
|
||||
result.add("results", results);
|
||||
result.add("failures", failed);
|
||||
result.add("conflicts", contested);
|
||||
return result;
|
||||
}
|
||||
|
||||
private static JsonArray canonicalAddresses(List<Address> addresses) {
|
||||
JsonArray result = new JsonArray();
|
||||
for (Address address : addresses) result.add(canonicalAddress(address));
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -2249,7 +2078,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireKeys(params, Set.of("document", "address"));
|
||||
requireDocument(params);
|
||||
Address query = requireAddress(params, "address");
|
||||
Function function = functionAtOrContaining(query);
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(query);
|
||||
if (function == null && query.isMemoryAddress()) {
|
||||
function = currentProgram.getFunctionManager().getFunctionContaining(query);
|
||||
}
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("query_address", canonicalAddress(query));
|
||||
if (function != null) {
|
||||
@@ -2265,9 +2097,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
|
||||
private JsonArray search(JsonObject params, boolean procedureSearch) throws Exception {
|
||||
Set<String> keys = new HashSet<>(Set.of("pattern", "mode", "case_sensitive", "document"));
|
||||
if (params.has("literal_ascii_candidate")) keys.add("literal_ascii_candidate");
|
||||
requireKeys(params, keys);
|
||||
requireKeys(
|
||||
params,
|
||||
Set.of("pattern", "mode", "case_sensitive", "document")
|
||||
);
|
||||
requireDocument(params);
|
||||
String expression = requireString(params, "pattern");
|
||||
String mode = requireString(params, "mode");
|
||||
@@ -2275,39 +2108,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
throw new RequestFailure("invalid_request", "mode must be literal or regex");
|
||||
}
|
||||
boolean caseSensitive = requireBoolean(params, "case_sensitive");
|
||||
ValueMatcher matcher;
|
||||
if (mode.equals("literal") && !caseSensitive) {
|
||||
if (!params.has("literal_ascii_candidate"))
|
||||
throw new RequestFailure("invalid_request", "Insensitive literal search requires an ASCII candidate");
|
||||
matcher = literalCandidateMatcher(optionalString(params, "literal_ascii_candidate"));
|
||||
} else {
|
||||
if (params.has("literal_ascii_candidate"))
|
||||
throw new RequestFailure("invalid_request", "ASCII candidate applies only to insensitive literals");
|
||||
matcher = mode.equals("literal")
|
||||
? literalMatcher(expression)
|
||||
: regexMatcher(expression, caseSensitive);
|
||||
}
|
||||
List<InventoryItem> inventory = procedureSearch
|
||||
? procedures().stream().map(FunctionEntry::item).toList()
|
||||
: strings();
|
||||
ValueMatcher matcher = mode.equals("literal")
|
||||
? literalMatcher(expression, caseSensitive)
|
||||
: regexMatcher(expression, caseSensitive);
|
||||
JsonArray items = new JsonArray();
|
||||
if (procedureSearch) {
|
||||
// Search the names exact lookup accepts, keeping one row per
|
||||
// procedure under its primary name.
|
||||
for (FunctionEntry entry : procedures()) {
|
||||
monitor.checkCancelled();
|
||||
ProcedureNameMatch match = matchProcedureNames(entry, matcher);
|
||||
if (!match.matched()) {
|
||||
continue;
|
||||
}
|
||||
JsonObject item = resultItem(entry.item, null);
|
||||
if (!match.aliases().isEmpty()) {
|
||||
JsonArray aliases = new JsonArray();
|
||||
for (String alias : match.aliases()) aliases.add(alias);
|
||||
item.add("matched_aliases", aliases);
|
||||
}
|
||||
items.add(item);
|
||||
}
|
||||
return items;
|
||||
}
|
||||
for (InventoryItem item : strings()) {
|
||||
for (InventoryItem item : inventory) {
|
||||
monitor.checkCancelled();
|
||||
if (!matcher.matches(item.value)) {
|
||||
continue;
|
||||
@@ -2317,36 +2125,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return items;
|
||||
}
|
||||
|
||||
/**
|
||||
* The one definition of the names that select a procedure, shared by exact
|
||||
* lookup and search: its primary name and every other symbol at its exact
|
||||
* entry, each in leaf or namespace-qualified spelling. A label inside the
|
||||
* body never names the procedure. Matching secondary symbols are reported
|
||||
* by their qualified names, sorted.
|
||||
*/
|
||||
private ProcedureNameMatch matchProcedureNames(FunctionEntry entry, ValueMatcher matcher) throws Exception {
|
||||
Function function = entry.function;
|
||||
boolean primary = spellingMatches(function.getName(), entry.item.value, matcher);
|
||||
Set<String> aliases = null;
|
||||
for (Symbol symbol : currentProgram.getSymbolTable().getSymbols(function.getEntryPoint())) {
|
||||
monitor.checkCancelled();
|
||||
String qualified = symbol.getName(true);
|
||||
if (!spellingMatches(symbol.getName(), qualified, matcher)) continue;
|
||||
if (qualified.equals(entry.item.value)) {
|
||||
primary = true;
|
||||
continue;
|
||||
}
|
||||
if (aliases == null) aliases = new TreeSet<>();
|
||||
aliases.add(qualified);
|
||||
}
|
||||
return new ProcedureNameMatch(primary, aliases == null ? List.of() : List.copyOf(aliases));
|
||||
}
|
||||
|
||||
/** A leaf is the suffix of its qualified spelling, so a literal found in it is found in both. */
|
||||
private static boolean spellingMatches(String leaf, String qualified, ValueMatcher matcher) {
|
||||
return matcher.matches(qualified) || (!leaf.equals(qualified) && matcher.matches(leaf));
|
||||
}
|
||||
|
||||
private List<InventoryItem> names() throws Exception {
|
||||
if (nameInventory != null) {
|
||||
return nameInventory;
|
||||
@@ -2360,8 +2138,13 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (!address.isMemoryAddress() && !address.isExternalAddress()) {
|
||||
continue;
|
||||
}
|
||||
JsonObject facts = symbolMetadata(symbol);
|
||||
result.add(new InventoryItem(address, symbol.getName(true), facts, null));
|
||||
JsonObject facts = new JsonObject();
|
||||
facts.addProperty("primary", symbol.isPrimary());
|
||||
facts.addProperty("dynamic", symbol.isDynamic());
|
||||
facts.addProperty("external", symbol.isExternal());
|
||||
facts.addProperty("type", normalizedSymbolType(symbol));
|
||||
facts.addProperty("source", normalizedSource(symbol.getSource()));
|
||||
result.add(new InventoryItem(address, symbol.getName(true), facts));
|
||||
}
|
||||
result.sort(INVENTORY_ORDER);
|
||||
nameInventory = List.copyOf(result);
|
||||
@@ -2405,8 +2188,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
InventoryItem item = new InventoryItem(
|
||||
function.getEntryPoint(),
|
||||
symbol == null ? function.getName() : symbol.getName(true),
|
||||
facts,
|
||||
symbol == null ? null : symbolMetadata(symbol)
|
||||
facts
|
||||
);
|
||||
destination.add(new FunctionEntry(function, item));
|
||||
}
|
||||
@@ -2437,7 +2219,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
instance.isMissingNullTerminator() ? "missing" : "present_or_not_required"
|
||||
);
|
||||
facts.addProperty("byte_length", Math.max(0, data.getLength()));
|
||||
result.add(new InventoryItem(data.getAddress(), value, facts, null));
|
||||
result.add(new InventoryItem(data.getAddress(), value, facts));
|
||||
}
|
||||
result.sort(INVENTORY_ORDER);
|
||||
stringInventory = List.copyOf(result);
|
||||
@@ -2449,13 +2231,26 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
// exact symbol name, even when every character is a hexadecimal digit.
|
||||
boolean explicitAddress = isExplicitReaAddress(value);
|
||||
if (!explicitAddress) {
|
||||
// Imported labels (for example Mach-O _main) can share a function
|
||||
// entry without becoming its primary symbol; search accepts the same.
|
||||
ValueMatcher selector = value::equals;
|
||||
List<Function> matches = new ArrayList<>();
|
||||
for (FunctionEntry entry : procedures()) {
|
||||
monitor.checkCancelled();
|
||||
if (matchProcedureNames(entry, selector).matched()) matches.add(entry.function);
|
||||
Function function = entry.function;
|
||||
Symbol symbol = function.getSymbol();
|
||||
String qualified = symbol == null ? function.getName() : symbol.getName(true);
|
||||
boolean matchesName = function.getName().equals(value) || qualified.equals(value);
|
||||
if (!matchesName) {
|
||||
// Imported labels (for example Mach-O _main) can share a
|
||||
// function entry without becoming its primary symbol. Only
|
||||
// exact entries qualify; an interior label is not a selector.
|
||||
for (Symbol entrySymbol : currentProgram.getSymbolTable().getSymbols(function.getEntryPoint())) {
|
||||
monitor.checkCancelled();
|
||||
if (entrySymbol.getName().equals(value) || entrySymbol.getName(true).equals(value)) {
|
||||
matchesName = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (matchesName) matches.add(function);
|
||||
}
|
||||
if (matches.size() > 1)
|
||||
throw new RequestFailure("ambiguous", "Ghidra procedure name is ambiguous: " + value +
|
||||
@@ -2479,7 +2274,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return function;
|
||||
}
|
||||
|
||||
/** Exact entries include externals; containment applies only to memory addresses. */
|
||||
private Function functionAtOrContaining(Address address) {
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
return function == null && address.isMemoryAddress()
|
||||
@@ -3184,7 +2978,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (!currentProgram.getLanguageID().getIdAsString().startsWith("AARCH64:")) {
|
||||
return null;
|
||||
}
|
||||
Function function = functionAtOrContaining(jumpTable.getSwitchAddress());
|
||||
Function function = containingFunction(jumpTable.getSwitchAddress());
|
||||
if (function == null) return null;
|
||||
|
||||
List<Instruction> instructions = new ArrayList<>();
|
||||
@@ -3567,22 +3361,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
DecompileResults results = decompiler.decompileFunction(
|
||||
function,
|
||||
decompileTimeoutSeconds,
|
||||
0,
|
||||
monitor
|
||||
);
|
||||
if (results.isCancelled()) {
|
||||
throw new RequestFailure("decompile_cancelled", "Ghidra decompilation was cancelled");
|
||||
}
|
||||
// DecompInterface disposes the native process at the limit and starts a
|
||||
// fresh one for the next request, so the session stays usable.
|
||||
if (results.isTimedOut()) {
|
||||
throw new RequestFailure(
|
||||
"decompile_timed_out",
|
||||
"Ghidra decompilation of " + canonicalAddress(function.getEntryPoint()) +
|
||||
" exceeded the " + decompileTimeoutSeconds +
|
||||
"-second configured decompilation limit (ghidra.decompile_timeout_seconds; launch seed REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS)"
|
||||
);
|
||||
}
|
||||
if (!results.decompileCompleted()) {
|
||||
throw new RequestFailure(
|
||||
"decompile_failed",
|
||||
@@ -3684,7 +3468,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
|
||||
private boolean unrepresentedCaller(Reference reference, Set<Function> callers) {
|
||||
if (!reference.getReferenceType().isCall()) return false;
|
||||
Function caller = functionAtOrContaining(reference.getFromAddress());
|
||||
Function caller = containingFunction(reference.getFromAddress());
|
||||
return caller == null || !callers.contains(caller);
|
||||
}
|
||||
|
||||
@@ -3831,7 +3615,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
void observe(Reference reference) {
|
||||
if (!reference.getReferenceType().isCall()) return;
|
||||
// Reference identity follows the same containing-function owner as referenceEdge.
|
||||
Function target = functionAtOrContaining(reference.getToAddress());
|
||||
Function target = containingFunction(reference.getToAddress());
|
||||
if (returnedCallees != null && (target == null || !returnedCallees.contains(target)))
|
||||
unrepresented++;
|
||||
if (target == null) unresolved(reference.getFromAddress(),
|
||||
@@ -4001,8 +3785,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("source_address", canonicalAddress(reference.getFromAddress()));
|
||||
result.addProperty("target_address", canonicalAddress(reference.getToAddress()));
|
||||
Function source = functionAtOrContaining(reference.getFromAddress());
|
||||
Function target = functionAtOrContaining(reference.getToAddress());
|
||||
Function source = containingFunction(reference.getFromAddress());
|
||||
Function target = containingFunction(reference.getToAddress());
|
||||
result.add(
|
||||
"source_procedure",
|
||||
source == null ? JsonNull.INSTANCE : procedureIdentity(source)
|
||||
@@ -4037,6 +3821,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return result;
|
||||
}
|
||||
|
||||
private Function containingFunction(Address address) {
|
||||
Function result = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
if (result == null && address.isMemoryAddress()) {
|
||||
result = currentProgram.getFunctionManager().getFunctionContaining(address);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private static String procedureName(Function function) {
|
||||
Symbol symbol = function.getSymbol();
|
||||
return symbol == null ? function.getName() : symbol.getName(true);
|
||||
@@ -4614,9 +4406,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (factsName != null) {
|
||||
result.add(factsName, item.facts.deepCopy());
|
||||
}
|
||||
if (item.symbol != null) {
|
||||
result.add("symbol", item.symbol.deepCopy());
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -4644,8 +4433,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("address", canonicalAddress(function.getEntryPoint()));
|
||||
result.addProperty("name", procedureName(function));
|
||||
Symbol symbol = function.getSymbol();
|
||||
result.add("symbol", symbol == null ? JsonNull.INSTANCE : symbolMetadata(symbol));
|
||||
result.add("classification", functionClassification(function));
|
||||
result.add("body", functionBody(function));
|
||||
return result;
|
||||
@@ -4874,32 +4661,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return value.isEmpty() ? "unknown" : value;
|
||||
}
|
||||
|
||||
private static JsonObject symbolMetadata(Symbol symbol) {
|
||||
JsonObject facts = new JsonObject();
|
||||
facts.addProperty("primary", symbol.isPrimary());
|
||||
facts.addProperty("dynamic", symbol.isDynamic());
|
||||
facts.addProperty("external", symbol.isExternal());
|
||||
facts.addProperty("type", normalizedSymbolType(symbol));
|
||||
facts.addProperty("source", normalizedSource(symbol.getSource()));
|
||||
return facts;
|
||||
}
|
||||
|
||||
private static String normalizedSource(SourceType source) {
|
||||
return source.name().toLowerCase(Locale.ROOT);
|
||||
}
|
||||
|
||||
private static ValueMatcher literalMatcher(String pattern) {
|
||||
return value -> value.contains(pattern);
|
||||
}
|
||||
|
||||
// REA's host owns Unicode simple folding. Keep all non-ASCII source values
|
||||
// for its final predicate, and filter ASCII-only values without Java regexes.
|
||||
private static ValueMatcher literalCandidateMatcher(String asciiPattern) {
|
||||
if (asciiPattern != null && asciiPattern.chars().anyMatch(character -> character > 127))
|
||||
throw new RequestFailure("invalid_request", "ASCII candidate contains a non-ASCII character");
|
||||
private static ValueMatcher literalMatcher(String pattern, boolean caseSensitive) {
|
||||
String needle = caseSensitive ? pattern : pattern.toLowerCase(Locale.ROOT);
|
||||
return value -> {
|
||||
if (value.chars().anyMatch(character -> character > 127)) return true;
|
||||
return asciiPattern != null && value.toLowerCase(Locale.ROOT).contains(asciiPattern);
|
||||
String candidate = caseSensitive ? value : value.toLowerCase(Locale.ROOT);
|
||||
return candidate.contains(needle);
|
||||
};
|
||||
}
|
||||
|
||||
@@ -4931,11 +4701,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
};
|
||||
}
|
||||
|
||||
// The host queue owns session lifetime, including concurrent cancellation.
|
||||
private static RequestFailure regexStackExhausted(String expression, String stage) {
|
||||
return new RequestFailure("regex_stack_exhausted",
|
||||
"Ghidra's Java regex engine exhausted its stack while " + stage +
|
||||
" pattern " + expression + ".");
|
||||
" pattern " + expression + ". Use literal mode or simplify the regex; the session remains usable.");
|
||||
}
|
||||
|
||||
private static SessionDescriptor readDescriptor(Path path) throws IOException {
|
||||
@@ -4946,17 +4715,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
Set<String> keys = new HashSet<>(DESCRIPTOR_KEYS);
|
||||
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("analysis_extensions"))
|
||||
keys.add("analysis_extensions");
|
||||
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("decompile_timeout_seconds"))
|
||||
keys.add("decompile_timeout_seconds");
|
||||
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("existing_project"))
|
||||
keys.add("existing_project");
|
||||
JsonObject object = requireObject(parsed, keys);
|
||||
int decompileTimeout = 0;
|
||||
if (object.has("decompile_timeout_seconds")) {
|
||||
decompileTimeout = requireInteger(object, "decompile_timeout_seconds");
|
||||
if (decompileTimeout < 1 || decompileTimeout > MAX_DECOMPILE_TIMEOUT_SECONDS)
|
||||
throw new IllegalArgumentException("REA session descriptor is invalid");
|
||||
}
|
||||
return new SessionDescriptor(
|
||||
requireString(object, "transport"),
|
||||
requireString(object, "endpoint_path"),
|
||||
@@ -4965,25 +4724,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireSha256(object, "target_sha256"),
|
||||
requireString(object, "provider_version"),
|
||||
requireString(object, "profile_digest"),
|
||||
object.has("analysis_extensions") ? object.getAsJsonArray("analysis_extensions") : new JsonArray(),
|
||||
decompileTimeout,
|
||||
object.has("existing_project") && requireBoolean(object, "existing_project")
|
||||
object.has("analysis_extensions") ? object.getAsJsonArray("analysis_extensions") : new JsonArray()
|
||||
);
|
||||
}
|
||||
|
||||
private static Request parseRequest(String line, String expectedToken) {
|
||||
JsonElement parsed = JsonParser.parseString(line);
|
||||
Set<String> keys = new HashSet<>(REQUEST_KEYS);
|
||||
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("decompile_timeout_seconds"))
|
||||
keys.add("decompile_timeout_seconds");
|
||||
JsonObject object = requireObject(parsed, keys);
|
||||
Integer timeout = null;
|
||||
if (object.has("decompile_timeout_seconds")) {
|
||||
JsonElement value = object.get("decompile_timeout_seconds");
|
||||
timeout = value.isJsonNull() ? 0 : requireInteger(object, "decompile_timeout_seconds");
|
||||
if (!value.isJsonNull() && (timeout < 1 || timeout > MAX_DECOMPILE_TIMEOUT_SECONDS))
|
||||
throw new IllegalArgumentException("Invalid per-request decompile deadline");
|
||||
}
|
||||
JsonObject object = requireObject(JsonParser.parseString(line), REQUEST_KEYS);
|
||||
int id = requireInteger(object, "id");
|
||||
if (id <= 0 || !constantTimeEquals(requireString(object, "token"), expectedToken)) {
|
||||
throw new IllegalArgumentException("Bridge request authentication failed");
|
||||
@@ -4992,7 +4738,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (params == null || !params.isJsonObject()) {
|
||||
throw new IllegalArgumentException("Bridge request parameters are invalid");
|
||||
}
|
||||
return new Request(id, requireString(object, "method"), params.getAsJsonObject(), timeout);
|
||||
return new Request(id, requireString(object, "method"), params.getAsJsonObject());
|
||||
}
|
||||
|
||||
private static JsonObject requireObject(JsonElement element, Set<String> keys) {
|
||||
@@ -5161,22 +4907,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
.thenComparingInt(Reference::getOperandIndex)
|
||||
.thenComparing(Reference::isPrimary);
|
||||
|
||||
private record InventoryItem(Address address, String value, JsonObject facts, JsonObject symbol) {}
|
||||
private record InventoryItem(Address address, String value, JsonObject facts) {}
|
||||
private record AddressNameRequest(String inputAddress, Address address, String name) {}
|
||||
private record AddressNameDestination(long namespace, String leaf) {}
|
||||
private record AddressNameTarget(Address address, Namespace namespace, String leaf) {
|
||||
AddressNameDestination destination() {
|
||||
return new AddressNameDestination(namespace.getID(), leaf);
|
||||
}
|
||||
}
|
||||
private record AddressNameConflict(Namespace namespace, String leaf, List<Address> requesters, List<Address> holders) {}
|
||||
private record AddressNameFailure(String reason, AddressNameConflict conflict) {}
|
||||
private record FunctionEntry(Function function, InventoryItem item) {}
|
||||
private record ProcedureNameMatch(boolean primary, List<String> aliases) {
|
||||
boolean matched() {
|
||||
return primary || !aliases.isEmpty();
|
||||
}
|
||||
}
|
||||
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage, boolean decompilerFailed) {}
|
||||
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
|
||||
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
|
||||
@@ -5196,11 +4929,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
String targetSha256,
|
||||
String providerVersion,
|
||||
String profileDigest,
|
||||
JsonArray analysisExtensions,
|
||||
int decompileTimeoutSeconds,
|
||||
boolean existingProject
|
||||
JsonArray analysisExtensions
|
||||
) {}
|
||||
private record Request(int id, String method, JsonObject params, Integer decompileTimeoutSeconds) {}
|
||||
private record Request(int id, String method, JsonObject params) {}
|
||||
|
||||
private static final class RequestFailure extends RuntimeException {
|
||||
private final String code;
|
||||
|
||||
@@ -1,193 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.database.mem.FileBytes;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.address.AddressSpace;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.mem.Memory;
|
||||
import ghidra.program.model.mem.MemoryBlock;
|
||||
import ghidra.program.model.mem.MemoryBlockSourceInfo;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.LinkOption;
|
||||
import java.nio.file.Path;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* Rebuild a GameCube/Wii DOL memory map from the validated load ranges REA's TypeScript
|
||||
* admission wrote. BinaryLoader maps the whole file once at address 0; this
|
||||
* script replaces that block with text, data, and BSS blocks at their load addresses.
|
||||
*/
|
||||
public final class ReaGhidraPrepareDol extends GhidraScript {
|
||||
// Canonical v2 header, 18 section rows, 12 gap rows and entry, using uint32 hex.
|
||||
private static final int MAX_PLAN_BYTES = 13 + 18 * 49 + 12 * 30 + 17;
|
||||
private static final class Section {
|
||||
final String kind;
|
||||
final int index;
|
||||
final long fileOffset;
|
||||
final long address;
|
||||
final long size;
|
||||
|
||||
Section(String kind, int index, long fileOffset, long address, long size) {
|
||||
this.kind = kind;
|
||||
this.index = index;
|
||||
this.fileOffset = fileOffset;
|
||||
this.address = address;
|
||||
this.size = size;
|
||||
}
|
||||
}
|
||||
|
||||
private record BssGap(long address, long size) {}
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] arguments = getScriptArgs();
|
||||
if (arguments.length != 1) {
|
||||
throw new IllegalArgumentException(
|
||||
"GameCube DOL preparation requires exactly one section table path"
|
||||
);
|
||||
}
|
||||
if (!currentProgram.getExecutableFormat().equals("Raw Binary") ||
|
||||
!currentProgram.getLanguageID().getIdAsString().startsWith("PowerPC:BE:32:")) {
|
||||
throw new IllegalArgumentException(
|
||||
"GameCube DOL preparation requires a BinaryLoader import in big-endian 32-bit PowerPC"
|
||||
);
|
||||
}
|
||||
Path tablePath = Path.of(arguments[0]);
|
||||
if (!Files.isRegularFile(tablePath, LinkOption.NOFOLLOW_LINKS)) {
|
||||
throw new IllegalArgumentException("GameCube DOL section table is not a regular file");
|
||||
}
|
||||
if (Files.size(tablePath) > MAX_PLAN_BYTES) {
|
||||
throw new IllegalArgumentException("GameCube DOL plan exceeds the canonical format's byte bound");
|
||||
}
|
||||
|
||||
List<Section> sections = new ArrayList<>();
|
||||
List<BssGap> bssGaps = new ArrayList<>();
|
||||
int expectedSections = -1;
|
||||
int expectedGaps = -1;
|
||||
long entry = -1;
|
||||
for (String line : Files.readAllLines(tablePath)) {
|
||||
String[] fields = line.trim().split("\\s+");
|
||||
switch (fields[0]) {
|
||||
case "plan" -> {
|
||||
requireFields(fields, 4, line);
|
||||
if (expectedSections >= 0 || !fields[1].equals("2")) {
|
||||
throw new IllegalArgumentException("Duplicate or unsupported GameCube DOL plan header");
|
||||
}
|
||||
expectedSections = Integer.parseInt(fields[2]);
|
||||
expectedGaps = Integer.parseInt(fields[3]);
|
||||
// A DOL has 18 slots, including at most 11 data slots and 12 BSS gaps.
|
||||
if (expectedSections < 1 || expectedSections > 18 || expectedGaps < 0 || expectedGaps > 12) {
|
||||
throw new IllegalArgumentException("GameCube DOL plan counts exceed the format's slot bounds");
|
||||
}
|
||||
}
|
||||
case "section" -> {
|
||||
requireFields(fields, 6, line);
|
||||
if (expectedSections < 0 || sections.size() >= expectedSections) {
|
||||
throw new IllegalArgumentException("GameCube DOL section exceeds its declared count");
|
||||
}
|
||||
sections.add(new Section(
|
||||
fields[1],
|
||||
Integer.parseInt(fields[2]),
|
||||
Long.decode(fields[3]),
|
||||
Long.decode(fields[4]),
|
||||
Long.decode(fields[5])
|
||||
));
|
||||
}
|
||||
case "bss-gap" -> {
|
||||
requireFields(fields, 3, line);
|
||||
if (expectedGaps < 0 || bssGaps.size() >= expectedGaps) {
|
||||
throw new IllegalArgumentException("GameCube DOL BSS gap exceeds its declared count");
|
||||
}
|
||||
long address = Long.decode(fields[1]);
|
||||
long size = Long.decode(fields[2]);
|
||||
if (address < 0 || size <= 0) {
|
||||
throw new IllegalArgumentException("GameCube DOL BSS gap is not a positive range");
|
||||
}
|
||||
bssGaps.add(new BssGap(address, size));
|
||||
}
|
||||
case "entry" -> {
|
||||
requireFields(fields, 2, line);
|
||||
entry = Long.decode(fields[1]);
|
||||
}
|
||||
default -> throw new IllegalArgumentException("Unknown GameCube DOL table line: " + line);
|
||||
}
|
||||
}
|
||||
if (expectedSections != sections.size() || expectedGaps != bssGaps.size() || entry < 0) {
|
||||
throw new IllegalArgumentException("GameCube DOL section table is incomplete");
|
||||
}
|
||||
|
||||
Memory memory = currentProgram.getMemory();
|
||||
List<FileBytes> files = memory.getAllFileBytes();
|
||||
if (files.size() != 1) {
|
||||
throw new IllegalStateException("GameCube DOL preparation requires exactly one imported file image");
|
||||
}
|
||||
FileBytes fileBytes = files.get(0);
|
||||
AddressSpace space = currentProgram.getAddressFactory().getDefaultAddressSpace();
|
||||
|
||||
// Remove the whole-file block before creating section blocks, so no two blocks claim one address.
|
||||
for (MemoryBlock block : memory.getBlocks()) {
|
||||
for (MemoryBlockSourceInfo source : block.getSourceInfos()) {
|
||||
if (source.getFileBytes().isPresent() && source.getFileBytes().get().equals(fileBytes)) {
|
||||
memory.removeBlock(block, monitor);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (Section section : sections) {
|
||||
if (section.fileOffset + section.size > fileBytes.getSize()) {
|
||||
throw new IllegalArgumentException(
|
||||
"GameCube DOL " + section.kind + " " + section.index + " exceeds the imported file"
|
||||
);
|
||||
}
|
||||
boolean text = section.kind.equals("text");
|
||||
if (!text && !section.kind.equals("data")) {
|
||||
throw new IllegalArgumentException("Unknown GameCube DOL section kind: " + section.kind);
|
||||
}
|
||||
MemoryBlock block = memory.createInitializedBlock(
|
||||
"." + section.kind + section.index,
|
||||
space.getAddress(section.address),
|
||||
fileBytes,
|
||||
section.fileOffset,
|
||||
section.size,
|
||||
false
|
||||
);
|
||||
block.setRead(true);
|
||||
block.setWrite(!text);
|
||||
block.setExecute(text);
|
||||
}
|
||||
|
||||
// Apply the domain owner's exact uninitialized ranges; do not reinterpret BSS here.
|
||||
int gapIndex = 0;
|
||||
for (BssGap gap : bssGaps) {
|
||||
MemoryBlock block = memory.createUninitializedBlock(
|
||||
".bss" + gapIndex++,
|
||||
space.getAddress(gap.address()),
|
||||
gap.size(),
|
||||
false
|
||||
);
|
||||
block.setRead(true);
|
||||
block.setWrite(true);
|
||||
block.setExecute(false);
|
||||
}
|
||||
|
||||
Address entryAddress = space.getAddress(entry);
|
||||
MemoryBlock entryBlock = memory.getBlock(entryAddress);
|
||||
if (entryBlock == null || !entryBlock.isExecute()) {
|
||||
throw new IllegalArgumentException("GameCube DOL entry must lie in an executable section");
|
||||
}
|
||||
addEntryPoint(entryAddress);
|
||||
if (!disassemble(entryAddress) || createFunction(entryAddress, "__start") == null) {
|
||||
throw new IllegalStateException("GameCube DOL entry cannot be decoded as a function");
|
||||
}
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setBoolean("REA raw binary prepared", true);
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setBoolean("REA GameCube DOL prepared", true);
|
||||
}
|
||||
|
||||
private static void requireFields(String[] fields, int count, String line) {
|
||||
if (fields.length != count) {
|
||||
throw new IllegalArgumentException("Malformed GameCube DOL table line: " + line);
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@@ -1,283 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.database.SpecExtension;
|
||||
import ghidra.program.model.data.DataType;
|
||||
import ghidra.program.model.data.DoubleDataType;
|
||||
import ghidra.program.model.data.Float16DataType;
|
||||
import ghidra.program.model.data.FloatDataType;
|
||||
import ghidra.program.model.data.FunctionDefinition;
|
||||
import ghidra.program.model.data.IntegerDataType;
|
||||
import ghidra.program.model.data.StructureDataType;
|
||||
import ghidra.program.model.lang.CompilerSpec;
|
||||
import ghidra.program.model.lang.Language;
|
||||
import ghidra.program.model.lang.PrototypeModel;
|
||||
import ghidra.program.model.lang.Register;
|
||||
import ghidra.program.model.listing.Function;
|
||||
import ghidra.program.model.listing.Parameter;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.listing.VariableStorage;
|
||||
import ghidra.program.model.pcode.Varnode;
|
||||
import ghidra.program.model.pcode.XmlEncode;
|
||||
import java.io.StringReader;
|
||||
import java.io.StringWriter;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.regex.Pattern;
|
||||
import javax.xml.parsers.DocumentBuilderFactory;
|
||||
import javax.xml.transform.OutputKeys;
|
||||
import javax.xml.transform.TransformerFactory;
|
||||
import javax.xml.transform.dom.DOMSource;
|
||||
import javax.xml.transform.stream.StreamResult;
|
||||
import org.w3c.dom.Document;
|
||||
import org.w3c.dom.Element;
|
||||
import org.w3c.dom.Node;
|
||||
import org.w3c.dom.NodeList;
|
||||
import org.xml.sax.InputSource;
|
||||
|
||||
/**
|
||||
* Apply the committed RISC-V calling-convention policy after analysis. Ghidra's generic RV32
|
||||
* compiler specification models ilp32d. Each policy derives a program-specific prototype from the
|
||||
* program's own default prototype, as its {@link Policy} row and docs/ghidra-riscv.md state. The
|
||||
* script installs the prototype as a specification extension and assigns it, with dynamic
|
||||
* storage, to every non-thunk function, external function and function definition.
|
||||
*/
|
||||
public final class ReaGhidraRiscvCallingConvention extends GhidraScript {
|
||||
static final String REPORT_KEY = "REA RISC-V calling convention";
|
||||
/** RISC-V floating-point registers by ABI name: ft0-ft11, fs0-fs11 and fa0-fa7. */
|
||||
private static final Pattern FLOAT_REGISTER = Pattern.compile("f[ast]\\d+");
|
||||
|
||||
/**
|
||||
* What a policy changes in the default ilp32d prototype. Every policy also passes values of 9
|
||||
* bytes or more, such as {@code long double}, by reference.
|
||||
*
|
||||
* @param floatEntryBytes the largest value a floating-point register may pass or return; 0
|
||||
* removes those entries and every rule that assigns floating-point storage
|
||||
* @param preservesFloatRegisters whether fs0-fs11 stay preserved across calls
|
||||
*/
|
||||
private record Policy(
|
||||
String id, String prototype, int floatEntryBytes, boolean preservesFloatRegisters) {}
|
||||
|
||||
private static final List<Policy> POLICIES =
|
||||
List.of(new Policy("ilp32-prototype-v1", "__rea_ilp32", 0, false));
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] arguments = getScriptArgs();
|
||||
Policy policy =
|
||||
POLICIES.stream()
|
||||
.filter(candidate -> arguments.length == 1 && candidate.id().equals(arguments[0]))
|
||||
.findFirst()
|
||||
.orElseThrow(
|
||||
() ->
|
||||
new IllegalArgumentException(
|
||||
"REA RISC-V calling convention requires one of the policies "
|
||||
+ POLICIES.stream().map(Policy::id).toList()));
|
||||
Language language = currentProgram.getLanguage();
|
||||
if (!language.getProcessor().toString().equals("RISCV")
|
||||
|| currentProgram.getDefaultPointerSize() != 4)
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.id() + " requires a 32-bit RISC-V program, not "
|
||||
+ language.getLanguageID());
|
||||
CompilerSpec spec = currentProgram.getCompilerSpec();
|
||||
new SpecExtension(currentProgram)
|
||||
.addReplaceCompilerSpecExtension(
|
||||
derive(spec.getDefaultCallingConvention(), spec, language, policy), monitor);
|
||||
PrototypeModel model =
|
||||
currentProgram.getCompilerSpec().getCallingConvention(policy.prototype());
|
||||
if (model == null)
|
||||
throw new IllegalStateException("REA " + policy.prototype() + " was not installed");
|
||||
selfCheck(model, policy, language);
|
||||
int functions = 0;
|
||||
List<Function> callables = new ArrayList<>();
|
||||
currentProgram.getFunctionManager().getFunctions(true).forEach(callables::add);
|
||||
currentProgram.getFunctionManager().getExternalFunctions().forEach(callables::add);
|
||||
for (Function function : callables) {
|
||||
// A thunk's signature is its target's; the target is in this list.
|
||||
if (function.isThunk()) continue;
|
||||
if (function.hasCustomVariableStorage()) function.setCustomVariableStorage(false);
|
||||
function.setCallingConvention(policy.prototype());
|
||||
if (policy.floatEntryBytes() == 0) {
|
||||
List<VariableStorage> storage = new ArrayList<>();
|
||||
storage.add(function.getReturn().getVariableStorage());
|
||||
for (Parameter parameter : function.getParameters())
|
||||
storage.add(parameter.getVariableStorage());
|
||||
// An untyped function has no assigned storage yet; the decompiler applies the
|
||||
// prototype when it recovers one.
|
||||
requireIntegerStorage(
|
||||
storage.toArray(VariableStorage[]::new), function.getName(true), true);
|
||||
}
|
||||
functions += 1;
|
||||
}
|
||||
int definitions = 0;
|
||||
Iterator<DataType> types = currentProgram.getDataTypeManager().getAllDataTypes();
|
||||
while (types.hasNext()) {
|
||||
if (!(types.next() instanceof FunctionDefinition definition)) continue;
|
||||
definition.setCallingConvention(policy.prototype());
|
||||
definitions += 1;
|
||||
}
|
||||
// Written last: any failure above leaves no report, and REA refuses the session.
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setString(
|
||||
REPORT_KEY,
|
||||
"{\"policy\":\"" + policy.id() + "\",\"functions\":" + functions
|
||||
+ ",\"definitions\":" + definitions + "}");
|
||||
println(
|
||||
"REA RISC-V " + policy.prototype() + ": functions=" + functions
|
||||
+ " definitions=" + definitions);
|
||||
}
|
||||
|
||||
/** The default ilp32d prototype changed as the policy states. */
|
||||
private static String derive(
|
||||
PrototypeModel base, CompilerSpec spec, Language language, Policy policy) throws Exception {
|
||||
XmlEncode encoder = new XmlEncode();
|
||||
base.encode(encoder, spec.getPcodeInjectLibrary());
|
||||
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
|
||||
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
|
||||
Document document =
|
||||
factory.newDocumentBuilder().parse(new InputSource(new StringReader(encoder.toString())));
|
||||
Element root = document.getDocumentElement();
|
||||
if (!root.getTagName().equals("prototype"))
|
||||
throw new IllegalStateException("Unexpected default prototype encoding " + root.getTagName());
|
||||
root.setAttribute("name", policy.prototype());
|
||||
List<Element> floatEntries = new ArrayList<>();
|
||||
NodeList entries = root.getElementsByTagName("pentry");
|
||||
for (int index = 0; index < entries.getLength(); index += 1) {
|
||||
Element entry = (Element) entries.item(index);
|
||||
if (entry.getAttribute("storage").equals("float")) floatEntries.add(entry);
|
||||
}
|
||||
if (floatEntries.isEmpty())
|
||||
throw new IllegalStateException("Default prototype has no floating-point entries");
|
||||
if (policy.floatEntryBytes() == 0) {
|
||||
List<Node> removed = new ArrayList<>(floatEntries);
|
||||
NodeList rules = root.getElementsByTagName("rule");
|
||||
for (int index = 0; index < rules.getLength(); index += 1) {
|
||||
Element rule = (Element) rules.item(index);
|
||||
if (usesFloatStorage(rule)) removed.add(rule);
|
||||
}
|
||||
if (removed.size() == floatEntries.size())
|
||||
throw new IllegalStateException("Default prototype has no floating-point rules to remove");
|
||||
for (Node node : removed) node.getParentNode().removeChild(node);
|
||||
} else
|
||||
for (Element entry : floatEntries)
|
||||
entry.setAttribute("maxsize", Integer.toString(policy.floatEntryBytes()));
|
||||
// Values over 2 x XLEN go by reference. Returns need no rule: a value that the return
|
||||
// registers cannot hold already becomes a caller buffer passed in a0.
|
||||
NodeList inputs = root.getElementsByTagName("input");
|
||||
if (inputs.getLength() != 1)
|
||||
throw new IllegalStateException("Default prototype has no single input list");
|
||||
Element input = (Element) inputs.item(0);
|
||||
Element last = null;
|
||||
for (Node child = input.getFirstChild(); child != null; child = child.getNextSibling())
|
||||
if (child instanceof Element element && element.getTagName().equals("rule")) last = element;
|
||||
if (last == null || !joinsAnyValue(last))
|
||||
throw new IllegalStateException("Default prototype's input list does not end with a join");
|
||||
Element byReference = document.createElement("rule");
|
||||
Element datatype = document.createElement("datatype");
|
||||
datatype.setAttribute("name", "any");
|
||||
datatype.setAttribute("minsize", "9");
|
||||
byReference.appendChild(datatype);
|
||||
byReference.appendChild(document.createElement("convert_to_ptr"));
|
||||
input.insertBefore(byReference, last);
|
||||
if (!policy.preservesFloatRegisters()) {
|
||||
NodeList killedLists = root.getElementsByTagName("killedbycall");
|
||||
Element killed = killedLists.getLength() == 0 ? null : (Element) killedLists.item(0);
|
||||
if (killed == null) {
|
||||
killed = document.createElement("killedbycall");
|
||||
root.appendChild(killed);
|
||||
}
|
||||
List<Element> moved = new ArrayList<>();
|
||||
NodeList preservedLists = root.getElementsByTagName("unaffected");
|
||||
for (int list = 0; list < preservedLists.getLength(); list += 1) {
|
||||
NodeList preserved = preservedLists.item(list).getChildNodes();
|
||||
for (int index = 0; index < preserved.getLength(); index += 1) {
|
||||
if (preserved.item(index) instanceof Element varnode
|
||||
&& isFloatRegister(varnode, language))
|
||||
moved.add(varnode);
|
||||
}
|
||||
}
|
||||
for (Element varnode : moved) {
|
||||
varnode.getParentNode().removeChild(varnode);
|
||||
killed.appendChild(varnode);
|
||||
}
|
||||
}
|
||||
StringWriter xml = new StringWriter();
|
||||
var transformer = TransformerFactory.newInstance().newTransformer();
|
||||
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
|
||||
transformer.transform(new DOMSource(document), new StreamResult(xml));
|
||||
return xml.toString();
|
||||
}
|
||||
|
||||
/** The installed prototype places each class of value as its policy states. */
|
||||
private void selfCheck(PrototypeModel model, Policy policy, Language language) {
|
||||
StructureDataType pair = new StructureDataType("rea_float_pair", 0);
|
||||
pair.add(FloatDataType.dataType);
|
||||
pair.add(FloatDataType.dataType);
|
||||
VariableStorage[] storage =
|
||||
model.getStorageLocations(
|
||||
currentProgram,
|
||||
new DataType[] {
|
||||
DoubleDataType.dataType, FloatDataType.dataType, DoubleDataType.dataType,
|
||||
IntegerDataType.dataType, pair, Float16DataType.dataType
|
||||
},
|
||||
false);
|
||||
VariableStorage quad = storage[storage.length - 1];
|
||||
if (quad.size() != currentProgram.getDefaultPointerSize() || quad.getRegister() == null)
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.prototype() + " does not pass a 16-byte value by reference: " + quad);
|
||||
if (policy.floatEntryBytes() == 0)
|
||||
requireIntegerStorage(storage, policy.prototype() + " self-check", false);
|
||||
if (policy.preservesFloatRegisters()) return;
|
||||
for (Varnode preserved : model.getUnaffectedList()) {
|
||||
Register register = language.getRegister(preserved.getAddress(), preserved.getSize());
|
||||
if (register != null && FLOAT_REGISTER.matcher(register.getName()).matches())
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.prototype() + " still preserves " + register.getName());
|
||||
}
|
||||
}
|
||||
|
||||
private static boolean joinsAnyValue(Element rule) {
|
||||
List<Element> children = new ArrayList<>();
|
||||
for (Node child = rule.getFirstChild(); child != null; child = child.getNextSibling())
|
||||
if (child instanceof Element element) children.add(element);
|
||||
return children.size() == 2
|
||||
&& children.get(0).getTagName().equals("datatype")
|
||||
&& children.get(0).getAttribute("name").equals("any")
|
||||
&& children.get(1).getTagName().equals("join");
|
||||
}
|
||||
|
||||
/** Whether a rule's action, at any depth, assigns floating-point storage. */
|
||||
private static boolean usesFloatStorage(Element element) {
|
||||
NodeList children = element.getChildNodes();
|
||||
for (int index = 0; index < children.getLength(); index += 1) {
|
||||
if (!(children.item(index) instanceof Element child)) continue;
|
||||
if (child.getAttribute("storage").equals("float") || usesFloatStorage(child)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private static boolean isFloatRegister(Element varnode, Language language) {
|
||||
if (!varnode.getTagName().equals("varnode")
|
||||
|| !varnode.getAttribute("space").equals("register"))
|
||||
return false;
|
||||
Register register =
|
||||
language.getRegister(
|
||||
language.getAddressFactory().getRegisterSpace()
|
||||
.getAddress(Long.decode(varnode.getAttribute("offset"))),
|
||||
Integer.parseInt(varnode.getAttribute("size")));
|
||||
return register != null && FLOAT_REGISTER.matcher(register.getName()).matches();
|
||||
}
|
||||
|
||||
private static void requireIntegerStorage(
|
||||
VariableStorage[] storage, String owner, boolean allowUnassigned) {
|
||||
for (VariableStorage item : storage) {
|
||||
if (item.isBadStorage() || (!allowUnassigned && item.isUnassignedStorage()))
|
||||
throw new IllegalStateException("REA RISC-V storage unresolved for " + owner);
|
||||
List<Register> registers = item.getRegisters();
|
||||
if (registers == null) continue; // stack-only storage
|
||||
for (Register register : registers) {
|
||||
if (FLOAT_REGISTER.matcher(register.getName()).matches())
|
||||
throw new IllegalStateException(
|
||||
"REA RISC-V storage still uses " + register.getName() + " for " + owner);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+39
-202
@@ -22,9 +22,6 @@ HOPPER_SYMBOL_NAME_UTF16_UNITS = 1024
|
||||
# that does not fit is null, not an older descriptor path.
|
||||
# 536870887 is Node's 536870888-character string ceiling minus the framing LF.
|
||||
# The ceiling is the encoded payload; JSON expansion is measured before it.
|
||||
# Encoded JSON allowance, in bytes, shared by one instruction list, one
|
||||
# native-caller list, and one caller-scan page. This release has no runtime override.
|
||||
CALLER_SCAN_ALLOWANCE_BYTES = 512 * 1024
|
||||
_BRIDGE_MAX_REQUEST_BYTES_SETTING = "REA_BRIDGE_MAX_REQUEST_BYTES"
|
||||
DEFAULT_BRIDGE_MAX_REQUEST_BYTES = 16 * 1024 * 1024
|
||||
MINIMUM_BRIDGE_MAX_REQUEST_BYTES = 7464
|
||||
@@ -235,9 +232,6 @@ EXHAUSTIVE_ANALYSIS_METHODS = frozenset(
|
||||
"procedure_assembly",
|
||||
"procedure_callees",
|
||||
"procedure_callers",
|
||||
"rea_caller_scan_page",
|
||||
"rea_caller_scan_target",
|
||||
"rea_caller_base",
|
||||
"procedure_info",
|
||||
"procedure_pseudo_code",
|
||||
"procedure_references",
|
||||
@@ -330,27 +324,20 @@ def _name_matches(document, address, name):
|
||||
return observed == name or (not name and observed is None)
|
||||
|
||||
|
||||
def _bookmark_api(document, name):
|
||||
method = getattr(document, name, None)
|
||||
if not callable(method):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose %s" % name
|
||||
)
|
||||
return method
|
||||
|
||||
|
||||
def _bookmark_name(document, address):
|
||||
return _bookmark_api(document, "getBookmarkName")(address)
|
||||
getter = getattr(document, "getBookmarkName", None)
|
||||
if not callable(getter):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose getBookmarkName"
|
||||
)
|
||||
return getter(address)
|
||||
|
||||
|
||||
def _address_selection(document, value=None):
|
||||
"""Own the coordinate and whether it was selected by symbol or address."""
|
||||
def _address(document, value=None):
|
||||
"""Keep explicit hexadecimal coordinates distinct from symbol names."""
|
||||
if value is None:
|
||||
return document.getCurrentAddress(), "cursor"
|
||||
return document.getCurrentAddress()
|
||||
_api_text(value, "Address")
|
||||
if value.startswith("file-offset:"):
|
||||
raise InvalidRequestError("Hopper does not support file-offset: selectors; select Ghidra for source-file coordinates or use a Hopper analysis address")
|
||||
source = "address"
|
||||
if value.lower().startswith("0x"):
|
||||
try:
|
||||
address = int(value, 16)
|
||||
@@ -361,8 +348,6 @@ def _address_selection(document, value=None):
|
||||
# Prefer an observed symbol, retaining bare-hex compatibility only
|
||||
# when Hopper has no symbol with the supplied name.
|
||||
address = document.getAddressForName(value)
|
||||
if address not in BAD_ADDRESSES:
|
||||
source = "symbol"
|
||||
if address in BAD_ADDRESSES:
|
||||
try:
|
||||
address = int(value, 16)
|
||||
@@ -370,11 +355,7 @@ def _address_selection(document, value=None):
|
||||
raise InvalidRequestError("Unknown Hopper address or name: %s" % value) from error
|
||||
if not 0 <= address < 0xFFFFFFFFFFFFFFFF:
|
||||
raise InvalidRequestError("Address is outside Hopper's usable unsigned 64-bit range: %s" % value)
|
||||
return address, source
|
||||
|
||||
|
||||
def _address(document, value=None):
|
||||
return _address_selection(document, value)[0]
|
||||
return address
|
||||
|
||||
|
||||
def _segment(document, address):
|
||||
@@ -385,15 +366,10 @@ def _segment(document, address):
|
||||
|
||||
|
||||
def _procedure(document, value=None):
|
||||
address, source = _address_selection(document, value)
|
||||
address = document.getCurrentAddress() if value is None else _address(document, value)
|
||||
result = _segment(document, address).getProcedureAtAddress(address)
|
||||
if result is None:
|
||||
raise InvalidRequestError("No procedure exists at the requested address")
|
||||
if source == "symbol" and address != result.getEntryPoint():
|
||||
raise InvalidRequestError(
|
||||
"Symbol %s at %s is inside procedure %s at %s; select its entry name/address or use resolve_containing_procedure for explicit containment"
|
||||
% (value, _hex(address), _procedure_name(result), _hex(result.getEntryPoint()))
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
@@ -406,7 +382,6 @@ def _procedure_identity(procedure):
|
||||
return {
|
||||
"address": _hex(procedure.getEntryPoint()),
|
||||
"name": _procedure_name(procedure),
|
||||
"symbol": None,
|
||||
"classification": None,
|
||||
"body": {
|
||||
"available": False,
|
||||
@@ -491,30 +466,19 @@ def _instruction_addresses(procedure):
|
||||
return result
|
||||
|
||||
|
||||
def _iter_native_calls(procedure, direction):
|
||||
def _native_calls(procedure, direction):
|
||||
references = procedure.getAllCallees() if direction == "outgoing" else procedure.getAllCallers()
|
||||
kinds = {0: "none", 1: "unknown", 2: "direct", 3: "objective_c"}
|
||||
calls = {}
|
||||
unresolved = []
|
||||
for reference in references:
|
||||
source, target, kind = reference.fromAddress(), reference.toAddress(), reference.type()
|
||||
if source in BAD_ADDRESSES:
|
||||
yield None, None, None
|
||||
continue
|
||||
if target in BAD_ADDRESSES:
|
||||
yield source, None, {"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind}
|
||||
unresolved.append({"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind})
|
||||
continue
|
||||
yield source, target, {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
|
||||
|
||||
|
||||
def _native_calls(procedure, direction):
|
||||
calls = {}
|
||||
unresolved = []
|
||||
for source, target, detail in _iter_native_calls(procedure, direction):
|
||||
if detail is None:
|
||||
continue
|
||||
if target is None:
|
||||
unresolved.append(detail)
|
||||
else:
|
||||
calls[(source, target)] = detail
|
||||
calls[(source, target)] = {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
|
||||
return calls, sorted(unresolved, key=lambda item: int(item["address"], 16))
|
||||
|
||||
|
||||
@@ -564,136 +528,6 @@ def _procedure_map(document):
|
||||
return result
|
||||
|
||||
|
||||
def _caller_target_data(procedure):
|
||||
addresses = []
|
||||
seen = set()
|
||||
size = 0
|
||||
for block in procedure.basicBlockIterator():
|
||||
for address in _block_instruction_addresses(procedure, block):
|
||||
if address in seen:
|
||||
continue
|
||||
encoded = _hex(address)
|
||||
size += len(encoded) + 4
|
||||
if size > CALLER_SCAN_ALLOWANCE_BYTES:
|
||||
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": False}
|
||||
addresses.append(encoded)
|
||||
seen.add(address)
|
||||
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": True}
|
||||
|
||||
|
||||
def _caller_scan_target(document, params):
|
||||
return _caller_target_data(_procedure(document, params.get("procedure")))
|
||||
|
||||
|
||||
def _caller_base(document, params):
|
||||
procedure = _procedure(document, params.get("procedure"))
|
||||
callers = []
|
||||
size = 0
|
||||
complete = True
|
||||
for item in procedure.getAllCallerProcedures():
|
||||
address = _hex(item.getEntryPoint())
|
||||
size += len(address) + 4
|
||||
if size > CALLER_SCAN_ALLOWANCE_BYTES:
|
||||
complete = False
|
||||
break
|
||||
callers.append(address)
|
||||
return {
|
||||
**_caller_target_data(procedure),
|
||||
"callers": sorted(callers, key=lambda value: int(value, 16)),
|
||||
"callers_complete": complete,
|
||||
}
|
||||
|
||||
|
||||
def _caller_scan_page(document, params):
|
||||
"""Observe forward relationships in one bounded, stateless bridge request."""
|
||||
start = params.get("start")
|
||||
if isinstance(start, bool) or not isinstance(start, int) or start < 0:
|
||||
raise InvalidRequestError("Caller scan start must be a non-negative integer")
|
||||
segments = document.getSegmentsList()
|
||||
counts = [segment.getProcedureCount() for segment in segments]
|
||||
total = sum(counts)
|
||||
end = min(start + 64, total)
|
||||
if start > total:
|
||||
raise InvalidRequestError("Caller scan start exceeds the procedure inventory")
|
||||
items = []
|
||||
item_bytes = 0
|
||||
page_limit = CALLER_SCAN_ALLOWANCE_BYTES
|
||||
oversized = False
|
||||
identities = {}
|
||||
containing = {}
|
||||
|
||||
def identity(procedure):
|
||||
entry = procedure.getEntryPoint()
|
||||
if entry not in identities:
|
||||
identities[entry] = _procedure_identity(procedure)
|
||||
return identities[entry]
|
||||
|
||||
def owner(address):
|
||||
if address not in containing:
|
||||
procedure, _ = _containing_procedure(document, address)
|
||||
containing[address] = identity(procedure) if procedure is not None else None
|
||||
return containing[address]
|
||||
|
||||
segment_start = 0
|
||||
for segment, count in zip(segments, counts):
|
||||
begin, stop = max(start - segment_start, 0), min(end - segment_start, count)
|
||||
segment_start += count
|
||||
for index in range(begin, max(begin, stop)):
|
||||
candidate = segment.getProcedureAtIndex(index)
|
||||
item = {"caller": identity(candidate), "callees": [], "references": []}
|
||||
size = len(json.dumps(item, ensure_ascii=True).encode("utf-8")) + 1
|
||||
remaining = page_limit - item_bytes
|
||||
too_large = size > remaining
|
||||
if not too_large:
|
||||
for callee in candidate.getAllCalleeProcedures():
|
||||
address = _hex(callee.getEntryPoint())
|
||||
size += len(address) + 4
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
item["callees"].append(address)
|
||||
if not too_large:
|
||||
callee_entries = set(item["callees"])
|
||||
for source, destination, call in _iter_native_calls(candidate, "outgoing"):
|
||||
# Account for every native record, even when its target cannot
|
||||
# be associated with an observed procedure.
|
||||
size += 32
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
if destination is None:
|
||||
continue
|
||||
source_procedure = item["caller"] if _hex(source) == item["caller"]["address"] else owner(source)
|
||||
target_address = _hex(destination)
|
||||
if target_address in callee_entries:
|
||||
target_entry = target_address
|
||||
else:
|
||||
target_procedure = owner(destination)
|
||||
if target_procedure is None:
|
||||
continue
|
||||
target_entry = target_procedure["address"]
|
||||
edge = {
|
||||
"source_address": _hex(source), "target_address": _hex(destination),
|
||||
"source_procedure": "caller" if source_procedure is not None and source_procedure["address"] == item["caller"]["address"] else source_procedure,
|
||||
"target_procedure": target_entry,
|
||||
"call": call,
|
||||
}
|
||||
size += len(json.dumps(edge, ensure_ascii=True).encode("utf-8")) + 1
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
item["references"].append(edge)
|
||||
if too_large:
|
||||
oversized = not items
|
||||
break
|
||||
items.append(item)
|
||||
item_bytes += size
|
||||
else:
|
||||
continue
|
||||
break
|
||||
return {"start": start, "next": start + len(items), "total": total, "items": items, "oversized": oversized}
|
||||
|
||||
|
||||
def _string_record(segment, address, display):
|
||||
"""Read one native typed object, retaining its shortened display separately."""
|
||||
length = segment.getObjectLength(address)
|
||||
@@ -1034,15 +868,6 @@ def _dispatch(method, params):
|
||||
if method in EXHAUSTIVE_ANALYSIS_METHODS:
|
||||
_api().require_analysis_complete(document, method)
|
||||
|
||||
if method == "rea_caller_scan_target":
|
||||
return _caller_scan_target(document, params)
|
||||
|
||||
if method == "rea_caller_base":
|
||||
return _caller_base(document, params)
|
||||
|
||||
if method == "rea_caller_scan_page":
|
||||
return _caller_scan_page(document, params)
|
||||
|
||||
if method == "analyze_function":
|
||||
return _analyze_function(document, params)
|
||||
if method == "read_function_instructions":
|
||||
@@ -1227,7 +1052,6 @@ def _dispatch(method, params):
|
||||
length = sum(max(0, _basic_block_end(procedure, block) - block.getStartingAddress()) for block in blocks)
|
||||
return {
|
||||
"name": _procedure_name(procedure),
|
||||
"symbol": None,
|
||||
"entrypoint": _hex(procedure.getEntryPoint()),
|
||||
"basicblock_count": procedure.getBasicBlockCount(),
|
||||
"length": length,
|
||||
@@ -1268,24 +1092,39 @@ def _dispatch(method, params):
|
||||
observed = getter(address)
|
||||
return observed == comment or (comment == "" and observed is None)
|
||||
if method == "list_bookmarks":
|
||||
return [{"address": _hex(item), "name": _bookmark_name(document, item)} for item in _bookmark_api(document, "getBookmarks")()]
|
||||
return [{"address": _hex(item), "name": _bookmark_name(document, item)} for item in document.getBookmarks()]
|
||||
if method == "set_bookmark":
|
||||
address = _address(document, params.get("address"))
|
||||
_segment(document, address)
|
||||
name = params.get("name")
|
||||
if name is not None:
|
||||
_api_text(name, "Bookmark name")
|
||||
has_bookmark = _bookmark_api(document, "hasBookmarkAtAddress")
|
||||
has_bookmark = getattr(document, "hasBookmarkAtAddress", None)
|
||||
if not callable(has_bookmark):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose hasBookmarkAtAddress"
|
||||
)
|
||||
exists = has_bookmark(address)
|
||||
if name is None and exists:
|
||||
# Omission means preserve the existing bookmark and its name.
|
||||
return True
|
||||
_bookmark_api(document, "getBookmarkName")
|
||||
if not callable(getattr(document, "getBookmarkName", None)):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose getBookmarkName"
|
||||
)
|
||||
if name == "" and exists:
|
||||
rename = _bookmark_api(document, "renameBookmarkAtAddress")
|
||||
rename = getattr(document, "renameBookmarkAtAddress", None)
|
||||
if not callable(rename):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose renameBookmarkAtAddress"
|
||||
)
|
||||
rename(address, name)
|
||||
else:
|
||||
setter = _bookmark_api(document, "setBookmarkAtAddress")
|
||||
setter = getattr(document, "setBookmarkAtAddress", None)
|
||||
if not callable(setter):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose setBookmarkAtAddress"
|
||||
)
|
||||
setter(address, name)
|
||||
if not has_bookmark(address):
|
||||
return False
|
||||
@@ -1297,12 +1136,10 @@ def _dispatch(method, params):
|
||||
address = _address(document, params.get("address"))
|
||||
# Permit removing a legacy orphan bookmark, but do not report success
|
||||
# for an unmapped address with no bookmark to remove.
|
||||
has_bookmark = _bookmark_api(document, "hasBookmarkAtAddress")
|
||||
remove_bookmark = _bookmark_api(document, "removeBookmarkAtAddress")
|
||||
if not has_bookmark(address):
|
||||
if not document.hasBookmarkAtAddress(address):
|
||||
_segment(document, address)
|
||||
remove_bookmark(address)
|
||||
return not has_bookmark(address)
|
||||
document.removeBookmarkAtAddress(address)
|
||||
return not document.hasBookmarkAtAddress(address)
|
||||
raise InvalidRequestError("Unknown bridge method")
|
||||
|
||||
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
import Foundation
|
||||
import CoreFoundation
|
||||
|
||||
struct AccessibilityValue {
|
||||
let value: Any
|
||||
let gap: String?
|
||||
}
|
||||
|
||||
/** Preserve JSON scalars; report successful reads that JSON cannot represent faithfully. */
|
||||
func captureAccessibilityValue(_ source: Any?) -> AccessibilityValue {
|
||||
guard let source else { return AccessibilityValue(value: NSNull(), gap: nil) }
|
||||
if let text = source as? String { return AccessibilityValue(value: text, gap: nil) }
|
||||
if let number = source as? NSNumber {
|
||||
if CFGetTypeID(number) == CFBooleanGetTypeID() {
|
||||
return AccessibilityValue(value: number.boolValue, gap: nil)
|
||||
}
|
||||
let value = number.doubleValue
|
||||
let encoding = String(cString: number.objCType)
|
||||
let floating = encoding == "f" || encoding == "d"
|
||||
if number is NSDecimalNumber || !value.isFinite || (value == 0 && value.sign == .minus) || (!floating && abs(value) > 9_007_199_254_740_991) {
|
||||
return AccessibilityValue(value: NSNull(), gap: "AXValue number \(number.stringValue) cannot be represented faithfully as a JSON number")
|
||||
}
|
||||
return AccessibilityValue(value: number, gap: nil)
|
||||
}
|
||||
return AccessibilityValue(value: NSNull(), gap: "AXValue type \(type(of: source)) is not a supported JSON scalar")
|
||||
}
|
||||
@@ -43,13 +43,10 @@ func windowBounds(_ element: AXUIElement) -> CGRect? {
|
||||
return CGRect(origin: point, size: size)
|
||||
}
|
||||
func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard let app = NSRunningApplication(processIdentifier: request.pid) else {
|
||||
try fail("pid-not-application", "LaunchServices does not expose an application for the selected PID; this does not establish that the process is absent")
|
||||
guard let app = NSRunningApplication(processIdentifier: request.pid), let url = app.executableURL else {
|
||||
try fail("target-mismatch", "Selected PID does not expose an application executable URL")
|
||||
}
|
||||
guard let url = app.executableURL else {
|
||||
try fail("application-executable-unavailable", "The selected application does not expose an executable URL; select a PID with an identifiable application executable")
|
||||
}
|
||||
guard let canonicalPointer = realpath(url.path, nil) else { try fail("executable-path-unavailable", "Running executable path cannot be resolved") }
|
||||
guard let canonicalPointer = realpath(url.path, nil) else { try fail("target-mismatch", "Running executable path cannot be resolved") }
|
||||
defer { free(canonicalPointer) }
|
||||
let canonicalExecutable = String(cString: canonicalPointer)
|
||||
guard canonicalExecutable == request.executable else {
|
||||
@@ -65,10 +62,7 @@ func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard SHA256.hash(data: executableBytes).map({ String(format: "%02x", $0) }).joined() == request.sha256 else {
|
||||
try fail("target-mismatch", "Running executable bytes differ from the active target digest")
|
||||
}
|
||||
// .optionIncludingWindow omits a selected window while it is off screen, which
|
||||
// is not absence: select from every window, and let capture report what it can.
|
||||
guard let windows = (CGWindowListCopyWindowInfo([.optionAll], kCGNullWindowID) as? [[String: Any]])?
|
||||
.filter({ ($0[kCGWindowNumber as String] as? NSNumber)?.uint32Value == request.window_id }),
|
||||
guard let windows = CGWindowListCopyWindowInfo([.optionIncludingWindow], request.window_id) as? [[String: Any]],
|
||||
windows.count == 1, let window = windows.first,
|
||||
(window[kCGWindowOwnerPID as String] as? NSNumber)?.int32Value == request.pid,
|
||||
(window[kCGWindowLayer as String] as? NSNumber)?.intValue == 0,
|
||||
@@ -123,9 +117,7 @@ func observe(_ request: Request) async throws -> [String: Any] {
|
||||
let childCountResult = childCount(element)
|
||||
var actions: CFArray?
|
||||
AXUIElementCopyActionNames(element, &actions)
|
||||
let observedValue = captureAccessibilityValue(attribute(element, kAXValueAttribute))
|
||||
if let gap = observedValue.gap { gaps.append("\(gap) at path \(path)"); truncated = true }
|
||||
nodes.append(["path": path, "role": text(element, kAXRoleAttribute), "title": text(element, kAXTitleAttribute), "value": observedValue.value, "actions": actions as? [String] ?? [], "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
|
||||
nodes.append(["path": path, "role": text(element, kAXRoleAttribute), "title": text(element, kAXTitleAttribute), "value": text(element, kAXValueAttribute), "actions": actions as? [String] ?? [], "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
|
||||
guard let totalChildren = childCountResult.value else {
|
||||
truncated = true
|
||||
gaps.append("AX child count unavailable at path \(path): AXError \(childCountResult.error ?? -1)")
|
||||
|
||||
@@ -661,15 +661,18 @@ def trace(debugger, config):
|
||||
open(observation_path, "wb")
|
||||
)
|
||||
process = target.Launch(_launch_info(target, config, listener), error)
|
||||
process_created = process.IsValid()
|
||||
if not error.Success() or not process_created:
|
||||
if not error.Success() or not process.IsValid():
|
||||
exit_observed = _retire_failed_launch(
|
||||
process, config, listener, stdout, stderr
|
||||
)
|
||||
return {
|
||||
"status": "launch-error",
|
||||
"error": error.GetCString(),
|
||||
"target_process_created": process_created,
|
||||
**(
|
||||
{"target_process_created": True}
|
||||
if process.IsValid()
|
||||
else {}
|
||||
),
|
||||
**(
|
||||
{"target_process_exit_observed": True}
|
||||
if exit_observed
|
||||
|
||||
@@ -25,73 +25,6 @@ spec.loader.exec_module(tracer)
|
||||
|
||||
|
||||
class NativeTraceBudgetTests(unittest.TestCase):
|
||||
def test_invalid_launch_process_reports_explicit_no_creation(self):
|
||||
class InvalidProcess:
|
||||
def IsValid(self):
|
||||
return False
|
||||
|
||||
def Kill(self):
|
||||
raise AssertionError("an invalid process cannot own target cleanup")
|
||||
|
||||
class LaunchError:
|
||||
def Success(self):
|
||||
return False
|
||||
|
||||
def GetCString(self):
|
||||
return "No such file or directory: fixture/absent"
|
||||
|
||||
class Target:
|
||||
def IsValid(self):
|
||||
return True
|
||||
|
||||
def Launch(self, _launch_info, _error):
|
||||
return InvalidProcess()
|
||||
|
||||
class Debugger:
|
||||
def SetAsync(self, _enabled):
|
||||
pass
|
||||
|
||||
def GetInstanceName(self):
|
||||
return "fixture"
|
||||
|
||||
def CreateTarget(self, *_arguments):
|
||||
return Target()
|
||||
|
||||
class Pipe:
|
||||
def __init__(self, *_arguments):
|
||||
self.closed = False
|
||||
pipes.append(self)
|
||||
|
||||
def start(self):
|
||||
pass
|
||||
|
||||
def close(self, expected_exit=False):
|
||||
self.closed = True
|
||||
|
||||
pipes = []
|
||||
debugger_api = types.SimpleNamespace(SetInternalVariable=lambda *_arguments: None)
|
||||
with patch.object(tracer.lldb, "SBDebugger", debugger_api, create=True), patch.object(
|
||||
tracer.lldb, "SBError", LaunchError, create=True
|
||||
), patch.object(tracer.lldb, "SBListener", lambda _name: object(), create=True), patch.object(
|
||||
tracer, "_hash_file", return_value="expected"
|
||||
), patch.object(tracer, "_file_identity", return_value=(1, 2)), patch.object(
|
||||
tracer, "_module_identity", return_value=("fixture", (1, 2), "fixture-uuid")
|
||||
), patch.object(tracer, "_launch_info", return_value=object()), patch.object(
|
||||
tracer, "_OutputPipe", Pipe
|
||||
):
|
||||
result = tracer.trace(Debugger(), {
|
||||
"executable": "fixture", "architecture": "arm64", "expected_sha256": "expected",
|
||||
"breakpoints": [], "working_directory": "fixture/absent",
|
||||
"stdout_path": "stdout", "stdout_capture_path": "stdout-capture",
|
||||
"stderr_path": "stderr", "stderr_capture_path": "stderr-capture",
|
||||
})
|
||||
self.assertEqual(result["status"], "launch-error")
|
||||
self.assertEqual(result["error"], "No such file or directory: fixture/absent")
|
||||
self.assertIs(result["target_process_created"], False)
|
||||
self.assertNotIn("target_process_exit_observed", result)
|
||||
self.assertEqual(len(pipes), 2)
|
||||
self.assertTrue(all(pipe.closed for pipe in pipes))
|
||||
|
||||
def test_failed_launch_does_not_infer_exit_from_a_failed_kill(self):
|
||||
class Process:
|
||||
def IsValid(self):
|
||||
|
||||
@@ -15,8 +15,6 @@ struct ProcessIdentityObservation: Encodable {
|
||||
let effective_uid: UInt32?
|
||||
let lifecycle: String?
|
||||
let relation: ProcessRelationObservation?
|
||||
/** Display-only kernel accounting name (p_comm, at most MAXCOMLEN bytes). */
|
||||
let name: String?
|
||||
let reason: String?
|
||||
}
|
||||
|
||||
@@ -33,7 +31,6 @@ struct ProcessMetadata {
|
||||
let parentPid: Int32
|
||||
let processGroupId: Int32
|
||||
let pointerSize: Int
|
||||
let name: String
|
||||
}
|
||||
|
||||
struct ProcessSnapshot {
|
||||
@@ -95,8 +92,7 @@ func readProcessSnapshot(
|
||||
lifecycle: process.kp_proc.p_stat == SZOMB ? "zombie" : "non-zombie",
|
||||
parentPid: process.kp_eproc.e_ppid,
|
||||
processGroupId: process.kp_eproc.e_pgid,
|
||||
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4,
|
||||
name: processName(process)
|
||||
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4
|
||||
)
|
||||
)
|
||||
},
|
||||
@@ -107,13 +103,6 @@ func readProcessSnapshot(
|
||||
return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(ENOMEM)")
|
||||
}
|
||||
|
||||
/** Decode the NUL-terminated accounting name; invalid UTF-8 is replaced for display. */
|
||||
private func processName(_ process: kinfo_proc) -> String {
|
||||
withUnsafeBytes(of: process.kp_proc.p_comm) { bytes in
|
||||
String(decoding: bytes.prefix { $0 != 0 }, as: UTF8.self)
|
||||
}
|
||||
}
|
||||
|
||||
func readProcessIdentities(pids: [Int32]) -> [ProcessIdentityObservation] {
|
||||
readProcessIdentities(pids: pids, snapshot: readProcessSnapshot())
|
||||
}
|
||||
@@ -141,7 +130,6 @@ func readProcessIdentities(
|
||||
effective_uid: nil,
|
||||
lifecycle: nil,
|
||||
relation: nil,
|
||||
name: nil,
|
||||
reason: "not-in-snapshot"
|
||||
)
|
||||
}
|
||||
@@ -156,7 +144,6 @@ func readProcessIdentities(
|
||||
parent_pid: metadata.parentPid,
|
||||
process_group_id: metadata.processGroupId
|
||||
),
|
||||
name: metadata.name,
|
||||
reason: nil
|
||||
)
|
||||
}
|
||||
@@ -170,25 +157,12 @@ private func unavailableIdentity(pid: Int32, reason: String) -> ProcessIdentityO
|
||||
effective_uid: nil,
|
||||
lifecycle: nil,
|
||||
relation: nil,
|
||||
name: nil,
|
||||
reason: reason
|
||||
)
|
||||
}
|
||||
|
||||
enum ProcargsMalformedStage: String, CaseIterable {
|
||||
case header
|
||||
case argumentCount = "argument_count"
|
||||
case executablePath = "executable_path"
|
||||
case pointerSize = "pointer_size"
|
||||
case pathPadding = "path_padding"
|
||||
case arguments
|
||||
case terminator
|
||||
case trailingEntries = "trailing_entries"
|
||||
}
|
||||
|
||||
enum RunTokenReadError: Error {
|
||||
case invalidBuffer
|
||||
case invalidBufferAt(ProcargsMalformedStage)
|
||||
case duplicateToken
|
||||
case invalidTokenEncoding
|
||||
case environmentUnavailable
|
||||
@@ -228,22 +202,6 @@ func processRunTokenObservation(
|
||||
parse: Result<ParsedRunToken, RunTokenReadError>,
|
||||
codeSigningFlags: () -> UInt32?
|
||||
) -> RunTokenObservation {
|
||||
// Native inspection restrictions apply even when argv is incomplete or
|
||||
// overwritten, before parsing reaches the environment. Keep this fact at
|
||||
// the native boundary; cleanup still decides whether the process is related.
|
||||
func withheldEnvironment() -> RunTokenObservation? {
|
||||
guard let flags = codeSigningFlags() else { return nil }
|
||||
if (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func unavailableEnvironment(_ reason: String) -> RunTokenObservation {
|
||||
withheldEnvironment() ?? unavailable(pid: pid, reason: reason)
|
||||
}
|
||||
switch parse {
|
||||
case .success(let parsed):
|
||||
if let runId = parsed.runId {
|
||||
@@ -254,25 +212,38 @@ func processRunTokenObservation(
|
||||
if parsed.environmentEntryCount > 0 {
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
}
|
||||
if let withheld = withheldEnvironment() { return withheld }
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
case .failure(let failure):
|
||||
switch failure {
|
||||
case .environmentUnavailable:
|
||||
return unavailableEnvironment("environment_unavailable")
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "environment_unavailable")
|
||||
case .ambiguousEnvironmentBoundary:
|
||||
return unavailableEnvironment("ambiguous_environment_boundary")
|
||||
// These failures already establish that token bytes were available.
|
||||
if let flags = codeSigningFlags(), (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "ambiguous_environment_boundary")
|
||||
case .duplicateToken:
|
||||
return unavailable(pid: pid, reason: "duplicate_run_token")
|
||||
case .invalidTokenEncoding:
|
||||
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
|
||||
case .appleVectorUnavailable:
|
||||
return unavailableEnvironment("apple_vector_unavailable")
|
||||
return unavailable(pid: pid, reason: "apple_vector_unavailable")
|
||||
case .invalidBuffer:
|
||||
return unavailableEnvironment("malformed_procargs")
|
||||
case .invalidBufferAt(let stage):
|
||||
return unavailableEnvironment("malformed_procargs_\(stage.rawValue)")
|
||||
return unavailable(pid: pid, reason: "malformed_procargs")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -331,30 +302,27 @@ func parseProcessEnvironment(
|
||||
in bytes: [UInt8],
|
||||
pointerSize: Int = MemoryLayout<UnsafeRawPointer>.size
|
||||
) throws -> ParsedRunToken {
|
||||
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBufferAt(.header) }
|
||||
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBuffer }
|
||||
let argc = bytes.withUnsafeBytes { $0.loadUnaligned(as: Int32.self).littleEndian }
|
||||
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBufferAt(.argumentCount) }
|
||||
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBuffer }
|
||||
var offset = MemoryLayout<Int32>.size
|
||||
guard let executablePath = readString(in: bytes, offset: &offset) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.executablePath)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBufferAt(.pointerSize) }
|
||||
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBuffer }
|
||||
let pathRecordSize = Array("executable_path=".utf8).count + executablePath.count + 1
|
||||
let padding = (pointerSize - pathRecordSize % pointerSize) % pointerSize
|
||||
guard offset + padding <= bytes.count,
|
||||
bytes[offset..<(offset + padding)].allSatisfy({ $0 == 0 }) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.pathPadding)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
offset += padding
|
||||
let tokenPrefix = Array("REA_PROCESS_RUN_ID=".utf8)
|
||||
var tokenInArgument = false
|
||||
for _ in 0..<argc {
|
||||
guard let argument = readString(in: bytes, offset: &offset, allowEmpty: true) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.arguments)
|
||||
guard readString(in: bytes, offset: &offset, allowEmpty: true) != nil else {
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
tokenInArgument = tokenInArgument || argument.starts(with: tokenPrefix)
|
||||
}
|
||||
guard bytes.last == 0 else { throw RunTokenReadError.invalidBufferAt(.terminator) }
|
||||
guard bytes.last == 0 else { throw RunTokenReadError.invalidBuffer }
|
||||
if offset == bytes.count {
|
||||
throw RunTokenReadError.environmentUnavailable
|
||||
}
|
||||
@@ -372,7 +340,7 @@ func parseProcessEnvironment(
|
||||
continue
|
||||
}
|
||||
guard let entry = readString(in: bytes, offset: &offset) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.trailingEntries)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
trailingEntries.append(entry)
|
||||
}
|
||||
@@ -393,28 +361,22 @@ func parseProcessEnvironment(
|
||||
throw RunTokenReadError.appleVectorUnavailable
|
||||
}
|
||||
if trailingEntries[appleVectorStart...].contains(where: {
|
||||
$0.starts(with: tokenPrefix)
|
||||
$0.starts(with: Array("REA_PROCESS_RUN_ID=".utf8))
|
||||
}) {
|
||||
throw RunTokenReadError.ambiguousEnvironmentBoundary
|
||||
}
|
||||
let environmentEntries = trailingEntries[..<appleVectorStart]
|
||||
var found: String?
|
||||
for entry in environmentEntries {
|
||||
if entry.starts(with: tokenPrefix) {
|
||||
if entry.starts(with: Array("REA_PROCESS_RUN_ID=".utf8)) {
|
||||
guard found == nil else { throw RunTokenReadError.duplicateToken }
|
||||
let value = entry.dropFirst(tokenPrefix.count)
|
||||
let value = entry.dropFirst("REA_PROCESS_RUN_ID=".utf8.count)
|
||||
guard let decoded = String(bytes: value, encoding: .utf8) else {
|
||||
throw RunTokenReadError.invalidTokenEncoding
|
||||
}
|
||||
found = decoded
|
||||
}
|
||||
}
|
||||
// A changed process title can overwrite the original argument boundaries,
|
||||
// causing a real environment entry to be consumed as an argument. A real
|
||||
// argument can have this form too, so it proves neither ownership nor absence.
|
||||
if tokenInArgument {
|
||||
throw RunTokenReadError.ambiguousEnvironmentBoundary
|
||||
}
|
||||
return ParsedRunToken(runId: found, environmentEntryCount: environmentEntries.count)
|
||||
}
|
||||
|
||||
|
||||
@@ -15,11 +15,6 @@ import sys
|
||||
PROFILE = "[email protected];[email protected];[email protected]"
|
||||
PACKAGES = {"pwntools": "4.15.0", "pyelftools": "0.33", "unicorn": "2.1.2"}
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
# (e_machine, ELF class bits, byte order) -> architecture pwntools must also report.
|
||||
PROFILES = {(62, 64, "little"): "amd64", (183, 64, "little"): "aarch64", (3, 32, "little"): "i386",
|
||||
(40, 32, "little"): "arm", (8, 32, "little"): "mips", (8, 32, "big"): "mips"}
|
||||
PROFILE_SCOPE = ("Layout profile requires ELF64 x86-64, ELF64 AArch64 little-endian, ELF32 i386, "
|
||||
"ELF32 ARM little-endian or ELF32 MIPS little- or big-endian objects.")
|
||||
|
||||
|
||||
class LayoutFailure(Exception):
|
||||
@@ -121,16 +116,13 @@ def inspect_elf(path, cache):
|
||||
raise LayoutFailure("format", "Selected object does not contain an ELF header.")
|
||||
if len(content) < 16:
|
||||
raise LayoutFailure("format", "Selected ELF identification is truncated.")
|
||||
if content[4] not in (1, 2) or content[5] not in (1, 2):
|
||||
raise LayoutFailure("unsupported-target", "Selected ELF declares an unknown class or byte order.")
|
||||
bits, header_size = (32, 52) if content[4] == 1 else (64, 64)
|
||||
order = "little" if content[5] == 1 else "big"
|
||||
if len(content) < header_size:
|
||||
raise LayoutFailure("format", f"Selected ELF{bits} header is truncated.")
|
||||
profile = (int.from_bytes(content[18:20], order), bits, order)
|
||||
if profile not in PROFILES:
|
||||
raise LayoutFailure("unsupported-target", PROFILE_SCOPE)
|
||||
if int.from_bytes(content[16:18], order) not in (1, 2, 3):
|
||||
if content[4] != 2 or content[5] != 1:
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires ELF64 little-endian objects.")
|
||||
if len(content) < 64:
|
||||
raise LayoutFailure("format", "Selected ELF64 header is truncated.")
|
||||
if int.from_bytes(content[18:20], "little") != 62:
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF objects.")
|
||||
if int.from_bytes(content[16:18], "little") not in (1, 2, 3):
|
||||
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
|
||||
from pwnlib.context import context
|
||||
with context.local(cache_dir=str(cache), log_level="warning"):
|
||||
@@ -140,11 +132,6 @@ def inspect_elf(path, cache):
|
||||
try:
|
||||
headers = ELFFile(io.BytesIO(content))
|
||||
section_count = headers.num_sections()
|
||||
if section_count:
|
||||
location(headers.header.e_shoff, section_count * headers.header.e_shentsize, len(content))
|
||||
segment_count = headers.num_segments()
|
||||
if segment_count:
|
||||
location(headers.header.e_phoff, segment_count * headers.header.e_phentsize, len(content))
|
||||
if headers.header.e_shstrndx == 0xffff and section_count == 0:
|
||||
raise LayoutFailure("format", "ELF extended section-name index requires a declared section-zero header.")
|
||||
raw_name_index = headers.header.e_shstrndx
|
||||
@@ -160,21 +147,9 @@ def inspect_elf(path, cache):
|
||||
if names_header.sh_type != "SHT_STRTAB":
|
||||
raise LayoutFailure("format", f"ELF section-name table {section_name_index} is not SHT_STRTAB.")
|
||||
location(names_header.sh_offset, names_header.sh_size, len(content))
|
||||
except (ELFError, ValueError, OverflowError, AssertionError, IndexError) as error:
|
||||
except ELFError as error:
|
||||
raise LayoutFailure("format", str(error)) from error
|
||||
class LayoutELF(ELF):
|
||||
def search(self, needle, writable=False, executable=False):
|
||||
# This adapter exposes structural facts, GOT/PLT and mitigation
|
||||
# indicators, not kernel config/version discovery. Upstream's
|
||||
# constructor searches virtual memory for those unrelated
|
||||
# heuristics, materializing NOBITS tails and interpreting
|
||||
# ordinary IKCFG_ST data as a compressed kernel config.
|
||||
return iter(())
|
||||
|
||||
def _populate_kernel_version(self):
|
||||
# Kernel banner decoding is outside this report too.
|
||||
return None
|
||||
|
||||
def _get_section_name(self, header):
|
||||
# SHN_UNDEF declares no names; upstream would otherwise read
|
||||
# strings at the inactive section-zero offset.
|
||||
@@ -193,8 +168,8 @@ def inspect_elf(path, cache):
|
||||
if type is None or section.header.sh_type == type)
|
||||
try:
|
||||
image = LayoutELF(str(path), checksec=False)
|
||||
if (image.arch, image.bits, image.endian) != (PROFILES[profile], bits, order):
|
||||
raise LayoutFailure("unsupported-target", PROFILE_SCOPE)
|
||||
if image.arch != "amd64" or image.bits != 64 or image.endian != "little":
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF64 little-endian objects.")
|
||||
if image.header.e_type not in ("ET_EXEC", "ET_DYN", "ET_REL"):
|
||||
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
|
||||
length = len(content)
|
||||
@@ -216,7 +191,7 @@ def inspect_elf(path, cache):
|
||||
})
|
||||
if h.sh_type in ("SHT_SYMTAB", "SHT_DYNSYM"):
|
||||
if h.sh_entsize < image.structs.Elf_Sym.sizeof():
|
||||
raise LayoutFailure("format", f"ELF symbol entry is smaller than the decoded Elf{bits}_Sym structure.")
|
||||
raise LayoutFailure("format", "ELF symbol entry is smaller than the decoded Elf64_Sym structure.")
|
||||
if h.sh_link >= len(all_sections) or all_sections[h.sh_link].header.sh_type != "SHT_STRTAB":
|
||||
raise LayoutFailure("format", f"ELF symbol table {index} links to invalid string table {h.sh_link}.")
|
||||
strings = all_sections[h.sh_link]
|
||||
@@ -319,7 +294,6 @@ def inspect_elf(path, cache):
|
||||
"Relocatable relocation targets with SHN_UNDEF or inactive SHT_NULL headers retain their reported section index/offset as unknown; they do not establish an associated target section.",
|
||||
"GOT/PLT are derived upstream convenience maps; PLT inference can emulate selected instructions in Unicorn. They do not establish target runtime behavior; aliases may collapse and warnings may indicate incomplete coverage. Their completeness is unknown.",
|
||||
"Mitigations are upstream static heuristics, not runtime protection. ET_DYN does not prove an executable; absent canary symbols do not prove every function unprotected.",
|
||||
"Kernel configuration/version discovery is not performed; these memory-content scans do not supply the reported structural, linkage or mitigation facts.",
|
||||
"This profile inspects ELF EXEC/DYN/REL layout only. It does not analyze recorded cores, resolve loaded libraries, launch the target as a host process or start a debugger.",
|
||||
"Symbol/relocation inventories reflect original section tables. A sectionless image can still report dynamic dependency names; missing tables do not prove absence of dynamic symbols or relocations.",
|
||||
*(["Absent section-name table: section-dependent RELRO/canary heuristics may be incomplete; protection coverage is unknown."] if all_sections and section_name_index == 0 else []),
|
||||
@@ -328,7 +302,7 @@ def inspect_elf(path, cache):
|
||||
}
|
||||
except LayoutFailure:
|
||||
raise
|
||||
except (ELFError, ValueError, OverflowError, AssertionError, IndexError) as error:
|
||||
except (ELFError, ValueError, AssertionError, IndexError) as error:
|
||||
raise LayoutFailure("format", "Selected ELF failed unchanged upstream structural parsing: " + str(error)) from error
|
||||
|
||||
|
||||
|
||||
@@ -88,18 +88,16 @@ def inspect_core(path, cache):
|
||||
segments, notes, padding, threads, signals = [], [], [], [], []
|
||||
for segment_index, segment in enumerate(image.iter_segments()):
|
||||
h = segment.header
|
||||
header_offset = header.e_phoff + segment_index * header.e_phentsize
|
||||
type_id = int.from_bytes(content[header_offset:header_offset + 4], "little")
|
||||
backing = type_id != 0 and h.p_filesz > 0
|
||||
backing = h.p_type != "PT_NULL" and h.p_filesz > 0
|
||||
if backing:
|
||||
location(h.p_offset, h.p_filesz, len(content))
|
||||
if type_id == 1 and h.p_filesz > h.p_memsz:
|
||||
if h.p_type == "PT_LOAD" and h.p_filesz > h.p_memsz:
|
||||
raise CoreFailure("format", "Recorded PT_LOAD file bytes exceed memory size.")
|
||||
segments.append({"index": segment_index, "type": h.p_type, "type_id": type_id,
|
||||
"header_location": location(header_offset, header.e_phentsize, len(content)),
|
||||
segments.append({"index": segment_index, "type": h.p_type,
|
||||
"header_location": location(header.e_phoff + segment_index * header.e_phentsize, header.e_phentsize, len(content)),
|
||||
"offset": hex(h.p_offset), "file_size": hex(h.p_filesz), "memory_size": hex(h.p_memsz),
|
||||
"virtual_address": hex(h.p_vaddr), "physical_address": hex(h.p_paddr), "alignment": hex(h.p_align), "flags": hex(h.p_flags), "file_backing": "file" if backing else "none"})
|
||||
if type_id != 4 or h.p_filesz == 0:
|
||||
if h.p_type != "PT_NOTE" or h.p_filesz == 0:
|
||||
continue
|
||||
position, end = h.p_offset, h.p_offset + h.p_filesz
|
||||
header_size = image.structs.Elf_Nhdr.sizeof()
|
||||
@@ -111,7 +109,6 @@ def inspect_core(path, cache):
|
||||
padding.append({"segment_index": segment_index, "location": location(position, len(raw), end), "bytes_base64": encoded(raw)})
|
||||
break
|
||||
note = struct_parse(image.structs.Elf_Nhdr, image.stream, stream_pos=position)
|
||||
note_type_id = int.from_bytes(content[position + 8:position + 12], "little")
|
||||
owner_start = position + header_size
|
||||
descriptor_start = owner_start + ((note.n_namesz + 3) // 4) * 4
|
||||
note_end = descriptor_start + ((note.n_descsz + 3) // 4) * 4
|
||||
@@ -122,10 +119,7 @@ def inspect_core(path, cache):
|
||||
display = owner.split(b"\0", 1)[0].decode("utf-8", "replace") if owner else None
|
||||
descriptor = content[descriptor_start:descriptor_start + note.n_descsz]
|
||||
note_index = len(notes)
|
||||
# pyelftools selects ET_CORE labels without owner context. Keep
|
||||
# those labels only for exact CORE owners; other kinds are opaque.
|
||||
note_type = note.n_type if owner == b"CORE\0" else note_type_id
|
||||
notes.append({"index": note_index, "segment_index": segment_index, "type": note_type, "type_id": note_type_id,
|
||||
notes.append({"index": note_index, "segment_index": segment_index, "type": note.n_type,
|
||||
"location": location(position, note_end - position, end),
|
||||
"owner_location": location(owner_start, len(owner), end), "owner_bytes_base64": encoded(owner), "owner_display": display,
|
||||
"descriptor_location": location(descriptor_start, len(descriptor), end), "descriptor_bytes_base64": encoded(descriptor)})
|
||||
@@ -133,7 +127,7 @@ def inspect_core(path, cache):
|
||||
# Only the initial Linux CORE ABI is interpreted. Other owners retain exact bytes.
|
||||
if owner != b"CORE\0":
|
||||
continue
|
||||
if note_type_id == 1:
|
||||
if note.n_type == "NT_PRSTATUS":
|
||||
if len(descriptor) < ctypes.sizeof(prstatus_type):
|
||||
raise CoreFailure("format", "Linux " + profile_label + " NT_PRSTATUS descriptor is truncated.")
|
||||
status = prstatus_type.from_buffer_copy(descriptor)
|
||||
@@ -142,7 +136,7 @@ def inspect_core(path, cache):
|
||||
relative = prstatus_type.pr_reg.offset + getattr(registers_type, name).offset
|
||||
registers.append({"name": name, "value": hex(getattr(status.pr_reg, name)), "location": location(descriptor_start + relative, ctypes.sizeof(kind), end)})
|
||||
threads.append({"note_index": note_index, "historical_pid": ctypes.c_int32(status.pr_pid).value, "recorded_current_signal": status.pr_cursig, "registers": registers})
|
||||
elif note_type_id == 0x53494749:
|
||||
elif note.n_type == "NT_SIGINFO":
|
||||
if len(descriptor) < ctypes.sizeof(elf_siginfo_64):
|
||||
raise CoreFailure("format", "Linux " + profile_label + " NT_SIGINFO descriptor is truncated.")
|
||||
signal = elf_siginfo_64.from_buffer_copy(descriptor)
|
||||
|
||||
@@ -33,11 +33,7 @@ export default defineConfig({
|
||||
{ text: "Model artifacts", link: "/model-artifacts" },
|
||||
{ text: "Rust dependencies", link: "/rust-binaries" },
|
||||
{ text: "PE certificates", link: "/pe-certificates" },
|
||||
{ text: "PE imports", link: "/pe-imports" },
|
||||
{ text: "PE TLS/load config", link: "/pe-load-config" },
|
||||
{ text: "ELF notes", link: "/elf-notes" },
|
||||
{ text: "Apple IPS crash reports", link: "/apple-ips" },
|
||||
{ text: "Linux kernel modules", link: "/kernel-modules" },
|
||||
{ text: "CPython pyc files", link: "/python-pyc" },
|
||||
{ text: "Erlang BEAM", link: "/erlang-beam" },
|
||||
{
|
||||
@@ -53,11 +49,7 @@ export default defineConfig({
|
||||
text: "Web network captures",
|
||||
link: "/web-network-captures",
|
||||
},
|
||||
{
|
||||
text: "Build RL environments",
|
||||
link: "/agent-environments-quickstart",
|
||||
},
|
||||
{ text: "Environment reference", link: "/agent-environments" },
|
||||
{ text: "Agent environments", link: "/agent-environments" },
|
||||
{ text: "Electron observation", link: "/electron-observation" },
|
||||
{ text: "Electron fuses", link: "/electron-fuses" },
|
||||
{ text: "Process capture", link: "/process-capture" },
|
||||
@@ -70,7 +62,6 @@ export default defineConfig({
|
||||
{ text: "Apple applications", link: "/apple-application-analysis" },
|
||||
{ text: "Desktop installations", link: "/desktop-applications" },
|
||||
{ text: "Firmware", link: "/firmware-analysis" },
|
||||
{ text: "iNES containers", link: "/ines-containers" },
|
||||
{ text: "SQLite databases", link: "/sqlite-databases" },
|
||||
{ text: "Packet captures", link: "/packet-captures" },
|
||||
{ text: "Signal recordings", link: "/signal-recordings" },
|
||||
@@ -87,7 +78,6 @@ export default defineConfig({
|
||||
{ text: "Ghidra raw binary", link: "/raw-binary" },
|
||||
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
|
||||
{ text: "Ghidra PowerPC", link: "/ghidra-ppc" },
|
||||
{ text: "Ghidra GameCube DOL", link: "/ghidra-dol" },
|
||||
{ text: "Ghidra semantics", link: "/ghidra-provider" },
|
||||
],
|
||||
},
|
||||
|
||||
@@ -103,10 +103,6 @@ existing analysis families directly, for example
|
||||
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
|
||||
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
|
||||
|
||||
Screen capture validates the PNG signature, IHDR framing, dimensions, and IHDR
|
||||
checksum before publishing. It does not validate encoding fields or decode and
|
||||
validate image contents; that limitation is retained in Evidence.
|
||||
|
||||
When `push_adb_file` receives an existing directory, adb places the local file
|
||||
inside it under the local basename; Evidence reports that effective file path
|
||||
while retaining the caller's directory path in the input parameters. The
|
||||
@@ -140,29 +136,6 @@ and during shutdown.
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- **Package numeric declarations retain failures.** `inspect_adb_package`
|
||||
admits the first valid `userId` or `appId` as `user_id`: a nonnegative safe
|
||||
integer, including zero. A null value means no valid declaration was found.
|
||||
Malformed declarations before or after a valid value remain in
|
||||
`numeric_issues`: the first ten issues with command, field, reason and original
|
||||
`raw_line`, plus the exact `total`. Reasons are `invalid_format` or
|
||||
`unsafe_integer`. Any issue makes package coverage partial; existing section
|
||||
omissions can also make it partial. `version_code` remains exact source text.
|
||||
- **Display nulls preserve uncertainty.** `inspect_adb_display` admits positive
|
||||
safe integers. Absent or malformed optional values are null; `numeric_issues`
|
||||
distinguishes malformed declarations from absence. It keeps the first ten
|
||||
issues and exact total, with command, field, original line and reason
|
||||
(`invalid_format`, `unsafe_integer` or `nonpositive`). The last recognized
|
||||
labeled declaration wins, even when malformed. A physical-density label
|
||||
suppresses bare-number fallback even when malformed; without that label, the
|
||||
first unsigned bare number wins, including zero or an unsafe integer that
|
||||
leaves the value null. Display coverage is complete only with known physical
|
||||
density and no numeric issues. Missing or malformed required physical size
|
||||
produces a protocol failure with `details.partial_observation`: an
|
||||
`inspect_adb_display` observation from provider `adb`, reason
|
||||
`physical_size_absent` or `physical_size_malformed`, and a `result` retaining
|
||||
client, serial, optional display facts, numeric issues, null physical size
|
||||
and partial coverage.
|
||||
- **Directory names retain whitespace and newlines.** Symlink targets do too,
|
||||
including trailing newlines. Records containing invalid UTF-8 remain as
|
||||
labelled hex in `unparsed_lines` and make directory coverage partial.
|
||||
@@ -257,25 +230,15 @@ beyond already-connected devices (`adb connect`) are not exposed yet.
|
||||
|
||||
## Real verification
|
||||
|
||||
`npm run verify:adb` exercises observation and transfer operations against a live device. It
|
||||
`npm run verify:adb` exercises every operation against a live device. It
|
||||
requires `adb` (or `REA_ADB_PATH`) and any attached emulator or phone;
|
||||
`--serial` selects one when several are attached, `--pull PACKAGE` pulls a
|
||||
real package and verifies the returned digests against the files on disk.
|
||||
The optional `--install-apk` lifecycle is unavailable until the selected APK
|
||||
identity and device cleanup ownership can be established safely.
|
||||
real package and verifies the returned digests against the files on disk,
|
||||
and `--install-apk PATH` drives a full install → resolve → uninstall
|
||||
lifecycle with a lane-owned probe.
|
||||
|
||||
Record: verified against adb 34.0.5 on Linux against an Android 14
|
||||
(API 34) x86_64 emulator — all operations above, a 224-package inventory,
|
||||
287 services, 92 features, a real two-APK split set pulled with byte-exact
|
||||
digests, a verified push/pull roundtrip, a 1.3 MB screen capture, and the
|
||||
complete package lifecycle; see `docs/testing.md` for the lane summary.
|
||||
|
||||
Intent integer extras use signed decimal text. The ADB argument encoder removes
|
||||
redundant leading zeros for Android's radix-sensitive `--ei` parser while retained
|
||||
Evidence keeps the caller's original spelling. Package inspection accepts both
|
||||
`userId` and Android 16's `appId` spelling for the legacy `user_id` result field.
|
||||
|
||||
A completed push and device digest verification are separate observations. A
|
||||
verification timeout, cancellation, or output budget failure returns the typed
|
||||
failure with partial Evidence of the completed transfer. An unavailable checksum
|
||||
command or unrecognized checksum output leaves verification explicitly unknown.
|
||||
@@ -1,204 +0,0 @@
|
||||
# Build RL environments with REA
|
||||
|
||||
REA turns a recorded web app session into a runnable environment for training
|
||||
and testing AI agents. An agent can practice tasks such as finding an issue,
|
||||
creating a folder, or editing a record without running the original app.
|
||||
|
||||
REA builds the tools the agent calls, the local data those tools use, and a
|
||||
test that checks the environment starts and responds. It supports changes
|
||||
when the recording confirms their effects. You can then use the environment
|
||||
with your RL framework to set task goals, check rewards, and run training.
|
||||
|
||||
## Before you start
|
||||
|
||||
Install [REA and its agent integration](installation.md). To record an app,
|
||||
you also need a running test instance and a Chrome-family browser. Choose an
|
||||
instance where the agent is allowed to create, edit, and delete data. REA
|
||||
records the app you provide; you start it and prepare its test data.
|
||||
|
||||
To run the exported environment, install the
|
||||
[AgentEnv framework](https://github.com/scaleapi/agentenv-framework#install)
|
||||
and start Docker. REA's [compatibility check](testing.md#exported-environment-bundles)
|
||||
uses AgentEnv version `0.9.1334` with Python 3.12.
|
||||
|
||||
There are two ways to build an environment: ask your agent to record the app,
|
||||
or start from a recording you already have.
|
||||
|
||||
## 1. Record and build with your agent
|
||||
|
||||
Give your agent the app URL, the operations you need, your browser's executable
|
||||
path, and an output directory. For an app with folders, adapt this prompt:
|
||||
|
||||
```text
|
||||
Use REA to build an AgentEnv environment from my test web app at
|
||||
http://127.0.0.1:3000. Cover listing, creating, renaming, and deleting folders.
|
||||
You may create and remove test folders. Leave the existing data unchanged.
|
||||
|
||||
Sign in as admin using the password in APP_PASSWORD. Use the browser at
|
||||
/absolute/path/to/chromium. Save the environment to
|
||||
/absolute/path/to/my-env, which does not exist yet.
|
||||
|
||||
Record the session, check which operations the environment supports, and
|
||||
export it. Report any unsupported operations and what is missing.
|
||||
```
|
||||
|
||||
Replace the URL, account, paths, and folder operations with your own. Set
|
||||
`APP_PASSWORD` in the environment that starts REA's MCP server so REA can use
|
||||
it without putting the password in the prompt.
|
||||
|
||||
Your agent inspects the app, records a session, checks the result, and exports
|
||||
a folder containing the runnable environment. It reports which operations
|
||||
work and which need more evidence. The app is used during recording; the
|
||||
exported environment runs from the saved data.
|
||||
|
||||
After your agent exports the folder, skip to
|
||||
[run the environment](#3-run-the-environment). The next section is an
|
||||
alternative for an existing recording.
|
||||
|
||||
## 2. Or build from a saved recording
|
||||
|
||||
A HAR file records a browser's network requests and responses. Export one
|
||||
with API response bodies from your browser's network panel, or use a
|
||||
recording you already have. To include create, edit, and delete operations,
|
||||
record those actions and the reads that show their results.
|
||||
|
||||
Build the environment into a new or empty directory:
|
||||
|
||||
```sh
|
||||
rea environment-build ./session.har ./my-env --name my-env --json
|
||||
```
|
||||
|
||||
You can also pass capture Evidence JSON saved by REA's
|
||||
`capture-browser-scenario` or `inspect-web-network-capture` commands. Here,
|
||||
Evidence means REA's saved recording with its source information. The build
|
||||
accepts HAR files and those JSON recordings. It reads local data and writes
|
||||
the environment folder without contacting the app.
|
||||
|
||||
Add `--dry-run` to review the build report without writing the folder. For an
|
||||
unreleased REA checkout, use `--package-tarball` with its packed package so
|
||||
the exported environment runs the same implementation.
|
||||
|
||||
The build report tells you what the environment supports:
|
||||
|
||||
| Report field | What to check |
|
||||
| ----------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `state` | Whether the environment can answer from local state or uses recorded-response replay. |
|
||||
| `coverage.reads` | Which read operations use state and which replay recorded answers. |
|
||||
| `coverage.writes` | Which create, edit, and delete operations are supported. Each withheld operation includes a reason and a recording hint. |
|
||||
| `warnings` | Recorded content or local paths to review before sharing the folder. |
|
||||
|
||||
A successful build means the folder was created. Check the report to confirm
|
||||
it supports the operations your tasks need. `coverage: null` means the build
|
||||
could not prepare state for this environment and it will use replay.
|
||||
|
||||
## 3. Run the environment
|
||||
|
||||
Whether your agent exported the folder or you built it from a HAR, run it
|
||||
with AgentEnv:
|
||||
|
||||
```sh
|
||||
agent-env run ./my-env
|
||||
```
|
||||
|
||||
Use the folder path your agent reported if it differs. The first run builds
|
||||
the container image, so it needs access to the image and package sources.
|
||||
|
||||
REA includes a basic test, called the smoke task. It checks the expected tool
|
||||
count and, when a suitable read tool exists, calls it and checks its response
|
||||
status. Passing this test shows the environment starts and responds in the
|
||||
tested setup. It does not yet score an agent on an application task.
|
||||
|
||||
The folder contains the recording, a tool manifest describing the available
|
||||
operations, a Dockerfile, environment configuration, and the test. The
|
||||
[export reference](agent-environments.md#export-a-bundle) describes each file.
|
||||
|
||||
## 4. Add tasks and reward checks
|
||||
|
||||
Start with one task the exported tools support. Give it a clear goal and a
|
||||
checkable result. For example, if folder creation is supported, ask your
|
||||
agent:
|
||||
|
||||
```text
|
||||
Use this exported environment to add an AgentEnv task that asks an agent
|
||||
to create a folder named Practice. Check the saved environment state to
|
||||
confirm the folder exists with that name, and use that check to score the
|
||||
result. Reset the environment before each run so attempts start with the
|
||||
same data. Use the AgentEnv task and verifier formats.
|
||||
```
|
||||
|
||||
This asks your agent to help build the task and its grading check around the
|
||||
environment REA generated. A retrieval task might check an answer against a
|
||||
recorded value. A task that changes data should check the resulting state.
|
||||
When state is available, the environment supports reset, load, and read-back
|
||||
operations for repeatable runs.
|
||||
|
||||
Use the [AgentEnv documentation](https://www.agentenvframework.com/docs) for
|
||||
task configuration, agent connections, scoring, and execution. Connect the
|
||||
resulting tasks to your chosen RL training workflow.
|
||||
|
||||
## Improve an operation that is missing
|
||||
|
||||
For REA to support a change, the recording needs to show what happened before
|
||||
and after it. When recording against your test app:
|
||||
|
||||
1. Read the list and an existing item before making changes. Repeat the
|
||||
initial reads to show which values change independently.
|
||||
2. Create a test item, read it back, and read the list before the next change.
|
||||
Repeat with a second item and different text.
|
||||
3. Edit each test item, then read the item and list again. Leave more than a
|
||||
second between creation and the first edit to show time-dependent changes.
|
||||
4. Delete the test items, then read them back and read the list to confirm
|
||||
their removal.
|
||||
|
||||
Also record filters, searches, sorting, and paging if your tasks need them.
|
||||
The agent should check that every recording step completed before using it.
|
||||
The [recording workflow](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/environments.md)
|
||||
has the detailed browser instructions, including timeouts and handling IDs
|
||||
assigned by the app.
|
||||
|
||||
Use the report's hints to decide what to record next:
|
||||
|
||||
| Reason | Next step |
|
||||
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `unconfirmed` | Read the item and its list after the change. |
|
||||
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
|
||||
| `no-observation` | Read back a created item. For an edit or delete, also read it before changing it. |
|
||||
| `contradicted` or `side-effect` | Inspect the reported disagreement. The operation may need behavior REA cannot model yet. |
|
||||
|
||||
See the [full reason table](agent-environments.md#build-a-bundle-in-one-step)
|
||||
for the other cases. Rebuild from the new recording and check the report
|
||||
again. A new recording replaces the build input; combining separate recording
|
||||
sessions is not supported.
|
||||
|
||||
## Compare with the original app
|
||||
|
||||
For a stronger check, record a separate sequence of calls against the real
|
||||
app, starting from the same test data as the original recording. This is a
|
||||
probe. Restore that starting data before each probe sequence, then compare
|
||||
the saved probe recordings with the environment:
|
||||
|
||||
```sh
|
||||
rea environment-compare ./my-env/envs/my-env/manifest.json \
|
||||
--evidence ./my-env/envs/my-env/evidence.json \
|
||||
--probe ./probe-create-edit-delete.json --probe ./probe-reads.json --json
|
||||
```
|
||||
|
||||
Each probe file must be capture Evidence JSON from REA. The comparison runs
|
||||
offline and reports differences in responses, unsupported calls, and missing
|
||||
observations. It accounts for supported differences in IDs, times, and
|
||||
modeled generated values. Agreement applies to the calls those probes check.
|
||||
See the [comparison reference](agent-environments.md#measure-fidelity-against-the-application)
|
||||
for the exact rules.
|
||||
|
||||
## Before sharing the environment
|
||||
|
||||
The exported folder includes retained response bodies and local source paths.
|
||||
Standard authentication headers are excluded, but custom headers, query
|
||||
values, and body tokens can remain. Review the content you plan to share.
|
||||
Credential-name warnings point to values worth reviewing; they do not remove
|
||||
those values or establish that the recording contains no credentials.
|
||||
|
||||
For direct MCP serving, replay behavior, state controls, and resource limits,
|
||||
see the [environment reference](agent-environments.md). The
|
||||
[implementation index](specs/agent-environments/README.md) links the contributing
|
||||
PRs and their verification records.
|
||||
+47
-433
@@ -1,27 +1,16 @@
|
||||
# Agent environments from recorded captures
|
||||
|
||||
For a first environment, start with the
|
||||
[quickstart](agent-environments-quickstart.md): record an app or use a saved
|
||||
session, build and run the environment, and add tasks and reward checks.
|
||||
The [implementation index](specs/agent-environments/README.md) links
|
||||
the contributing PRs and their verification records. This page is the detailed
|
||||
command and behavior reference.
|
||||
|
||||
Use `rea environment-build` to turn one recorded capture into an agent
|
||||
environment bundle in one offline step: a folder `agent-env run` builds and
|
||||
runs, whose MCP server answers from what the capture observed. See
|
||||
[Build a bundle in one step](#build-a-bundle-in-one-step).
|
||||
|
||||
The steps it composes are separate commands too. `rea environment-serve` serves
|
||||
an environment manifest as an agent environment: an MCP server whose tools
|
||||
answer from what one retained web network capture observed, plus the
|
||||
environment card the agentenv framework reads to discover it. By default the
|
||||
server derives the [state the capture observed](#the-state-a-capture-observed)
|
||||
at startup and answers read tools from it, and the write tools whose effect the
|
||||
capture confirmed by changing it; `--replay-only` replays the capture's records
|
||||
Use `rea environment-serve` to serve an environment manifest as an agent
|
||||
environment: an MCP server whose tools answer from what one retained web
|
||||
network capture observed, plus the environment card the agentenv framework
|
||||
reads to discover it. By default the server derives the
|
||||
[state the capture observed](#the-state-a-capture-observed) at startup and
|
||||
answers read tools from it, and the write tools whose effect the capture
|
||||
confirmed by changing it; `--replay-only` replays the capture's records
|
||||
instead. Nothing is fetched from the recorded application; every answer comes
|
||||
from retained Evidence. `rea environment-export` writes the same pair as a
|
||||
bundle folder; see [Export a bundle](#export-a-bundle).
|
||||
bundle folder `agent-env run` builds and runs; see
|
||||
[Export a bundle](#export-a-bundle).
|
||||
|
||||
```sh
|
||||
rea inspect-web-network-capture /absolute/session.har har --format json > capture.json
|
||||
@@ -47,190 +36,6 @@ The third command serves that manifest from the state it derives from the
|
||||
capture; see [Serve from the state](#serve-from-the-state). Both served files
|
||||
are complete REA Evidence JSON as the commands write it with `--format json`;
|
||||
the `evidence` member of `view.json` is the whole of `capture.json`.
|
||||
A `view.json` can also name the capture file as
|
||||
`{"kind": "evidence-file", "path": "capture.json"}`, which projects it as
|
||||
retained Evidence instead of charging the whole capture as inline input.
|
||||
|
||||
## Build a bundle in one step
|
||||
|
||||
```sh
|
||||
rea environment-build grafana.har ./grafana --json
|
||||
agent-env run ./grafana
|
||||
```
|
||||
|
||||
`rea environment-build <capture> [directory]` takes a `.har` file, or capture
|
||||
Evidence JSON from `inspect-web-network-capture` (HAR) or
|
||||
`capture-browser-scenario`, and writes the bundle
|
||||
[Export a bundle](#export-a-bundle) describes. It inspects a HAR as
|
||||
`inspect-web-network-capture` does (Evidence input skips this), projects the
|
||||
environment manifest from that Evidence as retained Evidence, authenticates
|
||||
the pair, prepares the state the bundle's server derives at startup, and
|
||||
writes the bundle with the export. For the same capture path, every file it
|
||||
writes is byte-identical to what `inspect-web-network-capture`,
|
||||
`inspect-analysis-view` and `environment-export` write, Evidence ids included,
|
||||
when `view.json` lists the same rules in the views' order and the export is
|
||||
given the build's name. It reads one file and writes one folder: it fetches
|
||||
nothing and launches no browser (HAR inspection runs REA's own decoder
|
||||
process), so record a browser scenario with `capture-browser-scenario` first
|
||||
and pass its Evidence. Other capture formats are refused, as the manifest view
|
||||
refuses them, and a HAR document saved as `.json` is refused with a hint to
|
||||
name it `.har`.
|
||||
|
||||
- `--template-rule <rule>`, repeated, replaces the default opt-in rules
|
||||
`after-uid-literal`, `opaque-token`, `sibling-variance` and
|
||||
`prefix-variance`, the rules the published capture-built environments use;
|
||||
`--format-rules-only` applies none, the views' own default. Rules apply in
|
||||
the views' fixed order, the report lists them, and a repeated rule is
|
||||
refused as the views refuse it.
|
||||
- `--name` names the env folder and env id. Without it the name is the
|
||||
manifest's, unless the capture's origin is a loopback name or an IP literal:
|
||||
the capture file then names it (`grafana.har` and `grafana.capture.json`
|
||||
become `grafana`) and a `name-from-file` warning says so. A file name that
|
||||
yields no portable label (kebab-case, at most 63 characters for a DNS service
|
||||
label, not a Windows device name or a label interpreted as an IP address) is
|
||||
refused; pass `--name`.
|
||||
- `--package-tarball` works as for the export, and `--capture-timeout-ms`
|
||||
as for `inspect-web-network-capture`.
|
||||
- `--dry-run` runs every stage and prints the report with `status: "planned"`
|
||||
and the files a build would write, but writes nothing and does not check the
|
||||
directory, which may then be omitted. It reads and hashes a
|
||||
`--package-tarball` as a build does before copying it, so it refuses the
|
||||
tarballs a build refuses, including one whose contents cannot be read.
|
||||
|
||||
The directory must be absent or empty, as for the export; the build never
|
||||
replaces a folder. A refusal before the write leaves nothing behind. A capture
|
||||
Evidence file, an inspected HAR's Evidence and the projected manifest must each
|
||||
fit the live heap headroom for parsing and sealing, checked before the file is
|
||||
read or the Evidence serialized; one that does not is refused as
|
||||
`resource_constraint`. Each
|
||||
refusal names its stage: `environment-build` for the build's own checks,
|
||||
`inspect_web_network_capture` and `inspect_analysis_view` for inspection and
|
||||
projection, `environment-export` for the bundle. SIGINT cancels the build
|
||||
(`cancelled`); before the bundle stage nothing is written, and a cancellation
|
||||
during the write names the bundle it wrote. Stage progress is logged to stderr
|
||||
at info level; the report is the only output on stdout:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "built",
|
||||
"directory": "/work/grafana",
|
||||
"name": "grafana",
|
||||
"capture": {
|
||||
"evidence_id": "ev_…",
|
||||
"operation": "inspect_web_network_capture",
|
||||
"records": 105,
|
||||
"input": "har"
|
||||
},
|
||||
"manifest": {
|
||||
"evidence_id": "ev_…",
|
||||
"tools": 48,
|
||||
"excluded_groups": 0,
|
||||
"template_rules": [
|
||||
"after-uid-literal",
|
||||
"opaque-token",
|
||||
"sibling-variance",
|
||||
"prefix-variance"
|
||||
]
|
||||
},
|
||||
"state": { "served": true, "evidence_id": "ev_…" },
|
||||
"coverage": {
|
||||
"reads": {
|
||||
"state": 36,
|
||||
"recorded": 6,
|
||||
"recorded_reasons": { "query-varies": 3, "…": 1 }
|
||||
},
|
||||
"writes": {
|
||||
"served": [{ "tool": "delete_folders_by_uid", "effect": "delete" }],
|
||||
"withheld": [
|
||||
{
|
||||
"tool": "post_folders",
|
||||
"effect": "create",
|
||||
"reason": "unconfirmed",
|
||||
"hint": "No later read showed the effect. Read the entity and its list after the write.",
|
||||
"exemplar_ordinal": 94,
|
||||
"contradicted_by": null
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"tool_count": 37,
|
||||
"package_source": { "kind": "registry", "specifier": "rea-agents@…" },
|
||||
"files": [{ "path": "README.md", "bytes": 4210 }],
|
||||
"warnings": [{ "code": "name-from-file", "message": "…" }],
|
||||
"next": ["agent-env run /work/grafana"]
|
||||
}
|
||||
```
|
||||
|
||||
`coverage` is the state's read views and write plans summarized, and `null`
|
||||
when the state is unavailable and the bundle serves replay. The
|
||||
[environment coverage](web-network-captures.md#environment-coverage) view
|
||||
reports the same summary for any capture without building, so an agent can
|
||||
check it after each recording. The view summarizes the state as derived; it
|
||||
does not apply the serve-time admission checks, so for an environment whose
|
||||
served catalog exceeds the discovery budget, or whose state exceeds what
|
||||
`data/add` loads, it still reports the state while the build reports `coverage:
|
||||
null` and the bundle serves replay. Each withheld write carries a hint about
|
||||
what to record next; hints never change what is served:
|
||||
|
||||
| Reason | Hint |
|
||||
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `no-state-target` | The write could not be linked to a modelled target. Check the path template and recorded views; record missing reads. Some route forms may not be supported. |
|
||||
| `no-call` | No recorded call of this tool was modelled, for example because every call failed. Record a successful call. |
|
||||
| `body-unusable` | The retained request or response cannot be modelled. Record a successful call with a complete supported request and reproducible response. |
|
||||
| `no-observation` | Nothing showed the entity or document this write changed. Read a created entity back by id right after the write; for an update or delete the state must already hold the entity, so read it before the write or record the create that made it. |
|
||||
| `unexplained-change` | The write changed fields it did not send, in a way no rule explains. A second, different call of the same tool can show the pattern. |
|
||||
| `contradicted` | A later read disagreed with the modelled effect. Check the read at contradicted_by. |
|
||||
| `side-effect` | The write changed something besides its own entity, which the environment does not model. |
|
||||
| `unconfirmed` | No later read showed the effect. Read the entity and its list after the write. |
|
||||
| `incomplete-entities` | The write is confirmed, but the views that show its entity cannot render it in full. Read the written entity through every view that lists it. |
|
||||
| `unknown-clock` | The learned rules generate timestamps, but the capture establishes no global served clock start. Record usable start times and completed durations for every request. |
|
||||
|
||||
The bundle ships `evidence.json`, the whole capture Evidence, response bodies
|
||||
included. Inspection excludes the values of the standard authentication headers
|
||||
(`Authorization`, `Cookie` and `Set-Cookie` among them) and URL userinfo; a
|
||||
custom key header such as `X-Api-Key`, a token in the query string and tokens
|
||||
an application returns in bodies remain. The environment views refuse captures
|
||||
inspected with `--sensitive-value`, so scrub a capture before building it. The
|
||||
report's `credential-named-members` warning counts recorded header names, query
|
||||
parameter names and JSON or form body members whose names look like
|
||||
credentials (whole name segments such as `token`, `secret`, `password`,
|
||||
`api_key`, `authorization` or `session`) and hold a value, and lists the first
|
||||
20 by record and pointer, never by value. A body is read as a form or as JSON by
|
||||
its recorded media type, or by its shape when it has none or one naming
|
||||
neither. A browser scenario is
|
||||
scanned by event (`event 4 response /token` names
|
||||
`normalized_result.events.items[4]`): every retained request and response
|
||||
content event, including a response whose request began before the browser was
|
||||
attached and a transaction no record represents, and each transaction URL's
|
||||
query once. Bodies larger than 4 MiB of UTF-8 text are not
|
||||
scanned; `credential-scan-incomplete` counts them and lists up to 20 record or
|
||||
event pointers, including base64 bodies whose decoded scan was cut. An empty name
|
||||
warning does not establish that these bodies were checked. A browser scenario's
|
||||
saved values (`save` and `save_response`) named like credentials but not marked
|
||||
`secret: true` are recorded unredacted; `credential-named-saved-values` lists up
|
||||
to 20 of them by parameter pointer. Names do not establish sensitivity,
|
||||
so the build never refuses on them. Capture
|
||||
|
||||
Evidence also records the capture file's absolute path (`subject.local_path`,
|
||||
`parameters.capture_path`, `locations`, `artifact.path`), and the bundle
|
||||
carries it: build from a neutral folder when that path should not ship.
|
||||
|
||||
Over MCP, `export_environment_bundle` builds the same bundle and returns the
|
||||
same report. Its `source` is capture Evidence the session retained
|
||||
(`{"kind": "retained-evidence", "evidence_id"}`), such as a
|
||||
`capture_browser_scenario` or `inspect_web_network_capture` result, or a
|
||||
capture Evidence file (`{"kind": "evidence-file", "path"}`); inspect a HAR with
|
||||
`inspect_web_network_capture` first. Retained Evidence is written into the
|
||||
bundle as `--format json` writes it, so the bundle is the one
|
||||
`environment-build` writes from that file. `output_directory` must be an
|
||||
absolute path that does not exist yet, whose parent does; the tool creates it
|
||||
and refuses an existing directory, empty or not. `name`, `template_rules`,
|
||||
`package_tarball` and `dry_run` work as `--name`, `--template-rule`,
|
||||
`--package-tarball` and `--dry-run` do, and retained Evidence has no file to
|
||||
name the environment after, so a loopback or IP origin needs `name`. Unlike
|
||||
`--dry-run`, `dry_run` still checks `output_directory`. Destination refusals
|
||||
name `export_environment_bundle`; the others name their stage as
|
||||
`environment-build`'s do.
|
||||
|
||||
## From a browser scenario
|
||||
|
||||
@@ -320,51 +125,6 @@ the replayed body is the bytes the browser exposes for each response, with any
|
||||
declared secret replaced by its redaction marker (a body that redaction had to
|
||||
empty replays no body).
|
||||
|
||||
To record writes, sign in through the UI and then send the application's API
|
||||
calls with [request steps](browser-scenario-contract.md#request-steps): each
|
||||
sends one request to the start origin through the page's own `fetch`, with the
|
||||
session's cookies, and a header can carry the CSRF cookie or stored token the
|
||||
application's own requests send. A write is served only when the capture shows
|
||||
its effect, so read each collection before writing, read every written entity
|
||||
back by id and in its list right after the write, and leave more than a second
|
||||
between a create and its first edit. A request step's path is fixed when the
|
||||
scenario is written and cannot use a value an earlier response returned, so an
|
||||
id must be known before the run: one the caller chooses, such as a `uid` in the
|
||||
create body, or one an earlier scenario created. When the server assigns ids,
|
||||
read the list after each create (the create's response or the list shows the
|
||||
new entity), and record updates and deletes in a second scenario that reads
|
||||
the first one's objects by id before editing them, or make them through the
|
||||
UI. The environment coverage view names each
|
||||
write it withholds and what to record next. A scenario with write requests
|
||||
changes the application's data: record against an instance whose data may
|
||||
change.
|
||||
|
||||
```json
|
||||
{
|
||||
"step_id": "create-folder-1",
|
||||
"action": "request",
|
||||
"method": "POST",
|
||||
"path": "/api/folders",
|
||||
"headers": [
|
||||
{
|
||||
"name": "X-CSRFToken",
|
||||
"value": { "source": "cookie", "name": "csrftoken" }
|
||||
}
|
||||
],
|
||||
"json": { "title": "Scratch folder 1" },
|
||||
"expected_status": [200, 201]
|
||||
}
|
||||
```
|
||||
|
||||
Each record takes its start and duration from the browser's timing (the time
|
||||
the scenario observed the request where the browser reported no start), so the
|
||||
state orders and overlaps
|
||||
records as it does a HAR's and learns a write clock: an update the application
|
||||
stamps with a modification time can be served. A scenario captured before
|
||||
these times were recorded has none; its records keep request order and the
|
||||
state learns no clock, so such updates stay `unexplained-change` until it is
|
||||
recorded again.
|
||||
|
||||
## What a manifest is
|
||||
|
||||
An environment manifest is a projection of one application's recorded request
|
||||
@@ -405,11 +165,7 @@ Each such MCP tool call selects one of the records its manifest tool cites and
|
||||
returns that record's recorded response. Selection compares the supplied path
|
||||
parameters against the record's path segments at the parameter indexes, then
|
||||
the supplied query values against the record's query string, and keeps the
|
||||
record with the most matches; ties keep manifest order. A query argument accepts
|
||||
one string, a number or boolean sent as its JSON text, or an array of strings for
|
||||
repeated values; the array is matched
|
||||
against every occurrence in captured order. The raw recorded query remains the
|
||||
replay evidence. The result reports
|
||||
record with the most matches; ties keep manifest order. The result reports
|
||||
where the answer came from:
|
||||
|
||||
```json
|
||||
@@ -431,14 +187,9 @@ where the answer came from:
|
||||
}
|
||||
```
|
||||
|
||||
`exact_match` is true when every supplied path parameter and query occurrence
|
||||
matches the recorded one; it says nothing about parameters the caller omitted.
|
||||
Comparison probes retain the raw query spelling and its decoded parameter pairs
|
||||
in request order, including the interleaving of repeated values across names.
|
||||
Standalone tool calls express repeated values as per-name arrays; when matching
|
||||
records differ only in cross-name order, `exact_match` is false because those arguments cannot
|
||||
identify that order. Recorded query parsing runs only when supplied query
|
||||
arguments affect selection.
|
||||
`exact_match` is true when every supplied path parameter and query value
|
||||
equals the recorded one; it says nothing about parameters the caller omitted.
|
||||
Recorded query parsing runs only when supplied query arguments affect selection.
|
||||
A path value matches a recorded segment as written or after percent-decoding;
|
||||
when both kinds of match are available, the record whose segment equals the
|
||||
value as written is chosen. `recorded_query` keeps the `?` of an explicit empty
|
||||
@@ -614,9 +365,7 @@ shape and carries no annotations.
|
||||
Path parameters are required strings whose
|
||||
description names the segment index, the templating rule and the observed
|
||||
values (sampled within the list budget above; the manifest keeps them all);
|
||||
query parameter names accept an optional string, number or boolean (a number or
|
||||
boolean is sent as its JSON text) or array of strings (for repeated values in
|
||||
captured order); write methods accept an
|
||||
query parameter names are optional strings; write methods accept an
|
||||
optional `body` object described by the recorded top-level key types. Each
|
||||
input keeps its own argument: a query name equal to a path parameter's name is
|
||||
advertised as `query_<name>`, a query name longer than 64 characters as
|
||||
@@ -673,58 +422,29 @@ member reports which applied:
|
||||
|
||||
Each read tool answers according to its view in the state:
|
||||
|
||||
| View | Answer |
|
||||
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`, whose `per_response_members` (such as a request id), like a list's or a document's, keep their recorded values. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`, except that a call matching one of the view's `projected_query_keys`, or on a view with `projected_record_ordinals` asking for some fields only with a value other than `*` no checked line gave or one a replayed read gave, replays its recording. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error, unless a read showed the application leaves that field out; a body is never invented. |
|
||||
| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"` or a line `all_record_ordinals` names, or the recorded members still held. A call with no query argument answers from such a line of the collection its path arguments name that only pages its list, with `replay.exact_match` false. On a line the view's `query` answers, and for a call whose every query argument has a role there (an argument-less call aside), the answer is the query evaluated over the state: filtered, searched, sorted and paged, with counters at the number listed and `query.totals` at the number matched, and `state.membership: "query"`; such a call answers on a query or whole line of its collection with `replay.exact_match` false. A value an `inert` parameter never recorded, a parameter without a role, a call matching one of the view's `projected_query_keys`, or a field selection the checked lines do not show (as for `item`), replays its recording. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
|
||||
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
|
||||
| `recorded` | The recording, as replay answers it. |
|
||||
|
||||
A read with repeated query values uses replay, because the state query model
|
||||
does not infer repeated-parameter semantics. A repeated-query write is refused
|
||||
before it can change state for the same reason.
|
||||
|
||||
List counters retain their recorded number or digit-string representation.
|
||||
Changing a digit-string counter preserves its observed zero-padding width;
|
||||
query-total inference remains limited to canonical decimal counts. Path matching
|
||||
uses the same segment parser as extraction, preserving empty internal segments.
|
||||
| View | Answer |
|
||||
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error, unless a read showed the application leaves that field out; a body is never invented. |
|
||||
| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"` or a line `all_record_ordinals` names, or the recorded members still held. A call with no query argument answers from such a line of the collection its path arguments name that only pages its list, with `replay.exact_match` false. On a line the view's `query` answers, and for a call whose every query argument has a role there (an argument-less call aside), the answer is the query evaluated over the state: filtered, searched, sorted and paged, with counters at the number listed and `query.totals` at the number matched, and `state.membership: "query"`; such a call answers on a query or whole line of its collection with `replay.exact_match` false. A value an `inert` parameter never recorded, or a parameter without a role, replays its recording. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
|
||||
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
|
||||
| `recorded` | The recording, as replay answers it. |
|
||||
|
||||
An item or list view that always spells a clock field otherwise than the
|
||||
entity's stored value names that spelling in `clock_formats`, and a value it
|
||||
never recorded, such as a created entity's clock, is rendered in it: a list that
|
||||
writes `+00:00` where item reads write `Z` lists a new entity the same way.
|
||||
A view that names a field in `sorted_arrays` lists that string array sorted, as
|
||||
a read-back showed the application does, while the state keeps it in the order
|
||||
it was written, so a write's own answer echoes it. If a later counted read
|
||||
contradicts that ordering, the view stays recorded and its earlier reads no
|
||||
longer confirm served writes. A view that names a field in `nested_fields`
|
||||
always showed that field's objects with the same keys, in any order, and fewer
|
||||
than an entity stores, such
|
||||
as a list showing a parent without the parent's own nested type; a stored
|
||||
object it never recorded, such as a created entity's parent, is rendered with
|
||||
those keys. Members a view names in
|
||||
`per_response_members`, such as a request id each response mints, keep the
|
||||
value its recording carried. A list's `counters` and `query.totals` are
|
||||
top-level members or members one level down in a paging object
|
||||
(`/pageInfo/totalRows`); either is set to the number listed or matched, and
|
||||
members beside it such as `isLastPage` stay as recorded.
|
||||
|
||||
A list or document call that names no checked line, including a line recorded
|
||||
after a write or with other query values, answers from the recording. Every
|
||||
answer has the replay shape plus `state`: `{"source": "state", "view": …}`
|
||||
naming the entity, collection or document path, or `{"source": "recording",
|
||||
"reason": …}` with the view's reason, `no-view`, `unrecorded-request`, `page-exceeded` (a call naming a page of a whole list that the collection has outgrown), `projection` (a call to an item or list tool with replayed reads asking for a selection of fields the checked lines do not show, or leaving out a projection parameter every checked line names) or `variant` (a call matching another `projected_query_keys` key: a parameter, or a value of one, only lines showing the entities in another representation named, or leaving out a parameter every other line named). A `projection` or `variant` answer is a recording, so it does not show served writes. A
|
||||
"reason": …}` with the view's reason, `no-view`, `unrecorded-request`, `page-exceeded` (a call naming a page of a whole list that the collection has outgrown) or `projection` (a call to an item tool whose recordings asked for some fields only, naming such a parameter for an entity the state holds; the answer is a recording, so it does not show served writes). A
|
||||
state answer carries the base record's status, content type and `replay`
|
||||
provenance with `truncated: false`; one whose JSON would exceed 256 KiB is
|
||||
refused as a `resource_constraint` tool error rather than cut. Extraction does
|
||||
not state-back a document whose rendered JSON exceeds 256 KiB; that read stays
|
||||
recorded (`rendered-oversize`) and replays the captured body. Query `POST`s,
|
||||
and `POST` tools the capture shows only reading (a lookup answering as a `GET`
|
||||
did, an RPC tunnel naming read operations, a session renewal), stay served and
|
||||
answer from their recordings with the reason `query-body`; they are not write
|
||||
tools.
|
||||
`HEAD`, `OPTIONS` and `TRACE` tools also replay their captured responses; their
|
||||
state views report `method-replay` and coverage counts them as recorded reads.
|
||||
recorded (`rendered-oversize`) and replays the captured body. Query `POST`s
|
||||
stay served and answer from their recordings with the reason `query-body`.
|
||||
|
||||
A line establishes whole-collection membership only from the beginning:
|
||||
`offset`, `start`, `_start` and `page` must be zero when present. Positive or
|
||||
@@ -739,58 +459,29 @@ tools are not served, so no call claims a change it does not make; each one's
|
||||
plan states why in its `reason`. A served write changes the live state, which
|
||||
every MCP session and `data/get` then see:
|
||||
|
||||
| Effect | Call |
|
||||
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
|
||||
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, nested clock, increment, slug, stamp, derived, composed, reference, unique and renewed changes the plan models. |
|
||||
| `update` (action) | A `POST`, `PUT` or `PATCH` to an item path plus an action segment (`/{type}/{id}/archive`) sets, on the entity before the segment, each scalar the recorded calls showed the action set (`set`), or flips each boolean they showed it flipping both ways (`toggle`), with any sent members; it is served only when a recorded call found what it sets already set, or flipped back what it toggles. |
|
||||
| `delete` | Removes the entity the path names. |
|
||||
| bulk `update` or `delete` | A `DELETE`, `PUT` or `PATCH` at a collection, or a `POST` to it plus `delete`, `remove` or `destroy`, applies its effect to every entity the body names, as recorded: an array of ids or of objects carrying the id field, at the root or as the body's one array member (`{"IDs": […], "Read": true}`). |
|
||||
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
|
||||
|
||||
An update's unsent changes follow the plan's rules. A `held-clock` is a clock
|
||||
the client sends back as it read it and the application stamps anew (Ghost's
|
||||
`updated_at`): the call must send the entity's value and the field takes the
|
||||
write's clock. A `nested-clock` sets the clocks inside an object the entity
|
||||
holds, in that object's recorded format. A `stamp` writes the acting user
|
||||
the entity already names under its `created…by` counterpart into a
|
||||
`modified…by` or `updated…by` field. A `derived` field takes a value showing
|
||||
the one sent member's text that the entity stores nowhere (a field Ghost
|
||||
derives from `html`), only with that member's recorded value. An action's
|
||||
`set` rule sets a scalar to the value the recorded calls left it at, and its
|
||||
`toggle` rule flips a boolean, refusing a call on an entity that holds none
|
||||
there (Miniflux's `PUT /v1/entries/{id}/bookmark` stars and unstars).
|
||||
| Effect | Call |
|
||||
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
|
||||
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, increment, slug, unique and renewed changes the plan models. |
|
||||
| `delete` | Removes the entity the path names. |
|
||||
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
|
||||
|
||||
Its answer has the status and content type of the plan's recorded response,
|
||||
with the written entity or its identity rendered in, no body, or the recorded
|
||||
body. A call whose body is not an object, sends a member the capture did not
|
||||
show stored, sends a fixed member with another value or a member of another
|
||||
JSON type, or lacks a required member is a tool error naming the accepted
|
||||
members; so is a call sending query parameters other than the exemplar's (they
|
||||
may change what the body does), a body not wrapped in the exemplar's envelope
|
||||
(Ghost's `{"posts": [{…}]}`), a bulk call naming an id the state does not hold,
|
||||
a `held-clock` member sent with another value than the entity holds, a call
|
||||
sending a `derived` rule's member to an entity that does not hold the value the
|
||||
recorded call found, a nested clock on an entity holding no such object, and a
|
||||
rendered answer missing a field the recorded response carried, and an update or delete of an entity the state does not hold answers
|
||||
members; so is a rendered answer missing a field the recorded response
|
||||
carried, and an update or delete of an entity the state does not hold answers
|
||||
the recorded 400, 404 or 410 of the tool or of an item view of the type, if
|
||||
any. A served delete of a type whose item view has a `deleted_flag` (the
|
||||
member, `deleted`, `isDeleted` or `is_deleted`, its reads set `true` on a
|
||||
record the application soft-deleted, as EspoCRM does) keeps the entity until
|
||||
`data/reset` or `data/add`, within the size `data/add` loads, so a read of it by
|
||||
id answers its last fields with that flag `true`. A refused call leaves the
|
||||
state unchanged. Values a write generates are
|
||||
any. A refused call leaves the state unchanged. Values a write generates are
|
||||
deterministic: the clock starts at `write_clock` and advances one second per
|
||||
served write, sequences take one past both the largest held value and the
|
||||
largest they generated, and UUIDs and tokens derive from the state Evidence
|
||||
id, tool, field and write count, skipping held values. If `write_clock` is null,
|
||||
plans needing `clock`, `held-clock`, or `nested-clock` retain their learned rules
|
||||
but are withheld with `unknown-clock`; other confirmed plans can still be served.
|
||||
`data/reset` and
|
||||
id, tool, field and write count, skipping held values. `data/reset` and
|
||||
`data/add` restart the clock, counter and sequences, so the same calls after a
|
||||
reset give the same state. Answers carry `state: {"source": "state", "view":
|
||||
"create" | "update" | "delete", …}` or `"document-write"`; a bulk answer adds
|
||||
`ids`, every entity it wrote in body order, with `id` naming the first.
|
||||
"create" | "update" | "delete", …}` or `"document-write"`.
|
||||
|
||||
The data plane serves three JSON-RPC methods:
|
||||
|
||||
@@ -882,17 +573,13 @@ id are unique; `view_fields` is optional and names item or list views of its
|
||||
type. Views render their values over the entity's fields, so a view's values
|
||||
must not set the type's identity field to a value whose text differs from the
|
||||
entity's `id`. Spell a collection as requests spell it, as `data/get` reports it: list
|
||||
calls find entities under the checked line's recorded path. State collection
|
||||
identity preserves repeated separators inside captured paths: `/api/books`
|
||||
and `/api//books` are distinct. A single trailing slash continues to name the
|
||||
same collection. Each document
|
||||
calls find entities under the checked line's recorded path. Each document
|
||||
names a `document` view once, at its initial document's `path`. The `version`,
|
||||
and the `state_evidence_id` and `types` that `data/get` reports, must match
|
||||
the served state, so a `data/get` result loads back unchanged. A number without
|
||||
an exact native value, or whose serialized spelling would not be read as exact
|
||||
(an integer from 2^53 below 1e21, such as `1e20`, in any spelling), is refused
|
||||
rather than altered. Caller data preserves all own JSON members, including
|
||||
`__proto__`; unknown members of the state document's structure are refused.
|
||||
(an integer from 2^53 below 1e21, such as `1e20`, in any spelling), or a
|
||||
`__proto__` member is refused rather than altered.
|
||||
An invalid document is `-32602` naming the first problem by JSON pointer, as in
|
||||
`/rea_state/entities/3/collection` (also in `error.data.path`); a problem
|
||||
inside caller data (`fields`, `value`, `types`) is located at that member, so
|
||||
@@ -946,25 +633,6 @@ after an update and referential rules on delete are not modelled. An exported
|
||||
bundle carries no state file: its server derives the state at startup, as the
|
||||
export does to predict what it serves.
|
||||
|
||||
Known limits of served writes:
|
||||
|
||||
- A recorded view answers from its recordings after a served write: Mailpit's
|
||||
`get_messages`, recorded as `list-differs`, still lists a message a served
|
||||
delete removed.
|
||||
- Served n8n workflow updates accept any `versionId`, while served Ghost post
|
||||
updates require `updated_at` to match the entity (a `held-clock`).
|
||||
- A slug derived from a title can repeat across served creates with the same
|
||||
title, where the application would make it unique.
|
||||
- A `composed` string copies sent text as sent: an application that escapes or
|
||||
renders it (HTML entities, Markdown) differs for text that needs it, as
|
||||
BookStack's `description_html` would for a description holding `&`.
|
||||
- A served create's sequence id is the state's next value, not the gaps an
|
||||
application's own sequence would leave.
|
||||
- Nested clocks are served in the spelling of the write's own response, which
|
||||
can differ from what a later read of the application shows.
|
||||
- A served write to an entity that a replayed (unserved) create made answers
|
||||
as for a missing entity, since the state never held it.
|
||||
|
||||
## Export a bundle
|
||||
|
||||
`rea environment-export` writes the same pair as a bundle folder the agentenv
|
||||
@@ -1001,8 +669,7 @@ serves replay. The complete tool catalog the bundle serves must also fit the
|
||||
server's advertised catalog budget before any files are written. Privately parsed JSON is sealed once so authentication
|
||||
and replay share immutable response trees rather than copying the capture repeatedly.
|
||||
`--name` replaces the manifest's portable kebab-case name as the env folder and env id
|
||||
(at most 63 characters for its DNS service label; Windows device names and
|
||||
labels interpreted as IP addresses are refused);
|
||||
(Windows device names such as `con` and `lpt1` are refused);
|
||||
`--package-tarball <path>` copies a packed `rea-agents` tarball into the env
|
||||
folder and makes the Dockerfile install it instead of the registry package and
|
||||
remove it in the same layer, which is how a checkout rather than a release is
|
||||
@@ -1020,12 +687,7 @@ Windows inherits ACLs and REA does not establish owner-only access there. Choose
|
||||
an already private Windows destination when captured content needs it.
|
||||
|
||||
The Dockerfile sets `MCP_HOST=0.0.0.0` and `MCP_PORT=18765`, the port the
|
||||
framework maps for an MCP server env, and `MCP_ALLOWED_HOSTS` naming the loopback
|
||||
addresses and the environment name, because the container binds outside loopback
|
||||
and the server refuses to serve on such a bind without an operator-declared
|
||||
allowlist: the framework's published port is reached over loopback and its
|
||||
gateway proxies to the env container by the compose service name (the
|
||||
environment name). The framework's own deploy supplies
|
||||
framework maps for an MCP server env; the framework's own deploy supplies
|
||||
`MCP_HOST`, `MCP_PORT` and `ENVIRONMENT_NAME`, so the served card carries the
|
||||
env's name. It installs Debian's `python3` because the framework's local
|
||||
sandbox probes a container's readiness with `python3` inside it, which the slim
|
||||
@@ -1093,53 +755,10 @@ recording replayed in its place and never from a recording whose body the probe
|
||||
two collections can reuse a live id, and an equal pair stays in that collection. A live id in a later request's path, query or body is translated before the
|
||||
environment answers, a path parameter by the collection the live path names (a nested collection keeps its parent values, each decoded once, and a literal route segment stays), except a query value that pages a list, such as `limit`
|
||||
or `page`. Each call is checked against the served tool's input schema first, as the MCP server checks it. After mapping, two clock values compare by format only (ISO-8601 strings, and epoch numbers under
|
||||
`at`, `time`, `date`, `timestamp`, `created`, `updated` or `modified` field names, including snake-case and camel-case forms).
|
||||
Numeric arrays retain that containing field's clock context through nested
|
||||
arrays; an object's own fields establish their own context. Numeric values at
|
||||
the body root, in root arrays, or under measurement fields remain literal.
|
||||
|
||||
Generated values normalize only where this confirmed, served write's entity
|
||||
response rules identify generation. A UUID identity rule must still compare two
|
||||
UUIDs, and a `unique` rule two UUIDs or two generated integers (10 to 20
|
||||
digits) of one JSON kind and length; a `renewed` rule applies only at its
|
||||
recorded pointers,
|
||||
with `*` matching actual array items. Token/sequence strings retain the existing
|
||||
format check: equal length of at least eight, the same character classes, and
|
||||
a digit. Numeric sequences continue to compare literally unless an ID mapping
|
||||
applies. String shape and names such as `token` or `sku` alone do not establish
|
||||
generation. Read and replay responses without these facts stay literal except
|
||||
for supported clocks and learned IDs. A missing write target replays an absence
|
||||
response and does not inherit the successful entity response's generation rules.
|
||||
Admitted generated values must also preserve aliases and distinct identities
|
||||
within that response. UUID identity uses the generator's case-insensitive
|
||||
matching; token and sequence strings retain their literal identity. This
|
||||
correspondence ends with the answer and is not inferred across calls.
|
||||
A `composed` response field must spell the value its parts build from that
|
||||
response's own fields. Its parts then compare by their rendered text or their
|
||||
source fields' clock and ID rules. A coherent archive URL can therefore differ
|
||||
only in normalized clock digits without reporting a value difference; an
|
||||
incoherent URL still differs even when both sides repeat the same stale text.
|
||||
Normalized compositions count under `generated`; source fields are counted once.
|
||||
The report counts each kind of normalised
|
||||
value; the generation rules remain model-derived facts, rather than new
|
||||
observations of the application.
|
||||
`at`, `time`, `date`, `timestamp`, `created`, `updated` or `modified` field names, including snake-case and camel-case forms), and two different values each side generated on its own compare equal: both UUIDs, or one length of
|
||||
eight or more with the same character classes and a digit. The report counts each kind of normalised value.
|
||||
ISO clock offsets are accepted only within RFC 3339's `00`–`23` hour and `00`–`59` minute ranges.
|
||||
Probe ordering retains every recorded fractional-second digit in ISO timestamps
|
||||
with a four-digit year, `T`, and a `Z` or `±HH:MM` suffix. Equal instants use ordinal order. Other native-Date
|
||||
spellings keep their existing millisecond interpretation; an absent or unreadable
|
||||
start time makes the whole probe use record order. Original capture timestamps
|
||||
remain unchanged. Clock-format comparison and respelling recognize four-digit
|
||||
ISO years `0000`–`0099` as well as later years, with normal calendar validation.
|
||||
Respelling preserves the original value when the target zone would require a year
|
||||
outside `0000`–`9999`; generated writes refuse a clock outside that format.
|
||||
Capture-state start ordering uses the same exact recorded timestamp facts,
|
||||
including minute-only times and fractions longer than nine digits. Recorded
|
||||
spellings may use `T` or a space, `Z` or an extended/compact offset, or no zone
|
||||
(interpreted as UTC). Equal starts use ordinal order; a missing or unreadable
|
||||
start makes the capture use record order. Request interval and overlap checks
|
||||
still use millisecond coordinates, and generated-clock progression compares
|
||||
milliseconds. Derived ISO clock formats still require seconds and at most nine
|
||||
fractional digits.
|
||||
Timestamp ordering retains submillisecond precision when the source records it.
|
||||
|
||||
Comparison keeps response representations distinct: JSON values compare structurally, text compares as text, and a
|
||||
retained base64 body compares by bytes (including with UTF-8 text carrying those same bytes), never by its base64
|
||||
@@ -1149,12 +768,7 @@ global conflict involving an equal pair is not used to normalize either pair; th
|
||||
|
||||
A step agrees when the statuses and the normalised bodies are equal. The report gives agreement overall, per probe and
|
||||
per tool, `agreement_ignoring_order` for steps that differ only in item order, and each step's differences, at most 20,
|
||||
with a JSON pointer and both values cut short. Each step reports `differences_total` before that cap,
|
||||
`differences_omitted`, and `differences_truncated` (true when entries were omitted). Agreement and categories use the
|
||||
complete comparison. Omitted entries are not retained, so `--full-output` cannot recover them; inspect the original
|
||||
capture and probe Evidence to investigate beyond the returned pointers.
|
||||
|
||||
The difference categories are:
|
||||
with a JSON pointer and both values cut short:
|
||||
|
||||
| Category | Meaning |
|
||||
| --------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
@@ -1171,16 +785,16 @@ The difference categories are:
|
||||
| `unmapped-id` | The application answered with success and the environment did not, for a path id the comparison never mapped |
|
||||
|
||||
What a probe did not observe is not counted as the application's behaviour. A record without a usable request line,
|
||||
such as a redacted URL or a transaction the capture cannot attribute, is not replayed, nor is a request to an origin different
|
||||
from the application's (`other-origin`: the probe's origin with the most JSON-like records, as the manifest
|
||||
such as a redacted URL or a transaction the capture cannot attribute, is not replayed, nor is a request to another
|
||||
origin than the application's (`other-origin`: the probe's origin with the most JSON-like records, as the manifest
|
||||
chooses its own), and a request with no recorded
|
||||
answer is made, since it may have changed the application, but not compared: each probe lists them under `unobserved` with the reason, and the summary counts them. A live
|
||||
body the probe did not record compares by status only and is counted as `unobserved_bodies`, unless the answer never
|
||||
carries one (204, 205, 304, `HEAD`). A browser probe captured without `capture.network.response_body` is refused, as
|
||||
is one that makes POST, PUT or PATCH calls without `capture.network.request_body`.
|
||||
|
||||
Agreement covers only the calls the probes make, after the mapping the answers allow. Clocks compared by format
|
||||
and fields normalized under explicit generation rules can still differ in content, and an equal answer after a write shows the state matched for that
|
||||
Agreement covers only the calls the probes make, after the mapping the answers allow. Values compared by form, clocks
|
||||
and generated values, can still differ in content, and an equal answer after a write shows the state matched for that
|
||||
read, not that the write had no other effect.
|
||||
|
||||
## Compatibility verification
|
||||
|
||||
+9
-380
@@ -30,115 +30,10 @@ code units; follow `coverage.next_offset`. The page preserves `source_status` an
|
||||
`reported_total_bytes`: exhausting retained text does not make a partial decoded
|
||||
manifest complete. Complete source Evidence remains exportable under the existing
|
||||
session lifetime. Servers without retention reject summary before starting JADX.
|
||||
One-shot CLI output remains complete by default. Select `--detail summary` with
|
||||
`--evidence_output /local/capture.json` to retain complete Evidence before returning
|
||||
a linked view. Older servers require the existing complete request after schema discovery. APK authenticity and runtime behavior remain
|
||||
One-shot CLI output remains complete. Older servers require the existing complete
|
||||
request after schema discovery. APK authenticity and runtime behavior remain
|
||||
unverified.
|
||||
|
||||
## Manifest component projection
|
||||
|
||||
`inspect_android_package` also returns a static inventory of declared
|
||||
activities, aliases, services, receivers and providers from the decoded
|
||||
manifest. The binary XML decoder preserves component class spelling, including
|
||||
`$` in nested classes, without applying JADX class aliases. `declared_name`
|
||||
preserves the decoded source attribute; `name` resolves a
|
||||
relative name only when the manifest package is known and the resolved copy
|
||||
fits the UTF-8 byte budget. An unresolved package or exhausted name budget
|
||||
retains the declared name and marks the inventory partial. Empty component
|
||||
names and names with forbidden DEX characters retain their raw spelling and
|
||||
are not treated as identities. Valid DEX spaces, including boundary spaces,
|
||||
are preserved when resolving component names and matching alias targets.
|
||||
Whitespace in the manifest package is rejected by its separate package grammar.
|
||||
Component permissions inherit the application
|
||||
permission when no component permission is declared, except an activity alias
|
||||
with no permission: Android documents that case as requiring no permission
|
||||
through the alias. `permission_source` distinguishes a local declaration,
|
||||
inherited value, no requirement (including an explicit empty override), and a
|
||||
value omitted because its resource/manifest-placeholder token is unresolved
|
||||
or the total permission-byte budget was reached. Resource and placeholder
|
||||
references are not expanded at this parser boundary. Repeated inherited
|
||||
values share one inventory-wide byte budget.
|
||||
Provider `android:readPermission` and `android:writePermission` overrides are
|
||||
not projected as separate fields; their presence marks the inventory partial
|
||||
and is called out in `limitations`. Relative component names are not resolved
|
||||
against a manifest package with surrounding whitespace.
|
||||
Class-name segments follow DEX `SimpleName` characters, including a leading
|
||||
digit or hyphen, rather than Java source identifier rules. Relative-name
|
||||
expansion does not prove class existence, DEX-version compatibility or runtime
|
||||
loadability. SDK defaults accept JADX's decimal and hexadecimal integer
|
||||
spellings; the original SDK declaration strings remain unchanged.
|
||||
|
||||
`exported` is the declared literal `true` or `false`, otherwise null.
|
||||
`exported_default` is the documented default only where it can be established;
|
||||
it remains null for unknown provider target SDKs, nonliteral declarations, or
|
||||
components with filters targeting API 31 or later that omit the now-required
|
||||
explicit declaration. A limitation records the latter validation error.
|
||||
Provider authorities and all inventory dimensions are bounded. `status:
|
||||
"partial"` and `limitations` identify omitted declarations or values; a
|
||||
capacity limitation appears only when an actual collection limit was reached.
|
||||
The projection does not merge manifests, expand resource references, or
|
||||
observe runtime enabled/export state.
|
||||
|
||||
`enabled` and `grant_uri_permissions` report only literal component-level
|
||||
`android:enabled` and provider-level `android:grantUriPermissions` declarations.
|
||||
They are `null` when absent or not a literal boolean; missing attributes are not
|
||||
filled from application defaults. A nonliteral declaration gets a limitation.
|
||||
`process` preserves the exact component-level `android:process` spelling (including
|
||||
empty or symbolic values), or `null` when absent or omitted under the inventory-wide
|
||||
1 MiB copy budget. Neither field resolves resources, merges manifest values or
|
||||
establishes the effective runtime process or enabled state. The exact boolean
|
||||
attributes are retained in `enabled_declaration` and
|
||||
`grant_uri_permissions_declaration`, including resource references.
|
||||
|
||||
`intent_filters` preserves each filter's source ordinal, exact priority spelling,
|
||||
actions, categories and individual data declarations (scheme, host, port, path,
|
||||
path prefix/pattern/advanced pattern/suffix and MIME type). These are declarations,
|
||||
not computed Android matching rules. `meta_data` lists names; values remain in the
|
||||
complete decoded manifest. Component, filter, data and metadata `location` values
|
||||
use one-based lines/columns and UTF-16 offsets into that manifest. A component's
|
||||
`declaration_index` preserves its original declaration ordinal, including gaps.
|
||||
|
||||
`declaration_coverage` distinguishes complete, partial and unknown observations.
|
||||
Historical Evidence omits newly introduced fields and defaults coverage to
|
||||
`unknown`; omission never establishes absence. Current records contain the fields,
|
||||
with null for absent or unresolved scalar declarations. The inventory retains at
|
||||
most 32 filters and 64 nested action/category/data/metadata records per component,
|
||||
and shares a 1 MiB structured-value copy budget across components. Omitted values
|
||||
or unsupported filter constructs mark coverage partial and include a limitation;
|
||||
complete retained manifest text remains accessible.
|
||||
|
||||
For a small answer, select `detail: "summary"` with an explicit view:
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": "android",
|
||||
"facet": "components",
|
||||
"actions": ["android.provider.Telephony.SMS_RECEIVED"],
|
||||
"fields": ["name", "permission", "enabled", "process", "intent_filters"],
|
||||
"offset": 0,
|
||||
"limit": 16
|
||||
}
|
||||
```
|
||||
|
||||
The same JSON works with CLI `--view` or retained `inspect_analysis_view`.
|
||||
`kinds`, `names` and `actions` use exact matching; names accept either the recorded
|
||||
declared or resolved name, actions match any requested action, and different
|
||||
filters combine with AND. `exported` matches only explicit declarations, keeping
|
||||
`exported_default` separate. Empty selectors match nothing. Every selected row
|
||||
retains its parent-array `source_index` and `declaration_coverage`; source coverage,
|
||||
parent identity and limitations remain alongside the page. Follow
|
||||
`coverage.next_offset`. The 1 MiB projection budget can shorten a page; a single
|
||||
oversized row requires fewer selected fields. Exhausting a partial source does
|
||||
not establish complete manifest coverage. `facet: "component-summary"` applies
|
||||
the same filters and counts kinds, declaration coverage and explicit/default
|
||||
exported values separately. Both adapters use the same retained projection;
|
||||
repeated pages do not reload JADX.
|
||||
|
||||
The service `foreground_service_type` field preserves the declared attribute,
|
||||
including an empty value or surrounding spaces; `null` means the attribute is
|
||||
absent or the component is not a service. This field does not validate service
|
||||
type tokens or establish runtime eligibility.
|
||||
|
||||
## Supply tools
|
||||
|
||||
Use a **full JDK 17 or newer** (including its compiler module) and
|
||||
@@ -252,50 +147,6 @@ queries accept the same `overload_index`; the bridge resolves the selected
|
||||
method node and the result reports its index and displayed signature. These
|
||||
indices are tied to the current artifact and engine, not stable across builds.
|
||||
|
||||
Method reference queries also accept an external class, such as
|
||||
`android.app.NotificationManager --method-name notify`, when the APK directly
|
||||
invokes that exact DEX class and method name. If multiple prototypes are observed,
|
||||
the input error lists their exact DEX descriptors and local overload indices;
|
||||
repeat with an explicit `--overload-index`. These are the prototypes observed in
|
||||
the APK, not the complete method inventory of an Android framework or library.
|
||||
|
||||
Such results report `target_definition: "external"` and
|
||||
`reference_scope: "direct_dex_invokes"`. Each using method has `source_selector`
|
||||
with `class_name`, `method_name` and `overload_index`; pass those fields with the
|
||||
same APK path to `inspect_android_method` to inspect the caller. Repeated direct
|
||||
invokes in one method remain one using node with multiple instruction offsets.
|
||||
This scans raw invoke instructions in JADX's admitted APK methods without
|
||||
generating their source. It does not resolve reflection, JNI, method handles,
|
||||
inherited method lookup or possible runtime dispatch. Class-only queries and
|
||||
class/method inspection still require an APK definition; an external target has
|
||||
no invented body. Unknown class/method names remain input errors.
|
||||
|
||||
External prototype and caller metadata use a bridge-frame-derived representation
|
||||
allowance that reserves space for retained raw calls and CLI/MCP copies. Exhausting
|
||||
it returns a typed `resource_constraint`, rather than a truncated caller list
|
||||
reported as complete. Direct instruction offsets have their separate explicit
|
||||
`direct_invokes_exhaustive` coverage. Older retained Evidence may omit the new
|
||||
definition, scope and caller-selector fields; omission does not establish them.
|
||||
The bridge checks prototype expansion before copying descriptors or joining
|
||||
displayed signatures. It reuses one bounded prototype inventory for the exact
|
||||
class/method selection in the current APK session; replacing the session or
|
||||
selection invalidates that retained result.
|
||||
|
||||
`inspect_android_class` also reports `dex_static_value` for each `static final`
|
||||
field whose class definition encodes a value in its DEX static values, so
|
||||
compile-time constants are readable even when the class has no `<clinit>`.
|
||||
Each value is typed: `boolean`, `byte`, `short`, `char` (UTF-16 code unit) and
|
||||
`int` values are JSON booleans or numbers, while `long`, `float` and `double`
|
||||
values are strings holding Java's text for the value, so 64-bit values stay
|
||||
exact. The field reports `null` when no value is encoded, the encoded value is
|
||||
null or of another kind, or the field is not `static final`. The value is read from parsed
|
||||
metadata without generating source, and a static initializer can still assign
|
||||
the field; use `inspect_android_method` on `<clinit>` for that evidence.
|
||||
Only `inspect_android_class` asks the bridge for these values.
|
||||
`inspect_android_method` and `trace_android_references` read the member-only
|
||||
class summary, so large encoded strings cannot push their member lookup past
|
||||
the 8 MiB protocol frame limit.
|
||||
|
||||
Smali fallbacks select one complete block using the original method name and DEX
|
||||
descriptor, before applying the source byte limit. This preserves the selected
|
||||
overload even when Java code generation renames a method. If the selected Java
|
||||
@@ -317,173 +168,11 @@ instructions into field declarations, leaving an empty or incomplete method
|
||||
body. The selected method's disassembly preserves those instructions and retains
|
||||
the same source byte limit and explicit truncation reporting.
|
||||
|
||||
## Apply an R8/ProGuard mapping
|
||||
|
||||
Release builds shrunk by R8 or ProGuard rename classes and members. Select the
|
||||
build's `mapping.txt` with `mapping_path` on `inspect_android_class` or
|
||||
`inspect_android_method`, or `--mapping-path` on the matching CLI command:
|
||||
|
||||
```sh
|
||||
rea inspect-android-class /path/Example.apk a.b --mapping-path /path/mapping.txt \
|
||||
--detail summary --evidence_output /tmp/class-evidence.json
|
||||
```
|
||||
|
||||
For agent workflows, select `detail: "summary"` on class and method inspection.
|
||||
It preserves the complete normalized member inventory or selected method body,
|
||||
including mapping annotations, while retaining duplicate raw provider calls in
|
||||
the complete parent Evidence. The reply links that parent's identity and digest;
|
||||
export it with `export_evidence_bundle` or read its summary again through
|
||||
`inspect_analysis_view`. CLI summaries require `--evidence_output` and return
|
||||
the portable bundle path. Omitted or `complete` detail preserves complete inline
|
||||
Evidence. Search and reference operations retain their existing contracts.
|
||||
Symbol summary views admit at most **1 MiB**, including view context. An admitted
|
||||
method's source can already consume that amount; a larger combined summary
|
||||
fails explicitly instead of silently truncating facts. MCP refusal includes
|
||||
the retained parent ID. Select complete detail, or inspect one method rather
|
||||
than a whole class, when the requested summary does not fit.
|
||||
|
||||
The mapping path must be absolute. Selectors still take the observed names, such as
|
||||
`a.b`, and observed rows are unchanged. The result adds one `mapping` block with
|
||||
the file's path, SHA-256 and size, the header values the file reports
|
||||
(`compiler`, `compiler_version`, `min_api`, `map_version`, `pg_map_id`,
|
||||
`pg_map_hash`), valid record counts and up to 16 malformed line samples. Its
|
||||
`symbols` list has one entry per observed class, inner class, method and field;
|
||||
`inspect_android_method` lists the class and the selected method. `coverage`
|
||||
counts each resolution status. With summary delivery, these unchanged facts are
|
||||
under the view's `summary` field; the view separately reports source coverage.
|
||||
|
||||
Lookups use the bridge's raw DEX names, not JADX display aliases such as a
|
||||
renamed reserved word. Each symbol resolves as:
|
||||
|
||||
- `mapped`: exactly one record matches. A class needs one record for its raw
|
||||
name (`verified_by: "class"`). A method must also match its DEX prototype
|
||||
(`"descriptor"`) and a field its DEX type (`"type"`). R8 residual signature
|
||||
metadata is used when present; otherwise original types are translated
|
||||
through the file's class records.
|
||||
- `ambiguous`: `multiple-records`, `descriptor-mismatch`, `type-mismatch`,
|
||||
`ambiguous-class` or `residual-signature-conflict`, with `candidate_count` and
|
||||
up to 8 candidates. Conflicting residual metadata never selects a winner. A name match
|
||||
whose prototype or type does not verify is never reported as mapped.
|
||||
- `unmapped`: no valid record matched the raw identity.
|
||||
- `unknown` with reason `projection-budget`: this claim did not fit the combined
|
||||
reply budget. The observed raw symbol remains present; `coverage.unknown`
|
||||
counts these omissions. Inspect that specific class or method separately,
|
||||
or consult the selected local mapping file for complete source records.
|
||||
|
||||
A line continues an inline frame stack when it repeats the previous line's
|
||||
minified range and obfuscated name and gives a single call-site line, such as
|
||||
`:7`. Only the outermost original of a stack becomes a candidate. Original
|
||||
line numbers and source files are not projected. Malformed records are counted
|
||||
and skipped, so an `unmapped` symbol might be described by one of them. A file
|
||||
that is missing, not a regular file, not UTF-8, larger than the byte limit or
|
||||
without any class record, such as `usage.txt` or `seeds.txt`, is refused before
|
||||
JADX is used. Requests read their mappings one at a time, in the same queue as
|
||||
the JADX work and under the same operation deadline. A mapping failure therefore never starts or retires the JADX
|
||||
session, and a different mapping reuses the loaded APK.
|
||||
|
||||
## Link native declarations to JNI candidates
|
||||
|
||||
When advertised, `inspect_android_jni_bindings` links original `ACC_NATIVE`
|
||||
declarations from standard root `classes*.dex` members to bundled ELF libraries.
|
||||
The supplied JAR's DEX reader preserves duplicate declarations before decompiler
|
||||
deduplication. This operation does not load the APK into the decompiler.
|
||||
MCP defaults to retained summary delivery when `detail` is omitted; select
|
||||
`complete` explicitly to receive the full report inline.
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/absolute/Example.apk",
|
||||
"class_name": "org.conscrypt.NativeCrypto",
|
||||
"method_names": ["EVP_PKEY_new_RSA"],
|
||||
"abis": ["arm64-v8a"],
|
||||
"detail": "summary"
|
||||
}
|
||||
```
|
||||
|
||||
Class/method names and optional `descriptors` select raw DEX identities.
|
||||
`library_paths` selects exact logical APK paths such as
|
||||
`lib/arm64-v8a/libconscrypt_jni.so`; each library's `source_path` preserves its
|
||||
original ZIP spelling. Omitted filters include all scoped facts; `[]` matches
|
||||
none. Class selection never narrows the global inventory used for table-class
|
||||
attribution.
|
||||
|
||||
Independent `declarations`, `libraries`, `candidates` and `bindings` collections
|
||||
have stable IDs and source ordinals. Candidates retain APK/library digests,
|
||||
ABI, function addresses and file offsets. Table candidates also retain section,
|
||||
table and entry identity plus all three name/signature/function pointer slots
|
||||
and their relocation sources. JNI exports follow the
|
||||
[UTF-16 escaping and short-before-long rules](https://docs.oracle.com/en/java/javase/21/docs/specs/jni/design.html#resolving-native-method-names);
|
||||
failed escaping does not acquire an invented export match. Tables require
|
||||
NUL-terminated [Modified UTF-8 strings](https://docs.oracle.com/en/java/javase/21/docs/specs/jni/types.html#modified-utf-8-strings),
|
||||
valid descriptors and executable load mappings. REL, RELA and RELR retain
|
||||
separate provenance; unsupported packed relocations and unresolved pointer
|
||||
spans stay explicit unknowns.
|
||||
|
||||
A static `JNINativeMethod` entry names no class. Attribute it uniquely only
|
||||
when exactly one declaration in the complete recorded native inventory has
|
||||
that `(name, descriptor)` pair. Multiple classes remain ambiguous after a
|
||||
class filter; a partial inventory cannot establish unique attribution.
|
||||
|
||||
Each selected declaration has a row per selected/observed ABI. `matched` means
|
||||
candidate presence was observed; `ambiguous` means multiple observed targets
|
||||
or class associations remain. Binding `coverage` is separate: a matched
|
||||
partial row does not establish complete uniqueness, and an empty partial scan
|
||||
is `unknown`, never `unmatched`. ELF/ABI label mismatches preserve facts without
|
||||
asserting a binding. No result establishes library loading, runtime registration
|
||||
or method invocation.
|
||||
|
||||
Summary returns the first 64 selected binding rows, their declaration/candidate
|
||||
closure and every selected library's identity, coverage and failure reason;
|
||||
counts and page coverage describe the full selected binding set. Selected
|
||||
declaration IDs have separate preview coverage, so native declarations remain
|
||||
visible even when the APK has no bundled ABI libraries. Complete
|
||||
Evidence stays retained. Read exact fields or further pages without reopening
|
||||
files or rerunning analysis:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": { "kind": "retained-evidence", "evidence_id": "ev_<digest>" },
|
||||
"view": {
|
||||
"kind": "android-jni",
|
||||
"collection": "bindings",
|
||||
"offset": 64,
|
||||
"limit": 64,
|
||||
"fields": ["declaration_id", "abi", "status", "coverage", "candidate_ids"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Collections also include `declarations`, `candidates` and `libraries`, with
|
||||
exact filters and batch lookup by declaration/candidate/library IDs. `row_index`
|
||||
is the original parent collection index; DEX and
|
||||
ZIP source ordinals stay separate. Complete detail returns the full report.
|
||||
CLI uses the same selection and delivery contracts. Its common one-shot
|
||||
default remains complete; summary delivery requires an explicit Evidence path:
|
||||
|
||||
```sh
|
||||
rea inspect-android-jni-bindings /absolute/Example.apk \
|
||||
--query '{"class_name":"org.conscrypt.NativeCrypto","method_names":["EVP_PKEY_new_RSA"],"abis":["arm64-v8a"]}' \
|
||||
--detail summary --evidence_output /absolute/fresh-jni-evidence.json
|
||||
```
|
||||
|
||||
Optional `--view` accepts the same JSON page and requires summary delivery.
|
||||
Evidence bundles never overwrite files. Complete declarations and one library
|
||||
selection are cached within the immutable APK session. Changing ABI/library
|
||||
selection replaces the scan; class/method selection reuses it. Fresh analysis
|
||||
still validates current caller APK bytes. Idle/close releases the caches;
|
||||
retained Evidence remains historical and independently readable.
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- Package inspection retains decoded manifest XML, summary declarations,
|
||||
permissions and class/resource counts. REA's bridge selects the exact root
|
||||
`AndroidManifest.xml` resource by original archive name and manifest type,
|
||||
independently of asset order. One namespace-aware JDK XML parse supplies
|
||||
summary attributes and direct permission/SDK declarations; namespace prefixes
|
||||
and XML entity spelling do not change their values. Raw decoded XML is retained
|
||||
exactly and only its displayed text is subject to the existing source byte cap.
|
||||
Summaries use the complete selected XML before that cap. Document types are
|
||||
unsupported; parsing does not load external DTDs, schemas, or XInclude files.
|
||||
permissions and class/resource counts. Summary fields are extracted by the
|
||||
upstream engine and should be checked against the retained XML when needed.
|
||||
Signature verification is explicitly `not_performed`.
|
||||
- Class search filters the engine's cached top-level names before transferring
|
||||
pages, then follows contiguous inventory positions without rescanning earlier
|
||||
@@ -498,22 +187,10 @@ retained Evidence remains historical and independently readable.
|
||||
full reported display types, fields and inner class names. Synthetic members
|
||||
that code generation hides can appear in the inventory; it stays consistent
|
||||
after method decompilation within the retained session.
|
||||
Class and method inspection and reference tracing accept either the exact
|
||||
reported class name or its original binary name, including `$` in nested and
|
||||
anonymous classes. The bridge keeps both identities from JADX; it does not
|
||||
replace separators or guess suffix matches. Ambiguous selectors are refused.
|
||||
Follow-up calls preserve the selected name, while results report the frozen
|
||||
JADX display name. Quote `$` names in the shell, for example
|
||||
`'com.example.Manager$WorkerThread'`.
|
||||
Exact DEX prototypes are reported from frozen JADX method metadata as
|
||||
`dex_descriptor`, for example `(I)V`. Display names can be JADX aliases;
|
||||
the descriptor alone is not a complete raw DEX method ID. Overload indices
|
||||
are tied to this artifact and engine, not stable identities across builds.
|
||||
- A `mapping` block appears only when the caller selected `mapping_path`. Its
|
||||
names are claims from that file, not observations of the APK. REA does not
|
||||
check that the mapping belongs to this build; `pg_map_id` is reported, not
|
||||
compared. Evidence keeps the selected path in its parameters and the file's
|
||||
SHA-256 in the result.
|
||||
- Method inspection reports Java or smali, fallback markers and complete/partial
|
||||
returned text. Decompiled source is a derived representation. A complete text
|
||||
result does not establish complete semantics or a byte-identical reconstruction.
|
||||
@@ -579,10 +256,6 @@ such as
|
||||
`libreactnativejni.so`, `libreactnative.so`, `libhermes.so`,
|
||||
`libhermes_executor.so`, `libflutter.so`, and `libunity.so`; unrelated names
|
||||
such as `libcommunity.so` and `libreactivity.so` do not identify those families.
|
||||
The signing list retains the `META-INF/MANIFEST.MF`, `.SF`, `.RSA`, `.DSA`, and
|
||||
`.EC` paths selected by Android's v1 verifier, including nested paths. These are
|
||||
inventory metadata candidates, not proof that a v1 signature verifies or that
|
||||
the APK is signed; use `inspect-apk-signature` for signature analysis.
|
||||
The bridge basis is inferred from the native path and repeated for each managed
|
||||
component, so it does not identify a specific JNI binding. Bridge hypotheses form a DEX/JVM-class by native-library
|
||||
Cartesian product. The projection computes the product and exact serialized
|
||||
@@ -603,19 +276,6 @@ target/configuration changes, failure, timeout, cancellation, MCP disconnect or
|
||||
server shutdown. One-shot CLI commands always join cleanup before returning.
|
||||
REA never writes to the original APK or launches target code.
|
||||
|
||||
JNI scans reuse the analyzer process without loading the APK into the
|
||||
decompiler. They admit at most 64 MiB per DEX member and 256 MiB of DEX bytes,
|
||||
with a 4 MiB native-declaration JSON allowance before MCP string escaping.
|
||||
Native libraries are processed sequentially: one 64 MiB buffer, 256 MiB total
|
||||
selected read work, and a 16 MiB parsed-library-facts allowance. ZIP inventory
|
||||
and library counts also have explicit resource bounds. Unexamined occurrences
|
||||
remain unknown. A complete relationship report has a separate 16 MiB serialized
|
||||
allowance; summaries/pages allow 1 MiB including their provenance envelope.
|
||||
These are resource admissions, not claims about maximum valid Android formats.
|
||||
Select fewer ABI/library occurrences or smaller retained pages when needed.
|
||||
Completed parsed library facts survive failed reader finalization in typed
|
||||
partial observations; the exact reader owner remains available for cleanup retry.
|
||||
|
||||
On Windows, stdin writes preserve stream backpressure and run outside the Node
|
||||
event loop. Cancellation terminates the retained job, including descendants,
|
||||
before workspace cleanup. On Windows, a detached REA helper owns the private
|
||||
@@ -676,17 +336,11 @@ export REA_JADX_OPERATION_TIMEOUT_MS=600000
|
||||
|
||||
The value must be a decimal millisecond count from 1 to 2147483647; anything
|
||||
else is refused before Java starts. REA reads it from the environment captured
|
||||
at process start. Changing that environment requires a new REA process. On a
|
||||
running MCP server, set `jadx.operation_timeout_ms` through
|
||||
`update_analysis_settings` to change the default for newly accepted operations;
|
||||
queued or running operations retain their accepted budget. The same deadline
|
||||
bounds each upstream call. Timeout recovery reports the effective milliseconds
|
||||
and range, with `details.limit` identifying the runtime setting. An invalid
|
||||
provider-local launch value can also be repaired by a runtime override; reset
|
||||
exposes that invalid launch value again. See the
|
||||
[settings design](design/limits-and-recovery.md#invalid-launch-configuration).
|
||||
A passing JADX configuration check in `rea doctor` reports its launch-derived
|
||||
`operation_timeout_ms`; it does not inspect another running server's overrides. Explicit
|
||||
at process start. A change applies on the next process start; an already
|
||||
running server keeps the deadline it started with. The same deadline bounds
|
||||
each upstream call. A timeout names `REA_JADX_OPERATION_TIMEOUT_MS`, the
|
||||
effective milliseconds, and that range. A passing JADX configuration check in
|
||||
`rea doctor` reports the effective `operation_timeout_ms`. Explicit
|
||||
cancellation still ends the operation immediately, and partial observations
|
||||
and owned-process cleanup are unchanged. Raising the deadline does not extend
|
||||
the bridge's 30-second startup handshake or the upstream **90-second**
|
||||
@@ -701,27 +355,6 @@ complete, case-sensitive search contract. A protocol frame above **8 MiB**, or c
|
||||
**32 MiB per operation**, fails with a diagnostic instead of returning an incomplete inventory
|
||||
as complete. These are memory-safety boundaries, not undocumented result caps.
|
||||
|
||||
A selected R8/ProGuard mapping is re-read and SHA-256 checked on every call, up
|
||||
to **64 MiB**. The serialized provider keeps one successful parsed index, keyed
|
||||
by byte length and digest with fixed parser options. Identical bytes reuse that
|
||||
index; changed bytes are reparsed, missing files still fail, and each response
|
||||
preserves the currently selected path. The index is released on provider idle
|
||||
retirement or close. Omitting `mapping_path` never applies the cached mapping.
|
||||
During replacement, the old index can coexist with the new parse until it
|
||||
succeeds; these transient allocations count toward process memory.
|
||||
|
||||
Before retaining records, the parser admits an estimated **128 MiB** index
|
||||
representation and **8 MiB** of expanded mapped claims per residual class,
|
||||
including shared header facts. The index allowance accounts for strings and
|
||||
containers; it is not a measured V8 heap cap. Parameter and metadata expansion
|
||||
is admitted before splitting or JSON decoding. These fixed resource bounds
|
||||
have no runtime override; refusal reports `resource_constraint`, the source
|
||||
line and mapping SHA-256. Ambiguous lookups count all candidates while keeping
|
||||
only the first eight in stable file order. A separate **8 MiB** combined claim
|
||||
budget applies to the actual query set, including outer and inner classes,
|
||||
before cloning the report. Claims that do not fit become explicit `unknown`
|
||||
entries; other admitted claims and all observed identities remain available.
|
||||
|
||||
## Real verification
|
||||
|
||||
Download the explicit public fixtures once; this installs no tools:
|
||||
@@ -765,7 +398,3 @@ containing node's unknown `dex_offset`. `direct_invokes_exhaustive: false`
|
||||
means instruction rows were bounded by the local bridge frame budget; `null`
|
||||
means the producer did not establish instruction coverage. Using-node totals
|
||||
and coverage do not imply complete instruction evidence.
|
||||
`direct_invokes_coverage` reports counts of using nodes with complete, partial,
|
||||
or unknown positions, plus the number of retained instruction sites. This
|
||||
aggregate makes position truncation visible without scanning every caller;
|
||||
older retained Evidence may omit it, which means the aggregate was unexamined.
|
||||
+21
-55
@@ -102,11 +102,7 @@ share the application workflow, projection, retention and publication owners.
|
||||
`SignerInfo`'s certificate and digest/signature algorithm OIDs. The PKCS#7
|
||||
`SignedData` envelope is read as BER, including constructed signature OCTET
|
||||
STRINGs; each embedded certificate is still interpreted as DER by the X.509
|
||||
owner. Issuer matching preserves the encoded Name and also admits equivalent
|
||||
ASCII PrintableString/UTF8String values for CN, C, L, ST, O and OU: case and
|
||||
repeated/edge spaces are normalized, RDN sequence order is preserved, and
|
||||
attribute order inside one multivalued RDN is ignored. Other name forms
|
||||
require exact encoded identity; semantic equivalence remains unestablished.
|
||||
owner.
|
||||
`certificate` is `null` when the block has more than one `SignerInfo` (with
|
||||
nothing verified there is no single signer) or names a certificate it does
|
||||
not carry. A block that cannot be read (malformed ASN.1, an unsupported
|
||||
@@ -148,10 +144,7 @@ share the application workflow, projection, retention and publication owners.
|
||||
- **Per signer:** the certificate's **SHA-256 fingerprint over the raw embedded
|
||||
DER** (the same value `apksigner --print-certs` prints), plus subject, issuer,
|
||||
serial, ISO 8601 validity and public-key algorithm when the X.509 parser can
|
||||
read the exact carried DER (each is `null` otherwise). The parser's returned
|
||||
DER must equal those bytes before readable facts are combined with their
|
||||
fingerprint; embedded PEM text and trailing data cannot select a different
|
||||
certificate. Raw signature algorithm IDs, declared
|
||||
read them (each is `null` otherwise); raw signature algorithm IDs, declared
|
||||
digest hex and digest algorithms, signature lengths and public-key length; and for
|
||||
v3/v3.1 the signer's declared min/max SDK.
|
||||
- **Key rotation (v3/v3.1):** the declared proof-of-rotation lineage, oldest
|
||||
@@ -179,31 +172,26 @@ share the application workflow, projection, retention and publication owners.
|
||||
- `schemes_agree_on_signer` (observed): every present block scheme presents
|
||||
the same _set_ of signer certificates. An extra signer in one scheme is
|
||||
disagreement; a signer without a certificate makes it `null` (unknown).
|
||||
A single present block scheme with known signers agrees with itself;
|
||||
v1 is compared separately below.
|
||||
Build-tools 33+ rotate through v3.1 by default, so a rotated app usually
|
||||
reads `false` here and `true` below.
|
||||
- `signer_change_declared_by_rotation` (declared): the signers differ, but
|
||||
each appears in a lineage the APK declares. Read the two together: block
|
||||
schemes with different signers covered by a declared lineage show `false`
|
||||
/ `true`; differing signers without such a lineage show `false` / `false`.
|
||||
A lineage alone does not establish a change between block schemes. It is
|
||||
`null` when a signer's identity is unknown, or when the signers differ and
|
||||
a declared lineage could not be decoded.
|
||||
each appears in a lineage the APK declares. Read the two together: a
|
||||
rotated APK shows `false` / `true`; signers that differ with no declared
|
||||
lineage containing them show `false` / `false`. It is `null` when a
|
||||
signer's identity is unknown, or when the signers differ and a declared
|
||||
lineage could not be decoded.
|
||||
- `v1_signers_match_block_signers` (observed): the v1 signer certificates
|
||||
equal what the lowest block scheme that has signers (v2, else v3, else
|
||||
v3.1) expects v1 to carry. That is its signer set, except that a v3/v3.1
|
||||
signer declaring a rotation lineage expects the lineage's first
|
||||
(original) certificate, as apksig requires. `false` is notable: the JAR
|
||||
signature names a different key than the block signature. When v1 uses
|
||||
the lineage's original key and v3 its newer key, this comparison reads
|
||||
`true`. If v3 is the only present block scheme, agreement is `true` and
|
||||
declared inter-block signer change is `false` even though rotation is
|
||||
present. If v2 also uses the original key, agreement is `false` and the
|
||||
declared inter-block change is `true`. `null` when
|
||||
v1 or a populated block scheme is absent, any signer's identity is unknown (a
|
||||
signature names a different key than the block signature. A key rotation
|
||||
keeps v1 (and v2) on the original key, so it reads `true` while
|
||||
`schemes_agree_on_signer` reads `false`, with or without v2. `null` when
|
||||
v1 or a block scheme is absent, any signer's identity is unknown (a
|
||||
`decode_error`, a `null` certificate or a `subjectKeyIdentifier`
|
||||
`SignerInfo`), or the lineage's original certificate cannot be
|
||||
identified. When identities or a lineage origin are unknown, limitations
|
||||
explain why; absence is shown in the scheme records.
|
||||
identified; a limitation then says why.
|
||||
- `strongest_scheme`, `rotation_present`, `source_stamp_present`,
|
||||
`self_consistent`.
|
||||
|
||||
@@ -226,9 +214,9 @@ Not yet read: v4 `.apk.idsig` (a sibling file), verity (`0x0421`/`0x0423`/`0x042
|
||||
## Bounds and errors
|
||||
|
||||
The archive is read once, bounded at 512 MiB, without following symlinks and
|
||||
with change detection. ZIP tail offsets, the central-directory size and signing
|
||||
block spans used by the reader are checked against their containing sections,
|
||||
including the complete central-directory extent before ZIP end records. A process-wide queue permits
|
||||
with change detection. Every size and offset in the ZIP tail and signing block
|
||||
is checked against its containing section before use, including the complete
|
||||
central-directory extent before ZIP end records. A process-wide queue permits
|
||||
one active whole-file inspection and eight pending scans across provider
|
||||
instances; pending scans retain no APK bytes and can be cancelled with their
|
||||
original reason. Further requests receive a capacity error. The active slot
|
||||
@@ -244,41 +232,19 @@ serialized size of the report together with its limitations (which repeat
|
||||
archive names) is then checked against an 8 MiB output budget (the exact check
|
||||
is authoritative); an over-budget report is refused, not truncated.
|
||||
|
||||
Signing blocks admit at most 4096 ID/value pairs; at most 4096 signing-relevant
|
||||
META-INF entries are retained. These fixed counts, signer/lineage counts and the
|
||||
report byte budget return nonretryable `resource_constraint` errors naming the
|
||||
effective allowance. This release has no runtime override for them. Waiting for
|
||||
an active scan cannot raise a fixed count, and complete CLI JSON cannot raise the
|
||||
report budget. Occupied scan-queue capacity remains retryable after a scan finishes.
|
||||
|
||||
The whole-file reader allocates the selected snapshot and bounded read scratch;
|
||||
total process memory exceeds the selected file's size. It also computes the complete
|
||||
The whole-file reader retains read chunks and a concatenated buffer; total
|
||||
process memory exceeds the selected file's size. It also computes the complete
|
||||
file SHA-256, so inspection is not header-only. Content digests are recomputed
|
||||
at most once per SHA-256/SHA-512 algorithm during one inspection. Input,
|
||||
inflation, decoded-record and serialized-output bounds keep work finite;
|
||||
resource refusals are distinct from malformed format errors.
|
||||
|
||||
If the stable read completes but closing its descriptor fails or is cancelled,
|
||||
the error retains `details.partial_observation` with the selected path, complete
|
||||
file SHA-256 and byte count. These facts describe the acquired file; they do not
|
||||
establish any decoded signing facts. Failed cleanup remains visible until the
|
||||
provider releases the descriptor on a cleanup retry.
|
||||
|
||||
A file that is not a ZIP is refused as an unsupported target. A damaged ZIP
|
||||
(local headers but no end-of-central-directory), a central directory that
|
||||
cannot be walked or whose declared record count does not account for all of
|
||||
its bytes, or a malformed signing block or signer is an input error naming the
|
||||
offset or scheme and reason. Zip64 tail damage includes a missing required
|
||||
locator, an impossible locator disk/count pair, a fixed end record that does
|
||||
not fit before its locator, or an incorrect end-record signature. These are
|
||||
input errors, not evidence of an unsupported format. Coherent multi-disk
|
||||
locator metadata and 64-bit offsets/counts/sizes beyond the reader's safe
|
||||
integer range remain unsupported. The locator offset is disk-relative, so
|
||||
multi-disk metadata is checked before applying local file bounds. This follows
|
||||
the record fields in [PKWARE APPNOTE 6.3.10, sections 4.3.14–4.3.15](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT);
|
||||
it does not establish full validation of Zip64 versions or extensible data.
|
||||
Signer framing includes signature records, the public-key field and matching
|
||||
signed/outer v3 SDK ranges. Suffix bytes beyond known
|
||||
scheme and reason. Signer framing includes signature records, the public-key
|
||||
field and matching signed/outer v3 SDK ranges. Suffix bytes beyond known
|
||||
fields are counted as uninterpreted, as Android readers may ignore extensions;
|
||||
their presence does not establish malformed signing.
|
||||
When the central directory is not immediately followed by
|
||||
|
||||
@@ -93,22 +93,9 @@ still suppresses all string projection, including in this mode.
|
||||
`target.bytes` bind the observation to the exact APK that was decoded.
|
||||
The Evidence subject carries the same snapshot digest and caller-selected
|
||||
path, including when the original file changes during decoding.
|
||||
A change during snapshot acquisition reports retryable `artifact_changed`;
|
||||
wait for a stable input before retrying. Host read permission failures report
|
||||
`access_denied`, while a missing or non-regular selected path reports
|
||||
`invalid_request` with the rejected `path` field.
|
||||
- **Metadata comes from apktool.yml as apktool printed it.** Version codes
|
||||
keep their own spelling (`'7'` versus `7`); `package_name` is read from
|
||||
the decoded manifest, not inferred.
|
||||
The `version: 2.9.0`-or-later header that Apktool 2.9+ and 3 emit selects
|
||||
their single-line string dialect: literal hashes, single quotes, empty
|
||||
strings, and the text `null` stay part of the value; producer backslashes
|
||||
and Unicode escapes decode once. Older/tagged documents retain YAML scalar
|
||||
semantics; double-quoted values also retain YAML string semantics.
|
||||
This distinction follows the released
|
||||
[2.9.0 metadata writer](https://github.com/iBotPeaches/Apktool/blob/b64e257c9b689eb9630ad0241f2fe4f4a749955b/brut.apktool/apktool-lib/src/main/java/brut/androlib/apk/VersionInfo.java#L38-L42),
|
||||
[3.0.3 metadata writer](https://github.com/iBotPeaches/Apktool/blob/18b5e99cb56ff9451e8aa55b065dcf5bbd616975/brut.apktool/apktool-lib/src/main/java/brut/androlib/meta/VersionInfo.java#L49-L55)
|
||||
and [string escape owner](https://github.com/iBotPeaches/Apktool/blob/18b5e99cb56ff9451e8aa55b065dcf5bbd616975/brut.j.yaml/src/main/java/brut/yaml/YamlStringEscapeUtils.java#L55-L116).
|
||||
- **The manifest is the decoded text**, bounded at 512 KiB and reported as
|
||||
truncated when larger.
|
||||
- **Strings are projected verbatim.** Escaped entities stay escaped, and
|
||||
@@ -183,8 +170,3 @@ exact facts/Evidence parity and bounded task costs are recorded in the
|
||||
[shared-decode and table-only checkpoint](https://github.com/morluto/rea/blob/42e1a17d6e6c0487f17be0bae248553c4caba7f7/experiments/analysis-efficiency/apktool-string-tables.md).
|
||||
This qualifies selected resource facts, not application execution or independent
|
||||
binary resource-decoder completeness.
|
||||
|
||||
If the selected APK changes while its private snapshot is copied, acquisition
|
||||
returns `artifact_changed`; request-schema correction cannot recover a stable
|
||||
snapshot. Launcher spawn permission denial is distinct from a missing launcher,
|
||||
and configuration diagnostics retain the selected command and system error.
|
||||
@@ -101,8 +101,8 @@ limitation reports how many inventoried entries were left out. IPA projection
|
||||
still lists every archive component, including `SwiftSupport/`.
|
||||
|
||||
`platforms` lists `macos` for application roots with `Contents/`. It lists `ios`
|
||||
for shallow application roots that have an `Info.plist` at their root. An
|
||||
application root with neither, such as an empty inventory, has no platform.
|
||||
for shallow application roots that have an `Info.plist`. An application root
|
||||
with neither, such as an empty inventory, has no platform.
|
||||
|
||||
## macOS bundle anatomy
|
||||
|
||||
@@ -149,11 +149,6 @@ directories rather than `Versions/Current`. A framework's `info_plist_path` is
|
||||
because `Versions/Current` decides which one applies) or when its single
|
||||
version directory has no `Resources/Info.plist`.
|
||||
|
||||
A `shallow` bundle's `info_plist_path` is its root `Info.plist`. Without one,
|
||||
it is `Resources/Info.plist`: a flattened macOS framework keeps its binary,
|
||||
`_CodeSignature/`, and `Resources/` directly in the `.framework`, like a single
|
||||
`Versions/<version>/` directory without the `Versions/` wrapper.
|
||||
|
||||
AppleDouble sidecar files (`._*` and `__MACOSX/`) are inventory facts, but they
|
||||
describe neighbouring files. They are excluded from roots, bundle roles, and the
|
||||
anatomy components.
|
||||
@@ -161,10 +156,6 @@ anatomy components.
|
||||
The projection does not read plists. Use `inspect_plist` on a bundle's
|
||||
`info_plist_path` or a launchd plist to read `CFBundleExecutable`, identifiers,
|
||||
`SMPrivilegedExecutables`, `BundleProgram`, URL schemes, and usage descriptions.
|
||||
MCP calls require an existing `open_binary` or `open_bundle_resources` session,
|
||||
including when `inspect_plist.path` is explicit. For a bundle without a resolved
|
||||
executable, open its resource session first. The CLI `rea inspect-plist <path>`
|
||||
opens the selected artifact for its one invocation.
|
||||
Use `inspect_signature` on an executable candidate for code-signing claims.
|
||||
|
||||
## Dylib load resolution
|
||||
@@ -218,9 +209,6 @@ Resolution follows dyld:
|
||||
already loaded image reuses it.
|
||||
- Symlinks such as `Versions/Current` are followed inside the analyzed root.
|
||||
An escape through `..` or an absolute link is `escapes-target`.
|
||||
- Dyld path components are POSIX names. On Windows hosts, link text uses `\`
|
||||
as its separator, and a dyld component that contains `\` or `:` cannot name
|
||||
a file in the analyzed root, so its candidate is `absent`.
|
||||
|
||||
Each edge lists every candidate path tried, with its outcome: `resolved`,
|
||||
`absent`, `not-mach-o`, `malformed`, `unsupported` (recognized Mach-O outside
|
||||
@@ -232,16 +220,9 @@ satisfy a dylib dependency. `platform-mismatch` means a matching-architecture
|
||||
dylib was found but its observed platform is incompatible with the process.
|
||||
These outcomes reject that candidate; dyld search may continue and resolve a
|
||||
later candidate. Candidate outcomes describe individual search paths, while
|
||||
`resolution.status` describes the final edge. For each selected process-root
|
||||
slice, candidate slices are matched by CPU type and dyld's subtype preference
|
||||
order, independently of FAT table order. An ARMv7s root slice prefers ARMv7s,
|
||||
then ARMv7; other known ARM32 subtypes use their compatibility rows from
|
||||
[Apple's dyld source](https://github.com/apple-oss-distributions/dyld/blob/dyld-852.2/src/dyld2.cpp#L2811-L3105).
|
||||
An `x86_64h` process also loads generic `x86_64`. Unknown ARM32 subtype policy
|
||||
remains undetermined when ARM32 candidate slices exist. This is a static model
|
||||
keyed to the selected root slice; it does not observe the target device's host
|
||||
CPU or prove which slice the device will execute. The edge's `resolution` is
|
||||
one of:
|
||||
`resolution.status` describes the final edge. Slices follow dyld's graded
|
||||
architectures, so an `x86_64h` process also loads `x86_64`. The edge's
|
||||
`resolution` is one of:
|
||||
|
||||
| Status | Meaning |
|
||||
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
@@ -254,7 +235,7 @@ A `conditional` image is still traced, because it is an in-root candidate
|
||||
that may load. Every edge below it has `loader_conditional: true`, and a
|
||||
request that reuses it by install name is `conditional` too. An in-root dylib
|
||||
with no platform load command remains unknown unless the root is an executable
|
||||
with observed macOS platform and the candidate's CPU type is x86_64, i386, or arm64. In that macOS context,
|
||||
with observed macOS platform and the candidate’s CPU type is x86_64, i386, or arm64. In that macOS context,
|
||||
[Apple's dyld](https://github.com/apple-oss-distributions/dyld/blob/fd8d0c4d52320ebf64db34f3cb280310d905c5ae/mach_o/UnsafeHeader.cpp#L1137-L1145) supplies an implicit macOS platform to legacy dylibs on those CPU
|
||||
types. The candidate records `platform_basis: "macos-dyld-implicit"`; the
|
||||
reported slice fields `platform: null` and `platforms: []` remain the parsed
|
||||
|
||||
@@ -1,123 +0,0 @@
|
||||
# Apple IPS crash reports
|
||||
|
||||
`inspect_apple_ips` / `inspect-apple-ips` answers whether one selected local
|
||||
Apple `.ips` file is a crash report (`bug_type` string `"309"`), which process,
|
||||
exception, and termination facts that report records, which threads and
|
||||
`usedImages` rows it lists, and which documented crash classes its strings
|
||||
match. It reads the file without executing a target and does not require Xcode,
|
||||
atos, a disassembler, or pwntools.
|
||||
|
||||
```bash
|
||||
rea inspect-apple-ips /absolute/path/to/report.ips --json
|
||||
```
|
||||
|
||||
The CLI resolves relative paths against the caller's working directory. MCP
|
||||
requires an absolute filesystem path and preserves that exact selection:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_apple_ips",
|
||||
"arguments": { "path": "/artifacts/report.ips" }
|
||||
}
|
||||
```
|
||||
|
||||
## Observations
|
||||
|
||||
The inline Evidence result contains the artifact path, SHA-256, and byte length.
|
||||
`metadata` is the first-line JSON object. `bug_type` stays a string. The crash
|
||||
body is parsed only when that string is `"309"`. Other types, including `"288"`
|
||||
stackshots, return no crash result.
|
||||
|
||||
`process`, `exception`, `termination`, `faulting_thread`, `threads`, and
|
||||
`images` reproduce documented body fields when present, including each image's
|
||||
`name` and each frame's `symbolLocation` (as `symbol_location`, the byte offset
|
||||
from the start of `recorded_symbol`). `termination` keeps the terminating
|
||||
process as `by_proc` and `by_pid` and its `flags`; `process.parent_pid` keeps
|
||||
`parentPid` next to `parent`; and `metadata` keeps `build_version` and
|
||||
`platform`. Integers keep their
|
||||
decimal literals and a hexadecimal form, including arm64e registers above
|
||||
2^53. Frame roles follow the recorded frame index: row zero of the resolved
|
||||
faulting thread is `faulting_instruction`, row zero of other threads is
|
||||
`current_instruction`, and later rows are `return_address`. Each runtime
|
||||
address uses that row's own image base and offset. The recorded `inline` flag
|
||||
does not supply a missing offset or authorize borrowing another row's address.
|
||||
When `faultingThread` is missing or does not resolve to a recorded thread
|
||||
object, row-zero roles are `unknown`; the recorded lookup remains available.
|
||||
A translated process records Rosetta's registers in a nested
|
||||
`threadState.rosetta` object; each one is reported as `rosetta.<name>` and
|
||||
counts toward the register budget. A [public ReportCrash `.ips` log](https://github.com/Mortennn/Dozer/issues/189)
|
||||
shows `rosetta.tmp2`, `tmp1`, and `tmp0` beside ordinary x86 registers. The
|
||||
test fixture quotes that register excerpt inside an authored report body; it
|
||||
does not generate a new Rosetta crash during the test.
|
||||
Observed JSON strings retain `""`; `null` means the field was missing or JSON
|
||||
`null`. This includes image paths, frame symbols, process text, and the older
|
||||
string `osVersion` form. Unlisted JSON keys and generic register names also
|
||||
retain empty spellings. An empty image architecture remains recorded as `""`
|
||||
while `uuid_unknown` stays true, since that spelling does not establish an
|
||||
architecture. An empty UUID is malformed, so its exact spelling appears in
|
||||
`coverage.type_mismatches` and its canonical UUID remains unknown.
|
||||
`process.os_version` keeps the producer's dictionary (`train`, `build`,
|
||||
`release_type`, `is_embedded`) or the older string form. Image UUIDs are uppercase 8-4-4-4-12; an all-zero UUID
|
||||
or a missing or empty `arch` is `uuid_unknown`. Image `source` letters keep their
|
||||
documented meaning (`P` process, `S` shared cache, `C` shared cache library,
|
||||
`K` kernel, `U` kernel cache, `T` kernel text exec, `A` absolute). Keys the
|
||||
documentation does not list remain in `unlisted_keys` or `unlisted_body_keys`.
|
||||
|
||||
Each frame's `image_index` refers to its entry in `images`. Image architecture,
|
||||
UUID and `uuid_unknown` are retained once on that image. Resolve the reference
|
||||
when using those facts; an absent or out-of-range index leaves the image
|
||||
unknown. This avoids repeating image text for every stack frame.
|
||||
|
||||
A present value whose JSON type differs from the documented one is never read
|
||||
as absent: it is listed in `coverage.type_mismatches` with its path, the
|
||||
expected type, and the observed type, and `coverage.status` becomes
|
||||
`partial`. A crash body without `threads` or `usedImages` is also `partial`.
|
||||
Malformed image UUID strings likewise make coverage partial; their exact spelling
|
||||
is retained in the mismatch's `recorded_value`, while the canonical image UUID
|
||||
remains unknown. Both 32 hexadecimal digits and canonical hyphenated UUIDs are accepted.
|
||||
Unresolved thread/image references also make coverage partial. A rejected
|
||||
integer lookup keeps its exact decimal literal in `recorded_integer`, including
|
||||
negative values and values above uint32. Valid uint32 lookup values remain in
|
||||
`faulting_thread` or `image_index` even when the referenced row is unavailable.
|
||||
|
||||
`classification` matches only documented SIGKILL codes (compared by their
|
||||
canonical hexadecimal form), `EXC_GUARD` subtypes,
|
||||
pointer-authentication suffixes, and dyld missing-library strings. A report
|
||||
that matches none of those rules is `unclassified`.
|
||||
|
||||
Opened Mach-O and dSYM UUID joins remain unobserved. Symbol names printed
|
||||
inside the report are recorded claims and are not verified. Linux ELF cores
|
||||
use the separate [recorded crash](recorded-crashes.md) workflow.
|
||||
|
||||
## Limits and failures
|
||||
|
||||
The selected file is limited to 1 MiB by default (16 MiB maximum). At most 256
|
||||
threads, 8192 frames, 1024 `usedImages` rows, and 16384 `threadState`
|
||||
register slots are examined by default (4096 / 65536 / 4096 / 262144
|
||||
maximum). Budgets are checked before each row is parsed, and body members the
|
||||
report does not interpret are syntax-checked without being kept. Fixed
|
||||
allowances bound JSON nesting (64 levels), unlisted key names per object
|
||||
(1024), and `termination.reasons` entries (256). Malformed metadata, a
|
||||
truncated crash body, invalid UTF-8, a NUL in the text, and exhausted budgets
|
||||
fail without partial success.
|
||||
|
||||
`max_integer_text_bytes` (`--max-integer-text-bytes` in the CLI) separately
|
||||
bounds retained decimal and hexadecimal integer text, including every derived
|
||||
runtime address and rejected lookup literal. Its default is 8 MiB of UTF-16 text storage and its maximum is
|
||||
64 MiB. Each appearance counts, even when an object could be shared in memory
|
||||
but would be serialized more than once. The parser reserves a conservative
|
||||
upper bound before numeric conversion or address expansion, then releases the
|
||||
unused reservation after each value. It reuses the parsed source integer when
|
||||
the same image base contributes to many frames.
|
||||
|
||||
Raise this per-call allowance on the next call when more room is needed; no
|
||||
restart is required. File-size and row budgets do not raise it. This allowance
|
||||
is separate from JSON wire size, MCP response limits and the client's receive
|
||||
buffer. An exhausted allowance reports a typed resource error without partial
|
||||
success or truncated integer facts.
|
||||
|
||||
```bash
|
||||
rea inspect-apple-ips /absolute/path/to/report.ips --max-integer-text-bytes 16777216 --json
|
||||
```
|
||||
|
||||
This workflow does not execute a target, launch atos, or search for dSYMs.
|
||||
+10
-47
@@ -1,6 +1,4 @@
|
||||
graph TD
|
||||
%% Selected layer owners and dependency/wiring paths, not a directory or import inventory.
|
||||
%% Current deep-provider candidates are wired in src/composition/binary.ts.
|
||||
subgraph "Entry Points"
|
||||
CLI["src/cli.ts + src/cli/<br/>One-shot CLI adapter<br/>setup, diagnostics, analysis, decompilation"]
|
||||
MAIN["src/main.ts + src/main/<br/>MCP server adapter: stdio default, streamable HTTP opt-in<br/>config, session, process lifetime"]
|
||||
@@ -8,7 +6,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Production Composition"
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>typed capability bindings: construction, availability and retryable lifetimes<br/>pure capability declarations feed provider catalogs"]
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>static WASM/EVM descriptors: named contracts, factories, retryable lifetimes"]
|
||||
end
|
||||
|
||||
subgraph "Adapters"
|
||||
@@ -21,15 +19,14 @@ graph TD
|
||||
subgraph "Application Layer"
|
||||
APP["src/application/<br/>Shared CLI/MCP workflows<br/>enhanced analysis and investigations"]
|
||||
SESSION["SessionProviderRouter + BinarySession<br/>one immutable deep binding per target<br/>disjoint auxiliary routing"]
|
||||
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback<br/>ProviderInspectionScope: prerequisite observation lifetime"]
|
||||
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback"]
|
||||
RECORDS["src/application/investigation/<br/>composed InvestigationRecords + narrow ports<br/>EvidenceLedger and Unknown ownership"]
|
||||
LIFETIMES["CapabilityScope<br/>host-owned service close capabilities<br/>released peers removed; failed cleanup retained for retry"]
|
||||
SNAPSHOT["BinarySessionRecords + AnalysisSnapshotCache<br/>binary target/profile snapshots and session file ownership<br/>record coordination and post-commit notifications"]
|
||||
SNAPSHOT["BinarySessionRecords + AnalysisSnapshotCache<br/>binary target/profile snapshots<br/>compatibility facade and post-commit notifications"]
|
||||
JS_RECON["src/application/javascript/ + src/artifacts/javascript/<br/>JavaScript artifact reconstruction<br/>directory/ASAR reads<br/>AST-only bundle projection"]
|
||||
end
|
||||
|
||||
subgraph "Provider Layers (selected adapters)"
|
||||
PROVIDER_OWNERS["src/providers/<br/>shared deferred client owners<br/>exact cleanup retry and attributed failures"]
|
||||
subgraph "Provider Layers"
|
||||
PROVIDER_OWNERS["src/providers/<br/>shared deferred client and inspection owners<br/>exact cleanup retry and attributed failures"]
|
||||
PWNTOOLS["src/native/pwntools/ + src/native/pwndbg/<br/>BYO offline ELF/core adapters<br/>owned snapshots/Python and optional core-only GDB<br/>unchanged upstream parsers, ABI structures and mapping API"]
|
||||
WASM["src/wasm/<br/>caller-supplied WABT; immutable artifact inspection<br/>owned process supervision and retryable workspace cleanup"]
|
||||
APKSIG["src/apkSignature/<br/>in-process APK Signing Block + v1 reader<br/>bounded single read, no device or execution"]
|
||||
@@ -37,13 +34,8 @@ graph TD
|
||||
HOPPER["src/hopper/<br/>Hopper launch & Unix-socket protocol<br/>BridgeLauncher, HopperClient, protocol"]
|
||||
GHIDRA["src/ghidra/<br/>Ghidra discovery, profile & function analysis<br/>snapshot, Unix/Windows transport, bounded lifecycle"]
|
||||
IDA["src/ida/<br/>Upstream MCP contract adaptation<br/>live GUI binding or owned headless database"]
|
||||
KUNA["src/kuna/<br/>BYO Kuna deep analysis candidate<br/>discovery reads metadata; version probe on target open"]
|
||||
IDA_MCP["External ida-pro-mcp<br/>GUI proxy or database supervisor<br/>upstream installation and engine lifecycle"]
|
||||
|
||||
KUNA["src/kuna/<br/>BYO Kuna decompiler, a deep analysis candidate<br/>discovery reads metadata only; version probe when a target opens"]
|
||||
AUX["src/artifacts/ + src/native/<br/>artifact inventory, bounded directory triage and extraction<br/>format readers and native capabilities"]
|
||||
|
||||
|
||||
BROWSER["src/browser/<br/>CDP / Playwright capture<br/>captured scripts and native module resolution<br/>execution/: coverage, listener and source attribution"]
|
||||
JS_CODEC["src/javascript/sourceMaps/<br/>pinned upstream map codec<br/>owned heap-limited Node process, complete point evidence"]
|
||||
INSPECTOR["src/inspector/<br/>passive V8 Inspector discovery and capture<br/>script and target locations"]
|
||||
@@ -55,19 +47,12 @@ graph TD
|
||||
FIRMWARE["src/firmware/<br/>Binwalk / Unblob adapters<br/>owned processes, report normalization, private extraction"]
|
||||
ANDROID["src/android/<br/>Static APK adapter<br/>owned SDK stdio, pinned headless JADX, normalization"]
|
||||
REFERENCE["src/reference/<br/>Filesystem reference-source reader<br/>bounded traversal, validation, errors"]
|
||||
|
||||
ANDROID_TOOLS["src/adb/ + src/apktool/ + src/jeb/<br/>BYO device, resource-decoding and JEB adapters<br/>owned adb and Apktool processes; JEB engine client over MCP"]
|
||||
CAPTURE_TOOLS["src/captures/ + src/pcap/ + src/signals/ + src/sqlite/<br/>BYO tshark, pcap, sigrok and SQLite child inspection<br/>selected captures, recordings and databases; owned processes"]
|
||||
NATIVE_READERS["src/go/ + src/rust/ + src/il2cpp/ + src/flutter/ + src/gpu/<br/>Bundled in-process binary metadata readers<br/>Go build info, Rust dependencies, IL2CPP, Dart build ID, GPU code objects<br/>src/compression/: shared codec-result helper for rust and gpu"]
|
||||
FORMAT_READERS["src/model/ + src/python/ + src/erlang/ + src/electron/<br/>Bundled in-process format readers<br/>GGUF and safetensors headers, Python .pyc, Erlang BEAM, Electron fuses"]
|
||||
RUNTIME["src/frida/ + src/cutter/<br/>Frida instrumentation sessions and live Cutter bridge client"]
|
||||
end
|
||||
|
||||
subgraph "Provider Process Foundation"
|
||||
PROCESS["src/process/<br/>owned process groups, private roots<br/>deadlines, bounded file reads and diagnostics, cleanup"]
|
||||
WINDOWS["src/windows/<br/>Windows host boundary adapters<br/>package loader, opaque handles, owned jobs and private runtimes"]
|
||||
WINDOWS_NATIVE["native/windows/src/<br/>Node-API implementation<br/>NTFS admission, protected DACLs and atomic Job Object assignment"]
|
||||
FILESYSTEM["src/filesystem/<br/>owned file and directory handles<br/>stable reads, staged output, caller paths"]
|
||||
end
|
||||
|
||||
subgraph "Contracts Layer"
|
||||
@@ -76,7 +61,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Domain Layer"
|
||||
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>format declarations, provider selection, Evidence, comparisons, application graphs"]
|
||||
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>evidence, comparisons, application graphs"]
|
||||
end
|
||||
|
||||
subgraph "Configuration"
|
||||
@@ -87,8 +72,8 @@ graph TD
|
||||
TESTS["tests/<br/>Vitest suite<br/>fixtures/ holds fake launcher, bridge, and CDP seams"]
|
||||
end
|
||||
|
||||
subgraph "Development Evaluation"
|
||||
EVALUATION["tests/evaluation/model-evals/<br/>Agent release evaluation<br/>session transcript scoring"]
|
||||
subgraph "Evaluation"
|
||||
EVALUATION["src/evaluation/<br/>Agent release evaluation<br/>session transcript scoring"]
|
||||
end
|
||||
|
||||
DISPATCH --> CLI
|
||||
@@ -100,7 +85,6 @@ graph TD
|
||||
MAIN --> COMPOSITION
|
||||
CLI --> COMPOSITION
|
||||
SERVER --> COMPOSITION
|
||||
COMPOSITION --> LIFETIMES
|
||||
COMPOSITION --> SESSION
|
||||
APP --> FIRMWARE
|
||||
FIRMWARE --> PROCESS
|
||||
@@ -128,7 +112,6 @@ graph TD
|
||||
COMPOSITION --> HOPPER
|
||||
COMPOSITION --> GHIDRA
|
||||
COMPOSITION --> IDA
|
||||
COMPOSITION --> KUNA
|
||||
COMPOSITION --> AUX
|
||||
IDA --> IDA_MCP
|
||||
IDA --> CONTRACTS
|
||||
@@ -151,14 +134,14 @@ graph TD
|
||||
OBJDUMP --> DOMAIN
|
||||
COMPOSITION --> RIZIN
|
||||
RIZIN --> DOMAIN
|
||||
COMPOSITION --> GDB
|
||||
CLI --> GDB
|
||||
SERVER --> GDB
|
||||
GDB --> PROCESS
|
||||
GDB --> DOMAIN
|
||||
HOPPER --> HOPPER_BRIDGE
|
||||
GHIDRA --> GHIDRA_BRIDGE
|
||||
HOPPER --> PROCESS
|
||||
GHIDRA --> PROCESS
|
||||
KUNA --> PROCESS
|
||||
GHIDRA --> PROVIDER_OWNERS
|
||||
APP --> PROVIDER_OWNERS
|
||||
PROVIDER_OWNERS --> PROCESS
|
||||
@@ -176,7 +159,6 @@ graph TD
|
||||
CONTRACTS --> DOMAIN
|
||||
HOPPER --> DOMAIN
|
||||
GHIDRA --> DOMAIN
|
||||
KUNA --> DOMAIN
|
||||
AUX --> DOMAIN
|
||||
BROWSER --> DOMAIN
|
||||
PROCESS --> DOMAIN
|
||||
@@ -193,22 +175,3 @@ graph TD
|
||||
APP -->|EVM interface port| EVM
|
||||
EVM --> PROCESS
|
||||
EVM --> DOMAIN
|
||||
COMPOSITION --> KUNA
|
||||
KUNA --> PROCESS
|
||||
KUNA --> FILESYSTEM
|
||||
KUNA --> DOMAIN
|
||||
COMPOSITION --> ANDROID_TOOLS
|
||||
ANDROID_TOOLS --> PROCESS
|
||||
ANDROID_TOOLS --> DOMAIN
|
||||
COMPOSITION --> CAPTURE_TOOLS
|
||||
CAPTURE_TOOLS --> PROCESS
|
||||
CAPTURE_TOOLS --> DOMAIN
|
||||
COMPOSITION --> NATIVE_READERS
|
||||
NATIVE_READERS --> DOMAIN
|
||||
COMPOSITION --> FORMAT_READERS
|
||||
FORMAT_READERS --> DOMAIN
|
||||
COMPOSITION --> RUNTIME
|
||||
RUNTIME --> DOMAIN
|
||||
PROCESS --> FILESYSTEM
|
||||
AUX --> FILESYSTEM
|
||||
APP --> FILESYSTEM
|
||||
@@ -1,118 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="960" height="263" viewBox="0 0 960 263" role="img" aria-labelledby="title desc">
|
||||
<title id="title">REA showcases</title>
|
||||
<desc id="desc">Three illustrations. A brick-breaker screen with a speaker on each side stands for the DX-Ball sound-pan showcase. A browser window joined by a wire to a clipboard stands for the Notion clipboard-bridge showcase. A ring of 24 dots with one orange dot around a dark centre stands for the TH04 bullet-ring showcase.</desc>
|
||||
<rect width="960" height="263" fill="#11111b"/>
|
||||
<rect x="43" y="16.5" width="262" height="235.5" fill="#181825"/>
|
||||
<rect x="354" y="14.5" width="258" height="237.5" fill="#181825"/>
|
||||
<rect x="664" y="16.5" width="260" height="235.5" fill="#181825"/>
|
||||
<g id="dx-ball">
|
||||
<rect x="108.5" y="54" width="140" height="160" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="106.85" y="52.1" width="136" height="157" fill="#181825" stroke="#cdd6f4" stroke-width="3.7"/>
|
||||
<rect x="110.9" y="55.9" width="128.2" height="149.6" fill="none" stroke="#6c7086" stroke-width="2.2"/>
|
||||
<rect x="112.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="145.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="178.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="211.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="112.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="145.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="178.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="211.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="111.8" y="57.900000000000006" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="144.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="177.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="210.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="111.8" y="75.0" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="144.79999999999998" y="75.0" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="177.79999999999998" y="75.0" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="210.79999999999998" y="75.0" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<ellipse cx="173.4" cy="157.2" rx="8.4" ry="8.2" fill="#000000" fill-opacity="0.45"/>
|
||||
<ellipse cx="171.9" cy="154.5" rx="7.3" ry="7.2" fill="#585b70" stroke="#cdd6f4" stroke-width="2"/>
|
||||
<rect x="165.5" y="149.5" width="3.2" height="2.6" fill="#a6adc8"/>
|
||||
<rect x="145.5" y="195.2" width="57.6" height="12.4" fill="#585b70" stroke="#cdd6f4" stroke-width="2.2"/>
|
||||
<rect x="144.5" y="204" width="59.3" height="6.5" fill="#cdd6f4"/>
|
||||
<g transform="translate(0 0)">
|
||||
<path d="M61.5 117.5H69.6L78 108.5V141.2L69.6 132.8H61.5Z" fill="#585b70" stroke="#cdd6f4" stroke-width="2.3" stroke-linejoin="miter"/>
|
||||
<path d="M78.1 106V143" stroke="#cdd6f4" stroke-width="3.4" fill="none"/>
|
||||
<g fill="none" stroke="#89b4fa" stroke-width="2.2" stroke-linecap="butt" stroke-dasharray="2.6 1.4">
|
||||
<path d="M84.8 118.2Q87.6 125 84.8 131.8"/>
|
||||
<path d="M90.2 112.4Q94.5 125 90.2 137.6"/>
|
||||
<path d="M95.8 108.2Q101.5 125 95.8 141.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(190 0)">
|
||||
<path d="M61.5 117.5H69.6L78 108.5V141.2L69.6 132.8H61.5Z" fill="#585b70" stroke="#cdd6f4" stroke-width="2.3" stroke-linejoin="miter"/>
|
||||
<path d="M78.1 106V143" stroke="#cdd6f4" stroke-width="3.4" fill="none"/>
|
||||
<g fill="none" stroke="#89b4fa" stroke-width="2.2" stroke-linecap="butt" stroke-dasharray="2.6 1.4">
|
||||
<path d="M84.8 118.2Q87.6 125 84.8 131.8"/>
|
||||
<path d="M90.2 112.4Q94.5 125 90.2 137.6"/>
|
||||
<path d="M95.8 108.2Q101.5 125 95.8 141.8"/>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="clipboard-bridge">
|
||||
<rect x="390.3" y="83.6" width="100.6" height="82.6" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="387.7" y="81.2" width="97.8" height="79.7" fill="#181825" stroke="#cdd6f4" stroke-width="2.9"/>
|
||||
<rect x="389.2" y="82.7" width="94.9" height="13" fill="#45475a"/>
|
||||
<path d="M387.7 97H485.5" stroke="#cdd6f4" stroke-width="3.1"/>
|
||||
<rect x="394.7" y="86.6" width="4.6" height="5" fill="#fab387"/>
|
||||
<rect x="402.7" y="86.6" width="4.6" height="5" fill="#f9e2af"/>
|
||||
<rect x="410.8" y="86.6" width="4.6" height="5" fill="#a6adc8"/>
|
||||
<rect x="398" y="107.4" width="10.6" height="9.4" fill="#7f849c"/>
|
||||
<rect x="412.4" y="107.9" width="16.8" height="6" fill="#7f849c"/>
|
||||
<rect x="434" y="107.9" width="42.4" height="6.2" fill="#7f849c"/>
|
||||
<rect x="412.2" y="117.3" width="64" height="2.6" fill="#6c7086"/>
|
||||
<rect x="412.2" y="124.9" width="55.3" height="3.6" fill="#6c7086"/>
|
||||
<rect x="412.2" y="133.2" width="50.3" height="3.8" fill="#6c7086"/>
|
||||
<rect x="412.2" y="142" width="13.2" height="3.4" fill="#6c7086"/>
|
||||
<rect x="430" y="142" width="14.4" height="3.4" fill="#6c7086"/>
|
||||
<rect x="449" y="142" width="27.6" height="3.2" fill="#313244"/>
|
||||
<rect x="412.2" y="150" width="12.6" height="2.9" fill="#6c7086"/>
|
||||
<path d="M455.2 162V187H476M497 187H508.2V163.2H523" fill="none" stroke="#89b4fa" stroke-width="3" stroke-linejoin="miter"/>
|
||||
<rect x="477.5" y="178.3" width="23" height="21" fill="#000000" fill-opacity="0.45"/>
|
||||
<path d="M484.7 173.2V177" stroke="#cdd6f4" stroke-width="2.2"/>
|
||||
<rect x="476.4" y="177.5" width="20.2" height="18.1" fill="#585b70" stroke="#cdd6f4" stroke-width="2.2"/>
|
||||
<rect x="481.4" y="178.6" width="4.6" height="16" fill="#313244"/>
|
||||
<rect x="477.2" y="191" width="4.6" height="4.4" fill="#f9e2af"/>
|
||||
<rect x="481.6" y="183.8" width="3.4" height="3" fill="#fab387"/>
|
||||
<rect x="525.4" y="114.6" width="60.6" height="75" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="524.7" y="112.5" width="56.4" height="69.4" fill="#45475a" stroke="#cdd6f4" stroke-width="3"/>
|
||||
<rect x="531.6" y="118.6" width="42.3" height="53.4" fill="#1e1e2e"/>
|
||||
<rect x="531.6" y="172" width="42.3" height="8.4" fill="#45475a"/>
|
||||
<path d="M531.6 175.8H575" stroke="#cdd6f4" stroke-width="1.8" stroke-opacity="0.8"/>
|
||||
<path d="M547.4 103.3H556.6V106.2H559.6V109.2H564.9V120.3H540.1V109.2H545V106.2H547.4Z" fill="#585b70" stroke="#cdd6f4" stroke-width="2" stroke-linejoin="miter"/>
|
||||
<rect x="549.4" y="108.4" width="2.8" height="1.9" fill="#181825"/>
|
||||
<rect x="536.6" y="131.8" width="32.3" height="3.4" fill="#7f849c"/>
|
||||
<rect x="536.6" y="138.8" width="32.3" height="3.4" fill="#7f849c"/>
|
||||
<rect x="536.6" y="145.8" width="32.3" height="3.4" fill="#7f849c"/>
|
||||
<rect x="536.6" y="152.8" width="26.8" height="3.4" fill="#7f849c"/>
|
||||
<rect x="536.6" y="159.8" width="14.6" height="3.4" fill="#7f849c"/>
|
||||
</g>
|
||||
<g id="bullet-ring">
|
||||
<circle cx="783" cy="74.5" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="798" cy="72" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="812.5" cy="74.5" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="769.5" cy="81" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="826.5" cy="81" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="756.5" cy="90" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="839" cy="90" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="746.5" cy="103" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="849" cy="103" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="740.5" cy="117" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="854.5" cy="117" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="738" cy="134" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="856.5" cy="133.5" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="741" cy="149" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="854" cy="149" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="747.5" cy="163" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="848" cy="163" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="757.5" cy="175" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="770" cy="184" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="826.5" cy="184" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="838.5" cy="175" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="784" cy="191" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="798" cy="192.5" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="812" cy="190.5" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="833.5" cy="109" r="4.1" fill="#fab387"/>
|
||||
<circle cx="797.4" cy="133" r="10.9" fill="#cdd6f4"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 8.7 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 48 KiB |
@@ -1,118 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="960" height="263" viewBox="0 0 960 263" role="img" aria-labelledby="title desc">
|
||||
<title id="title">REA showcases</title>
|
||||
<desc id="desc">Three illustrations. A brick-breaker screen with a speaker on each side stands for the DX-Ball sound-pan showcase. A browser window joined by a wire to a clipboard stands for the Notion clipboard-bridge showcase. A ring of 24 dots with one orange dot around a dark centre stands for the TH04 bullet-ring showcase.</desc>
|
||||
<rect width="960" height="263" fill="#fafbfb"/>
|
||||
<rect x="43" y="16.5" width="262" height="235.5" fill="#f4f6f5"/>
|
||||
<rect x="354" y="14.5" width="258" height="237.5" fill="#f4f6f5"/>
|
||||
<rect x="664" y="16.5" width="260" height="235.5" fill="#f4f6f5"/>
|
||||
<g id="dx-ball">
|
||||
<rect x="108.5" y="54" width="140" height="160" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="106.85" y="52.1" width="136" height="157" fill="#f4f6f5" stroke="#191a1d" stroke-width="3.7"/>
|
||||
<rect x="110.9" y="55.9" width="128.2" height="149.6" fill="none" stroke="#a5a6a9" stroke-width="2.2"/>
|
||||
<rect x="112.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="145.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="178.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="211.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="112.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="145.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="178.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="211.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="111.8" y="57.900000000000006" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="144.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="177.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="210.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="111.8" y="75.0" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="144.79999999999998" y="75.0" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="177.79999999999998" y="75.0" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="210.79999999999998" y="75.0" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<ellipse cx="173.4" cy="157.2" rx="8.4" ry="8.2" fill="#000000" fill-opacity="0.27"/>
|
||||
<ellipse cx="171.9" cy="154.5" rx="7.3" ry="7.2" fill="#4d5863" stroke="#191a1d" stroke-width="2"/>
|
||||
<rect x="165.5" y="149.5" width="3.2" height="2.6" fill="#aeb1b4"/>
|
||||
<rect x="145.5" y="195.2" width="57.6" height="12.4" fill="#525c65" stroke="#191a1d" stroke-width="2.2"/>
|
||||
<rect x="144.5" y="204" width="59.3" height="6.5" fill="#191a1d"/>
|
||||
<g transform="translate(0 0)">
|
||||
<path d="M61.5 117.5H69.6L78 108.5V141.2L69.6 132.8H61.5Z" fill="#56616b" stroke="#191a1d" stroke-width="2.3" stroke-linejoin="miter"/>
|
||||
<path d="M78.1 106V143" stroke="#191a1d" stroke-width="3.4" fill="none"/>
|
||||
<g fill="none" stroke="#274b77" stroke-width="2.2" stroke-linecap="butt" stroke-dasharray="2.6 1.4">
|
||||
<path d="M84.8 118.2Q87.6 125 84.8 131.8"/>
|
||||
<path d="M90.2 112.4Q94.5 125 90.2 137.6"/>
|
||||
<path d="M95.8 108.2Q101.5 125 95.8 141.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(190 0)">
|
||||
<path d="M61.5 117.5H69.6L78 108.5V141.2L69.6 132.8H61.5Z" fill="#56616b" stroke="#191a1d" stroke-width="2.3" stroke-linejoin="miter"/>
|
||||
<path d="M78.1 106V143" stroke="#191a1d" stroke-width="3.4" fill="none"/>
|
||||
<g fill="none" stroke="#274b77" stroke-width="2.2" stroke-linecap="butt" stroke-dasharray="2.6 1.4">
|
||||
<path d="M84.8 118.2Q87.6 125 84.8 131.8"/>
|
||||
<path d="M90.2 112.4Q94.5 125 90.2 137.6"/>
|
||||
<path d="M95.8 108.2Q101.5 125 95.8 141.8"/>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="clipboard-bridge">
|
||||
<rect x="390.3" y="83.6" width="100.6" height="82.6" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="387.7" y="81.2" width="97.8" height="79.7" fill="#f4f6f5" stroke="#191a1d" stroke-width="2.9"/>
|
||||
<rect x="389.2" y="82.7" width="94.9" height="13" fill="#5a636d"/>
|
||||
<path d="M387.7 97H485.5" stroke="#191a1d" stroke-width="3.1"/>
|
||||
<rect x="394.7" y="86.6" width="4.6" height="5" fill="#c3693b"/>
|
||||
<rect x="402.7" y="86.6" width="4.6" height="5" fill="#be9a53"/>
|
||||
<rect x="410.8" y="86.6" width="4.6" height="5" fill="#b3b3b4"/>
|
||||
<rect x="398" y="107.4" width="10.6" height="9.4" fill="#666671"/>
|
||||
<rect x="412.4" y="107.9" width="16.8" height="6" fill="#666c74"/>
|
||||
<rect x="434" y="107.9" width="42.4" height="6.2" fill="#666c74"/>
|
||||
<rect x="412.2" y="117.3" width="64" height="2.6" fill="#7a7c82"/>
|
||||
<rect x="412.2" y="124.9" width="55.3" height="3.6" fill="#7a7c82"/>
|
||||
<rect x="412.2" y="133.2" width="50.3" height="3.8" fill="#6d7279"/>
|
||||
<rect x="412.2" y="142" width="13.2" height="3.4" fill="#6d7279"/>
|
||||
<rect x="430" y="142" width="14.4" height="3.4" fill="#6d7279"/>
|
||||
<rect x="449" y="142" width="27.6" height="3.2" fill="#e8e8e8"/>
|
||||
<rect x="412.2" y="150" width="12.6" height="2.9" fill="#6d7279"/>
|
||||
<path d="M455.2 162V187H476M497 187H508.2V163.2H523" fill="none" stroke="#274b77" stroke-width="3" stroke-linejoin="miter"/>
|
||||
<rect x="477.5" y="178.3" width="23" height="21" fill="#000000" fill-opacity="0.27"/>
|
||||
<path d="M484.7 173.2V177" stroke="#191a1d" stroke-width="2.2"/>
|
||||
<rect x="476.4" y="177.5" width="20.2" height="18.1" fill="#525c65" stroke="#191a1d" stroke-width="2.2"/>
|
||||
<rect x="481.4" y="178.6" width="4.6" height="16" fill="#323435"/>
|
||||
<rect x="477.2" y="191" width="4.6" height="4.4" fill="#be9a53"/>
|
||||
<rect x="481.6" y="183.8" width="3.4" height="3" fill="#c3693b"/>
|
||||
<rect x="525.4" y="114.6" width="60.6" height="75" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="524.7" y="112.5" width="56.4" height="69.4" fill="#545e69" stroke="#191a1d" stroke-width="3"/>
|
||||
<rect x="531.6" y="118.6" width="42.3" height="53.4" fill="#ededec"/>
|
||||
<rect x="531.6" y="172" width="42.3" height="8.4" fill="#545e69"/>
|
||||
<path d="M531.6 175.8H575" stroke="#191a1d" stroke-width="1.8" stroke-opacity="0.8"/>
|
||||
<path d="M547.4 103.3H556.6V106.2H559.6V109.2H564.9V120.3H540.1V109.2H545V106.2H547.4Z" fill="#545e69" stroke="#191a1d" stroke-width="2" stroke-linejoin="miter"/>
|
||||
<rect x="549.4" y="108.4" width="2.8" height="1.9" fill="#f4f6f5"/>
|
||||
<rect x="536.6" y="131.8" width="32.3" height="3.4" fill="#706f74"/>
|
||||
<rect x="536.6" y="138.8" width="32.3" height="3.4" fill="#706f74"/>
|
||||
<rect x="536.6" y="145.8" width="32.3" height="3.4" fill="#706f74"/>
|
||||
<rect x="536.6" y="152.8" width="26.8" height="3.4" fill="#706f74"/>
|
||||
<rect x="536.6" y="159.8" width="14.6" height="3.4" fill="#706f74"/>
|
||||
</g>
|
||||
<g id="bullet-ring">
|
||||
<circle cx="783" cy="74.5" r="3.8" fill="#274b77"/>
|
||||
<circle cx="798" cy="72" r="3.8" fill="#274b77"/>
|
||||
<circle cx="812.5" cy="74.5" r="3.8" fill="#274b77"/>
|
||||
<circle cx="769.5" cy="81" r="3.8" fill="#274b77"/>
|
||||
<circle cx="826.5" cy="81" r="3.8" fill="#274b77"/>
|
||||
<circle cx="756.5" cy="90" r="3.8" fill="#274b77"/>
|
||||
<circle cx="839" cy="90" r="3.8" fill="#274b77"/>
|
||||
<circle cx="746.5" cy="103" r="3.8" fill="#274b77"/>
|
||||
<circle cx="849" cy="103" r="3.8" fill="#274b77"/>
|
||||
<circle cx="740.5" cy="117" r="3.8" fill="#274b77"/>
|
||||
<circle cx="854.5" cy="117" r="3.8" fill="#274b77"/>
|
||||
<circle cx="738" cy="134" r="3.8" fill="#274b77"/>
|
||||
<circle cx="856.5" cy="133.5" r="3.8" fill="#274b77"/>
|
||||
<circle cx="741" cy="149" r="3.8" fill="#274b77"/>
|
||||
<circle cx="854" cy="149" r="3.8" fill="#274b77"/>
|
||||
<circle cx="747.5" cy="163" r="3.8" fill="#274b77"/>
|
||||
<circle cx="848" cy="163" r="3.8" fill="#274b77"/>
|
||||
<circle cx="757.5" cy="175" r="3.8" fill="#274b77"/>
|
||||
<circle cx="770" cy="184" r="3.8" fill="#274b77"/>
|
||||
<circle cx="826.5" cy="184" r="3.8" fill="#274b77"/>
|
||||
<circle cx="838.5" cy="175" r="3.8" fill="#274b77"/>
|
||||
<circle cx="784" cy="191" r="3.8" fill="#274b77"/>
|
||||
<circle cx="798" cy="192.5" r="3.8" fill="#274b77"/>
|
||||
<circle cx="812" cy="190.5" r="3.8" fill="#274b77"/>
|
||||
<circle cx="833.5" cy="109" r="4.1" fill="#c3693b"/>
|
||||
<circle cx="797.4" cy="133" r="10.9" fill="#1c222a"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 8.7 KiB |
+10
-119
@@ -8,14 +8,9 @@ ordinary binary inspection use the same modular capability.
|
||||
|
||||
## Bring your own engine
|
||||
|
||||
Real verification covers **Linux x64** hosts and **EXEC/DYN/REL** objects in
|
||||
these profiles: ELF64 x86-64, ELF64 little-endian AArch64, ELF32 i386, ELF32
|
||||
little-endian ARM and ELF32 little- or big-endian MIPS. Other machines, classes
|
||||
and byte orders, including big-endian ARM, return `unsupported_target`. Other
|
||||
hosts, including Linux arm64, return `capability_unavailable` naming the
|
||||
detected host before any target is read. Provide an absolute
|
||||
Python executable whose environment already contains unchanged pwntools 4.15.0,
|
||||
pyelftools 0.33 and Unicorn 2.1.2:
|
||||
Initial real verification covers **Linux x64, ELF64 x86-64 little-endian
|
||||
EXEC/DYN/REL**. Provide an absolute Python executable whose environment already
|
||||
contains unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2:
|
||||
|
||||
```sh
|
||||
REA_PWNTOOLS_PYTHON=/absolute/isolated-env/bin/python \
|
||||
@@ -34,16 +29,6 @@ the selected object as a host process, or requests runtime library resolution. T
|
||||
uses isolated mode and an owned cache. Exact upstream profiles are recorded in
|
||||
[upstream provenance](https://github.com/morluto/rea/blob/main/third_party/pwntools/README.md).
|
||||
|
||||
The adapter skips pwntools kernel configuration and version discovery: those
|
||||
memory-content scans supply none of this report's fields. Ordinary `IKCFG_ST`
|
||||
data is not interpreted as a kernel-config request, and virtual zero-filled
|
||||
LOAD tails are reported from their headers without materializing them for that
|
||||
scan. GOT/PLT inference and the advertised static mitigation indicators still
|
||||
use the pinned upstream implementations. Header-table file ranges are checked
|
||||
before dependency seeks. Structural inventory and the convenience-map engine
|
||||
still parse the snapshot separately; eliminating that remaining duplication
|
||||
requires preserving the upstream GOT/PLT and mitigation facts.
|
||||
|
||||
## Interpreting results
|
||||
|
||||
- Artifact path, SHA-256 and size identify original selected bytes. All parsing
|
||||
@@ -105,12 +90,6 @@ one section or symbol, or a stable page without re-running the decoder. CLI
|
||||
one-shot invocations pass portable inline Evidence JSON; an MCP session uses
|
||||
the retained `evidence_id`. Page `limit` is required and bounded by the
|
||||
measured MCP stdio budget documented on the tool contract.
|
||||
An empty retained section or symbol collection has no valid item index. Its page
|
||||
view can still return an empty page.
|
||||
Malformed authenticated layout input remains rejected with the owning schema's
|
||||
safe field diagnostics under `source.normalized_result`, including the field
|
||||
path and type or value reason. Evidence authentication and decoder validation
|
||||
remain separate boundaries.
|
||||
|
||||
Over MCP, `inspect_binary_layout` with `"detail": "summary"` retains the
|
||||
complete layout Evidence in the session and returns only its summary view.
|
||||
@@ -191,31 +170,6 @@ subject to earlier cancellation or the startup deadline. `output_incomplete`
|
||||
marks a snapshot taken before stream closure; cleanup ownership remains with
|
||||
the session until release succeeds.
|
||||
|
||||
A kernel or container OOM killer ends Ghidra with `SIGKILL` and prints no
|
||||
`OutOfMemoryError`. When a process or protocol failure records
|
||||
`exit_signal: "SIGKILL"` and no `stop_requested`, REA reports the signal
|
||||
instead of generic `rea doctor` advice. The process supervisor records
|
||||
`stop_requested: true` once REA begins stopping Ghidra, before it sends any
|
||||
signal, so REA's own SIGKILL is never reported as an outside kill.
|
||||
|
||||
On Linux with cgroup v2, REA also reads its own cgroup's `memory.events` before
|
||||
launch and again after the kill, locating the group through the cgroup2 mount
|
||||
in `/proc/self/mountinfo`. `memory_cgroup` records the group path,
|
||||
`oom_kill_before`, `oom_kill_after` and `memory_max`. If the count rose, REA
|
||||
returns `resource_constraint` with `resource: "memory"` and `cgroup_memory_max`
|
||||
in `reported_limits`. The count covers every process in REA's cgroup and its
|
||||
child groups, which can include processes outside REA's process tree when the
|
||||
group is shared, so a rise shows that the kernel OOM killer ended a process in
|
||||
that cgroup or a child group, not which process it chose. `memory_max` is the
|
||||
limit in bytes, `"max"` when the group sets none, or `null` when it could not be
|
||||
read; a limit on a parent group is not read. The limit also covers Ghidra's
|
||||
native decompiler processes, whose number follows the available CPUs, so the
|
||||
advice does not recommend a larger Java heap. When the session had already
|
||||
completed an operation, the advice keeps the instruction to reopen the target
|
||||
with `open_binary`. Without a cgroup v2 memory group, or when the count did not
|
||||
rise, the failure stays `execution_failure` and names the signal as a possible
|
||||
out-of-memory kill.
|
||||
|
||||
## Portable PE resources
|
||||
|
||||
`inspect_pe_resources` / `inspect-pe-resources` inspects an explicit PE32 or
|
||||
@@ -239,14 +193,6 @@ reported. Directory headers, entries, data entries and payloads retain original
|
||||
file offsets and lengths. Each opaque payload has its own SHA-256; code page and
|
||||
reserved data-entry fields are preserved.
|
||||
|
||||
Every `directories[]` row retains its identity `path` and the matching incoming
|
||||
`entry_locations`: both are empty for the root, then contain the type and
|
||||
optionally the name. Empty branches keep these identities even when they have
|
||||
no payload leaves. Shared directory bytes produce a row for each incoming path,
|
||||
with the original directory range and each path's own entry locations. These
|
||||
rows are included in the metadata budget. A present empty root remains distinct
|
||||
from an absent resource data directory (`directory: null`, `directories: []`).
|
||||
|
||||
RT_GROUP_ICON records expose original image descriptors and RT_ICON candidates.
|
||||
A reference resolves only when an icon with the same numeric ID and language
|
||||
exists. Missing references, other-language candidates and declared-size mismatch
|
||||
@@ -372,48 +318,7 @@ rea inspect-native-exports ./runtimes/linux-x64/native/libexample.so --json
|
||||
```
|
||||
|
||||
The complete Evidence identifies the stable artifact snapshot by path, byte
|
||||
count and SHA-256. Complete delivery remains the default. For a small initial
|
||||
MCP reply, select `detail: "summary"`; this retains the complete Evidence and
|
||||
returns artifact identity, table coverage and row counts. Use its
|
||||
`normalized_result.parent_evidence_id` with `inspect_analysis_view`:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_<parent digest>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "native-exports",
|
||||
"collection": "exports",
|
||||
"names": ["selected_name"],
|
||||
"fields": ["name"],
|
||||
"offset": 0,
|
||||
"limit": 32
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Names match exactly and preserve every same-name version or alias. Optional
|
||||
`indices` selects original row positions, intersecting a name filter when both
|
||||
are supplied; empty filters select nothing. `source_index` always remains and,
|
||||
with the parent ID, identifies the original export or issue row. Fields default
|
||||
to the complete row; an empty field list returns only its identity. ELF `name`
|
||||
selection also returns raw-byte and encoding qualifications. Fields from the
|
||||
other format are rejected. `collection: "issues"` pages the complete diagnostic
|
||||
records, with optional original indices. Offset and limit default to 0 and 64
|
||||
and apply to matching rows in original order.
|
||||
|
||||
View `coverage` describes the selected page; `summary.coverage` or
|
||||
`exports.coverage` preserves source-table coverage. An exhausted or empty page
|
||||
does not make a partial source complete. Retained views never reopen the file.
|
||||
The CLI uses the same projector and requires a durable bundle for a summary:
|
||||
|
||||
```sh
|
||||
rea inspect-native-exports ./libexample.so --detail summary --evidence_output ./exports.bundle.json --json
|
||||
# Pass the summary's parent ID and the same view as JSON to inspect-analysis-view,
|
||||
# together with --evidence-bundle ./exports.bundle.json.
|
||||
```
|
||||
count and SHA-256.
|
||||
|
||||
- PE rows reproduce the export directory: each used export-address-table slot
|
||||
with its biased `ordinal` (`OrdinalBase` plus the slot index), one row per
|
||||
@@ -426,10 +331,8 @@ rea inspect-native-exports ./libexample.so --detail summary --evidence_output ./
|
||||
without an export directory is a complete empty inventory.
|
||||
- ELF rows list defined `.dynsym` symbols with global, weak or GNU-unique
|
||||
binding and default or protected visibility. Each row keeps `value`
|
||||
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`,
|
||||
`binding` and `visibility` (`"default"` or `"protected"`; `"unknown"` in older
|
||||
historical Evidence where visibility was not recorded). ELF string tables do
|
||||
not declare a text encoding. Names that
|
||||
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`
|
||||
and `binding`. ELF string tables do not declare a text encoding. Names that
|
||||
contain non-ASCII bytes are decoded as UTF-8 when those bytes form valid
|
||||
UTF-8; `name_utf8_decodable` records whether the non-ASCII name bytes are
|
||||
valid UTF-8 and `name_bytes_base64` retains those exact bytes (`null` for
|
||||
@@ -438,32 +341,20 @@ rea inspect-native-exports ./libexample.so --detail summary --evidence_output ./
|
||||
report null for the newly added encoding fields. `version` is null for an
|
||||
unversioned symbol. A versioned
|
||||
symbol reports its GNU index, a name resolved from `.gnu.version_d` or
|
||||
`.gnu.version_r`, the corresponding `name_bytes_base64` and
|
||||
`name_utf8_decodable` facts, and the `hidden` flag from `.gnu.version`.
|
||||
Version names use the same UTF-8-when-valid, otherwise byte-preserving
|
||||
Latin-1 display mapping as symbol names; ELF does not declare either
|
||||
string's text encoding. Older retained results report null for the added
|
||||
version-name encoding fields. `"unknown"`
|
||||
`.gnu.version_r`, and the `hidden` flag from `.gnu.version`. `"unknown"`
|
||||
means the symbol-version table could not be read. A hidden version is not
|
||||
the default version that a by-name `dlsym` lookup selects.
|
||||
Definition names take precedence when both tables name the same index.
|
||||
|
||||
Unreadable names, out-of-range name ordinals and unreadable forwarders are
|
||||
listed in `issues`, their entries are not reported, and coverage is `partial`.
|
||||
PE export names are bounded null-terminated byte strings and may continue
|
||||
across adjacent file-backed section mappings when the virtual bytes are
|
||||
unambiguous. Gaps, zero-fill and conflicting mappings remain unreadable names.
|
||||
Unresolved symbol versions keep their rows: the version name is null when no
|
||||
definition or requirement names the index, and `version` is
|
||||
`"unknown"` when the version table cannot be read; coverage is likewise
|
||||
`partial`. A dynamic symbol table without its null entry is malformed. An ELF
|
||||
without section headers is `unsupported`. With section headers, a missing
|
||||
`SHT_DYNSYM` section is also `unsupported` when a `PT_DYNAMIC` segment is
|
||||
observed, including a directory containing only the inactive null section.
|
||||
Enumerating dynamic symbols from that segment is not implemented. A present
|
||||
section directory with neither a dynamic symbol table nor a dynamic segment
|
||||
reports a complete empty inventory. Mach-O exports are reported by
|
||||
`inspect_macho`.
|
||||
without section headers is `unsupported`:
|
||||
enumerating dynamic symbols from the dynamic segment is not
|
||||
implemented. Mach-O exports are reported by `inspect_macho`.
|
||||
|
||||
`max_file_bytes` defaults to 256 MiB (maximum 512 MiB; CLI `--max-file-bytes`).
|
||||
Export directories in PE headers are read like those in sections. An export or
|
||||
|
||||
+21
-191
@@ -12,20 +12,13 @@ For request/response bodies and traffic debugging, see
|
||||
[Fiddler Everywhere captures and MCP](web-network-captures.md#reading-fiddler-captures-saz).
|
||||
Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate MCP server.
|
||||
|
||||
HTTP discovery and browser WebSocket inputs require an explicit port. An
|
||||
explicit default `:80` remains valid and is retained in the canonical endpoint;
|
||||
omitting the port remains invalid. HTTP `/json/list` discovery preserves usable
|
||||
siblings when optional metadata is malformed: attachment becomes unknown and
|
||||
row/field diagnostics appear in `limitations`. Target identity and socket
|
||||
authority are validated independently before selection and connection.
|
||||
|
||||
## Shipped surfaces
|
||||
|
||||
- `list_browser_targets` / `rea list-browser-targets` discovers all eligible page targets at the endpoint by default, with an optional exact-origin filter.
|
||||
- `inspect_web_page` / `rea inspect-web-page` captures DOM structure, accessibility nodes, scripts, resources, safe response/DOM metadata, attach-window network and console metadata, WebSocket frame sizes, workers, quota, and optionally selected storage key names, redacted storage-content fingerprints, or script sources.
|
||||
- `analyze_web_bundle` / `rea analyze-web-bundle` parses selected script artifacts without execution and derives chunk edges, route and endpoint candidates, vendor fingerprints, static WebMCP declarations, and optionally source-map/original-source evidence.
|
||||
- `observe_web_session` / `rea observe-web-session` arms an observation window of the requested duration for an external user action and records ordered reload, SPA navigation, redirect, failure, lifecycle, and target-termination metadata.
|
||||
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return observed in-scope registrations and their complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. An omitted optional `inputSchema` is a valid parameterless registration, represented by null schema and digest. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. Declared `readOnly`, `untrustedContent`, `consequential`, and `autosubmit` annotation hints are reported per registration: `true`/`false` are declared values, `null` means the declaration did not provide the hint in that capture, and a missing `consequential` key means the hint was not recorded, either in a historical capture or because the observed annotation was invalid. Consult discovery completeness and exclusions to distinguish these cases. Malformed annotation values or registration stacks retain the registration with incomplete coverage and an `invalid_protocol_value` exclusion. Other malformed protocol payloads can fail discovery explicitly. REA never exposes `WebMCP.invokeTool`.
|
||||
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. Declared `readOnly`, `untrustedContent`, `consequential`, and `autosubmit` annotation hints are reported per registration: `true`/`false` are declared values, `null` means the declaration did not provide the hint in that capture, and a missing `consequential` key means the hint was not recorded, either in a historical capture or because the observed annotation was invalid. Consult discovery completeness and exclusions to distinguish these cases. Malformed annotation values or registration stacks retain the registration with incomplete coverage and an `invalid_protocol_value` exclusion. Other malformed protocol payloads can fail discovery explicitly. REA never exposes `WebMCP.invokeTool`.
|
||||
|
||||
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
|
||||
|
||||
@@ -52,11 +45,7 @@ observations may omit the coverage field; treat omission as `unknown` rather
|
||||
than inferring completeness from the array.
|
||||
|
||||
Passive DOM comparison checks complete node inventories and complete frame
|
||||
parent/child structure independently. Node value lengths are comparable only
|
||||
when both nonempty snapshots declare the same `node_value_length_unit`.
|
||||
Missing or different units leave length comparison unknown; independent node
|
||||
or frame structural differences can still prove change. Empty inventories have
|
||||
no value lengths to compare. Frame IDs and sibling enumeration order
|
||||
parent/child structure independently. Frame IDs and sibling enumeration order
|
||||
are capture-local and do not establish a change; duplicate IDs, missing parents,
|
||||
and cyclic ancestry remain unknown. A complete frame difference can still prove
|
||||
change when DOM nodes are incomplete, and vice versa. Partial DOM or metadata
|
||||
@@ -65,57 +54,6 @@ from a protocol omission. Rejected DOM facts also mark derived URL and agent-hin
|
||||
metadata incomplete, while valid facts from other nodes and documents remain
|
||||
available in the capture.
|
||||
|
||||
Page network and console coverage starts independently at the first observed
|
||||
event from that domain or its successful enable acknowledgment. The capture's
|
||||
lifecycle start is earlier and does not establish domain readiness. Events
|
||||
received before the enable reply remain part of the observation.
|
||||
|
||||
Worker output is a final target-inventory snapshot. `attached` describes any
|
||||
debugging client, not an REA-owned observing session. The selected page-session
|
||||
collector does not collect worker-session requests, console events, or scripts,
|
||||
and workers that disappeared before the snapshot may be missing; worker
|
||||
coverage is therefore attach-limited. Missing ownership metadata is reported as
|
||||
`unattributed_target` with unavailable worker attribution, separately
|
||||
from a proven foreign owner excluded as `out_of_target_scope`. A shared origin
|
||||
alone establishes neither relationship.
|
||||
|
||||
Passive comparison keeps lookup identity, cross-capture correspondence, content
|
||||
evidence, and inventory coverage separate:
|
||||
|
||||
| Observation | Comparison outcome |
|
||||
| ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- |
|
||||
| Renamed WebMCP tool keys/frame IDs with one matching name, owner origin, and frame URL | Compare the registration content; local IDs do not prove change |
|
||||
| Duplicate WebMCP owners with the same name, origin, and frame URL | Unknown correspondence; complete group count differences still prove change |
|
||||
| Matching script inventory keys with retained source digests | Compare the retained SHA-256 values even without a producer hash |
|
||||
| One corresponding script source is unavailable, or neither source nor producer hash is available | Source equality remains unknown; inventory changes and other proven differences survive |
|
||||
| A partial HTTP response group is a subset of the complete peer | Unknown; omitted repeated responses can explain the difference |
|
||||
| An observed HTTP response fingerprint is absent from the complete peer group | Changed; omissions cannot explain that retained response |
|
||||
| A WebSocket connection or frame inventory is incomplete | Do not align retained payloads by position |
|
||||
|
||||
Storage fingerprints decode supported CDP primitive/JSON values before hashing.
|
||||
Debugger descriptions, previews, and remote references do not represent storage
|
||||
content. Actual JSON properties with those names remain content. Reference-only
|
||||
objects, unsupported values, and opaque cookie partition identities remain
|
||||
incomplete. Previously exported fingerprints are not recomputed; recapture both
|
||||
sides when comparing captures produced across this decoder change.
|
||||
IndexedDB schema fingerprints with `projection_version: 2` include the observed
|
||||
database version and object-store definitions. A missing, malformed, or imprecise
|
||||
version leaves the schema fingerprint incomplete while retaining observed store
|
||||
facts. Historical fingerprints without this marker covered object stores only:
|
||||
matching historical hashes cannot establish unchanged database versions, and a
|
||||
historical hash is not compared with a version-bearing hash. Recapture both
|
||||
sides for a complete IndexedDB version comparison; other proven storage changes
|
||||
remain reportable.
|
||||
|
||||
Script comparison groups observations by reported URL and compares unordered
|
||||
content samples. Producer inventory keys include optional metadata, so a changed
|
||||
key cannot by itself prove script removal or addition. Missing module, length,
|
||||
language or source-hash facts remain unknown; shared known values and complete
|
||||
retained source digests can still prove a group change. Partial inventories use
|
||||
the same omission rule as repeated HTTP samples. WebMCP dropped-tool counts also
|
||||
prevent equality, and two incomplete owner inventories cannot prove tool
|
||||
correspondence merely from their different retained registrations.
|
||||
|
||||
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
|
||||
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
|
||||
- The CLI comparison commands accept each capture, screenshot artifact or normalization policy as inline JSON or a JSON file path. Use files for screenshot artifacts and scenario captures, whose embedded PNG bytes can exceed the host's command-line length limit.
|
||||
@@ -159,11 +97,6 @@ preserve these authority and lifecycle differences. They share one exclusion
|
||||
vocabulary (`BrowserExclusionReason`): every denial reason maps onto it, even
|
||||
where a wire schema keeps a narrower historical bucket.
|
||||
|
||||
Passive page and Inspector capture enable the `Debugger` domain. A target's
|
||||
`debugger` statement can then pause execution; these observation paths do not
|
||||
suppress pauses with `Debugger.setSkipAllPauses` or resume the target. Their
|
||||
lack of explicit pause commands does not guarantee uninterrupted execution.
|
||||
|
||||
CDP attachment is not a guarantee that the browser preserves all state.
|
||||
Chromium can clear another client's `navigator.platform` emulation when REA's
|
||||
page session disconnects, even if that session sent no emulation commands.
|
||||
@@ -334,11 +267,7 @@ read as registration state only after domain availability is confirmed.
|
||||
Discovery is attach-only: the command needs a reachable loopback CDP endpoint
|
||||
and a target ID from `list_browser_targets`, and it never navigates, reloads,
|
||||
or evaluates the page. Observe the page in the state you want to measure;
|
||||
registered tools reflect what the protocol delivered. If an in-scope subframe
|
||||
already exists when discovery attaches, `webmcp_tools` remains attach-limited:
|
||||
Chromium replay may omit its older registrations. Later live registrations
|
||||
retain their frame ownership but do not establish recovery of those missing
|
||||
descriptors. REA does not reload, evaluate, or invoke tools to fill that gap.
|
||||
registered tools reflect the document as loaded.
|
||||
|
||||
Accessibility names/descriptions, console primitive text, JSON body shapes, WebSocket shapes, script content, and storage key names are selected independently:
|
||||
|
||||
@@ -380,20 +309,6 @@ are reported in the result.
|
||||
Call `list_browser_targets` first because target IDs are browser-instance-specific. REA rechecks the selected target's current type and origin immediately before attaching. Without an origin filter, discovery lists all eligible pages; an individual capture defaults to the selected page's current origin.
|
||||
|
||||
REA establishes a main-frame origin boundary before enabling Runtime, Debugger, or Network observation, rechecks it before document capture, and validates it again afterward. If the main frame navigates during final capture, REA discards the mixed result and returns `target_changed`.
|
||||
An observed navigation also takes precedence over failures from stale
|
||||
document reads, such as a script ID invalidated by reload; cancellation remains
|
||||
cancellation. Retry after navigation settles, or refresh target discovery if
|
||||
the selected page was replaced. Origin-scope refusals from page inspection,
|
||||
runtime observation, screenshots, and WebMCP discovery explain the permitted
|
||||
HTTP(S) origin scope and direct the caller to select a permitted document or
|
||||
explicitly revise `allowed_origins`. Page capture also preserves an observed
|
||||
main-frame scope violation when a later document read fails. An unresolved
|
||||
initial main frame does not establish an origin violation; its bounded startup
|
||||
retries retain a distinct refusal.
|
||||
|
||||
Screenshot capture requires a valid main-frame identity. Malformed identity
|
||||
produces a protocol failure; a readable child frame cannot substitute for the
|
||||
main frame.
|
||||
|
||||
The web-session timeline uses CDP monotonic seconds where the event producer
|
||||
provides a comparable timestamp; absent or invalid timestamps are `null` rather
|
||||
@@ -404,37 +319,15 @@ contextual evidence with `timestamp: null`, accompanied by a limitation stating
|
||||
that it does not establish activity in the selected window. Main-frame scope
|
||||
enforcement still applies to contextual navigation events. Network redirect
|
||||
events retain their paired prior-hop response context when the timestamped
|
||||
next-request event falls inside the window. A redirect row's optional
|
||||
`redirect_response` contains the prior response `url`, CDP-reported `status`,
|
||||
and `scope`. The URL has transport credentials removed; URL and status are
|
||||
withheld unless its origin is approved. Missing or malformed response URLs have
|
||||
null scope and add `invalid_protocol_value` timeline coverage; a valid URL with an
|
||||
unsupported protocol has `unsupported` scope. Outside-policy and unsupported
|
||||
responses mark the timeline as policy-filtered with `disallowed_origin` or
|
||||
`unsupported_url`. Missing status remains null rather than zero. A supplied
|
||||
non-numeric status is also null and records `invalid_protocol_value` coverage;
|
||||
the other valid response facts remain available. Historical rows may omit this
|
||||
field; omission does not establish that no prior response existed. Response
|
||||
headers and bodies are discarded. The row timestamp belongs to the next request,
|
||||
not an exact prior response time. The pre-arm timeline buffer is bounded
|
||||
next-request event falls inside the window. The pre-arm timeline buffer is bounded
|
||||
by original CDP message bytes. If clock sampling completes after that budget is
|
||||
exhausted, the operation returns `payload_limit` instead of a partial window.
|
||||
Cancellation and clock-request failures retain their own outcomes. Session
|
||||
`load_failed` rows require a previously observed main-frame `Document` request;
|
||||
subresources, child-frame requests and failures without admitted request ownership
|
||||
are excluded. Redirects reuse their request identity, and completed/failed requests
|
||||
release it. Active request identity metadata uses the existing 8 MiB retained-event
|
||||
allowance; exhaustion fails the observation explicitly instead of guessing scope.
|
||||
Cancellation and clock-request failures retain their own outcomes.
|
||||
|
||||
## Data minimization
|
||||
|
||||
Response metadata parses repeated CDP header field lines independently. The
|
||||
Link projection retains their declarations in order. `csp.directives` lists the
|
||||
directives of every CSP policy in order, across field lines and comma-separated
|
||||
policies, without recording policy boundaries, so a directive name can repeat
|
||||
across policies. As in CSP3, a repeated name within one policy is ignored.
|
||||
Single-valued policy fields combine their lines as one comma-separated value,
|
||||
as RFC 9110 specifies. Each link keeps its target in `href` and
|
||||
The Link projection parses repeated CDP header field lines independently and
|
||||
retains their declarations in order. Each link keeps its target in `href` and
|
||||
its relationship context in `context_url` with a separate `context_scope`;
|
||||
absent `anchor` uses the response URL, while a relative `anchor` resolves
|
||||
against that URL. Anchored links remain in the projection, but a `link_rel`
|
||||
@@ -451,31 +344,20 @@ complete Structured Field Item, including parameters such as `report-to`.
|
||||
These are declarations, not an assessment of the document's effective policy;
|
||||
absent, malformed or non-token items project as `null`.
|
||||
|
||||
Console entries, WebSocket frame summaries, and selected request/response JSON
|
||||
shape outputs share one 8 MiB retained-observation allowance per capture. Shape
|
||||
inference admits expanded path bytes before allocating those paths, including
|
||||
escaped and Unicode property names. Replacing or removing a request's shapes
|
||||
refunds their retained charges. A refused body shape preserves network metadata
|
||||
and any captured peer shape with `resource_budget_exhausted` coverage; a refused
|
||||
WebSocket frame is counted as dropped rather than reclassified as non-JSON text.
|
||||
The allowance has no runtime override. Protocol messages and JSON input parsing
|
||||
remain subject to their separate transport admission; this output allowance does
|
||||
not bound the whole process heap.
|
||||
|
||||
REA retains local observation data and removes only URL userinfo credentials, explicitly declared scenario secrets, and values from structured credential/storage fields:
|
||||
|
||||
- URL observations preserve the complete local URL, including query values, fragments, duplicate parameters, and their order. Only username/password userinfo is removed; malformed original URL strings remain available with `origin: null`.
|
||||
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values. `dom.node_value_length_unit` is `utf16-code-units` for web observations and applies to every node's `node_value_length`; it counts JavaScript string code units, not Unicode code points or UTF-8 bytes. Electron observations declare `utf8-bytes` instead. Historical captures without the unit leave it unknown; reading them does not infer a unit or convert their stored lengths.
|
||||
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values.
|
||||
- Supported DOM URL attributes are projected separately as resolved destinations. Empty `action` on HTML or XHTML forms and empty `formaction` on HTML or XHTML submit controls resolve to the document URL; other relative URL attributes resolve against the captured document base.
|
||||
- Accessibility structure and reported roles are retained by default, while names and descriptions require `include_accessibility_text: true`. The browser protocol may omit an individual role; that role remains unknown for comparison. Other observed accessibility differences can still establish a change.
|
||||
- Accessibility structure and roles are retained by default, while names and descriptions require `include_accessibility_text: true`.
|
||||
- Network observations retain method, status, MIME type, size, type, initiator stack location, and complete URLs with userinfo credentials removed. Redirects reported by CDP are attached as ordered `redirects` on the one request ID, with the prior request URL, CDP response URL, method/type, response status/MIME/encoded length, request timestamp, and the next `requestWillBeSent` event timestamp. CDP delivers `redirectResponse` with that next request event, so the latter is an event boundary rather than an exact response timestamp. Redirect response headers and bodies are discarded; hops outside the selected origins are discarded with the request chain. Headers are discarded after an allowlisted projection of length/encoding, structured CSP/Link/policy fields, and untrusted agent hints. Cookies and authorization headers are never retained.
|
||||
- Request/response bodies are not requested or parsed by default. When selected, only allowed-origin JSON media types are read, converted immediately to complete value-free typed paths, types and observation counts, and discarded. Each path segment is either `{ "kind": "property", "name": "raw property name" }` or `{ "kind": "array-element" }`; array-element types include primitives and nested arrays. Literal `*` properties remain distinct from array elements. `Network.getResponseBody` is never sent unless JSON body shape capture is selected. A JSON-typed request body that CDP reports through `hasPostData` but omits from `postData`, or that cannot be parsed as JSON, makes `json_body_shapes` unavailable. That request's `body_shapes.status` is `partial`, not `included`, when only its response shape was captured. Base64 response bytes must be valid UTF-8 before JSON interpretation; invalid bytes leave the response shape unavailable and preserve any observed request shape and network metadata. Producer-decoded strings and genuine U+FFFD characters retain their reported meaning. Comparison uses body shapes only when both captures selected them, and compares request/response shapes only where both observations provide those facts. Unavailable or truncated selected coverage makes equivalence unknown; observed differences still report changed.
|
||||
- Request/response bodies are not requested or parsed by default. When selected, only allowed-origin JSON media types are read, converted immediately to complete value-free typed paths, types and observation counts, and discarded. Each path segment is either `{ "kind": "property", "name": "raw property name" }` or `{ "kind": "array-element" }`; array-element types include primitives and nested arrays. Literal `*` properties remain distinct from array elements. `Network.getResponseBody` is never sent unless JSON body shape capture is selected. A JSON-typed request body that CDP reports through `hasPostData` but omits from `postData`, or that cannot be parsed as JSON, makes `json_body_shapes` unavailable. That request's `body_shapes.status` is `partial`, not `included`, when only its response shape was captured. Comparison uses body shapes only when both captures selected them, and compares request/response shapes only where both observations provide those facts. Unavailable or truncated selected coverage makes equivalence unknown; observed differences still report changed.
|
||||
For normalized passive inputs with complete request inventories on both sides,
|
||||
request additions and removals remain observable even when selected body shapes,
|
||||
response metadata, or WebSocket coverage are unavailable. HTTP request and
|
||||
WebSocket inventory coverage are evaluated separately; complete HTTP coverage
|
||||
cannot prove a missing WebSocket connection. A missing request in a limited inventory cannot prove absence; a retained
|
||||
request absent from a complete peer can prove a change. This comparison rule does not make live CDP's
|
||||
cannot prove a missing WebSocket connection. Limited request inventory still
|
||||
prevents request-absence claims. This comparison rule does not make live CDP's
|
||||
attach-limited inventory complete or establish body-shape equality.
|
||||
An unavailable HTTP resource type also leaves request identity incomplete:
|
||||
observations with the same method and URL remain possible matches across types,
|
||||
@@ -490,17 +372,13 @@ REA retains local observation data and removes only URL userinfo credentials, ex
|
||||
uses `websocket_connections` and `websocket_frames`. Drops in other event kinds
|
||||
and the aggregate total do not by themselves limit HTTP inventory. Proven
|
||||
modifications remain reportable despite incomplete inventory.
|
||||
Repeated HTTP samples retain multiplicity. A partial group that is contained
|
||||
in a complete peer cannot prove modification, even when the omitted response
|
||||
had different content. A retained sample absent from the complete peer can
|
||||
prove modification. Two partial groups cannot establish response
|
||||
correspondence from their different retained samples alone.
|
||||
Repeated identical HTTP content samples establish multiplicity changes only
|
||||
with complete request inventories; limited inventories retain known content
|
||||
comparisons without interpreting duplicate sample counts as changes.
|
||||
WebSocket stream inventories and connection counts likewise require complete
|
||||
WebSocket coverage. Retained frame positions are compared only with complete
|
||||
connection/frame inventories; matching direction/opcode sequences alone do
|
||||
not prove alignment after drops. Redirect resource types use only shared known metadata:
|
||||
WebSocket coverage. Redirect resource types use only shared known metadata:
|
||||
an omitted type leaves equality unknown rather than proving a modification.
|
||||
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. Source locations are CDP's zero-based lines and UTF-16 columns. A negative column, which Chrome reports on the first line of some inline scripts, is kept as reported and does not locate a column in the source URL; an invalid line or a non-integer column is `null` (WebMCP `registration_source` is `null`) and is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
|
||||
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
|
||||
- WebSocket observations retain direction, opcode, and payload byte length. A missing or malformed CDP opcode is returned as `null`, with opcode-dependent payload byte length and shape also `null`, and marks `websocket_frames` incomplete. Missing or wrong-typed payload data is not treated as empty; its byte length is `null`, and a text payload shape is unknown. Numeric opcode `0` and an observed empty payload remain distinct from missing data. Opcode is nullable in the public inspection schema, so clients must handle unknown protocol values. When shape capture is selected, text messages are classified as text or complete value-free JSON shape; binary bytes, hashes, prefixes, and raw payloads are never retained.
|
||||
- Storage observations always redact values. Key names, IndexedDB names, and cache names require `include_storage_keys`; stable content fingerprints require the additional `include_storage_fingerprints` selection. Cookie fingerprints cover cookies applicable to the current main-frame URL, including its path, rather than every cookie path on that origin. Other storage remains scoped to the origin. Cache fingerprints hash all decoded response bytes, including bodies above 64 KiB, subject to the [64 MiB per-message CDP limit](#request-boundary). Missing or undecodable cache bodies and partial IndexedDB remote objects make fingerprint coverage incomplete, so identical observations remain `unknown`.
|
||||
- Unavailable or malformed storage inventories mark `storage_keys` unavailable, even without fingerprints. Comparisons suppress inferred key additions and removals while preserving modifications proven by complete fingerprints on both sides. Previously exported captures are not retroactively corrected; recapture them to obtain these coverage markers.
|
||||
@@ -510,11 +388,6 @@ Frames, resources, scripts, events, and workers outside the selected origin scop
|
||||
Retained workers include validated opener-target and parent-frame IDs when CDP
|
||||
provides them; these relationships do not expand the exact-origin boundary or
|
||||
prove static module ownership.
|
||||
Workers without either relationship are withheld with an `unattributed_target`
|
||||
exclusion and `workers` in `unavailable_sections`: a same-origin URL does not
|
||||
establish which page owns a worker. A supplied relationship to another target or
|
||||
frame remains `out_of_target_scope`. Missing ownership also remains unknown when
|
||||
the worker URL is empty; withheld target details are not returned.
|
||||
|
||||
## Completeness and limits
|
||||
|
||||
@@ -591,12 +464,6 @@ An explicitly empty MIME type is valid. The optional content size may be absent;
|
||||
an invalid supplied size remains unknown with incomplete resource coverage while
|
||||
the resource's other facts are retained. Zero and nonnegative fractional sizes
|
||||
are preserved.
|
||||
`resource_key` remains the exact observation identity used by script
|
||||
reconciliation. Capture comparison groups resources by URL, origin, type and
|
||||
MIME type, then compares known sizes within a group. If a matched group contains
|
||||
an unknown size, comparison reports unknown rather than treating the observation
|
||||
as a resource addition or removal; complete inventory coverage still establishes
|
||||
when an entire group is absent.
|
||||
|
||||
CDP discovery has a 10-second response deadline. HTTP discovery documents
|
||||
use an 8 MiB budget. Browser-socket discovery (`Browser.getVersion`,
|
||||
@@ -626,44 +493,10 @@ Network and console coverage starts only after REA attaches and enables the rele
|
||||
|
||||
Source maps are not fetched by inspection. Bundle analysis fetches them only when `fetch_source_maps` is selected, with exact-origin redirect checks and no credentials, cookies, or referrer. Each fetch operation uses `source_map_fetch_timeout_ms` (default 30000) and a 64 MiB aggregate streamed-response budget across redirects. The byte budget has no runtime override. A fetch overrun is an item limitation, not a thrown analysis timeout. A deadline or byte-limit failure remains explicitly unavailable and makes bundle completeness partial. Source-map decoding also bounds raw resolved identities to 64 MiB and aggregate inline evidence to 32 MiB. Failed maps preserve their failure context when it fits; if another context cannot fit, collection stops with the full requested count, the returned processed count, and an explicit unknown-results limitation. Retained mappings and original sources are returned inline. See [source-location limits](web-source-location.md) for the structural decoder bounds. Original source artifacts with `.ts`, `.tsx`, `.mts`, or `.cts` suffixes carry `text/typescript`; other source names retain `text/javascript`.
|
||||
|
||||
A fully read response that fails UTF-8, JSON, mapping, or structural validation
|
||||
retains `artifact: null`, empty original-source, module-edge, and mapping
|
||||
arrays, and an explicit failure limitation. Invalid content is `invalid`;
|
||||
decoder or structural capacity refusal is `fetch_failed`. Its optional
|
||||
`response_identity` records the complete body's SHA-256 digest, byte count,
|
||||
and `fetch-response-body-bytes` representation without retaining malformed
|
||||
text or charging that text to the 32 MiB output budget. These are bytes
|
||||
delivered by Fetch after HTTP content decoding and before UTF-8 decoding, not
|
||||
HTTP wire bytes. The identity is absent when the body was not fully read, and
|
||||
older observations may omit it. A response whose text cannot fit the existing
|
||||
output budget is likewise `fetch_failed` with text omitted and completed-body
|
||||
identity; JSON parsing and source-map decoding are not attempted. Successful and partial maps keep their
|
||||
full text artifacts. Their original text, including an initial BOM or XSSI
|
||||
line, remains distinct from the parsing view that removes that framing.
|
||||
Collection reads the body once and does not raise its response or output limits.
|
||||
|
||||
When fetching is selected, source-map `excluded` facts preserve the capture
|
||||
ledger's reasons and counts for declarations rejected before download. No
|
||||
disallowed URL details are retained or fetched. `requested` and `processed`
|
||||
count the admitted fetch requests; exclusions are counted separately. Excluded
|
||||
declarations make the collection unavailable, or partial when maps were retained.
|
||||
An empty collection with no declarations remains `not_requested`.
|
||||
|
||||
If the caller cancels while later source maps are being fetched, the failure's
|
||||
partial Evidence retains the captured page and every completed map item. Its
|
||||
`normalized_result.kind` is `web-bundle-acquisition` and
|
||||
`analysis_status` is `not_performed`: no script-analysis result is claimed.
|
||||
`source_maps.requested` remains the admitted request count,
|
||||
`source_maps.processed` counts completed items, and any unprocessed requests
|
||||
have an explicit unknown-results limitation. Capture exclusions remain in the
|
||||
same collection. The cancellation error keeps the caller's exact reason;
|
||||
an already observed response resource or cleanup failure keeps its own error
|
||||
category and cleanup owner.
|
||||
|
||||
Source-map items retain the script's `declared_url` separately from the observed
|
||||
`effective_url`. Successful maps use the latter as their source-resolution base.
|
||||
Failed fetches retain the last response URL observed within the approved origins;
|
||||
`effective_url` is `null` when no response was observed. Query parameters are
|
||||
source-map items retain the script's `declared_url` separately from the observed
|
||||
`effective_url`. successful maps use the latter as their source-resolution base.
|
||||
failed fetches retain the last response URL observed within the approved origins;
|
||||
`effective_url` is `null` when no response was observed. query parameters are
|
||||
preserved, and both URLs count toward the inline evidence budget.
|
||||
Current fetches emit a string or null; historical observations may omit
|
||||
`effective_url` without changing their stored content or Evidence identity.
|
||||
@@ -678,9 +511,6 @@ validation.
|
||||
## Non-goals and threat model
|
||||
|
||||
The browser provider deliberately does not expose generic CDP commands, `Runtime.evaluate`, WebMCP invocation, navigation, input, downloads, page closure, or browser closure. Screenshot capture is the sole pixel surface and returns an image only when requested. REA disables the domains it enabled, detaches the target session, and closes only its own WebSocket.
|
||||
Discovery and attachment errors remain primary if closing REA's socket also
|
||||
fails. The cleanup failure is reported alongside the original command failure;
|
||||
it cannot replace the operation diagnostic.
|
||||
|
||||
This feature is not a browser sandbox or network containment mechanism. The attached page and browser continue running with their existing privileges and may make external requests independently of REA. CDP gives deep access to the selected browser profile, so use a dedicated profile, include only origins you intend to inspect, and treat other same-user processes as outside this boundary.
|
||||
|
||||
|
||||
@@ -10,39 +10,27 @@ liveness deadlines. `INPUT_JSON` may be inline JSON or a JSON file path.
|
||||
The minimal request contains only browser launch/connect selection, `start_url`,
|
||||
and at least one `actions` entry. Launch `headless` defaults to `true` and can
|
||||
be set to `false` where the host supports a visible browser. Navigations follow
|
||||
the destinations named by the scenario and the page's ordinary browser requests,
|
||||
and [request steps](#request-steps) send only to the start URL's origin;
|
||||
there is no separate origin allowlist. Request steps use a temporary dispatch
|
||||
gate for cancellation; ordinary page requests retain their browser behavior. Environment
|
||||
the destinations named by the scenario and the page's ordinary browser requests;
|
||||
there is no separate origin allowlist or request-interception layer. Environment
|
||||
settings (including service workers blocked by default), empty storage, and a
|
||||
final sanitized URL capture are supplied by default. Set
|
||||
`environment.service_workers` to `allow` when the application requires them.
|
||||
The connected service-worker policy owner retains its exact CDP transport
|
||||
through discovery, attachment and restoration. Failed admission preserves its
|
||||
primary diagnostic when socket cleanup also fails; later policy cleanup retries
|
||||
that retained transport and clears cleanup uncertainty only after close succeeds.
|
||||
For attached browsers, `block` rejects captures when the selected page or its
|
||||
frames are controlled by a service worker or already have a registration,
|
||||
including after navigation. Policy checks bracket each artifact capture, including
|
||||
the final step. If an external worker takes control while artifacts are being
|
||||
captured, REA rejects the capture and retains the completed artifacts in
|
||||
`details.diagnostics.collected_artifacts`; they do not establish policy-compliant
|
||||
completion. Existing caller registrations are never removed.
|
||||
including after navigation. Existing caller registrations are never removed.
|
||||
Navigation can run destination code before a newly discovered worker is rejected;
|
||||
that rejection does not roll back browser effects. REA removes its owned
|
||||
future-document scripts, restores live frame methods, then disconnects.
|
||||
Add secret declarations, storage seeds, and additional artifact or event
|
||||
capture only when needed. Missing environment-backed secrets fail as input errors before browser acquisition, naming the declaration and required variable. An explicitly set empty string is a resolved value. Action, secret, and storage counts are not capped.
|
||||
capture only when needed. Action, secret, and storage counts are not capped.
|
||||
Duration, action, and navigation timeouts remain separate from CDP command
|
||||
deadlines. `command_timeout_ms` (default 20000; omit it to keep that default
|
||||
and leave it out of Evidence) bounds launch startup, the connect-over-CDP
|
||||
handshake, target lookup, environment commands, and storage CDP commands,
|
||||
including replies from attached child targets. It also bounds the initial
|
||||
`start_url` navigation through the load event; a redirect or replacement
|
||||
navigation cannot leave startup waiting indefinitely. Set it in the scenario JSON.
|
||||
including replies from attached child targets. Set it in the scenario JSON.
|
||||
`capture-browser-scenario` has no `--command-timeout-ms` option.
|
||||
It does not change action `timeout_ms`, `wait_for_timeout.duration_ms`, or
|
||||
action navigation. Service-worker restoration keeps a fixed 5000 ms grace and does
|
||||
navigation. Service-worker restoration keeps a fixed 5000 ms grace and does
|
||||
not inherit a raised `command_timeout_ms`. CDP discovery stays on its own
|
||||
fixed deadline. When service workers are blocked, the registration check
|
||||
before discovery has its own fixed 10000 ms deadline; `command_timeout_ms`
|
||||
@@ -51,30 +39,6 @@ covers endpoint discovery only. The following WebSocket handshake and
|
||||
`Target.attachToTarget` each use `command_timeout_ms`. `capture.limits` controls retained event and network-content
|
||||
budgets; it does not change those liveness deadlines.
|
||||
|
||||
## Validate a scenario before running it
|
||||
|
||||
`validate-browser-scenario INPUT_JSON` checks one scenario against the same
|
||||
`browserScenarioSchema` refinement that `capture_browser_scenario` admits, and
|
||||
stops there. `INPUT_JSON` may be inline JSON, a JSON file path, or `-` for
|
||||
standard input.
|
||||
|
||||
```bash
|
||||
rea validate-browser-scenario ./browser-scenario.json --json
|
||||
```
|
||||
|
||||
A valid scenario reports `{"valid":true,"actions":<count>,"secrets":<count>}`
|
||||
and exits 0. An invalid one exits 1 and reports the same field paths capture
|
||||
would report, so a correction made here is the correction capture needs:
|
||||
duplicate or reserved step IDs, duplicate secret identities and
|
||||
environment-variable names, undeclared secret references, request-step
|
||||
violations, and unknown fields.
|
||||
|
||||
Validation performs no browser discovery, launch, attachment, or navigation and
|
||||
constructs no provider. It does not read secret environment variables, so a
|
||||
scenario whose declared variables are unset still validates; the missing-value
|
||||
failure belongs to capture. A schema-valid scenario does not prove that the
|
||||
executable or target is available on the host.
|
||||
|
||||
## Minimal launch example
|
||||
|
||||
Save this as `browser-scenario.json`, replacing the executable path with an
|
||||
@@ -197,12 +161,6 @@ distinction: it closes and deletes provider-owned profiles, but only disconnects
|
||||
from an external CDP browser. Real-browser verification checks both outcomes and
|
||||
confirms that an attached external browser remains alive.
|
||||
|
||||
CDP connections leave Playwright's default-context overrides disabled. The
|
||||
selected page receives the declared scenario environment and temporary focus
|
||||
emulation, so actions can complete while that page is in the background.
|
||||
Its environment owner clears these overrides before disconnecting; other pages
|
||||
keep their media settings. Connecting does not create a bootstrap page.
|
||||
|
||||
That cleanup requires REA to remain running. Abrupt termination, including
|
||||
Windows process-tree termination or POSIX `SIGKILL`, can leave the launch
|
||||
profile and Playwright artifacts behind. A profile can retain cookies, history,
|
||||
@@ -236,9 +194,7 @@ language environments.
|
||||
HTTP(S) URLs may include ordinary query values and fragments directly. Use
|
||||
ordered structured query entries when a value needs to reference a declared
|
||||
environment-backed secret. Form, storage, and cookie values are either literal
|
||||
strings or declared secret references; request step headers can also read a
|
||||
cookie or local storage value from the page, and request steps and `goto`
|
||||
destinations can send [saved values](#saved-values). Raw URL userinfo credentials are rejected.
|
||||
strings or declared secret references. Raw URL userinfo credentials are rejected.
|
||||
Captured credential-header
|
||||
values are not retained, and declared secret values are redacted automatically.
|
||||
A secret may be declared solely to
|
||||
@@ -255,9 +211,8 @@ validation.
|
||||
|
||||
Cookies are installed before the initial navigation. Local storage entries are
|
||||
seeded once per browser storage partition; session storage entries are seeded
|
||||
once per origin in each instrumented top-level browsing context. Seeding occurs
|
||||
before the first document scripts for the selected page and its instrumented
|
||||
frames. An origin first reached by a later action
|
||||
once per origin in each top-level browsing context. Seeding occurs before the
|
||||
first document scripts for that area. An origin first reached by a later action
|
||||
receives its seed on that first visit. Application updates, deletions, and
|
||||
empty values then persist across later navigations in the same area; a new
|
||||
scenario starts with its own seeds. Partitioned frames can receive separate
|
||||
@@ -265,17 +220,6 @@ local storage seeds even when they share an origin. The pre-script mechanism
|
||||
uses CDP debugger instrumentation. The observer resumes only pauses whose top
|
||||
frame belongs to its private seed script; page breakpoints and `debugger`
|
||||
statements remain under the attached debugger client's control.
|
||||
If Chromium leaves the private isolated world's origin empty, REA looks up
|
||||
that exact frame's reported security origin before selecting a seed. It does
|
||||
not infer the origin from a frame URL or serialized storage key. Unresolved
|
||||
frame identity or security origin fails initialization rather than silently
|
||||
running with an unknown initial state.
|
||||
Popup discovery does not establish that its first script was paused by the
|
||||
selected page's storage instrumentation. When non-cookie seeds are declared
|
||||
and a popup is discovered, the capture retains completed observations and
|
||||
marks `storage` coverage missing, with equality ineligible. This applies even
|
||||
if popup lifecycle events were not selected. Popup first-script storage
|
||||
initialization is not currently guaranteed.
|
||||
|
||||
The result is Evidence with an initial state followed by one record per
|
||||
declared action. Each step reports action status, elapsed time, sanitized URLs,
|
||||
@@ -349,263 +293,6 @@ external browser without closing it. The request itself defines the operation;
|
||||
there is no additional REA grant step. The browser and host still enforce their
|
||||
own access and process rules.
|
||||
|
||||
## Request steps
|
||||
|
||||
A `request` step sends one HTTP request from the page with the page's own
|
||||
`fetch`, so the browser attaches the session's cookies and the request is
|
||||
recorded in the step's network events like any request the application makes.
|
||||
Use it to record an application's API, including writes, when driving the UI
|
||||
for each call would be slower or less exact than calling the endpoint the UI
|
||||
calls. It is the only action that changes application state without a user
|
||||
gesture, so run scenarios with write requests only against an instance whose
|
||||
data you may change. This scenario creates one item:
|
||||
|
||||
```json
|
||||
{
|
||||
"browser": {
|
||||
"mode": "launch",
|
||||
"executable_path": "/absolute/path/to/chromium"
|
||||
},
|
||||
"start_url": { "url": "http://127.0.0.1:3000" },
|
||||
"actions": [
|
||||
{
|
||||
"step_id": "create-item",
|
||||
"action": "request",
|
||||
"method": "POST",
|
||||
"path": "/api/items",
|
||||
"query": [
|
||||
{ "name": "notify", "value": { "source": "literal", "value": "false" } }
|
||||
],
|
||||
"headers": [
|
||||
{
|
||||
"name": "X-CSRFToken",
|
||||
"value": { "source": "cookie", "name": "csrftoken" }
|
||||
},
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": {
|
||||
"source": "local_storage",
|
||||
"key": "auth",
|
||||
"json_pointer": "/token"
|
||||
},
|
||||
"prefix": "Bearer "
|
||||
}
|
||||
],
|
||||
"json": { "title": "Scratch item 1" },
|
||||
"expected_status": [201],
|
||||
"timeout_ms": 10000
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- `method` is `GET`, `HEAD`, `POST`, `PUT`, `PATCH` or `DELETE`. `json` is the
|
||||
body, sent with `Content-Type: application/json` unless a header sets one; a
|
||||
`GET` or `HEAD` step cannot carry it.
|
||||
- `path` starts with one `/` and must resolve to the start URL's origin, which
|
||||
validation checks before the browser starts. When the step runs, the page must
|
||||
be on that origin, checked again in the page as the request is sent; add a
|
||||
`goto` step first if an earlier action left it. A query in `path` is sent as
|
||||
written (a fragment is not sent), and `query` entries are appended after it
|
||||
in order and may reference declared secrets.
|
||||
- Request steps follow no redirects: a redirect answer fails the step, and
|
||||
nothing is sent to the location it names, though a write the redirect
|
||||
answered may already have been applied. `expected_status` cannot name a 3xx
|
||||
status. Manual redirects can be opaque to the page and may not produce a
|
||||
response event; their status and `Location` can therefore be unavailable even
|
||||
when network events and header values were selected. Such captures report
|
||||
incomplete event coverage.
|
||||
- Header values are literals, declared secrets, a cookie the browser holds for
|
||||
the request URL (`{"source": "cookie", "name"}`, HttpOnly cookies included,
|
||||
whose value the page's scripts can then see while the request runs), or a
|
||||
local storage entry on the start origin (`{"source": "local_storage",
|
||||
"key"}`, with an optional `json_pointer` to a string or number in a stored
|
||||
JSON value). `prefix` is sent before the value. A value read from the page is
|
||||
redacted like a declared secret, as `[REDACTED:page:<step_id>.<header>]`,
|
||||
from records made after the step reads it; a response recorded earlier, such
|
||||
as the login response that returned a token, keeps the value. A value read
|
||||
again keeps its first marker, and a value shorter than 8 characters is not
|
||||
redacted, since redacting it would rewrite ids, ports and hosts throughout
|
||||
the capture. A value is redacted as stored and as `fetch` sends it, without
|
||||
surrounding spaces and tabs. A missing cookie, key or pointer fails the step.
|
||||
Reading a page value raises the estimated cost of redacting every later
|
||||
body, so the bodies kept within `max_network_body_allocation_bytes` (1 MiB
|
||||
by default) get smaller: after a 32-character `X-CSRFToken` value, a body over
|
||||
about 47 KiB is kept as `unavailable`. Raise `capture.limits` when the
|
||||
application's responses are larger.
|
||||
- A header can send Basic credentials, `{"source": "basic_credentials", "parts":
|
||||
[username, password]}`: the base64 of `username:password`, after a `prefix`
|
||||
such as `"Basic "` or with none, as some APIs take encoded credentials in
|
||||
their own header. Each part is a literal, a declared secret, a saved value, a
|
||||
cookie or a local storage entry, so a header a frontend builds from a login
|
||||
token can be sent too; the password is never a literal other than empty, since
|
||||
Evidence records literals as written, and a username with a colon fails the
|
||||
step, since Basic authentication ends the username at its first colon. The
|
||||
encoded value is redacted however short, as
|
||||
`[REDACTED:page:<step_id>.<header>]`, and parts read from the page are
|
||||
redacted as page values are.
|
||||
- Validation refuses a header name given twice in a step, header names a
|
||||
page's `fetch` cannot set, such as `Cookie`, `Origin`, `Host` or `Sec-*`, and
|
||||
`User-Agent`, which Chrome drops; it refuses line breaks and other ASCII
|
||||
control characters in literal values and prefixes, and the step fails before sending
|
||||
when a secret or page value holds them.
|
||||
- The step completes when the whole response has been read. With
|
||||
`expected_status`, any other status fails the step and the later actions are
|
||||
recorded as `cancelled`; without it, any response completes the step. A
|
||||
network error fails the step. `timeout_ms` covers reading the header values
|
||||
and the request: a step whose time runs out, or that is cancelled, before
|
||||
its request is sent sends nothing, but a request already sent can still reach
|
||||
the application after the step fails.
|
||||
An absolute deadline also covers browser evaluation queued behind page work.
|
||||
A temporary route checks cancellation again at network dispatch and is removed
|
||||
when the request evaluation settles. A page controlled by a service worker
|
||||
is refused because its fetch can bypass that dispatch gate; use the default
|
||||
blocked service-worker policy for request steps.
|
||||
|
||||
The request runs in the page's main world, so an application that wraps
|
||||
`fetch` sees it, and it goes through the same network observation as other
|
||||
page requests: select network content (bodies or header values) to record it.
|
||||
Captures with request steps state in `limitations` that the steps sent their
|
||||
declared requests through the page's fetch. A scenario without request steps
|
||||
is unchanged.
|
||||
|
||||
## Saved values
|
||||
|
||||
A step can keep a string or number from a JSON response, by name, and a later
|
||||
step can send it, so a scenario can read back, edit and delete an object whose
|
||||
id the application assigns:
|
||||
|
||||
```json
|
||||
{
|
||||
"browser": {
|
||||
"mode": "launch",
|
||||
"executable_path": "/absolute/path/to/chromium"
|
||||
},
|
||||
"start_url": { "url": "http://127.0.0.1:3000/items" },
|
||||
"actions": [
|
||||
{
|
||||
"step_id": "save-item-1",
|
||||
"action": "click",
|
||||
"timeout_ms": 10000,
|
||||
"locator": { "kind": "role", "role": "button", "name": "Save" },
|
||||
"save_response": {
|
||||
"method": "POST",
|
||||
"path": "/api/items",
|
||||
"save": [{ "name": "item_1", "json_pointer": "/id" }]
|
||||
}
|
||||
},
|
||||
{
|
||||
"step_id": "open-item-1",
|
||||
"action": "goto",
|
||||
"timeout_ms": 10000,
|
||||
"destination": { "url": "http://127.0.0.1:3000/items/{saved:item_1}" },
|
||||
"wait_until": "load"
|
||||
},
|
||||
{
|
||||
"step_id": "copy-item-1",
|
||||
"action": "request",
|
||||
"timeout_ms": 10000,
|
||||
"method": "POST",
|
||||
"path": "/api/items/{saved:item_1}/copies",
|
||||
"json": {
|
||||
"source_id": "{saved:item_1}",
|
||||
"title": "Copy of {saved:item_1}"
|
||||
},
|
||||
"expected_status": [201]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- A `request` step saves with `save: [{name, json_pointer, secret}]`, from its
|
||||
own response. A `click` or `press` step saves with `save_response: {method,
|
||||
path, save}`, from a newly observed matching response while the step waits.
|
||||
The recorder watches before the action and excludes requests already running,
|
||||
such as an earlier autosave. Method/path matching does not prove that the UI
|
||||
caused the request: use a path that distinguishes the intended operation from
|
||||
background polling. Multiple observed matches before binding fail as ambiguous;
|
||||
requests starting after this observation window cannot be ruled out. It matches the method and a path on the
|
||||
start origin as the browser spells it, so a path may name `My List` or
|
||||
`My%20List` (`{*}` matches one segment, and the query is ignored), and fails
|
||||
the step unless a 2xx JSON response arrives before its `timeout_ms`; one still
|
||||
being read when the step times out or is cancelled saves nothing. A pointer
|
||||
that names no string or number fails the step, so later steps never send a
|
||||
missing value, as does a response over 4,000,000 characters: a request step
|
||||
stops reading and aborts its fetch there. A UI step reads a body only when
|
||||
its decoded size is bounded at 12,000,000 bytes or less: its received size,
|
||||
or the declared Content-Length of one served from cache, times 1032 for each
|
||||
gzip or deflate coding, the most DEFLATE expands. Other codings and unknown
|
||||
or zero sizes are refused before materializing the body; decoded text over
|
||||
the character limit also fails. The UI action may already have applied.
|
||||
- `{saved:name}` sends a saved value in a request step's `path`, including its
|
||||
query, and in a `goto` destination's `url`, percent-encoded; a destination's
|
||||
structured `query` values are sent as declared. A saved value that makes a
|
||||
path segment `.` or `..`, alone or with the text beside it (including `%2e`),
|
||||
fails the step, since the URL would drop it. In a request step's `json` body,
|
||||
a string that is only `{saved:name}` becomes the saved value with the JSON
|
||||
type it was saved with, so a saved number id is sent as a number, and
|
||||
`{saved:name}` inside a longer string is filled as text. A header value or
|
||||
Basic credentials part sends one as `{"source": "saved", "name"}`. Other
|
||||
braces, such as `{id}`, are sent as written, and so is a percent-encoded
|
||||
`%7Bsaved:name%7D` anywhere but a `goto` URL, whose parsed form spells the
|
||||
braces that way.
|
||||
- Validation refuses a saved value that no earlier step saves. A later save of
|
||||
the same name replaces the value. An integer past 2^53, which JSON parsing
|
||||
would round to another number, and a number that is not finite, such as
|
||||
`1e400`, negative zero, or a value that underflows to zero, such as `1e-400`, fail the step
|
||||
that saves it.
|
||||
- A scenario keeps at most 4,000,000 characters of saved values at once,
|
||||
counting distinct retained redaction spellings, including trimmed variants,
|
||||
of values saved as secret. Replacing a binding releases its live value
|
||||
charge, while secret spellings remain protected for the scenario lifetime.
|
||||
A save past the budget fails before changing the binding.
|
||||
- A value saved with `secret: true` is redacted however short, as a declared
|
||||
secret is, as `[REDACTED:page:<step_id>.<name>]`. The selected source
|
||||
response is protected before retaining its body or header values, including
|
||||
when only headers were requested. If that body cannot be read safely, its
|
||||
body and header values remain unavailable. Before publishing the capture,
|
||||
the selected source transaction's earlier URL, request headers and request
|
||||
body are also protected using the admitted values. Unresolved source values
|
||||
withhold that transaction's affected metadata/content with an explicit
|
||||
limitation. Before publication, retained events are rechecked
|
||||
with the final marked-response values, including immediate follow-up requests
|
||||
and their response bodies, console/page errors, WebSocket text and event URL/name fields. Learned redaction facts are independent of source-identity readiness. Unknown source values withhold only the selected
|
||||
source transaction. Other saved values, such as ids, stay visible, since
|
||||
environments model them. The environment build warns about saved values
|
||||
named like credentials that are not marked secret
|
||||
(`credential-named-saved-values`).
|
||||
- Page-derived cookie/local-storage values and constructed Basic credentials
|
||||
remain redacted only from observations made after they were read/built.
|
||||
They do not retroactively rewrite earlier facts. The final event projection
|
||||
borrows only declared and response-marked patterns; promotion of an already
|
||||
held page value to an explicit response secret preserves its first marker.
|
||||
Projection state is session-local, references admitted pattern/byte owners,
|
||||
and is invalidated on admission or response classification changes.
|
||||
- Screenshots are withheld as `missing` from the start when the scenario plans
|
||||
any response save marked `secret: true`, page-derived or Basic authentication
|
||||
header values, or whenever nonempty sensitive values
|
||||
are already known. Raw pixels cannot be safely text-redacted, so the provider
|
||||
does not request pixels in those cases. This also protects earlier steps and
|
||||
partial failures without retaining an image to scrub later. Other requested
|
||||
artifacts remain available; affected screenshot/aggregate completeness is
|
||||
partial and is not eligible for equality.
|
||||
- Saved URLs and expanded request JSON are admitted before building their expanded strings or
|
||||
serializing. Its allowance is the smaller of Node's string limit and
|
||||
one sixteenth of current V8 available heap bytes, allowing for the expanded
|
||||
tree, serialization, Playwright transport and browser-side copies. Size
|
||||
accounting includes JSON escaping and repeated references; each distinct
|
||||
saved value's size is measured once per request. An oversized request fails
|
||||
before browser dispatch; the complete request envelope, including repeated
|
||||
header values, is checked as well. URL encoding uses a conservative
|
||||
nine-characters-per-UTF-16-unit allowance before encoding a saved value.
|
||||
Reduce the payload/repetitions, or raise the parent
|
||||
Node heap and restart; this is a conservative allocation estimate, not a
|
||||
measurement or guarantee of browser/process-family memory.
|
||||
- Evidence records a request step's empty `save` list as absent, so a scenario
|
||||
that saves nothing records the same parameters and `scenario_sha256` as
|
||||
before saved values existed, and REA versions without saved values read its
|
||||
captures.
|
||||
|
||||
## Network evidence
|
||||
|
||||
`capture.events: ["network"]` retains request, response, completion, failure,
|
||||
@@ -646,18 +333,6 @@ and unfinished-request metadata. Select content independently:
|
||||
All three options default to `false`. Any selected network content also enables
|
||||
network metadata observation; listing `network` in `events` is then optional.
|
||||
Each observed request object receives a capture-scoped `transaction_id`.
|
||||
Each `request` event carries `observed_at`, the wall-clock time REA observed the
|
||||
request start. Playwright reports the browser's own start only once a request
|
||||
ends, so each `request-finished` or `request-failed` event carries `started_at`,
|
||||
the browser's wall-clock start, and `elapsed_ms`, its time from that start to
|
||||
the end of the response, when the browser reports them. The environment views
|
||||
time scenario records from these, as HAR entries are timed: the browser's start
|
||||
where it reported one, else `observed_at`, which trails it by up to tens of
|
||||
milliseconds while REA reads other responses. A duration is retained only from a
|
||||
unique terminal event that reports it; missing or ambiguous durations stay
|
||||
omitted, while an explicitly reported zero remains zero. Captures made before these fields existed lack them and
|
||||
are read in record order. REA versions before them refuse captures that carry
|
||||
them. Scenario comparisons ignore all three (`ignore_network_request_timing`).
|
||||
Requests sharing a URL remain distinct. Redirects link to the preceding
|
||||
transaction only when it was observed; a null predecessor does not establish
|
||||
that no earlier hop existed. Request events require both transaction fields;
|
||||
@@ -691,16 +366,8 @@ Proxy-Authorization, Cookie, and Set-Cookie values are null with
|
||||
and their standard URI/form or JSON-escaped representations are replaced without
|
||||
decoding unrelated binary bytes. Other encodings are not interpreted. Digests describe the
|
||||
retained representation after redaction.
|
||||
The selected body-byte limit also bounds that retained representation after
|
||||
secret expansion, before base64 serialization. Credential header values are
|
||||
discarded before retained-header allocation accounting; their names and null
|
||||
projections still count. The transport/API has already materialized incoming
|
||||
bodies and headers, so these retained limits do not bound producer-side input
|
||||
allocation.
|
||||
|
||||
Only completed responses are read. The producer-observed terminal state is
|
||||
recorded independently of whether its event metadata fit the retention budget. An omitted completion event cannot turn a
|
||||
finished transaction into an unfinished one. When the final step ends, observation
|
||||
Only completed responses are read. At the end of the final step, observation
|
||||
stops; requests still in flight emit `request-unfinished`, and their observed
|
||||
responses have explicitly unavailable bodies. Already-admitted content reads
|
||||
drain before browser cleanup. At most 16 reads run at once; queued work is
|
||||
@@ -755,8 +422,8 @@ and compared as an unordered collection after caller rules apply, because
|
||||
concurrent responses arrive in any order and can land in a later step; a
|
||||
redirect retains the complete predecessor ancestry, including each hop's own
|
||||
request, response outcome, and selected network content. Each transaction's
|
||||
capture-local IDs, sequence/index fields, content source-sequence anchor and
|
||||
request times (`observed_at`, `started_at`, `elapsed_ms`) are removed. Caller rules normalize retained strings before hashing. The
|
||||
capture-local IDs, sequence/index fields, and content source-sequence anchor
|
||||
are removed. Caller rules normalize retained strings before hashing. The
|
||||
`preserve_redirect_transaction_ancestry` built-in rule fingerprints each hop
|
||||
as canonical SHA-256 of `{ events, redirected_from }`: `events` is the sorted
|
||||
list of its normalized event SHA-256 digests, and `redirected_from` is its
|
||||
@@ -777,10 +444,7 @@ scope/flags, value fingerprints, and duplicate entries even when redaction or
|
||||
literal rules make names collide. Captured storage arrays retain their original
|
||||
order. The `compare_storage_entries_as_unordered_collection` built-in rule
|
||||
participates in the normalization commitment; new storage comparison digests
|
||||
therefore differ from earlier order-sensitive comparisons. The
|
||||
`ignore_network_request_timing` built-in rule likewise joins the commitment, so
|
||||
normalization digests differ from comparisons made before it, though captures
|
||||
with and without request times compare alike.
|
||||
therefore differ from earlier order-sensitive comparisons.
|
||||
|
||||
Each aligned step reports changed, unchanged, or unknown. Changed and unknown
|
||||
action, screenshot, DOM, accessibility, URL, history, storage, and event
|
||||
@@ -794,67 +458,7 @@ selection. Captures with different selections report event evidence as unknown
|
||||
with a recovery instruction, even when no selected family emitted an event.
|
||||
Captures written before this field existed omit it, so their event evidence is
|
||||
also unknown.
|
||||
Retained-capture readers admit that omission without deriving a selection from
|
||||
observed events; other artifact families remain comparable. Current capture
|
||||
producers still require and publish the selected event families. Manifest,
|
||||
environment and script-export readers share the retained-capture admission
|
||||
rules and preserve the same missing fact.
|
||||
|
||||
Network normalization also commits the removal of capture-local transaction IDs,
|
||||
attribution to the requesting step, and unordered transaction comparison. These
|
||||
built-in rule identities participate in the normalization digest.
|
||||
|
||||
## Summary delivery and retained views
|
||||
|
||||
`capture_browser_scenario` accepts `detail: "summary"` to retain the complete
|
||||
capture in the MCP session and return its summary. The default remains
|
||||
`complete`. Delivery detail does not change scenario parameters or identity.
|
||||
|
||||
The summary reports each step's status, error, URL and captured/missing/truncated
|
||||
section names. It omits `not_requested` sections and repeated `before_url` values.
|
||||
The accessibility tree comes from the first unsuccessful step, or the final
|
||||
step on success; when that step captured no tree, its nearest earlier captured
|
||||
tree is identified explicitly. Event and network counts cover retained records,
|
||||
with capture omissions reported separately.
|
||||
|
||||
For the one-shot CLI, save the complete Evidence before selecting a summary:
|
||||
|
||||
```bash
|
||||
rea capture-browser-scenario scenario.json --detail summary --evidence_output /absolute/path/scenario-evidence.json --json
|
||||
```
|
||||
|
||||
Use the returned `normalized_result.parent_evidence_id` with
|
||||
`inspect_analysis_view` source `{kind: "retained-evidence", evidence_id: "..."}`.
|
||||
For a CLI bundle, use an `evidence-file` source with its absolute path and the
|
||||
parent Evidence ID. Views read the recorded capture; they never replay actions.
|
||||
|
||||
- `{kind: "summary"}` returns the same overview.
|
||||
- `{kind: "browser-scenario", collection: "steps", step_indices: [2], artifacts: ["accessibility"], offset: 0, limit: 1}` returns selected recorded steps and artifacts.
|
||||
- `{kind: "browser-scenario", collection: "events", event_kinds: ["response"], transaction_ids: ["request-2"], offset: 0, limit: 20}` pages selected events with original item indices.
|
||||
|
||||
Omitted filters select all; empty arrays select none. Pagination counts matching
|
||||
retained rows. Exhausting a page does not establish capture completeness or
|
||||
current application state.
|
||||
|
||||
Transaction pages use `{kind: "browser-scenario", collection: "transactions", limit: 20}`.
|
||||
Their stable identity is the parent Evidence ID plus `transaction_id`; each row
|
||||
keeps original event indices. Metadata events are returned by default. Select
|
||||
`event_kinds: ["network-content"]` for recorded bodies and header values, or
|
||||
`event_kinds: []` for identity and indices alone. A `step_indices` filter selects
|
||||
a whole transaction when any retained event belongs to a selected step; this
|
||||
is receipt attribution, not proof that the action caused the request.
|
||||
|
||||
For a large retained tree, request `{kind: "browser-scenario", collection: "text",
|
||||
step_index: 10, artifact: "accessibility", offset: 0, limit: 2048}`. `dom` is also
|
||||
supported. Offsets and limits count UTF-16 code units. The full artifact digest
|
||||
and byte size remain attached to each page. Missing, truncated and unrequested
|
||||
text stay explicit; source truncation counts are preserved without treating an
|
||||
exhausted page as a complete capture.
|
||||
|
||||
Network summary counts include every retained transaction identity, including
|
||||
those whose request event was dropped. Missing request events, conflicting
|
||||
metadata and observed request failures have separate counts. `transactions` is
|
||||
`null` when network observation was unrequested or its legacy selection is
|
||||
unknown and there are no retained transactions; an observed empty inventory is
|
||||
`0`. Full capture omissions remain available and each page carries source
|
||||
completeness, dropped-event counts and omitted-transaction accounting.
|
||||
@@ -1,62 +0,0 @@
|
||||
# Bun standalone executables
|
||||
|
||||
`inspect-artifact` inventories embedded Bun module graphs in Mach-O `__BUN,__bun`,
|
||||
PE `.bun`, ELF `.bun`, and appended trailers. `extract-artifact` writes their stored
|
||||
module contents through the existing safe output tree. Neither operation executes
|
||||
the executable or decompiles bytecode.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./hello --json
|
||||
rea extract-artifact ./hello --json
|
||||
```
|
||||
|
||||
In MCP, open the executable with `open_binary`, then call `inspect_artifact`.
|
||||
Use `detail: "summary"` for large graphs and `inspect_analysis_view` to page retained
|
||||
Evidence. The root metadata's `entry_point_logical_path` is the reader-owned
|
||||
location of its entry module. Use it as an inventory `logical_paths` filter,
|
||||
then read selected node fields with the returned content identity. This also
|
||||
works for names mapped to indexed extraction paths. The complete Evidence stays
|
||||
retained; no source-name normalization is needed in the caller.
|
||||
|
||||
The root reports the carrier, layout, module count, entry point, raw flags and
|
||||
compile exec-argv. Each module retains its source name, loader and encoding values,
|
||||
entry-point status, sizes and SHA-256. Sourcemaps, bytecode and module-info regions
|
||||
are separate children. Source names such as `/$bunfs/root/main.js` remain in
|
||||
metadata; extraction uses a relative path. Unsafe or non-UTF-8 names receive an
|
||||
indexed path and retain their source spelling or bytes. Virtual names deeper
|
||||
than 256 logical path parts also use an indexed path with an explicit limitation,
|
||||
preventing unbounded path-tree work while retaining the source fact. Duplicate names remain
|
||||
separate occurrences; ordinary extraction collision rules apply.
|
||||
|
||||
The recognized record shapes are the 36-byte Bun 1.3.3 layout and the 52-byte
|
||||
Bun 1.3.9 layout. These labels identify structural layouts, not a detected compiler
|
||||
version. Bun does not store an explicit revision identifier: unknown table sizes
|
||||
and ambiguous known layouts are unsupported. A future layout that is byte-for-byte
|
||||
compatible cannot be distinguished. Invalid pointers and truncated claimed sections
|
||||
are malformed. An absent trailer without a named section does not establish a Bun
|
||||
graph. Unknown enum values remain raw, with a null name.
|
||||
|
||||
Sourcemaps are stored Bun regions, not necessarily standalone JSON source maps.
|
||||
Extracted contents retain their original encoding. Use JavaScript analysis only on
|
||||
compatible source files; bytecode is retained as evidence, not decoded.
|
||||
|
||||
Module parsing reads exact metadata ranges rather than loading stored asset
|
||||
bodies or whole pages around small pointers. A cache retains at most 256 KiB
|
||||
and 1,024 ranges for one reader preparation. Cumulative uncached metadata reads
|
||||
are limited to 256 MiB, checked before allocation or I/O. The module table,
|
||||
decoded strings and combined logical path tree have separate admission checks;
|
||||
at most 250,000 distinct path prefixes are admitted before inventory construction.
|
||||
Stored bodies are range-streamed for hashing and extraction. Header/suffix observations are reused
|
||||
only for the same admitted file state, with selected-path checks before publication.
|
||||
|
||||
See [limits and recovery](design/limits-and-recovery.md) for admission ceilings.
|
||||
Synthetic tests cover the carriers and malformed declarations. Run
|
||||
`node scripts/verify-bun-standalone.mjs` after `npm run build:cached` for local Bun
|
||||
verification, and add `--docker` to compile a Linux fixture with `oven/bun:1.4.0`.
|
||||
Bun and Docker are fixture producers only and are not installed by REA.
|
||||
|
||||
Layout references: [Bun 1.3.3 serializer](https://github.com/oven-sh/bun/blob/bun-v1.3.3/src/StandaloneModuleGraph.zig)
|
||||
and [Bun 1.3.9 serializer](https://github.com/oven-sh/bun/blob/bun-v1.3.9/src/StandaloneModuleGraph.zig).
|
||||
|
||||
The current 52-byte record order can also be checked in the
|
||||
[Bun 1.4.4 Rust serializer](https://github.com/oven-sh/bun/blob/bun-v1.4.4/src/standalone_graph/StandaloneModuleGraph.rs).
|
||||
@@ -1,107 +0,0 @@
|
||||
# Capability composition and ownership
|
||||
|
||||
REA keeps domain contracts and shared workflows separate from provider adapters
|
||||
and CLI/MCP translation. A capability can have several provider identities; a
|
||||
provider can contribute operations to several capabilities. Those associations
|
||||
are source facts, while host availability is an observation.
|
||||
|
||||
## Source declarations and runtime bindings
|
||||
|
||||
`src/composition/capabilityDeclarations.ts` contains static provider identities,
|
||||
named contract-group references and precise operation subsets. Complete groups
|
||||
refer to `TOOL_CONTRACT_GROUPS` in the existing contract owner; family definitions
|
||||
and provider catalogs read the same arrays, so a new group operation is included
|
||||
without updating a copied provider list. Metadata imports only the group ID type
|
||||
and pure metadata leaves; it must not load factories, schemas, engines or
|
||||
generated catalogs. Catalog generation resolves groups and subsets against the
|
||||
canonical contracts and rejects unknown groups/operations and conflicting
|
||||
identities. New provider provenance must come from the implementation
|
||||
that actually produces the Evidence, including format-specific capture adapters
|
||||
and derived workflows.
|
||||
|
||||
`src/composition/capabilityServices.ts` binds typed service factories to those
|
||||
capability keys. Each runtime entry owns construction, cleanup and any available
|
||||
host/configuration facts or live probes. A service entry requires a close
|
||||
capability; entries adapting managers retain their existing `closeAll` operation.
|
||||
The host maps entries without erasing their individual service/port types. There
|
||||
is no generic JSON operation dispatcher.
|
||||
|
||||
These two declarations have different loading needs. Static facts can be read
|
||||
without loading a schema or provider graph. The full runtime composition is
|
||||
materialized only when the MCP runtime needs it. One-shot CLI adapters import the
|
||||
selected factory rather than constructing a complete capability host. CLI and MCP
|
||||
reuse the same provider/service factories and application workflows, while
|
||||
keeping their respective argument translation and lifetime.
|
||||
|
||||
## Resource lifetime
|
||||
|
||||
`CapabilityScope` registers a module's actual close capability when its instance
|
||||
is composed. There is no independent server provider-close list to update.
|
||||
Concurrent shutdown calls join the same cleanup attempt. A successful peer is
|
||||
removed from pending cleanup; failed peers remain owned and are retried on the
|
||||
next close. Synchronous throws and rejections without a reason remain failures,
|
||||
and cannot prevent other peers from attempting cleanup. Shutdown seals further
|
||||
ownership registration.
|
||||
|
||||
The host distinguishes these lifetimes:
|
||||
|
||||
- Host-owned service instances are released by that host.
|
||||
- Caller-owned optional observations remain borrowed by SDK server instances;
|
||||
their process-level owner closes them, including discarded discovery probes.
|
||||
- CLI invocations use their existing finalizers to join cleanup and preserve the
|
||||
original observation, cancellation and cleanup errors.
|
||||
- Binary engines, bundle readers and investigation records retain their existing
|
||||
owners. The `BinarySession` transition/drain barrier still coordinates target
|
||||
changes, reset and disposal. A module refactor must not create independent
|
||||
barriers that allow late work to repopulate reset records.
|
||||
|
||||
An application workflow may borrow a shared resource scope, such as byte entropy
|
||||
using the host's artifact scope. Its wrapper does not acquire a second ownership
|
||||
claim over the borrowed scope.
|
||||
|
||||
## Availability and registration
|
||||
|
||||
Module bindings provide host support, selected configuration and precisely typed
|
||||
live availability probes through `CapabilityAvailabilityFacts`. Constructors do
|
||||
not run these probes or launch engines. Session discovery invokes the existing
|
||||
provider-owned probes only for advertised families; live failures remain fresh
|
||||
observations. Caller policy, selected environments and injected providers retain
|
||||
their existing precedence.
|
||||
|
||||
MCP registration remains in the MCP adapter. It consumes composed services,
|
||||
record ports, admission and logging; it must not add fallback constructors that
|
||||
can escape the host owner. Named contracts bind handlers and schemas. The complete
|
||||
tool-family registrar map is checked by TypeScript, and catalog order never
|
||||
selects a handler or schema.
|
||||
|
||||
## Extending a capability
|
||||
|
||||
1. Add or reuse the domain contract and application workflow, with explicit
|
||||
Evidence provenance, coverage, unknowns and caller choices.
|
||||
2. Put concrete construction in `src/composition/` and keep its default factory
|
||||
lazy. Reference adapter-owned pure identity metadata from both the provider
|
||||
and its static capability declaration.
|
||||
3. Add the runtime binding if a host-owned service is needed. Bind cleanup and
|
||||
availability/probes there, preserving borrowed versus owned resources.
|
||||
4. Add the CLI/MCP argument adapters and their named registrations. A one-shot
|
||||
command uses the shared selected factory and existing finalization helpers.
|
||||
5. Run source-boundary, discovery, SDK-schema and relevant public workflow checks.
|
||||
Exercise partial cleanup, retry, cancellation and caller-owned peers at the
|
||||
applicable production seam; use actual applications/providers for routing and
|
||||
complete Evidence verification. Recording ports do not prove real providers.
|
||||
|
||||
The constructor boundary guard covers the centralized Frida, Cutter, GDB and
|
||||
Rizin constructors without rejecting legitimate type-only injected interfaces.
|
||||
Other settled pure/provider boundaries continue checking type imports too. The
|
||||
shared parser records module-edge erasure once. Whole-declaration type imports
|
||||
are erased; inline type bindings retain their runtime module edge under
|
||||
`verbatimModuleSyntax`. Static dynamic imports remain runtime edges too.
|
||||
Tests belong with their semantic owner and must be
|
||||
found exactly once; provider roots need the corresponding adapter discovery lane.
|
||||
|
||||
This is an internal composition contract. External plugin distribution, automatic
|
||||
provider installation and a universal provider interface are separate concerns.
|
||||
|
||||
The [initial verification report](verification/capability-composition-2026-11-01.md)
|
||||
records the executed public workflows, cleanup boundaries, resource measurements
|
||||
and remaining provider qualification gaps for this composition increment.
|
||||
@@ -1,90 +0,0 @@
|
||||
# Static analysis in GitHub Actions
|
||||
|
||||
The [minimal workflow](https://github.com/morluto/rea/blob/main/examples/github-actions/rea-analysis.yml) builds a small
|
||||
JavaScript application, analyzes its files without executing them, and uploads
|
||||
machine-readable results. It uses the existing CLI, not a separate GitHub Action
|
||||
or an MCP client. It does not promise support for every REA provider.
|
||||
|
||||
## Run the example
|
||||
|
||||
1. Copy the example to `.github/workflows/rea-analysis.yml` in your repository and
|
||||
commit it to the default branch. No existing application or package manifest is
|
||||
needed for the self-contained demonstration.
|
||||
2. Enable GitHub Actions if necessary. Select **REA static analysis example** in
|
||||
the Actions tab and choose **Run workflow**.
|
||||
3. Download the `rea-analysis` artifact after the run. Its files are:
|
||||
|
||||
| File | Meaning |
|
||||
| ----------------- | -------------------------------------------------------------------------------------------------- |
|
||||
| `analysis.json` | Original CLI JSON, including complete Evidence on success or the error report when one is emitted. |
|
||||
| `analysis.stderr` | Original diagnostics/progress; an empty file is normal when there are none. |
|
||||
| `exit-code.txt` | Original analysis process exit code. |
|
||||
|
||||
The example pins Node.js 24.22.1, REA 6.7.0, and action commit revisions. It installs
|
||||
REA into the runner's temporary directory rather than changing the application's
|
||||
dependencies. This pins the tool version, not every transitive npm dependency;
|
||||
use a reviewed, committed package lock and `npm ci` if your integration requires
|
||||
that stronger dependency-reproducibility guarantee.
|
||||
|
||||
## Prerequisites and trust
|
||||
|
||||
The runner is GitHub-hosted Ubuntu 24.04 with Bash, npm, and `ps` from `procps`.
|
||||
Other Linux images must supply `procps`; REA's supervised process controls use it.
|
||||
This static JavaScript operation needs no Hopper, Ghidra, IDA, Java, browser,
|
||||
Docker, GPU, external API key, or user-supplied GitHub secret. The workflow only
|
||||
requests `contents: read`; artifact upload uses the runner's normal Actions
|
||||
credentials.
|
||||
|
||||
Replace **Build a known artifact** with your reviewed build, and set
|
||||
`ARTIFACT_PATH` to its application directory (relative to the checkout) or an ASAR.
|
||||
The demonstration creates its own tiny application; it deliberately throws if
|
||||
executed. REA reads it statically. Your replacement build itself can execute code:
|
||||
review it separately and do not add untrusted pull-request builds to a privileged
|
||||
`pull_request_target` workflow. The example is manually triggered, not an
|
||||
untrusted-PR execution policy.
|
||||
|
||||
Reports can contain paths, URLs, source-derived facts, and caller-selected data.
|
||||
Only analyze artifacts and publish reports you are authorized to share. Do not
|
||||
rely on REA inferring secrets from their names. Review artifact access and
|
||||
retention for your repository; the example retains reports for seven days.
|
||||
|
||||
## Failure, partial results, and unsupported capabilities
|
||||
|
||||
The analysis step captures stdout and stderr separately, records `$?`, and exits
|
||||
with that same status. There is no `continue-on-error`, `|| true`, output filter,
|
||||
or pipeline that could turn a failed analysis green. Before checkout or other
|
||||
fallible steps, the workflow creates a fresh report directory under `RUNNER_TEMP`.
|
||||
Analysis and upload both use that directory, never a checked-in `reports/rea` or
|
||||
other workspace files. The upload step uses `always()` when that directory was
|
||||
prepared, so available diagnostics survive an analysis failure. If installation
|
||||
or the build failed before analysis began, it is empty and upload warns rather
|
||||
than publishing stale results or inventing a successful result. Runner loss or
|
||||
cancellation can still prevent upload.
|
||||
|
||||
`--json --full-output` selects a machine-readable envelope; it does **not** mean
|
||||
that analysis was complete. A successful operation can retain partial analysis,
|
||||
unknowns, and limitations. Exit zero is not a security verdict, proof of complete
|
||||
coverage, or a policy gate for those findings. Inspect the saved Evidence and
|
||||
coverage before making such a decision. See [output and exit status](cli.md#output-and-exit-status).
|
||||
|
||||
A nonzero exit can represent invalid input, unavailable tooling, an unsupported
|
||||
host/capability, or another operation failure. Keep the original JSON error (when
|
||||
emitted), stderr, and exit code, and use their category/message to diagnose it.
|
||||
An early startup failure may leave `analysis.json` empty; do not parse that as
|
||||
successful analysis. Select a supported operation or explicitly provision its
|
||||
[documented prerequisites](installation.md); do not silently switch providers,
|
||||
execute the artifact, or ignore the failure to obtain a green job.
|
||||
|
||||
To exercise the failure path, temporarily set `ARTIFACT_PATH` to
|
||||
`build/missing-artifact` without changing the demo build. Analysis must fail,
|
||||
`exit-code.txt` must be nonzero, and the available reports must still upload.
|
||||
To exercise unsupported input, set `ARTIFACT_PATH` to `build/rea-ci-demo/index.js`:
|
||||
this operation accepts an application directory or ASAR, not a single source file.
|
||||
The retained JSON error has category `unsupported_target`; the step exits nonzero.
|
||||
Restore the correct path before using the example normally.
|
||||
|
||||
To exercise partial analysis, append `export const broken = ;` to the demo's
|
||||
JavaScript. REA 6.7.0 retains `data.normalized_result.statistics.parse_failures`
|
||||
and an incomplete-coverage limitation while the operation can still exit zero.
|
||||
Inspect those facts instead of treating process success as full coverage. The
|
||||
report JSON and diagnostics remain unfiltered in all three cases.
|
||||
+31
-756
@@ -3,8 +3,6 @@
|
||||
Use the CLI for a direct inspection or a script. It uses the same application
|
||||
workflows and evidence contracts as REA's MCP server. Each CLI invocation is a
|
||||
separate process; save results when you want to use them in a later command.
|
||||
For a pinned, noninteractive workflow with retained reports, see
|
||||
[static analysis in GitHub Actions](ci-analysis.md).
|
||||
|
||||
## Run a command
|
||||
|
||||
@@ -39,11 +37,6 @@ pass an absolute path. REA reads and authenticates the file, then projects it
|
||||
like retained Evidence, so a large file is bounded by the file reader rather
|
||||
than charged as inline input.
|
||||
|
||||
Local file inspection commands reject empty paths and NUL bytes before resolving
|
||||
relative inputs, auxiliary files or output destinations. An empty string never
|
||||
selects the working directory; use an explicit `.` when selecting that directory
|
||||
is supported by the command.
|
||||
|
||||
For example, a 334 MB Obsidian application Evidence file yields a summary view
|
||||
of about 10 KB. Replace the view with
|
||||
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
|
||||
@@ -51,145 +44,6 @@ module page. See
|
||||
[JavaScript application workflows](javascript-application-workflows.md) for
|
||||
views, feature traces and comparisons that take the same saved Evidence.
|
||||
|
||||
Add `--timings` to a command to write one JSON timing record to stderr. For
|
||||
example, `rea inspect-elf-notes program --json --timings > result.json` keeps
|
||||
stdout as a parseable result and writes
|
||||
`{"rea_timings":{"version":1,"command_ms":42}}` to stderr. `command_ms`
|
||||
measures CLI command dispatch and result delivery; it is not provider-only or
|
||||
analysis-only time. The flag is off by default, and the record omits stages
|
||||
that REA did not measure. Root version reporting does not dispatch a command
|
||||
and has no command timing record.
|
||||
|
||||
## JSON Lines record output
|
||||
|
||||
Use `--format jsonl` to consume row-oriented results one record per line:
|
||||
|
||||
```bash
|
||||
cat > observations-view.json <<'JSON'
|
||||
{"source":{"kind":"evidence-file","path":"app-evidence.json"},"view":{"kind":"javascript-facts","property_keys":[],"offset":0,"limit":32}}
|
||||
JSON
|
||||
rea inspect-analysis-view observations-view.json --format jsonl > observations.jsonl
|
||||
```
|
||||
|
||||
This request selects up to 32 observation identities and their provenance,
|
||||
without observation property values.
|
||||
|
||||
This changes JSONL object-collection framing. Scripts that need one complete
|
||||
report or a scalar field should use `--format json`. For a full application
|
||||
graph, first project a supported analysis view rather than selecting an
|
||||
arbitrary nested graph array.
|
||||
|
||||
A top-level array emits its elements in source order. An object must have one
|
||||
clear `rows`, `items`, `records`, or `occurrences` array, directly or under
|
||||
`page`. Evidence uses the same boundaries inside `normalized_result`.
|
||||
Any other selection is refused before a record is written, including an
|
||||
arbitrary report, a scalar, a nested array elsewhere, and a columnar object such
|
||||
as the `inspect-sqlite-database --table` `normalized_result.rows`
|
||||
(`{columns, values}`). The refusal is the caller error `invalid_request`
|
||||
(`invalid_input`) and names the selection and its observed shape; the analysis
|
||||
itself succeeded, so use `--format json` for the complete document. If several
|
||||
collections exist, select one with `--filter-output`, for example
|
||||
`normalized_result.rows` for an analysis view.
|
||||
|
||||
Object collections finish with a control record:
|
||||
|
||||
```json
|
||||
{
|
||||
"$rea": "jsonl/v1",
|
||||
"type": "metadata",
|
||||
"records": {
|
||||
"path": ["normalized_result", "rows"],
|
||||
"emitted": 2,
|
||||
"source": 10,
|
||||
"filter": "normalized_result.rows[0,2]"
|
||||
},
|
||||
"context": {
|
||||
"evidence_id": "ev_…",
|
||||
"normalized_result": {
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 10,
|
||||
"total": 100,
|
||||
"next_offset": 10,
|
||||
"exhausted": false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This abbreviated example illustrates framing. The actual `context` preserves
|
||||
all non-row fields, including Evidence subject, provider, authority, confidence,
|
||||
locations, links, limitations, parent/view digests, pagination and coverage.
|
||||
With `--full-output`, it also retains the command envelope and its metadata.
|
||||
A data filter selects row fields/slices while this context remains available;
|
||||
excluded peer record collections and unselected Evidence `raw_result` are not
|
||||
restored. An explicitly selected `raw_result` remains selected. Retain the
|
||||
original Evidence for complete observations.
|
||||
|
||||
`records.emitted` and `records.source` count the selected output and its source
|
||||
array; they do not assert analysis completeness. Source coverage and unknowns
|
||||
retain their original meaning. Existing row identities and indices remain
|
||||
unchanged unless your field selection excludes them. Empty object collections
|
||||
still emit metadata; an empty top-level array emits no lines. Bare top-level
|
||||
arrays have no synthetic coverage record.
|
||||
|
||||
Command and output-validation failures emit `{"$rea":"jsonl/v1","type":"error","ok":false,"result":…}`
|
||||
and retain the original diagnostic payload. SDK validation/sentinel errors use
|
||||
the SDK's exit status rather than guessing from data fields. `$rea` is reserved
|
||||
at the top level of a data record, so control records are unambiguous; collisions
|
||||
are refused before rows are written. Nested `$rea` values are ordinary data.
|
||||
A missing final metadata record or nonzero exit can indicate an interrupted
|
||||
export; do not treat already emitted rows as a complete result.
|
||||
|
||||
`--token-count` estimates this exact representation, including metadata, one
|
||||
line at a time. Errors remain error records even with token counting enabled.
|
||||
Token windows cannot preserve JSON Lines and are rejected. JSON encoding and
|
||||
stdout writes are incremental, including large escaped strings; this is output
|
||||
streaming over a completed result, not incremental analysis. The analysis still
|
||||
needs its normal resources. A single record must fit the runtime string limit
|
||||
for token estimation; remove `--token-count` to stream a larger record.
|
||||
|
||||
Standalone version and MCP doctor diagnostics remain single JSON documents
|
||||
in JSON Lines mode. Incur's streaming generator events and successful sentinel
|
||||
responses retain their SDK protocol. MCP application results and Evidence
|
||||
contracts are unchanged.
|
||||
|
||||
## Shell completion
|
||||
|
||||
The CLI includes optional shell completion. Run `rea completions --help` to
|
||||
see the available shells and their setup commands. For Bash, load the hook
|
||||
into the current shell:
|
||||
|
||||
```bash
|
||||
eval "$(rea completions bash)"
|
||||
```
|
||||
|
||||
For zsh, initialize completion before loading the hook. If your shell already
|
||||
runs `compinit`, place the hook after that existing initialization:
|
||||
|
||||
```zsh
|
||||
autoload -Uz compinit
|
||||
compinit
|
||||
eval "$(rea completions zsh)"
|
||||
```
|
||||
|
||||
The Bash hook supports macOS system Bash 3.2. It uses optional completion
|
||||
features when the shell provides them and falls back to portable registration.
|
||||
Completion includes standard global options and the active command's options;
|
||||
`rea mcp ` offers only `add` and `doctor`. Output-option values remain data when
|
||||
resolving the command scope.
|
||||
|
||||
For future interactive non-login Bash sessions, add the command to
|
||||
`~/.bashrc`. Interactive login Bash sessions read the first available file in
|
||||
`~/.bash_profile`, `~/.bash_login`, and `~/.profile`; add the command to the
|
||||
file your shell reads, or have that file source `~/.bashrc`. Remove the command
|
||||
from the startup file to stop loading completion, and run `complete -r rea` to
|
||||
remove the registration from the current shell. These are manual, opt-in
|
||||
steps.
|
||||
Completion is a discovery aid; use `rea COMMAND --help` to confirm the
|
||||
available options and subcommands.
|
||||
|
||||
## Managed assemblies (.NET)
|
||||
|
||||
For a .NET PE/CLI assembly, inspect its identity and deployment form before
|
||||
@@ -259,96 +113,6 @@ Formats 2, 3 and 4 retain header, directory and integrity facts. Compact deliver
|
||||
keeps complete Evidence in the requested bundle. See [Godot PCK](godot-pck.md)
|
||||
for embedded packs, batch queries and unknown encrypted coverage.
|
||||
|
||||
Inspect a UnityFS AssetBundle without running Unity:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact content.bundle --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact content.bundle --json
|
||||
```
|
||||
|
||||
Format versions 6 through 8 retain header, block and directory facts with
|
||||
none/LZ4/LZ4HC block decompression. See [UnityFS](unityfs.md) for encryption
|
||||
and LZMA coverage boundaries.
|
||||
|
||||
Inspect an ESP32-family app or bootloader image without esptool:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact firmware.bin --json
|
||||
rea extract-artifact firmware.bin --json
|
||||
```
|
||||
|
||||
The inventory lists each segment with its ESP-IDF load classification, the
|
||||
checksum and SHA-256 results, the descriptor and any Secure Boot v2 signature
|
||||
blocks. See [ESP images](esp-image.md) for recognition, coverage and refusals.
|
||||
|
||||
Inspect an ESP flash dump, or one read from a flash address other than 0:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact flash.bin --json
|
||||
rea inspect-artifact dump.bin --target-format esp-flash-image --flash-offset 0x1000 --json
|
||||
rea extract-artifact flash.bin --json
|
||||
```
|
||||
|
||||
The inventory lists the bootloader, the partition table and each partition,
|
||||
with ESP image facts for app partitions and the OTA records. See
|
||||
[ESP flash images](esp-flash-image.md) for table location, inputs and limits.
|
||||
|
||||
Inspect a Node.js SEA carrier without running it:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./application --json
|
||||
rea extract-artifact ./application --json
|
||||
```
|
||||
|
||||
See [Node.js SEA](node-sea.md) for supported layouts, script and asset paths,
|
||||
fuse observations, and the real-fixture verification lane.
|
||||
|
||||
Inventory and extract Intel HEX or S-record regions:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./firmware.hex --json
|
||||
rea extract-artifact ./firmware.hex --json
|
||||
```
|
||||
|
||||
See [Intel HEX and S-record images](hex-records.md) for recognition, region
|
||||
names, metadata and limits.
|
||||
|
||||
Inspect and extract stored U-Boot legacy uImage payloads and components:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./firmware.bin --detail summary --evidence_output firmware.json --json
|
||||
rea extract-artifact ./firmware.bin --json
|
||||
```
|
||||
|
||||
See [U-Boot uImage](uimage.md) for verified CRCs, physical component ranges,
|
||||
unknown names, resource limits and retained views. Compressed bytes stay stored.
|
||||
|
||||
`inspect-artifact` and `extract-artifact` also read ZIP members compressed with
|
||||
Zstandard (standard method 93). They retain the shared ZIP CRC, declared-size,
|
||||
overlap and archive expansion checks. The native streaming decoder keeps the
|
||||
existing 128 MiB Zstandard window cap; frames requiring a larger window are
|
||||
refused. Deprecated method 20 and other unregistered ZIP codecs remain unsupported.
|
||||
|
||||
Inventory and extract UF2 or DfuSe regions:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./firmware.uf2 --json
|
||||
rea extract-artifact ./firmware.dfu --json
|
||||
```
|
||||
|
||||
See [UF2 and DfuSe images](uf2-dfuse.md) for recognition, families, numbering,
|
||||
the DfuSe suffix CRC and limits.
|
||||
|
||||
Inventory and extract U-Boot FIT images:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./boot.itb --json
|
||||
rea extract-artifact ./boot.itb --json
|
||||
```
|
||||
|
||||
See [U-Boot FIT images](fit-images.md) for recognition, data sources, hashes
|
||||
and limits.
|
||||
|
||||
Inspect local model declarations without loading weights:
|
||||
|
||||
```bash
|
||||
@@ -372,8 +136,8 @@ The command decodes a version 24, 27, 29, 31 or 39 `global-metadata.dat` and rep
|
||||
images, type, method and member names, string literals, index ranges, header
|
||||
sections and SHA-256. With `--binary PATH` it also reads the matching
|
||||
`GameAssembly.dylib`, `GameAssembly.dll` or `libil2cpp.so` (never runs it) and
|
||||
links each method to its code address; `--generic-instances` adds generic
|
||||
method instances and `--ghidra-seeds PATH` then writes Ghidra function seeds. It does not decrypt metadata. MCP defaults to compact counts
|
||||
links each method to its code address; `--ghidra-seeds PATH` then writes Ghidra
|
||||
function seeds. It does not decrypt metadata. MCP defaults to compact counts
|
||||
with retained Evidence; query indexed rows with filters, batch indices and
|
||||
selected fields. The CLI defaults to complete Evidence;
|
||||
`--detail summary --evidence_output FILE` exports the full result to a new file
|
||||
@@ -403,17 +167,6 @@ embedded X.509 serial, issuer, subject, and validity fields. A missing security
|
||||
directory is observed absence of that table. See
|
||||
[PE Authenticode certificate tables](pe-certificates.md).
|
||||
|
||||
Inspect PE import and delay-load tables without executing the image:
|
||||
|
||||
```bash
|
||||
rea inspect-pe-imports /absolute/path/to/application.exe --json
|
||||
```
|
||||
|
||||
The workflow reports DLL names, named and ordinal functions, delay-load
|
||||
descriptors, and a pefile-compatible import hash of the standard import table.
|
||||
A missing import directory is observed absence of that table. See
|
||||
[PE import and delay-load tables](pe-imports.md).
|
||||
|
||||
Inspect ELF PT_NOTE and SHT_NOTE tables without executing the image:
|
||||
|
||||
```bash
|
||||
@@ -424,33 +177,6 @@ The workflow reports note owners, types, descriptor hashes, and GNU build-id
|
||||
or ABI-tag fields when those descriptors match. A missing note table is
|
||||
observed absence of those declarations. See [ELF note tables](elf-notes.md).
|
||||
|
||||
Inspect an Apple `.ips` crash report without executing a target:
|
||||
|
||||
```bash
|
||||
rea inspect-apple-ips /absolute/path/to/report.ips --json
|
||||
```
|
||||
|
||||
The workflow reports first-line metadata, process/exception/termination facts,
|
||||
threads, usedImages, lossless integer registers, and documented crash classes
|
||||
when `bug_type` is `"309"`. Other DiagnosticReports types return no crash
|
||||
result. See [Apple IPS crash reports](apple-ips.md).
|
||||
Inspect Linux kernel module metadata without executing the module or requiring
|
||||
`modinfo`:
|
||||
|
||||
```bash
|
||||
rea inspect-kernel-module /absolute/path/to/module.ko --json
|
||||
```
|
||||
|
||||
The workflow reports ordered `.modinfo` keys (duplicate `alias` and `parm`
|
||||
entries preserved), `__versions` symbol CRCs, presence of
|
||||
`.gnu.linkonce.this_module` and `__ksymtab` sections, observed `.init.text`
|
||||
and `.exit.text` sizes, and an appended signature trailer. `struct module` is
|
||||
not decoded and the signature is not validated. Other ELF types are not
|
||||
applicable. gzip, xz, and zstd containers are unsupported and name
|
||||
`external_decompression` as the next probe: decompress with an external
|
||||
utility, then inspect the decompressed file. See
|
||||
[Linux kernel modules](kernel-modules.md).
|
||||
|
||||
Inspect a CPython 3.7–3.13 `.pyc` header and nested marshal code objects
|
||||
without executing bytecode or installing Python:
|
||||
|
||||
@@ -490,21 +216,11 @@ rea trace /absolute/path/to/program "search" --provider ghidra --json
|
||||
a function dossier; `decompile` returns pseudocode. For assembly instructions
|
||||
alone, use `rea instructions`. A macOS `.app` bundle can be supplied directly.
|
||||
|
||||
A complete dossier can reach several megabytes of value-flow and references.
|
||||
`rea function ... --detail summary --evidence_output /absolute/path/dossier.json` returns the
|
||||
compact summary and saves the complete dossier as an Evidence bundle, never
|
||||
overwriting an existing file. Page that bundle without reanalysis by naming it
|
||||
as an `inspect-analysis-view` source, selecting the summary's
|
||||
`normalized_result.parent_evidence_id` when the bundle holds several records:
|
||||
`{"source": {"kind": "evidence-file", "path": "/absolute/path/dossier.json", "evidence_id": "ev_…"}, "view": {"kind": "native", "facet": "pseudocode", "offset": 0, "limit": 4096}}`.
|
||||
MCP `analyze_function` defaults to the same summary and retains the complete
|
||||
dossier in the session for `inspect_analysis_view`.
|
||||
|
||||
Run `rea --help` or a command's `--help` for arguments and output options.
|
||||
The [generated catalog](mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
|
||||
tools. Provider-specific target support is described in the
|
||||
[native guide](native-investigation.md), [DOS guide](ghidra-dos.md),
|
||||
[raw binary guide](raw-binary.md), [GameCube DOL guide](ghidra-dol.md), and
|
||||
[raw binary guide](raw-binary.md), and
|
||||
[Windows Ghidra guide](windows-ghidra-p0.md).
|
||||
|
||||
## Choose a provider
|
||||
@@ -541,13 +257,7 @@ The selected provider remains bound to the session until an explicit switch
|
||||
or close. Provider failures are returned with their original reason. For an
|
||||
`ambiguous` selection error, choose from `details.candidate_ids`; for
|
||||
`provider_unavailable`, follow `message`, `remediation.action`, and the original
|
||||
reasons in `details.rejections`. Each rejection names its `origin`: `availability`
|
||||
entries describe local readiness failures, including missing configuration or an
|
||||
invalid existing registration; they do not establish target support. `target`
|
||||
entries record a refused kind, format, architecture, or role. An availability-only
|
||||
message describes the recorded rejections without claiming that every provider
|
||||
is unconfigured or that target support is known.
|
||||
For Hopper, Ghidra, IDA, JADX, and Kuna, run
|
||||
reasons in `details.rejections`. For Hopper, Ghidra, IDA, JADX, and Kuna, run
|
||||
`rea doctor --provider ID --json` to diagnose the selected engine. Other adapters,
|
||||
such as `pwntools-elf`, report their configuration requirements directly and are
|
||||
not accepted by `doctor --provider`. See
|
||||
@@ -561,25 +271,6 @@ be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and
|
||||
[Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
|
||||
for import, client timeouts and recovery.
|
||||
|
||||
A one-shot command releases its provider session and REA runtime before it
|
||||
prints a failure, so it omits the same-session reuse and
|
||||
`update_analysis_settings` guidance an MCP connection receives. Its
|
||||
`details.limit` reports `scope: "rea-invocation"` and
|
||||
`applies_at: "next_invocation"`; set the named `environment` (or the per-call
|
||||
`parameter`) for a new invocation and rerun. For example, a Ghidra decompile
|
||||
deadline names `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS`.
|
||||
|
||||
`rea debug-with-gdb` and `rea debug-with-rizin` hold their debugger only for the
|
||||
command's own process. If cleanup cannot confirm the debugger stopped, they print
|
||||
`cleanup_incomplete` and exit nonzero without stopping it. `remediation.action`
|
||||
names the process, as does `retained_debugger_pids` in `details.diagnostics`, or
|
||||
in `details.diagnostics.cleanup_error.details.diagnostics` when the command also
|
||||
failed or was interrupted. Exiting closes the debugger's command and output pipes.
|
||||
REA does not establish what the debugger or its target does next, so check that
|
||||
process. An MCP server that shuts down likewise exits with status 1 instead of
|
||||
waiting on a debugger it could not stop. It names the process on stderr and as
|
||||
`retained_debugger_pids` in its error log.
|
||||
|
||||
If one-shot analysis fails and closing its session also fails, the original
|
||||
error details remain inline and `details.cleanup_error` describes the later
|
||||
close failure. Each diagnostic retains its own reported resources; the later
|
||||
@@ -593,29 +284,10 @@ an exact result when the target bytes, operation, parameters, provider and
|
||||
settings match. Mutations and cursor-dependent calls are excluded from the
|
||||
cache. Snapshot files are local and use owner-only permissions.
|
||||
|
||||
One-shot binary CLI Evidence reports the exact path argument in `caller_input_path`,
|
||||
including relative spelling and symlink aliases. Its subject identifies the
|
||||
inspected artifact; `caller_input_path` does not change the Evidence ID or
|
||||
snapshot query. A replay reports the current invocation's path while leaving
|
||||
the saved Evidence unchanged. Provider inspections continue to include the
|
||||
established `parameters.input_path` field.
|
||||
|
||||
Importing an Evidence bundle refreshes retained request and subject display metadata.
|
||||
It preserves cached analysis results, including observations for other targets.
|
||||
Snapshot bindings that use a replaced record retain its updated display path.
|
||||
|
||||
Dispatching a provider metadata write invalidates the active session's snapshot cache,
|
||||
Dispatching a metadata write invalidates the active session's snapshot cache,
|
||||
even if its reply fails: the provider may already have applied the write.
|
||||
Close and reopen the target before importing or saving another snapshot.
|
||||
|
||||
Saved snapshots include only cache entries whose subject identifies the selected
|
||||
target. Unbound observations and observations of child artifacts remain available
|
||||
in the open session and its Evidence bundle; they are excluded from the target's
|
||||
saved replay entries without changing their subjects.
|
||||
The snapshot also retains available Evidence linked by selected records, including
|
||||
transitive sources with an unbound or different subject. Those sources are not
|
||||
replay entries for the selected target. Unrelated records remain excluded.
|
||||
|
||||
Snapshots retain eligible target-scoped question histories in full, including
|
||||
mutation Evidence recorded while another target was active. Observations and
|
||||
related questions must still belong to the saved target. If those dependencies
|
||||
@@ -637,44 +309,6 @@ cached result. `close_binary` accepts `snapshot_path` and optional
|
||||
`overwrite: true` to save before releasing provider resources. A failed save
|
||||
leaves the session open so the caller can resolve the output failure.
|
||||
|
||||
When a validated snapshot does not match the selected binding, the error's
|
||||
`details.binding_mismatch` reports confirmed target identities, provider
|
||||
identities, or profile digests that differ. `snapshot` is the saved value and
|
||||
`selected` is the requested value. Multiple differences can be reported. A
|
||||
`profile_comparison: "not_compared"` means the profile was not available at
|
||||
that boundary; it does not establish a match. A profileless snapshot can report
|
||||
`workflow_scope: "incompatible"` even when its target and profile match.
|
||||
Choose a matching target, provider, and profile, or analyze without the snapshot
|
||||
and save a new one for the intended binding.
|
||||
|
||||
For a profile mismatch, the saved snapshot contains
|
||||
`binding.analysis_profile.parameters`. If `binary_session` reports an active
|
||||
profile whose digest matches `binding_mismatch.profile_digest.selected`, its
|
||||
`analysis_provider_binding.analysis_profile.parameters` can be compared with
|
||||
those saved parameters. A rejected switch leaves the previous session active,
|
||||
so its profile may be different from the attempted selection; check the digest
|
||||
before using it for this comparison. A configuration difference does not by
|
||||
itself establish why analysis results differ.
|
||||
|
||||
If no target is open and you intend to inspect the selected binding, open that
|
||||
target without the snapshot, then check its profile digest in `binary_session`
|
||||
before comparing parameters. This may start the selected provider; the saved
|
||||
snapshot remains a separate file.
|
||||
|
||||
If a selected snapshot's file descriptor cannot close, `open_binary` reports
|
||||
`cleanup_incomplete`. `binary_session` reports `cleanup_pending` even when no
|
||||
target opened. Retry `close_binary` to release the retained descriptor before
|
||||
opening another target.
|
||||
|
||||
If a snapshot switch fails after the previous target was retired, restoration
|
||||
keeps that target's analysis commitments with the replacement client. This
|
||||
includes the native dispatch workflow profile on a profileless route, so its
|
||||
existing snapshot and fresh observations retain the same provenance.
|
||||
|
||||
For native dispatch, the snapshot binding retains the reader profile; workflow
|
||||
Evidence and `workflow_entries` retain the composed profile, which commits that
|
||||
reader and the workflow's analysis caveats. Restoration preserves both profiles.
|
||||
|
||||
The MCP save receipt reports `primitive_entries`, `workflow_entries`, and
|
||||
`evidence_records` separately. Zero primitive bindings can still accompany retained workflow
|
||||
results and Evidence. These are cached observations, not a saved provider
|
||||
@@ -692,100 +326,21 @@ canonical form, or compare two supplied bundles:
|
||||
rea evidence-import /absolute/path/to/evidence/bundle.json
|
||||
rea evidence-export /absolute/path/to/evidence/bundle.json /absolute/path/to/evidence/canonical.json
|
||||
rea compare /absolute/path/to/evidence/left.json /absolute/path/to/evidence/right.json
|
||||
rea compare left.json right.json --record_pairs '[{"left_evidence_id":"ev_<left-sha256>","right_evidence_id":"ev_<right-sha256>"}]' --json
|
||||
```
|
||||
|
||||
Migration: replace `rea compare-bundles` with `rea compare`; the former command
|
||||
has been removed. The MCP tool remains `compare_bundles`.
|
||||
|
||||
`rea compare` compares records by Evidence identity and explicit pairs. It does
|
||||
not read artifact contents: the inventories of two builds are unresolved
|
||||
records, or one opaque change when you pair them. Use `compare-artifacts`,
|
||||
described at the end of this section, to compare their contents.
|
||||
|
||||
`--record_pairs` supplies an explicit JSON array of observation pairs, matching
|
||||
MCP `compare_bundles.record_pairs`. Pair IDs must exist on their named sides and
|
||||
be one-to-one. Without pairs, one-sided membership remains unknown; REA does
|
||||
not infer correspondence from position or display names.
|
||||
|
||||
Exports preserve an existing destination unless `--overwrite` is explicit.
|
||||
|
||||
Atomic Evidence output stages on the selected destination's filesystem. If the
|
||||
OS pathname ceiling prevents its internal staging name, REA retries in a shorter
|
||||
ancestor on that same filesystem and removes the unused stage. Existing output
|
||||
protection still applies. A genuine pathname refusal uses `resource_constraint`
|
||||
with `details.reason: path-limit` and `boundary: filesystem-path`.
|
||||
|
||||
For application traces and version comparisons, see
|
||||
[JavaScript application workflows](javascript-application-workflows.md).
|
||||
MCP [retained Evidence references](mcp-contracts.md#retained-application-evidence-inputs)
|
||||
belong to one connection. Separate CLI calls can consume full saved records or
|
||||
resolve exact retained IDs from an explicitly selected local bundle:
|
||||
|
||||
```bash
|
||||
rea trace-application-feature request.json --evidence-bundle /absolute/path/evidence.json --json
|
||||
```
|
||||
|
||||
Here `request.json` uses the same logical Evidence input as MCP, for example
|
||||
`"application": {"kind":"retained-evidence","evidence_id":"ev_<64 lowercase hex characters>"}`.
|
||||
Inline records and references may be mixed in the request's declared source
|
||||
fields, including arrays and nested inputs. The bundle is read and authenticated
|
||||
once for that command; no target is reopened, no producer is rerun, and no
|
||||
record is selected by file order. Source identity, provenance, coverage and
|
||||
partial or unknown facts agree with the complete inline form. When starting
|
||||
from summary or inspection Evidence, select its
|
||||
`normalized_result.parent_evidence_id` for a workflow requiring complete source
|
||||
facts. Omit `--evidence-bundle` when all sources are inline.
|
||||
|
||||
The JSON workflow commands supporting this option are:
|
||||
|
||||
- `trace-application-feature`, `trace-javascript-semantics`,
|
||||
`compare-application-versions`, `compare-javascript-export-shapes`,
|
||||
`compare-source-to-bundle`, and `reconcile-javascript-runtime`.
|
||||
- `import-managed-reconstruction`, `verify-managed-native-boundaries`, and
|
||||
`project-managed-application-graph`.
|
||||
- `project-android-application-graph`, `project-apple-application-graph`, and
|
||||
`project-harmony-application-graph`.
|
||||
- `compare-wasm-interfaces` and `inspect-analysis-view`.
|
||||
|
||||
For `inspect-analysis-view`, the request's source reference selects a parent
|
||||
from the bundle for every supported view kind. Bundles exported from MCP work
|
||||
for these commands. Its inline source form cannot also select a bundle.
|
||||
A missing ID is refused with `details.reason: "missing"`;
|
||||
supplying a bundle does not make incompatible Evidence valid for a workflow.
|
||||
`compare-managed-members` keeps its two assembly paths, and
|
||||
`compare-process-captures` keeps its two capture Evidence JSON paths. The bundle
|
||||
option above selects an input; producer options that write a new bundle, such
|
||||
as `inspect-managed-members --evidence-bundle`, retain their existing meaning.
|
||||
File selectors remain available only on contracts that explicitly support them.
|
||||
|
||||
Compare two artifact inventories with `compare-artifacts`. It takes the same
|
||||
`left` and `right` input as MCP `compare_artifacts`: complete Evidence, or an
|
||||
`evidence-file` source with an optional `evidence_id` that selects one record
|
||||
from a bundle. Relative paths resolve from the working directory:
|
||||
|
||||
```bash
|
||||
rea inspect-artifact old.zip --json > old.json
|
||||
rea inspect-artifact new.zip --json > new.json
|
||||
rea compare-artifacts '{"left":{"kind":"evidence-file","path":"old.json"},"right":{"kind":"evidence-file","path":"new.json"}}' --json
|
||||
```
|
||||
|
||||
A one-shot `compare-artifacts` command keeps no session Evidence, so it refuses a
|
||||
`retained-evidence` source with `details.reason: "missing"`. Select the record
|
||||
from a saved bundle instead.
|
||||
For an artifact summary selected from a bundle, include its complete parent
|
||||
inventory in the same bundle. The command resolves that parent from the
|
||||
authenticated file records; a summary-only file cannot supply missing facts.
|
||||
belong to one connection. Separate CLI calls can consume full saved records.
|
||||
`inspect-analysis-view REQUEST.json --evidence-bundle BUNDLE.json` resolves
|
||||
the request's `retained-evidence` source ID from that authenticated bundle,
|
||||
including bundles exported from an MCP session. The same view JSON works for
|
||||
every supported parent kind; no target is reopened and no record is selected
|
||||
by file order. Omit `--evidence-bundle` when the request supplies inline Evidence.
|
||||
|
||||
## Import historical source
|
||||
|
||||
Historical source inventories preserve exact filenames, including literal backslashes
|
||||
on POSIX hosts. Relative import lookup uses the recorded source file's owning
|
||||
directory and decodes URL percent escapes once; encoded slash, NUL and malformed
|
||||
escapes remain unresolved. A decoded backslash remains part of a filename rather
|
||||
than a directory separator. This inventory lookup does not claim that Node can
|
||||
load that filename or apply Node's runtime file-URL admission policy.
|
||||
|
||||
Import an older source tree to compare with the current artifact:
|
||||
|
||||
```bash
|
||||
@@ -795,26 +350,12 @@ rea import-reference-source /absolute/path/to/source --detail summary --outline-
|
||||
|
||||
The default (`--detail full`) returns the complete `HistoricalSourceGraph`.
|
||||
Choose `--detail summary` for a `HistoricalSourceSummary` of that same committed
|
||||
inventory: its `HistoricalSourceManifest` contains the semantic graph and root
|
||||
inventory: its `HistoricalSourceManifest` contains the full graph and root
|
||||
digests, with provenance, VCS facts, languages, manifest paths, entry and
|
||||
relationship counts, parse-failure counts, file/directory states, exclusions
|
||||
and limitations. The summary is a presentation view, not a graph accepted by
|
||||
source-to-bundle comparison; keep the full output for that workflow.
|
||||
|
||||
Known filesystem failures commit the operation, parsed errno, entry-relative
|
||||
location and recovery guidance. Entry `local_diagnostics`, when present,
|
||||
retains the exact host message, including local paths, outside the semantic
|
||||
graph commitment. Equivalent permission failures therefore retain their manifest
|
||||
identity after relocation while preserving original source diagnostics in full
|
||||
output and visible summary outline entries. Other substantive limitations still
|
||||
participate in the graph commitment.
|
||||
|
||||
An unreadable child directory remains a partial inventory entry. Directory-scoped
|
||||
ignore policy is required before enumerating a directory that can be opened;
|
||||
an unreadable or invalid policy still fails the import. Project negations cannot
|
||||
override defaults or caller exclusions; caller patterns retain their own ordered
|
||||
negations. Configured secret exclusions remain final.
|
||||
|
||||
`--outline-depth` requires `--detail summary` and accepts a nonnegative safe
|
||||
integer. Depth 1 shows root children, depth 2 (the default) also shows their
|
||||
children, and depth 0 omits all outline entries. Paths retain canonical Unicode
|
||||
@@ -834,26 +375,11 @@ output, not the amount of source examined. A shallow tree or many exclusions can
|
||||
still produce a large summary; depth 0 removes the outline but preserves the
|
||||
other facts. No fixed output-size or runtime improvement is promised.
|
||||
|
||||
Import failures use the shared error contract (`code`, `category`, `message`,
|
||||
`retryable`, and `remediation`). When reads completed before failure,
|
||||
`details.diagnostics.partial` retains the reader facts, with file bytes as
|
||||
arrays of byte values. Unresolved cleanup uses `cleanup_incomplete` and
|
||||
retains the source failure and resource paths in `details`.
|
||||
|
||||
The import records hashes and metadata for the supplied files, separately
|
||||
from observations of the current app. For a directory inside a Git checkout,
|
||||
it records the HEAD of the nearest enclosing repository; whether the working
|
||||
tree matches that commit remains unknown. Root selection follows filesystem
|
||||
symlink/parent semantics before canonical identity is derived. It honors
|
||||
`.gitignore` files at the selected root and in traversed child directories,
|
||||
with patterns relative to their containing directory and deeper negations
|
||||
applied only when the parent was not excluded. Matching uses the nearest
|
||||
repository's local `core.ignorecase` setting (false when unset), including the
|
||||
shared configuration of linked worktrees. Policy and Git metadata reads retain
|
||||
their descriptor cleanup under the import's resource owner; unreadable policy
|
||||
is a typed failure, not an empty policy. Above-root `.gitignore` files, global
|
||||
excludes, `.git/info/exclude`, config includes and worktree-specific config are
|
||||
not loaded. The import also applies default exclusions for common generated and dependency paths such
|
||||
tree matches that commit remains unknown. It honors the source tree's `.gitignore`
|
||||
and applies default exclusions for common generated and dependency paths such
|
||||
as `node_modules/`, `dist/`, and `*.log`. Pattern-based exclusions record the
|
||||
matched pattern and whether it came from project, default, caller, or sensitive
|
||||
path policy. Links into excluded paths retain their internal target metadata;
|
||||
@@ -875,10 +401,7 @@ top-level limitation. That partial inventory does not establish absence among
|
||||
unexamined children. POSIX directory-entry bytes that are not valid UTF-8 cannot
|
||||
be represented as graph paths; the containing directory (or root) is marked
|
||||
unsupported, and REA does not access the entry through a replacement-character
|
||||
name. Symlink targets are read as bytes too: a target that is not valid UTF-8
|
||||
is retained as an unreadable symlink with its bytes reported in the limitation,
|
||||
and is never resolved through a replacement-character path. Readable siblings
|
||||
remain inventoried.
|
||||
name. Readable siblings remain inventoried.
|
||||
|
||||
JavaScript and TypeScript import parsing requires valid UTF-8. Malformed source
|
||||
bytes retain their original hashes and sizes with a decoding diagnostic; REA
|
||||
@@ -886,40 +409,6 @@ does not infer module targets from replacement characters. Exact dynamic-import
|
||||
strings, expression-free templates and runtime-transparent TypeScript wrappers
|
||||
retain their targets; computed imports remain unknown. TypeScript import-equals
|
||||
dependencies remain `requires` relationships whether or not they are exported.
|
||||
Historical dependencies also include TypeScript import types (`import("./dep").T`
|
||||
and `typeof import("./dep")`) and declarations inside ambient module bodies;
|
||||
these source relationships do not assert runtime loading. Language labels retain
|
||||
the conventional case-sensitive `.C` (C++) and `.M` (Objective-C++) suffixes.
|
||||
|
||||
REA derives historical CommonJS paths only when the selected loader and its
|
||||
base are proven: this includes the exact
|
||||
`createRequire(new URL("./file.cjs", import.meta.url))` form and
|
||||
`require.resolve` with `{ paths: [__dirname] }`. Exact optional-call forms such
|
||||
as `require.resolve?.("./dep.cjs")` use the same loader and mutation facts. The
|
||||
nearest inventoried package manifest supplies `.js`/`.ts` package facts to the
|
||||
shared parser; explicit filename modes take precedence. A known ES module has no
|
||||
implicit CommonJS wrapper, including when a factory is derived from an unbound
|
||||
`require`. An unreadable, excluded or malformed manifest, or package scope outside
|
||||
the selected inventory, leaves otherwise unproven loader resolution unknown with
|
||||
the requested specifier and a reason. A nearest manifest without a `type` field
|
||||
uses the shared parser's JavaScript module-syntax facts. Unresolved TypeScript,
|
||||
JSX compilation and declaration contexts do not establish runtime wrapper bases.
|
||||
Other custom bases and search options remain partial with unknown resolution and
|
||||
the literal requested specifier. Overwritten resolver methods retain that uncertainty; they are not
|
||||
attributed to Node. `require.main` is a module object, so calling it does not
|
||||
establish a loader reference. These relationships do not establish execution;
|
||||
Node's [`require.resolve`](https://nodejs.org/docs/v24.22.1/api/modules.html#requireresolverequest-options)
|
||||
performs lookup without loading the target. An excluded `package.json`
|
||||
also leaves package-directory selection unknown rather than treating the
|
||||
manifest as absent and applying an `index` fallback.
|
||||
|
||||
Import requests use URL pathname semantics: query and fragment remain request
|
||||
identity, percent escapes decode exactly once for inventory lookup, and encoded
|
||||
separators or malformed escapes retain an unresolved relationship and diagnostic.
|
||||
URL-bearing relationships retain the authored `source_specifier` separately from
|
||||
the inventoried `to` path; distinct query/fragment requests remain distinct even
|
||||
when they name the same file. CommonJS requests retain literal filename semantics
|
||||
for percent signs, query marks and fragments.
|
||||
|
||||
Historical-source import requires safe no-follow file opens on Linux or
|
||||
macOS. Native Windows returns `unsupported_host`; use Linux REA inside WSL
|
||||
@@ -952,24 +441,8 @@ rea mcp --http --port 8765
|
||||
```
|
||||
|
||||
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the environment,
|
||||
then to `127.0.0.1` and an ephemeral port. A bind outside loopback requires
|
||||
`MCP_ALLOWED_HOSTS` or `--allowed-hosts`, a comma-separated list of client host
|
||||
names or IP addresses without URLs or ports.
|
||||
|
||||
Repeat `--allowed-hosts` to append names, or put multiple names in one
|
||||
comma-separated value. All occurrences are validated together and deduplicated
|
||||
after hostname normalization; the explicit list replaces `MCP_ALLOWED_HOSTS`.
|
||||
|
||||
For example:
|
||||
|
||||
```sh
|
||||
rea mcp --http --host 0.0.0.0 --allowed-hosts 127.0.0.1,rea.lan
|
||||
```
|
||||
|
||||
Requests with a different `Host` or browser `Origin` receive `403`; a missing
|
||||
`Origin` is accepted. The announced client URL uses the bound hostname when
|
||||
admitted, otherwise the first declared hostname, with the actual bound port.
|
||||
It is written to stderr as one JSON log line (`"msg":"MCP transport listening"`) whatever `REA_LOG_LEVEL`
|
||||
then to `127.0.0.1` and an ephemeral port. The bound URL is written to stderr as
|
||||
one JSON log line (`"msg":"MCP transport listening"`) whatever `REA_LOG_LEVEL`
|
||||
is set to; nothing is written to stdout. `rea mcp --http --help` lists the accepted options. The listener serves
|
||||
one client at a time: a session ends with the client's `DELETE`, and a client
|
||||
that connects while another session is open takes it over, so a client that
|
||||
@@ -1007,47 +480,13 @@ rea environment-serve manifest.json --evidence capture.json --port 8765
|
||||
```
|
||||
|
||||
`--host` and `--port` resolve like `rea mcp --http`: flags, then `MCP_HOST` and
|
||||
`MCP_PORT`, then `127.0.0.1` and an ephemeral port. A non-loopback bind requires
|
||||
`MCP_ALLOWED_HOSTS` with the same hostname rules and Host/Origin gates as above.
|
||||
`environment-serve` uses this environment setting; `--allowed-hosts` belongs to
|
||||
`rea mcp --http`. For example:
|
||||
|
||||
```sh
|
||||
MCP_ALLOWED_HOSTS=127.0.0.1,rea.lan rea environment-serve manifest.json --evidence capture.json --host 0.0.0.0
|
||||
```
|
||||
|
||||
`ENVIRONMENT_NAME` renames
|
||||
`MCP_PORT`, then `127.0.0.1` and an ephemeral port. `ENVIRONMENT_NAME` renames
|
||||
the served environment; set but blank, it is refused as `configuration_invalid`
|
||||
like a blank `MCP_HOST`. The admitted client URL is written to stderr as one JSON log line
|
||||
like a blank `MCP_HOST`. The bound URL is written to stderr as one JSON log line
|
||||
(`"msg":"MCP transport listening"`). See
|
||||
[agent environments](agent-environments.md) for what the state and replay
|
||||
return and do not establish.
|
||||
|
||||
## Build an environment bundle
|
||||
|
||||
`rea environment-build <capture> [dir] [--name <name>] [--template-rule <rule> ...]
|
||||
[--format-rules-only] [--package-tarball <path>] [--capture-timeout-ms <ms>] [--dry-run]` builds the bundle
|
||||
`environment-export` writes from one recorded capture: a `.har` file, which it
|
||||
inspects as `inspect-web-network-capture` does, or capture Evidence JSON from
|
||||
`inspect-web-network-capture` (HAR) or `capture-browser-scenario`. It projects
|
||||
the environment manifest from the in-memory Evidence with the opt-in rules
|
||||
`after-uid-literal`, `opaque-token`, `sibling-variance` and `prefix-variance`
|
||||
unless `--template-rule` or `--format-rules-only` selects others, prepares the
|
||||
state, and writes the bundle; every file is byte-identical to the three-step
|
||||
path's when that path lists the same rules in the views' order and exports
|
||||
under the build's name. A loopback or IP-literal capture is named after its file unless `--name`
|
||||
is given. It prints one JSON report: the capture and manifest identities, the
|
||||
state, a `coverage` summary of which tools answer from the state, which replay
|
||||
and why each write is withheld with a hint, the files, warnings and the next
|
||||
command. `--dry-run` writes nothing and may omit the directory.
|
||||
|
||||
```sh
|
||||
rea environment-build grafana.har ./grafana --json
|
||||
```
|
||||
|
||||
See [agent environments](agent-environments.md#build-a-bundle-in-one-step)
|
||||
for the report, the hints and what the bundle ships.
|
||||
|
||||
## Export an environment bundle
|
||||
|
||||
`rea environment-export <manifest.json> --evidence <capture.json> <dir>
|
||||
@@ -1097,58 +536,8 @@ without response bodies, or without the request bodies its writes need. See
|
||||
rea environment-compare manifest.json --evidence capture.json --probe probe.json --json
|
||||
```
|
||||
|
||||
## Report version and runtime
|
||||
|
||||
`rea --version` prints the package version. Add an output format to get the
|
||||
version with the runtime and platform context that a support request needs:
|
||||
|
||||
Version reports do not initialize analysis providers. Invalid global settings
|
||||
or output options still report their usual errors.
|
||||
|
||||
```bash
|
||||
rea --version --json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "rea-agents",
|
||||
"version": "6.7.0",
|
||||
"runtime": { "id": "node", "version": "24.22.1" },
|
||||
"platform": "linux",
|
||||
"architecture": "x64",
|
||||
"supported_runtime": {
|
||||
"range": "^22.23.4 || ^24.22.1 || >=26.12.1",
|
||||
"satisfied": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Meaning |
|
||||
| ------------------------------------ | -------------------------------------------------------------------------------------------------------------- |
|
||||
| `name`, `version` | Published package name and version; `version` matches plain `--version`. |
|
||||
| `runtime.id` | `node` or `bun`. |
|
||||
| `runtime.version` | Version of that runtime. |
|
||||
| `runtime.node_compatibility_version` | Bun only: the Node.js version it reports compatibility with. |
|
||||
| `platform`, `architecture` | Node.js `process.platform` and `process.arch`, such as `linux` and `x64`. |
|
||||
| `supported_runtime.range` | Supported Node.js range. |
|
||||
| `supported_runtime.satisfied` | Whether this runtime is supported, judged as `rea doctor` judges it; Bun by its Node.js compatibility version. |
|
||||
|
||||
The field names are stable. The report contains no paths, environment values,
|
||||
user or host names, and it starts no provider and reads no target. `--format`
|
||||
selects any CLI output format, such as `--format yaml`. JSON follows the usual
|
||||
rule: indented on a terminal, compact when piped or redirected. `--help` still
|
||||
takes precedence. With other output controls, `--version` keeps its existing
|
||||
behavior: valid controls such as `--filter-output` or `--full-output` print the
|
||||
plain version, and invalid combinations report the usual output-flag error,
|
||||
such as `UNSUPPORTED_OUTPUT_COMBINATION` for `--json` with `--token-limit`. Use
|
||||
`rea doctor` for provider and host readiness.
|
||||
|
||||
## Output and exit status
|
||||
|
||||
Commands reject surplus positional arguments before execution, including commands
|
||||
that accept no positionals. Use `rea <command> --help` for the declared argument
|
||||
shape; a final variadic argument continues to accept multiple values.
|
||||
|
||||
`rea mcp doctor --json` checks production MCP startup and reports child logs in
|
||||
`diagnostics`. It drains stderr while retaining at most its first 64 KiB of raw
|
||||
bytes as text, omitting a final UTF-8 code point if it crosses that byte limit.
|
||||
@@ -1160,65 +549,15 @@ truncation alone does not change the diagnostic health or exit status.
|
||||
The default terminal format is TOON. Use `--json` when saving results for a
|
||||
JSON consumer. Valued output flags (`--format`, `--filter-output`,
|
||||
`--token-limit`, `--token-offset`) accept `--flag value` or `--flag=value`; a
|
||||
missing value fails with `MISSING_OUTPUT_VALUE` before the command runs.
|
||||
Malformed format values and token windows fail with `INVALID_OUTPUT_VALUE` in the
|
||||
selected structured format. Token limits and offsets must be nonnegative safe
|
||||
integers; zero selects an empty window. These checks run before command execution. `--json` is indented on a terminal and compact when piped or
|
||||
missing value fails with `MISSING_OUTPUT_VALUE` before the command runs. `--json` is indented on a terminal and compact when piped or
|
||||
redirected, which keeps large results smaller for agents and scripts; pipe
|
||||
through `jq .` for indented files. A formatting refusal exits nonzero even when
|
||||
the underlying operation succeeded.
|
||||
|
||||
`--format md` escapes backslashes, pipes, and inline Markdown syntax in table
|
||||
headers and cells so source text is displayed literally, and renders cell line
|
||||
breaks as `<br>`. Underscores between letters or digits, as in `source_index`,
|
||||
cannot form emphasis and stay unescaped. These are presentation changes; retained
|
||||
Evidence and JSON output keep the original values. Nested objects and arrays
|
||||
use JSON inside their owning Markdown cells, retaining the field names and
|
||||
values instead of replacing them with an object label. Table widths are measured
|
||||
iteratively, so an admitted row page is not limited by the JavaScript function
|
||||
argument count.
|
||||
|
||||
Markdown refuses selected NUL characters and unpaired UTF-16 surrogates because
|
||||
renderers or UTF-8 output would substitute them. TOON refuses selected unpaired
|
||||
surrogates for the same UTF-8 reason. Use JSON or YAML to retain those values.
|
||||
An authored replacement character and valid supplementary characters are not
|
||||
treated as malformed.
|
||||
|
||||
`--format csv` and `--format tsv` export one selected row collection. Select an
|
||||
exact array path with `--filter-output` and provide comma-separated direct
|
||||
scalar columns with `--columns`; column order is the output column order:
|
||||
|
||||
```bash
|
||||
rea inspect-analysis-view request.json --format csv \
|
||||
--filter-output normalized_result.rows --columns name,address
|
||||
```
|
||||
|
||||
The first line is a REA metadata record,
|
||||
`# rea-delimited-v1 <JSON>`. It carries the selected path, view request, limitations and unknowns,
|
||||
coverage, source coverage and completeness, command, and duration. A metadata
|
||||
value of `null` means that the selected row collection did not provide that
|
||||
fact; it does not mean complete coverage. The following line is the header.
|
||||
CSV records use CRLF line endings; TSV records use LF. Cells containing the
|
||||
delimiter, quotes, or line breaks are quoted and embedded quotes are doubled.
|
||||
Values are emitted as UTF-8 without Unicode normalization. Missing and `null`
|
||||
cells are empty; booleans and finite numbers use their ordinary text forms.
|
||||
String cells and headers that begin with spreadsheet formula syntax are refused
|
||||
with a structured error; unpaired surrogates in selected cells or headers are
|
||||
also refused. JSON retains those values unchanged.
|
||||
|
||||
Delimited output does not flatten nested Evidence or nested field values.
|
||||
It returns a structured error and a nonzero status if the filter does not
|
||||
select exactly one row array, a row is not an object, or a selected cell is
|
||||
nested or formula-leading. JSON remains the complete canonical output; use it
|
||||
to retain fields outside the selected columns or preserve exact formula-leading
|
||||
text. Token counting and token windows are not
|
||||
available for CSV/TSV because slicing would corrupt records or metadata.
|
||||
through `jq .` for indented files. Output selection and formatting do not change
|
||||
operation status.
|
||||
|
||||
`--filter-output` selects fields from successful command data. Classified failures
|
||||
retain their complete typed diagnostic projection even when the selector matches
|
||||
a field such as `code` or `message`, or names a missing field. `--full-output`
|
||||
controls whether the outer `{ok, error, meta}` envelope is included.
|
||||
Selected own JSON members, including `__proto__`, remain ordinary output data.
|
||||
|
||||
Sibling array slices that select different fields are combined by original row
|
||||
index and emitted in source order, regardless of selector order. Selecting a
|
||||
@@ -1232,10 +571,10 @@ Cutter when available. This is separate from `--format json`, which selects the
|
||||
CLI result representation; the global `--json` flag selects that same CLI
|
||||
representation.
|
||||
|
||||
Cutter no longer accepts command-local `--json=true` or `--json=false`.
|
||||
Replace these with `--json-output=true` or `--json-output=false`, respectively.
|
||||
Combine `--json-output` with bare `--json` when both Cutter command output and
|
||||
the CLI result should use JSON; either argument order is accepted.
|
||||
Cutter still accepts deprecated `--json=true` and `--json=false` compatibility
|
||||
values. When both forms are present, `--json-output` takes precedence in either
|
||||
argument order. Bare `--json` retains its global meaning of selecting JSON CLI
|
||||
output formatting.
|
||||
|
||||
| Status | Meaning |
|
||||
| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
@@ -1255,24 +594,11 @@ Refusals before binding (unreadable files, non-Evidence JSON, capture Evidence
|
||||
in a format other than HAR, a manifest or state whose parent is not the
|
||||
supplied evidence, an unbindable port) print a structured
|
||||
error and exit `1`, as does a listener that fails after binding.
|
||||
If shutdown fails, the command reports `cleanup_incomplete` with the retained
|
||||
server resources. It preserves a primary listener diagnostic, or a
|
||||
`shutdown_observation` naming the requested shutdown and its URL, environment,
|
||||
tool count and Evidence identifiers. The successful `stopped` document is
|
||||
returned only after cleanup is confirmed. The same in-process server owner
|
||||
retains failed stages for retry; the CLI exposes no separate retry command.
|
||||
`rea environment-build` is one-shot too: it prints its report and exits `0`,
|
||||
or prints the refusing stage's structured error and exits `1`, writing nothing
|
||||
when the refusal comes before the bundle is written. SIGINT cancels it with a
|
||||
`cancelled` error; a cancellation before the bundle stage writes nothing.
|
||||
`rea environment-export` is one-shot: it prints the written files and exits
|
||||
`0`, or prints a structured `invalid_request` error and exits `1` for the same
|
||||
pair refusals, a non-empty or file-occupied directory, a missing package
|
||||
tarball or a name that is not a kebab-case label or is a Windows device name,
|
||||
writing nothing in each case. A file occupying the destination is reported as
|
||||
requiring a directory; identity-change diagnostics are reserved for admitted
|
||||
paths that fail later identity checks. Host permission failures identify whether
|
||||
input reading or bundle writing was denied and retain the reported path.
|
||||
writing nothing in each case.
|
||||
A write that fails after the first file (a permission, disk-space or
|
||||
pathname-length failure on a later path) propagates as the host's failure,
|
||||
naming its cause and, when it failed on one bundle file or the package
|
||||
@@ -1293,9 +619,7 @@ browser, web and Electron commands (including `analyze-web-bundle`,
|
||||
for Android/ADB, Apktool/JEB/Flutter, firmware, PCAP and signal analysis,
|
||||
WASM/EVM, Go/Rust and binary diagnostics, Frida instrumentation, the Cutter
|
||||
command, objdump, Rizin and GDB, and the agent environment server
|
||||
(`environment-serve`). The handler also wraps `doctor`,
|
||||
`inspect-sqlite-database`, `recover-javascript-sources`,
|
||||
`inspect-il2cpp-metadata`, and `inspect-desktop-application`. In these paths, a returned structured
|
||||
(`environment-serve`). In these paths, a returned structured
|
||||
`cancelled` result can be printed before the wrapper assigns the
|
||||
signal-derived status. This describes the current source routing; it is not a
|
||||
claim that every command or provider runtime has been independently exercised.
|
||||
@@ -1308,26 +632,14 @@ new file. A publication that has committed can win a later cancellation; the
|
||||
after commit, the `cleanup_incomplete` error reports the same file as
|
||||
`details.published_path` and `details.bytes`.
|
||||
|
||||
When a failure retains a completed observation and no bundle committed,
|
||||
`details.source_closure` states what that observation cites. `publication` is
|
||||
`not_published`. `records` holds authentic source Evidence this invocation
|
||||
captured, within 64 KiB. `omitted` lists up to 64 other identities and
|
||||
`omitted_count` the total: `not_captured` means the invocation holds no such
|
||||
record, so nothing is shown for it or its ancestors; `not_inlined` means it was
|
||||
captured but is not in this response. `complete` is true only when every cited
|
||||
record is in `records` or embedded in the observation. If examining the closure
|
||||
itself fails, `collection_failure` names the error type and the closure state is
|
||||
unknown; the original failure is still returned. Nothing is retained after exit,
|
||||
and running the command again is a new observation.
|
||||
|
||||
When the reader of piped output closes early, as `rea ... | head` does, the
|
||||
CLI stops writing, prints no error and exits `141` (`128 + 13`, the status of a
|
||||
SIGPIPE-terminated writer). The operation itself may have completed; only its
|
||||
output was declined.
|
||||
|
||||
`inspect-artifact` also returns a structured cancelled result. Its logging
|
||||
wrapper sits outside the cancellation handler and replaces the signal-derived
|
||||
status with the analysis failure status, so it exits `1`. Setup has its own contract: cancellation exits `0`. Treat the
|
||||
`inspect-artifact` also returns a structured cancelled result, but its direct
|
||||
analysis cancellation path does not set a signal-derived exit status, so it
|
||||
exits `1`. Setup has its own contract: cancellation exits `0`. Treat the
|
||||
structured result and process status as separate signals when scripting
|
||||
cancellation.
|
||||
|
||||
@@ -1343,40 +655,3 @@ REA's failure status:
|
||||
set -o pipefail
|
||||
rea inspect-artifact ./app.asar --json | jq . > inspection.json
|
||||
```
|
||||
|
||||
`--token-count` reports a tokenx estimate for the selected emitted document,
|
||||
including a requested full-output envelope and metadata. JSON uses the same
|
||||
terminal indentation or compact piped representation as normal output; JSONL
|
||||
uses its emitted line representation. The final newline is included (tokenx
|
||||
assigns whitespace no tokens). This is an estimate, not a model tokenizer count.
|
||||
Counting retains the existing bounded formatted-string allocation after capacity
|
||||
admission; omit the flag to stream large JSON documents without materializing
|
||||
the complete formatted text. Exhausted token windows display an empty range at
|
||||
the total token count and have no next offset.
|
||||
|
||||
### `inspect-devicetree`
|
||||
|
||||
Inspect an explicit local FDT v16/v17 blob without execution or an external tool.
|
||||
Returns header, reservations, node/property tree, derived standard values and
|
||||
selected phandle references. See [devicetree inspection](devicetree.md) for
|
||||
limits and retained-view pagination.
|
||||
|
||||
```bash
|
||||
rea inspect-devicetree /artifacts/board.dtb --json
|
||||
rea inspect-devicetree /artifacts/board.dtb --detail summary --evidence_output /artifacts/board-evidence.json --json
|
||||
```
|
||||
|
||||
### PE TLS and load configuration
|
||||
|
||||
```sh
|
||||
rea inspect-pe-load-config /absolute/path/to/application.exe --json
|
||||
```
|
||||
|
||||
Reports TLS callbacks and size-covered load-config fields without execution.
|
||||
Use `--max-callbacks` and `--max-file-bytes` to set read budgets. See
|
||||
[PE TLS and load configuration](pe-load-config.md).
|
||||
|
||||
Bun standalone executables use the existing `inspect-artifact` and
|
||||
`extract-artifact` commands to inventory and materialize embedded source,
|
||||
sourcemaps and bytecode without running the executable. See
|
||||
[Bun standalone executables](bun-standalone.md) for layout coverage and limits.
|
||||
+23
-344
@@ -6,102 +6,14 @@ for the reviewed setup transaction and [client IDs](installation.md#supported-ag
|
||||
The versioned verification notes below describe their stated runs; live service
|
||||
acceptance is verified only where explicitly recorded.
|
||||
|
||||
## Launchers and PATH
|
||||
|
||||
A client launched from a desktop app, IDE or service can inherit a different
|
||||
`PATH` from your terminal. An absolute path to `rea-agents` does not resolve its
|
||||
interpreter: the npm launcher's `#!/usr/bin/env node` still searches `PATH` for
|
||||
Node. Running that command successfully in a terminal does not establish that
|
||||
the client can launch it.
|
||||
|
||||
For JavaScript installations, setup registers the selected runtime
|
||||
executable and REA script by absolute path. On POSIX, it prefers executable
|
||||
`node` or `bun` and REA aliases on setup's `PATH` only when they resolve to the
|
||||
same selected files. This preserves manager-owned aliases across version updates;
|
||||
otherwise setup uses the selected paths directly. This also applies when setup
|
||||
runs through a package runner: retain the selected cache entry and runtime while
|
||||
the registration uses them. Clearing either makes startup fail without resolving
|
||||
another package from the client's project. Restore the same installation before
|
||||
repairing it; an unidentified registration remains untouched. See
|
||||
[installation and recovery](installation.md#package-runners-and-global-installation).
|
||||
Setup does not copy your ambient environment into client
|
||||
settings. To refresh an older registration, preview
|
||||
`rea setup --client <client-id> --skill=false --dry-run`, then run the same command
|
||||
without `--dry-run` and approve the displayed changes. Setup preserves unrelated
|
||||
settings, backs up changed configuration and leaves an aligned registration
|
||||
unchanged. Restart the client after applying the update.
|
||||
|
||||
If a runtime manager removes the registered executable and no surviving alias
|
||||
was available, doctor reports its identity as unknown and setup retains the
|
||||
registration. REA does not infer that an arbitrary missing executable was Node
|
||||
from its script argument. Back up the client's configuration, verify the absolute
|
||||
path and supported version of the runtime you intend to use, then update only
|
||||
the runtime command, preserving the REA script arguments and other settings.
|
||||
Rerun setup to review the registration and restart the client.
|
||||
|
||||
`rea mcp doctor --json` tests the production MCP adapter using the runtime
|
||||
executable and environment of the terminal running doctor. It does not launch
|
||||
the client's configured command or reproduce the client's environment. If doctor
|
||||
passes but the client disconnects, inspect the client's startup error and registered
|
||||
launcher; see [verification scope](client-verification.md#launch-environment-verification).
|
||||
|
||||
`rea doctor` has the same limit for provider checks, and reports a provider setting
|
||||
the registration's file does not carry as a
|
||||
[non-blocking unknown](installation.md#check-readiness-for-your-task).
|
||||
|
||||
## Runtime settings and client deadlines
|
||||
|
||||
`update_analysis_settings` changes curated defaults inside the running REA
|
||||
server. It does not edit the client registration, extend the client's tool-call
|
||||
deadline, or recompute timeout values previously derived by `rea setup`. When
|
||||
raising a REA startup or operation deadline, separately ensure the client allows
|
||||
enough time for the full request. Use `inspect_analysis_settings` for the running
|
||||
server's defaults; `rea doctor` checks launch configuration and cannot observe
|
||||
another process's runtime overrides. See the
|
||||
[settings design](design/limits-and-recovery.md) for ownership and lifetime.
|
||||
|
||||
## Client tool names and discovery
|
||||
|
||||
REA advertises canonical names such as `binary_session`. The MCP host decides
|
||||
how to expose them to its model: a name such as `mcp__rea__binary_session` is a
|
||||
client convention, not a prefix REA can enforce. Use the active client's tool
|
||||
listing and server association to find the operation. A valid registration or
|
||||
installed skill does not prove that those tools reached the model. Doctor reports
|
||||
live-server identity as unknown; query `binary_session` through the active client
|
||||
to observe the running server's identity.
|
||||
|
||||
If a client or fork exposes several servers under the same prefix, a wildcard
|
||||
for that prefix does not isolate REA permissions. Inspect that client's actual
|
||||
tool names and approval rules before selecting a permission scope. REA setup
|
||||
does not repair a client's naming behavior or infer permission drift from an
|
||||
unfamiliar prefix. The unavailable fork reported in
|
||||
[#5512](https://github.com/morluto/rea/issues/5512) has not been reverified.
|
||||
|
||||
## Fiddler Everywhere alongside REA
|
||||
|
||||
Connect Fiddler Everywhere's MCP server separately for HTTP traffic debugging,
|
||||
preserving your REA client entry. See the
|
||||
[setup instructions and public skills](web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
|
||||
|
||||
## VS Code
|
||||
|
||||
`rea setup --client vscode` configures the default VS Code user profile's
|
||||
`mcp.json`. Setup honors `VSCODE_PORTABLE` and `VSCODE_APPDATA` when they are
|
||||
present in its environment. It cannot infer a running window's named profile
|
||||
or `--user-data-dir`; configure that profile separately when needed.
|
||||
|
||||
`rea setup --client copilot_cli` instead configures Copilot's user
|
||||
`mcp-config.json`. VS Code can forward eligible profile servers to Copilot
|
||||
sessions; see the [VS Code MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration).
|
||||
|
||||
## GitHub Copilot CLI
|
||||
|
||||
Setup preserves an existing `tools` selection in the REA registration, including
|
||||
an empty list. New registrations default to `["*"]`. A restricted selection is
|
||||
aligned configuration; doctor does not widen it to expose every REA tool. See
|
||||
[Copilot's MCP configuration](https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/add-mcp-servers)
|
||||
for tool-selection settings.
|
||||
|
||||
For GitHub Copilot CLI 1.0.95, `gpt-4.1` model metadata can block ordinary
|
||||
chat with REA's full input-schema profile before sending a model request
|
||||
(`compaction_static_context_blocked`; [#1554](https://github.com/morluto/rea/issues/1554)).
|
||||
@@ -224,25 +136,14 @@ and Claude Code reports that autocompact is thrashing. To avoid this:
|
||||
`mcpServers.rea` entry in `~/.claude.json` (`$CLAUDE_CONFIG_DIR/.claude.json`
|
||||
when that variable is set), then restart Claude Code. Setup keeps this
|
||||
override.
|
||||
- To remove entire tool families from the upfront catalog, run
|
||||
`rea setup --client claude_code --tool-families <ids>` with the families the
|
||||
workflow needs (see [the advertising scope contract](mcp-contracts.md#tool-family-advertising-scope)).
|
||||
Out-of-scope tools are not advertised and their calls fail with the SDK's
|
||||
disabled-tool error, so widen the selection when a task needs another
|
||||
family. Setup writes the scope into the entry's `env` as
|
||||
`REA_MCP_TOOL_FAMILIES`; rerun setup with a different selection to refresh
|
||||
it, then restart Claude Code.
|
||||
|
||||
The sizes above come from REA's own `tools/list`. A live Claude Code session
|
||||
against DeepSeek's API is unverified.
|
||||
|
||||
For a client whose stdio reader rejects the complete catalog frame, set
|
||||
`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "compact"` to bound every advertised output
|
||||
schema, or `"off"` to omit them, in REA's `env` object, then restart the client.
|
||||
Both keep canonical result validation and complete structured results; `compact`
|
||||
keeps a widened root shape and `off` advertises nothing. On catalogs large
|
||||
enough that full input schemas alone approach the frame limit, also keep the
|
||||
input profile at its `compact` default. See [MCP output schema
|
||||
For a client whose stdio reader rejects the complete catalog frame, add
|
||||
`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "off"` to REA's `env` object and restart the
|
||||
client. This omits only advertised output schemas; REA still validates complete
|
||||
structured results against its canonical contracts. See [MCP output schema
|
||||
profiles](mcp-contracts.md#output-schemas).
|
||||
|
||||
## Kimi Code
|
||||
@@ -258,16 +159,8 @@ startup; an explicit Ghidra startup timeout of 2700000 ms produces a 2820000 ms
|
||||
tool timeout. The startup allowance follows an explicitly selected Ghidra or
|
||||
Hopper provider, and otherwise uses the larger known Ghidra/Hopper deadline.
|
||||
An explicit `REA_JEB_MCP_TIMEOUT_MS` deadline, shared by catalog discovery and
|
||||
each tools/call, also contributes to that maximum. An explicit `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS` is added to the Ghidra startup allowance, because a single-decompile call can import the target and then decompile up to that limit; setup forwards it into the registration. For `ida` or `auto` selection, a valid IDA registration's parsed
|
||||
each tools/call, also contributes to that maximum. For `ida` or `auto` selection, a valid IDA registration's parsed
|
||||
`timeoutMs` also contributes to that maximum; its default is 300000 ms.
|
||||
The derived value is a single-decompile estimate, not a deadline for an
|
||||
unbounded batch. For example, startup 330 seconds plus two 600-second
|
||||
decompiles and 120 seconds of headroom needs an estimated 1650 seconds,
|
||||
before any additional listing or output work. Select the client tool deadline
|
||||
for the intended workload, split `batch_decompile`, use `decompile_all`
|
||||
limit/resume, or reduce `trace_native_values` `max_functions`. The trace can
|
||||
decompile several functions serially; its depth and graph budgets do not
|
||||
make the single-decompile allowance a whole-traversal deadline. Existing positive caller deadlines are preserved.
|
||||
Setup forwards these explicitly selected REA deadlines and the input schema
|
||||
profile into the registration; unrelated process values are omitted. An
|
||||
explicit `REA_JADX_OPERATION_TIMEOUT_MS` is forwarded with the JADX
|
||||
@@ -275,21 +168,6 @@ registration but does not contribute to the derived tool timeout; raise
|
||||
`toolTimeoutMs` to cover a longer selected JADX deadline.
|
||||
This headroom is a practical default, not a guarantee for every analysis
|
||||
duration. Setup preserves positive timeout values already in the `rea` entry.
|
||||
|
||||
Setup also forwards explicitly selected input/output schema profiles,
|
||||
`REA_MCP_MAX_RESPONSE_BYTES`, `REA_MCP_MAX_INPUT_ELEMENTS`, provider paths and supported Ghidra, JADX and Kuna
|
||||
settings. It also forwards explicit executable and resource selections for ADB,
|
||||
apktool, packet capture, GDB/Rizin/objdump, Cutter, WABT, Wakaru, firmware and
|
||||
signal tools, browser inspection, and pwntools/pwndbg. This includes their
|
||||
supported prlimit overrides, Darwin Swift compiler/SDK/module-cache selections,
|
||||
and compile timeout, logging level and explicitly configured reference-secret
|
||||
patterns. A provider being unavailable does not discard its selected paths;
|
||||
doctor reports its readiness separately. Explicit selections override discovered
|
||||
defaults, including when setup runs again with a changed selection. Unrelated
|
||||
environment variables, process ownership tokens and one-shot target paths are not
|
||||
copied. For an explicitly configured startup target, see the
|
||||
[initial-target settings](mcp-contracts.md#initial-target-at-startup). Existing
|
||||
client environment overrides remain intact unless setup supplies the same key.
|
||||
You can edit `startupTimeoutMs` and `toolTimeoutMs` there; doctor keeps positive
|
||||
caller-selected values aligned and reports a non-blocking advisory when a tool
|
||||
timeout is shorter than the known provider allowance.
|
||||
@@ -347,15 +225,6 @@ Setup and doctor follow Qwen Code's tilde and working-directory-relative home
|
||||
overrides. Qwen Code also discovers personal skills in `~/.agents/skills`,
|
||||
where setup installs the bundled workflow.
|
||||
|
||||
If an existing direct-script registration fails with `env: node: No such file or
|
||||
directory`, refresh it through setup as described under [Launchers and PATH](#launchers-and-path).
|
||||
For an immediate Qwen-specific workaround, when Node is installed in
|
||||
`/opt/homebrew/bin`, add `"PATH": "/opt/homebrew/bin:${PATH}"` to that server's
|
||||
existing `env` object and restart Qwen. Qwen expands `${PATH}` from its parent
|
||||
environment; other clients can use different substitution syntax. Qwen's stdio
|
||||
launcher inherits the parent `PATH` and applies server overrides; a reduced
|
||||
parent environment does not establish that Qwen removed Homebrew from it.
|
||||
|
||||
## Antigravity
|
||||
|
||||
Antigravity 2.0 discovers personal skills in `~/.gemini/config/skills`.
|
||||
@@ -373,58 +242,12 @@ and the selected `trustedFolders.json` (including
|
||||
`GEMINI_CLI_TRUSTED_FOLDERS_PATH`). Settings in an untrusted workspace do not
|
||||
participate. A workspace or system `mcpServers.rea` replaces the entire user
|
||||
entry; setup stops if the winning definition conflicts with its planned entry.
|
||||
Setup checks user policy against the file snapshot it is inspecting or about to
|
||||
update, including confirmed absence; an earlier discovery snapshot does not veto
|
||||
a repaired user policy. Independent system and workspace sources are still checked.
|
||||
Review those files manually: setup never edits administrator/workspace settings
|
||||
or grants workspace trust. Trust supplied only by a running IDE is outside this
|
||||
standalone CLI check.
|
||||
|
||||
A model API's `400 INVALID_ARGUMENT` alone does not identify a rejected schema,
|
||||
tool-count limit or skill defect. Preserve the response's parameter path and
|
||||
diagnostic, the actual running REA version, Gemini CLI version and selected
|
||||
model. For schema rejection, review the [input presentation profiles](mcp-contracts.md#compact-input-schema-profile)
|
||||
and reconnect after changing REA's registration environment. Compact is the
|
||||
current default; an explicitly stored `full` choice is preserved. The generic
|
||||
error in [#4620](https://github.com/morluto/rea/issues/4620) does not establish
|
||||
which constraint failed, and local schema checks do not prove live API acceptance.
|
||||
|
||||
## Codex and Devin
|
||||
|
||||
REA manages its launcher and selected provider settings in the client
|
||||
registration. Codex's server and per-tool approval policies are user choices.
|
||||
Setup patches REA's connection fields in the selected file and retains other
|
||||
server settings, including `default_tools_approval_mode`, nested
|
||||
`[mcp_servers.rea.tools.<name>]` tables, tool allow/deny lists, and unrecognized
|
||||
client settings. Doctor accepts Codex's per-tool policy tables. Known malformed
|
||||
policy values block setup before it writes the configuration or a backup, and
|
||||
the diagnostic identifies the setting to repair. See the
|
||||
[Codex configuration reference](https://developers.openai.com/codex/config-reference/)
|
||||
for the supported policy values.
|
||||
|
||||
When direct launcher paths prove the same executable or Node entrypoint,
|
||||
setup retains the existing caller-selected spelling. An otherwise aligned
|
||||
registration therefore needs no rewrite or recovery backup merely because
|
||||
REA was invoked through another symlink. Setup also retains that spelling when
|
||||
another field, such as an invalid timeout, needs repair. Unresolved identity
|
||||
does not prove equivalence.
|
||||
|
||||
| Settings | Setup ownership |
|
||||
| ------------------------------------------- | --------------------------------------------------------------------------------------- |
|
||||
| Launcher, transport and activation | Refreshes the REA stdio connection and enables it. |
|
||||
| Explicit server environment | Preserves configured overrides and merges selected provider settings. |
|
||||
| Codex timeouts | Retains positive values; repairs missing or invalid values to recommended REA defaults. |
|
||||
| Codex approval policies and tool selections | Retains caller choices; refuses known malformed values. |
|
||||
| Unrelated or future server settings | Retains their parsed values. |
|
||||
|
||||
Dry-run applies the same policy validation and writes no files. A successful
|
||||
refresh keeps an exact recovery snapshot of the original configuration;
|
||||
repeating setup with the same settings performs no write. Uninstall removes
|
||||
the owned REA entry, including its policy, while retaining other servers and
|
||||
the recovery snapshot. A green registration check establishes known
|
||||
configuration alignment; it does not prove that a particular client or fork
|
||||
successfully exposes every REA tool.
|
||||
|
||||
Codex treats an unset or empty `CODEX_HOME` as `~/.codex`. Its MCP server
|
||||
registration supports `startup_timeout_sec` for connection initialization and
|
||||
`tool_timeout_sec` for each tool call. New REA registrations use a 30-second
|
||||
@@ -446,9 +269,7 @@ not enforce Codex's 30 seconds as the doctor deadline. The 120000 ms
|
||||
tool-operation headroom and the 64 KiB doctor stderr cap stay fixed. Setup preserves positive caller-selected timeout
|
||||
values in the REA entry, and doctor reports a non-blocking advisory when a tool
|
||||
timeout is shorter than the known provider allowance. An explicitly selected
|
||||
JEB request budget also contributes to the allowance, and an explicit
|
||||
`REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS` is added to the Ghidra startup
|
||||
allowance. A configured IDA
|
||||
JEB request budget also contributes to the allowance. A configured IDA
|
||||
registration's parsed `timeoutMs` contributes for `ida` or `auto` selection.
|
||||
An explicit `REA_JADX_OPERATION_TIMEOUT_MS` is forwarded with the JADX
|
||||
registration but does not contribute; raise `tool_timeout_sec` to cover a
|
||||
@@ -457,44 +278,12 @@ These values are client-side deadlines, so increase them in `~/.codex/config.tom
|
||||
workload needs more time. Devin honors `XDG_CONFIG_HOME` for its
|
||||
`devin/mcp_config.json` on non-Windows systems.
|
||||
|
||||
### Codex with Moonshot or Kimi models
|
||||
|
||||
An older REA input containing acyclic local references was rejected by a
|
||||
Responses-compatible Kimi gateway as “infinite recursion”; expanding those
|
||||
references succeeded. A separate full-profile input failed the provider's
|
||||
schema-size limit. These are distinct reported failures, not proof that the
|
||||
canonical schema is recursive. Current REA inputs are already self-contained;
|
||||
see [schema advertisement](mcp-contracts.md#schema-advertisement).
|
||||
|
||||
For this failure, inspect the active MCP launch command and refresh the REA
|
||||
installation that command actually runs. A configured package selector or the
|
||||
installed skill version alone does not identify a cached or still-running
|
||||
server. Follow [installation and update guidance](installation.md#diagnose-update-and-remove).
|
||||
Then merge this setting into the REA entry in `$CODEX_HOME/config.toml`
|
||||
(`~/.codex/config.toml` by default), preserving its launcher and other settings:
|
||||
|
||||
```toml
|
||||
[mcp_servers.rea.env]
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE = "compact"
|
||||
```
|
||||
|
||||
If that table already exists, add or update the key there. Restart/reconnect
|
||||
Codex so REA reads the new environment. Compact is now REA's default, but an
|
||||
explicit `full` override remains effective until changed. It preserves the
|
||||
tool inventory and canonical server validation while reducing advertised input
|
||||
detail. The probes in [#5160](https://github.com/morluto/rea/issues/5160) covered
|
||||
individual tools through one gateway, not the entire current catalog or a
|
||||
model-driven analysis workflow.
|
||||
|
||||
## OMP
|
||||
|
||||
For OMP, setup writes a `type: "stdio"` entry to the user-level
|
||||
`~/.omp/agent/mcp.json`. A relative `PI_CONFIG_DIR` selects a configuration root
|
||||
under the home directory. For the default profile, `PI_CODING_AGENT_DIR`
|
||||
overrides the agent directory; a relative value uses the setup working
|
||||
directory. A named profile selected by `OMP_PROFILE` or `PI_PROFILE` instead
|
||||
uses `<configuration-root>/profiles/<name>/agent/mcp.json` and ignores
|
||||
`PI_CODING_AGENT_DIR`. Setup also removes
|
||||
`~/.omp/agent/mcp.json`. It follows `PI_CONFIG_DIR` and
|
||||
`PI_CODING_AGENT_DIR` (relative to the setup working directory), and the profile selected by `OMP_PROFILE` or
|
||||
`PI_PROFILE` (`~/.omp/profiles/<name>/agent/mcp.json`). Setup also removes
|
||||
`rea` from that file's `disabledServers` list, which would otherwise hide the
|
||||
registration. Doctor treats an `enabled: false` entry as active when
|
||||
`enabledServers` lists `rea`, as OMP does, unless `disabledServers` also lists
|
||||
@@ -507,51 +296,13 @@ comments and unrelated settings. It resolves that file from `HERMES_HOME`, else
|
||||
from the platform default Hermes itself uses — `%LOCALAPPDATA%\hermes` on
|
||||
Windows, `~/.hermes` elsewhere — including any `HERMES_DATA_DIR_SUFFIX`. Hermes
|
||||
connects an entry unless `enabled` is false, so setup states `enabled: true`.
|
||||
Aliases whose anchor is defined in the `rea` entry retain their original resolved
|
||||
values when setup refreshes or uninstall removes that entry. Setup does not
|
||||
redirect another server's alias to REA's new launcher or policy.
|
||||
This also covers an anchor on the removed registration key. References to
|
||||
definitions outside the entry keep sharing the original value, including when
|
||||
another definition later reuses the same anchor name.
|
||||
|
||||
Hermes personal skills install under the active Hermes home's `skills` directory.
|
||||
Setup, doctor and uninstall share REA's sticky-profile resolver.
|
||||
A custom `HERMES_HOME` outside the platform default home
|
||||
reads its own `active_profile`. An override physically inside the default home
|
||||
uses that default home's sticky selection and profile root together, including
|
||||
when the final override directory does not yet exist.
|
||||
|
||||
For example, with `HERMES_HOME=~/.hermes/sub` and `~/.hermes/active_profile` set
|
||||
to `coder`, setup selects `~/.hermes/profiles/coder/config.yaml` and that
|
||||
profile's skills directory. It does not read `~/.hermes/sub/active_profile`.
|
||||
Without an outer named selection, this override uses `~/.hermes/sub/config.yaml`.
|
||||
Use a home outside the platform default tree for an independent sticky selection.
|
||||
|
||||
This describes REA's current routing, not full Hermes CLI equivalence.
|
||||
At Hermes commit `748c79a1`, the
|
||||
[sticky lookup](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_cli/main.py#L612-L632)
|
||||
also reads the outer selection, but
|
||||
[`resolve_profile_env`](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_cli/profiles.py#L2431-L2461)
|
||||
resolves it beneath the configured `HERMES_HOME`. For nested overrides, confirm
|
||||
the profile Hermes actually uses and point `HERMES_HOME` directly at that
|
||||
`profiles/<name>` directory before setup. Compatibility is tracked in
|
||||
[#3939](https://github.com/morluto/rea/issues/3939).
|
||||
|
||||
REA resolves existing ancestor directories before checking physical containment,
|
||||
including symlinks and junctions, even when the override directory does not exist.
|
||||
An override beneath a symlinked default home therefore uses the default home's
|
||||
sticky selection. If `~/.hermes/link` points outside the default home,
|
||||
`HERMES_HOME=~/.hermes/link/new` instead uses that override root, whether or not
|
||||
`new` exists. Previewed destinations retain the configured path spelling.
|
||||
An unreadable path or dangling alias leaves the home unresolved; repair that path
|
||||
before configuring Hermes. Inspect setup's previewed destinations before applying
|
||||
changes.
|
||||
|
||||
An explicit `HERMES_HOME` pointing to `profiles/<name>` selects that named
|
||||
profile directly, bypassing sticky selection. An invalid, missing or deleted
|
||||
sticky profile is reported instead of writing into the default home; repair the
|
||||
profile or run `hermes profile use default` before retrying. It also remains an
|
||||
unresolved skill selection, so setup cannot report a successful empty installation.
|
||||
Setup, doctor and uninstall follow the root's `active_profile` selection, as
|
||||
plain Hermes CLI commands do. An explicit `HERMES_HOME` pointing to
|
||||
`profiles/<name>` selects that named profile directly. An invalid, missing or
|
||||
deleted sticky profile is reported instead of writing into the default home;
|
||||
repair the profile or run `hermes profile use default` before retrying.
|
||||
REA follows `HERMES_HOME` whitespace trimming, environment-variable expansion,
|
||||
and home expansion before resolving that profile. Setup previews the resolved
|
||||
configuration and skill destinations; doctor and uninstall use those same paths.
|
||||
@@ -568,12 +319,10 @@ installs its skill copy there.
|
||||
## Factory Droid
|
||||
|
||||
For [Factory Droid](https://docs.factory.com/harness/mcp), setup writes
|
||||
`mcpServers.rea` into the user-level `~/.factory/mcp.json` (the
|
||||
`FACTORY_HOME_OVERRIDE` override replaces the home that contains `.factory`).
|
||||
Entries carry `command`, `args`, and `env`. Droid discovers personal skills
|
||||
from `~/.factory/skills` ([skills](https://docs.factory.com/harness/skills),
|
||||
following `FACTORY_HOME_OVERRIDE` when set), so setup installs its skill copy
|
||||
there.
|
||||
`mcpServers.rea` into the user-level `~/.factory/mcp.json`. Entries carry
|
||||
`command`, `args`, and `env`. Droid discovers personal skills from
|
||||
`~/.factory/skills` ([skills](https://docs.factory.com/harness/skills)), so
|
||||
setup installs its skill copy there.
|
||||
|
||||
## Trae
|
||||
|
||||
@@ -621,43 +370,14 @@ the fallback file untouched. Existing REA environment overrides are preserved
|
||||
when refreshing its command. A malformed or unreadable fallback file fails setup instead of
|
||||
being silently suppressed; repair or remove it and rerun. Uninstall removes an
|
||||
REA-owned `rea` entry from both the native table and the fallback, backing up
|
||||
each changed file. If removing the last native server would activate a retained
|
||||
fallback server, uninstall refuses before changing either file; review the
|
||||
fallback configuration before retrying. Other fallback servers, settings and a
|
||||
foreign `rea` entry are kept. ZCode disables a
|
||||
each changed file, so an emptied native table cannot load a leftover fallback
|
||||
registration; other fallback servers, settings and a foreign `rea` entry are
|
||||
kept. ZCode disables a
|
||||
server with `enable: false` on its entry; doctor reports such a registration
|
||||
as stale, and rerunning setup removes the flag to repair it. Native configuration
|
||||
uses strict JSON; setup refuses comments, trailing commas and a BOM rather than
|
||||
writing a registration ZCode cannot load.
|
||||
|
||||
## FreeBuff
|
||||
|
||||
For FreeBuff, the proposed integration writes `mcpServers.rea` into the user-scope
|
||||
`~/.agents/mcp.json` shared-agent file. Entries carry `command`, `args`, and
|
||||
`env`; stdio is the default transport. The proposed personal skill location is
|
||||
`~/.agents/skills`, so setup installs its skill copy there. Because the same
|
||||
file doubles as ZCode's user-scope fallback (see above), a `freebuff`
|
||||
registration is also in the file ZCode uses until ZCode defines its own native server.
|
||||
Setup and doctor detect FreeBuff from its own profile directory
|
||||
(`~/.config/freebuff-desktop`), not from the shared `.agents` folder that
|
||||
REA's skill installs create for every client.
|
||||
|
||||
The configuration write and personal skill locations are proposed integration
|
||||
boundaries. FreeBuff MCP loading and skill discovery have not been verified
|
||||
with an installed native FreeBuff client/version; filesystem tests cover REA's
|
||||
editing, discovery, and ownership behavior only. Automatic applicability
|
||||
requires the FreeBuff profile directory even when the shared MCP file exists.
|
||||
When uninstall cannot establish FreeBuff or another shared-storage reader from
|
||||
product discovery, a malformed or unreadable shared MCP file is retained
|
||||
unchanged and does not stop other REA removals, including `--purge-data`. When
|
||||
FreeBuff or ZCode product applicability is established or unresolved, the same
|
||||
malformed shared file stops uninstall before removals. See
|
||||
[Diagnose, update, and remove](installation.md#diagnose-update-and-remove).
|
||||
|
||||
FreeBuff setup support requires `[email protected]` or later. Older versions
|
||||
reject `rea setup --client freebuff` as an unsupported client. See the
|
||||
[installation guide](installation.md#supported-agents) for client availability.
|
||||
|
||||
## DeepSeek Harness
|
||||
|
||||
DeepSeek Harness registers MCP clients through a Cordis YAML patch sequence
|
||||
@@ -723,10 +443,8 @@ become active again. Malformed applicable files block setup before changes.
|
||||
Grok Build loads `[mcp_servers.rea]` from `$GROK_HOME/config.toml`, or from
|
||||
`~/.grok/config.toml` when `GROK_HOME` is unset or empty. Setup edits that server
|
||||
table, `[mcp_servers.rea.env]`, and a root `disabled_mcp_servers` entry that
|
||||
names `rea`. It defaults `startup_timeout_sec` to 30 seconds, preserves an
|
||||
existing positive timeout, and leaves every other name in that list.
|
||||
[Grok supports longer startup timeouts](https://docs.x.ai/build/features/mcp-servers)
|
||||
for cold package-runner launches. The shared skill installed under `~/.agents/skills` is already
|
||||
names `rea`. It sets `startup_timeout_sec = 30` and leaves every other name
|
||||
in that list. The shared skill installed under `~/.agents/skills` is already
|
||||
on Grok Build's skill path.
|
||||
|
||||
Grok Bot (`grok_bot`) is detected from `~/.grokbot`, or from `SAND_DATA_ROOT`
|
||||
@@ -745,42 +463,3 @@ npx -y rea-agents@<version> mcp
|
||||
|
||||
Do not put credentials in that command or its arguments. `rea doctor --client grok_bot`
|
||||
reports this manual step. `rea uninstall` does not remove the account connector.
|
||||
|
||||
## Skill selection and maintenance
|
||||
|
||||
OpenCode's selected configuration layers determine the registration dialect.
|
||||
If any loaded layer uses native `mcp.servers`, setup writes a native REA entry
|
||||
in the selected upper file even when that file is absent or has no MCP table.
|
||||
Legacy activation is normalized before layer merging, so an explicit enable
|
||||
can override an inherited disable. Lower files and other server entries remain
|
||||
unchanged; a legacy server named `servers` that conflicts with the native table
|
||||
requires manual repair before setup can write.
|
||||
|
||||
With no client selected, the interactive skill choice targets the shared
|
||||
`~/.agents/skills` copy. Selecting a client targets its personal destination
|
||||
when supported. Use `rea setup --skill-client shared --skill` for the shared
|
||||
copy, or `--skill-client claude_code --skill` for that client's copy. Setup
|
||||
resolves the final selection before deciding which files need installation.
|
||||
Doctor retains each selected stale or missing path and names the matching
|
||||
repair command; a current copy elsewhere does not establish freshness.
|
||||
|
||||
Unresolved profile or directory selections remain diagnostics and never become
|
||||
write targets. Update maintenance considers only known enabled, owned REA
|
||||
registrations and owned skills. If identity or a selected skill root is
|
||||
unresolved, it reports maintenance as unavailable with the known scope and
|
||||
remaining diagnostics. An empty writable scope alone does not prove current.
|
||||
|
||||
When configuration publication fails, setup retains the affected path, operation
|
||||
and available filesystem code in the client result. Owner/group and mode
|
||||
preservation failures identify that operation rather than implying that the
|
||||
configuration bytes or parent directory are unwritable. Setup preserves the
|
||||
original and reports its recovery backup; review local ownership and group
|
||||
membership before retrying.
|
||||
|
||||
For an installed POSIX CLI, setup prefers an executable absolute `rea` alias
|
||||
from its selected `PATH` when filesystem identity proves that it names the
|
||||
running REA entrypoint. This keeps package-manager aliases intact instead of
|
||||
persisting an internal versioned script path. Unresolved aliases and wrappers
|
||||
that are different files do not establish that proof; setup retains its selected
|
||||
entrypoint. A manager-specific upgrade remains subject to that manager's alias
|
||||
policy.
|
||||
@@ -1,26 +1,11 @@
|
||||
# Client and agent verification
|
||||
|
||||
FreeBuff MCP loading and personal skill discovery are currently unverified with
|
||||
an installed native client/version. Its temporary-filesystem coverage verifies
|
||||
REA's configuration and applicability boundaries only; it does not establish
|
||||
that FreeBuff consumes the written registration or skill.
|
||||
|
||||
Use these optional lanes to verify an installed client or evaluate an agent's
|
||||
REA workflow. [Testing REA](testing.md) provides the normal development commands
|
||||
and provider lanes. Each record below states its host, client version, model
|
||||
fixture, and remaining coverage; a native-client run with a loopback model does
|
||||
not establish live model-service acceptance.
|
||||
|
||||
## Launch environment verification
|
||||
|
||||
`rea mcp doctor --json` starts the production MCP adapter with the current runtime
|
||||
executable and environment. A healthy result establishes that adapter's
|
||||
initialization, discovery and target-free request flow; it does not verify a
|
||||
client's configured launcher or its inherited environment. Test the generated
|
||||
registration through the actual client, including a reduced `PATH` when checking
|
||||
desktop or service startup. Record the client version and launch environment
|
||||
separately from terminal doctor results. See [launcher troubleshooting](client-configuration.md#launchers-and-path).
|
||||
|
||||
## Published Pi code-mode renderer limit
|
||||
|
||||
A separate probe of the published `@earendil-works/pi-codemode` 1.1.0 and 1.2.0
|
||||
@@ -63,14 +48,6 @@ skill setting; Hermes discovers it through the selected `HERMES_HOME`.
|
||||
Set `REA_VERIFY_HERMES_STICKY_PROFILE=1` to exercise native Hermes selection of
|
||||
a named sticky profile with `HERMES_HOME` still pointing to its root.
|
||||
|
||||
For Qwen on POSIX, set `REA_VERIFY_QWEN_REDUCED_PATH=1` and select its JavaScript
|
||||
entry point with `REA_VERIFY_QWEN_COMMAND` to test a restricted client environment.
|
||||
Setup retains the caller's `PATH`; the native Qwen `chat` or `call` run then uses
|
||||
an isolated `PATH` containing only Bash for the shell-tool fixture. The lane
|
||||
checks that `node`, `npm` and `npx` cannot be found and uses the generated REA
|
||||
registration unchanged. Its receipt records `reducedPath`, `clientPath` and
|
||||
`registrationCommand`; the option alone does not establish a passing native run.
|
||||
|
||||
The Qwen lane also supports native Windows (verified with Qwen Code 0.25.1 and
|
||||
Node 24.19.0 for both `chat` and `call`). With Node 24.18.0, the same `call`
|
||||
journey reached full Evidence readback but the client aborted during exit with
|
||||
@@ -172,12 +149,6 @@ REA_VERIFY_COPILOT_MODEL=gpt-4.1 REA_VERIFY_COPILOT_SCHEMA_PROFILE=compact npm r
|
||||
REA_VERIFY_COPILOT_MODEL=gpt-4.1 REA_VERIFY_COPILOT_SCHEMA_PROFILE=compact npm run verify:copilot-client -- call
|
||||
```
|
||||
|
||||
The verifier explicitly writes either selected profile into its disposable
|
||||
registration and compares the input schemas forwarded by Copilot with that
|
||||
profile's advertised schemas. A run must not infer `full` from an unset setting:
|
||||
REA's runtime default is now `compact`. The recorded client results above remain
|
||||
scoped to their executed revisions; this verifier correction is not a new client run.
|
||||
|
||||
Both native workflows passed with the complete tool inventory. The verifier selects the
|
||||
profile only in its disposable Copilot registration and records it in the
|
||||
receipt. Compact advertisements can reduce nested validation structure; the
|
||||
@@ -481,11 +452,6 @@ real local Codex CLI with:
|
||||
npm run verify:agent
|
||||
```
|
||||
|
||||
The verifier's scenarios, transcript scoring, fixture rubrics, and report gates
|
||||
are development verification code under `tests/evaluation/model-evals/`. The
|
||||
source-owned evaluator runs alongside the compiled REA tools; it is excluded
|
||||
from the shipped runtime package.
|
||||
|
||||
The lane runs scenarios sequentially in standalone Codex (`--no-daemon`) with a
|
||||
disposable HOME and CODEX_HOME under `~/.cache/rea-agent-evaluations/`. Set
|
||||
`REA_AGENT_EVAL_ROOT` to choose another parent directory outside the OS temporary
|
||||
@@ -509,16 +475,10 @@ is removed after confirmed process cleanup even with
|
||||
`REA_AGENT_EVAL_KEEP_FIXTURES=true`. No user configuration,
|
||||
rules, plugins or history is copied; shell snapshots and additional agents are
|
||||
disabled. Existing API-key environment authentication remains available.
|
||||
The native provider scenario preapproves only `open_binary` and
|
||||
The native missing-provider scenario preapproves only `open_binary` and
|
||||
`close_binary` in the disposable client so it reaches REA's actual provider
|
||||
boundary instead of stopping at Codex approval. Its scenario contract accepts
|
||||
one `provider_unavailable` open refusal without a close, or one successful open
|
||||
followed by a successful close. Other errors, an unclosed session, and a close
|
||||
without acquisition fail. These outcomes and the prompt are owned together in
|
||||
`tests/evaluation/model-evals/AgentEvaluationScenarios.ts`; generic successful-tool sequences
|
||||
still apply to scenarios that require them. The provider scenario checks routing,
|
||||
reported limitations, and cleanup; free-form answer correctness remains
|
||||
unassessed. It does not validate a real native engine or authorize runtime execution.
|
||||
boundary instead of stopping at Codex approval. This does not validate a real
|
||||
native engine or authorize arbitrary runtime execution.
|
||||
|
||||
Select a smaller run with `REA_AGENT_EVAL_SCENARIOS`, a comma-separated list of
|
||||
scenario IDs. `asar-zh` exercises the packaged desktop rubric from a Chinese
|
||||
@@ -531,10 +491,8 @@ limit for the whole process tree when testing on a constrained host; a Node
|
||||
heap limit alone does not bound client and child-process memory.
|
||||
|
||||
`missing-target` leaves desktop fixtures visible without selecting one in the
|
||||
request. It requires zero REA calls and a JSON-only request for an app name or
|
||||
artifact path, with no preceding prose or Markdown fences. Its prompt and
|
||||
whole-answer validator are defined together in `AgentEvaluationScenarios.ts`.
|
||||
Its `expectedFirstTool` is `null`, and
|
||||
request. It requires zero REA calls and a structured request for an app name or
|
||||
artifact path. Its `expectedFirstTool` is `null`, and
|
||||
`targetClarificationPassed` grades that limited routing outcome independently
|
||||
of analysis prose heuristics. Analyzing a nearby example cannot pass this case.
|
||||
|
||||
@@ -596,10 +554,7 @@ tool results and final answers for review.
|
||||
Report schema version 4 adds negative routing scenarios with nullable
|
||||
`expectedFirstTool` and a `targetClarificationPassed` outcome. Consumers must
|
||||
allow an expected absence of REA calls; this does not make an analysis scenario
|
||||
pass without its required tools. Native provider records additionally report
|
||||
`scenarioOutcome` (`provider-unavailable`, `opened-and-closed`, or `failed`) and
|
||||
`scenarioOutcomePassed`; a failed outcome fails the gate independently of prose
|
||||
heuristics. Version 3 changed the top-level
|
||||
pass without its required tools. Version 3 changed the top-level
|
||||
`factualCorrectness` from a constant
|
||||
string to an assessment summary with status, scope, and assessed/passed/failed/
|
||||
not-assessed scenario counts. Scenario records include the factual assessment and
|
||||
|
||||
@@ -1,121 +1,35 @@
|
||||
# Analysis settings, limits, and recovery
|
||||
# Admission limits and recovery
|
||||
|
||||
This guide explains the runtime settings design for maintainers and developers:
|
||||
who owns each value, when an update takes effect, how providers enforce it, and
|
||||
how to extend the system. The public tool contract is in
|
||||
[MCP contracts](../mcp-contracts.md#editable-analysis-settings); recorded runtime
|
||||
checks and their limitations are in the
|
||||
[verification report](../verification/runtime-analysis-settings.md).
|
||||
|
||||
## Problem and design boundary
|
||||
|
||||
A valid workload can exceed a conservative default. An oversized ASAR header
|
||||
may prevent analysis of the entire archive; a slow Ghidra function may exceed
|
||||
its decompile deadline. Requiring a new REA process to change either budget
|
||||
interrupts the investigation and can discard reusable provider state.
|
||||
|
||||
REA exposes a curated set of analysis defaults through MCP so callers can
|
||||
adjust a reported limit and retry the same workload in the running runtime.
|
||||
Each setting has a typed value, resource bounds, an enforcement owner, and a
|
||||
defined application point. The design applies across archive admission,
|
||||
provider startup, operation deadlines, and worker memory requests.
|
||||
|
||||
The setter changes in-memory defaults. It does not edit arbitrary environment
|
||||
variables, configuration files, MCP registrations, or provider installations.
|
||||
Those mechanisms have different owners and lifetimes. It also cannot change a
|
||||
client deadline, extend accepted work, release retained resources, or remove a
|
||||
fixed representation limit. Exposing a setting requires a working enforcement
|
||||
path, not just an entry in the tool schema.
|
||||
|
||||
Further ceilings that should become settings are tracked in
|
||||
How to add a REA-owned admission ceiling, budget, or deadline, and what a caller
|
||||
does when one is hit. This is a design rule, not an inventory. Ceilings that
|
||||
should become settings are tracked in
|
||||
[#6176](https://github.com/morluto/rea/issues/6176). Naming honesty for limit
|
||||
errors is [#5910](https://github.com/morluto/rea/issues/5910).
|
||||
|
||||
## Ownership and data flow
|
||||
|
||||
| Owner | Responsibility |
|
||||
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| [Domain definitions](https://github.com/morluto/rea/blob/main/src/domain/analysisSettingDefinitions.ts) and [types](https://github.com/morluto/rea/blob/main/src/domain/analysisSettings.ts) | Exact setting IDs, value types, shared bounds/defaults, units, provider/operation associations, application points, and structured limit metadata. Numeric constants stay with their domain owner. |
|
||||
| [Launch parser](https://github.com/morluto/rea/blob/main/src/config/analysisSettings.ts) | Parse selected launch values once, preserving an invalid value as a typed configuration failure rather than substituting a default. |
|
||||
| [AnalysisSettings](https://github.com/morluto/rea/blob/main/src/application/AnalysisSettings.ts) | Own one runtime's launch baseline, immutable override revisions, atomic updates, inspection, and accepted-operation snapshots. It does not own provider lifecycle or cleanup. |
|
||||
| [Composition](https://github.com/morluto/rea/blob/main/src/main/eagerRuntime.ts) and [typed configuration views](https://github.com/morluto/rea/blob/main/src/config/runtimeAnalysisSettings.ts) | Inject the same settings reader into providers and workflows; translate overrides into the configuration each consumer understands. |
|
||||
| Provider or resource owner | Resolve the effective budget, enforce it at the applicable boundary, and attach the actual value and known source to failures. |
|
||||
| [MCP registration boundary](https://github.com/morluto/rea/blob/main/src/server/EvidenceMcpServerCore.ts) | Enter the operation's settings scope before invoking its handler. |
|
||||
| [Settings contracts](https://github.com/morluto/rea/blob/main/src/contracts/analysisSettingsContracts.ts) and [tool registration](https://github.com/morluto/rea/blob/main/src/server/registerAnalysisSettingsTools.ts) | Validate named requests and deliver inspection/update results without requiring an open target or healthy analysis provider. |
|
||||
|
||||
The normal path is: composition creates the store, an MCP handler captures its
|
||||
current revision, and providers read that snapshot when resolving budgets.
|
||||
If an owner rejects work, it reports the effective limit through the shared
|
||||
error projection. A subsequent settings update publishes a new revision for
|
||||
later handlers. It does not reach into the failed operation or retry it.
|
||||
|
||||
### Accepted-operation snapshots
|
||||
|
||||
`AnalysisSettings.run()` installs the current immutable snapshot in
|
||||
`AsyncLocalStorage`. Its `read` function returns that snapshot inside the
|
||||
operation and the latest revision outside it. Providers depend on the reader
|
||||
contract rather than on the mutable store or an ambient environment lookup.
|
||||
|
||||
Acceptance here is entry into the registered handler's settings scope, not
|
||||
arrival of bytes at the MCP transport. Work queued or awaited within that scope
|
||||
keeps its snapshot. Deferred server initialization must still dispatch through
|
||||
the canonical scoped handler.
|
||||
|
||||
| Event | Revision used |
|
||||
| --------------------------------------------------------- | ----------------------------------- |
|
||||
| Operation A enters its handler, then waits for a provider | Current revision 0 |
|
||||
| A settings update publishes revision 1 | Future handlers can read revision 1 |
|
||||
| Operation A resumes and starts its worker | Its accepted revision 0 |
|
||||
| Operation B enters its handler | Revision 1 |
|
||||
|
||||
This gives each operation stable budget semantics even if another connection
|
||||
updates settings while it waits. `next_worker` and `next_startup` describe where
|
||||
an accepted value is applied; they do not cause workers or startups belonging
|
||||
to an older operation to adopt a newer revision.
|
||||
|
||||
Inspection deliberately reads the store's current defaults, rather than the
|
||||
calling operation's snapshot. It reports `active_values.status: "not_observed"`:
|
||||
an inspection result cannot establish the budget or remaining time of an
|
||||
already-running worker.
|
||||
|
||||
### Composition and embedding
|
||||
|
||||
The MCP runtime creates one store and supplies its reader to provider factories.
|
||||
`createServer` uses a supplied store or creates one, and wires its default
|
||||
providers to that store. An embedder supplying a custom binary session or other
|
||||
provider must also inject the same reader into that provider's composition and
|
||||
pass the store to `createServer`. Merely exposing the settings tools cannot
|
||||
make a custom provider consume updates.
|
||||
|
||||
Keep default factories, explicitly supplied providers, transport composition,
|
||||
and deferred handler dispatch covered separately where their wiring differs.
|
||||
Resolve settings at the operation boundary; copying a runtime value into a
|
||||
provider constructor would freeze it at launch. For entrypoints outside the
|
||||
MCP wrapper, an embedder needing the same snapshot guarantee must establish
|
||||
the operation scope with `AnalysisSettings.run()`.
|
||||
|
||||
## Name a default, then a setting when operators need headroom
|
||||
|
||||
Put the number in a named constant with a unit. Do not leave it as a magic
|
||||
number at the call site.
|
||||
|
||||
When a workload may need more room, add a setting whose unset value is that
|
||||
constant. Pure numeric contracts live in `src/domain/analysisResourceLimits.ts`
|
||||
or their existing domain owner. `analysisSettingDefinitions.ts` binds the
|
||||
curated runtime IDs to those same bounds, defaults and application points.
|
||||
Environment parsers and recovery text stay under `src/config/`; register launch
|
||||
schemas on `environmentSchema` so startup and doctor agree. Add launch settings
|
||||
to `SELECTED_REGISTRATION_SETTINGS` when setup should forward them. Unset keeps
|
||||
today's behavior. An invalid launch value remains a configuration failure,
|
||||
never a silent fallback; its validation boundary determines whether MCP can
|
||||
start (see [invalid launch configuration](#invalid-launch-configuration)). An
|
||||
invalid runtime write is `invalid_request` and changes nothing.
|
||||
constant. One module under
|
||||
[`src/config/`](https://github.com/morluto/rea/blob/main/src/config) owns the
|
||||
default, any floor, the maximum, the schema, the parser, and the recovery
|
||||
text. Register that schema on `environmentSchema` in
|
||||
[`environment.ts`](https://github.com/morluto/rea/blob/main/src/config/environment.ts).
|
||||
`parseConfig` calls `parseEnvironment`. Doctor uses that same parser through
|
||||
`nativeAnalysisConfigurationError`, so startup and `rea doctor` accept the
|
||||
same values. Add the name to `SELECTED_REGISTRATION_SETTINGS` so `rea setup`
|
||||
copies it into the client registration. Unset keeps today's behavior. An
|
||||
invalid value is a `ConfigurationError` and projects as `configuration_invalid`.
|
||||
Do not fall back to the default.
|
||||
|
||||
For example, `regex.matching_deadline_ms` and its launch seed
|
||||
`REA_REGEX_MATCHING_DEADLINE_MS` share a 5000 ms default, 1 ms floor and
|
||||
2147483647 ms maximum. Node schedules a longer `setTimeout` delay as 1 ms.
|
||||
Changing the runtime setting affects the next accepted search; changing the
|
||||
launch environment affects the next REA process.
|
||||
[`REA_REGEX_MATCHING_DEADLINE_MS`](https://github.com/morluto/rea/blob/main/src/config/regexMatchingDeadline.ts)
|
||||
([#6267](https://github.com/morluto/rea/pull/6267)) is the reference. Default
|
||||
5000 ms, floor 1 ms, maximum 2147483647 ms, because Node schedules a longer
|
||||
`setTimeout` delay as 1 ms. Setup copies the setting. A hit says:
|
||||
|
||||
> The effective REA_REGEX_MATCHING_DEADLINE_MS value is 5000 ms (source default; default 5000 ms, floor 1 ms, maximum 2147483647 ms). Set REA_REGEX_MATCHING_DEADLINE_MS in the environment that starts REA to a positive decimal integer from 1 to 2147483647, then restart REA. The worker old-generation heap does not raise this deadline.
|
||||
|
||||
[`REA_MCP_MAX_RESPONSE_BYTES`](https://github.com/morluto/rea/blob/main/src/config/mcpResponseBudget.ts)
|
||||
is the byte-budget shape: a decimal safe integer with a 10 MiB floor
|
||||
@@ -125,8 +39,7 @@ raise REA's reserved string limit or Node's string ceiling. The setup
|
||||
allowlist does not copy this setting. A new headroom setting should, following
|
||||
the regex deadline.
|
||||
|
||||
A per-call budget can have a runtime default without having an environment
|
||||
setting. JavaScript `max_heap_mb`
|
||||
A per-call budget is not an environment setting. JavaScript `max_heap_mb`
|
||||
(CLI `--max-heap-mb`) is an integer from 128 to 16384 MiB, default 1024. It
|
||||
applies on the next analysis, which starts a new worker, and does not restart
|
||||
REA. It does not raise the parent result budget, transfer ceilings, or fixed
|
||||
@@ -138,144 +51,8 @@ does: its recovery says to set `NODE_OPTIONS` (for example
|
||||
`--max-old-space-size=<MiB>`) on the process that starts REA, then restart
|
||||
that process. `max_heap_mb` does not raise it.
|
||||
|
||||
## Runtime settings through MCP
|
||||
|
||||
`inspect_analysis_settings` lists the curated settings, their current and launch
|
||||
values, source, bounds, application point and relevant operation names. Filters
|
||||
by exact setting IDs, provider or operation intersect. Active worker values are explicitly `not_observed`; a configured default is
|
||||
not a measurement of active work. Inspection also separates known server limits from unknown client deadlines and receive buffers.
|
||||
|
||||
`update_analysis_settings` accepts an atomic `changes` array. Each entry names
|
||||
an exact `setting` and either `action: "set"` with a schema-validated `value`, or
|
||||
`action: "reset"`. Reset restores the launch value. `expected_revision` is an
|
||||
optional compare-and-set guard; conflicts and duplicate keys change nothing.
|
||||
An unchanged write does not advance the revision.
|
||||
|
||||
The revision tracks override state, not only effective numeric values. Setting
|
||||
an explicit override equal to the launch value still changes its source to
|
||||
`runtime` and advances the revision. Reset removes that override, even if the
|
||||
effective number stays the same. Validation, revision checks, and duplicate-key
|
||||
checks all happen before publishing the replacement snapshot; a rejected batch
|
||||
cannot partially apply. Clients coordinating writes can supply
|
||||
`expected_revision`; without it, valid updates apply to the current state.
|
||||
|
||||
| Runtime setting | Application point |
|
||||
| ------------------------------------------------------------- | ---------------------------------------------------------------------- |
|
||||
| `asar.max_header_bytes` | Next archive operation, including full JavaScript application analysis |
|
||||
| `ghidra.startup_timeout_ms`, `hopper.startup_timeout_ms` | Next provider startup attempt |
|
||||
| `ghidra.decompile_timeout_seconds` | Next operation on the existing Ghidra Program; `null` means unbounded |
|
||||
| `jadx.operation_timeout_ms` | Next JADX operation |
|
||||
| `regex.matching_deadline_ms`, `regex.worker_heap_request_mib` | Next search / new regex worker |
|
||||
| `capture.decoder_old_generation_mib` | Next owned capture decoder |
|
||||
| `javascript.analysis_timeout_ms`, `javascript.max_heap_mb` | Next source analysis / new JavaScript worker |
|
||||
| `wasm.command_timeout_ms` | Next WABT operation |
|
||||
| `native.matching_representation_bytes` | Next `match_native_functions` request |
|
||||
|
||||
### Precedence and lifetime
|
||||
|
||||
Where a setting supports an explicit per-call argument, precedence is:
|
||||
|
||||
1. The caller's explicit argument.
|
||||
2. The accepted operation's runtime override.
|
||||
3. Valid launch configuration, where that setting has an environment seed.
|
||||
4. The built-in default when launch configuration is absent.
|
||||
|
||||
An invalid launch value does not fall through to the built-in default. Preserve
|
||||
omitted caller arguments in Evidence: resolving an internal default must not
|
||||
rewrite the request as though the caller supplied it. `null` is the explicit
|
||||
unbounded value for `ghidra.decompile_timeout_seconds`; it is not a substitute
|
||||
for an unknown or omitted value elsewhere.
|
||||
|
||||
A write does not extend active work, relaunch a provider, clear retained
|
||||
cleanup, or invalidate successful Evidence.
|
||||
|
||||
Overrides belong to one REA runtime, survive target close/open, and disappear
|
||||
when REA exits. Connections sharing that runtime share its overrides. No files,
|
||||
ambient environment, registrations or upstream software are edited. Overrides
|
||||
are runtime-wide, not scoped to a target or an individual connection.
|
||||
|
||||
### Failure and retry
|
||||
|
||||
An ASAR admission error supplies `details.limit` with the exact setting,
|
||||
effective value, units, bounds, scope, application point, restart requirement
|
||||
and, when known, required bytes and value source. The agent can immediately call:
|
||||
|
||||
```json
|
||||
{
|
||||
"changes": [
|
||||
{ "setting": "asar.max_header_bytes", "action": "set", "value": 33554432 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Then retry the same full-archive request once work and cleanup have settled.
|
||||
No preliminary inspection or source subset is required. The setter does not
|
||||
retry automatically. If an explicit per-call value caused the failure, correct
|
||||
that argument; raising a runtime default cannot override it.
|
||||
|
||||
The resource owner constructs `details.limit` from the registry plus the budget
|
||||
it actually enforced. Metadata includes `agent_editable: true`,
|
||||
`scope: "rea-runtime"`, and `rea_restart_required: false`. Optional
|
||||
`required_value`, `source`, and `parameter` fields are supplied only where known
|
||||
or applicable. Do not infer a source because a value happens to equal a default.
|
||||
Error copying, attribution, redaction, and delivery must preserve these facts.
|
||||
Human-readable remediation explains the same failure; clients should not need
|
||||
to parse that sentence to identify an editable setting.
|
||||
|
||||
`scope` and `applies_at` describe a continuing caller, such as an MCP
|
||||
connection. A one-shot invocation, such as a CLI command, has released its
|
||||
provider session and exited its runtime before its caller reads the failure,
|
||||
so its adapter projects with an explicit `"invocation"` caller lifetime.
|
||||
`details.limit` then keeps every other field and reports
|
||||
`scope: "rea-invocation"`, `applies_at: "next_invocation"` and the launch
|
||||
`environment` seed when one exists. `agent_editable: true` still holds, because
|
||||
the caller chooses that environment value or per-call `parameter` for the new
|
||||
invocation; `rea_restart_required: false` means no running REA process needs a
|
||||
restart. A producer keeps guidance that holds only while its caller retains the
|
||||
session, such as "the Ghidra session remains usable" or a same-session runtime
|
||||
update, in `continuingCallerGuidance`; the shared presentation adds it only for
|
||||
a continuing caller. The producer's own message and remediation must stay true
|
||||
after the session is released. A composed error records each embedded child
|
||||
failure through `childFailureDiagnostics`, which snapshots it at composition
|
||||
for both lifetimes when they differ, so cleanup history keeps its state and
|
||||
the final projection selects the caller's form. Failures retained inside
|
||||
Evidence keep the continuing form, so CLI and MCP Evidence stay identical.
|
||||
|
||||
Cache policy is part of enforcement. Successful reusable analysis can survive
|
||||
a budget update. Transient Ghidra decompile failures remain uncached so a raised
|
||||
deadline can recover on the same Program. Admission checks still apply to reused
|
||||
data: lowering an ASAR limit must reject a cached oversized header. Neither
|
||||
raising a limit nor clearing a transient analysis failure authorizes skipping
|
||||
cleanup or changing the identity of the provider session.
|
||||
|
||||
### Invalid launch configuration
|
||||
|
||||
Launch validation has two relevant boundaries. Settings validated by global
|
||||
configuration parsing can prevent REA from starting; an MCP setter cannot
|
||||
repair a server that is not running. Correct the launch environment or
|
||||
registration, use `rea doctor` for its configuration checks, and start REA again.
|
||||
|
||||
Provider-local validation can leave MCP available. For example, an invalid
|
||||
`REA_JADX_OPERATION_TIMEOUT_MS` is inspectable with `source: "invalid"`, a
|
||||
`configuration_error`, and omitted `value` and `launch_value`. A valid runtime
|
||||
override repairs the effective JADX timeout without editing that launch seed.
|
||||
Inspection then reports the runtime value while retaining the launch error.
|
||||
Reset exposes the invalid launch state again; it does not repair or replace it
|
||||
with the built-in default. Provider availability and execution must resolve the
|
||||
same effective setting so readiness changes along with repair and reset.
|
||||
`REA_NATIVE_MATCHING_REPRESENTATION_BYTES` is validated the same way, by
|
||||
`match_native_functions` alone.
|
||||
|
||||
## Keep a fixed ceiling honest
|
||||
|
||||
Inspection's separate
|
||||
[constraints](https://github.com/morluto/rea/blob/main/src/server/analysisSettingsConstraints.ts)
|
||||
describe operator-owned, external, and fixed controls. REA can report its
|
||||
configured response budget and parent heap limit; client deadlines and receive
|
||||
buffers remain explicitly unknown. These are not entries accepted by the
|
||||
runtime setter. Raising a worker heap request cannot raise the parent heap,
|
||||
and raising a provider deadline cannot make the client wait longer.
|
||||
|
||||
If the ceiling must stay fixed, say so in the failure. Name the limit and the
|
||||
effective value, say this release has no runtime override, and name what does
|
||||
not raise it. Do not invent a setting.
|
||||
@@ -291,24 +68,20 @@ The string the caller sees includes:
|
||||
|
||||
- the limit or setting name
|
||||
- the effective value and unit
|
||||
- the source when known (`default`, `setting`, `runtime`, or `request`)
|
||||
- the source (`default` or `setting`)
|
||||
- the default, the maximum, and the floor when the setting has one
|
||||
- the next action, including where to set it ("in the environment that starts REA")
|
||||
- whether a restart is required
|
||||
- what the change does not raise
|
||||
|
||||
Distinguish an invalid configuration from a valid budget exhausted by work.
|
||||
Preserve the configuration failure and its setting identity; apply the
|
||||
[launch validation boundary](#invalid-launch-configuration) when describing
|
||||
whether live repair is possible.
|
||||
Invalid settings never take this path. They are `configuration_invalid`.
|
||||
Doctor reports `runtime-configuration:<setting>` and says to correct it in the
|
||||
shell or the MCP registration, then rerun.
|
||||
|
||||
## What agents do
|
||||
|
||||
Use `details.limit` for a direct runtime update when it names an editable
|
||||
setting with `scope: "rea-runtime"`. For `scope: "rea-invocation"`, set its
|
||||
`environment` or `parameter` for a new invocation instead.
|
||||
Follow `remediation.action` for per-call, fixed and external controls.
|
||||
Inspect settings only when bounds or application details are missing. Restart only when it says the value is
|
||||
Follow `remediation.action`. Change a request field or a `REA_*` setting only
|
||||
when that text recommends the change. Restart only when it says the value is
|
||||
read at startup. A sentence that a control does not raise the limit is not a
|
||||
request to change it.
|
||||
|
||||
@@ -330,90 +103,22 @@ request to change it.
|
||||
- `resource_constraint` for capacity, when `retryable` is true: wait for the
|
||||
active admission to finish, then retry within the reported limits. Do not
|
||||
restart REA to free the slot.
|
||||
- `configuration_invalid`: when MCP remains available and the failure belongs
|
||||
to a curated provider-local setting, inspect and set a valid runtime override.
|
||||
Otherwise correct the named setting in the environment that starts REA, then
|
||||
start REA again. Run `rea doctor` for the
|
||||
- `configuration_invalid`: correct the named setting in the environment that
|
||||
starts REA, then start REA again. Run `rea doctor` for the
|
||||
`runtime-configuration:` check. Re-run `rea setup` for that client when the
|
||||
setting is on the registration allowlist, then reconnect so the new process
|
||||
sees it.
|
||||
- `invalid_request`: follow the specific recovery and reported issue. Inspect
|
||||
the schema or CLI help when the input violates the contract or the response
|
||||
asks for `inspect_input_contract`. A valid selector can instead require a
|
||||
different choice, such as selecting a reported address; an inventory miss
|
||||
does not establish that the target lacks the item. Do not repeat the
|
||||
unchanged call when the response says it cannot succeed unchanged.
|
||||
- `invalid_request`: correct the reported fields from the current schema or
|
||||
CLI help. Do not repeat the unchanged call.
|
||||
|
||||
A restart is required to adopt a changed launch environment or registration.
|
||||
The curated runtime settings above require no REA restart. It is not the first step for a transient
|
||||
A restart is required for an environment setting and for a client still
|
||||
running the previous registration. It is not the first step for a transient
|
||||
MCP drop, a capacity wait, or retained cleanup.
|
||||
|
||||
## Adding a runtime setting
|
||||
## Tests
|
||||
|
||||
1. Identify the enforcing owner and the real resource constraint. Decide whether
|
||||
callers need a per-call argument, a runtime default, a launch-only control,
|
||||
or a fixed limit. A registry entry must not advertise authority REA lacks.
|
||||
2. Define the value type and named numeric bounds with the domain owner. Add the
|
||||
ID and metadata to `analysisSettings.ts` and `analysisSettingDefinitions.ts`.
|
||||
Reuse those bounds in runtime validation, launch parsing, and error metadata.
|
||||
Choose the application point and provider/operation associations from actual
|
||||
consumers; a shared control need not apply to every provider.
|
||||
3. If there is a launch seed, add its parser and appropriate configuration/doctor
|
||||
validation. Decide whether an invalid value blocks the runtime or only the
|
||||
provider. Add it to setup's `SELECTED_REGISTRATION_SETTINGS` when it should
|
||||
be forwarded; runtime editability does not automatically propagate it.
|
||||
4. Thread the shared reader through every applicable composition path. Resolve
|
||||
the effective value within the accepted operation, honor explicit arguments,
|
||||
and preserve their omission in Evidence. Keep availability checks consistent
|
||||
with execution. A worker heap request must not be described as a measured
|
||||
process memory limit.
|
||||
5. Enforce the value at its resource boundary and construct structured limit
|
||||
facts there. Preserve those facts through shared error projection and copies.
|
||||
Check cache reuse, lowered admission, transient failures, and retained cleanup
|
||||
against the same ownership rules.
|
||||
6. Update the public contracts and affected guides, then regenerate catalogs
|
||||
and packaged skill metadata using the repository commands. Keep externally
|
||||
owned and fixed controls in the separate inspection constraints, not in the
|
||||
editable registry. Validate both store semantics and a public recovery path.
|
||||
|
||||
## Tests and verification
|
||||
|
||||
Cover the unset default, accepted overrides, rejected floor/maximum values,
|
||||
and invalid syntax through the owning parser and configuration boundary.
|
||||
Lock the unset default, one accepted override, rejected floor, maximum, and
|
||||
syntax, and the recovery sentence through `parseEnvironment` and `parseConfig`.
|
||||
[`regexMatchingDeadline.test.ts`](https://github.com/morluto/rea/blob/main/src/config/regexMatchingDeadline.test.ts)
|
||||
is one example. Assert error codes, setting IDs, values, units, provenance,
|
||||
application points, and resulting behavior. Check that adapters deliver the
|
||||
canonical recovery guidance, with independent assertions for its underlying
|
||||
facts. Avoid freezing prose or deriving all expected values from the same
|
||||
registry used by the implementation; either can hide the wrong failure.
|
||||
See [test design](../testing.md#test-design-and-pruning) for the assertion policy.
|
||||
|
||||
Runtime checks cover atomic validation, stale writes, reset, runtime isolation,
|
||||
accepted-operation snapshots, and explicit argument precedence. Public MCP
|
||||
acceptance must reject an oversized ASAR, raise admission, and successfully
|
||||
retry the same complete archive on the same connection. Lowering admission must
|
||||
also refuse previously cached headers. The real Ghidra timeout verifier checks
|
||||
recovery after a live update without replacing its session. Also cover invalid
|
||||
provider-local launch configuration, repair/reset availability, default and
|
||||
custom composition paths, and preservation of structured limits through error
|
||||
copies. Gates used to overlap work and updates need bounded acknowledgements
|
||||
and release/join in `finally`.
|
||||
|
||||
The [verification report](../verification/runtime-analysis-settings.md) separates
|
||||
public MCP checks, simulated provider coverage, real Ghidra execution, and
|
||||
measured task costs. It does not establish real-provider behavior for every
|
||||
editable setting or active worker telemetry.
|
||||
|
||||
## Bun standalone module graphs
|
||||
|
||||
`MAX_BUN_MODULE_COUNT` admits at most 100,000 module records.
|
||||
`MAX_BUN_RETAINED_BYTES` admits at most 256 MiB of serialized graph payload and
|
||||
separately bounds retained entry metadata. Physical metadata read work is also
|
||||
limited to 256 MiB per preparation, including repeated uncached ranges. A
|
||||
256 KiB / 1,024-entry exact-range cache avoids full-page amplification. The
|
||||
combined logical path tree admits at most 250,000 distinct prefixes; this is
|
||||
checked before adding nodes, with shared prefixes counted once. The shared
|
||||
artifact expansion budget also bounds the sum of referenced member lengths. Refusals name the boundary.
|
||||
There is no runtime override; additional host memory or `NODE_OPTIONS` does not
|
||||
raise these ceilings. Use a smaller producer fixture or a separately reviewed
|
||||
reader for larger graphs. No partial extraction is claimed on admission failure.
|
||||
is the pattern. Project the failure and assert `configuration_invalid` for a
|
||||
bad value and the recovery wording for a hit.
|
||||
@@ -53,7 +53,7 @@ Every collection supports `fields`, `offset` and `limit`; its stable row ID is
|
||||
always included. Selection changes delivery, not source inspection. Pages keep
|
||||
source coverage and parent Evidence links. Page exhaustion does not establish
|
||||
absence in a partial source. Complete Evidence remains accessible through
|
||||
`get_evidence_bundle` and Evidence bundle export.
|
||||
`get_evidence` and Evidence bundle export.
|
||||
|
||||
The summary's `probe_issues` counts files with reported issues and their retained
|
||||
issue entries. It previews at most two such files, including stable `file_id`,
|
||||
@@ -103,12 +103,11 @@ same bundle. Export writes only the caller-selected output file.
|
||||
|
||||
When the four-byte metadata magic is present but version bytes are incomplete,
|
||||
the fact retains `raw_version: null`; it does not offer an IL2CPP metadata
|
||||
inspection probe. A complete magic/version prefix with supported raw version
|
||||
24, 27, 29, 31 or 39 offers an `inspect_il2cpp_metadata` follow-up at its
|
||||
admitted file path, including when no native library was found. This probe does
|
||||
not establish full metadata validity or correspondence to a native binary;
|
||||
version 24 inspection still applies only to supported header layouts. See
|
||||
[IL2CPP metadata](il2cpp-metadata.md).
|
||||
inspection probe. A complete magic/version prefix with supported raw version 29,
|
||||
31 or 39 offers an
|
||||
`inspect_il2cpp_metadata` follow-up at its admitted file path, including when no
|
||||
native library was found. This probe does not establish full metadata validity
|
||||
or correspondence to a native binary. See [IL2CPP metadata](il2cpp-metadata.md).
|
||||
|
||||
Candidates are derived or inferred from these facts and do not prove loading or
|
||||
execution. BepInEx, MelonLoader, Easy Anti-Cheat and BattlEye labels identify
|
||||
@@ -135,13 +134,11 @@ observations, not an atomic directory snapshot. Unread bytes remain unknown.
|
||||
## Resource bounds and reuse
|
||||
|
||||
Defaults are 20,000 examined entries and 8 MiB of total probe reads. Callers may
|
||||
select up to 100,000 entries and 64 MiB. A separate 32 MiB representation
|
||||
budget bounds retained rows, paths and facts before growth. Each retained path
|
||||
is charged at the larger of its UTF-16 heap size and its escaped JSON size, so
|
||||
typical macOS bundle paths fit about 20,000 rows, and control-character paths
|
||||
fewer. Limits and failed probes produce partial coverage. Traversal and reads
|
||||
are serial. Each reader admits one inspection at a time; overlapping requests
|
||||
receive a retryable capacity refusal without cancelling the admitted caller.
|
||||
select up to 100,000 entries and 64 MiB. A separate 32 MiB conservative
|
||||
representation budget bounds retained paths and facts before growth. Limits
|
||||
and failed probes produce partial coverage. Traversal and reads are serial.
|
||||
Each reader admits one inspection at a time; overlapping requests receive a
|
||||
retryable capacity refusal without cancelling the admitted caller.
|
||||
|
||||
Unity reference decoding stops after a witness in each `Data/Managed` directory;
|
||||
other assemblies' references remain unexamined, while their headers are still
|
||||
|
||||
@@ -1,105 +0,0 @@
|
||||
# Flattened devicetree inspection
|
||||
|
||||
`inspect_devicetree` / `rea inspect-devicetree` reads a local DTB without an
|
||||
engine, target execution or external tools. It supports FDT versions 16 and 17;
|
||||
other versions return `unsupported_target`. Corrupt recognized blobs return
|
||||
`invalid_request`, and exhausted budgets return `resource_constraint` without
|
||||
partial success.
|
||||
|
||||
```bash
|
||||
rea inspect-devicetree /artifacts/board.dtb --json
|
||||
rea inspect-devicetree /artifacts/board.dtb --detail summary --evidence_output /artifacts/board-evidence.json --json
|
||||
```
|
||||
|
||||
The complete Evidence includes the artifact SHA-256, validated header fields,
|
||||
64-bit memory reservations as exact hexadecimal strings, and a flat node tree
|
||||
with explicit parent paths. Each property retains its file offset, byte length,
|
||||
and up to 256 raw bytes as hex. `truncated` qualifies the preview only; parsing
|
||||
coverage includes the entire declared tree. Version 16 has no declared structure
|
||||
size, so `header.size_dt_struct` is null.
|
||||
|
||||
Derived fields carry `confidence: derived`: standard ASCII string lists, scalar
|
||||
phandles and cell counts, and `reg` tuples using the parent's `#address-cells`
|
||||
and `#size-cells` (defaults 2 and 1). Zero-cell components are null (omitted), distinct from an encoded zero. Cell
|
||||
counts above four, invalid counts and incomplete tuples remain undecoded with a
|
||||
limitation. Addresses are bus-local; no `ranges` translation is attempted.
|
||||
Invalid or ambiguous strings remain raw, rather than being guessed.
|
||||
|
||||
The summary is labeled derived and includes root compatible/model, chosen
|
||||
bootargs/stdout-path and aliases. An undecodable alias retains its name with a
|
||||
null path; it is not silently treated as absent. Single-cell `interrupt-parent`, `remote-endpoint` and `msi-parent`
|
||||
references resolve through phandle declarations. Missing or duplicate targets
|
||||
have null target paths. A node whose `phandle` and `linux,phandle` disagree
|
||||
adds a limitation, and references to either value stay unresolved. A present
|
||||
`phandle` that is not one cell also adds a limitation; `linux,phandle` cannot
|
||||
resolve that node when the canonical declaration is malformed. Binding-specific
|
||||
reference arrays and overlays are not resolved; an empty reference list does not
|
||||
prove absence of hardware links.
|
||||
|
||||
MCP `detail: summary` retains complete Evidence in the session. Follow its
|
||||
`parent_evidence_id` with `inspect_analysis_view`:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": { "kind": "retained-evidence", "evidence_id": "ev:<digest>" },
|
||||
"view": {
|
||||
"kind": "devicetree",
|
||||
"collection": "properties",
|
||||
"path": "/serial@1000",
|
||||
"offset": 0,
|
||||
"limit": 32
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Collections are `nodes` (identity and property counts), `properties` (requires an
|
||||
exact node path), `reservations`, and `references`. Pages use retained parsed
|
||||
facts and do not reopen the file. Each page carries fixed-size context
|
||||
(artifact, header, coverage and counts) rather than the summary strings, so a
|
||||
large `bootargs` value does not repeat on every page. Source indices are scoped to the selected
|
||||
collection and parent Evidence; property indices are scoped to their node.
|
||||
|
||||
| Input budget | Default | Ceiling |
|
||||
| ---------------- | ------: | ------: |
|
||||
| `max_file_bytes` | 4 MiB | 16 MiB |
|
||||
| `max_nodes` | 8,192 | 65,536 |
|
||||
| `max_properties` | 32,768 | 262,144 |
|
||||
| `max_depth` | 64 | 256 |
|
||||
|
||||
Decoded names, paths, values and row overhead share a fixed 32 MiB representation
|
||||
allowance, charged before expansion. It has no runtime override.
|
||||
|
||||
The reservation count also uses `max_properties`. CLI flags use kebab case,
|
||||
for example `--max-nodes 16384`. View pages contain at most 256 rows.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic tests cover v16/v17, malformed headers/blocks/names/tokens, resource
|
||||
limits, derived values, and CLI/MCP delivery including retained pages after
|
||||
removing the input. The optional real lane requires **dtc 1.8.1**:
|
||||
|
||||
```bash
|
||||
npm run build:cached
|
||||
node scripts/verify/firmware/devicetree.mjs
|
||||
```
|
||||
|
||||
It compiles a DTS with reserved-memory, chosen, aliases and phandles, compares
|
||||
REA's node paths against `dtc -I dtb -O dts`, and checks the derived report.
|
||||
`REA_DTC_PATH` selects the verification executable; it is not runtime setup.
|
||||
|
||||
The dtc 1.8.1 lane ran on macOS arm64 with Node 26.5.0. Its seven node paths
|
||||
matched the decompiled tree. The report included `model: "REA DTB verification"`,
|
||||
`bootargs: "console=ttyS0"`, alias `serial0 -> /serial@1000`, reservation
|
||||
`0x80000000 + 0x1000`, and a resolved reference from `/consumer` to
|
||||
`/serial@1000`. The deliberately minimal remote-endpoint fixture produces dtc
|
||||
graph-binding warnings; this lane verifies FDT structure and selected property
|
||||
derivations, not hardware-binding validity.
|
||||
|
||||
A repeat run of that complete compile/decompile/CLI verification took 2.13 s
|
||||
elapsed, 1.52 s user CPU and 0.27 s system CPU under `/usr/bin/time -l`. This is a
|
||||
small fixture with warm filesystem caches, not a cold-start or large-board
|
||||
benchmark. Peak process-family memory and whole-workflow token usage were not
|
||||
measured; the sandbox denied the memory-reporting system query. No pre-feature
|
||||
equivalent REA workflow exists for comparison. The pinned Node 24.22.1/npm
|
||||
11.16.0 toolchain was unavailable; dependency installation used the local cache
|
||||
with Node 26.5.0/npm 12.2.0 instead.
|
||||
@@ -1,110 +0,0 @@
|
||||
# .NET single-file bundles
|
||||
|
||||
Use `rea inspect-artifact /absolute/path/host --json` to inventory an ordinary
|
||||
.NET single-file host and `rea extract-artifact /absolute/path/host --json` to
|
||||
materialize its members into a fresh, REA-owned output directory. With a host
|
||||
selected through `open_binary`, the corresponding MCP tools are
|
||||
`inspect_artifact` and `extract_artifact`. No .NET installation is needed for
|
||||
inspection or extraction.
|
||||
|
||||
For a compact inventory, call `inspect_artifact` with `detail: "summary"`,
|
||||
then use `inspect_analysis_view` with the returned `parent_evidence_id`.
|
||||
Inventory occurrence pages accept `logical_paths`, `occurrence_ids`, selected
|
||||
`fields`, `offset`, and `limit`; node pages accept batches of `artifact_ids`
|
||||
to retrieve expanded-byte digests. Include `format_metadata` and
|
||||
`source_location` when member declarations and stored extents matter. Identity
|
||||
fields survive projection, and the complete inspection Evidence stays retained.
|
||||
|
||||
REA detects the apphost marker and its preceding header offset from PE, ELF,
|
||||
and thin Mach-O bytes, regardless of filename. An unbundled apphost's zero
|
||||
offset remains an ordinary native artifact. Universal Mach-O slice inventory
|
||||
keeps its existing behavior; bundled contents inside slices are not expanded.
|
||||
When a native file has a recognized PyInstaller CArchive, apphost-marker
|
||||
scanning stops before its payload. A .NET helper stored inside that archive
|
||||
does not replace the outer container's format; an outer .NET host can still
|
||||
contain a PyInstaller helper as a member.
|
||||
|
||||
## Supported layout
|
||||
|
||||
| Header | Producer generation | Member encoding |
|
||||
| ------ | ---------------------------------- | --------------------------- |
|
||||
| 1.0 | .NET Core 3.x | Uncompressed |
|
||||
| 2.0 | .NET 5 | Uncompressed |
|
||||
| 6.0 | .NET 6 and later using this layout | Uncompressed or raw DEFLATE |
|
||||
|
||||
Versions, minor versions, and flags outside this set are refused explicitly;
|
||||
they do not silently become an ordinary PE or a successful empty inventory.
|
||||
Flags 0 and the v2+ compatibility flag 1 are admitted. Unknown file-type bytes
|
||||
are retained with a limitation, rather than treated as executable authority.
|
||||
Encrypted/protected wrappers and recovering their payloads are outside this
|
||||
reader's scope.
|
||||
|
||||
The host keeps its whole-file SHA-256. Each embedded file gets its expanded-byte
|
||||
SHA-256, path, declared size, compressed size, and `format_metadata` declaration
|
||||
(`major_version`, `minor_version`, `file_type`). Its `source_location` describes
|
||||
the stored host extent, including the compressed length when applicable.
|
||||
The root manifest's `root_format` is `dotnet-bundle`; the root content node
|
||||
still records its native PE/ELF/Mach-O format. A file-type declaration or `.dll`
|
||||
suffix does not establish a valid assembly. Inspect an extracted component
|
||||
explicitly with the managed tools to establish PE/CLI facts.
|
||||
|
||||
## Admission and containment
|
||||
|
||||
Metadata is range-read with a 64 KiB cache. Policy budgets are 16 MiB of
|
||||
manifest metadata, 100,000 files, 64 KiB per UTF-8 string, 512 MiB of stored or
|
||||
expanded bytes per file, 256 path parts per member, 250,000 distinct in-memory
|
||||
path-tree nodes, and 2 GiB of aggregate expanded bytes. The node budget counts
|
||||
shared directory prefixes only once. These are REA
|
||||
resource policies, not limits imposed by Microsoft's format. Marker scanning
|
||||
streams 64 KiB chunks and reuses completed probe facts during extraction;
|
||||
ordinary native files without a marker incur one extra
|
||||
read pass before host hashing.
|
||||
|
||||
Extraction hashes the whole host again through the same descriptor used to read
|
||||
members and compares it with the inventory's root digest before creating an
|
||||
output directory. This adds a full-host read pass to extraction, including when
|
||||
the embedded members themselves are small.
|
||||
|
||||
Malformed/truncated headers, unsafe or conflicting paths, overlapping member
|
||||
ranges, members crossing the manifest, contradictory configuration extents,
|
||||
invalid UTF-8, corrupt/trailing compressed data, and incorrect expanded sizes
|
||||
fail before output publication. Source descriptor/path identity is checked
|
||||
during reading; extraction verifies the host digest and member bytes against
|
||||
the inventory. Existing output roots are never overwritten. Cancellation and
|
||||
failures release owned streams/descriptors through the shared extraction
|
||||
rollback scope.
|
||||
|
||||
## Reproduce producer fixtures
|
||||
|
||||
The source-owned generator uses the installed .NET 8 SDK's **Microsoft
|
||||
HostModel** producer to emit x64 v1, v2, plain v6, and compressed v6 bundles
|
||||
for Windows, Linux, or macOS. It does not run generated hosts or download runtime packs. Supply an
|
||||
SDK apphost template from
|
||||
`packs/Microsoft.NETCore.App.Host.<rid>/<version>/runtimes/<rid>/native/apphost`
|
||||
(`apphost.exe` on Windows). Pass `windows` (the default), `linux`, or `osx`
|
||||
as the optional third generator argument to match that template.
|
||||
|
||||
```bash
|
||||
dotnet build scripts/fixtures/dotnet-bundle/BundleFixture.csproj -c Release --configfile scripts/fixtures/dotnet-bundle/NuGet.Config -o /absolute/path/producer -p:BaseIntermediateOutputPath=/absolute/path/producer-obj/
|
||||
dotnet /absolute/path/producer/BundleFixture.dll /absolute/path/apphost.exe /absolute/path/absent-bundles-directory
|
||||
rea inspect-artifact /absolute/path/absent-bundles-directory/net8-compressed/Probe.exe --json
|
||||
rea extract-artifact /absolute/path/absent-bundles-directory/net8-compressed/Probe.exe --json
|
||||
```
|
||||
|
||||
The generator writes `oracle.json` beside each host. An independent C#
|
||||
`BinaryReader`/`DeflateStream` pass records version, member types and stored
|
||||
extents, then compares every expanded member digest with its source file.
|
||||
Compare those records with inspection Evidence and the extracted bytes.
|
||||
This verifies Microsoft's compression writer separately from synthetic parser
|
||||
fixtures; it is not evidence that the bundled host executes on a target runtime.
|
||||
|
||||
Deterministic boundary and public-entrypoint coverage:
|
||||
|
||||
```bash
|
||||
npm run test:focused -- tests/boundary/filesystem/dotnetBundleArtifacts.test.ts tests/acceptance/dotnetBundleJourney.test.ts
|
||||
```
|
||||
|
||||
Layout references: Microsoft's [manifest writer](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/Bundle/Manifest.cs),
|
||||
[file entry writer](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/Bundle/FileEntry.cs),
|
||||
[apphost marker handling](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/AppHost/HostWriter.cs),
|
||||
and [compression producer](https://github.com/dotnet/runtime/blob/v8.0.22/src/installer/managed/Microsoft.NET.HostModel/Bundle/Bundler.cs).
|
||||
@@ -29,20 +29,15 @@ files, including pages a packaged application serves from inside `app.asar`.
|
||||
For those, the first `.asar` path component that is a regular file is the
|
||||
archive, the remainder must name a regular file in its header, and the reported
|
||||
path is the canonical archive path joined with that member. Remote hosts,
|
||||
encoded separators in the pathname, ASAR directories or links, and nonexistent paths are
|
||||
encoded path separators, ASAR directories or links, and nonexistent paths are
|
||||
rejected. The endpoint exposes every eligible target and its local path
|
||||
metadata. Query strings and fragments do not participate in filesystem path
|
||||
admission, including their encoded separator text. Listing and inspection
|
||||
Evidence both retain the observed browser product/revision; host platform and
|
||||
architecture remain explicitly unknown.
|
||||
metadata.
|
||||
|
||||
Cancellation stops further file authorization after an already-started filesystem
|
||||
operation settles. ASAR descriptor-close failures remain typed incomplete cleanup
|
||||
with an exact retryable owner; they are not treated as a missing or unauthorized
|
||||
page. A later authorization and provider shutdown retry retained descriptors
|
||||
before reporting their release. Failed listing retains its already-authorized
|
||||
target prefix and explicit remaining-target uncertainty through cleanup errors
|
||||
and exact-owner retries.
|
||||
before reporting their release.
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -75,47 +70,13 @@ DOM node indices retain their snapshot positions. Invalid rows are omitted with
|
||||
completeness diagnostics, so `total_nodes` can exceed the retained node count
|
||||
and a parent index can refer to an omitted source node.
|
||||
|
||||
`dom.node_value_length_unit` is `utf8-bytes`: each node's `node_value_length`
|
||||
counts the bytes needed to encode its value as UTF-8, not characters. Web
|
||||
observations instead declare `utf16-code-units`. These existing measurements
|
||||
are preserved; the unit is reported once for the whole DOM, without retaining
|
||||
text values. Historical captures without the unit leave it unknown; reading
|
||||
them does not infer a unit or convert their stored lengths.
|
||||
|
||||
Script metadata retains its execution-context frame ID when CDP supplies one and
|
||||
records admission separately in `frame_scope`. An `authorized` scope refers to
|
||||
the observed document generation; `unknown` retains the known frame ID without
|
||||
claiming authorization and makes script coverage attach-limited. Historical
|
||||
captures may omit this field, which leaves that admission fact unknown.
|
||||
Scripts from document generations whose frames were explicitly excluded by
|
||||
local-file policy are omitted before script-source reads. Frame identity can
|
||||
come from the script event's own auxiliary data or its observed execution
|
||||
context; contradictory identities are not aligned.
|
||||
`frame_claims.script_event_frame_id` and
|
||||
`frame_claims.execution_context_frame_id` preserve the actual observed claims
|
||||
from those two sources. Both remain available when their disagreement leaves
|
||||
the resolved `frame_id` null and `frame_scope` unknown. An omitted claim is
|
||||
unobserved, not proof that a frame was absent.
|
||||
Frames describe the current inventory; script observations can retain earlier
|
||||
authorized frames that have since detached.
|
||||
|
||||
`has_source_url` preserves CDP's sourceURL declaration flag as true, false, or
|
||||
unknown (`null`, or omitted in historical captures). A script's canonical
|
||||
`file_path` is the admitted local-file URL or supplied sourceURL label. It does
|
||||
not verify that the runtime bytes came from that disk file; captured source
|
||||
bytes have their own digest. `reported_url` retains the original URL or label
|
||||
before canonical path admission. Labels remain distinct from filesystem provenance.
|
||||
the observed frame generation is authorized. Frames describe the current inventory;
|
||||
script observations can retain earlier authorized frames that have since detached.
|
||||
The capture also
|
||||
inventories worker, service-worker, and shared-worker targets with
|
||||
validated opener-target and parent-frame IDs. Worker discovery uses passive
|
||||
target metadata; REA does not attach to or execute code in those targets.
|
||||
Script metadata uses the shared 8 MiB JSON-byte allowance for each retained
|
||||
representation: incoming drafts and normalized capture metadata. Admission
|
||||
counts escaped strings, frame claims, and provenance fields before retaining
|
||||
them or serializing metadata for a digest. Exhaustion preserves earlier records
|
||||
and reports counted `resource_budget_exhausted` script omissions; a later
|
||||
smaller record may still fit. This allowance adds no script-count cap and is
|
||||
separate from script-source bytes.
|
||||
Script-source collection retains at most 32 MiB of UTF-8 text across scripts
|
||||
and shares a 20-second collection deadline. Omitted sources retain their script
|
||||
metadata and an explicit reason; later smaller sources may still fit after an
|
||||
@@ -126,11 +87,6 @@ target, frame, script, and resource, a worker URL must resolve to a local
|
||||
file before it is retained. Relationship IDs improve
|
||||
attribution but do not prove which static module started a worker or that its
|
||||
work completed.
|
||||
Missing opener and parent relationships leave worker ownership unknown. Such
|
||||
targets are withheld with an `unattributed_target` exclusion and unavailable
|
||||
`workers` coverage, even when their URL names a local file. Explicit foreign
|
||||
relationships remain `out_of_target_scope`; malformed supplied relationship IDs
|
||||
are `invalid_protocol_value`.
|
||||
|
||||
Inspection does not retain DOM values, execute renderer code, navigate, click,
|
||||
invoke Electron IPC, close a target, or terminate the application.
|
||||
@@ -147,16 +103,6 @@ window-targeted renderer reload/crash actions, and synthetic `open-url` or
|
||||
WebContents identities, process metrics, and IPC channel and value-shape metadata.
|
||||
Payload values are not retained.
|
||||
|
||||
For the exact Playwright launch sequence, the active hook removes only REA's
|
||||
bootstrap arguments from the application-visible `process.argv`. The vector keeps
|
||||
the selected entry point and every caller argument, in order, including repeated
|
||||
hook paths, `.asar` filenames, and debug-like options. The private launch handoff
|
||||
is consumed before application code runs. If the observed sequence cannot be
|
||||
matched exactly, argv stays unchanged and capture coverage reports `hook_conflict`;
|
||||
REA does not guess an entry point or remove arguments by filename patterns.
|
||||
|
||||
`window_index` selects from Playwright's current Page list at the start of each action. REA resolves that Page to its corresponding BrowserWindow once; the action's `target` uses the BrowserWindow's capture-scoped `window_id`, matching the window inventory. Reordering either enumeration after selection does not retarget the action. Handle cleanup failures retain the completed action records and remain owned for provider cleanup retry.
|
||||
|
||||
Run the real fixture verifier with an operator-provided Electron runtime:
|
||||
|
||||
```bash
|
||||
@@ -181,10 +127,8 @@ native-addon, process, and IPC timeline events. IPC channels and argument-shape
|
||||
metadata retain their observed lengths; payload values are never retained.
|
||||
Renderer crash/restart and deep-link actions are synthetic
|
||||
scenario controls; their attempted and observed outcomes remain in the
|
||||
timeline. The active hook blocks and records the shell methods listed below
|
||||
and selected navigation, permission, download, popup, updater, and
|
||||
OS-integration APIs. These method patches do not contain direct filesystem,
|
||||
network, native-addon, or other unpatched target effects. The timeline
|
||||
timeline. The active hook blocks and records external shell/navigation,
|
||||
permission, download, popup, updater, and OS-integration effects. The timeline
|
||||
is explicitly partial when attachment starts after application activity. The
|
||||
startup, including the initial state snapshot, has a 60-second deadline.
|
||||
Actions wait until completion or request cancellation; a click whose target
|
||||
@@ -193,17 +137,11 @@ window, remains pending until cancelled. After cancellation, the final state
|
||||
read has 5 seconds before REA keeps the initial snapshot instead; the
|
||||
cancellation keeps the caller's reason and records that expiry in its
|
||||
diagnostics.
|
||||
The shell policy is owned by `blockedShellResults` in
|
||||
`scripts/electron-active-hook-boundaries.cjs`. Its synchronous blocks are:
|
||||
Blocked synchronous shell APIs keep their return contracts:
|
||||
`showItemInFolder` returns `undefined` and `writeShortcutLink` returns `false`.
|
||||
Their `shell-attempt` events record the blocked phase and argument shapes;
|
||||
neither result means the requested host operation occurred. The existing
|
||||
`openExternal`, `openPath`, and `trashItem` blocks reject their promises.
|
||||
The `trashItem` block prevents that call from reaching the OS trash operation;
|
||||
it does not intercept other ways the application can remove files.
|
||||
Process rejection and uncaught-exception events are observed at dispatch without
|
||||
adding exception handlers. The runtime and application retain their warning,
|
||||
handler, and fatal-exit policies, including for rejected shell-block promises.
|
||||
`openExternal` and `openPath` blocks continue to reject their promises.
|
||||
CLI SIGINT/SIGTERM requests cancellation and awaits owned-process cleanup.
|
||||
Window IDs, URLs, and cached titles come from one main-process Electron snapshot,
|
||||
so a crashed renderer does not require a successful renderer-side title query.
|
||||
@@ -217,15 +155,6 @@ If cleanup cannot be verified, the error retains collected observations in
|
||||
`details.partial_observation` and marks their application cleanup `unverified`.
|
||||
Cleanup uncertainty does not authorize signaling unrelated processes.
|
||||
|
||||
On Linux, Chromium-based processes can overwrite the initial environment memory
|
||||
exposed through `/proc/<pid>/environ`. The environment may be readable while
|
||||
`REA_PROCESS_RUN_ID` is absent, even if the application still sees it through
|
||||
`process.env`. REA reports the missing marker and leaves lineage ownership and
|
||||
cleanup unverified; the PID, executable, or parent relationship cannot replace
|
||||
that ownership proof. This limitation can affect owned Electron captures on
|
||||
native Linux as well as WSL. It does not establish that all Electron versions or
|
||||
Linux environments behave the same way.
|
||||
|
||||
Main-process IPC recording keeps original-function removal for `on` and `once`
|
||||
registrations and records a fired `once` callback once. Native Node.js owns
|
||||
`once` registration, firing and callable dispatch as well as removal order and
|
||||
@@ -292,9 +221,7 @@ initialization, the hook leaves listener callbacks unwrapped and records a
|
||||
callback and receiver; callback execution remains unobserved. This comparison
|
||||
assumes Node's EventEmitter prototype has not itself been globally decorated
|
||||
before hook initialization. The provider loads the hook before the application
|
||||
and excludes `NODE_OPTIONS` from the child environment. It preserves selected
|
||||
`DISPLAY`, `XAUTHORITY` and `WAYLAND_DISPLAY` values, including the caller-selected
|
||||
X11 authorization-file path.
|
||||
and excludes `NODE_OPTIONS` from the child environment.
|
||||
|
||||
An opaque later listener or unsupported emitter state can prevent proof of
|
||||
removal order. The hook then delegates removal
|
||||
@@ -323,13 +250,8 @@ capture provides them.
|
||||
|
||||
ASAR observation reads the header through one open file handle and validates its
|
||||
declared lengths against the archive size. Header/index admission is the same
|
||||
budget used by static ASAR reads: 16 MiB / 16777216 bytes by default, seeded by
|
||||
`REA_ASAR_MAX_HEADER_BYTES` (at most 256 MiB). In MCP,
|
||||
`update_analysis_settings` can override `asar.max_header_bytes` for new
|
||||
operations without restarting REA. Cached headers are checked against each
|
||||
operation's admission, including after a decrease. See the
|
||||
[settings design](design/limits-and-recovery.md#failure-and-retry).
|
||||
An oversized header is
|
||||
budget used by static ASAR reads: 16 MiB / 16777216 bytes by default, or
|
||||
`REA_ASAR_MAX_HEADER_BYTES` (at most 256 MiB) when set. An oversized header is
|
||||
not parsed; selecting a member path does not bypass admission, and the page
|
||||
stays unauthorized. The header cache separately retains at most that many bytes
|
||||
of already-admitted encoded headers across eight archives. Evicting that cache does not raise admission.
|
||||
+5
-36
@@ -24,26 +24,12 @@ requires an absolute filesystem path and preserves that exact selection:
|
||||
|
||||
The inline Evidence result contains the artifact path, SHA-256, and byte length.
|
||||
`tables` lists each declared note program header or section, including the
|
||||
section name when a string table is present. Zlib- and Zstandard-compressed
|
||||
section-name tables are decompressed within the selected file-size budget. An unsupported
|
||||
compression type leaves section names unknown and adds a limitation; note bytes
|
||||
remain independently examined. Each `entries[]` item reports owner, type,
|
||||
descriptor SHA-256, and a hex preview for descriptors of 64 bytes or less.
|
||||
|
||||
A section name also stays null when its offset is outside the declared string
|
||||
table or its bytes lack a NUL terminator. A limitation explains that missing
|
||||
name. Note coverage remains complete when all declared note tables and entries
|
||||
were examined; an unavailable section name does not erase those note facts.
|
||||
section name when a string table is present. Each `entries[]` item reports owner,
|
||||
type, descriptor SHA-256, and a hex preview for descriptors of 64 bytes or less.
|
||||
|
||||
For GNU notes, `gnu.build_id_hex`, `gnu.abi_tag`, or `gnu.gold_version` are
|
||||
filled when the descriptor matches that type. Other owners keep `gnu` null.
|
||||
|
||||
For Linux `CORE` notes, `core.prpsinfo`, `core.file`, `core.siginfo`, or
|
||||
`core.auxv` are decoded for process arguments and status (`NT_PRPSINFO`), mapped
|
||||
file memory regions and paths (`NT_FILE`), fault signal and code (`NT_SIGINFO`), and
|
||||
auxiliary vectors (`NT_AUXV`) across 32-bit and 64-bit images. Other owners or unparsed
|
||||
notes keep `core` null.
|
||||
|
||||
`PT_NOTE` and `SHT_NOTE` are examined independently. The same file range may
|
||||
appear twice when both tables declare it.
|
||||
|
||||
@@ -54,31 +40,14 @@ the image never carried notes in another container or overlay.
|
||||
|
||||
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
|
||||
32 note entries are examined by default across every table. Retained note
|
||||
names, section names, GNU descriptor text, and core dump string representations share a scan budget based on
|
||||
the source bytes plus any decoded section-name table, capped at
|
||||
`max_file_bytes`. The raw source and decoded table must also fit together
|
||||
within that ceiling. If admission fails, the error reports the required
|
||||
`max_file_bytes`; raising it can admit the decoded table up to the operation's
|
||||
512 MiB maximum. Once admitted, raising that setting does not increase the
|
||||
scan budget for the same source and decoded names. Repeated table ranges count again. Each descriptor range is hashed once,
|
||||
names, section names, and GNU descriptor text share that file's byte budget,
|
||||
and repeated table ranges count again. Each descriptor range is hashed once,
|
||||
and each GNU version's trailing NUL run is scanned once; neither walk may
|
||||
exceed that byte ceiling. Malformed ELF headers, truncated header
|
||||
tables, declared note tables that extend past the file, and truncated
|
||||
note entries fail without partial success. ELF extended
|
||||
numbering (`PN_XNUM`, including a section count stored when `e_shnum` is 0) is
|
||||
unsupported and returns an unsupported-target outcome rather than invalid-input
|
||||
recovery.
|
||||
|
||||
Zstandard section-name tables also use a 128 MiB decoder window limit.
|
||||
|
||||
`SHF_COMPRESSED` note sections return `unsupported_target` before their payload
|
||||
is interpreted or decompressed. Their compression header is storage metadata,
|
||||
not a note header. The current result requires physical file offsets for each
|
||||
note and descriptor, which decoded compressed notes cannot provide. This refusal
|
||||
does not label compressed note storage as malformed or allocate its declared
|
||||
decoded size. Compressed section-name tables remain supported within the limits
|
||||
above. Inspect a separately decompressed copy when these notes are needed,
|
||||
retaining the original artifact for provenance.
|
||||
unsupported.
|
||||
|
||||
This workflow does not execute the image or verify GNU build-id uniqueness
|
||||
against package catalogs.
|
||||
+1
-83
@@ -27,11 +27,8 @@ The inline Evidence result contains the artifact path, SHA-256, and byte length.
|
||||
Repeated names retain separate chunk occurrences. Decoded tables and the Code
|
||||
header use the last occurrence of each name, matching OTP's loader; the chunk
|
||||
inventory still exposes earlier payloads and their source ranges.
|
||||
The BEAM form size bounds the parsed chunks. Bytes after the declared form are
|
||||
ignored by the parser but remain part of the selected artifact's byte length
|
||||
and SHA-256.
|
||||
|
||||
When a selected `AtU8` or `Atom` table is decoded, `atoms` is the 1-based name table and
|
||||
When `AtU8` or `Atom` is present, `atoms` is the 1-based name table and
|
||||
`module` is atom 1. Both the pre-OTP 28 table and the OTP 28+ compact table
|
||||
(negated count, compact-encoded name lengths) are read. `exports`, `imports`, and `lambdas` resolve atom indices to
|
||||
those names when the table is present; unresolved indices keep `function` or
|
||||
@@ -41,85 +38,6 @@ those names when the table is present; unresolved indices keep `function` or
|
||||
function count, and the instruction stream's file range. Missing optional
|
||||
tables are null, not empty-everywhere.
|
||||
|
||||
A zero-byte `AtU8` falls back to an available legacy `Atom` table, following
|
||||
[OTP 20.3's atom selection](https://github.com/erlang/otp/blob/a113f6117fd696ea6f84ed754055b4ec97a7ccb2/erts/emulator/beam/beam_load.c).
|
||||
A nonempty `AtU8` retains UTF-8 precedence, including an explicit zero atom
|
||||
count; malformed nonempty data fails instead of falling back. An empty atom
|
||||
chunk without an alternate table still fails its required count header.
|
||||
Absent and zero-byte optional `FunT` chunks produce `lambdas: null`, while a
|
||||
four-byte zero row count produces `lambdas: []`. This follows
|
||||
[OTP 29's positive-size lambda parsing rule](https://github.com/erlang/otp/blob/550d7b7898706c7822362c42e7b93120c1d1f29a/erts/emulator/beam/beam_file.c).
|
||||
Empty payloads retain their ranges and SHA-256 in the chunk inventory. These
|
||||
rules apply to the last occurrence of each chunk name; an empty final occurrence
|
||||
does not revive an earlier payload of that name. Nonempty truncated tables remain
|
||||
format failures.
|
||||
|
||||
## Select retained facts
|
||||
|
||||
Complete output remains the default. Request a summary with a `beam` view to
|
||||
read a few facts in the first call while retaining the complete inspection:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_erlang_beam",
|
||||
"arguments": {
|
||||
"path": "/artifacts/app.beam",
|
||||
"detail": "summary",
|
||||
"view": {
|
||||
"kind": "beam",
|
||||
"collection": "exports",
|
||||
"names": ["module_info"],
|
||||
"fields": ["function_index", "function", "arity", "label"],
|
||||
"limit": 20
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`normalized_result.page.items` contains the matching rows. `source_index` is
|
||||
zero-based in the original collection and scoped to `parent_evidence_id` and
|
||||
that collection. It does not replace the original one-based atom `index` or a
|
||||
lambda's own `index`. Every matching function name survives, including different
|
||||
arities. Names are exact retained display strings; raw indexes remain available
|
||||
when names are unresolved or displayed names collide.
|
||||
|
||||
Collections are `atoms`, `exports`, `imports`, `lambdas`, and `chunks`. `names`
|
||||
selects atom/chunk names or function names; imports additionally support exact
|
||||
`modules`. Filters intersect. `indices` selects original collection indexes;
|
||||
`fields` selects that collection's own fields. Empty selector arrays match
|
||||
nothing, and `fields: []` returns only `source_index`. Offset/limit page matching
|
||||
rows in original order (defaults 0/64). Unfiltered pages slice the retained
|
||||
collection; queries do not decode the file again.
|
||||
|
||||
For `chunks`, `names` matches the four-character IFF chunk tag (for example
|
||||
`"AtU8"`, `"Code"`, `"ExpT"`); available `fields` are `index`, `name`,
|
||||
`location`, and `payload`. For `lambdas`, `names` matches the resolved function
|
||||
name from the atom table; `null` names (unresolved atom indices) are never
|
||||
selected by a name filter. For `atoms`, `names` matches the decoded atom name
|
||||
string.
|
||||
|
||||
`beam` keeps the module, form/Code bounds, original source coverage, and table
|
||||
counts. A null count means no table was decoded; zero means a decoded empty
|
||||
table. Missing tables and unresolved atom names remain explicit, and an empty
|
||||
page does not establish runtime absence. Page coverage is separate from the
|
||||
source's examined chunks and parser budgets.
|
||||
|
||||
CLI summary delivery requires a destination for the complete Evidence bundle:
|
||||
|
||||
```sh
|
||||
rea inspect-erlang-beam /artifacts/app.beam --detail summary \
|
||||
--view '{"kind":"beam","collection":"exports","names":["module_info"],"fields":["function","arity"]}' \
|
||||
--evidence_output /reports/app-beam.evidence.json --json
|
||||
rea inspect-analysis-view '{"source":{"kind":"evidence-file","path":"/reports/app-beam.evidence.json"},"view":{"kind":"beam","collection":"imports","modules":["erlang"]}}' --json
|
||||
```
|
||||
|
||||
The bundle retains the complete report. Subsequent MCP queries may use
|
||||
`source: {kind: "retained-evidence", evidence_id: parent_evidence_id}` or an
|
||||
Evidence file. Retained and offline views still work after the original BEAM
|
||||
file is deleted. Supplying `view` requires `detail: "summary"`; omitting it
|
||||
returns only the structural summary. Selection changes delivery, not parsing or
|
||||
coverage, and executes no bytecode.
|
||||
|
||||
## Limits and failures
|
||||
|
||||
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
|
||||
|
||||
@@ -52,15 +52,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -124,15 +115,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -196,15 +178,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -268,15 +241,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -340,15 +304,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -412,15 +367,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -484,15 +430,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -556,15 +493,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -628,15 +556,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -700,15 +619,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -772,15 +682,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -790,8 +691,12 @@
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"logical_path": { "type": "string" },
|
||||
"declared_sha256": { "type": ["string", "null"] },
|
||||
"calculated_sha256": { "type": ["string", "null"] },
|
||||
"declared_sha256": {
|
||||
"anyOf": [{ "type": "string" }, { "type": "null" }]
|
||||
},
|
||||
"calculated_sha256": {
|
||||
"anyOf": [{ "type": "string" }, { "type": "null" }]
|
||||
},
|
||||
"unpacked": { "type": "boolean" }
|
||||
},
|
||||
"required": [
|
||||
@@ -800,7 +705,7 @@
|
||||
"calculated_sha256",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": {}
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
@@ -826,33 +731,7 @@
|
||||
"calculated_md5",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": {}
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"logical_path": { "type": "string" },
|
||||
"algorithm": { "type": "string", "const": "dfu-crc32" },
|
||||
"declared_crc32": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-f0-9]{8}$"
|
||||
},
|
||||
"calculated_crc32": {
|
||||
"anyOf": [
|
||||
{ "type": "string", "pattern": "^[a-f0-9]{8}$" },
|
||||
{ "type": "null" }
|
||||
]
|
||||
},
|
||||
"unpacked": { "type": "boolean" }
|
||||
},
|
||||
"required": [
|
||||
"logical_path",
|
||||
"algorithm",
|
||||
"declared_crc32",
|
||||
"calculated_crc32",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": {}
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -918,15 +797,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -990,15 +860,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1062,15 +923,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1134,15 +986,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1206,15 +1049,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1278,15 +1112,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1350,15 +1175,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1422,15 +1238,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1494,15 +1301,6 @@
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "retry_cleanup" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -1,348 +0,0 @@
|
||||
# ESP flash images
|
||||
|
||||
REA reads a dump of an ESP32-family SPI flash, such as the output of
|
||||
`esptool read-flash 0 ALL dump.bin` or `esptool merge-bin`, through
|
||||
`inspect_artifact` and `extract_artifact`. The reader is static: no device,
|
||||
esptool or ESP-IDF is needed, and no code is run.
|
||||
|
||||
The container format is `esp-flash-image`. Entries are the bootloader region,
|
||||
the partition table sector and one file per partition. The bootloader and
|
||||
each app partition are also decoded as [ESP images](esp-image.md), so their
|
||||
headers, segments, integrity and descriptors appear in the root
|
||||
`format_metadata`. Layout facts are cited to ESP-IDF v6.1
|
||||
(`esp_flash_partitions.h`, `flash_partitions.c`, `gen_esp32part.py`,
|
||||
`bootloader_common_loader.c`, `bootloader_utility.c` and each chip's
|
||||
`esp_rom_caps.h`).
|
||||
|
||||
## Recognition
|
||||
|
||||
A file is a flash image when REA finds a partition table in the entry
|
||||
structure ESP-IDF's bootloader verifies (`esp_partition_table_verify`):
|
||||
32-byte `0x50AA` entries, at most one `0xEBEB` MD5 entry over the entries
|
||||
before it, and a terminating entry within `0xC00` bytes. ESP-IDF accepts a
|
||||
table with no partition entries, but REA requires one, so that neither
|
||||
erased flash nor a lone MD5 entry is read as a table. A table whose MD5 does not match, or that has a second MD5 entry, is
|
||||
still read, with `md5.status` `invalid` and a limitation, because the
|
||||
bootloader would refuse it. The table is located in this order:
|
||||
|
||||
1. at the caller's `partition_table_offset`, when given; no scan follows;
|
||||
2. at the default `0x8000`;
|
||||
3. by scanning every `0x1000`-aligned flash sector for a table whose MD5
|
||||
verifies. This location is derived, and the inventory says so.
|
||||
|
||||
A table never lies inside the located bootloader image, so the default
|
||||
offset and the scan are tried only after the image's last segment. Without
|
||||
a caller selection, the scan runs only when an ESP image sits at a ROM
|
||||
bootloader offset, and a single app or bootloader image ends at its last
|
||||
segment, so neither ordinary files nor single images are scanned. A C3 or
|
||||
S3 dump begins with its bootloader, so flash layouts are detected before a
|
||||
file is read as one ESP image.
|
||||
|
||||
Unless the caller names the table with `partition_table_offset`, REA
|
||||
refuses two kinds of dump before looking for a table:
|
||||
|
||||
- an image of a known chip at another chip's ROM offset, such as an ESP32
|
||||
bootloader at the start of a dump read from `0x1000`. The dump's origin
|
||||
is not the flash address REA assumed, given or default, and a table found
|
||||
there would place every partition at the wrong file offset; pass the
|
||||
`flash_offset` where the dump starts. When a partition table sits where
|
||||
the image's own ROM offset puts `0x8000`, after the image's segments, the
|
||||
refusal names that offset.
|
||||
Without a selected format such a dump opens as its bootloader image, whose
|
||||
[ESP image](esp-image.md) facts say how to reopen it.
|
||||
- an ESP8266 image at a ROM offset. ESP8266 dumps can hold the same table
|
||||
format, and they keep the [ESP image](esp-image.md) reader's ESP8266
|
||||
refusal.
|
||||
|
||||
When no table is found the layout is not recognized. This is an unknown
|
||||
outcome, not a malformed file: flash encryption, which encrypts the
|
||||
bootloader and the table, is one cause. When a table is found without a
|
||||
located bootloader and `flash_offset` is omitted, a limitation says that
|
||||
each partition's file offset assumes the dump starts at flash `0x0`.
|
||||
|
||||
## Caller inputs
|
||||
|
||||
| Input | Meaning |
|
||||
| ------------------------- | ----------------------------------------------------------------------------- |
|
||||
| `format: esp-flash-image` | Read the file as a flash dump; with no table found, refuse it |
|
||||
| `flash_offset` | Flash address (`0x…`) of the dump's first byte; default 0 |
|
||||
| `partition_table_offset` | Flash address (`0x…`, a multiple of `0x1000`) of the table; disables the scan |
|
||||
|
||||
MCP takes these on `open_binary`; the CLI takes `--target-format
|
||||
esp-flash-image`, `--flash-offset` and `--partition-table-offset` on
|
||||
`inspect-artifact` and `extract-artifact`. The offsets require the format, and
|
||||
the format takes no `architecture` or `entry_point`. The inputs are part of
|
||||
the target's identity, so reopening the same bytes with other inputs reads
|
||||
them again instead of reusing the earlier reading. The `open_binary` summary
|
||||
reports them as `flash_offset` and `partition_table_offset`, null when
|
||||
omitted.
|
||||
|
||||
## Layout facts
|
||||
|
||||
The root `format_metadata` (`esp-flash-image`) records:
|
||||
|
||||
- `flash_offset`: its `value` and whether the `caller` gave it or it is the
|
||||
`default`;
|
||||
- `chip`: from the bootloader's `chip_id`, or null without a located
|
||||
bootloader;
|
||||
- `bootloader`: the region from the chip's ROM bootloader offset to the
|
||||
table, with its ESP image facts;
|
||||
- `partition_table`: its flash address, the `method` that found it
|
||||
(`default`, `caller` or `scan`), the partition count and the MD5 `status`
|
||||
(`valid`, `invalid` or `absent`) with the stored and computed digests;
|
||||
- `apps`: ESP image facts for each app partition the dump holds; app
|
||||
partitions declared over the same bytes share one decode;
|
||||
- `otadata`: the otadata partition's records, or null without one;
|
||||
- `boot_selection`: the app the bootloader would select, for every table;
|
||||
- `partitions_outside_dump`: declared partitions that begin before the
|
||||
dump's first byte, which have no entry;
|
||||
- `undescribed`: ranges outside every region, each from a byte that is not
|
||||
erased flash to the end of its gap.
|
||||
|
||||
The ROM loads the bootloader from `ESP_ROM_BOOTLOADER_OFFSET_FLASH`:
|
||||
`0x1000` for ESP32 and ESP32-S2; `0x0` for ESP32-S3, C2, C3, C6, C61, H2 and
|
||||
H21; and `0x2000` for ESP32-C5, H4, P4 and S31. A bootloader is located only
|
||||
where its own `chip_id` says the ROM loads it.
|
||||
|
||||
### Partitions
|
||||
|
||||
Each partition entry is `partitions/<label>.bin` when the label is a safe
|
||||
name of up to 16 letters, digits, `.`, `_` and `-` that starts with a
|
||||
letter, digit or underscore and has at least one letter or underscore, is
|
||||
not a Windows device name such as `NUL` or `COM1`, alone or before a dot,
|
||||
and no other label repeats it, ignoring case as some file systems do; it is
|
||||
`partitions/<index>.bin` otherwise. `source_path` keeps the label, the
|
||||
table's own name for the partition. Its `format_metadata`
|
||||
(`esp-flash-partition`) records the table `index`, `label` (null when not
|
||||
UTF-8), `label_hex` (the label's bytes when it is not UTF-8, else null),
|
||||
`type`, `subtype`, ESP-IDF's `type_name` and `subtype_name`,
|
||||
`flash_address`, `size`, the `encrypted` and `readonly` flags and any
|
||||
`unknown_flags`. The `encrypted` flag says how a device with flash
|
||||
encryption stores the partition; it does not establish that the dump's
|
||||
bytes are ciphertext.
|
||||
|
||||
Each partition entry also records where the bytes the dump holds are
|
||||
written, relative to the partition's start: `written_offset`, the first
|
||||
byte that is not erased flash (`0xFF`), `written_end`, just past the last
|
||||
such byte, and `written_bytes`, how many bytes are not `0xFF`. A partition
|
||||
erased throughout has null offsets and `written_bytes` `0`; a partition the
|
||||
file holds none of has all three null. The range bounds the written
|
||||
bytes and does not say whether erased flash lies between them; extract the
|
||||
partition to read them. REA reads the bytes the partitions cover once, so
|
||||
overlapping partitions share that read however many the table declares.
|
||||
|
||||
ESP-IDF 5.3 and later may declare primary `bootloader` and `partition_table`
|
||||
entries over the bootloader region and the table sector. They are
|
||||
partitions like any other, whose bytes are also `bootloader.bin` and
|
||||
`partition-table.bin`, and they are exempt from the rule that partitions
|
||||
follow the table, as in `gen_esp32part.py`.
|
||||
|
||||
Every partition from the dump's first byte on is an entry holding the bytes
|
||||
the file has. One that the file holds only in part, as when an `esptool
|
||||
merge-bin` dump ends inside its last partition, has an entry limitation that
|
||||
gives the size the table declares; one that lies past the end of the file is
|
||||
an entry with no bytes and a limitation, and has no image facts. A partition
|
||||
that begins before the dump's first byte has no entry and is listed in
|
||||
`partitions_outside_dump`. One unreadable partition does not hide the
|
||||
others: partitions outside the dump or overlapping another are each reported
|
||||
while the rest stay readable.
|
||||
|
||||
### App partitions and the bootloader
|
||||
|
||||
Each app partition and the bootloader region carry their entry's `path`
|
||||
and ESP image `facts`:
|
||||
`status` is `esp-image`, `erased` (the image header bytes are erased flash,
|
||||
as in an empty OTA slot) or `unrecognized` (ciphertext or other data). For an
|
||||
image they hold the [ESP image](esp-image.md) root facts, `segments`,
|
||||
`signature_blocks`, `coverage` and `limitations`, with offsets relative to
|
||||
the region. Extract a partition and inspect it as an ESP image for its
|
||||
segments as entries.
|
||||
|
||||
The summary view (`inspect_artifact` with `detail: "summary"`, or the
|
||||
inventory summary of `inspect_analysis_view`) reports the root facts under
|
||||
their own kind, `esp-flash-image-summary`, and gives each of these images a
|
||||
`summary` instead of its `facts`: `status`, `chip`, `entry_addr`,
|
||||
`segment_count`, `load_ranges` (the load address, declared length and
|
||||
classification of each segment that is mapped, loaded or unclassified),
|
||||
`signature_block_count`, the `descriptor` kind with the fields that tell
|
||||
builds apart (an app's `project_name`, `version`, `idf_ver`,
|
||||
`secure_version` and `app_elf_sha256`; a bootloader's `version`, `idf_ver`
|
||||
and `secure_version`), the `checksum` and `appended_sha256` statuses,
|
||||
`coverage` and `limitations`. The default complete detail keeps the full
|
||||
facts, every segment included, and the summary's `inventory_evidence_ids`
|
||||
name the Evidence that holds them.
|
||||
|
||||
`erased` is the bootloader's view: with no image header at the start of the
|
||||
slot, it finds no app there. An erased slot can still hold data further in,
|
||||
even a complete app image. When it does, its entry's `written_offset` and
|
||||
`written_end` say where, and a limitation says that the slot holds bytes the
|
||||
bootloader does not read; REA does not look for images in them.
|
||||
|
||||
### OTA data
|
||||
|
||||
The otadata partition holds two `esp_ota_select_entry_t` records, one at the
|
||||
start of each 4 KiB sector. Like the bootloader, REA reads the last otadata
|
||||
partition the table declares, and the last factory app and OTA app of each
|
||||
subtype; a limitation names any repeated declaration. REA reports each
|
||||
record the partition and the dump hold: its `ota_seq`, `seq_label`,
|
||||
`ota_state` and its name, `crc`, whether the CRC matches (CRC-32 of
|
||||
`ota_seq` seeded with `0xFFFFFFFF`) and whether the bootloader would treat
|
||||
it as valid.
|
||||
|
||||
### Boot selection
|
||||
|
||||
`boot_selection` is present for every table. It is derived as
|
||||
`bootloader_utility_get_selected_boot_partition` does without app rollback
|
||||
or anti-rollback, and `basis` names its branch:
|
||||
|
||||
- `otadata`: the valid record with the higher `ota_seq` (record 0 on a tie)
|
||||
selects slot `(ota_seq - 1) % ota_app_count`, indexed by OTA subtype;
|
||||
- `factory-fallback`: no record is usable, or the table has no OTA app, and
|
||||
a factory app is declared; or records are set but none is valid, as after
|
||||
a CRC mismatch, which selects the factory index whether or not a factory
|
||||
app is declared;
|
||||
- `ota_0-fallback`: no record is usable, or the table has no OTA app, and no
|
||||
factory app is declared, so the bootloader selects boot index 0;
|
||||
- `no-otadata`: the table declares no otadata partition, or one at offset 0,
|
||||
which the bootloader takes for none, so it selects the factory index, as in
|
||||
the common factory-only layout;
|
||||
- `none`: the bootloader rejects the table, for an MD5 mismatch or a second
|
||||
MD5 entry, or cannot read an otadata partition smaller than two 4 KiB
|
||||
sectors, and resets without selecting an app;
|
||||
- `unknown`: the dump lacks an otadata record the bootloader reads.
|
||||
|
||||
Without an OTA app the records do not matter, so a dump that lacks them
|
||||
still selects a fallback. REA does not compare the partitions with the
|
||||
flash size in the bootloader header, which the bootloader also checks.
|
||||
|
||||
`active_record` and `slot` are set for `otadata` and `slot` for
|
||||
`ota_0-fallback`. `partition_index` is null when the table declares no
|
||||
partition for the selected slot or fallback; the bootloader then tries the
|
||||
other apps. `assumptions` list what the derivation leaves out: records used
|
||||
as written (for the branches that read them), anti-rollback, load fallback,
|
||||
and factory-reset and test-app GPIO holds and a wake from deep sleep; they
|
||||
are empty for `none` and `unknown`. `running_partition` is always
|
||||
`unknown`: at boot the bootloader may first mark a pending-verify record
|
||||
aborted, check the app's secure version against eFuse, and fall back to
|
||||
another partition when an app fails to load.
|
||||
|
||||
## Coverage and extraction
|
||||
|
||||
Erased flash (`0xFF`) between regions is described. In a gap between
|
||||
regions, the first byte that is not erased starts an `undescribed` range to
|
||||
the end of the gap, and makes coverage `unknown`; the rest of that range is
|
||||
not examined. Coverage is also `unknown` when no bootloader is located, or
|
||||
when a declared partition begins before the dump's first byte, since it has
|
||||
no entry. A partition cut short or left out by the end of the file does not
|
||||
make it `unknown`: its entry holds the bytes the file has, none for one past
|
||||
the end, and its limitation says what the file lacks. Coverage describes the
|
||||
inventory of the file, not the observation of every declared flash byte.
|
||||
|
||||
So artifact comparison classifies every partition entry of two dumps. Where
|
||||
both lack the same bytes, as two `merge-bin` dumps of the same layout do,
|
||||
the entries are equal; a dump against a longer one reports each partition
|
||||
it cuts short or lacks as `changed`, and that entry's limitation says the
|
||||
file did not hold the bytes.
|
||||
|
||||
`extract_artifact` writes every entry as an ordinary file, `nvs_keys`
|
||||
partitions included. On a device without flash encryption an `nvs_keys`
|
||||
partition holds the NVS encryption keys in plain text, and a dump of it
|
||||
contains them. REA decrypts nothing, recovers no keys and does not interpret
|
||||
NVS.
|
||||
|
||||
## Comparison
|
||||
|
||||
Artifact comparison uses each partition's `label`, `label_hex`, `type`,
|
||||
`subtype`, `flash_address`, `size` and flags as its declaration, so a
|
||||
partition that moves, changes type or flags, or is renamed with unchanged
|
||||
bytes is `changed` in `metadata`. Partitions pair by path. The root facts
|
||||
are not compared as a declaration: they derive from the dump's bytes, which
|
||||
content comparison covers, and from the caller's `flash_offset` and
|
||||
`partition_table_offset`, which are part of the target's identity and
|
||||
decide each partition's byte range.
|
||||
|
||||
## Limits
|
||||
|
||||
- A flash dump inside another container, such as a ZIP, is classified from
|
||||
its first bytes and so reads as an ESP image, not as a flash layout; extract
|
||||
it first.
|
||||
- ESP8266 dumps are refused; Secure Boot v1 bootloader digests and
|
||||
flash-encrypted layouts are not read.
|
||||
- Partitions are not checked against the flash chip size, which the
|
||||
bootloader also verifies.
|
||||
- NVS, FAT, SPIFFS and other data partitions are extracted, not decoded.
|
||||
- An image nested in a partition is decoded for its facts; its segments are
|
||||
not inventory entries.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic fixtures with computed MD5s and CRCs, and one CRC otatool wrote,
|
||||
cover the table rules, every way the table is located, the scan's chunk and
|
||||
sector edges, the misplaced-origin guard with the origin omitted or given,
|
||||
the ESP8266 refusal, the `0x0`, `0x1000` and `0x2000` ROM bootloader
|
||||
offsets, single images that are not scanned, overlaps, primary entries,
|
||||
truncation and partitions outside the dump, index names for unsafe labels,
|
||||
repeated declarations, the otadata states and records, and each branch of
|
||||
the bootloader's selection. CLI and stdio MCP tests exercise the inputs and
|
||||
their identity and extraction. MCP `compare_artifacts` tests compare
|
||||
partition declarations, labels that are not UTF-8 included, two dumps that
|
||||
end inside their last partition, and such a dump against a full one.
|
||||
|
||||
The optional real lane builds its dumps with esptool and ESP-IDF's own tools:
|
||||
|
||||
```sh
|
||||
REA_RISCV_CLANG=/path/to/clang REA_RISCV_LLD=/path/to/ld.lld \
|
||||
REA_ESPTOOL_PYTHON=/path/to/python-with-esptool \
|
||||
REA_ESP_IDF_PATH=/path/to/esp-idf \
|
||||
npm run verify:esp-flash
|
||||
```
|
||||
|
||||
It converts the conformance ELFs with esptool, builds a default, a
|
||||
relocated and a primary-entry partition table with `gen_esp32part.py`,
|
||||
assembles dumps with `esptool merge-bin`, one of them written from flash
|
||||
`0x1000` with `--target-offset`, and writes OTA records with `otatool`
|
||||
through `tests/conformance/esp/flash_file_tool.py`. That script replaces
|
||||
parttool's esptool calls with reads and writes of the dump file, so
|
||||
otatool's own OTA logic runs unchanged. The lane then checks:
|
||||
|
||||
- the bootloader and app facts against `esptool image-info`;
|
||||
- every partition against `gen_esp32part.py`'s decoding of the table REA
|
||||
extracted;
|
||||
- both OTA records against `otatool read_otadata` after switching to slot 0,
|
||||
then to slot 1, then erasing the otadata, and the derived selection after
|
||||
each: slot 0, slot 1, then the factory fallback;
|
||||
- that each extracted file is the dump's bytes at its range and that the
|
||||
source is unchanged;
|
||||
- a table with the primary `bootloader` and `partition_table` entries that
|
||||
`--primary-bootloader-offset` fills, read with complete coverage and no
|
||||
overlap or misplacement limitation; it has no otadata, so its boot
|
||||
selection is `no-otadata` with the factory app;
|
||||
- the relocated table, found by scanning and named with
|
||||
`partition_table_offset`, and the dump written from `0x1000`, read with
|
||||
`flash_offset` and, without it, left an ESP image whose `flash_dump`
|
||||
names `flash_offset` `0x1000`;
|
||||
- an app with a flipped byte beside valid partitions, and a truncated dump,
|
||||
read with complete coverage and its missing partitions as entries with no
|
||||
bytes.
|
||||
|
||||
The lane ran on macOS arm64 with Node 24.22.1, Homebrew Clang and LLD 21.1.8
|
||||
and ESP-IDF v5.5.2's `gen_esp32part.py`, `otatool.py` and `parttool.py`,
|
||||
once with esptool v5.6.0 and once with 4.12.0. Its `gen_esp32part.py` is
|
||||
identical to v6.1's, and its otatool and parttool differ only in import style
|
||||
and esptool command spelling. All checks passed in both runs. With v5.6.0 and
|
||||
the build already cached, the run took 24.1 s elapsed, 16.9 s user CPU and
|
||||
6.3 s system CPU under `/usr/bin/time -l`; the largest single process peaked
|
||||
at a 278 MiB resident set, and the combined peak of all processes was not
|
||||
measured.
|
||||
|
||||
The lane does not establish:
|
||||
|
||||
- dumps read from a device: its dumps come from `merge-bin`;
|
||||
- chips other than ESP32-C3 and ESP32;
|
||||
- `ota_state` and record validity, which `read_otadata` does not print, or
|
||||
OTA states the bootloader writes itself, app rollback or anti-rollback;
|
||||
- selection with gaps or repeated OTA subtypes, which otatool does not
|
||||
write;
|
||||
- flash-encrypted dumps, which are not recognized by design.
|
||||
|
||||
It does not run in CI.
|
||||
@@ -1,333 +0,0 @@
|
||||
# ESP images
|
||||
|
||||
REA reads ESP32-family firmware images through `inspect-artifact` /
|
||||
`inspect_artifact` and `extract-artifact` / `extract_artifact`. OTA downloads,
|
||||
app partitions and second-stage bootloaders are all ESP images. Reading is
|
||||
static: no esptool, device or flash access is needed, and no code is run or
|
||||
disassembled.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./firmware.bin --json
|
||||
rea inspect-artifact ./ota.bin --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact ./firmware.bin --json
|
||||
```
|
||||
|
||||
For MCP, open the file with `open_binary`, then call `inspect_artifact`. The
|
||||
summary's `root_format_metadata` holds the image facts. Use its
|
||||
`parent_evidence_id` with `inspect_analysis_view` to select segment rows.
|
||||
`extract_artifact` writes one file per segment and per signature block.
|
||||
|
||||
For agent sessions, advertise the `session,artifact,application` tool families
|
||||
and consider setting both `REA_MCP_INPUT_SCHEMA_PROFILE` and
|
||||
`REA_MCP_OUTPUT_SCHEMA_PROFILE` to `compact`. The output profile bounds each
|
||||
advertised output schema to 4,096 bytes by relaxing nested client validation;
|
||||
REA still validates complete results against the canonical contracts. Pair
|
||||
this with retained summaries and selected inventory views to limit the context
|
||||
needed for one question. See [MCP contracts](mcp-contracts.md) for the profiles
|
||||
and their client compatibility tradeoffs.
|
||||
|
||||
The layout facts follow ESP-IDF v6.1 (`esp_app_format.h`, `esp_app_desc.h`,
|
||||
`esp_bootloader_desc.h`, `esp_image_format.c`, the ROM `secure_boot.h`
|
||||
headers and `docs/en/security/secure-boot-v2.rst`) and esptool v5.6.0
|
||||
(`bin_image.py`, `cmds.py`, `loader.py`, espsecure).
|
||||
|
||||
## Recognition
|
||||
|
||||
The container format is `esp-image`. A file is an ESP image when its bytes
|
||||
show all of these; the file name does not matter:
|
||||
|
||||
- magic `0xE9` at offset 0 and 1 to 16 segments. A count of 0 is valid but
|
||||
loads nothing, and accepting it would admit x86 programs that begin
|
||||
`E9 00`;
|
||||
- a flash mode and flash speed code that ESP-IDF's enums define;
|
||||
- a `hash_appended` byte of 0 or 1, as esptool requires;
|
||||
- when the file holds the first segment header, a segment length that is a
|
||||
multiple of 4.
|
||||
|
||||
Later segments, the checksum and the digest are integrity facts. A truncated
|
||||
download is still an ESP image, with its integrity reported as unavailable.
|
||||
|
||||
| Input | Outcome |
|
||||
| --------------------------------------------------- | ---------------------------------------------------------------- |
|
||||
| ESP8266 image (v1 or v3): same magic, 8-byte header | Refused: "ESP8266 firmware images are not supported…" |
|
||||
| Flash-encrypted image (ciphertext) | Not recognized: the bytes carry no magic. Unknown, not malformed |
|
||||
| ESP8266 v2 image (magic `0xEA`) | Not recognized |
|
||||
|
||||
An ESP8266 image has its first segment header at offset 8, and that segment
|
||||
loads into ESP8266 DRAM, IRAM (Arduino's eboot links at `0x4010F000`) or the
|
||||
IROM flash window, where v3 and Arduino app images begin. In an ESP32-family
|
||||
header the same bytes hold `wp_pin` and the drive settings, which esptool and
|
||||
ESP-IDF write as `0xEE` and zeros. Inside another container, an ESP8266 image
|
||||
is classified as a plain file.
|
||||
|
||||
Recognition cannot exclude every file that starts with an x86 near jump
|
||||
(`0xE9`), such as a DOS `.com` program. An explicit `--target-format dos-com`
|
||||
or `raw-binary` selection skips ESP recognition.
|
||||
|
||||
## Image facts
|
||||
|
||||
Root `format_metadata` has `kind: "esp-image"`:
|
||||
|
||||
| Field | Label | Meaning |
|
||||
| ------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
|
||||
| `header` | observed | Every `esp_image_header_t` field, with ESP-IDF's names, for example `entry_addr`, `chip_id`, `min_chip_rev_full` |
|
||||
| `chip` | derived | `target`, `isa` and the table's `source` (`ESP-IDF v6.1`) from `chip_id`; `null` for an ID the table lacks |
|
||||
| `checksum` | derived | `offset`, `stored`, `computed` and `status`: `valid`, `invalid` or `unavailable` |
|
||||
| `appended_sha256` | derived | The digest `hash_appended` adds: `offset`, `stored`, `computed` and `status`, including `not-appended` |
|
||||
| `image_end` | derived | End of the checksum block and any appended digest; `null` when the file ends before it |
|
||||
| `signature_sector_offset` | derived | The Secure Boot v2 signature sector, when it holds at least one valid block |
|
||||
| `undescribed_trailing` | derived | Byte ranges after the image that nothing describes; empty when every byte is described |
|
||||
| `descriptor` | observed | `esp_app_desc_t` or `esp_bootloader_desc_t` fields, with ESP-IDF's names, and the descriptor's file `offset` |
|
||||
|
||||
The descriptor's `kind`, derived from its magic, classifies the image as an
|
||||
`app` or a `bootloader`. Without a descriptor the role is unknown.
|
||||
|
||||
The ISA comes from `chip_id`, never from the series letter: ESP32-S31 is
|
||||
RISC-V. The chip table covers every `esp_chip_id_t` in ESP-IDF v6.1:
|
||||
|
||||
| `chip_id` | Target | ISA | `chip_id` | Target | ISA |
|
||||
| --------- | --------- | ------ | --------- | ---------- | ----- |
|
||||
| `0x0000` | `esp32` | xtensa | `0x0012` | `esp32p4` | riscv |
|
||||
| `0x0002` | `esp32s2` | xtensa | `0x0014` | `esp32c61` | riscv |
|
||||
| `0x0005` | `esp32c3` | riscv | `0x0017` | `esp32c5` | riscv |
|
||||
| `0x0009` | `esp32s3` | xtensa | `0x0019` | `esp32h21` | riscv |
|
||||
| `0x000C` | `esp32c2` | riscv | `0x001C` | `esp32h4` | riscv |
|
||||
| `0x000D` | `esp32c6` | riscv | `0x0020` | `esp32s31` | riscv |
|
||||
| `0x0010` | `esp32h2` | riscv | | | |
|
||||
|
||||
Other IDs, for example esptool v5.6.0's ESP32-E22 (`0x1F`), have `chip: null`
|
||||
and unclassified segments.
|
||||
|
||||
**Integrity.** The checksum is the XOR of every segment data byte, seeded with
|
||||
`0xEF`, in the last byte of the next 16-byte block. The appended SHA-256 covers
|
||||
every byte through the checksum. Both are facts, not admission rules: a
|
||||
mismatch is reported with a limitation and the segments stay readable.
|
||||
|
||||
**Descriptors.** Both descriptors begin segment 0: ESP-IDF's bootloader reads
|
||||
the app descriptor there, and esptool places the bootloader descriptor there.
|
||||
Strings stop at the first NUL and are omitted, with a limitation, when they
|
||||
are not valid UTF-8. The bootloader descriptor's magic is a single byte
|
||||
(`0x50`), so its reserved bytes, which ESP-IDF zero-fills, must also be zero.
|
||||
A `--ram-only-header` image puts RAM segments first, so its app descriptor is
|
||||
not in segment 0 and its role is unknown. `mmu_page_size` keeps ESP-IDF's
|
||||
name and value: the MMU page size as a base-2 logarithm, as
|
||||
`esp_app_desc_t` declares it, so `16` means 64 KiB. It is `0` in images built
|
||||
before ESP-IDF v5.4. `spi_flash_mode` is the next byte, which ESP-IDF v6.1
|
||||
names, kept as stored and not interpreted; releases that reserve it leave it
|
||||
`0`. A file can end inside the app descriptor: every field it holds in full
|
||||
is kept, a field it cuts off is `null`, and `observed_length` gives the
|
||||
descriptor bytes the file holds. The segment's truncation is reported as usual.
|
||||
|
||||
## Segments
|
||||
|
||||
Each header-declared segment is one entry named `segment-<index>.bin`, for
|
||||
example `segment-00.bin`, so two builds pair segments by position. Its
|
||||
`source_location` is the segment data in the image file. `format_metadata`
|
||||
(`esp-image-segment`) records `index`, `load_addr`, `data_len`,
|
||||
`header_offset` and `classification`.
|
||||
|
||||
`classification` follows ESP-IDF's bootloader (`should_map` and
|
||||
`should_load` in `esp_image_format.c`) on a cold boot:
|
||||
|
||||
| Value | Meaning |
|
||||
| ------------ | ------------------------------------------------------------------------------------------------- |
|
||||
| `mapped` | Inside the chip's IROM, DROM or, where the MMU is per memory type, PSRAM range (`soc.h`) |
|
||||
| `loaded` | `should_load` selects it: not mapped, at or above `0x10000000`. RAM ranges are not checked |
|
||||
| `not-loaded` | Below `0x10000000`: the padding segments esptool writes at address 0, including `--secure-pad-v2` |
|
||||
| `unknown` | The chip is unknown, or the mapping depends on the bootloader's MMU page size |
|
||||
|
||||
`loaded` does not mean the bootloader accepts the segment. Before loading,
|
||||
ESP-IDF's `verify_load_addresses` rejects the image when a loaded segment is
|
||||
not entirely in the chip's DRAM, IRAM, RTC memory or SPM, or overlaps the
|
||||
bootloader's own stack, data or loader code. For example, an ESP32-C6
|
||||
segment at `0x43000000` is `loaded` here and rejected there. REA models no
|
||||
RAM ranges.
|
||||
|
||||
Padding segments are inventoried and extractable, but are never load regions.
|
||||
When waking from deep sleep, ESP-IDF also skips RTC-memory segments; that
|
||||
runtime case is not modelled. ESP32-C6 and ESP32-H2 map flash from
|
||||
`0x42000000` through 256 pages of the bootloader's configured MMU page size,
|
||||
which no image records. The app descriptor's page size describes the app's
|
||||
build, not the bootloader's. With page sizes from 8 KiB (which the hardware
|
||||
supports and esptool builds for) to 64 KiB, segments from `0x42200000` to
|
||||
`0x43000000` are `unknown`. Ranges follow ESP-IDF's `should_map`, not esptool's
|
||||
wider placement ranges.
|
||||
|
||||
A segment cut short by the end of the file is the bytes the file holds, with
|
||||
an entry limitation that gives the declared length.
|
||||
|
||||
Artifact comparison uses each segment's `load_addr` and `data_len` as its
|
||||
declaration, so a segment that moves with unchanged bytes is `changed` in
|
||||
`metadata`. Segments pair by index: inserting or removing one shifts the
|
||||
paths of the segments after it. Header offsets and `classification` are not
|
||||
compared; the image's own change covers the header and chip. The image's and
|
||||
the signature blocks' `format_metadata` carry no declaration: their fields
|
||||
come from their own bytes, which content comparison covers, and a block's
|
||||
`image_digest_matches` follows the image's content.
|
||||
|
||||
## Secure Boot v2 signatures
|
||||
|
||||
A signature sector starts at the next 4 KiB boundary after the image and
|
||||
holds up to three 1216-byte slots. A slot holds a block when it has the `0xE7`
|
||||
magic and a matching CRC32 at offset 1196, the rule ESP-IDF and espsecure
|
||||
apply. Each slot is examined independently, so an erased or damaged slot does
|
||||
not hide a later block. Each block is an entry, `signature-block-<slot>.bin`,
|
||||
with `esp-image-signature-block` metadata:
|
||||
|
||||
| Field | Meaning |
|
||||
| ---------------------------------- | ------------------------------------------------------------------------- |
|
||||
| `index`, `version`, `scheme` | Slot 0 to 2, version byte, and `rsa` (`0x02`) or `ecdsa` (`0x03`) |
|
||||
| `sha_version`, `curve_id` | For ECDSA, the third byte (0 is SHA-256, 1 SHA-384) and the curve ID byte |
|
||||
| `digest_algorithm`, `image_digest` | The digest the block records |
|
||||
| `image_digest_matches` | Whether it equals the digest of every byte before the sector |
|
||||
| `public_key_sha256` | SHA-256 of the block's public key, the value eFuse key digests hold |
|
||||
| `signature_status` | Always `unverified` |
|
||||
|
||||
Signatures are observed, not verified: REA checks no public key, signature or
|
||||
eFuse. `public_key_sha256` is the digest espsecure `signature-info-v2`
|
||||
prints, so it can be compared with `espefuse summary`. An unknown version or
|
||||
`sha_version` leaves the layout-dependent fields `null`. A slot with the magic
|
||||
but no matching CRC32 is not a block: it is reported in a limitation, whether
|
||||
or not another slot holds a block, and its bytes are undescribed. Secure Boot
|
||||
v1 signatures are not decoded.
|
||||
|
||||
## Coverage and limitations
|
||||
|
||||
Bytes after the image are described when they are erased flash (`0xFF`) or a
|
||||
signature sector's blocks; erased slots and slack in that sector count as
|
||||
erased flash. Every other byte is in `undescribed_trailing`.
|
||||
|
||||
| Situation | Outcome |
|
||||
| ---------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
|
||||
| Segment data or a segment header beyond the end of the file | Truncation limitation; checksum and SHA-256 `unavailable`; coverage `unknown` |
|
||||
| File ends before the checksum or the appended digest | That status is `unavailable`, with a limitation; coverage `unknown`, since a signature sector could follow |
|
||||
| Checksum or SHA-256 mismatch | `invalid`, with a limitation; coverage is unchanged |
|
||||
| Erased flash after the image that runs past the three slots | Described; no limitation |
|
||||
| Erased flash that ends before the three slots are complete | Limitation; coverage `unknown`, since a signed image cut there would have lost its sector |
|
||||
| A signature sector the file cuts short | Limitation; coverage `unknown` |
|
||||
| Any other bytes after the image | Ranges in `undescribed_trailing`, a limitation, and coverage `unknown` |
|
||||
| Segment length that is not a multiple of 4 | Limitation: ESP-IDF rejects such an image |
|
||||
| Segment of 16 MiB or more, except on chips with 32-bit cache addresses | Limitation: ESP-IDF rejects such an image |
|
||||
|
||||
ESP32-S3, ESP32-P4, ESP32-C5, ESP32-C61, ESP32-H4 and ESP32-S31 map flash
|
||||
through 32-bit cache addresses (`SOC_SPI_MEM_SUPPORT_CACHE_32BIT_ADDR_MAP`), so
|
||||
ESP-IDF sets no 16 MiB segment limit for them. For an unknown chip the limit
|
||||
is not applied.
|
||||
|
||||
Undescribed bytes include other flash contents such as an app partition's
|
||||
stale data, a Secure Boot v1 signature, and the flash segments that esptool
|
||||
`--ram-only-header` writes after the checksum without counting them. REA
|
||||
reports their ranges and does not guess their structure. In the padding, in a
|
||||
sector without a valid block, in the sector's slack and after the sector, the
|
||||
first byte that is not erased starts a range to the end of that region. A
|
||||
slot that is neither a block nor erased is reported whole. Adjacent ranges
|
||||
are merged.
|
||||
|
||||
A whole-flash dump whose partition table REA finds is read as an
|
||||
[ESP flash image](esp-flash-image.md). Without one, a dump of a chip whose
|
||||
bootloader is at offset 0, such as ESP32-C3 or ESP32-S3, is recognized as that
|
||||
bootloader image, and the partition table, apps and data follow as
|
||||
undescribed bytes.
|
||||
|
||||
A dump read from a later flash address, such as an ESP32 dump read from
|
||||
`0x1000` with `read-flash 0x1000 …` or `merge-bin --target-offset 0x1000`,
|
||||
opens as its bootloader image. REA says so when the file shows it: the image
|
||||
at file offset 0 is not an app, its chip's ROM loads the bootloader from a
|
||||
later address (`0x1000` for ESP32 and S2, `0x2000` for C5, H4, P4 and S31),
|
||||
and a partition table sits where that origin puts `0x8000` (file offset
|
||||
`0x7000` for `0x1000`), after the image ends. Then `flash_dump` records
|
||||
`flash_offset` and
|
||||
`table_file_offset`, a limitation says to reopen the file with
|
||||
`format: "esp-flash-image"` and that `flash_offset`, and `inspect_artifact`
|
||||
lists the same `open_binary` call as a next probe. `flash_dump` is null for
|
||||
every other image. Evidence that REA recorded before it reported the field
|
||||
omits it; REA still reads and compares such Evidence, and the omission says
|
||||
nothing about where the file was read from.
|
||||
|
||||
## Limits
|
||||
|
||||
An image has at most 16 segments and 3 signature blocks. Decoding reads each
|
||||
byte once and retains only fixed-size fields. It reads through erased flash to
|
||||
find the first byte that is not erased, and stops there. There is no
|
||||
file-size limit.
|
||||
|
||||
## Analyzing a segment
|
||||
|
||||
Inventory does not disassemble code or build a memory map. `open_binary`
|
||||
admits neither RISC-V nor Xtensa as a raw-binary architecture, so a segment
|
||||
cannot yet be opened as a raw image. Each segment's `source_location` and
|
||||
`load_addr` give the file offset and address to map it at by hand, for
|
||||
example with the Rizin tool's `om` command.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic fixtures with computed checksums, digests and CRCs cover
|
||||
recognition, ESP8266 layouts, the ESP32, ESP32-C3, ESP32-C6 and ESP32-P4
|
||||
memory maps, truncation at each structure, integrity mismatches, erased and
|
||||
undescribed trailing bytes, signature slots including damaged and unknown
|
||||
layouts, and the refusals. CLI and stdio MCP tests exercise the same artifact
|
||||
provider and advertised schemas.
|
||||
|
||||
The optional real lane builds ELF files from source, converts them with
|
||||
esptool and compares REA's facts with `esptool image-info`:
|
||||
|
||||
```sh
|
||||
REA_RISCV_CLANG=/path/to/clang REA_RISCV_LLD=/path/to/ld.lld \
|
||||
REA_ESPTOOL_PYTHON=/path/to/python-with-esptool \
|
||||
npm run verify:esp-images
|
||||
```
|
||||
|
||||
It needs a Clang with the RISC-V target, `ld.lld` and a Python that can run
|
||||
`esptool` and `espsecure`. It checks each tool before building anything and
|
||||
names the variable to set when one is missing, and it warns when esptool is
|
||||
not v5.6.0, the version whose sources the layout facts cite. It builds an
|
||||
ESP32-C3 app, the same app with `--dont-append-digest`, `--secure-pad-v2`,
|
||||
`--ram-only-header` and `--pad-to-size`, a bootloader, and ESP8266 v1 and v3
|
||||
images. It then checks:
|
||||
|
||||
- entry point, chip ID, each segment's length, address and header offset,
|
||||
checksum, digest and descriptor fields against `image-info`;
|
||||
- each segment's classification in the app, its variants and the
|
||||
bootloader, whose linker scripts fix every section's region, including the
|
||||
`not-loaded` padding that `--secure-pad-v2` adds;
|
||||
- after signing with RSA-3072 and appending ECDSA P-256 and P-384 blocks,
|
||||
each block's `public_key_sha256` against espsecure `signature-info-v2`,
|
||||
and that its recorded digest matches;
|
||||
- extraction of the app and the signed image: each segment file equals the
|
||||
bytes after esptool's segment header offset, each signature block equals
|
||||
its slot in the sector espsecure appended, and the source's SHA-256 is
|
||||
unchanged and equals the reported `root_sha256`;
|
||||
- the `--pad-to-size` erased tail is described; the `--ram-only-header`
|
||||
flash segments are undescribed;
|
||||
- a flipped checksum, a flipped digest byte, a truncated segment (where
|
||||
`image-info` stops with "End of file reading segment"), and both ESP8266
|
||||
refusals.
|
||||
|
||||
`REA_XTENSA_ESP32_GCC` adds an ESP32 (Xtensa) app built with Espressif's
|
||||
`xtensa-esp32-elf-gcc`. Without it the report marks that image as skipped.
|
||||
|
||||
Each command runs as a process tree owned by the shared verifier command
|
||||
owner, so a timeout also stops the linker or assembler a compiler driver
|
||||
starts. The lane removes its workspace only after every tree has stopped and
|
||||
keeps it when that cleanup cannot be confirmed.
|
||||
|
||||
The lane ran on macOS arm64 with Node 24.22.1, Homebrew Clang and LLD 21.1.8,
|
||||
and `xtensa-esp32-elf-gcc` 14.2.0 (crosstool-NG esp-14.2.0_20251107), once
|
||||
with esptool and espsecure v5.6.0 and once with 4.12.0. All checks passed for
|
||||
all 13 images in both runs. With v5.6.0 the complete run, including the builds
|
||||
and every esptool call, took 21.7 s elapsed, 16.9 s user CPU and 6.1 s system
|
||||
CPU under `/usr/bin/time -l` with warm caches. The largest single process
|
||||
peaked at a 253 MiB resident set; the combined peak of all processes was not
|
||||
measured.
|
||||
|
||||
The lane does not establish:
|
||||
|
||||
- the other chips: only ESP32-C3 (RISC-V) and ESP32 (Xtensa) images come
|
||||
from a real producer; the other rows of the chip table are checked against
|
||||
the ESP-IDF v6.1 headers by reading them;
|
||||
- images from a real ESP-IDF project build, an OTA download or a device's
|
||||
flash, none of which it inspects;
|
||||
- that a signature is valid: it compares digests, and nothing verifies a
|
||||
signature.
|
||||
|
||||
It does not run in CI.
|
||||
+9
-27
@@ -19,24 +19,13 @@ rea inspect-evm-interface ./runtime.bin raw --json
|
||||
|
||||
Initial real verification covers flat legacy bytecode on Linux x64 and unchanged
|
||||
bundled EVMole 0.9.3. EOF-style EF00 containers return `unsupported_target`,
|
||||
with the selected path and format reason. Selected-path resolution errors
|
||||
`ENOENT`, `ENOTDIR`, `ELOOP`, and `ENAMETOOLONG` are `invalid_request` path
|
||||
issues. Permission, resource, cancellation, provider, and cleanup failures retain
|
||||
their distinct classifications.
|
||||
with the selected path and format reason. Provider/configuration failures retain
|
||||
their distinct unavailable-provider or capability classification.
|
||||
The host must already provide util-linux `prlimit`, normally `/usr/bin/prlimit`;
|
||||
`REA_EVM_PRLIMIT_COMMAND` can select another absolute executable. REA installs
|
||||
no system tools and changes no user configuration. [Upstream provenance and
|
||||
license](https://github.com/morluto/rea/blob/main/third_party/evmole/README.md) are packaged.
|
||||
|
||||
Limiter discovery validates the selected path without launching it. A relative
|
||||
path or directory is reported as `not_configured`, a missing file as
|
||||
`executable_missing`, and denied access or another filesystem-probe failure as
|
||||
`runtime_unavailable`. Recovery names the selected path and
|
||||
`REA_EVM_PRLIMIT_COMMAND`; denied access asks for execute permission and
|
||||
filesystem access. The original admission or system code stays in diagnostics.
|
||||
An available filesystem probe does not verify the selected tool's version or
|
||||
util-linux compatibility, and REA does not substitute another command.
|
||||
|
||||
## Evidence semantics
|
||||
|
||||
- `encoding` is required: raw bytes or UTF-8 hex, with optional `0x` prefix and
|
||||
@@ -54,8 +43,7 @@ util-linux compatibility, and REA does not substitute another command.
|
||||
map from the selected bytes. Integers use decimal strings; byte strings use hex,
|
||||
and scalar keys retain their types. The recorded offset/length addresses the
|
||||
original bytecode, so summary delivery does not copy an opaque CBOR payload.
|
||||
Nested containers have partial scalar coverage; map keys are still checked for
|
||||
duplicates throughout nested maps and arrays. Unsupported or malformed
|
||||
Nested containers have partial scalar coverage; unsupported or malformed
|
||||
candidates report `not-decoded` with a diagnostic. Decoding is bounded to 64
|
||||
container levels and the terminal payload's encoded size.
|
||||
- An unavailable trailer does not prove metadata absence or Solidity origin.
|
||||
@@ -89,10 +77,8 @@ without bytecode text or duplicated raw stdout:
|
||||
}
|
||||
```
|
||||
|
||||
The result is distinct view Evidence linked to its complete parent. Like the
|
||||
parent, it has `confidence: inferred`; projection never strengthens inferred
|
||||
candidates. Its `normalized_result.summary` preserves every producer-returned
|
||||
selector, decoded
|
||||
The result is distinct derived Evidence linked to its complete parent. Its
|
||||
`normalized_result.summary` preserves every producer-returned selector, decoded
|
||||
byte offset, dispatch, argument/mutability string including null, carrier/decoded
|
||||
identities, original limitations and explicit unknowns. Payload `evidence_kind`
|
||||
remains `inferred`. It preserves stderr/truncation and records stdout text length.
|
||||
@@ -119,19 +105,15 @@ CLI keeps its existing complete result.
|
||||
## Resource and lifecycle limits
|
||||
|
||||
Inputs are limited to 4 MiB, complete replies to 16 MiB and combined diagnostic
|
||||
output to 1 MiB. An owned worker has a 256 MiB JavaScript heap; its profile
|
||||
maxima are 3 GiB virtual address space and 30 CPU seconds, with tighter inherited
|
||||
limits retained. The worker command deadline is
|
||||
output to 1 MiB. An owned worker has a 256 MiB JavaScript heap, 3 GiB virtual
|
||||
address-space limit and 30 CPU seconds. The worker command deadline is
|
||||
`timeout_ms` (default 30000 ms, integer 1 through 2147483647) on the next call,
|
||||
or `--timeout-ms` on `rea inspect-evm-interface`. Omission keeps 30000 ms.
|
||||
Those byte, address-space, and CPU ceilings have no runtime override, and
|
||||
`timeout_ms` does not raise them. An oversized selected carrier returns a
|
||||
non-retryable `resource_constraint` with resource `file-size` and its fixed
|
||||
input allowance in `reported_limits.maximum_input_bytes`.
|
||||
Complete-reply allowance refusals use the same non-retryable resource family,
|
||||
with `reported_limits.maximum_reply_bytes`; malformed JSON remains unreadable
|
||||
output. This refusal leaves no complete report to retrieve through another
|
||||
delivery channel. Virtual address space differs from resident memory. The worker retains inherited tighter
|
||||
input allowance in `reported_limits.maximum_input_bytes`. Virtual
|
||||
address space differs from resident memory. The worker retains inherited tighter
|
||||
soft/hard limits, reports its configured soft limits inline and leaves hard limits
|
||||
unchanged. WASM trap reservations are disabled
|
||||
so the engine can run within that virtual limit; actual memory is not equated to
|
||||
|
||||
+6
-107
@@ -26,14 +26,8 @@ REA does not install these tools, their dependencies or system packages. It
|
||||
checks the tool's `--version` banner when executing a request. Binwalk must
|
||||
print `binwalk <version>` and Unblob must print the version alone. Inspection and
|
||||
extraction have independent prerequisites; one does not require the other.
|
||||
Capability discovery recommends `REA_BINWALK_COMMAND` for missing inspection
|
||||
and `REA_UNBLOB_COMMAND` for missing extraction. Configuring the other engine
|
||||
does not enable the unavailable operation.
|
||||
Both require util-linux `prlimit`, normally `/usr/bin/prlimit`; override its
|
||||
absolute path with `REA_FIRMWARE_PRLIMIT_COMMAND` if needed. `rea doctor` does
|
||||
not currently inspect firmware readiness. Engine selection/version checks run
|
||||
when the firmware operation is requested; successful general doctor checks do
|
||||
not establish firmware tool or external-extractor availability.
|
||||
absolute path with `REA_FIRMWARE_PRLIMIT_COMMAND` if needed.
|
||||
|
||||
REA retains at most 8 MiB of combined stdout and stderr per command. Exceeding
|
||||
that diagnostic budget stops analysis with an explicit truncated-output error;
|
||||
@@ -77,20 +71,6 @@ Unblob submodule. In Git Bash, prefix the `sparse-checkout` command with
|
||||
`MSYS_NO_PATHCONV=1`; otherwise `!/tests/integration/` is rewritten as a
|
||||
Windows path.
|
||||
|
||||
The extraction destination must be absent with an existing writable parent.
|
||||
REA checks that selection before starting an engine and still acquires it
|
||||
exclusively during publication. Invalid sources and destinations identify the
|
||||
caller argument; failures inside an acquired workspace keep their own cause.
|
||||
Published file `relative_path` preserves producer spelling for lineage; `path`
|
||||
uses the writer's actual normalized filename. An extracted filename that cannot
|
||||
be represented losslessly as UTF-8 fails publication explicitly with its raw
|
||||
name bytes in hexadecimal instead of silently omitting the entry.
|
||||
Unpublished symlinks retain the extracted tree's target bytes in lowercase
|
||||
`link_target_hex`; `link_target` is null when those bytes are not valid UTF-8.
|
||||
Older retained Evidence may omit the hexadecimal field. Unblob may have rewritten
|
||||
the target before REA observed it, so these bytes are not necessarily those stored
|
||||
in the firmware image.
|
||||
|
||||
## CLI and MCP
|
||||
|
||||
```sh
|
||||
@@ -143,19 +123,14 @@ build's source revision, so that field remains unknown.
|
||||
selected input have a known `root_file_range`, adjusted by the selected offset.
|
||||
- Decompressed children have unknown original byte ranges and runtime addresses.
|
||||
Task lineage does not establish a byte-for-byte mapping through decompression.
|
||||
- `partial` marks unknown/encrypted chunks, selected input bytes outside all
|
||||
reported chunk intervals, reported extraction problems,
|
||||
- `partial` marks unknown/encrypted chunks, reported extraction problems,
|
||||
unpublished links/special files or reached depth limits. `complete` means the
|
||||
selected input is covered by reported chunk intervals and the engine finished
|
||||
its configured extraction without those reported conditions;
|
||||
engine finished its configured extraction without those reported conditions;
|
||||
default skip rules and format coverage still apply. It does not establish
|
||||
complete firmware reconstruction.
|
||||
- `extraction_gaps` makes those conditions recoverable: a row identifies an
|
||||
unextracted region or a failed task/stage. A chunk retains its identity and
|
||||
byte range, plus its original-file range when it belongs to the selected input.
|
||||
`unclassified-range` rows identify selected input intervals with no reported
|
||||
chunk; their provider identity and handler stay null. An empty chunk inventory
|
||||
therefore cannot establish complete extraction of a nonempty selected input.
|
||||
A multi-file stage retains its reported ID and handler with unknown byte ranges;
|
||||
a standalone task failure keeps unavailable stage identity null. Rows retain exact dependency names from
|
||||
the producer's missing-dependency reports; an accepted report with no names
|
||||
@@ -164,11 +139,6 @@ build's source revision, so that field remains unknown.
|
||||
share recovery prose without losing any region identity. Instructions use
|
||||
already configured capabilities where one exists. Every extraction rerun
|
||||
requires a fresh absent `output_directory`, preserving the prior evidence.
|
||||
Unblob's declared `ERROR` severity establishes an extraction failure,
|
||||
including standalone `UnknownError` task exceptions. Warnings and resolved
|
||||
extraction problems remain diagnostics with partial coverage; they do not
|
||||
establish failed extractor stages. Standalone errors do not acquire a nearby
|
||||
chunk's handler or byte range.
|
||||
Nested unknown chunks use `inspect_firmware_regions` on the actual published
|
||||
child rather than repeating the same extraction with unchanged handlers.
|
||||
Inspection requires separately configured Binwalk (`REA_BINWALK_COMMAND`);
|
||||
@@ -240,15 +210,10 @@ tree semantics, vulnerability scanning, emulation or hardware runtime capture.
|
||||
Diagnostic output is bounded with exact captured byte counts.
|
||||
- Default staging budget: **256 MiB** and **10,000 entries**, configurable using
|
||||
`max_output_bytes` / `max_output_files` (CLI `--max-output-bytes` /
|
||||
`--max-output-files`). Exceeding either selected staging budget returns a
|
||||
nonretryable resource constraint naming the selected limit. Depth defaults to
|
||||
3, allowed range 1–10.
|
||||
`--max-output-files`). Depth defaults to 3, allowed range 1–10.
|
||||
|
||||
Staging budgets are polled and rechecked after exit; short-lived overshoot is
|
||||
possible. The per-file process limit is the larger of the selected payload budget
|
||||
and the fixed report/diagnostic file allowances, so a small payload selection
|
||||
still permits the mandatory report. A tighter inherited host limit still applies.
|
||||
The inherited file-size limit bounds each file, not total storage;
|
||||
possible. The inherited file-size limit bounds each file, not total storage;
|
||||
address-space limits are per process, not an aggregate memory quota. Worker
|
||||
count is not CPU affinity for every external extractor. These limits do not
|
||||
claim cgroup/container containment. Cancellation cleans the owned process group
|
||||
@@ -283,7 +248,7 @@ Additional lanes are explicit, so the base lane stays small:
|
||||
REA_FIRMWARE_VERIFY_EXT4=1 npm run fixtures:firmware
|
||||
REA_FIRMWARE_VERIFY_EXT4=1 npm run verify:firmware
|
||||
|
||||
# Existing Ghidra 12.x (>=12.1) and its declared JDK required.
|
||||
# Existing Ghidra 12.1.x and its declared JDK required.
|
||||
REA_FIRMWARE_VERIFY_GHIDRA=1 npm run verify:firmware
|
||||
```
|
||||
|
||||
@@ -293,69 +258,3 @@ REA_FIRMWARE_VERIFY_GHIDRA=1 npm run verify:firmware
|
||||
`scripts/rea.mjs` to verify the packaged CLI/MCP workflow. Real coverage here is
|
||||
generated gzip/USTAR/ext4 with a Linux x64 ELF child; SquashFS, UBIFS, proprietary
|
||||
firmware, other hosts and architectures remain unverified.
|
||||
|
||||
## Inspect ESP images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on an ESP32-family app or
|
||||
bootloader image, such as an OTA download or app partition, for its chip,
|
||||
classified segments, checksum and SHA-256 results, descriptor and Secure Boot
|
||||
v2 signature blocks. `extract-artifact` / `extract_artifact` writes one file
|
||||
per segment and signature block. Reading is portable and needs no external
|
||||
tool. See [ESP images](esp-image.md).
|
||||
|
||||
## Inspect ESP flash dumps
|
||||
|
||||
Use the same tools on a whole-flash dump, such as `esptool read-flash 0 ALL`
|
||||
output. The inventory reports the bootloader, the partition table and how it
|
||||
was found, each partition with its declared type, flags and digest, ESP image
|
||||
facts for the bootloader and each app partition, and the OTA records with a
|
||||
derived selection; the running partition stays unknown. Extraction writes
|
||||
each partition, `nvs_keys` included, as an ordinary file. For a dump that
|
||||
does not start at flash address 0, select `esp-flash-image` with
|
||||
`flash_offset`. See [ESP flash images](esp-flash-image.md).
|
||||
|
||||
## Inspect extracted devicetrees
|
||||
|
||||
Use [`inspect_devicetree`](devicetree.md) / `rea inspect-devicetree` on an
|
||||
extracted DTB for its node/property tree, reservations, hardware summary and
|
||||
selected phandle relationships. FDT v16/v17 inspection is portable and requires
|
||||
no external tool or disassembler.
|
||||
|
||||
## Inspect Intel HEX and S-record images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on an Intel HEX or Motorola S-record
|
||||
file for its address regions, declared start address and record facts, then
|
||||
`extract-artifact` / `extract_artifact` for one raw file per region. Decoding is
|
||||
portable and requires no external tool. See
|
||||
[Intel HEX and S-record images](hex-records.md).
|
||||
|
||||
## Inspect U-Boot legacy uImages
|
||||
|
||||
`inspect-artifact` / `extract-artifact` recognize a legacy uImage by its
|
||||
`0x27051956` magic and inventory it without running or decompressing it.
|
||||
The root facts record the 64-byte header (architecture, OS, type, and
|
||||
compression as raw values and names, load and entry addresses, image name,
|
||||
and timestamp) with the header and data CRC32 verified; the stored payload
|
||||
is extractable as `payload.bin`, and multi-file and script images additionally
|
||||
expose each sub-image as `image-<n>.bin` from the stored size table. Declared
|
||||
addresses are facts from the header, not proven entry points, and compressed
|
||||
payloads stay as stored. Intel HEX/S-record and FIT images are separate
|
||||
readers and are not covered here. See [U-Boot legacy uImage](uimage.md) for
|
||||
component padding, preserved names, resource policies and verification scope.
|
||||
|
||||
## Inspect UF2 and DfuSe images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on a UF2 file for each familyID's
|
||||
address regions and its block numbering, or on a DfuSe file for its targets,
|
||||
elements and DFU suffix, then `extract-artifact` / `extract_artifact` for one
|
||||
raw file per region or element. A DfuSe suffix CRC mismatch follows
|
||||
`integrity_policy`. Decoding is portable and requires no external tool. See
|
||||
[UF2 and DfuSe images](uf2-dfuse.md).
|
||||
|
||||
## Inspect U-Boot FIT images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on a U-Boot FIT (`.itb`) for its
|
||||
images, declared load and entry addresses, configurations and verified hashes,
|
||||
then `extract-artifact` / `extract_artifact` for one raw file per image.
|
||||
Decoding is portable and requires no external tool. See
|
||||
[U-Boot FIT images](fit-images.md).
|
||||
@@ -1,172 +0,0 @@
|
||||
# U-Boot FIT images
|
||||
|
||||
REA inventories U-Boot Flattened Image Tree (FIT, `.itb`) files through
|
||||
`inspect-artifact` / `inspect_artifact`, and materializes the stored bytes of
|
||||
each image through `extract-artifact` / `extract_artifact`. Decoding is static
|
||||
and needs no external tool.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./boot.itb --json
|
||||
rea inspect-artifact ./boot.itb --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact ./boot.itb --json
|
||||
```
|
||||
|
||||
For MCP, open the file with `open_binary`, then call `inspect_artifact`. Summary
|
||||
delivery retains the complete Evidence; use `parent_evidence_id` with
|
||||
`inspect_analysis_view` to read selected images and fields. `extract_artifact`
|
||||
returns a fresh `output_root` containing one raw file per image. FIT summaries
|
||||
use the current projection identity; there is no pre-coverage FIT summary variant.
|
||||
|
||||
## Recognition
|
||||
|
||||
A FIT is a flattened devicetree (FDT v16 or v17) with a top-level `/images`
|
||||
node. Recognition walks the structure block within the first 4 KiB of the file
|
||||
only; the file name does not matter. A devicetree whose `/images` node begins
|
||||
after 4 KiB, a devicetree without `/images`, and any other FDT version stay on
|
||||
the plain devicetree route: `open_binary` refuses them and
|
||||
[`inspect_devicetree`](devicetree.md) still parses them. A U-Boot control DTB
|
||||
that contains `/images` is treated as a FIT. The container format is `fit`.
|
||||
|
||||
Recognition is a probe. The complete devicetree parse decides, so a file whose
|
||||
prefix declares `/images` but whose structure is malformed fails as a
|
||||
malformed FIT rather than falling back to plain classification.
|
||||
|
||||
## Images and metadata
|
||||
|
||||
Each child of `/images` is one inventory entry named `images/<unit-name>.bin`,
|
||||
listed in devicetree order. Its bytes come from exactly one data source:
|
||||
|
||||
| Source | Stored bytes |
|
||||
| --------------- | --------------------------------------------------------------------------------------- |
|
||||
| `data` | The embedded property value |
|
||||
| `data-offset` | `data-size` bytes at `data-offset` past the devicetree end, aligned to 4 (`mkimage -E`) |
|
||||
| `data-position` | `data-size` bytes at `data-position` from the start of the file (`mkimage -p`) |
|
||||
|
||||
`data-position` is read as U-Boot reads it: an offset from the start of the
|
||||
FIT, not a machine address.
|
||||
|
||||
Image `format_metadata` (`fit-image`) records `index`, `name`, `description`,
|
||||
`type`, `arch`, `os`, `compression`, `load` and `entry`, the `data_source`,
|
||||
the raw `declared_offset` for external data, `image_data` (the shared-data
|
||||
link), `cipher_present`, `cipher_algo`, `hash_algorithms`, `hashes_verified`,
|
||||
`hashes_unverified`, `signature_count`, `signatures_unreadable` and
|
||||
`signature_algorithms`. Algorithm lists are arrays. Nullable declaration fields
|
||||
are absent unless named in `unreadable_fields`, which records parsing failures
|
||||
per field. Hash or signature warnings do not make unrelated declarations unknown.
|
||||
`load` and `entry` fold one or two 32-bit cells into a lowercase
|
||||
`0x…` value, as U-Boot does, independent of `#address-cells`.
|
||||
|
||||
Root `format_metadata` (`fit`) records `fdt_version`, `fdt_totalsize`,
|
||||
`external_data_base` (the aligned devicetree end), the root `description`,
|
||||
`timestamp` (seconds since 1970) and `address_cells`, plus
|
||||
`default_configuration` and `configurations`. A missing description or
|
||||
timestamp is `null`.
|
||||
|
||||
Image bytes are reported as stored. Compressed images are not decompressed and
|
||||
ciphered images are not decrypted: entries report `encrypted: false` and the
|
||||
cipher declaration in `cipher_present` and algorithm in `cipher_algo`. A missing
|
||||
or malformed algorithm stays `null`; it does not establish unciphered bytes.
|
||||
Embedded `image-data` links require byte-identical stored properties; external
|
||||
links require matching physical ranges.
|
||||
|
||||
The extracted file holds the stored bytes. Direct-analysis suggestions require
|
||||
`compression: "none"` and `cipher_present: false`.
|
||||
|
||||
## Hashes and signatures
|
||||
|
||||
Hash subnodes are children whose name starts with `hash`. Inventory verifies
|
||||
`crc32` (4 bytes, big-endian), `md5`, `sha1`, `sha256`, `sha384` and `sha512`
|
||||
over the stored bytes, the same bytes mkimage hashed after any ciphering. All
|
||||
hashes of an image are computed in one streamed pass while the inventory is
|
||||
prepared, so a mismatch fails the whole inventory, naming the image, hash
|
||||
node, algorithm, and the declared and computed values. A supported algorithm
|
||||
whose value has the wrong length also fails.
|
||||
|
||||
Other algorithms such as `crc16-ccitt`, `crc8` or `sm3_256`, a hash without a
|
||||
readable `algo`, and a hash without a `value` are not verified; each is
|
||||
counted in `hashes_unverified` and reported as an image limitation.
|
||||
|
||||
Signature subnodes (names starting with `signature`) on images and
|
||||
configurations retain the total declaration count, readable algorithms and
|
||||
`signatures_unreadable`. Missing or malformed algorithms carry source limitations
|
||||
so unknown declarations remain distinguishable from absence. Signatures are never
|
||||
verified.
|
||||
|
||||
## Configurations
|
||||
|
||||
`/configurations` is optional. Each child records its `name`, `description`,
|
||||
and the image references `kernel`, `firmware`, `fdt` (base plus overlays),
|
||||
`ramdisk` and `loadables`, always as lists, plus its signature algorithms.
|
||||
`compatible`, `fpga`, `script`, `cmdline` and `load-only` are not reported. A
|
||||
reference to a name that is not an `/images` child, and a `default` that names
|
||||
no configuration, are reported as limitations and kept as declared. Each
|
||||
configuration carries `unresolved_reference_count`; dangling references share one
|
||||
configuration-level diagnostic while every original reference remains available.
|
||||
|
||||
## Facts, limitations and failures
|
||||
|
||||
| Situation | Outcome |
|
||||
| ---------------------------------------------------------------------------------------------- | ------------------------------------ |
|
||||
| Malformed devicetree, truncated header, `totalsize` past the end of the file | Fails as malformed |
|
||||
| No data source, more than one data source, external data without `data-size` | Fails as malformed |
|
||||
| `data-offset`, `data-position` or `data-size` not one 32-bit cell, `image-data` not one string | Fails as malformed |
|
||||
| External data outside the file, overlapping the devicetree or another image | Fails as malformed |
|
||||
| Two images with an identical external range linked by `image-data` | Accepted as shared data |
|
||||
| A dangling, chained or mismatched `image-data` link | Fails as malformed |
|
||||
| Hash mismatch, or a supported hash value with the wrong length | Fails integrity |
|
||||
| Unsupported, algorithm-less or value-less hashes | Unverified, reported as a limitation |
|
||||
| Malformed `description`, `type`, `arch`, `os`, `compression`, `load`, `entry` or `timestamp` | `null`, reported as a limitation |
|
||||
| Dangling configuration reference, missing default configuration | Reported as a limitation |
|
||||
|
||||
Failures apply to the whole file and return no partial inventory.
|
||||
|
||||
## Limits
|
||||
|
||||
The devicetree, including any embedded image data, must be at most 16 MiB; a
|
||||
larger `totalsize` fails as a memory limit. Store large images externally
|
||||
(`mkimage -E`): external data of any size is streamed from the file and never
|
||||
loaded into memory. A FIT may declare at most 1,024 images and 1,024
|
||||
configurations, and an image at most 8 hash nodes; the devicetree parser's
|
||||
node, property and depth bounds also apply. The sum of declared image sizes
|
||||
is bounded by the shared archive expansion budget. These are fixed resource
|
||||
bounds with no runtime override. Configurable ceilings are tracked in
|
||||
[#6176](https://github.com/morluto/rea/issues/6176).
|
||||
|
||||
## Analyzing an image
|
||||
|
||||
Inventory does not execute anything, and `load` and `entry` are declarations.
|
||||
After extraction:
|
||||
|
||||
- inspect `flat_dt` images with [`inspect_devicetree`](devicetree.md);
|
||||
- open an uncompressed image with `open_binary` using format `raw-binary`, an
|
||||
architecture matching its declared `arch`, and `entry_point` set to its
|
||||
declared `load` address; raw-binary maps file offset 0 and seeds analysis at
|
||||
`entry_point`. Automatic suggestions require nonempty bytes, an admitted
|
||||
architecture/address/extent, and a declared entry that is absent or equal to
|
||||
load. Distinct or unreadable entry declarations remain in Evidence without
|
||||
a raw-loader suggestion, since this loader cannot represent separate mapping
|
||||
and analysis entry addresses.
|
||||
|
||||
## Comparison
|
||||
|
||||
Artifact comparison reports changes to an image's payload interpretation,
|
||||
including load and entry declarations. Equivalent external offset forms and
|
||||
inventory positions leave those declarations unchanged. Unavailable
|
||||
interpretations remain unknown, while independently known changes survive.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic fixtures with computed hashes cover embedded, `data-offset` and
|
||||
`data-position` layouts, shared data, overlap and bounds failures, every
|
||||
verified hash algorithm, configurations, resource limits, inventory,
|
||||
extraction and reader lifecycle. CLI and stdio MCP tests exercise the same
|
||||
artifact provider and advertised schemas. Real U-Boot `mkimage` 2023.01 output
|
||||
has also been verified for embedded, `-E -B 0x200`, and
|
||||
`-E -p 0x2000 -B 0x200` layouts, with byte-exact extraction checked against
|
||||
`dumpimage`. The OpenWrt 24.10.3 ABT ASR3000 recovery FIT was checked against
|
||||
`mkimage`, `fdtget` and `dumpimage`, including its kernel, initrd and devicetree.
|
||||
No image was executed.
|
||||
|
||||
For small containers, complete delivery avoids an additional view request.
|
||||
Choose summary and selected views when the omitted facts justify that extra
|
||||
request; summary delivery does not reduce the underlying inventory work.
|
||||
+10
-38
@@ -62,12 +62,6 @@ server; older servers keep the complete request after schema discovery.
|
||||
[Dart 2.14 layout](https://github.com/dart-lang/sdk/blob/2.14.0/runtime/vm/snapshot.h).
|
||||
Kind numbers vary across Dart releases; other values are preserved with
|
||||
partial coverage, without guessing their interpretation.
|
||||
Unified `_kDartSnapshotData` / `_kDartSnapshotText` exports are a different
|
||||
unsupported profile. The [upstream VM-isolate removal](https://github.com/dart-lang/sdk/commit/d36adbacafee01d9852cf8e960cf2d8654d91e7e)
|
||||
changes both the source layout and the FullAOT kind value. REA reports the
|
||||
observed export names and library identity with a typed unsupported-profile
|
||||
refusal when only unified exports are available; it does not alias one header
|
||||
into two independent sources.
|
||||
Address resolution examines every overlapping `PT_LOAD` interpretation,
|
||||
including partial overlaps. Equivalent mappings are coalesced; conflicting
|
||||
mappings leave affected symbols unresolved and qualify coverage. Dynamic
|
||||
@@ -100,8 +94,6 @@ server; older servers keep the complete request after schema discovery.
|
||||
operation, and when they **disagree** the distinct values are
|
||||
kept in `snapshot_hash_candidates` with `snapshot_hash: null` and
|
||||
`coverage: "partial"` rather than silently picking one.
|
||||
Identification can retain a single byte-scanned hash from a unified payload;
|
||||
its partial status does not establish truncation or support for that profile.
|
||||
- **`elf_header_valid`** checks the supported little-endian ELF32/ELF64
|
||||
header, table entry widths, and full table extents against the library's
|
||||
actual byte length. Truncated, malformed, or unsupported headers report
|
||||
@@ -109,30 +101,18 @@ server; older servers keep the complete request after schema discovery.
|
||||
header check does not establish valid table contents or exported symbols.
|
||||
- **`libflutter` facts** are per-ABI observations: byte size, SHA-256
|
||||
digest, GNU build IDs from ELF-declared `PT_NOTE` / `SHT_NOTE` ranges,
|
||||
and Dart/toolchain strings the build carries. `dart_version` accepts the
|
||||
labeled `Dart VM version:` form and the unlabeled Dart
|
||||
`Version::String()` form (`<version> (<channel>) (<commit time>)` with an
|
||||
optional ` on "<os>_<arch>"` suffix) that release engines embed. An
|
||||
unlabeled match is a candidate based on its shape; it is not bound to a
|
||||
Dart symbol and another string can match. `null` means no supported pattern
|
||||
was found, not proof that no version exists. Android clang/LLVM toolchain
|
||||
and any _labeled_ strings the build
|
||||
carries. Release engine builds often strip the `Dart VM version` string —
|
||||
absence is reported as `null`, not guessed. Android clang/LLVM toolchain
|
||||
lines are reported verbatim as build provenance.
|
||||
`build_id_candidates` preserves distinct IDs. `build_id` is populated only
|
||||
when all declared note ranges were examined and one ID was found;
|
||||
conflicting IDs or malformed/bounded note metadata make coverage partial.
|
||||
Nonempty GNU build-ID descriptors up to 64 bytes are retained as lowercase
|
||||
hexadecimal, including IDs shorter than four bytes.
|
||||
Matching bytes outside declared note ranges cannot supply a build ID.
|
||||
Compressed `SHT_NOTE` sections are skipped with an unsupported-compression
|
||||
limitation and partial coverage; IDs from other note ranges remain candidates.
|
||||
Encoded note storage is never interpreted as note headers or inflated.
|
||||
- Version and toolchain strings must fit their complete grammar.
|
||||
`dart_version_capped` and `toolchain_capped` independently report labels
|
||||
omitted by grammar or projection bounds; either makes identification
|
||||
coverage partial. Values split across read chunks are retained without
|
||||
publishing truncated prefixes. Unlabeled runs that miss the
|
||||
`Version::String()` grammar are ignored and do not set
|
||||
`dart_version_capped`.
|
||||
- Labeled strings must fit their complete grammar. `dart_version_capped`
|
||||
and `toolchain_capped` independently report labels omitted by grammar or
|
||||
projection bounds; either makes identification coverage partial. Values
|
||||
split across read chunks are retained without publishing truncated prefixes.
|
||||
- **`coverage: "partial"`** also appears when an ABI carries `libapp.so`
|
||||
without `libflutter.so` (or vice versa).
|
||||
- The APK itself is digested before parsing, so `target.sha256` binds the
|
||||
@@ -154,9 +134,8 @@ server; older servers keep the complete request after schema discovery.
|
||||
- **Bounded reads.** APK snapshots are bounded at 2 GiB, including while
|
||||
copying. Identification permits 768 MiB of cumulative library extraction;
|
||||
declared sizes are checked before extraction. Library reads are bounded at 256 MiB each and ZIP
|
||||
metadata at 8 MiB. Before reading, the AOT image buffer reserves the
|
||||
admitted declared size plus one byte; the reported byte count is the number
|
||||
actually read. Exceeded budgets report
|
||||
metadata at 8 MiB; the AOT image buffer is allocated at exactly the
|
||||
declared size before reading. Exceeded budgets report
|
||||
`resource_constraint`.
|
||||
- **Engine notes.** Engine libraries are spooled into the owned root. ELF
|
||||
headers, note tables and note payloads share an 8 MiB metadata budget;
|
||||
@@ -173,16 +152,9 @@ server; older servers keep the complete request after schema discovery.
|
||||
- **Selected ABI.** AOT inspection applies the library budget to the selected
|
||||
ABI after selection; an oversized unselected library does not block it.
|
||||
Undefined dynamic-symbol imports are not treated as snapshot exports.
|
||||
A retained section directory without `SHT_DYNSYM` uses the same bounded
|
||||
`PT_DYNAMIC` symbol description as a section-stripped image. Malformed
|
||||
present symbol-table sections remain failures.
|
||||
Section-stripped images require a bounded symbol count from `DT_HASH` or
|
||||
`DT_GNU_HASH`; layouts without either are refused. Symbol names and their
|
||||
terminators must fit their linked string table. Only dynamic entries used for
|
||||
symbol discovery are interpreted as numeric bounds. ABI-ignored payloads,
|
||||
including [`DT_BIND_NOW`](https://gabi.xinuos.com/elf/08-dynamic.html#dynamic-section),
|
||||
do not change discovery; their stored bytes still contribute to the selected
|
||||
artifact's digest.
|
||||
terminators must fit their linked string table.
|
||||
- REA performs **no hash-to-SDK lookup**: mapping the snapshot hash to a
|
||||
Dart SDK release requires an external dataset and is deliberately out of
|
||||
scope for this family.
|
||||
|
||||
@@ -6,8 +6,6 @@ For remote devices, pass a host with an optional port (`host[:port]`; bracket IP
|
||||
|
||||
MCP exposes `list_frida_devices`, `list_frida_processes`, `start_frida_session`, `load_frida_script`, `resume_frida_session`, `unload_frida_script`, `frida_session_status`, and `close_frida_session`. A spawned process starts paused so scripts can load first. Closing unloads session scripts and detaches. If a spawn remains paused, REA resumes it only after script cleanup or session detach confirms the instrumentation is released; if both cleanup paths fail, the target stays paused and the cleanup error is retained. REA never kills the target. MCP sessions end with the server process. The one-shot CLI command `instrument-with-frida` accepts one inline script or script file, observes messages for a bounded duration, and returns Evidence with the source hash and captured messages.
|
||||
|
||||
`instrument-with-frida --json` returns the Evidence record directly, matching MCP. Consumers of the earlier `{evidence, cleanup_error}` CLI wrapper should read the record at the JSON root and the session cleanup observation at `normalized_result.cleanup_error`. If final provider cleanup fails or cancellation arrives during it, the typed error retains completed Evidence in `details.partial_observation`. Discovery commands retain their device or process report in the same field.
|
||||
|
||||
For state and capture-coverage checks, pass `{"session_id":"SESSION_ID","detail":"summary"}` to `frida_session_status`. Each call reads the observed session state and retains the complete observation. The reply is an `inspect_analysis_view` Evidence summary with the session identity, state, scripts, `messages_retained`, and `messages_truncated`; message payloads remain in its `parent_evidence_id`. The count covers retained queued messages, not every target event. Omitted `detail` or `detail=complete` preserves the existing complete reply.
|
||||
|
||||
A summary's selected fields look like this (excerpt):
|
||||
|
||||
@@ -1,129 +0,0 @@
|
||||
# GameCube DOL analysis through Ghidra
|
||||
|
||||
This lane imports 32-bit big-endian PowerPC GameCube/Wii DOL images through
|
||||
Ghidra. It does not execute the image. DOL is a headerless format with no magic
|
||||
number, so REA never detects it: the caller must select it explicitly.
|
||||
|
||||
Only the cached MEM1 window `0x80000000..0x81800000` is admitted. Wii MEM2
|
||||
and uncached aliases are outside this lane.
|
||||
|
||||
Not covered: small-data anchors (`r2`/`r13`), REL, RPX, and IDA or Hopper
|
||||
support. See [Not supported yet](#not-supported-yet).
|
||||
|
||||
## Select the format
|
||||
|
||||
CLI:
|
||||
|
||||
```bash
|
||||
rea inspect /absolute/path/to/main.dol --target-format gamecube-dol --provider ghidra --json
|
||||
rea function /absolute/path/to/main.dol 0x80003130 --target-format gamecube-dol --provider ghidra --json
|
||||
```
|
||||
|
||||
MCP, with the existing `open_binary` tool:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/absolute/path/to/main.dol",
|
||||
"provider_id": "ghidra",
|
||||
"format": "gamecube-dol"
|
||||
}
|
||||
```
|
||||
|
||||
The architecture comes from the DOL header (`powerpc`). Do not pass
|
||||
`architecture` with `gamecube-dol`; it is rejected. `entry_point` is optional,
|
||||
and if given it must equal the header entry.
|
||||
|
||||
Omitting `format` never imports a file as a DOL. Detection does not run for DOL.
|
||||
|
||||
## What the import does
|
||||
|
||||
REA reads the 256-byte DOL header at admission. It checks every section's file
|
||||
range and load address. It records BSS/data overlaps and computes the exact
|
||||
uninitialized BSS gaps once. Malformed headers are refused before Ghidra starts.
|
||||
|
||||
Ghidra then runs in two steps:
|
||||
|
||||
1. The standard `BinaryLoader` maps the whole file at base `0x0`, with
|
||||
`-processor` and `-cspec` taken from the language selection below.
|
||||
2. The pre-script `bridge/ghidra/ReaGhidraPrepareDol.java` reads a section file
|
||||
that REA wrote from the parsed header, with versioned collection counts. It creates one initialized block per
|
||||
used text and data section at its load address, with file bytes from the
|
||||
section's file offset. The whole-file block is removed before the section
|
||||
blocks are created. Text is read-execute and data is read-write. It sets the
|
||||
entry point and labels the entry
|
||||
`__start`.
|
||||
|
||||
Sections that the header marks unused are not mapped. The 256-byte header and
|
||||
any bytes between sections are not mapped either.
|
||||
|
||||
## BSS
|
||||
|
||||
A DOL has one BSS range, and it is not always disjoint from the data sections.
|
||||
The domain parser computes the parts of BSS that no data section covers. Both
|
||||
the pre-script and load-image attestation consume those parsed ranges. The
|
||||
pre-script creates uninitialized read-write blocks only in those gaps. Where BSS overlaps a data slot, that slot
|
||||
keeps its file bytes. Overlapping slot indexes are reported by the parser.
|
||||
|
||||
## Bound language
|
||||
|
||||
The default language is `PowerPC:BE:32:default` with compiler spec `default`,
|
||||
which ships with Ghidra. The load-image attestation checks the language Ghidra
|
||||
reports, not only the requested one.
|
||||
|
||||
### Optional Gekko/Broadway override
|
||||
|
||||
`REA_GHIDRA_LANGUAGE_ID=PowerPC:BE:32:Gekko_Broadway` selects the Gekko/Broadway
|
||||
variant. The language is not in stock Ghidra. It comes from the GameCubeLoader
|
||||
extension, by Cuyler, licensed under Apache-2.0. The extension ships
|
||||
`ppc_gekko_broadway.ldefs` under its `data/languages` directory and must be
|
||||
installed in the Ghidra installation's `Ghidra/Extensions` directory. The stock
|
||||
lane never requires it.
|
||||
|
||||
Any other language id must start with `PowerPC:BE:32:`. Other languages are
|
||||
refused with a specific constraint.
|
||||
|
||||
## Paired-single limitation
|
||||
|
||||
The stock `PowerPC:BE:32:default` language does not decode the paired-single
|
||||
instructions that Gekko and Broadway add. Those words decode as bad or undefined
|
||||
instructions. Use the Gekko override to decode them.
|
||||
|
||||
## Verification
|
||||
|
||||
The source-owned fixture is built from `tests/conformance/c/ppc.c`,
|
||||
`tests/conformance/c/dol.c`, `tests/conformance/ghidra/ppc-probe.S` and the
|
||||
linker script `tests/conformance/ghidra/dol-fixture.ld`. It is converted to a DOL
|
||||
by `tests/conformance/ghidra/dol-fixture.mjs`. Its text is at `0x80003100`, and
|
||||
its single BSS range overlaps data slot 1 and leaves a gap. The fixture is
|
||||
never executed. The real GameCube or Wii game images are never used.
|
||||
|
||||
```bash
|
||||
GHIDRA_INSTALL_DIR=/absolute/path/to/ghidra npm run verify:ghidra:dol
|
||||
```
|
||||
|
||||
The lane needs the same toolchain as the PowerPC ELF lane: `REA_PPC_CLANG`,
|
||||
`REA_PPC_READELF` and LLD. An independent DOL header reader and GNU readelf
|
||||
(run on the source ELF) check the section table, file bytes, entry and BSS gap
|
||||
before Ghidra starts. The checks never use REA's parser.
|
||||
|
||||
After import, the lane checks the load-image attestation (`verified`), the
|
||||
bound language, the `__start` entry procedure, a direct call to the leaf
|
||||
function, decompilation, and bytes at the data, tail, marker and BSS-gap
|
||||
addresses. It also checks CLI identity.
|
||||
|
||||
The Gekko pass runs only when `PowerPC:BE:32:Gekko_Broadway` is in the Ghidra
|
||||
installation's language catalog, the same catalog REA uses to admit
|
||||
`REA_GHIDRA_LANGUAGE_ID`. A language that exists only in a user extension under
|
||||
`~/Library/ghidra` is not admitted, so the lane prints
|
||||
`skipped: PowerPC:BE:32:Gekko_Broadway (not in the Ghidra installation catalog that REA admits); stock lane still verified`
|
||||
and still succeeds on the stock checks.
|
||||
|
||||
The ELF lane (`npm run verify:ghidra:ppc`) is separate. See
|
||||
[PowerPC ELF analysis through Ghidra](ghidra-ppc.md).
|
||||
|
||||
## Not supported yet
|
||||
|
||||
- Small-data anchors: `r2` and `r13` are not seeded from the DOL header.
|
||||
- REL (relocatable modules) and RPX (Wii U) are not admitted.
|
||||
- IDA and Hopper refuse `gamecube-dol`. Ghidra is the only provider.
|
||||
- Disc images (ISO/GCM) are not supported. Pass an extracted DOL file.
|
||||
+8
-13
@@ -3,7 +3,7 @@
|
||||
REA can analyze DOS MZ executables and explicitly selected COM images through the bring-your-own Ghidra adapter.
|
||||
The Linux x64 and macOS arm64 verification lanes exercise actual 16-bit disassembly and
|
||||
decompilation through the CLI and stdio MCP. Provider admission accepts Ghidra
|
||||
12.x (>=12.1) and that installation's declared JDK range. The verification lane uses
|
||||
12.1.x and that installation's declared JDK range. The verification lane uses
|
||||
Ghidra 12.1.4 and JDK 21; no DOS emulator or cross-compiler is required.
|
||||
|
||||
## Open and inspect
|
||||
@@ -29,18 +29,13 @@ point using the available structural bytes and actual file length. It reports
|
||||
those DOS facts separately from Windows-header evidence. A recognized PE
|
||||
signature remains PE; recognized NE, LE, and LX headers are explicitly
|
||||
unsupported. A nonzero Windows-header pointer without a recognized signature
|
||||
usually remains an unresolved candidate, even when the DOS structure validates:
|
||||
automatic target selection fails closed and reports both observations. When a
|
||||
nonempty relocation table ends inside a complete DOS header, the remaining
|
||||
header bytes belong to DOS padding; a nonzero dword there is not a Windows
|
||||
declaration. An unread signature beyond the current probe is requested before
|
||||
that padding rule applies. This avoids manufacturing a Windows header from
|
||||
original DOS padding while preserving rejection of damaged Windows
|
||||
declarations.
|
||||
Inventory classification is a bounded-prefix format hint; it returns `unknown`
|
||||
for an unresolved candidate or when its prefix cannot establish the format.
|
||||
That result is distinct from complete DOS validation and does not decide target
|
||||
admission.
|
||||
remains an unresolved candidate, even when the DOS structure validates:
|
||||
automatic target selection fails closed and reports both observations.
|
||||
REA does not infer DOS intent from padding or infer a damaged Windows header
|
||||
from the pointer alone. Inventory classification is a bounded-prefix format
|
||||
hint; it returns `unknown` for that candidate or when its prefix cannot
|
||||
establish the format. That result is distinct from complete DOS validation and
|
||||
does not decide target admission.
|
||||
|
||||
Ghidra imports a private target snapshot with `MzLoader`, language
|
||||
`x86:LE:16:Real Mode`, and compiler specification `default`. These settings,
|
||||
|
||||
+18
-32
@@ -24,8 +24,7 @@ specific unsupported constraint rather than guessing a compatible profile.
|
||||
|
||||
A second admission covers recordless legacy firmware images
|
||||
([#1620](https://github.com/morluto/rea/issues/1620)): ELF32 `ET_EXEC` headers
|
||||
declaring `EF_MIPS_ARCH_1` (MIPS-I, a zero ISA field), `EF_MIPS_ARCH_32` or
|
||||
`EF_MIPS_ARCH_32R2` with an unspecified ABI
|
||||
declaring `EF_MIPS_ARCH_32` or `EF_MIPS_ARCH_32R2` with an unspecified ABI
|
||||
field, `EF_MIPS_ABI2` clear, and an _inspected_ absence of any ABI flags
|
||||
record (a target whose record was merely not observed stays refused). For
|
||||
these images o32 is an analysis assumption derived from the ELF32/ET_EXEC
|
||||
@@ -39,19 +38,11 @@ bridge-reported `language_id` after import. A caller-selected
|
||||
pin; combining the pinned profile with an environment override is a typed
|
||||
refusal. A header that declares an ABI field but carries no record is still
|
||||
refused. Two further boundaries apply: an inspection that could not examine
|
||||
every ABI declaration source does not establish the recordless admission
|
||||
(target opening already refuses a carved ELF whose section table extends past
|
||||
the captured bytes, as described below), and an `EF_MIPS_ARCH_1` or
|
||||
`EF_MIPS_ARCH_32` header that _declares_ an ABI is refused rather than
|
||||
relabeled, because the declared-ABI lane is verified for
|
||||
`EF_MIPS_ARCH_32R2` headers only. The recordless MIPS-I lane pins the same
|
||||
generic 32-bit language, which also decodes later MIPS32 instructions; the
|
||||
session states that conformance to the MIPS-I subset is not established from
|
||||
the header alone, and names the committed language when the caller selected a
|
||||
variant instead of the pin. A committed profile is bound to the current
|
||||
target's lane and captured ELF metadata before startup; reusing a profile
|
||||
resolved for the other recordless lane is refused rather than exporting
|
||||
attribution for a target it did not describe.
|
||||
every ABI declaration source (for example a carved ELF whose section table
|
||||
extends past the captured bytes without a program-header record) does not
|
||||
establish the recordless admission, and an `EF_MIPS_ARCH_32` header that
|
||||
_declares_ an ABI is refused rather than relabeled, because the declared-ABI
|
||||
lane is verified for `EF_MIPS_ARCH_32R2` headers only.
|
||||
|
||||
Explicit `EF_MIPS_FP64` and `EF_MIPS_NAN2008` declarations are refused. Their
|
||||
absence is insufficient: Clang can emit identical `e_flags` for FP32, FP64 and
|
||||
@@ -77,16 +68,13 @@ argument ABI and unknown nonzero MSA values remain unsupported. Without a
|
||||
separate ABI flags record, FP attribute values are admitted only when absent,
|
||||
`ANY` (0), double precision (1), or FPXX (5), the values covered by the
|
||||
recordless profile. Parsed values are retained in the MIPS analysis profile.
|
||||
A carved ELF whose section header table extends beyond the captured file is
|
||||
refused when the target is opened, before provider selection, as is every ELF
|
||||
declaration beyond EOF ([installation](installation.md#ghidra)). The refusal
|
||||
names the declared range and the file size. It applies even when a
|
||||
program-header ABI record was captured: an uncaptured section header could
|
||||
carry an unsupported or conflicting declaration, so the captured bytes are not
|
||||
analyzed as a complete smaller program. The provider also refuses any ABI
|
||||
declaration inspection that reports unexamined headers. A missing
|
||||
program-header record does not make ABI declarations in uncaptured sections
|
||||
absent or supply a default ABI.
|
||||
For carved ELFs whose section table extends
|
||||
beyond the captured file, REA examines every complete captured section header
|
||||
and records the unavailable headers as a limitation. The provider refuses an
|
||||
incomplete ABI declaration inspection even when a program-header ABI record is
|
||||
valid, because an unavailable section header could carry an unsupported or
|
||||
conflicting declaration. A missing program-header record does not make ABI
|
||||
declarations in uncaptured sections absent or supply a default ABI.
|
||||
The field layout follows
|
||||
[`Elf_MIPS_ABIFlags_v0`](https://github.com/bminor/glibc/blob/master/elf/elf.h).
|
||||
|
||||
@@ -260,13 +248,11 @@ observations and does not relabel the upstream result as verified.
|
||||
|
||||
CLI/MCP discovery and decompilation, direct calls, target/profile/Evidence
|
||||
identity, snapshot round-trip and final process cleanup remain covered.
|
||||
For both byte orders, the lane also cuts the image one byte before its section
|
||||
header table, after checking that every program header and payload stays
|
||||
captured. `open_binary` and the CLI must both refuse that copy before provider
|
||||
selection as `target_unavailable` with constraint `file_format_rejected`,
|
||||
naming the declared section-table range and the captured size, and leave it
|
||||
unchanged. Caller-selected Ghidra heap limits are passed to the lane's CLI and
|
||||
MCP child workflows.
|
||||
For both byte orders, the lane also carves away the section table while retaining
|
||||
all program-header payloads. It checks the actual Ghidra language, entry-address
|
||||
decompilation, distinct artifact digest and missing-section limitations through
|
||||
CLI Evidence and an MCP-produced snapshot. Caller-selected Ghidra heap limits
|
||||
are passed to these child workflows.
|
||||
For soft-float fixtures the lane also checks, through `inspect_native_api`, the
|
||||
exact parameter and return types and storage of float, double, mixed (pair
|
||||
alignment and stack spill) and struct-returning functions; that a caller's
|
||||
|
||||
Loaded 100 of 5344 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user