mirror of
https://github.com/morluto/rea.git
synced 2026-11-10 17:30:32 -10:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 gensokyo-dev
|
a54464548e |
No files matched your search
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"name": "rea",
|
||||
"interface": {
|
||||
"displayName": "REA"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "rea",
|
||||
"source": {
|
||||
"source": "npm",
|
||||
"package": "rea-agents",
|
||||
"version": "6.8.0"
|
||||
},
|
||||
"policy": {
|
||||
"installation": "AVAILABLE",
|
||||
"authentication": "ON_INSTALL"
|
||||
},
|
||||
"category": "Developer Tools"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -10,16 +10,11 @@ equivalent completed investigations. A useful outcome can also be a measured
|
||||
decision to leave an implementation alone.
|
||||
|
||||
Follow the checkout's [performance priorities](../../../AGENTS.md#performance),
|
||||
[tool design](../../../docs/development/tool-design.md), and
|
||||
[testing guide](../../../docs/development/testing.md). This skill supplies an optimization
|
||||
[tool design](../../../docs/tool-design.md), and
|
||||
[testing guide](../../../docs/testing.md). This skill supplies an optimization
|
||||
workflow, not additional permission to execute targets, publish, or merge.
|
||||
Use the current contracts, provider prerequisites, and user authorization.
|
||||
|
||||
Reserve performance measurements for changes that target performance or have a
|
||||
concrete regression concern. Routine cleanup uses correctness checks and does
|
||||
not require this optimization workflow or synthetic benchmarks merely because
|
||||
helper calls, imports, or representations change.
|
||||
|
||||
Read [the Kuna case study](references/kuna-case-study.md) when selecting a
|
||||
reuse strategy or interpreting a benchmark. Its pinned observations illustrate
|
||||
tradeoffs; they are not current-provider guarantees or universal speedups.
|
||||
|
||||
@@ -6,7 +6,7 @@ description: Prepare or rewrite REA release changelogs and GitHub release notes
|
||||
# REA changelog update
|
||||
|
||||
Use this for release-note preparation and contributor attribution. Read
|
||||
[docs/development/releasing.md](../../../docs/development/releasing.md) from the repository root for
|
||||
[docs/releasing.md](../../../docs/releasing.md) from the repository root for
|
||||
publication authority and the existing checkpoint checks. This skill prepares
|
||||
notes; it does not authorize merging, tagging, or publishing.
|
||||
|
||||
@@ -68,10 +68,9 @@ closing-issue reporters. Grouping preserves all those credits in one line.
|
||||
When Highlights, detailed entries or migrations reference the same work,
|
||||
put its thanks in one of those entries, not all of them. Include issue links.
|
||||
Direct-commit bullets carry the commit link and known human credit.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic, Cursor,
|
||||
Codex, and Copilot co-author trailers resolve to `@claude`, `@cursoragent`,
|
||||
`@codex`, and `@Copilot` with GitHub type `User`; the credit policy excludes
|
||||
them while retaining their metadata.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic, Cursor
|
||||
and Codex co-author trailers resolve to `@claude`, `@cursoragent` and `@codex`
|
||||
with GitHub type `User`; the credit policy excludes them while retaining their metadata.
|
||||
At the REA maintainer's request, omit `@morluto` self-thanks while preserving
|
||||
authorship in the inventory. Other human contributors remain eligible.
|
||||
Use explicit GitHub links for cross-repository references to avoid ambiguous
|
||||
|
||||
@@ -9,7 +9,7 @@ Establish which checks still catch real regressions and which costs can be
|
||||
removed without losing that protection. The outcome may be a verified fix,
|
||||
an evidence-backed recommendation, or an explicit decision to stop.
|
||||
|
||||
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/development/testing.md),
|
||||
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/testing.md),
|
||||
and [contribution rules](../../../CONTRIBUTING.md). Use their current commands
|
||||
and pinned toolchain, rather than commands remembered from an earlier audit.
|
||||
Preserve the requested scope: investigation alone does not authorize edits;
|
||||
|
||||
@@ -1,43 +1,20 @@
|
||||
---
|
||||
name: rea-tool-design
|
||||
description: Audit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
|
||||
description: Design or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
|
||||
---
|
||||
|
||||
# REA tool design
|
||||
|
||||
Use the [tool-design guide](../../../docs/development/tool-design.md) for tool boundaries,
|
||||
Use the [tool-design guide](../../../docs/tool-design.md) for tool boundaries,
|
||||
contract semantics, provider ownership, and discoverability. Inspect the current
|
||||
contract and nearest existing tool relevant to the analyst question; source,
|
||||
documentation, and live catalogs may describe different versions.
|
||||
|
||||
For provider-backed work, exercise the analyst's task through REA and compare
|
||||
relevant native provider and independent artifact observations where available.
|
||||
Check applicable partial results, recovery and session boundaries; report what
|
||||
was not exercised. For failure guidance, follow [design recovery around the
|
||||
failure](../../../docs/development/tool-design.md#design-recovery-around-the-failure).
|
||||
Use [workflow verification](../../../docs/development/testing.md#provider-backed-workflow-verification)
|
||||
to select evidence for the proposed contract instead of seeking provider API
|
||||
parity.
|
||||
|
||||
For an audit, follow [audit existing tools](../../../docs/development/tool-design.md#audit-existing-tools):
|
||||
scope representative tasks, exercise the live workflow, and distinguish
|
||||
reproduced bugs, supported limitations, design opportunities and unverified
|
||||
claims. Check relevant rendered MCP prompts, guided workflows and packaged skill
|
||||
advice against the live contracts and exercise their suggested handoffs. Report
|
||||
working alternatives and evidence limits as well as findings.
|
||||
|
||||
For wording audits and agent trials, use
|
||||
[focused tool-description trials](../../../docs/development/client-verification.md#focused-tool-description-trials).
|
||||
Start with representative workflow boundaries and repeat failures or ambiguous
|
||||
outcomes; keep actual arguments and supported final claims separate. Reuse the
|
||||
repository's evaluation tooling and report actual runtime/model versions and
|
||||
synthetic versus production coverage rather than copying a harness into a skill.
|
||||
|
||||
For a design request, return the proposed boundary, input/result shape, Evidence
|
||||
semantics, nearest alternative, and verification approach. Implement when
|
||||
requested, preserving the user's scope and existing authorization.
|
||||
|
||||
For implementation or verification, consult [testing](../../../docs/development/testing.md)
|
||||
For implementation or verification, consult [testing](../../../docs/testing.md)
|
||||
for the affected consumer and provider lanes, and
|
||||
[contributing](../../../CONTRIBUTING.md) for generated-file ownership. Read
|
||||
provider/workflow guides only as the task requires.
|
||||
@@ -1,14 +1,14 @@
|
||||
---
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "62"
|
||||
version: "45"
|
||||
---
|
||||
|
||||
# REA
|
||||
|
||||
Use REA when a claim depends on a shipped binary or package, decompilation,
|
||||
requested runtime evidence, or comparison with
|
||||
passive application runtime evidence, or comparison with
|
||||
behavior not established by available source. For ordinary analysis of a
|
||||
complete source repository, use normal repository tools and do not run REA
|
||||
readiness or provider commands.
|
||||
@@ -17,9 +17,6 @@ readiness or provider commands.
|
||||
|
||||
Installing this skill supplies instructions; it does not register the REA MCP
|
||||
server, install analysis engines, or add tools to an already-running agent.
|
||||
The exception is a Claude Code or Codex plugin install: the plugin bundles
|
||||
this skill and registers its own `rea` server, outside the state that
|
||||
`rea setup` and `rea doctor` manage.
|
||||
|
||||
If REA tools are available and their registration is not known to be stale,
|
||||
proceed directly to the target. Do not run diagnostics or setup before every
|
||||
@@ -40,23 +37,43 @@ repeat or known not to have started. If it may already have changed the
|
||||
target, or it fails again, tell the user and ask them to restart the MCP
|
||||
client. If REA could not start its connection, retry once, then tell the user
|
||||
and ask them to restart the MCP client. Do not retry a startup failure that
|
||||
returns the same unchanged result. For plugin installs, follow
|
||||
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin).
|
||||
For setup-owned registrations, run `rea doctor` when REA could not start
|
||||
returns the same unchanged result. Run `rea doctor` when REA could not start
|
||||
or tools stay missing after that restart. For `cleanup_incomplete`, a retryable
|
||||
capacity limit, or `configuration_invalid`, follow the recovery guide's ladder
|
||||
before asking for a restart. Read that guide before a second repair attempt.
|
||||
|
||||
Choose the installation owner before repairing registration. Plugin installs use
|
||||
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin);
|
||||
only setup-owned registrations use the guide's
|
||||
[registration repair plan](references/connection-and-recovery.md#repair-registration-when-needed).
|
||||
A plugin-only doctor result of `missing` does not call for `rea setup`.
|
||||
1. Diagnose without changing REA or client configuration. `npx` may download
|
||||
the package and write to npm's cache. For Codex, run
|
||||
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
|
||||
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
|
||||
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
|
||||
`commandcode`, `vscode`, `grok_build`, `omp`, or `grok_bot`. `grok_bot` has no local
|
||||
file registration: doctor reports the Grok Bot chat step, and setup does not
|
||||
write `mcp.json`. If the client is unknown, use `doctor --json` and
|
||||
inspect its registration results before choosing a setup scope.
|
||||
2. Distinguish the reason. Missing, malformed, or stale registration needs a
|
||||
scoped configuration repair. An aligned registration with no tools in the
|
||||
active session needs a restart/reconnection; doctor cannot prove that the
|
||||
current agent has connected. A missing provider affects only tasks requiring
|
||||
that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
|
||||
and Android inspection has separate bring-your-own JADX/Java prerequisites.
|
||||
3. For a configuration repair, prepare the read-only plan:
|
||||
`npx -y rea-agents@latest setup --client codex --dry-run --json`.
|
||||
Use the current client's ID, show its exact proposed paths, backups, and
|
||||
changes, and obtain approval before setup writes configuration or installs
|
||||
Hopper. Setup normally installs the matching bundled skill too; include that
|
||||
replacement in the reviewed plan. After approval, apply the same scope with
|
||||
`npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
|
||||
only if that separate installation was needed and explicitly approved.
|
||||
4. Restart/reconnect the affected agent. Verify that REA tools actually appear
|
||||
in the session, then resume the original investigation. If they remain
|
||||
absent, inspect the client's MCP launch error rather than repeating setup.
|
||||
|
||||
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
|
||||
JADX, Kuna, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
|
||||
software to repair MCP registration. For an unsupported client, use manual stdio
|
||||
registration with an existing installation’s absolute runtime and entry script, or continue through the CLI.
|
||||
software to repair
|
||||
MCP registration. For an unsupported client, use manual stdio registration
|
||||
with a version-pinned `rea-agents` package or continue through the CLI.
|
||||
|
||||
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
|
||||
|
||||
@@ -69,13 +86,11 @@ graph, limitations, and unknowns with the same care as an MCP result; the CLI
|
||||
returns the Evidence record directly. This fallback does not establish that
|
||||
MCP is configured. Native CLI tasks still require their selected engine.
|
||||
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
|
||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place from a
|
||||
trusted directory outside the target project, with absolute target paths. Managed
|
||||
setup binds the current installation directly; retain its runtime and package
|
||||
cache entry, or restore that installation before repairing its registration.
|
||||
For setup-owned registration, configuration repair starts with a read-only setup plan; show paths, backups and
|
||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
|
||||
Configuration repair starts with a read-only setup plan; show paths, backups and
|
||||
skill replacement before applying authorized changes. Preserve existing approval
|
||||
and target-execution boundaries.
|
||||
and target-execution boundaries. REA setup never installs or upgrades unrelated
|
||||
software, including Node.js, npm, Homebrew, Java, Ghidra, IDA, JADX, Kuna, Binwalk or Unblob.
|
||||
|
||||
## Route the target first
|
||||
|
||||
@@ -94,15 +109,6 @@ explicit path or endpoint and do not need it.
|
||||
than extra calls and schemas. Runtime options are metadata, not files.
|
||||
Compact delivery still inventories and hashes members. See the
|
||||
[Python packaged applications guide](https://github.com/morluto/rea/blob/main/docs/python-packaged-applications.md).
|
||||
- Bun `--compile` executable: `open_binary`, then `inspect_artifact`; the Bun
|
||||
module graph is the application layer. Extract stored modules and analyze the
|
||||
JavaScript. See [Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
- Deno `deno compile` executable: `open_binary`, then `inspect_artifact`; the
|
||||
embedded virtual filesystem is the application layer. Extract stored files and
|
||||
analyze compatible JavaScript. Environment-file values and CA certificate bytes
|
||||
are retained in local Evidence with their counts and byte lengths. The npm
|
||||
snapshot length is reported and not decoded, and the executable is not run. See
|
||||
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
|
||||
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
|
||||
`open_bundle_resources`, then the selected resource tool. This does not
|
||||
require `CFBundleExecutable` or a deep provider. Resource sessions have no
|
||||
@@ -114,42 +120,15 @@ explicit path or endpoint and do not need it.
|
||||
- Firmware image: `inspect_firmware_regions` when advertised. Use
|
||||
`extract_firmware` when extraction is requested, with a caller-selected new
|
||||
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
|
||||
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md).
|
||||
- ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact`
|
||||
without a provider; see the ESP section of `references/native-and-artifacts.md`.
|
||||
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
|
||||
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
|
||||
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
|
||||
optional metadata recovery and its supported host/layout boundary.
|
||||
- Native program behavior, live calls, instrumentation, or command-run effects:
|
||||
choose the matching [native runtime workflow](references/runtime-observation.md#native-runtime-questions).
|
||||
`open_binary` and `trace_native_values` establish static facts; they do not
|
||||
observe execution. Runtime tools have distinct launch, attach, mutation and
|
||||
cleanup effects, and do not require a preliminary static session unless their
|
||||
own contract says so.
|
||||
- User-owned browser page already open: `list_browser_targets`.
|
||||
- Passive traffic capture of a SPA: the default scope is the page's own origin,
|
||||
and data APIs usually live on another origin. Discover API origins from
|
||||
absolute URLs in loaded bundles or script text via JavaScript analysis tools
|
||||
(another passive capture without those origins still scopes to the page
|
||||
origin, and `completeness.excluded` counts lack URLs). Pass both the page
|
||||
origin and those API origins in `allowed_origins` — the list replaces the
|
||||
default scope — and read `completeness.excluded` before reporting that an
|
||||
API is absent. For long captures with body shapes, use
|
||||
`export_evidence_bundle` instead of expecting an inline result.
|
||||
- Browser action the agent must perform (click, type, navigate):
|
||||
`capture_browser_scenario` with declared actions. See
|
||||
[runtime observation](references/runtime-observation.md) for connect/launch
|
||||
and the focused follow-up browser tools.
|
||||
- Agent environment (AgentEnv bundle) of a web application: read
|
||||
[environments](references/environments.md) before recording; it records with
|
||||
`capture_browser_scenario` and writes the bundle with `export_environment_bundle`.
|
||||
- Task for a desktop application's exported environment, from a
|
||||
`capture-desktop-application` folder: read
|
||||
[desktop application tasks](references/environments.md#tasks-for-a-desktop-application-environment).
|
||||
- Environment of a Linux desktop app with a task, from a request such as
|
||||
"build me an Inkscape environment where the agent changes a poster's
|
||||
headline": read [desktop builds](references/desktop-build.md). You drive the
|
||||
app on a build computer from `start_build_computer`, then export and witness.
|
||||
- Fiddler Everywhere traffic or a native `.saz` capture: read the
|
||||
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
|
||||
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
|
||||
@@ -160,7 +139,7 @@ explicit path or endpoint and do not need it.
|
||||
old-generation heap is `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` (default 192 MiB,
|
||||
floor 8, maximum 16384), set where REA starts. Host memory and `NODE_OPTIONS` do not
|
||||
raise it. See
|
||||
[historical captures](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md) for the exact upstream
|
||||
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
|
||||
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
|
||||
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
|
||||
or establish network loss. Prefer detail summary, then kind qlog trace/stream
|
||||
@@ -172,10 +151,8 @@ explicit path or endpoint and do not need it.
|
||||
and field names/types before choosing fields for `inspect_packet_capture`.
|
||||
Query/page the relevant records; registrations do not establish observed traffic.
|
||||
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and
|
||||
TShark 4.x on an absolute `PATH` entry or via the optional absolute
|
||||
`REA_TSHARK_COMMAND` override; see
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/guides/data/packet-captures.md).
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
|
||||
Packet limits count input packets before filtering. Output defaults to 50 matching
|
||||
frames; use `match_offset` / `match_limit` for later pages, or explicit null
|
||||
limit for complete matching output. Preserve the source digest across pages.
|
||||
@@ -183,7 +160,7 @@ explicit path or endpoint and do not need it.
|
||||
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
|
||||
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
|
||||
and capinfos, with Linux prlimit or Windows Job Objects; see
|
||||
[packet summaries](https://github.com/morluto/rea/blob/main/docs/guides/data/pcap-analysis.md).
|
||||
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
|
||||
Select an exact DNS question for address pages; page exhaustion does not prove
|
||||
network completeness. This reads recordings and does not acquire traffic.
|
||||
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
|
||||
@@ -191,7 +168,7 @@ explicit path or endpoint and do not need it.
|
||||
relevant details before choosing settings; this does not scan hardware.
|
||||
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
|
||||
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
|
||||
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/guides/signals/signal-recordings.md).
|
||||
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
|
||||
Output defaults to 50 literal lines; select annotation classes and page with
|
||||
`line_offset` / `line_limit`, or explicit null for all remaining output.
|
||||
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
|
||||
@@ -200,7 +177,7 @@ explicit path or endpoint and do not need it.
|
||||
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
|
||||
Select JavaScript glue and candidate paths explicitly and preserve URL/path
|
||||
ambiguity. No implicit fetch or target execution; WAT is not original source.
|
||||
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/guides/wasm/wasm-artifacts.md).
|
||||
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/wasm-artifacts.md).
|
||||
- Explicit local Android APK (or ZIP/AAB) signing: `inspect_apk_signature`.
|
||||
Prefer its summary or selected signer/digest/lineage page; retain complete
|
||||
Evidence and use `inspect_analysis_view` kind `apk-signature` for follow-ups.
|
||||
@@ -219,17 +196,13 @@ explicit path or endpoint and do not need it.
|
||||
Evidence stays retained under its original identity. Use the returned
|
||||
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
|
||||
Candidate coverage does not establish interface completeness. See the
|
||||
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/guides/evm/evm-bytecode.md).
|
||||
- Embedded Go compiler, module, build-setting metadata, or static `(ptr, len)`
|
||||
string literals: `inspect_go_binary` with the explicit local file (pass
|
||||
`include_string_literals: true` and optional `max_string_literals` for bounded
|
||||
inferred static string-header recovery, and page retained rows with
|
||||
`inspect_analysis_view` kind `go` collection `string_literals`). This built-in
|
||||
target-free inspection requires no Go installation or disassembler. Preserve
|
||||
raw module bytes and source ranges; absent metadata does not prove a binary is
|
||||
not Go, static string recovery excludes compiler-inlined stack or immediate
|
||||
strings, and unparsed text records remain explicit. See the
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/go-binaries.md).
|
||||
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
|
||||
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
|
||||
with the explicit local file. This built-in target-free inspection requires no
|
||||
Go installation or disassembler. Preserve raw module bytes and source ranges;
|
||||
absent metadata does not prove a binary is not Go, and unparsed text records
|
||||
remain explicit. See the
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
|
||||
- Embedded CUDA/HIP code objects: `inspect_gpu_code_objects`; inventory targets
|
||||
first, then decode selected payloads and query retained GPU Evidence.
|
||||
- Bundled GGUF or safetensors model declarations: `inspect_model_artifact`
|
||||
@@ -274,90 +247,12 @@ explicit path or endpoint and do not need it.
|
||||
absence of that table, not proof the image is unsigned in every catalog sense.
|
||||
Signatures are not verified. See the
|
||||
[PE certificate guide](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
- PE import and delay-load tables: `inspect_pe_imports` with the explicit
|
||||
local PE32/PE32+ file. This built-in target-free inspection requires no
|
||||
Windows SDK. Preserve DLL names, named and ordinal functions, ILT/IAT RVAs,
|
||||
delay-load descriptors, and the pefile import hash of the standard import
|
||||
table. A missing import directory is observed absence of that table, not
|
||||
proof the image never imported APIs through another table. Bound snapshots
|
||||
and loaded IAT contents remain unobserved. See the
|
||||
[PE import guide](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
|
||||
- Windows Shell Link shortcuts: `inspect_windows_shortcut` with the explicit
|
||||
local `.lnk` file. This built-in target-free inspection requires no Windows
|
||||
SDK or COM. Preserve header flags and FILETIMEs, IDList items, LinkInfo
|
||||
volume and network names, StringData arguments and icon location, ExtraData
|
||||
signatures, and Tracker droid facts labelled derived. Unknown IDList items
|
||||
and ExtraData signatures stay raw with SHA-256. The target path is not
|
||||
resolved or executed. See the
|
||||
[Windows shortcut guide](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
|
||||
- PE TLS callbacks and load configuration: `inspect_pe_load_config` with the
|
||||
explicit local PE32/PE32+ file. This built-in target-free inspection requires
|
||||
no Windows SDK. Preserve callback VAs as strings with their RVA, file offset
|
||||
and section, and the size-covered security cookie, SafeSEH, CFG,
|
||||
EH-continuation and XFG fields. Guard-flag mitigation claims are inferred
|
||||
linker declarations, not proof of enforcement. A missing directory is observed
|
||||
absence of that declaration. See the
|
||||
[PE load configuration guide](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
|
||||
- ELF, PE, and Mach-O hardening: `inspect_binary_hardening` with the explicit
|
||||
local file. This built-in target-free inspection requires no pwntools or other
|
||||
host tool. One image per Mach-O slice. Each property is `enabled`, `disabled`,
|
||||
`partial`, `unknown`, or `not_applicable`, with the header field or import
|
||||
behind it. A missing stack-protector or fortify import is `unknown`, not
|
||||
proof the protection was omitted. Authenticode and Mach-O code signatures
|
||||
report presence, not validity. Hardened runtime stays unknown and names
|
||||
`inspect_signature`. The image is not executed. See the
|
||||
[binary hardening guide](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
|
||||
- ELF note tables: `inspect_elf_notes` with the explicit local ELF32/ELF64
|
||||
file. This built-in target-free inspection requires no disassembler. Preserve
|
||||
PT_NOTE and SHT_NOTE ranges, owners, types, descriptor SHA-256, and GNU
|
||||
build-id or ABI-tag fields. A missing table is observed absence of those
|
||||
declarations, not proof the image never carried notes. See the
|
||||
[ELF notes guide](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
- Code that runs before `main` or at exit in ELF or Mach-O:
|
||||
`inspect_native_initializers` with the explicit local file (and
|
||||
`architecture` for a universal Mach-O). This built-in inspection requires no
|
||||
disassembler. Preserve slot and target addresses, the `resolution` of each
|
||||
entry, and its limitations. An `unresolved` slot is not a zero pointer, and a
|
||||
missing table is not proof that no constructor exists. See the
|
||||
[initializer guide](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
|
||||
|
||||
- ELF unwind function ranges: `inspect_elf_unwind_ranges` with the explicit
|
||||
local ELF32/ELF64 file. This built-in target-free inspection requires no
|
||||
disassembler. Preserve FDE start, end, size, containing section, exact
|
||||
symbol name, CIE index, LSDA, and personality. Page with `offset` and
|
||||
`limit`. A missing `.eh_frame` is observed absence of that declaration,
|
||||
not proof the image has no functions. `.debug_frame` is reported present
|
||||
or absent and is not decoded. CFA instructions are counted and not
|
||||
evaluated. See the
|
||||
[ELF unwind ranges guide](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
|
||||
- Apple `.ips` crash reports: `inspect_apple_ips` with the explicit local
|
||||
file. This built-in target-free inspection requires no Xcode, atos, or
|
||||
pwntools. Preserve first-line metadata, `bug_type` as a string, process and
|
||||
exception facts, lossless integer registers, usedImages UUIDs, and
|
||||
documented classification matches. Only `bug_type` `"309"` yields a crash
|
||||
body; other types, including `"288"` stackshots, return no crash result.
|
||||
Opened Mach-O and dSYM UUID joins remain unobserved. See the
|
||||
[Apple IPS guide](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
|
||||
- Linux kernel module metadata: `inspect_kernel_module` with the explicit local
|
||||
ELF32/ELF64 relocatable file. This built-in target-free inspection requires
|
||||
no Linux host or `modinfo`. Preserve `.modinfo` key order and duplicate keys,
|
||||
`__versions` CRC hex, section file ranges, and the appended signature
|
||||
trailer's PKCS#7 SHA-256. A missing section is observed absence when section
|
||||
names are known, not proof the module never carried that metadata.
|
||||
`struct module` is not decoded. The signature is not validated. gzip, xz, and
|
||||
zstd containers are unsupported; decompress into a separate file with an
|
||||
external codec utility, then inspect the decompressed `ET_REL` image. See the
|
||||
[kernel module guide](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
|
||||
- Linux kernel version and build config (`CONFIG_*` options such as
|
||||
`CONFIG_SECURITY_SELINUX`): `inspect_kernel_build_config` with the explicit
|
||||
local `vmlinux`, raw `Image`, x86 `bzImage`, or `configs.ko`. This built-in
|
||||
target-free inspection requires no Linux host or `extract-ikconfig`. Preserve
|
||||
the banner version, config SHA-256, option count, and value kinds; use `name`
|
||||
for one option. An `IKCFG_ST` string is a config only with a valid gzip member
|
||||
and `IKCFG_ED`. An absent config may mean the kernel was built without
|
||||
`CONFIG_IKCONFIG`. Non-gzip payloads are unsupported; decompress externally,
|
||||
then inspect the result. See the
|
||||
[kernel build config guide](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
|
||||
- CPython 3.7–3.13 `.pyc` header, nested marshal code objects and bytecode
|
||||
identity: `inspect_python_pyc` with the explicit local file. This built-in
|
||||
target-free inspection requires no Python installation. Preserve magic,
|
||||
@@ -368,10 +263,8 @@ explicit path or endpoint and do not need it.
|
||||
- Erlang BEAM IFF container: `inspect_erlang_beam` with the explicit local
|
||||
`.beam` file. This built-in target-free inspection requires no Erlang
|
||||
installation. Preserve chunk names, atom order, export/import/lambda tables,
|
||||
and the Code header. For selected facts, prefer `detail: "summary"` plus a
|
||||
`beam` view (collection, names/indices, fields and paging), retaining the full
|
||||
Evidence. CLI summary requires `--evidence_output`. Missing table counts stay
|
||||
null; decoded empty tables count zero. Bytecode is not executed. See the
|
||||
and the Code header. Missing optional tables stay null. Bytecode is not
|
||||
executed. See the
|
||||
[Erlang BEAM guide](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
|
||||
|
||||
- Explicit local SQLite database snapshot: `inspect_sqlite_database`. Inspect
|
||||
@@ -381,17 +274,17 @@ explicit path or endpoint and do not need it.
|
||||
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
|
||||
report truncation and have unspecified order. Use a quiescent snapshot rather
|
||||
than assuming a live database/WAL pair was copied atomically. See the
|
||||
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
|
||||
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
- Explicit Linux ELF file for offline layout, symbols, relocations or static
|
||||
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
|
||||
caller-supplied pwntools without opening a disassembler database. Preserve its
|
||||
raw locations and inference/coverage limits; see the
|
||||
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
- Supplied Linux x86-64 ELF core: `inspect_recorded_crash`. Read every recorded
|
||||
thread, raw note and source-bound register without opening a live target.
|
||||
Historical PIDs do not select live processes. Request `include_debugger_context`
|
||||
only when core-only mapping candidates help; see the
|
||||
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md).
|
||||
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md).
|
||||
- PE resource type/name/language trees and static icon group references:
|
||||
`inspect_pe_resources` with an explicit path. This portable operation requires
|
||||
no active target or engine; opaque payloads retain file ranges and SHA-256.
|
||||
@@ -407,12 +300,7 @@ The legacy attached profile binds the already-open GUI input by SHA-256 and
|
||||
leaves its database open. The modern headless profile creates and closes an
|
||||
owned private database without saving. Installation and registration reuse
|
||||
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
|
||||
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md).
|
||||
For migration to the separate native IDA MCP plugin, follow the guide's
|
||||
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md#migrate-a-legacy-attached-registration).
|
||||
Replace the upstream registration explicitly; modern mrexodia tools are not the
|
||||
native profile. Native attachment preserves the database but does not provide
|
||||
direct callers; check session availability and the guide's verification limits.
|
||||
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
|
||||
Proceed directly to `analyze_function`, pseudocode, or function/string searches
|
||||
after opening; IDA does not supply `binary_overview`. Consult session availability
|
||||
when composing broader workflows. Modern direct callers and unsupported dossier
|
||||
@@ -443,28 +331,18 @@ native target whose analysis tool operates on the active target. Do not call an
|
||||
unadvertised tool; inspect `binary_session` with `{}` and its
|
||||
`result.tool_availability` for availability reasons and remediation. Expand each
|
||||
group's `names` into entries, inheriting the inventory-level client requirements
|
||||
unless the group overrides them. The grouped name union is the complete tool
|
||||
inventory; use `tools/list` for canonical tool schemas. The default summary keeps
|
||||
each provider candidate's availability and target-support status, code and
|
||||
reason; `omitted_fields` lists what it leaves out. Request `{"detail": "complete"}`
|
||||
only when you need an unavailable candidate's remediation, provider
|
||||
diagnostics, candidate capability sets, or tool surface, effect and annotation
|
||||
flags; in those closed flag vocabularies, listed
|
||||
flags are true and omitted known flags are false. Provider candidate facts
|
||||
describe possible providers, while top-level session capabilities describe the
|
||||
current route and `analysis_provider_binding` names the selected provider. Do
|
||||
not treat those scopes as interchangeable.
|
||||
|
||||
For editable Ghidra analysis that must survive a later agent process, select
|
||||
the [owned Program checkpoint workflow](references/native-and-artifacts.md#continue-editable-ghidra-analysis-across-processes)
|
||||
before making metadata edits. It is separate from Evidence/Unknown persistence.
|
||||
unless the group overrides them. In its closed effect and annotation vocabularies,
|
||||
listed flags are true and omitted known flags are false. The grouped name union
|
||||
is the complete tool inventory; use `tools/list` for canonical tool schemas.
|
||||
Provider candidate facts describe possible providers, while top-level session
|
||||
capabilities describe the current route and `analysis_provider_binding` names the
|
||||
selected provider. Do not treat those scopes as interchangeable.
|
||||
|
||||
For a Ghidra startup timeout on a large native target, follow
|
||||
[deadline recovery](references/native-and-artifacts.md#ghidra-startup-deadline-recovery)
|
||||
before abandoning deep analysis or repeating the same cold import. The default
|
||||
330 seconds can be changed for new startup attempts through
|
||||
`update_analysis_settings` using `ghidra.startup_timeout_ms`. Changing the
|
||||
server's launch environment instead requires a new REA process.
|
||||
330 seconds is configurable; changing the server's environment requires a new
|
||||
REA process.
|
||||
|
||||
## Work summary-first
|
||||
|
||||
@@ -472,22 +350,7 @@ Start with the default result and use its inline Evidence and graph context.
|
||||
For a JavaScript application or ELF layout whose size is unknown or large,
|
||||
call `analyze_javascript_application` or `inspect_binary_layout` with
|
||||
`"detail": "summary"`, then inspect the retained `parent_evidence_id` with
|
||||
`inspect_analysis_view`. Over MCP, `analyze_function` is summary-first without
|
||||
a flag: its default reply is the summary, and the complete dossier stays
|
||||
retained under `parent_evidence_id` for facet paging. Pass that ID as a
|
||||
`retained-evidence` reference to `compare_functions`, or inside
|
||||
`match_native_functions` `left_observations`/`right_observations` or
|
||||
`build_call_path.functions`; request `"detail": "complete"` only when the
|
||||
dossier must travel inline.
|
||||
Use the same exact-reference form in other advertised logical Evidence inputs,
|
||||
including optional sources, arrays and nested sources across domains. Each
|
||||
source can independently be inline or retained; preserve its source identity,
|
||||
coverage, partial facts and unknowns. A summary or inspection page has its own
|
||||
Evidence ID: use `normalized_result.parent_evidence_id` when the consumer needs
|
||||
the complete source. For portable CLI follow-ups, commands advertising
|
||||
`--evidence-bundle` resolve references from that selected exported bundle.
|
||||
See [Evidence composition](references/evidence-workflows.md) for scope and
|
||||
recovery; file selectors are supported only where explicitly advertised.
|
||||
`inspect_analysis_view`.
|
||||
Page module identities and select the returned `node_id` before requesting an
|
||||
item; a single file path can represent several graph roles. Do not repeat an
|
||||
identical analysis or rejected call without a changed input or resolved blocker.
|
||||
@@ -506,10 +369,7 @@ output also streams without a single MCP frame.
|
||||
When the retained parent ID is unknown, use `get_evidence_bundle` with
|
||||
`detail: "summary"` and exact filters for operation, target SHA-256,
|
||||
analysis-profile digest or canonical procedure address. Select its returned
|
||||
Evidence ID for the next view instead of requesting the complete bundle. When
|
||||
that record has no supported view (for example an annotation), read it with
|
||||
`detail: "records"` and an `evidence_id` filter; `missing_evidence_links`
|
||||
names linked sources this session does not retain.
|
||||
Evidence ID for the next view instead of requesting the complete bundle.
|
||||
`retention: "complete-record"` means the whole record is stored; native
|
||||
value-flow truncation and unknown metadata remain separate. For an exported
|
||||
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
|
||||
@@ -578,26 +438,15 @@ as complete while required questions remain open.
|
||||
[references/javascript-applications.md](references/javascript-applications.md)
|
||||
- Android APK declarations, classes, methods, and static references:
|
||||
[references/android-applications.md](references/android-applications.md)
|
||||
- Native instrumentation, debugger and process capture; browser/Electron
|
||||
observation; static/runtime comparison:
|
||||
- Browser/Electron observation (passive and scenario) and static/runtime
|
||||
reconciliation:
|
||||
[references/runtime-observation.md](references/runtime-observation.md)
|
||||
- Evidence paging, comparisons, residual unknowns, and verification:
|
||||
[references/evidence-workflows.md](references/evidence-workflows.md)
|
||||
- Agent environments (AgentEnv bundles) of a web application, recorded from
|
||||
a live instance or a HAR:
|
||||
[references/environments.md](references/environments.md)
|
||||
- Tasks for a desktop application's exported environment:
|
||||
[references/environments.md](references/environments.md#tasks-for-a-desktop-application-environment)
|
||||
- A desktop app environment from a request, built and witnessed on build
|
||||
computers: [references/desktop-build.md](references/desktop-build.md)
|
||||
|
||||
## Finish the task
|
||||
|
||||
Explain findings in plain language and tie them to returned evidence. When the
|
||||
user asks to build something, use normal coding tools and separate observed
|
||||
behavior from design choices. Close an opened native session with
|
||||
`close_binary` when its engine is no longer needed. Evidence references and Unknown
|
||||
history remain in memory on the same MCP connection. End the investigation with
|
||||
`reset_investigation`; supply `save_path` to checkpoint the complete bundle before
|
||||
clearing. Import the saved bundle to restore original references and Unknown
|
||||
history in a new process. Retention is bounded and is not automatic persistence.
|
||||
`close_binary` when the investigation is complete.
|
||||
@@ -13,34 +13,11 @@ own APK path and explicit selectors; do not open a native session first.
|
||||
Choose an `overload_index` from the class inventory when methods are ambiguous
|
||||
instead of guessing.
|
||||
|
||||
For JNI implementations, discover `inspect_android_jni_bindings`. Select raw
|
||||
DEX class/method identities and optional descriptors, ABI or logical library
|
||||
paths. Prefer `detail: "summary"`; complete Evidence remains retained and
|
||||
`android-jni` views page declarations, libraries, candidates or binding rows
|
||||
with exact fields. Global native-pair ambiguity survives class selection.
|
||||
Preserve slot/relocation provenance and source coverage: matched means observed
|
||||
candidate presence; partial scans do not establish uniqueness or absence.
|
||||
This reads DEX/ELF bytes without loading the APK into the decompiler or executing
|
||||
target code. Runtime registration and loading remain unknown. See the
|
||||
[JNI guide](https://github.com/morluto/rea/blob/main/docs/guides/android/android-analysis.md#link-native-declarations-to-jni-candidates).
|
||||
|
||||
For an R8/ProGuard release build with its `mapping.txt`, pass the absolute
|
||||
`mapping_path` to `inspect_android_class` or `inspect_android_method`. Prefer
|
||||
`detail: "summary"` to receive normalized symbols or the selected body without
|
||||
duplicate raw class metadata. Complete Evidence stays retained under the linked
|
||||
parent ID and can be exported; CLI summary delivery requires `--evidence_output`.
|
||||
Omitted or complete detail preserves the full inline response. Keep
|
||||
selecting by observed names; the separate `mapping` block reports each symbol
|
||||
as `mapped`, `ambiguous` or `unmapped` after checking DEX types and prototypes.
|
||||
REA does not verify that the mapping belongs to the APK build.
|
||||
|
||||
The provider requires a separately supplied headless JADX MCP JAR 0.7.1 and
|
||||
Java 17 or newer. `REA_JADX_MCP_JAR` selects the absolute JAR path; `JAVA_HOME`
|
||||
can select Java. Use `rea doctor --provider jadx --json` to check the JADX
|
||||
configuration and obtain remediation for missing prerequisites. Java and the
|
||||
headless JADX MCP JAR remain caller-supplied; REA setup does not install or
|
||||
configure them. Configuration readiness does not establish successful analysis
|
||||
of a particular APK.
|
||||
can select Java. REA setup does not install or configure that JAR, and doctor
|
||||
provider scopes cover Hopper/Ghidra rather than JADX. Use existing tools and
|
||||
report missing prerequisites for the Android task specifically.
|
||||
|
||||
Linux and macOS use the POSIX owned-process boundary. Windows x64 requires the
|
||||
matching bundled native controls and uses an owned Job Object with bidirectional
|
||||
@@ -49,11 +26,8 @@ stdio. A full JDK, including `jdk.compiler`, is required on every host; Windows
|
||||
verification coverage. Calls use one worker and provider deadlines; optional
|
||||
`REA_JADX_HEAP_MIB` selects the JVM heap and `REA_JADX_OPERATION_TIMEOUT_MS`
|
||||
the per-operation deadline (integer milliseconds from 1 to 2147483647, default
|
||||
120000). To change that deadline on a running MCP server, set
|
||||
`jadx.operation_timeout_ms` through `update_analysis_settings`, then retry after
|
||||
work and cleanup settle. Newly accepted operations use the override; accepted
|
||||
work keeps its budget. Changing the launch environment requires a new process.
|
||||
The MCP client's tool timeout must also allow the selected deadline. The APK is never launched,
|
||||
120000; a change applies on the next process start; the MCP client's tool
|
||||
timeout must also allow it). The APK is never launched,
|
||||
and no emulator is required.
|
||||
|
||||
Manifest entries and decompiled text are static observations, not execution or
|
||||
|
||||
@@ -7,22 +7,18 @@ operation and the next useful step; after repair, continue the investigation.
|
||||
Commands below use an installed `rea`. If none is available, use the package
|
||||
invocation in `SKILL.md` when acquiring REA is authorized. Follow the connected
|
||||
server's schemas and the installed CLI's help rather than assuming repository
|
||||
main and an older release expose the same arguments. Tool-name prefixes belong
|
||||
to the client: find the canonical REA operation in its actual tool listing and
|
||||
server association before concluding that it is absent. A shared prefix alone
|
||||
does not establish which server an approval wildcard covers. See
|
||||
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/getting-started/client-configuration.md#client-tool-names-and-discovery).
|
||||
main and an older release expose the same arguments.
|
||||
|
||||
## Choose the remedy from the observed failure
|
||||
|
||||
| Observation | Meaning and next step |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| No REA tools in the active session | If REA was installed as a client plugin, follow the plugin section below. Otherwise inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| MCP connection dropped or REA could not start | If the connection drops, retry that call once only when it is safe to repeat or known not to have started. If it may already have changed the target, or the same failure returns, tell the user REA's connection failed and ask them to restart the MCP client. If REA could not start its connection, retry once, then tell the user and ask them to restart the MCP client. Do not retry a startup failure that returns the same unchanged result. Run `rea doctor` when REA could not start or tools stay missing after restart. Doctor cannot restore a live connection by itself. |
|
||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. Request `{"detail": "complete"}` only for provider diagnostics. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. |
|
||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
||||
|
||||
If the client cannot launch its shell/sandbox helper or read the installed skill,
|
||||
@@ -116,32 +112,6 @@ is not a recommendation to change it. Do not invent a setting. Host memory and
|
||||
- `invalid_request`: correct the reported fields from the current schema. Do
|
||||
not repeat the unchanged call.
|
||||
|
||||
## REA installed as a client plugin
|
||||
|
||||
A Claude Code or Codex plugin owns its `rea` server and this skill. Doctor's
|
||||
`--client` and `--skill` checks and `rea setup` read and write only
|
||||
setup-owned state, so for a plugin-only install they report `missing`, and
|
||||
applying setup would add a second `rea` server. If this skill was loaded from
|
||||
a plugin, or the client lists a `rea` plugin, do not prepare a setup plan for
|
||||
missing tools. Instead:
|
||||
|
||||
- Check that the plugin is installed and enabled with the client's own
|
||||
listing: `claude plugin list` and `/mcp` in Claude Code, `codex plugin list`
|
||||
in Codex.
|
||||
- Restart or reconnect the client, then read its launch error for the `rea`
|
||||
server if tools are still absent. The plugin starts its bundled runtime; a missing build or version mismatch
|
||||
requires a reinstall, or a fresh build when developing from source.
|
||||
- Check analysis engines through the connected server's
|
||||
`binary_session.result.tool_availability`. `rea doctor --provider <id> --json`
|
||||
from a shell reports that shell's environment, which can differ from the
|
||||
environment the client gave the plugin server. If you still run it through
|
||||
the package runner, use the version pinned in the plugin manifest's server
|
||||
entry (`rea-agents@<version>`), not `@latest`, so it matches the server the
|
||||
plugin starts.
|
||||
|
||||
Use the setup repair below only when the user wants a setup-owned registration
|
||||
instead of the plugin, and remove or disable the plugin first.
|
||||
|
||||
## Repair registration when needed
|
||||
|
||||
For Codex:
|
||||
@@ -154,12 +124,9 @@ rea setup --client codex --dry-run --json
|
||||
Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
|
||||
`cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`,
|
||||
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `trae`, `grok_build`, `omp`, `pi`,
|
||||
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, `freebuff`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
||||
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
||||
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
|
||||
For unsupported clients, use manual stdio registration with the selected
|
||||
installation’s absolute runtime and entry script, or continue through the CLI.
|
||||
Acquire packages from a trusted directory outside the target project. Keep a
|
||||
package-runner installation and its runtime available while registrations use it.
|
||||
For unsupported clients, use version-pinned manual stdio registration or CLI.
|
||||
|
||||
Show the plan's exact paths, backups, changes and bundled-skill replacement.
|
||||
Obtain approval for writes when that concrete scope is not already authorized,
|
||||
@@ -204,17 +171,6 @@ writing. Managed policy can constrain client settings. Consult the
|
||||
for the installed client's supported settings. This is an optional recovery
|
||||
choice, not a prerequisite for every REA task.
|
||||
|
||||
## Repair an editable REA limit
|
||||
|
||||
If `details.limit` names an editable setting and its bounds, call
|
||||
`update_analysis_settings` directly with a `changes` entry containing `setting`,
|
||||
`action: "set"` and `value`. `inspect_analysis_settings` is optional when those
|
||||
facts are already present. Retry the same input after work and cleanup settle.
|
||||
Runtime defaults affect new work only; explicit per-call arguments take
|
||||
precedence and must be corrected directly when selected. Reset restores launch
|
||||
values. Updates survive target close/open but disappear on REA restart. They do
|
||||
not edit host deadlines, response buffers, registrations or fixed limits.
|
||||
|
||||
## Continue through an installed CLI
|
||||
|
||||
Check the command environment only if the launcher fails:
|
||||
|
||||
@@ -1,860 +0,0 @@
|
||||
# Desktop app environments from a request
|
||||
|
||||
Use this when someone asks for an environment of a Linux desktop app with a
|
||||
task, such as "Build me an Inkscape environment where the agent changes a
|
||||
poster's headline and saves it". You are the building agent: you drive the
|
||||
app on a build computer through REA's MCP tools, write the task, export the
|
||||
bundle and witness it in a fresh computer. REA runs no model. The tools are
|
||||
MCP-only and need a Linux host with Docker, agentenv and REA's packed
|
||||
runtime. When a start reports one missing, tell the person to run
|
||||
`rea doctor --provider desktop_build` and apply its fixes, or
|
||||
`rea setup --client <client> --desktop-build` and restart the client, and
|
||||
stop.
|
||||
|
||||
For the whole build:
|
||||
|
||||
- Drive the app only with `act_on_build_computer`. Never script it, run
|
||||
commands in the computer or write its files another way. The one command
|
||||
you write for the computer is a task's reader, which REA runs to check a
|
||||
file; it never makes the seed or the edit.
|
||||
- Ask the upfront questions once, then work on your own. Pause only when
|
||||
blocked.
|
||||
- The seed's content is synthetic. Never put real personal data in it.
|
||||
- Close every computer you start, on success and on failure.
|
||||
- Note the time you start; the report gives the build time.
|
||||
|
||||
## Ask once, upfront
|
||||
|
||||
In one message, ask only what the request leaves open, and keep the list for
|
||||
the report. Skip each question the request answers; when it answers them all,
|
||||
start without asking:
|
||||
|
||||
1. The app and the task, if either is unclear: what changes, in which
|
||||
document.
|
||||
2. Whether they have a starting file of their own; otherwise you make one.
|
||||
3. Whether they want the tasks measured after the witness: with which
|
||||
model, within which budget, and whether to harden a task until it lands
|
||||
in the band.
|
||||
4. If so, whether they want to review the tasks before those solves.
|
||||
|
||||
When you cannot ask, as in a headless `claude -p` run, do not stop: take
|
||||
these defaults for whatever the request leaves open, say which you took in
|
||||
your first message and in the report, and go on. Install from the Ubuntu
|
||||
archive, make a synthetic seed, skip the review, and measure each task
|
||||
within $4 in all, with the model the request names, without hardening.
|
||||
|
||||
## When you are blocked
|
||||
|
||||
Stop, say what you need and wait only for:
|
||||
|
||||
- a licence the app or its installer asks someone to accept: never accept it
|
||||
on your own; name it and where to read it, and go on only once the person
|
||||
says they accept it. Then record their words with `record_build_consent`,
|
||||
kind `licence`, and accept it on their behalf;
|
||||
- credentials: ask the person to write each secret, and nothing else, in a
|
||||
file named after a handle you choose, such as `keepassxc-master`, in
|
||||
`~/.config/rea/build-secrets/` on this host (under `$XDG_CONFIG_HOME` when
|
||||
it is set), readable only by them (`chmod 600`), and wait until they say it
|
||||
is there; then type it with `type_secret` (see Drive the app). Never ask
|
||||
for a password or key in chat, and never read, list or print that folder;
|
||||
- a signed-in state in the seed: by default the export withholds it (see
|
||||
Sign-in and licence state). Seed it only when the person says the bundle
|
||||
may hold it: record their words with `record_build_consent`, kind
|
||||
`seed_sign_in`, and export with `seed_sign_in: true`;
|
||||
- an install this flow cannot do, below.
|
||||
|
||||
Decide everything else yourself and note it in the report.
|
||||
|
||||
## Sign-in and licence state
|
||||
|
||||
What an app writes when someone signs in or activates a licence is withheld
|
||||
from the seed: the bundle's receiver supplies their own. Mark
|
||||
`pre-sign-in` with `mark_build_checkpoint` just before the sign-in or
|
||||
activation and `post-sign-in` just after, both before `seed`: every file
|
||||
under the owned paths that changed between them is withheld, and the draft
|
||||
leaves it out. A file the app writes at first launch, such as a trial's
|
||||
licence file, changes before you can mark anything: name it in
|
||||
`export_build_environment`'s `withhold`. The export lists each withheld file
|
||||
in `withheld_files` with the environment variable a deploy reads it from and
|
||||
the handle of the secret REA keeps its bytes in; a check may not name one.
|
||||
The export refuses a seed file holding a withheld file's exact bytes, such as
|
||||
a `.bak` copy, but a backup that differs from it is not caught: trash backups
|
||||
before `seed`.
|
||||
Explicitly withhold any other files the person identifies as sensitive;
|
||||
REA does not infer secrecy from a value's shape or a file name.
|
||||
|
||||
## Choose the install
|
||||
|
||||
The computer is Ubuntu 24.04 (noble) on this host's architecture only. Use
|
||||
the first source that has the app:
|
||||
|
||||
| Source | `install` |
|
||||
| ---------------------------- | ------------------------------------------------------------------------------------------- |
|
||||
| Ubuntu archive | `{"kind": "archive", "package": "inkscape"}` |
|
||||
| PPA or vendor apt repository | `{"kind": "apt_repository", "package", "uri", "suite", "components", "key_file"}` |
|
||||
| `.deb` by URL | `{"kind": "package_file", "url", "sha256", "architecture", "dependencies"}` |
|
||||
| AppImage by URL | `{"kind": "appimage", "url", "sha256", "architecture", "extra_packages"}` |
|
||||
| Tarball by URL | `{"kind": "tarball", "url", "sha256", "architecture", "program", "name", "extra_packages"}` |
|
||||
|
||||
- A PPA's `uri` is `https://ppa.launchpadcontent.net/OWNER/NAME/ubuntu`,
|
||||
`suite` `noble`, `components` `["main"]`. `key_file` is the repository's
|
||||
signing key saved on this host; for a PPA, fetch it from
|
||||
keyserver.ubuntu.com by the fingerprint its Launchpad page shows.
|
||||
- When the app exits at launch for want of a runtime piece its package does
|
||||
not install, start a new build computer with that package in
|
||||
`extra_packages`, up to 16 archive names; the export installs them too.
|
||||
MyPaint needs
|
||||
`{"kind": "archive", "package": "mypaint", "extra_packages": ["librsvg2-common"]}`
|
||||
for its SVG icons, and Freeplane `["openjdk-17-jre"]`, since its launcher
|
||||
refuses noble's Java 21. Name one up front, too, when a package the app
|
||||
only recommends holds its content, such as QElectroTech's element library
|
||||
`qelectrotech-data`, without which no component can be placed.
|
||||
- Take the `.deb` path only when the app is not in the archive at all, not
|
||||
to add a package. Ask the requester for its URL and SHA-256; do not fetch
|
||||
a vendor's download page yourself. List its `Depends` as `dependencies`,
|
||||
with any runtime piece it lacks: it installs with the network off.
|
||||
- Take an AppImage or a tarball only when no package has the app. Pin it by
|
||||
the release file's URL for this host's architecture and the SHA-256 its
|
||||
project publishes, such as a GitHub release asset's digest or the
|
||||
release's `SHA256SUMS`; ask the requester when the project publishes none.
|
||||
The computer downloads it when the bundle is put, refuses other bytes, and
|
||||
unpacks it under `/opt/rea-apps/<desktop_entry>` with the network off; the
|
||||
export downloads the same file.
|
||||
- An AppImage installs its own desktop entry, so `desktop_entry` is the
|
||||
`.desktop` file at the top of the image, such as FreeCAD's
|
||||
`org.freecad.FreeCAD`. A wrong name fails the start with the entries the
|
||||
image has: start again with one of them. The image brings its libraries,
|
||||
so `extra_packages` is rarely needed; FreeCAD 1.1.4's needs none.
|
||||
- A tarball has no entry of its own. `program` is the program's path inside
|
||||
the archive, such as `firefox/firefox`; REA writes an entry named
|
||||
`desktop_entry` that runs it with the files it opens, and `name` is what
|
||||
the Applications menu shows. A tarball uses the computer's libraries: name
|
||||
what its program lacks in `extra_packages`. gzip, bzip2 and xz archives
|
||||
are read, zstd ones are not.
|
||||
- Flatpak and snaps are not supported: say so and stop. Flatpak's sandbox
|
||||
needs user namespaces the computer's container does not grant, so a
|
||||
Flathub app installs but never opens; ask for the project's AppImage,
|
||||
tarball or `.deb` when it ships one. An archive package that only installs
|
||||
a snap, such as noble's `firefox`, counts as a snap. A package with no
|
||||
build for this host's architecture (some PPAs are amd64 only) cannot be
|
||||
installed either.
|
||||
- `desktop_entry` is the package's `.desktop` file name under
|
||||
`/usr/share/applications` without `.desktop`, such as
|
||||
`org.inkscape.Inkscape`. Add `window_class` when you know it; Inkscape's
|
||||
is `Inkscape`. AbiWord's entry is `abiword` and its class `Abiword`.
|
||||
- To find the entry, read the package's file list: its page on
|
||||
packages.ubuntu.com, or `apt-get download PKG && dpkg -c PKG_*.deb` where
|
||||
apt runs. The entry may ship in a companion `-data` or `-common` package,
|
||||
as Planner's in `planner-data`: read that package's list too. The entry's `StartupWMClass` may not be the window's real
|
||||
class, so do not take `window_class` from it: MuseScore 3's says
|
||||
`mscore3`, its window's is `MuseScore3`. The export uses the class the
|
||||
capture saw.
|
||||
|
||||
## Drive the app
|
||||
|
||||
- Pass `computer_id` and the action's fields at the top level: click
|
||||
`{x, y, count}`, scroll `{x, y, direction, amount}`, a screenshot region
|
||||
`[x0, y0, x1, y1]`. The rest are in
|
||||
[the build-computer guide](https://github.com/morluto/rea/blob/main/docs/desktop-build-computer.md).
|
||||
- Take a `screenshot` before pointing: points are pixels of the latest
|
||||
frame. If a response is too large, ask for a `region`, or `jpeg` with a
|
||||
lower `quality`. A screenshot with `region` does not capture the screen
|
||||
again; it crops the latest frame, so take a full screenshot, or a `wait`
|
||||
with `screenshot: true`, first. The crop then becomes the latest frame:
|
||||
take a full screenshot before pointing again.
|
||||
- A drawn caret does not prove keyboard focus. After using a toolbar
|
||||
control, click into the document, then type a few characters and look
|
||||
before a long text.
|
||||
- Confirm a dialog with `key` Return, not a trailing `\n` in `type`.
|
||||
- An action's own screenshot is taken at once. After opening a menu, a
|
||||
dialog or a window, look with
|
||||
`{"action": "wait", "ms": 500, "screenshot": true}`.
|
||||
- On a computer started from a request the app is not open at first, and
|
||||
`start_build_capture` opens it: never open it yourself before that. Do not
|
||||
close it while capturing; the capture ends when the app exits. When the
|
||||
app is closed at any other time, open it from Applications → Other → the
|
||||
environment's name.
|
||||
- Type a secret with `{"action": "type_secret", "secret": "<handle>"}`,
|
||||
only into a field that masks what is typed: check in a screenshot that it
|
||||
is a password field, which shows dots. A screenshot shows any field that
|
||||
shows its text, such as many licence-key boxes, so never type a secret into
|
||||
one: say the app needs a secret the build cannot type unseen, and stop.
|
||||
`type_secret` takes no `screenshot`; look with a `wait` after it. A refusal
|
||||
names the secret's file and what is wrong with it: tell the person, never
|
||||
the value.
|
||||
- Prefer shortcuts and typing to long pointer paths: `key` takes `keys`, an
|
||||
array holding one chord's keys, such as
|
||||
`{"action": "key", "keys": ["Control", "s"]}`, and `type` turns `\n` into
|
||||
Enter.
|
||||
- Before `type`, check in a screenshot that a text cursor or a focused
|
||||
field is there. In apps with single-key shortcuts, such as Inkscape, text
|
||||
typed without one runs as commands.
|
||||
- Save As differs by app. Check the dialog's current folder first: most
|
||||
open in the home, so click Documents in the sidebar, but Scribus opens in
|
||||
`~/Documents` and its sidebar has no Documents entry. Type only the file
|
||||
name. A dialog taller than the 1280×800 screen hides its buttons:
|
||||
maximize it from its title bar, or confirm it with `key` Return, as when
|
||||
HomeBank's Save button is off-screen. When a GTK Save dialog ignores
|
||||
clicks or runs off the screen, type `~/Documents/<name>` in its Name field
|
||||
and press Return.
|
||||
- Answer the app's save questions, such as keeping the format, then check
|
||||
the saved path in the title or status bar, or the dialog's folder when
|
||||
the app shows only the name, and that it shows no unsaved mark, before
|
||||
you mark a checkpoint or check. Where the title never shows one, as in
|
||||
HomeBank, a greyed Save or Revert, or the file reopened from recent
|
||||
files, confirms the save.
|
||||
- Qt spin boxes change their value on scroll: move the pointer off them
|
||||
before you scroll.
|
||||
- gLabels edits an object's text in Object properties → Text, not on the
|
||||
canvas.
|
||||
- In Grisbi's transaction form Return moves to the next field; only Return
|
||||
in the last field, Notes, records the transaction.
|
||||
- ghostwriter autosaves a new document as `~/Documents/untitled-1.md`: name
|
||||
it with File → Rename, not Save As; Ctrl+Shift+S does nothing, and Enter
|
||||
after a checklist item starts another `- [ ]` item.
|
||||
- KMyMoney's first launch opens the New File Setup wizard; Finish asks for a
|
||||
storage type (XML), then a Save As where typing `Documents/<name>.kmy`
|
||||
works. Its date field is a sectioned Qt editor: double-click the month and
|
||||
type each part, since Ctrl+A garbles it. A new transaction's payee comes
|
||||
prefilled and selected. Amounts are saved as reduced fractions, such as
|
||||
`-1749/20`, so compare them exactly. Untick Help → Tip of the Day → "Show
|
||||
tips on startup" before `seed`, or the tip appears at every boot.
|
||||
- QElectroTech: an element dropped from Collections follows the pointer until
|
||||
a click on the folio places it. Its title shows only the app, so it needs
|
||||
`state_paths`; its settings are in `~/.qet`.
|
||||
- Rosegarden: after a dialog its Transport window can take keyboard focus,
|
||||
so save with File → Save.
|
||||
- Pencil2D: the Tools dock is too narrow for its buttons, so use shortcuts
|
||||
(B for the brush); double-click a layer's name to rename it.
|
||||
- Umbrello: the title shows no unsaved mark, so confirm a save with a probe
|
||||
checkpoint. An Untitled document autosaves `~/autosave.xmi`: delete it
|
||||
before `seed`. Add attributes from the tree: right-click the class → New →
|
||||
Attribute.
|
||||
- Hydrogen: the computer has no audio device, so Hydrogen falls back to a
|
||||
null driver and its BPM field snaps back to 120 within about 100 ms.
|
||||
Before `seed`, set Preferences → Audio System → ALSA with device `null`
|
||||
and click "Apply and restart output", which `~/.hydrogen/hydrogen.conf`
|
||||
keeps for the seed; also set autosaves per hour to 0, and dismiss the
|
||||
development-version dialog with "Don't show again". `pulseaudio` as an
|
||||
extra package does not help.
|
||||
- Poedit cannot add entries by hand. Use File → New, pick the language and
|
||||
save to `Documents` first (Sources Paths stays locked until then), then
|
||||
Properties → Sources Paths → + → Add Files with a source such as
|
||||
`/usr/lib/python3.12/argparse.py`. It writes `fr.mo` beside the `.po`, and
|
||||
the language box's autocomplete popup takes the first OK click.
|
||||
- CherryTree: its first-run autosave "Choose Storage Type" dialog swallows
|
||||
keys, so save once early as `.ctd` and untick autosave.
|
||||
- VSCodium (its tarball, with `extra_packages: ["libnss3",
|
||||
"libasound2t64"]`) runs on outside REA's reach, so `stop_build_capture`
|
||||
fails to clean it up: after `end`, quit it with Ctrl+Q, then stop. Enter
|
||||
accepts its word suggestions, so press Escape after typing a line.
|
||||
- Glade: a new widget's id is empty until you set it, and a second save
|
||||
writes a `.glade~` backup, which shows in the draft as an unrelated
|
||||
change. A backup present at `seed` is seeded with the document, so trash
|
||||
it before `seed`. The draft names a button's `<child>` by the id
|
||||
its widget carries, `[name=child][>object@id=submit_button]`, which moving
|
||||
the button keeps; give the widget an id before `seed` so it can.
|
||||
- VYM: the title never names the map, so pass `Documents/<map>.vym` in
|
||||
`state_paths`. Its settings live in `.config/InSilmaril` and are written
|
||||
only at a clean exit. It writes `<map>.vym.lock` while a map is open, so
|
||||
close the map before `seed`. Press A only after clicking a branch on the
|
||||
canvas: keys typed in the tree editor run other shortcuts.
|
||||
- KmPlot: Create → Cartesian Plot, then edit the formula in the Function
|
||||
tab and press Return. It stores minus as U+2212 (`−`). Its file:// title
|
||||
still names the file.
|
||||
- RedNotebook keeps each month in `~/.rednotebook/data/YYYY-MM.txt`, YAML
|
||||
read as text, folded at about column 80: pick check phrases the fold keeps
|
||||
on one line.
|
||||
- Qt5 apps saving with QDom, such as KmPlot, write attributes in a random
|
||||
order in each process, so check their XML by element, never by whole line.
|
||||
- Gnumeric's desktop entry is `org.gnumeric.gnumeric`. Its Save As dialog
|
||||
runs off the 1280×800 screen: type `~/Documents/<name>` in Name and press
|
||||
Return. Rename a sheet with right-click on its tab, then Rename….
|
||||
- Parley's title shows the collection's title, so name the file in
|
||||
`state_paths`. File → New opens the new collection's Properties dialog:
|
||||
confirm it with OK, since Cancel keeps the old collection and its autosave
|
||||
lock open. A new collection starts with 15 blank entries: delete them.
|
||||
Pick languages from the dropdown; a typed name keeps the previous locale.
|
||||
While a document is open KDE keeps an autosave and its `.lock` in
|
||||
`~/.local/share/stalefiles/parley/`, and there is no File → Close: before
|
||||
`seed`, use File → New, then reopen the collection from Open Recent.
|
||||
- Sigil: pass `extra_packages: ["libqt5svg5"]`, or its SVG toolbar icons are
|
||||
blank. Its title names the book, so it needs no `state_paths`. It unpacks
|
||||
the open book into `.local/share/sigil-ebook/sigil/workspace/Sigil-XXXXXX/`
|
||||
and deletes it on exit, so never name the whole `sigil-ebook` folder: that
|
||||
seeds the stale copy.
|
||||
- What the app saved under the home before `seed`, such as a document in
|
||||
`~/Documents` or its settings, is part of the seed. A file you move to the
|
||||
Trash is left out, and the export lists it in `limitations`. Trash a stray
|
||||
file rather than starting over. JabRef 3.8 writes `<file>.bak` and
|
||||
`<file>.sav` beside a `.bib` it saves; in the owned folder they are seeded
|
||||
and show in the draft as noise, so trash them before `seed` if they should
|
||||
not ship.
|
||||
- Dismiss first-run dialogs, such as a welcome screen, before `seed`. A
|
||||
choice the app saves only later, such as Inkscape's "show this every
|
||||
time", may not reach the seed, so expect the dialog in the witness too.
|
||||
- Some apps write their settings late: Freeplane writes
|
||||
`~/.config/freeplane/1.7.x/auto.properties` only once Preferences is
|
||||
confirmed, and a seed without it reopens its documentation map. Before
|
||||
`seed`, open the app's preferences and confirm them, then read a probe
|
||||
checkpoint's `changed_files` to see the settings file written.
|
||||
- Before `seed`, click into the document to clear any selection the app
|
||||
would save: Gnote saves its cursor and selection with the note.
|
||||
- An app whose state folder is named after it, such as
|
||||
`~/.local/share/gnote`, or after it and its version, such as TreeLine's
|
||||
`~/.treeline-3.1`, is found without `state_paths`.
|
||||
- TreeLine: File → New opens a template window. Book List's CATEGORY makes
|
||||
AUTHOR children by default; change it under Data → Configure Data Types →
|
||||
Default Child Type. A new child opens an inline rename showing "", so
|
||||
press Escape and fill its fields in the Data Edit tab. Ctrl+A adds a child
|
||||
node even while a field has focus. Clicking a node already selected starts
|
||||
an inline rename, so Ctrl+X then cuts its text: cut a node from its
|
||||
context menu. It saves nodes sorted by their
|
||||
generated uid1, so their order changes from run to run. The draft names a
|
||||
new node by its format and a field of its `data`,
|
||||
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, and checks its other
|
||||
fields through that, so give each new node a title no other node has.
|
||||
- After an app crashes, capture again on a fresh computer, so the seed
|
||||
carries no crash-recovery state, such as MyPaint's "Recover interrupted
|
||||
work?".
|
||||
- Give what the task changes a stable name where the app has one, such as an
|
||||
id in Inkscape's Object Properties, so a check can select it.
|
||||
- Expect apps to split text into runs: Writer's revision ids split an
|
||||
edited paragraph into spans and may give it an automatic style such as
|
||||
`P1`, and AbiWord wraps each insertion in `<c>`. Check such text whole
|
||||
with `text_equals` (see Write the task), and find the paragraph by
|
||||
something the edit keeps, such as AbiWord's `xid`.
|
||||
- Freeplane's first launch opens a documentation map: close it with Ctrl+W.
|
||||
Its Ctrl+N asks for a template.
|
||||
- MyPaint 2.0.1 crashes on brush strokes on the build computer: build its
|
||||
seed with flood fill and layer operations.
|
||||
- GnuCash's Ctrl+S on a new book opens a Save dialog with no format choice:
|
||||
use File > Save As to pick XML. Close the book (File > New File) before
|
||||
`seed`, so its `.LCK` lock is not seeded.
|
||||
|
||||
## Build the seed and the edit
|
||||
|
||||
1. `start_build_computer` with
|
||||
`{"source": {"kind": "request", "desktop_entry": ..., "install": ...}}`.
|
||||
The first start of an app builds its image, and starts at the same time
|
||||
on this host wait for each other; either can take minutes. A failed
|
||||
start returns the build's error: fix the request, or stop when the
|
||||
install cannot work, and close any `computer_id` the error names.
|
||||
2. `start_build_capture`. It returns once the app's window is open. A
|
||||
`package_lists_error` may stop the capture finding the package's
|
||||
sources; the export says when that matters. When the app exits before
|
||||
its window opens, the error gives its exit code and the end of its
|
||||
stderr, which may name what it lacks, such as an icon loader or a Java
|
||||
runtime: close the computer and start a new one with that package in
|
||||
`extra_packages` (see Choose the install), or stop as for an install
|
||||
this flow cannot do.
|
||||
- REA counts a file as the app's document only when the app's window
|
||||
title shows its file name or `~/` path. When the title shows something
|
||||
else, such as MuseScore's score title, pass the path you will save the
|
||||
document at in `state_paths`, relative to the home:
|
||||
`{"computer_id": ..., "state_paths": ["Documents/MuseScore3/Scores/Morning_Song.mscz"]}`.
|
||||
It need not exist yet. Add the app's settings folder when its name
|
||||
differs from the app's, such as `.config/MuseScore`, so the seed keeps
|
||||
the dismissed first-run dialogs. Do not pass the document as a launch
|
||||
argument to make REA see it: `state_paths` is the way. gLabels' title
|
||||
leaves out the file's extension and Grisbi's shows the entity's name,
|
||||
so both need it.
|
||||
- An app whose document is a folder of pages, such as a Zim notebook
|
||||
(`Notebooks/Notes`, a `.txt` file per page), never shows a file in its
|
||||
title: pass the folder in `state_paths`. Every page in it is then a
|
||||
document, so the bundle opens none at launch and the app starts where
|
||||
it keeps its place; check the page the task changes.
|
||||
3. Make the seed through the app and save it from the app under
|
||||
`~/Documents`: build the document in the app, or write its source where
|
||||
the app edits its format as text, then save it again from the app. Make
|
||||
it look real but synthetic: the thing the task changes, plus a few things
|
||||
the change must leave alone. Keep it to one document: the bundle's app
|
||||
opens the seed's document at launch only when there is exactly one, and
|
||||
otherwise starts on its own screen. For a starting file of the requester's,
|
||||
read it here and rebuild it in the app with synthetic data in place of
|
||||
personal data; REA cannot copy a file into the computer.
|
||||
- An app that keeps its own data and saves after every change, such as
|
||||
Osmo's contacts in `~/.local/share/osmo/contacts_records.xml`, has no
|
||||
document: that state is the seed. Build it in the app and mark `seed`
|
||||
once the app has written. To see whether it has, mark a throwaway
|
||||
checkpoint, such as `probe-1`, and read its `changed_files`: up to ten
|
||||
files that changed since the checkpoint before, those a window title
|
||||
named first, then those under `state_paths` and those the app held
|
||||
open, caches last, with `more_changed_files` counting the rest.
|
||||
- The app's own compressed format works when it is gzip, such as
|
||||
Gnumeric's `.gnumeric`, Dia's `.dia` or Inkscape's `.svgz`: REA reads
|
||||
the document inside, and checks point into it as into the same
|
||||
document uncompressed. Compression may be off by default: Dia's
|
||||
"Compress diagram files" is unticked, which REA reads as plain XML. A format REA reads as binary, such as one
|
||||
compressed with xz, has no view: save in a format the app offers that
|
||||
REA reads, then reopen the file once and check that a plain Ctrl+S
|
||||
keeps that format. When the task is about the app's own binary
|
||||
format, such as GIMP's `.xcf` or Blender's `.blend`, keep it and check
|
||||
it through a reader (see Read a binary file with a reader).
|
||||
4. Only once the seed is saved, `mark_build_checkpoint` with name `seed`.
|
||||
5. Make the change the task asks for, as a user would, and save it.
|
||||
6. Only once the change is saved, `mark_build_checkpoint` with name `end`.
|
||||
For more tasks in this environment, make each next task's change on top
|
||||
of the last as a user would, save it, and mark `end-<task name>`, such as
|
||||
`end-add-bread`. Each task is checked at its own end, so its checks must
|
||||
not read what a later task changes, and a solving agent starts each from
|
||||
the seed, so a task must not need an earlier one done first.
|
||||
Then, for each task you can, make a near miss of its change on top, a
|
||||
plausible wrong edit such as a typo or a wrong number, save it and mark
|
||||
`near-<task name>`: the export refuses a task whose near miss passes every
|
||||
check. Take the near misses after the last end, since each later draft
|
||||
starts from the end before it.
|
||||
7. `stop_build_capture`. Read `draft.files` (each file that changed, with
|
||||
its net change and its `values`), `draft.task.checks` (suggested
|
||||
checks), `draft.guards`, `draft.notes` and `limitations`; each
|
||||
`later_drafts` entry is the same for one later task, from the end
|
||||
checkpoint before its own, so it shows only that task's change. When
|
||||
`draft` is null, read `draft_refusal` first: a missing `seed` or `end`
|
||||
checkpoint means capturing again on a new build computer, while a failed
|
||||
draft command names its own cause to fix. If a declared secret was used,
|
||||
automatic drafting is withheld: write the task manually and export it.
|
||||
When `draft.files` is empty,
|
||||
`draft.notes` says why:
|
||||
- `no_change`: nothing was saved between `seed` and `end`. Capture again
|
||||
on a new build computer, and save before marking `end`. When the app
|
||||
keeps the task's state only on screen, such as a calculator's result,
|
||||
judge the screen instead (see Judge the screen).
|
||||
- `unowned_changes` with `no_document`: the app saved your document
|
||||
where REA did not count it as the app's. Its `files` list what changed,
|
||||
with their roles; the document is among them. Close the computer and
|
||||
capture again on a new one with that path in `state_paths`; the export
|
||||
refuses a check on a file outside the owned paths.
|
||||
- `uncheckable_changes`: the app saved the change in dconf, as gedit does
|
||||
its preferences, and no check reads dconf. Saving again does not help:
|
||||
choose a change the app saves in a file, and capture again on a new
|
||||
build computer, or judge the screen (see Judge the screen).
|
||||
- `unreadable_changes`: a dconf database changed but REA could not read
|
||||
it; `error` says why. This is not evidence the app saved there: report
|
||||
the error rather than changing the task.
|
||||
|
||||
A note with a `file` says why that file's suggestions are thin:
|
||||
- `by_position`: only position names what changed; its `elements` lack
|
||||
a naming attribute. Name a record that keeps its fields as child
|
||||
elements by one of them, as Osmo's contact by
|
||||
`[name=contact][>first_name=Delphine]`, a JSON record by a field
|
||||
inside it, `[>data/Title=…]` (`~1` for the `/` inside a pointer), or
|
||||
an element by an id its child carries, `[>object@id=…]`; count records
|
||||
with `member_count`, `matching` `[name=contact]`; otherwise check an
|
||||
ancestor's text, or give the element an id in the app and capture again.
|
||||
- `view_state_only`: every change is view state, a renumbered id or a
|
||||
time the app stamps, and its `pointers` are the checks kept. The edit
|
||||
was likely not saved: capture again and save it before `end`.
|
||||
- `long_text`: the element's text is too long to check whole; check a
|
||||
phrase in it with `text_contains`.
|
||||
- `cut_view` or `unreadable_view`: REA's view holds only part of the
|
||||
file, or none a check can read, such as a binary format (step 3). For
|
||||
a binary file, read it with a reader (see Read a binary file with a
|
||||
reader).
|
||||
|
||||
## Write the task
|
||||
|
||||
Write `{"name", "prompt", "checks"}` from the draft and what you did, one
|
||||
per task, each with its own name.
|
||||
|
||||
- `name`: lowercase letters, digits and hyphens, not `smoke`.
|
||||
- `prompt`: what the requester wants, in their words and the app's terms,
|
||||
starting with the app's name, naming the document as a user would and
|
||||
ending with saving it: "In Inkscape, change the poster's headline in
|
||||
Documents/poster.svg to Summer Bake Sale and save it." Name the app: the
|
||||
solving agent's own shell cannot see the computer's files, and the app's
|
||||
name sends it to the screen. Leave saving out for an app that saves on its own, after every change
|
||||
or a few seconds after the last one, such as Osmo or Gnote: grading waits
|
||||
until the app's files have held for 3 s, so the solving agent need not
|
||||
wait for the save. A save more than 3 s after the agent stops is still
|
||||
missed; name such an app in the report. Never say how the file stores it,
|
||||
and never give the checks.
|
||||
- Checks are file-based: content checks that prove the edit, never only that
|
||||
a file exists or changed. Start from the suggestions; keep those that say
|
||||
what the user did and drop what follows from how the app saves.
|
||||
- Compare exact values, so a typo or a wrong number fails.
|
||||
- Read each file's `values` before writing checks: every value the change
|
||||
added, changed and removed, with `before` and `after` (an uncut scalar, or
|
||||
a preview cut at 600 characters) at the pointer a check names it by. Copy
|
||||
the pointer. Copy `after` into `value` only when it is an uncut scalar: a
|
||||
string without a trailing `…`, a number, or a boolean. Compact XML and a
|
||||
JSON array or object are preview strings, so a `value_equals` check needs
|
||||
the structured value from the file's view, and element text uses
|
||||
`text_equals` or `text_contains`.
|
||||
Only XML files, archives, JSON and key files have values: a text file's
|
||||
or SQLite database's `change` already lists its lines and rows.
|
||||
`refers_to` says what a number or id stands for, such as `category="44"`
|
||||
naming `<cat key="44" name="Groceries"/>`, or a new KMyMoney
|
||||
transaction's splits naming their account and payee; use it for the
|
||||
prompt's words. `alternatives` gives a changed value's pointer by the
|
||||
element's other names, such as a label beside an id, to pick another key
|
||||
without another export.
|
||||
Skip entries whose `note` is `by_position` (no check can use that
|
||||
pointer) or noise (`view_state`, `renumbered`, `volatile`,
|
||||
`timestamp`, `upkeep`, `layout`). `values_cut` means a limit left some out.
|
||||
- Select elements by what they hold, never by position: SVG text by id,
|
||||
`/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0`,
|
||||
or an ODF paragraph by style,
|
||||
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Title]/children/0`.
|
||||
A selector must match exactly one element, so give the target a unique id
|
||||
or style in the seed. The draft's pointers already do this: a name-like
|
||||
attribute such as Scribus's `ANNAME` before a coordinate, and two
|
||||
attributes together where one is not enough, as a spreadsheet cell's
|
||||
`[@Col=3][@Row=2]`. Only an element's text is read by position, as
|
||||
`children/0`.
|
||||
- When an app may split an element's text into runs, check it whole:
|
||||
`text_equals` (or `text_contains` for a value inside a sentence) on the
|
||||
element compares all its text, however it is split. Add
|
||||
`normalize_whitespace: true` when the app may reflow spaces. Example, an
|
||||
AbiWord paragraph by its stable id:
|
||||
`{"kind": "text_contains", "file": "Documents/memo.abw", "pointer": "/root/children/[name=section]/children/[name=p][@xid=6]", "value": "at 3:30 PM"}`.
|
||||
Point a text check at the paragraph or another element the edit keeps,
|
||||
never at a run such as AbiWord's `[name=c]`: a reopened document splits
|
||||
the text into several runs. The draft lists only what changed; an
|
||||
unchanged element is in `draft.guards` when it is a like sibling of a
|
||||
changed one, and otherwise use the app's usual numbering, such as another
|
||||
paragraph's `xid`, and let the export confirm it.
|
||||
- In an ODS spreadsheet no attribute names a row: the draft names it by its
|
||||
label cell, `[name=table:table-row][>table:table-cell=February]`. Where no
|
||||
cell names it alone, name the sheet in the seed, and check its whole
|
||||
text: `text_equals` on
|
||||
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:spreadsheet]/children/[name=table:table][@table:name=Q1 Sales]`,
|
||||
or a stretch of it with `text_contains`. Cell texts join with nothing
|
||||
between, so a row reads `February23100`.
|
||||
- For a line added to free text, such as an item on a note's list, the
|
||||
draft compares the whole text with `text_equals`. This rejects longer or
|
||||
negated items, but can reject a correct edit with different ordering or
|
||||
spacing. Report that limitation when the task permits such variations.
|
||||
- For a note placed by its own `x` and `y`, such as Xournal++'s `<text>`
|
||||
on a layer, the draft checks its text on the layer with `text_contains`,
|
||||
which passes wherever the user puts it, and guards each note it kept.
|
||||
- Some apps keep text in an attribute, one element per run, such as
|
||||
Scribus's `<ITEXT CH=...>`: add `"text_attribute": "CH"` to `text_equals`
|
||||
or `text_contains` on the frame,
|
||||
`/root/children/[name=DOCUMENT]/children/[name=PAGEOBJECT][@ANNAME=Headline]`,
|
||||
to read its runs' values joined.
|
||||
- When the draft shows no values or no checks, a scratch export can expose
|
||||
`envs/<env>/seed/` only after the task has at least one valid check: it must
|
||||
pass on the captured end state and fail on the seed. Before the first
|
||||
successful export, use `act_on_build_computer` to open the saved file in an
|
||||
available GUI text editor or viewer and inspect its actual representation
|
||||
without changing it. Use the visible saved content and the task's edit to
|
||||
write a check; an export refusal can also name the end state's value, such
|
||||
as `is "44", not "1"`. If the file cannot be inspected or expressed by a
|
||||
supported check, report the limitation instead of guessing a check.
|
||||
- Where easy, add one check that the rest is untouched, such as another
|
||||
text's value the edit must leave alone. `draft.guards` offers them at
|
||||
their seed values, which the end holds too: a field on the nearest like
|
||||
siblings, one the user sets, such as Planner's `work`, before a default
|
||||
they all share; the text an addition kept; a text file's lines beside its
|
||||
new one. Copy one or two into `checks`; they pass on the seed, which is
|
||||
fine, since the export needs only one check to fail there.
|
||||
- The draft suggests a decimal the app writes as text, such as `146.83`,
|
||||
as a number with `tolerance: 0.001`; keep the tolerance for any number
|
||||
you check that way.
|
||||
- When the draft shows an XML document's changes as `lines`, REA read it
|
||||
as text, for example because its DOCTYPE declares entities. Selectors are
|
||||
impossible there, and `line_count` with `contains` only approximates a
|
||||
value: `3:30` also matches `13:30`. Bound it with neighbouring text and
|
||||
name the check as approximate in the report's limitations.
|
||||
- Some apps save a document reloaded from disk differently from one typed
|
||||
in the same session: AbiWord wraps each new insertion in its own markup.
|
||||
Your build's end state hides this, which is why whole-text checks matter;
|
||||
expect the witness to show the real format.
|
||||
|
||||
The kinds and pointer rules are in
|
||||
[the desktop tasks section](environments.md#pick-the-checks).
|
||||
|
||||
### Read a binary file with a reader
|
||||
|
||||
When the draft notes `unreadable_view` on a binary document, add a reader to
|
||||
the task: the app's own headless command, which REA runs in the computer on a
|
||||
private copy of the file, and whose output a check reads instead of the file.
|
||||
Write it in the app's scripting mode, print each fact on a line starting
|
||||
with `REA ` (with the space), and name the copy `{file}`:
|
||||
|
||||
- GIMP `.xcf`, layer names, as `lines`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "layers",
|
||||
"file": "Documents/poster.xcf",
|
||||
"argv": ["gimp-console-2.10", "-i", "-d", "-f", "-b", "-"],
|
||||
"stdin": "(let ((image (car (gimp-file-load RUN-NONINTERACTIVE \"{file}\" \"{file}\")))) (for-each (lambda (layer) (display \"\\nREA layer \") (display (car (gimp-item-get-name layer)))) (vector->list (cadr (gimp-image-get-layers image)))) (newline))\n(gimp-quit 0)\n",
|
||||
"output": "lines"
|
||||
}
|
||||
```
|
||||
|
||||
checked with `{"kind": "lines_present", "file": "Documents/poster.xcf", "reader": "layers", "lines": ["layer Summer Sale"]}`.
|
||||
Script-Fu's prompt `ts> ` starts a line, so print a newline before each
|
||||
marked line, as `\nREA` does. A text layer's text is
|
||||
`(car (gimp-text-layer-get-text layer))`.
|
||||
|
||||
- Blender `.blend`, objects with their type and location, as `json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "objects",
|
||||
"file": "Documents/scene.blend",
|
||||
"argv": [
|
||||
"blender",
|
||||
"-b",
|
||||
"--factory-startup",
|
||||
"{file}",
|
||||
"--python-expr",
|
||||
"import bpy, json; print('REA ' + json.dumps({o.name: {'type': o.type, 'location': [round(v, 4) for v in o.location]} for o in bpy.data.objects}, sort_keys=True))"
|
||||
],
|
||||
"output": "json"
|
||||
}
|
||||
```
|
||||
|
||||
checked with `{"kind": "value_equals", "file": "Documents/scene.blend", "reader": "objects", "pointer": "/Crate/location/0", "value": 2, "tolerance": 0.001}`.
|
||||
Print every value a check compares as a number or string a check can
|
||||
name, rounded where the app stores floats.
|
||||
Blender 4 shows its Quick Setup splash at every launch until its
|
||||
preferences are saved: with `.config/blender` in `state_paths`, confirm
|
||||
Quick Setup with Continue, save the preferences (Edit → Preferences, then
|
||||
Save Preferences in the menu at its bottom left), and read a probe
|
||||
checkpoint's `changed_files` for `.config/blender/4.0/config/userpref.blend`
|
||||
before `seed`, so solves start on the scene. Blender often takes the first click on a
|
||||
menu or a dialog's button only to focus it: click again.
|
||||
|
||||
- A format LibreOffice opens but REA cannot read, such as a legacy `.xls`,
|
||||
converted with `--convert-to`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "cells",
|
||||
"file": "Documents/budget.xls",
|
||||
"argv": [
|
||||
"sh",
|
||||
"-c",
|
||||
"soffice --headless --convert-to csv --outdir out \"$1\" >/dev/null && sed 's/^/REA /' out/*.csv",
|
||||
"sh",
|
||||
"{file}"
|
||||
],
|
||||
"output": "lines",
|
||||
"timeout_s": 120
|
||||
}
|
||||
```
|
||||
|
||||
Pass `{file}` to `sh -c` as an argument, as here, rather than inside the
|
||||
script.
|
||||
|
||||
The reader runs without the display, in a fresh home with the app's default
|
||||
settings, so use the app's headless or batch mode; set `display: true` only
|
||||
for an app that has none. It must exit 0 within `timeout_s` (60 s by
|
||||
default, at most 300) and print at most 1 MiB. Pick facts the change sets and
|
||||
the near miss gets wrong, such as a name or a number, not the app's whole
|
||||
output.
|
||||
|
||||
Give the task the reader in `readers`, and each check through it `reader`
|
||||
and the reader's `file`. `json` readers serve `value_equals`, `value_absent`
|
||||
and `member_count`; `lines` readers `lines_present`, `lines_absent` and
|
||||
`line_count`. The export runs the reader in the build computer on the seed
|
||||
and the end: when a check through it fails on the end, the refusal shows
|
||||
what the reader printed, and a reader that fails shows its error and the end
|
||||
of its stderr. Fix the reader or the check from that and export again.
|
||||
|
||||
### Judge the screen
|
||||
|
||||
Only when no file shows the change, as when the draft notes `no_change`
|
||||
because the app keeps its state on screen, write the task's checks as
|
||||
screen judges: `{"id": "shown", "kind": "screen_judge", "question": "Does the
|
||||
calculator's display show 8.568?"}`. A model, the judge, answers the question
|
||||
on a screenshot three times, and the check passes when most answers are yes.
|
||||
|
||||
- REA keeps a screenshot at each checkpoint. Mark `seed` with the screen
|
||||
showing the state before the task, where the answer is no, and `end` with
|
||||
the change showing, where it is yes: the bundle ships both as the judge's
|
||||
examples.
|
||||
- Ask about what the change makes visible, with exact values, so the near
|
||||
miss is a no: a result, a label, a selected item. Avoid what a window
|
||||
moved or a cursor blinking can change.
|
||||
- The prompt asks to leave the result showing, since the judge sees only
|
||||
the screen when the agent stops.
|
||||
- A task with screen judges holds only screen judges and no readers. The
|
||||
export refuses one when a file shows the change, and checks none of its
|
||||
questions: the witness does, running the bundle's own judge with the
|
||||
runner's model endpoint (`LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
|
||||
`[model]` in `.agentenv/config.toml`) on a fresh screenshot at each stage.
|
||||
Without an endpoint the witness's check fails: say what to set and stop.
|
||||
- The judge's model is `[model.roles] judge`, else `[model] default`, else
|
||||
`anthropic/claude-sonnet-5-5`. `agent-env run --judge-model` does not
|
||||
reach it. Each grade costs about $0.02.
|
||||
|
||||
## Export
|
||||
|
||||
`export_build_environment` with `tasks` and `directory`, the absolute path of
|
||||
a new or empty folder here, such as `<home>/rea-builds/poster/bundle`. Each
|
||||
entry of `tasks` is `{"task": ..., "end_checkpoint": ...}`: leave
|
||||
`end_checkpoint` out for the task checked at `end`, and give
|
||||
`end-<task name>` for each later one. The
|
||||
export seeds what was saved at `seed`, replaces the computer's host name,
|
||||
`user@host` and home path (a bare user name, such as AbiWord's author
|
||||
`user`, stays), and checks each task against `seed`, its own end and its
|
||||
`near-<task name>` when you took one. It refuses a task
|
||||
unless at least one check fails on the seed, every check passes at its
|
||||
end and one fails at its near miss, and says which task and why, such as a
|
||||
pointer that names nothing at the end. A
|
||||
check that the rest is untouched passes on the seed too, which is fine. On
|
||||
a refusal, fix the task and export again, into another folder if that one
|
||||
is no longer empty. The bundle's eval lists every task, so a run of the
|
||||
bundle runs them all. A refusal naming a secret's handle means a file of the
|
||||
bundle holds that secret, most often an app's sign-in or licence state in
|
||||
the seed: do not work around it, but report it and stop. Keep
|
||||
`seed_files`, `scrubbed_files`, `unscrubbed_files` and `limitations` for the
|
||||
report.
|
||||
|
||||
The limitations name the folders the app wrote that the bundle neither owns
|
||||
nor seeds, such as `~/.config/MuseScore`, so the app starts without them:
|
||||
expect its first-run wizard, a tour or a "restore session?" question at every
|
||||
boot. If the witness shows one, capture again on a new build computer with
|
||||
those folders in `state_paths`, and dismiss the dialogs before `seed`. Lock
|
||||
files the app holds while it runs, such as LibreOffice's `.lock` and
|
||||
`.~lock.<name>#`, GnuCash's `<book>.LCK` or Grisbi's `.<file>.lock`, are
|
||||
left out of the seed and listed after "Left out of the seed". An app can
|
||||
still start fine yet refuse every save when a lock was seeded, so close the
|
||||
document before `seed` where the app lets you (apps other than LibreOffice,
|
||||
such as Rosegarden, also write `.~lock.<name>#` and autosaves beside a
|
||||
document, and closing it covers them too), and treat "file is locked"
|
||||
on a save in the witness as a seeded lock: delete it from the bundle's seed
|
||||
and witness again in a new computer. In LibreOffice, own the profile
|
||||
(`state_paths: [".config/libreoffice"]`) for a clean boot, and close the
|
||||
document before `seed`, then reopen it from Recent Documents to make the
|
||||
change: the export also leaves out the recovery list LibreOffice writes while
|
||||
a document is open, which would start the bundle into document recovery. A
|
||||
limitation naming `~/.config/dconf/user` as unread means the app's dconf
|
||||
settings, such as a first-run tip you turned off, are not seeded: capture
|
||||
again on a new build computer. The limitations also name seeded WebKit,
|
||||
Chromium or QtWebEngine storage and caches; when the witness's boot is clean,
|
||||
list them in the report.
|
||||
|
||||
Keep the build computer open until the witness holds: a task fix exports
|
||||
again from it.
|
||||
|
||||
## Witness
|
||||
|
||||
Witness each task in its own fresh computer of the bundle, from the prompt,
|
||||
as a solving agent would; a computer is witnessed once, so the steps below
|
||||
repeat for each task, each with its own record folder. Do not replay the build's actions or coordinates, and do not
|
||||
check between stages to steer the edit: a solving agent never sees the
|
||||
checks. Give every `check_build_task` the task exactly as exported.
|
||||
|
||||
1. `start_build_computer` with
|
||||
`{"source": {"kind": "bundle", "path": "<bundle>"}}`, adding
|
||||
`"settings": {"<variable>": "<handle>"}` for each of the export's
|
||||
`withheld_files`, or the deploy is refused for want of them, and agentenv
|
||||
keeps that bundle's build failed, so start from a fresh copy of it. The app starts
|
||||
with the seed's files in place and opens the seed's document when the
|
||||
seed holds exactly one and the app takes a file. When the export's
|
||||
`limitations` say it opens no document, open it from the prompt's path,
|
||||
as a solving agent would. Take a `wait` screenshot first, since the
|
||||
desktop may not have drawn yet. When the start fails because the
|
||||
computer never got ready, and its error names lock files in the bundle's
|
||||
seed, delete them from the bundle and start again.
|
||||
2. `check_build_task` with `stage: "seed"` and `directory`, the absolute
|
||||
path of a new or empty folder for the record. A check must fail.
|
||||
Recording starts. Later stages take no `directory`.
|
||||
3. Do the task from the prompt with `act_on_build_computer` and save.
|
||||
`check_build_task` with `stage: "after_edit"`: every check must pass.
|
||||
Each check waits until the app's files have held for 3 s, as the
|
||||
bundle's verifier does, so an app that saves on its own needs no wait of
|
||||
yours. A result with `settled: false` means the files were still
|
||||
changing at the 15 s bound, and `read_cut: true` that a slow read was
|
||||
still running there.
|
||||
4. Make a near miss on top of it: a plausible wrong edit, such as a typo or
|
||||
a wrong number, and save. `check_build_task` with `stage: "near_miss"`:
|
||||
a check must fail.
|
||||
5. `act_on_build_computer` with `action: "reset"`, then `check_build_task`
|
||||
with `stage: "after_reset"`: a check must fail again.
|
||||
6. The last stage ends the witness: `witness.record` has the verdict, the
|
||||
path of `witness.json` and the recording's segments in `directory`.
|
||||
`close_build_computer` on the witness computer.
|
||||
|
||||
A stage that does not hold ends the witness as `failed`. Read each check's
|
||||
result and decide:
|
||||
|
||||
- `after_edit` failed though the saved file is right: the check is brittle.
|
||||
Fix it, export again and witness in a new computer.
|
||||
- `after_edit` failed because your edit was wrong: witness again in a new
|
||||
computer; a computer is witnessed once.
|
||||
- `near_miss` passed: the checks are too loose. Tighten them, export again
|
||||
and witness again.
|
||||
- `after_reset` passed: the reset did not restore the seed. Report it as a
|
||||
limitation.
|
||||
|
||||
Stop after three witnesses of a task and report what still fails.
|
||||
|
||||
## Measure
|
||||
|
||||
Only if the requester asked for it, or by the headless default above. If
|
||||
they wanted to review the tasks first, show them each prompt and checks and
|
||||
wait for their approval. Then `measure_build_task` each task with `bundle`,
|
||||
`task`, `model` and `budget_usd`, its share of the budget. It runs the task
|
||||
in fresh computers, 3 times, then 2 more unless they passed none or all, and
|
||||
keeps a task that passes 1 to 3 of 5 (`band: "in_band"`). It stops before a
|
||||
solve the largest cost so far would take past the budget, and writes the
|
||||
record into the bundle's `difficulty.json` and README. The solver needs a
|
||||
model endpoint where REA starts: `LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
|
||||
`base_url` and `api_key` under `[model]` in `.agentenv/config.toml`; when the
|
||||
measure says a solve did not run for want of one, tell the person what to
|
||||
set and stop. Read the result's `failures_per_check` and each solve's
|
||||
`failed_checks` before calling a task hard: a check that fails in most failed
|
||||
solves while the app shows the right change is brittle, so fix the check,
|
||||
export and witness again, rather than count it as difficulty.
|
||||
|
||||
## Harder tasks
|
||||
|
||||
When the requester wants tasks worth evaluating or training on, harden a
|
||||
task that came out `too_easy` and measure it again, within the budget. The
|
||||
solver sees and fixes its slips on screen, so one edit with the value in the
|
||||
prompt is solved almost every time; so far, on TreeLine with Sonnet 5.5, so
|
||||
were one more dialog, a conditional move of seven records, and ten
|
||||
corrections read from a second document and applied across the list. Try,
|
||||
one at a time, measuring each:
|
||||
|
||||
- A longer chain of exact edits than that, each a value the checks compare
|
||||
whole, so one slip in the chain fails the task.
|
||||
- Facts read in one view and used in another: values the solver must read
|
||||
from a second document the app opens, or from another record, rather than
|
||||
from the prompt, such as "the same year as The Quiet Glacier's".
|
||||
- Distractors beside each target: records with near-identical names or
|
||||
values, and a guard check on each that it stays as it was.
|
||||
- Mistakes that do not show once saved: a value the app displays rounded or
|
||||
formatted unlike what it stores, or an edit the app keeps only when its
|
||||
field loses focus.
|
||||
|
||||
Stop when the budget left cannot cover 3 solves of the hardened task, and
|
||||
report the bands measured.
|
||||
|
||||
Each hardened task stays sound: its checks still fail on the seed and pass on
|
||||
its end, its near miss fails, and another valid way of doing it passes, so
|
||||
check values, never the order or method the solver used. Put the second
|
||||
document in the seed. Count records with `member_count` and check each
|
||||
value by a selector that names its record, since no check joins two records.
|
||||
|
||||
## Close and report
|
||||
|
||||
`close_build_computer` on every computer still open, also after a failure.
|
||||
If a close fails, call it again; if it still fails, tell the requester what
|
||||
is left.
|
||||
|
||||
Report:
|
||||
|
||||
- the bundle path and the environment name;
|
||||
- each task's name, prompt and end checkpoint;
|
||||
- each check, and why it proves the edit and fails the near miss, and each
|
||||
reader's command;
|
||||
- each task's witness stages with their results, the verdict, and the paths of
|
||||
`witness.json` and the recording;
|
||||
- the seed's contents: `seed_files` and what each holds;
|
||||
- `scrubbed_files` and `unscrubbed_files`; an unscrubbed file still names
|
||||
the host or user, so it needs review before sharing. `recompressed_files`
|
||||
are gzip seeds whose headers were cleared, their content unchanged: give
|
||||
their count;
|
||||
- limitations: the capture's, the export's and anything you could not do;
|
||||
- each measured task's band, passes of finished solves, turns, cost and
|
||||
failures per check, from `difficulty.json`;
|
||||
- the build time and the questions you asked.
|
||||
|
||||
Never publish or share the bundle yourself.
|
||||
@@ -1,399 +0,0 @@
|
||||
# Agent environments from a live application
|
||||
|
||||
Use this when the user asks for an agent environment (an AgentEnv bundle) of a
|
||||
web application: a folder `agent-env` deploys, whose MCP tools answer from what
|
||||
one recording of the application observed and serve the writes that recording
|
||||
confirmed. The workflow records the application once with
|
||||
`capture_browser_scenario`, checks what the environment would serve, records
|
||||
again if a hint names a gap, then writes the bundle with
|
||||
`export_environment_bundle`. For a HAR the user already has, skip to
|
||||
[export](#export-and-hand-off) through `inspect_web_network_capture`, or use
|
||||
`rea environment-build <capture.har> <dir>`. For a task in a desktop
|
||||
application's exported environment, read
|
||||
[its section](#tasks-for-a-desktop-application-environment).
|
||||
|
||||
## Before recording
|
||||
|
||||
- Record only against an instance the user says is disposable or seeded for
|
||||
this purpose: every write step creates, edits or deletes its data. If the
|
||||
user has not said so, ask before any scenario that writes. REA does not
|
||||
start, seed or reset applications.
|
||||
- Credentials come from environment variables the user names. Declare each as a
|
||||
scenario secret, `{secret_id, environment_variable}`; the variable must be set
|
||||
in the environment REA's MCP server started with. Never ask for a password in
|
||||
chat. Declared secrets are redacted as the capture records.
|
||||
- Map the application first with `inspect_web_page` (and `capture_web_screenshot`
|
||||
when layout matters): the login form's locators, and the API calls behind the
|
||||
pages that list and edit the entities the user asked about.
|
||||
- `inspect_web_page` cannot sign in. To see pages behind sign-in, run a short
|
||||
`capture_browser_scenario` that signs in and opens them, with
|
||||
`"capture": {"after_each_step": ["accessibility"]}` (`"dom"` for markup).
|
||||
Read a step's tree with `inspect_analysis_view`, `view: {kind:
|
||||
"browser-scenario", collection: "text", step_index, artifact:
|
||||
"accessibility", offset: 0, limit: 2048}`, paging with `offset`. Capture
|
||||
options are only `after_each_step`, `at_end`, `events`, `network` and
|
||||
`limits`.
|
||||
|
||||
## Record a session an environment can learn writes from
|
||||
|
||||
One `capture_browser_scenario` records the whole session. Start from this
|
||||
shape, with the application's own locators, paths and headers:
|
||||
|
||||
```json
|
||||
{
|
||||
"browser": { "mode": "launch", "executable_path": "/usr/bin/chromium" },
|
||||
"start_url": { "url": "http://127.0.0.1:3000/login" },
|
||||
"secrets": [
|
||||
{ "secret_id": "admin_password", "environment_variable": "APP_PASSWORD" }
|
||||
],
|
||||
"capture": {
|
||||
"network": {
|
||||
"request_body": true,
|
||||
"response_body": true,
|
||||
"header_values": true
|
||||
}
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"step_id": "user",
|
||||
"action": "fill",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=username]" },
|
||||
"value": { "source": "literal", "value": "admin" }
|
||||
},
|
||||
{
|
||||
"step_id": "password",
|
||||
"action": "fill",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=password]" },
|
||||
"value": { "source": "secret", "secret_id": "admin_password" }
|
||||
},
|
||||
{
|
||||
"step_id": "sign-in",
|
||||
"action": "press",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=password]" },
|
||||
"key": "Enter"
|
||||
},
|
||||
{
|
||||
"step_id": "signed-in",
|
||||
"action": "wait_for",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "nav a[href='/settings']" },
|
||||
"state": "visible"
|
||||
},
|
||||
{
|
||||
"step_id": "folders-1",
|
||||
"action": "request",
|
||||
"timeout_ms": 15000,
|
||||
"method": "GET",
|
||||
"path": "/api/folders",
|
||||
"expected_status": [200]
|
||||
},
|
||||
{
|
||||
"step_id": "new-folder-1",
|
||||
"action": "click",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "role", "role": "button", "name": "New folder" }
|
||||
},
|
||||
{
|
||||
"step_id": "folder-title-1",
|
||||
"action": "fill",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=title]" },
|
||||
"value": { "source": "literal", "value": "Scratch folder 1" }
|
||||
},
|
||||
{
|
||||
"step_id": "create-folder-1",
|
||||
"action": "click",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "role", "role": "button", "name": "Create" },
|
||||
"save_response": {
|
||||
"method": "POST",
|
||||
"path": "/api/folders",
|
||||
"save": [{ "name": "folder_1", "json_pointer": "/id" }]
|
||||
}
|
||||
},
|
||||
{
|
||||
"step_id": "read-folder-1",
|
||||
"action": "request",
|
||||
"timeout_ms": 15000,
|
||||
"method": "GET",
|
||||
"path": "/api/folders/{saved:folder_1}",
|
||||
"expected_status": [200]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Give each step that accepts `timeout_ms` a deadline. Without one, a step waits
|
||||
until it completes or the scenario is cancelled, so a locator that never
|
||||
matches hangs the capture and the client with it. The `wait_for_timeout` pause
|
||||
instead uses `duration_ms` and does not accept `timeout_ms`. Match `wait_for`
|
||||
locators to exactly one element.
|
||||
|
||||
1. Sign in through the UI: `fill` the user and password fields (the password
|
||||
as a secret reference), submit, then `wait_for` an element only a signed-in
|
||||
user sees.
|
||||
2. Read with `request` steps: each collection the environment needs, twice
|
||||
before any write, so fields that change on their own show up; then each
|
||||
object a list shows by id, up to ten per collection. An object known only
|
||||
from a list that leaves out fields its own read shows cannot be read by id
|
||||
or updated in the environment.
|
||||
3. Write the way the application's UI does: fill its forms and click its
|
||||
buttons, so each request carries the fields the UI sends, including ones
|
||||
the server would otherwise derive. Give the click or key press that submits
|
||||
a create `save_response` to keep the new object's ID, then `goto` its page
|
||||
(`/items/{saved:item_1}`) or read it with a `request` step. Never guess an
|
||||
ID, and never create a probe object to learn the next one. Write with
|
||||
`request` steps instead only when the UI cannot reach the operation (an
|
||||
API-only resource, or a UI that does not call the JSON API), or when coverage
|
||||
withholds the UI-shaped write as `unexplained-change` or `contradicted`;
|
||||
that counts as one of the two re-recordings. A `request` step write that
|
||||
coverage withholds that way usually lacks fields the UI sends: record it
|
||||
through the UI instead. If a journey cannot be scripted, leave the write
|
||||
unserved and report the limitation.
|
||||
Record two numbered scratch objects per write type,
|
||||
with varied free text. After each write, and before the next one, read the
|
||||
object back by id and then its list: create 1, read 1, list, create 2,
|
||||
read 2, list; never several writes before their reads.
|
||||
4. Leave more than a second between two creates and between a create and its
|
||||
first edit (`wait_for_timeout` with `duration_ms: 1100`), so lists sorted
|
||||
by creation time show them in order.
|
||||
5. Update both objects and read each back, changing the fields a person
|
||||
would change (a name, an email address, a status, a due date) rather than
|
||||
only a description: a served update accepts only the members a recorded
|
||||
update stored. Run an action that turns something on or off (star,
|
||||
bookmark, archive) twice in a row on one object, reading it back after
|
||||
each: the second call shows whether it sets or flips the field, and the
|
||||
environment serves the action only once a call has shown it. Delete both
|
||||
and read each back (expect 404, or the object marked deleted) and the list.
|
||||
6. Vary what the two creates send, however you write them: the first every
|
||||
field the application takes, the second only what a person must give (its
|
||||
title or name), with the other fields left out or empty. A served create
|
||||
requires each member both creates filled in and lets a caller leave out one
|
||||
a create left out or sent empty, so an agent that names only a title is not
|
||||
refused for fields it had no reason to set.
|
||||
|
||||
About `request` steps:
|
||||
|
||||
- `path` is on the start URL's origin; the page must be on that origin.
|
||||
- Header values are literals, declared secrets, a cookie the browser holds
|
||||
(`{source: "cookie", name}`), a local storage entry
|
||||
(`{source: "local_storage", key, json_pointer}`), a saved value
|
||||
(`{source: "saved", name}`), or Basic credentials
|
||||
(`{source: "basic_credentials", parts: [username, password]}`, sent as the
|
||||
base64 of `username:password` and redacted), with an optional `prefix` such as
|
||||
`"Bearer "` or `"Basic "`. Each credential part is a literal, secret, saved
|
||||
value, cookie or local storage entry, so a header the frontend builds from a
|
||||
login token (the username and the token) can be copied too; the password is
|
||||
a secret or saved value, never a literal. Copy the
|
||||
headers the application's own requests send (a CSRF header such as
|
||||
`X-CSRFToken` from the `csrftoken` cookie, a bearer token). Values read from the page are redacted from records made after they
|
||||
are read, not from the login response that returned them, and values
|
||||
shorter than 8 characters are not redacted at all.
|
||||
- `save: [{name, json_pointer, secret}]` keeps a string or number from the
|
||||
step's JSON response, such as the created object's `/id`, for later steps;
|
||||
`secret: true` redacts it, as for a token the API returns, so mark every
|
||||
saved token, key or session value secret. This also protects the selected
|
||||
source response before its body or headers are retained. Compressed or
|
||||
unknown-size UI responses cannot supply saved values. On a `click` or `press`,
|
||||
`save_response: {method, path, save}` keeps values from the response to the
|
||||
new request observed while the step waits (`{*}` in `path` matches one segment).
|
||||
Matching does not prove UI causality: choose a path distinct from polling;
|
||||
multiple observed matches fail as ambiguous. The step
|
||||
fails unless a 2xx JSON response arrives before its `timeout_ms`. Later
|
||||
steps send a saved value as `{saved:name}` in a request `path` (and its
|
||||
query) or a `goto` URL; in the `json` body, where a string that is only
|
||||
`{saved:name}` keeps the saved JSON type; or as `{source: "saved", name}` in
|
||||
a header or credential part.
|
||||
- Reading a page value makes later bodies costlier to keep: after a CSRF
|
||||
token, bodies over about 47 KiB are kept as `unavailable`. When the API
|
||||
returns larger responses, raise `capture.limits`
|
||||
(`max_network_body_allocation_bytes`, `max_total_event_allocation_bytes`).
|
||||
- Request steps follow no redirects: a redirect fails the step, and
|
||||
`expected_status` cannot name a 3xx status. Use the path the redirect names,
|
||||
such as the one with the trailing slash.
|
||||
- Set `expected_status` on every step, so a refused call fails its step and
|
||||
the later steps report not having run, instead of recording read-backs of
|
||||
nothing.
|
||||
|
||||
Keep it to about 60 to 160 requests. A scenario with no `request` step is
|
||||
recorded exactly as before.
|
||||
|
||||
The capture returns its whole Evidence, every recorded body included, which is
|
||||
often more than a client shows inline. Before checking coverage, confirm that
|
||||
every entry of `normalized_result.steps` has `status: "completed"`; if the
|
||||
client saved the result to a file, search it for `"status":"failed"` and read
|
||||
that step's `error`. A failed sign-in or write cancels every step after it.
|
||||
Use the result's `evidence_id` for coverage and export; do not paste the
|
||||
Evidence back into another tool.
|
||||
|
||||
## Check coverage
|
||||
|
||||
`inspect_analysis_view` with `source: {kind: "retained-evidence",
|
||||
evidence_id}` (the capture's id) and `view: {kind: "environment-coverage",
|
||||
template_rules: ["after-uid-literal", "opaque-token", "sibling-variance",
|
||||
"prefix-variance"]}`. It reports read tools
|
||||
answered from the state or replayed (`reads.recorded_reasons`), the served write
|
||||
tools, and each withheld write with its `reason` and `hint`. It does not apply
|
||||
the serve-time catalog and state size checks; the export reports those.
|
||||
|
||||
Re-record a withheld write at most twice, and only for a recording gap. Stop at
|
||||
once when the reason is application behaviour, and report the write as
|
||||
withheld with its hint:
|
||||
|
||||
| Reason | What to do |
|
||||
| --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
|
||||
| `no-observation` | Re-record: read the entity back by id after the write; read it first for an update or delete. |
|
||||
| `unconfirmed` | Re-record: read the entity and its list after the write. |
|
||||
| `incomplete-entities` | Re-record: read the entity through every view that lists it. |
|
||||
| `unexplained-change` | Re-record once with a second, different call of the same write. |
|
||||
| `no-call` | Re-record once with a write that succeeds. |
|
||||
| `side-effect` | Stop: the application changes other entities. |
|
||||
| `contradicted` | Stop: a later read disagrees with the modelled effect. |
|
||||
| `body-unusable` | Re-record a UI write once with `request` steps (its body was not exposed); else stop. |
|
||||
|
||||
Use new scratch names on each attempt, and delete what an earlier attempt
|
||||
created before recording again (through the UI, or a `request` step with the
|
||||
id that capture shows): the environment starts from what the recording first
|
||||
reads, so a leftover object becomes seeded data that write tasks then find.
|
||||
New browser scenario captures include request times, so a fully timed capture
|
||||
can learn a write clock and serve updates that stamp modification times. Older
|
||||
captures or ones missing a request start keep record order and learn no clock.
|
||||
When coverage reports that limitation, record again with the current version
|
||||
before concluding that the timestamp change cannot be served.
|
||||
|
||||
## Export and hand off
|
||||
|
||||
`export_environment_bundle` with `source: {kind: "retained-evidence",
|
||||
evidence_id}` (or `{kind: "evidence-file", path}`), an absolute
|
||||
`output_directory` that does not exist yet and whose parent does, and a `name`
|
||||
(required when the application runs on `localhost` or an IP address). It returns
|
||||
the coverage again, `tool_count`, the written files and warnings; `dry_run:
|
||||
true` reports without writing. The bundle installs `rea-agents` of this REA's
|
||||
version from the registry; when the user runs an unreleased REA build, pass the
|
||||
`package_tarball` they give you (a packed `rea-agents` tarball) instead.
|
||||
|
||||
Tell the user how many tools answer from the state, which writes are served and
|
||||
which are withheld and why, and the run command (`agent-env run <folder>`). Say
|
||||
that the bundle ships the recorded responses and the capture's local paths
|
||||
(for a browser scenario, the browser executable's path), and
|
||||
list any `credential-named-members` warning pointers for review. Never publish
|
||||
or share the bundle yourself.
|
||||
|
||||
See
|
||||
[agent environments](https://github.com/morluto/rea/blob/main/docs/agent-environments.md)
|
||||
for what the state, replay and the bundle establish, and
|
||||
[evidence workflows](evidence-workflows.md) for the manifest and state views.
|
||||
|
||||
## Tasks for a desktop application environment
|
||||
|
||||
Use this when the user has a folder from `rea capture-desktop-application` and
|
||||
wants a task for the environment `rea export-desktop-application` makes from
|
||||
it: a prompt for a computer-use agent and checks that grade what the agent
|
||||
left in the app's files. You write the prompt and the checks; REA validates
|
||||
the task against the capture and exports it. These commands are CLI only. To
|
||||
build the environment and its task from a request instead, follow
|
||||
[desktop builds](desktop-build.md).
|
||||
|
||||
### Describe the change
|
||||
|
||||
Run `rea describe-desktop-application-task CAPTURE --format json`. Read:
|
||||
|
||||
- `owned_paths`: the only paths whose files the app serves, so the only files
|
||||
a check can read.
|
||||
- `files`: each file there whose state differs between the capture's first
|
||||
and last version, with its `role`, the `baseline` and `final` versions and
|
||||
the net `change`. Text changes list each added and removed line once, and
|
||||
`line_counts` gives lines found at both ends whose count changed.
|
||||
- `values` in each of those files: what the change added, changed and
|
||||
removed, `before` and `after`, at the pointer a check names it by, with
|
||||
`refers_to` naming the element a key such as `category="44"` stands for.
|
||||
Skip entries noted `by_position` or as noise. Only XML files, archives,
|
||||
JSON and key files have values: a text file's or database's change
|
||||
already lists its lines and rows.
|
||||
- `task`: a draft whose `checks` are suggestions from the net change of the
|
||||
documents the user worked on, or of the app's own state when no document
|
||||
changed.
|
||||
- `guards`: optional checks that the rest kept its seed values, such as a
|
||||
like sibling's field or a text file's neighbouring lines; add one or two
|
||||
to show the rest is untouched.
|
||||
- `notes`: why `files` is empty, or why a file's suggestions are thin, such
|
||||
as `by_position` when no attribute names the changed element.
|
||||
|
||||
### Write the prompt
|
||||
|
||||
Say what the user wants done, in their words and the app's terms, naming the
|
||||
app: "In Mousepad, add buy milk to my to-do list and save it". A solving
|
||||
agent's own shell cannot see the computer's files; the app's name sends it to
|
||||
the screen. Never say how the file stores it: no
|
||||
paths the user would not name, line formats, XML, keys or SQL, and never the
|
||||
checks. Replace the draft's placeholder; the export refuses it.
|
||||
|
||||
### Pick the checks
|
||||
|
||||
Checks must express only the observed change:
|
||||
|
||||
- Keep a suggestion only if it says what the user did. Drop those that follow
|
||||
from how the app saves: a format rewrite, renumbered lines, counters. A
|
||||
LibreCAD drawing gaining four holes was rewritten from DXF R12 to 2007, and
|
||||
the draft suggests counts of `SECTION`, `ENDSEC` and `CIRCLE`; only
|
||||
`CIRCLE` equals 9 is the user's change.
|
||||
- Never check unchanged content, whole files, or values the app stamps:
|
||||
times, ids it assigns, window sizes. REA refuses ISO 8601 and `asctime`
|
||||
date-times (`Thu Nov 5 09:20:31 2026`); the
|
||||
rest is your judgment.
|
||||
- Prefer a few robust checks: whole lines over positions, rows by meaningful
|
||||
columns over generated ids, `at_least` when the user may add more.
|
||||
|
||||
Kinds: `file_present` and `file_absent`; `lines_present` and `lines_absent`
|
||||
(`lines`); `line_count` (`exact` or `contains`, and `equals`, `at_least` or
|
||||
`at_most`); `value_equals` and `value_absent` (a JSON `pointer` into the view,
|
||||
such as `/groups/GROUP/KEY` for a key file); `text_equals` and
|
||||
`text_contains` (an XML element's whole text at a `pointer`, optional
|
||||
`normalize_whitespace`); `row_present` and `row_absent` (`table`, `where` by
|
||||
column) and `row_count`. Each names its `file` as a path under the home. A
|
||||
document saved as a zip archive, such as FreeCAD's `.FCStd`, reads as its
|
||||
entries: point into an entry's view, and name array members by selector
|
||||
rather than position, as
|
||||
`/entries/[name=Document.xml]/view/root/children/[name=Object][@name=Hole]`,
|
||||
so the check survives an app version that writes them in another order. A
|
||||
gzip-compressed document, such as Gnumeric's `.gnumeric`, reads as the
|
||||
document inside: point into it as into the same document uncompressed. Give
|
||||
`value_equals` a `tolerance` to compare a number the app writes as text, such
|
||||
as `"5.0000000000000000"`, with a number. Check what the user wrote in a
|
||||
document with `text_equals` on its paragraph element, not `value_equals` on a
|
||||
text node: apps such as AbiWord and Writer split a paragraph into runs as they
|
||||
save, so its first text node holds only part of it; use `text_contains` for a
|
||||
value within a sentence, such as a time. A binary document, such as GIMP's
|
||||
`.xcf`, has no view: give the task a `readers` entry, the app's own headless
|
||||
command printing `REA ` lines, and a check naming it in `reader`, as
|
||||
[desktop builds](desktop-build.md#read-a-binary-file-with-a-reader) shows. The
|
||||
[desktop application tasks](https://github.com/morluto/rea/blob/main/docs/desktop-application-tasks.md)
|
||||
guide has the exact rules.
|
||||
|
||||
### Export the task
|
||||
|
||||
Write the task as JSON with a `name`, the `prompt` and the `checks`, then run
|
||||
`rea export-desktop-application CAPTURE DIRECTORY --task TASK.json`. REA keeps
|
||||
the task only when its checks fail on the seed and all pass on the captured
|
||||
end state. A refusal names the check: one that fails on the end state does not
|
||||
match what the user left; "every check already passes on the seed" means no
|
||||
check captures the change. For several tasks from one capture, repeat
|
||||
`--task` and name each later task's end with `--task-end NAME=CHECKPOINT`.
|
||||
|
||||
When the export refuses an application that no apt repository installs, ask
|
||||
the user for the `.deb` file they installed it from and capture with
|
||||
`--package-file`, then ask where the computer may download that file and pass
|
||||
it as `--package-url`. Never fetch a vendor's download page yourself: such
|
||||
sites often refuse automated downloads, and the copy and its licence are the
|
||||
user's to choose.
|
||||
|
||||
Tell the user the task's name, prompt and checks, and how to measure a solve
|
||||
rate: run `agent-env run DIRECTORY --task NAME --agent claude-code` with
|
||||
`--model MODEL` three times and count the runs reported `passed`. The runs
|
||||
need a model endpoint: `LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
|
||||
`base_url` and `api_key` under `[model]` in `.agentenv/config.toml`. Do not run
|
||||
agents unless asked. The bundle's seed is the user's real data: keep it local.
|
||||
@@ -11,44 +11,6 @@ returned inline. When complete `inspect_binary_layout` or
|
||||
`inspect_analysis_view` for a summary, one section/symbol/module, or a stable
|
||||
page instead of repeating the producer.
|
||||
|
||||
For any logical Evidence source that advertises retained references, pass
|
||||
`{kind: "retained-evidence", evidence_id}` instead of copying the complete
|
||||
record back into a call. The same contract covers individual, optional, array
|
||||
and nested sources across native, process, JavaScript, managed, mobile and WASM
|
||||
workflows. Sources can independently mix inline records and exact references.
|
||||
Resolution preserves the source's identity, provenance, coverage, partial facts
|
||||
and unknowns; it neither reruns analysis nor makes incomplete facts complete.
|
||||
Keep domain role and identity requirements: a native dossier cannot substitute
|
||||
for a managed member observation, for example.
|
||||
|
||||
When starting from a summary or inspection page, use its
|
||||
`normalized_result.parent_evidence_id` to reference the complete source.
|
||||
The view's own `evidence_id` names
|
||||
only that view. `compare_artifacts` explicitly supports artifact summary inputs
|
||||
and resolves their parents; do not assume every consumer does. When a workflow
|
||||
follows source links, keep those linked records available too.
|
||||
|
||||
MCP references belong to the current connection's ledger. If the ID is known,
|
||||
pass it directly; no discovery call or provider setup is needed. If it is missing,
|
||||
import the exported bundle or supply complete inline Evidence. Do not retry an
|
||||
unchanged reference or rerun a producer merely to move an existing record.
|
||||
Export before closing the session when the source must survive it. For CLI JSON
|
||||
workflows that advertise `--evidence-bundle`, the same reference JSON resolves
|
||||
against one explicitly selected authenticated bundle:
|
||||
|
||||
```bash
|
||||
rea trace-application-feature request.json --evidence-bundle /absolute/path/evidence.json --json
|
||||
```
|
||||
|
||||
This includes JavaScript traces/comparisons and runtime reconciliation, managed
|
||||
reconstruction/native verification/graph projection, Android/Apple/Harmony graph
|
||||
projection, WASM interface comparison and retained analysis views. The managed
|
||||
member comparison CLI still takes two assembly paths; process comparison takes
|
||||
two capture Evidence JSON paths. Check command help for the installed version.
|
||||
An input bundle grants no implicit file lookup: `evidence-file` and historical
|
||||
graph file selectors are accepted only by contracts advertising them. See the
|
||||
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/getting-started/cli.md#import-export-and-compare-evidence).
|
||||
|
||||
Completed `inspect_artifact` inventory also supports a summary and
|
||||
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
|
||||
Omit `logical_paths` to page all available occurrences; `[]` selects none.
|
||||
@@ -73,14 +35,6 @@ Parent Evidence ID plus source index preserves same-address
|
||||
alternatives. Follow `coverage.next_offset` only as far as the question needs;
|
||||
exhausting retained rows does not establish complete program discovery.
|
||||
|
||||
`batch_decompile` returns every body unless you select `detail: "summary"` when
|
||||
the connected tool advertises it. Use its parent Evidence ID with
|
||||
`view: {kind: "batch-decompile", collection: "items"}` for body-free rows and
|
||||
exact `where.indices`, `addresses`, `procedure_addresses` or `statuses`. Add
|
||||
`include_pseudocode: true` for several chosen bodies, or use
|
||||
`collection: "pseudocode"` with one row's `index` to page a large body. Batch
|
||||
limitations aggregate every item, so do not attribute them to one successful row.
|
||||
|
||||
Use `record_unknown` to track unresolved questions in the user's investigation
|
||||
and name the authority or environment still required. Supply supporting and
|
||||
contradicting evidence IDs. It updates the session registry directly.
|
||||
@@ -92,22 +46,16 @@ Use comparisons with complete, compatible page sets when claiming equivalence
|
||||
or absence:
|
||||
|
||||
- `compare_artifacts` compares inventory pages by occurrence path, content,
|
||||
metadata, and graph relations. Name each side as
|
||||
`{kind: "retained-evidence", evidence_id}` (an `inspect_artifact` result,
|
||||
its nested inventory or its summary) or
|
||||
`{kind: "evidence-file", path, evidence_id?}` instead of resending Evidence.
|
||||
metadata, and graph relations.
|
||||
- `compare_functions` compares explicit function Evidence, not fuzzy
|
||||
whole-binary matches. It rejects an `analyze_function` summary reply; pass
|
||||
the summary's `parent_evidence_id` as a `retained-evidence` reference or
|
||||
the complete dossier.
|
||||
whole-binary matches.
|
||||
- `compare_bundles` compares canonical bundle membership and unknown history.
|
||||
- `find_changed_behavior` combines existing comparisons; static differences
|
||||
remain candidates, not causal proof. Its `comparisons` accept exact references.
|
||||
remain candidates, not causal proof.
|
||||
- `build_call_path` needs explicit function dossiers from one artifact/provider;
|
||||
pass complete parent IDs in `functions`. An incomplete frontier makes absence
|
||||
unknown.
|
||||
an incomplete frontier makes absence unknown.
|
||||
- `correlate_static_and_runtime` uses explicit mappings; matching patterns are
|
||||
hypotheses, not causality. Both comparison arrays accept exact references.
|
||||
hypotheses, not causality.
|
||||
- `verify_reconstruction` evaluates a finite typed specification. A pass covers
|
||||
only declared comparable claims, not global equivalence.
|
||||
|
||||
@@ -156,21 +104,3 @@ each plan states why in its `reason`. It replays the capture instead with
|
||||
`--replay-only` or when the state is unavailable (not derivable, or a catalog
|
||||
over the discovery budget), and its stopped document's `state` says which
|
||||
applied.
|
||||
Use `view: {kind: "environment-coverage"}` with the same rules to see what an
|
||||
environment built from a capture would serve: read tools answered from the
|
||||
state or replayed with their reason, and each write tool served or withheld
|
||||
with a hint about what to record next; it does not apply the serve-time
|
||||
catalog and state size checks. `rea environment-build <capture.har> <dir>`
|
||||
inspects, projects and exports in one step and prints the same coverage, or
|
||||
`null` when the bundle serves replay.
|
||||
|
||||
For a PE/ELF export question, `inspect_native_exports` accepts `detail: "summary"`
|
||||
and retains the complete table. Query `inspect_analysis_view` with
|
||||
`view: {kind: "native-exports", collection: "exports", names: ["selected_name"],
|
||||
fields: ["name"], offset: 0, limit: 32}` and the reported parent Evidence ID.
|
||||
All same-name versions and aliases survive; use `source_index` with the parent ID
|
||||
for stable identity. `collection: "issues"` pages interpretation failures. Page
|
||||
coverage describes selected retained rows; `summary.coverage` or `exports.coverage`
|
||||
still describes the source table, so an empty partial page does not prove absence.
|
||||
CLI `inspect-native-exports --detail summary --evidence_output PATH` saves the
|
||||
complete bundle for later `inspect-analysis-view --evidence-bundle PATH` requests.
|
||||
@@ -6,7 +6,7 @@ For captured or bundled JavaScript that needs readable source recovery, use
|
||||
`additional_paths` when advertised, then analyze the returned `analysis_input`
|
||||
directory together. Each input retains its own snapshot, digest and byte-range
|
||||
provenance; inputs and recovered code are never executed. Recovery requires a
|
||||
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/guides/javascript/javascript-recovery.md).
|
||||
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
|
||||
|
||||
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
|
||||
extracted tree. When the question concerns known source paths and the connected
|
||||
@@ -17,20 +17,15 @@ the complete artifact inventory and identity but reads source text and analyzes
|
||||
only selected files. The result and summary record `source_selection`; files
|
||||
outside scope and their relationships remain unknown, so partial application
|
||||
coverage does not mean the selected source failed. Use repeated `--source-path`
|
||||
on either static application CLI route, or `--source-paths-json` with a JSON
|
||||
file for selections too long for one command line. Omit the selection when the
|
||||
question requires complete application analysis; `source_paths: []` is invalid,
|
||||
not an unfiltered request. `detail: "summary"` reduces delivery
|
||||
on either static application CLI route. Omit the selection when the question
|
||||
requires complete application analysis; `source_paths: []` is invalid, not an
|
||||
unfiltered request. `detail: "summary"` reduces delivery
|
||||
and does not by itself reduce analysis work.
|
||||
|
||||
ASAR header/index admission defaults to 16 MiB / 16777216 bytes;
|
||||
an admission error names `asar.max_header_bytes` and its bounds (at most
|
||||
256 MiB). Call `update_analysis_settings` with that setting, `action: "set"`
|
||||
and enough bytes, then retry the same full-archive request. No inspection or
|
||||
REA restart is required. `REA_ASAR_MAX_HEADER_BYTES` seeds future REA processes.
|
||||
`source_paths`, `detail`, and `max_heap_mb` do not bypass or raise header admission.
|
||||
Runtime JavaScript timeout/heap defaults are also editable; explicit per-call
|
||||
`analysis_timeout_ms` and `max_heap_mb` take precedence.
|
||||
`REA_ASAR_MAX_HEADER_BYTES` (at most 256 MiB, set where REA starts) changes it.
|
||||
`source_paths`, `detail`, and `max_heap_mb` do not bypass or raise it. Or
|
||||
analyze an extracted directory.
|
||||
|
||||
Exact `#alias` specifiers resolve through the importer's
|
||||
enclosing package `imports` map to package-local files; nested `node_modules`
|
||||
@@ -78,15 +73,6 @@ classification, and unavailable sources never establish route reachability or
|
||||
the complete application route denominator. See the
|
||||
[route inventory guide](https://github.com/morluto/rea/blob/main/docs/javascript-route-inventory.md).
|
||||
|
||||
For captured endpoint observations and inference facts, `analyze_web_bundle`
|
||||
also accepts `view: {kind: "web-bundle", collection: "endpoints", limit: 32}`
|
||||
with `detail: "summary"`. Read later `endpoints`, `inferences`, or `summary`
|
||||
pages from the retained parent. Filter before requesting detail; retain
|
||||
`source_index` together with parent ID and collection, and request `location`
|
||||
or `basis` when tracing sources. Page exhaustion does not mean producer
|
||||
completeness: check `completeness`, structured `source_unknowns`, and limitations.
|
||||
Endpoint candidates remain inferences rather than confirmed network behavior.
|
||||
|
||||
## Read a module without repeating analysis
|
||||
|
||||
Use the actual returned parent ID, on the same connection, as `source.evidence_id`.
|
||||
@@ -169,10 +155,7 @@ leaving it, and `both` when the question needs both. Trace traversal includes
|
||||
containment relationships, so even a narrow seed can expand many nodes; a
|
||||
bidirectional trace can reach unrelated sibling modules. A module item or page
|
||||
is preferable when its observations already answer the question. Do not treat
|
||||
all reached nodes as participants in the feature. For a broad trace, pass
|
||||
`view: {"kind": "feature-trace", "collection": "paths"}` (or `summary`) to
|
||||
retain the complete trace and read terminal paths first; page `nodes` or `edges`
|
||||
with exact `where.node_ids` through `inspect_analysis_view` as needed.
|
||||
all reached nodes as participants in the feature.
|
||||
|
||||
Reuse retained parent Evidence when the server advertises it, or supply complete
|
||||
Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest
|
||||
@@ -239,19 +222,10 @@ This notation is
|
||||
a template: replace it with the actual returned ID. Versions before 4.1.0 accept only full
|
||||
inline Evidence. Use the exact ID
|
||||
returned by the producer on the same MCP connection. Resolution does not run
|
||||
analysis or select a provider; findings remain inline. For summary or inspection
|
||||
replies, select the complete `normalized_result.parent_evidence_id`. Native
|
||||
observation arrays and runtime reconciliation's `static_layers[].analysis` and
|
||||
`runtime_observations[]` also accept exact references where advertised. Each
|
||||
source can independently be inline or retained, preserving identity, coverage
|
||||
and unknowns. CLI workflows advertising `--evidence-bundle` resolve these inputs
|
||||
from the explicitly selected bundle. `close_binary` releases the engine while
|
||||
retaining references and complete Unknown history on the same MCP connection.
|
||||
`reset_investigation` explicitly clears them; optional `save_path` checkpoints the
|
||||
complete bundle before clearing. Import that bundle in another connection to
|
||||
restore its original references. Retention is bounded, in-memory and never silently
|
||||
evicted; `binary_session.investigation` reports the budget. A missing reference
|
||||
includes its ID and recovery guidance.
|
||||
analysis or select a provider; findings remain inline. Native observation
|
||||
arrays still take full Evidence. `close_binary` clears retained references;
|
||||
export a bundle before closing, or supply portable inline Evidence in another
|
||||
connection. A missing reference includes its ID and recovery guidance.
|
||||
|
||||
For exact file/native metadata or processing gaps, use
|
||||
`view: {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"], limit: 32}`.
|
||||
|
||||
@@ -1,35 +1,16 @@
|
||||
# Native, managed, and packaged artifacts
|
||||
|
||||
For release comparisons on one MCP connection, retain `list_procedures` Evidence
|
||||
from each target and reuse its exact IDs after target switches or `close_binary`.
|
||||
For a handoff or new process, export the complete bundle and import it there,
|
||||
including its source closure. `reset_investigation` explicitly clears retained
|
||||
records; optional `save_path` checkpoints them before clearing.
|
||||
`match_native_functions` pairs unique exact names and, when supplied,
|
||||
For release comparisons, export `list_procedures` Evidence from each target before
|
||||
closing it. Close clears session records: import both bundles after collecting the
|
||||
two targets, or supply portable inline Evidence. `match_native_functions` pairs unique exact names and, when supplied,
|
||||
complete observed body bytes. Reuse optional same-context `list_names`,
|
||||
`procedure_info`, `read_bytes` and existing `analyze_function` dossiers as
|
||||
`left_observations`/`right_observations`; an MCP `analyze_function` summary
|
||||
reply contributes its `parent_evidence_id` as a `retained-evidence` reference,
|
||||
not the summary itself. Avoid decompiling every function merely to match it. Its summary retains a parent for
|
||||
`procedure_info`, `read_bytes` and existing `analyze_function` Evidence; avoid
|
||||
decompiling every function merely to match it. Its summary retains a parent for
|
||||
`inspect_analysis_view` `function-matches` collections `matches` and `functions`.
|
||||
Select columns, batch IDs and page individual facts; aliases remain unresolved.
|
||||
Unpaired observations do not prove additions/removals. Existing matched dossier
|
||||
pairs include inputs for `compare_functions`; otherwise analyze just chosen pairs.
|
||||
|
||||
Aggregate completed comparisons with `find_changed_behavior`, selecting each
|
||||
comparison's exact `retained-evidence` ID instead of resending full records.
|
||||
Inline comparison Evidence remains accepted. References belong to the current
|
||||
server: after handoff, import the original bundle including its source closure.
|
||||
Do not substitute dossier IDs or summary views. Static differences remain
|
||||
behavior candidates, with runtime behavior unknown unless separately observed.
|
||||
|
||||
For a matching representation refusal, inspect `details.limit` and raise
|
||||
`native.matching_representation_bytes` with `update_analysis_settings` before
|
||||
retrying the unchanged MCP request. CLI launch seed:
|
||||
`REA_NATIVE_MATCHING_REPRESENTATION_BYTES`. The default is 96 MiB; the policy
|
||||
range is 1 byte to 1 GiB and does not bound process RSS. Keep the complete
|
||||
inventories: splitting them changes global name/body uniqueness.
|
||||
|
||||
## Apple app resources
|
||||
|
||||
Use `open_bundle_resources` with the selected app path, then `inspect_plist`,
|
||||
@@ -67,18 +48,11 @@ See [desktop installations](https://github.com/morluto/rea/blob/main/docs/deskto
|
||||
Use `inspect_go_binary` for an explicit local ELF, PE or Mach-O (universal
|
||||
files need `architecture`) when the
|
||||
question concerns the embedded compiler version, module/dependency versions,
|
||||
local replacements, build settings, or static Go string literals packed without
|
||||
NUL terminators. Pass `include_string_literals: true`
|
||||
(`--include-string-literals`, optional `max_string_literals` /
|
||||
`--max-string-literals`) to recover bounded inferred word-aligned `(ptr, len)`
|
||||
UTF-8 string slices from read-only data and page them through `go`
|
||||
`string_literals` views. No `open_binary`, Go installation or
|
||||
local replacements or build settings. No `open_binary`, Go installation or
|
||||
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
|
||||
Preserve artifact SHA-256, original file ranges, framed module bytes and every
|
||||
unparsed module-text line. A missing record is an unknown, not proof of a
|
||||
non-Go binary; observed metadata does not authenticate its claims, and static
|
||||
`(ptr, len)` string recovery excludes compiler-inlined stack/immediate strings
|
||||
without proving runtime reachability. Function,
|
||||
non-Go binary; observed metadata does not authenticate its claims. Function,
|
||||
source and type recovery remain a separate native-analysis question.
|
||||
|
||||
## Model files
|
||||
@@ -136,58 +110,6 @@ not proof the image is unsigned in every catalog or overlay sense. Signatures,
|
||||
chains, timestamps, and Authenticode digest coverage remain unobserved. See
|
||||
[PE Authenticode certificate tables](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
|
||||
## PE import and delay-load tables
|
||||
|
||||
Use `inspect_pe_imports` for an explicit local PE32/PE32+ image when the
|
||||
question concerns imported DLL and function names, ordinals, delay-load
|
||||
descriptors, or the pefile import hash. No `open_binary`, Windows SDK, or
|
||||
target execution is required. The CLI equivalent is `inspect-pe-imports PATH`.
|
||||
Preserve artifact SHA-256, directory file ranges, original DLL spellings, and
|
||||
ILT/IAT RVAs. A missing `IMAGE_DIRECTORY_ENTRY_IMPORT` directory is observed
|
||||
absence of that table, not proof the image never imported APIs through another
|
||||
table or overlay. Bound snapshots, loaded IAT contents, and export-forward
|
||||
resolution remain unobserved. See
|
||||
[PE import and delay-load tables](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
|
||||
|
||||
## Windows Shell Link shortcuts
|
||||
|
||||
Use `inspect_windows_shortcut` for an explicit local `.lnk` file when the
|
||||
question concerns the recorded target path, arguments, working directory, icon
|
||||
location, volume serial, network share names, ExtraData blocks, or Tracker
|
||||
droid identifiers. No `open_binary`, Windows SDK, COM, or target execution is
|
||||
required. The CLI equivalent is `inspect-windows-shortcut PATH`. Preserve
|
||||
artifact SHA-256, FILETIME raw decimals, IDList type bytes, and ExtraData
|
||||
signatures. Unknown IDList items and ExtraData signatures remain raw with
|
||||
SHA-256; they are not guessed. Version-1 UUID MAC and timestamp values are
|
||||
labelled derived. A missing optional section is observed absence of that
|
||||
structure. See
|
||||
[Windows Shell Link shortcuts](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
|
||||
|
||||
## PE TLS callbacks and load configuration
|
||||
|
||||
Use `inspect_pe_load_config` for an explicit local PE32/PE32+ image when the
|
||||
question concerns code that runs before the entry point through TLS callbacks,
|
||||
or the security cookie, SafeSEH, CFG, EH-continuation and XFG declarations in
|
||||
the load configuration. No `open_binary`, Windows SDK, or target execution is
|
||||
required. The CLI equivalent is `inspect-pe-load-config PATH`. Preserve artifact
|
||||
SHA-256, directory and callback-array file ranges, and callback VAs as
|
||||
hexadecimal strings. Fields outside the declared `Size` are `null`. Guard-flag
|
||||
mitigations carry `evidence_kind: "inferred"`; enforcement remains unobserved.
|
||||
See
|
||||
[PE TLS callbacks and load configuration](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
|
||||
|
||||
## Binary hardening
|
||||
|
||||
Use `inspect_binary_hardening` for an explicit local ELF, PE, or Mach-O file
|
||||
when the question concerns PIE or ASLR, NX, RELRO, stack-protector or FORTIFY
|
||||
signs, control-flow flags, or stripped state. No `open_binary`, pwntools, or
|
||||
target execution is required. The CLI equivalent is
|
||||
`inspect-binary-hardening PATH`. Universal Mach-O files return one image per
|
||||
slice. A missing import is `unknown`, not proof the protection was omitted.
|
||||
Authenticode and Mach-O code-signature rows report presence, not validity.
|
||||
`hardened_runtime` stays unknown and names `inspect_signature`. See
|
||||
[binary hardening](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
|
||||
|
||||
## ELF note tables
|
||||
|
||||
Use `inspect_elf_notes` for an explicit local ELF32/ELF64 image when the
|
||||
@@ -199,88 +121,6 @@ absence of those declarations, not proof the image never carried notes in
|
||||
another container. See
|
||||
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
|
||||
## ELF and Mach-O initializer functions
|
||||
|
||||
Use `inspect_native_initializers` for an explicit local ELF or Mach-O image
|
||||
when the question is which functions run before `main` or at exit: ELF
|
||||
`DT_PREINIT_ARRAY`, `DT_INIT`, `DT_INIT_ARRAY`, `DT_FINI`, `DT_FINI_ARRAY` and
|
||||
legacy `.ctors`/`.dtors`, or Mach-O `__mod_init_func`, `__init_offsets` and
|
||||
`__mod_term_func`. Pass `architecture` for a universal Mach-O. The CLI
|
||||
equivalent is `inspect-native-initializers PATH`. PIE slots are resolved
|
||||
through RELATIVE, RELR and section relocations, and Mach-O slots through
|
||||
chained fixups. Keep each entry's `resolution`: `unresolved` means the value was
|
||||
not established, and `unresolved_import` names an external symbol. Page with
|
||||
`offset` and `limit`. See
|
||||
[ELF and Mach-O initializer functions](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
|
||||
|
||||
## ELF unwind function ranges
|
||||
|
||||
Use `inspect_elf_unwind_ranges` for an explicit local ELF32/ELF64 image when
|
||||
the question concerns function starts and lengths that survive `strip`, which
|
||||
functions have an LSDA, or which ranges have no exact symbol. No
|
||||
`open_binary` or target execution is required. The CLI equivalent is
|
||||
`inspect-elf-unwind-ranges PATH`. Preserve artifact SHA-256, FDE start and
|
||||
end, containing section, exact symbol name, CIE index, LSDA, and personality.
|
||||
Page later ranges with `offset` and `limit`. A missing `.eh_frame` is
|
||||
observed absence of that declaration and of `PT_GNU_EH_FRAME`, not proof the
|
||||
image has no functions. Header disagreement is a limitation and does not drop
|
||||
decoded ranges. `.debug_frame` is reported present or absent and is not
|
||||
decoded. CFA instructions are counted and not evaluated. Indirect EH pointers
|
||||
are not dereferenced. See
|
||||
[ELF unwind function ranges](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
|
||||
|
||||
## Apple IPS crash reports
|
||||
|
||||
Use `inspect_apple_ips` for an explicit local Apple `.ips` file when the
|
||||
question concerns crash metadata, exception and termination facts, threads,
|
||||
usedImages, lossless registers, or documented crash classes. No `open_binary`,
|
||||
Xcode, atos, or target execution is required. The CLI equivalent is
|
||||
`inspect-apple-ips PATH`. Preserve artifact SHA-256, `bug_type` as a string,
|
||||
uppercase image UUIDs, and integer literals that exceed IEEE-754 range. Only
|
||||
`bug_type` `"309"` yields a crash body; other DiagnosticReports types return
|
||||
no crash result. Opened Mach-O and dSYM UUID joins remain unobserved. See
|
||||
[Apple IPS crash reports](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
|
||||
|
||||
## Linux kernel modules
|
||||
|
||||
Use `inspect_kernel_module` for an explicit local ELF32/ELF64 relocatable
|
||||
object when the question concerns `.modinfo` license, vermagic, alias,
|
||||
depends, or parm strings, the `__versions` symbol CRCs, or an appended module
|
||||
signature trailer. No `open_binary`, Linux host, or `modinfo` program is
|
||||
required. The CLI equivalent is `inspect-kernel-module PATH`. Preserve
|
||||
artifact SHA-256, key order, duplicate keys, CRC hex, and section file ranges.
|
||||
A missing section is observed absence when section names are known, not proof
|
||||
the module never carried that metadata. `.gnu.linkonce.this_module` is reported
|
||||
as present or absent; `struct module` stays unknown because its layout changes
|
||||
with the kernel version. `__ksymtab` records are not decoded. The PKCS#7 range
|
||||
and its SHA-256 are observed; certificate, signer, and digest coverage remain
|
||||
unvalidated. An absent trailer is not proof the module is unsigned in another
|
||||
container. gzip, xz, and zstd whole-file containers are unsupported and name
|
||||
`external_decompression` as the next probe: use an external codec utility to
|
||||
decompress into a separate file, then call `inspect_kernel_module` on that file. See
|
||||
[Linux kernel modules](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
|
||||
|
||||
## Linux kernel build config
|
||||
|
||||
Use `inspect_kernel_build_config` for an explicit local kernel image (`vmlinux`
|
||||
ELF, raw arm64 `Image`, x86 `bzImage`/`zImage`, or `configs.ko`) when the
|
||||
question is which kernel version it is or how it was configured. No
|
||||
`open_binary`, Linux host, or `extract-ikconfig` is required. The CLI
|
||||
equivalent is `inspect-kernel-build-config PATH`; pass `name` for one exact
|
||||
`CONFIG_*` option and `offset`/`limit` to page the rest. Preserve the banner
|
||||
version, builder, and compiler, the config SHA-256 and option count, and each
|
||||
entry's value kind (`y`, `m`, `string`, `number`, `not_set`). A config is
|
||||
accepted only when `IKCFG_ST` is immediately followed by a valid gzip member
|
||||
and `IKCFG_ED`, so an ordinary `IKCFG_ST` string is not a config. Offsets with
|
||||
`space: "inflated_payload"` are positions in an inflated bzImage payload, not
|
||||
file offsets. An absent config describes only the searched bytes; the kernel
|
||||
may have been built without `CONFIG_IKCONFIG` or with it as a module. Truncated
|
||||
payloads are incomplete, not absent. A framed blob that does not decode, has a
|
||||
bad CRC, or holds non-config text is `malformed`, not absent. xz, lz4, lzma, zstd, bzip2, and lzo
|
||||
payloads are unsupported and name `external_decompression`: decompress with an
|
||||
external utility, then call `inspect_kernel_build_config` on the result. See
|
||||
[Linux kernel build config](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
|
||||
|
||||
## CPython pyc files
|
||||
|
||||
Use `inspect_python_pyc` for an explicit local CPython 3.7–3.13 `.pyc` when the
|
||||
@@ -298,11 +138,7 @@ question concerns IFF chunks, atom names, exports, imports, lambdas, or the
|
||||
Code header. No `open_binary`, Erlang installation, or bytecode execution is
|
||||
required. The CLI equivalent is `inspect-erlang-beam PATH`. Preserve artifact
|
||||
SHA-256, chunk file ranges, 1-based atom indices, and original atom spellings.
|
||||
Missing optional tables stay null. For a few facts, use `detail: "summary"` with
|
||||
a `beam` view selecting a collection, exact names/indices and fields; read later
|
||||
pages from `parent_evidence_id` or the complete Evidence bundle. Same-name
|
||||
exports retain every arity and original atom index. CLI summary delivery requires
|
||||
`--evidence_output`. Null table counts remain distinct from decoded empty tables. See
|
||||
Missing optional tables stay null. See
|
||||
[Erlang BEAM files](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
|
||||
|
||||
## SQLite snapshots
|
||||
@@ -318,7 +154,7 @@ Choose `row_limit` when a finite sample is useful and keep truncation and
|
||||
unspecified row order explicit. Virtual tables, shadow tables and views are
|
||||
schema facts, not row-read targets. REA inspects private copies and leaves
|
||||
the source files untouched; obtain a coherent quiescent database/WAL snapshot
|
||||
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
|
||||
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
|
||||
## Recorded Linux crashes
|
||||
|
||||
@@ -333,7 +169,7 @@ The optional `include_debugger_context` facet adds mapping candidates through
|
||||
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
|
||||
establish current file identity; zero reported flags leave permissions unknown.
|
||||
Requested unavailable context fails with setup guidance. See
|
||||
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md)
|
||||
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md)
|
||||
for exact upstream profiles, bounds and verification coverage.
|
||||
|
||||
## Offline ELF layout
|
||||
@@ -392,24 +228,9 @@ cross-references. Addresses and recovered pseudocode are analysis observations,
|
||||
not original source. Provider unavailability and unsupported metadata remain
|
||||
unknown rather than false.
|
||||
|
||||
For large pseudocode, request `procedure_pseudo_code` with `detail: "summary"`
|
||||
on the first MCP call. It retains the complete observation and returns the
|
||||
original selection, text availability and length without the body. Use
|
||||
`normalized_result.parent_evidence_id` with `inspect_analysis_view` and
|
||||
`view: {kind: "native", facet: "pseudocode", offset: 0, limit: 20000}`;
|
||||
follow `coverage.next_offset` for later pages. Choose a smaller limit if one
|
||||
page exceeds the client or transport budget. Offsets and limits count UTF-16
|
||||
code units, not lines or tokens. Pages reuse the same observation without
|
||||
repeating decompilation. Null and empty text remain distinct. Omitted or
|
||||
`complete` detail preserves full inline delivery. Summary delivery requires a
|
||||
retaining session and does not reduce the work or memory of decompilation.
|
||||
|
||||
For a completed `analyze_function` dossier, use `inspect_analysis_view` with
|
||||
`view: {kind: "native", facet: "value_flow_operations", offset: 0, limit: 32}`
|
||||
and the exact retained Evidence ID to read selected facts without reanalysis.
|
||||
The default MCP reply is already a summary of that dossier: its
|
||||
`normalized_result.parent_evidence_id` is the retained ID to page, and its
|
||||
null counts mark facets the provider did not record or declared unavailable.
|
||||
Native facets also include procedure, pseudocode, assembly, basic blocks,
|
||||
comments, callers/callees, references, unresolved calls, referenced strings and
|
||||
names, the native API record, and value-flow summaries. Pseudocode offsets and limits count UTF-16
|
||||
@@ -429,84 +250,36 @@ run, use `observe_native_calls` with explicit breakpoints and a bounded
|
||||
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
|
||||
runs and may change files or use the network. Hardened-runtime targets need the
|
||||
`get-task-allow` entitlement.
|
||||
Function breakpoints use exact LLDB runtime names: pass an `inspect_macho` export
|
||||
or import `runtime_name` (`main`), not its symbol-table `name` (`_main`). REA does
|
||||
not translate breakpoint names. Set `module` to the image
|
||||
file name when selecting that image only; otherwise loaded system images also
|
||||
match. Inspect retained locations and module paths before attributing a call.
|
||||
|
||||
Use `binary_session` with no arguments to check the open target, selected
|
||||
provider, and alignment. Its default summary `result.tool_availability` carries
|
||||
the complete tool inventory grouped by current status. Expand each group's
|
||||
`names` and inherit the inventory-level client requirements unless a group
|
||||
overrides them; availability, reasons, remediation, and modes remain inline.
|
||||
Provider candidates keep availability and target-support status, code, and
|
||||
reason. `omitted_fields` names what the summary leaves out; `{"detail":
|
||||
"complete"}` adds each unavailable candidate's `remediation`, provider
|
||||
diagnostics, candidate capability sets, and tool surface, `effects`, and
|
||||
`annotations`. Those arrays name true flags from fixed
|
||||
vocabularies, so an omitted known flag is false. `tools/list` remains the
|
||||
canonical source for full tool schemas when target or provider state changes.
|
||||
provider, and alignment. Its default `result.tool_availability` carries a
|
||||
complete inventory grouped by shared facts. Expand each group's `names` and
|
||||
inherit the inventory-level client requirements unless a group overrides them;
|
||||
availability, reasons, remediation, modes, and flags remain inline. The
|
||||
`effects` and `annotations` arrays name true flags from fixed vocabularies, so an
|
||||
omitted known flag is false. `tools/list` remains the canonical source for full
|
||||
tool schemas when target or provider state changes.
|
||||
|
||||
Provider capability groups carry provider identity and ordered limitations.
|
||||
Prepend each set's `shared_limitations` to a group's `limitations` to recover the
|
||||
operation's full ordered list. Candidate capabilities, returned with complete
|
||||
detail, describe possible providers. Top-level session capabilities describe the current route, and
|
||||
operation's full ordered list. Candidate capabilities describe possible
|
||||
providers. Top-level session capabilities describe the current route, and
|
||||
`analysis_provider_binding` identifies the selected provider.
|
||||
|
||||
## Continue editable Ghidra analysis across processes
|
||||
|
||||
For a task spanning agent sessions, check `binary_session` after selecting
|
||||
Ghidra: the current provider's `capabilities[].shared_limitations` states whether
|
||||
Program checkpoints are enabled or the session is ephemeral. Successful native
|
||||
queries distinguish initial analysis from an actually restored packed Program;
|
||||
do not infer that restoration happened merely because checkpoint mode is enabled.
|
||||
|
||||
In ephemeral mode, choose a private absolute directory and set
|
||||
`REA_GHIDRA_CHECKPOINT_DIR` in the environment starting REA. Restart the MCP server
|
||||
after changing that launch configuration; no current tool call or runtime
|
||||
analysis-setting update enables it. Stock ELF/PE/Mach-O imports on Linux/macOS
|
||||
are supported; see the [checkpoint setup and limitations](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program).
|
||||
|
||||
With the setting enabled, normal opens restore compatible saved Programs
|
||||
automatically. Initial completed analysis is saved, and `close_binary` saves
|
||||
later metadata edits; no separate save/restore tool is needed. Repair a reported
|
||||
save failure and retry close. Cancellation or process failure can lose edits
|
||||
after the last successful save; unfinished initial analysis cannot resume.
|
||||
Keep the same checkpoint configuration for later processes. Evidence/Unknown
|
||||
history and immutable REA query snapshots have separate persistence workflows.
|
||||
|
||||
## Reuse caller-selected Ghidra analysis
|
||||
|
||||
When the caller supplies an expanded project, configure `REA_GHIDRA_PROJECT_PATH`
|
||||
(the `.gpr` marker) and optionally `REA_GHIDRA_PROGRAM_NAME` (one root program),
|
||||
then restart/reconnect before `open_binary`. CLI uses the same configuration.
|
||||
REA opens only a fingerprint-bound owned copy with `-process -noanalysis -readOnly`;
|
||||
original project files are not opened by the engine or saved. Database edits,
|
||||
import overrides and repair scripts are unavailable in this mode. Treat original
|
||||
analysis completeness/settings as unknown and prior edits as caller-project state.
|
||||
The program's recorded import digest must match the target; changed project content
|
||||
requires reopening for a fresh profile. See the
|
||||
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#reuse-an-existing-ghidra-project).
|
||||
|
||||
## Ghidra startup deadline recovery
|
||||
|
||||
`provider_timeout` during the first Ghidra query can cover import, auto-analysis,
|
||||
bridge startup, and readiness. The default 330,000 ms is configurable through
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS`; a smaller query still needs the same cold import.
|
||||
|
||||
When the error names `ghidra.startup_timeout_ms`, call
|
||||
`update_analysis_settings` with `action: "set"` and a larger value within the
|
||||
reported bounds. Resolve retained cleanup before the next startup attempt.
|
||||
No REA restart is needed; an already accepted startup keeps its original
|
||||
budget. Follow the error's session-specific restart guidance, and preserve useful Evidence before closing
|
||||
and reopening the target when remediation requires that lifecycle step.
|
||||
|
||||
For a per-function timeout, update `ghidra.decompile_timeout_seconds` and retry
|
||||
the same function on the existing Program. Explicit `null` removes that
|
||||
per-function deadline. Inspection is optional when the error supplies the
|
||||
setting and bounds. Runtime updates disappear when REA exits; environment
|
||||
changes only seed future REA processes.
|
||||
Preserve useful Evidence before closing the session or restarting REA. Check
|
||||
`close_binary` and resolve reported cleanup failures. Set a larger startup budget
|
||||
in the environment that launches REA, then restart REA and reconnect the MCP
|
||||
client before reopening the same target. Closing and reopening a binary in the
|
||||
existing server keeps its original configuration and starts import again.
|
||||
Environment changes in an agent's shell do not update an already running MCP
|
||||
server. If its launch configuration cannot be changed, report that constraint
|
||||
and the exact retry setting instead of claiming a fixed technical limit.
|
||||
|
||||
The client's tool-call deadline must allow startup plus the requested query;
|
||||
server settings and progress notifications cannot override it. CLI analysis
|
||||
@@ -521,7 +294,7 @@ and memory controls in place. Disable specific expensive analyzers only when
|
||||
their missing coverage is acceptable and retain the resulting limitations.
|
||||
Exported C and embedded resources remain useful independent evidence, but a
|
||||
timeout does not establish complete analysis or absence of code. See the
|
||||
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
|
||||
## Bridge request ceiling
|
||||
|
||||
@@ -552,21 +325,6 @@ encoded in bounded batches.
|
||||
The 25000 ms Cutter request deadline and the 256-candidate discovery budget
|
||||
still have no runtime override.
|
||||
|
||||
## Selected native facts and call graphs
|
||||
|
||||
For a large `get_call_graph`, or repeated `procedure_address`,
|
||||
`procedure_callees`, `procedure_callers` and `resolve_containing_procedure`
|
||||
queries, select `detail: "summary"` when the first 64 recorded observations
|
||||
suffice. Read further `native-observation` pages from the returned parent Evidence
|
||||
ID. Graph message IDs index that page's exact-text dictionary; keep level,
|
||||
source index, failed nodes, and partial/unknown relationship coverage.
|
||||
Read referenced source context through `native-context` facets `overview`,
|
||||
`analysis-profile`, `limitations` and `result-limitations`. Reuse inspected
|
||||
context/profile values only for the same digest; until caveats are read,
|
||||
interpretation beyond the returned facts remains unknown. Complete Evidence
|
||||
remains retained and exportable. Small complete replies can cost fewer tokens
|
||||
than additional context/page calls; count the completed task and discovery.
|
||||
|
||||
## Static memory and original file bytes
|
||||
|
||||
For Ghidra, `read_bytes` reads initialized analysis memory, including loader
|
||||
@@ -594,14 +352,6 @@ loaded bytes, file mappings, relocations and the entry against the immutable
|
||||
snapshot. Keep `mismatch` and `unsupported` explicit. Verified import covers the
|
||||
reported static image, not runtime DOS or PC-98 hardware behavior.
|
||||
|
||||
For a large load-image report, select `detail: "summary"` and query its
|
||||
`parent_evidence_id` with `inspect_analysis_view` kind `native-load-image`.
|
||||
Choose a collection such as `relocations` or `mappings`, select only needed
|
||||
fields, and page with offset/limit. Original indices preserve same-address
|
||||
alternatives. Null collections remain unrecorded or inapplicable; paging does
|
||||
not upgrade unsupported verification. CLI requires `--evidence_output` for
|
||||
summary delivery so complete source Evidence remains accessible.
|
||||
|
||||
For Ghidra decompilation, `procedure_pseudo_code` also accepts `procedure: "file-offset:0x..."`; CLI `rea decompile <path> --file-offset=0x... --provider ghidra` resolves the source byte offset internally. An unresolved 0x-prefixed numeric procedure selector also falls back to a unique file mapping. Existing symbols and analysis addresses win; multiple mappings fail as ambiguous. Preserve the returned file-offset and address locations and raw selection observations. ELF virtual addresses are distinct from file offsets; never assume an arbitrary offset plus image base is an analysis address.
|
||||
|
||||
Complete function body ranges use inclusive ends and can contain gaps. Do not
|
||||
@@ -645,11 +395,6 @@ names), coverage and per-collection counts. Page rows with
|
||||
`view: {kind: "managed-members", collection: "call_edges", offset: 0, limit: 64}`
|
||||
over `types`, `exported_types`, `fields`, `methods`, `member_refs`, `call_edges`,
|
||||
`field_accesses` or `user_strings`.
|
||||
Use exact `names` and `declaring_types` filters before paging supported
|
||||
collections. For a compact method lookup, request `fields: ["name",
|
||||
"declaring_type", "signature"]`; omit `body` to avoid anchors. Identity tokens,
|
||||
original `source_index`, module/MVID and source coverage remain. Check
|
||||
`unresolved_filter_rows` before interpreting an empty match as absence.
|
||||
|
||||
### .NET NativeAOT
|
||||
|
||||
@@ -672,7 +417,7 @@ broader metadata recovery on Linux; it requires an explicitly configured
|
||||
NativeAOT JAR. These are separate routes. The built-in parser also runs on
|
||||
Windows Ghidra P0; read frozen literals from its report, because
|
||||
`search_strings` does not include them. See the
|
||||
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-nativeaot.md)
|
||||
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/ghidra-nativeaot.md)
|
||||
for setup, supported layouts, and host restrictions. Preserve binary identity
|
||||
and distinguish derived metadata from Ghidra database observations. Recovered
|
||||
types and native pseudocode are not original C# source. Windows Ghidra P0 has
|
||||
@@ -701,22 +446,12 @@ every executable in the bundle. It also lists missing or weak loads and earlier
|
||||
provides them.
|
||||
|
||||
Use `extract_artifact` when materialized files from a supported format are needed.
|
||||
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md).
|
||||
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md).
|
||||
It materializes all regular files into a fresh temporary directory chosen by
|
||||
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
|
||||
The result reports the materialized directory; callers do not choose the
|
||||
destination. No separate permission grant is needed.
|
||||
|
||||
For classic iNES cartridges, `open_binary` recognizes the byte signature without
|
||||
an executable architecture. `inspect_artifact` retains header facts and each
|
||||
PRG/CHR file unit's offset, length and SHA-256; its nested inventory's
|
||||
`ines.reconstruction` verifies the actual ordered region bytes against the
|
||||
whole-file digest. Extraction includes the original header, optional trainer
|
||||
and uninterpreted trailing bytes. These file units are not mapper/CPU bank
|
||||
maps. NES 2.0, UNIF, dirty legacy headers and truncated payloads are explicitly
|
||||
refused. No emulator or 6502 analysis is performed. See the
|
||||
[iNES guide](https://github.com/morluto/rea/blob/main/docs/ines-containers.md).
|
||||
|
||||
Native DMG traversal automatically uses a read-only mount on supported macOS
|
||||
hosts, with an owned temporary mount directory and cleanup. It needs no approval
|
||||
flag. Unsupported hosts or mount failures remain explicit limitations; do not
|
||||
@@ -729,7 +464,7 @@ temporary, not a retained export of the image's contents.
|
||||
For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
|
||||
`--target-format dos-com`). Inspect the load image before trusting function
|
||||
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
|
||||
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-dos.md).
|
||||
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
|
||||
|
||||
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
|
||||
|
||||
@@ -743,34 +478,11 @@ otherwise. Big-endian ARM, Thumb-only, and Cortex-M variants are unsupported.
|
||||
`0x0000..0xffff`. 65C02, 65816, cartridge containers, bank switching, and mapper
|
||||
semantics are unsupported. `m68k` requires an even entry and decodes the 68040
|
||||
superset of the 68000 family; Amiga/Atari containers and vector-table entry
|
||||
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/providers/raw-binary.md).
|
||||
|
||||
Intel HEX, S-record, UF2 and DfuSe files are containers, not raw images:
|
||||
`inspect_artifact` lists address regions (UF2 per familyID, DfuSe per element
|
||||
with its alternate setting), and `extract_artifact` writes each region. Open an
|
||||
extracted region as `raw-binary` with its start address as `entry_point`. UF2
|
||||
numbering gaps leave coverage unknown; a DfuSe suffix CRC mismatch follows
|
||||
`integrity_policy`. See [UF2 and DfuSe](https://github.com/morluto/rea/blob/main/docs/uf2-dfuse.md).
|
||||
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
|
||||
|
||||
### Function annotations in Ghidra
|
||||
|
||||
Fresh-import sessions admit annotations; existing-project reuse is read-only.
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits. Normal close saves them when owned Program checkpoint mode is enabled; ephemeral mode discards them. CLI `annotate-native-function` follows the same setting when closing after its updated result. Select the [checkpoint workflow](#continue-editable-ghidra-analysis-across-processes) before edits that must survive a later process. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier. Ghidra rejects batch addresses that request the same name or a name another selected address holds; `raw_result.failures` gives each failure's reason.
|
||||
|
||||
A provider switch or different target can retire the active Ghidra database and discard
|
||||
its edits, even if the new provider fails on a later call. Same-target,
|
||||
same-profile reopen keeps the active database. Retained annotation Evidence
|
||||
records what happened; it does not restore the edits.
|
||||
|
||||
For `compare_functions`, obtain `analyze_function` Evidence before and after the
|
||||
edit while the MCP session is open; annotation Evidence is not a comparison
|
||||
input. Export the needed Evidence before close if comparison will happen later.
|
||||
Authenticated same-artifact coordinates preserve renamed entries as
|
||||
`matched / same-entry`, with the canonical `entry_address` inline; name and
|
||||
exact-text changes remain separate dimensions. This requires compatible
|
||||
architecture, provider-owned mapping and selected-image commitments. Equal
|
||||
numeric addresses alone cannot establish correspondence; inspect retained
|
||||
source Evidence and coverage when the match remains ambiguous.
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier.
|
||||
|
||||
## Offline PE resource inventory
|
||||
|
||||
@@ -782,7 +494,7 @@ code pages, payload hashes and static icon-group references. Same-language icon
|
||||
resolution is explicit; missing references and size mismatch remain visible.
|
||||
Other payloads are opaque, and this operation does not decode images or simulate
|
||||
Windows resource loading. Consult the budgets and supported tree profile in
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
|
||||
## Offline PE and ELF export tables
|
||||
|
||||
@@ -792,15 +504,13 @@ directory or ELF dynamic symbol table on any host without opening a native
|
||||
target. Ordinal-only, forwarded and hidden-version exports stay distinct;
|
||||
partial or unsupported coverage means the inventory is incomplete, not that
|
||||
exports are absent. Use `inspect_macho` for Mach-O. See
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
|
||||
For exact core-module import/export differences, use `compare_wasm_interfaces`
|
||||
with two complete inspection Evidence records (or current-session retained
|
||||
references) and explicit scope. Preserve unknown types/duplicate names; static
|
||||
interface equality is not runtime compatibility. CLI accepts complete inline
|
||||
Evidence in `rea compare-wasm-interfaces input.json --json`, or exact references
|
||||
with `--evidence-bundle /absolute/path/evidence.json`. Use the complete parent
|
||||
ID when the inspection returned a summary.
|
||||
Evidence in `rea compare-wasm-interfaces input.json --json`.
|
||||
|
||||
## Godot PCK
|
||||
|
||||
@@ -820,112 +530,3 @@ page cannot prove absence in unknown coverage. Retained views are historical,
|
||||
not fresh filesystem checks. No decryption, Godot resource decoding, GDScript
|
||||
decompilation or target execution is provided. See
|
||||
[Godot PCK](https://github.com/morluto/rea/blob/main/docs/godot-pck.md).
|
||||
|
||||
UnityFS AssetBundle containers (`.bundle`, `.ab`, YooAsset packages and other
|
||||
byte-identified files) list `CAB-*` SerializedFiles and `.resS` members with
|
||||
header and node facts; none/LZ4/LZ4HC member blocks stream on demand. Bundles
|
||||
declaring asset-bundle encryption keep header facts with unknown directory
|
||||
coverage; LZMA member bytes retain named occurrences with unavailable hashes.
|
||||
Virtual member offsets are not physical carrier locations. Format 6–8 readers
|
||||
cache one decoded block and support retained filtered/paged views. No
|
||||
decryption or SerializedFile object decoding is provided. See
|
||||
[UnityFS](https://github.com/morluto/rea/blob/main/docs/unityfs.md).
|
||||
|
||||
WeChat mini-program packages (`.wxapkg` files with `0xBE`/`0xED` marks) list
|
||||
members with declared names and absolute carrier offsets; member bytes are
|
||||
stored verbatim. `V1MMWX` PC-client packages report `encrypted` directory
|
||||
status with unknown coverage and are refused on extraction; no decryption is
|
||||
attempted. See
|
||||
[wxapkg](https://github.com/morluto/rea/blob/main/docs/wxapkg.md).
|
||||
|
||||
U-Boot legacy uImages use the same inspection/extraction workflows. Prefer a
|
||||
retained summary and selected inventory views for header facts or component
|
||||
IDs. Header addresses/CPU/OS are declarations; payloads remain as stored.
|
||||
Multi-file components retain their aligned physical ranges; the full payload
|
||||
is also materialized. Unknown name text retains its original 32 bytes. See
|
||||
[U-Boot uImage](https://github.com/morluto/rea/blob/main/docs/uimage.md).
|
||||
|
||||
## ESP images
|
||||
|
||||
For an ESP32-family app or bootloader image, `open_binary` then
|
||||
`inspect_artifact` with `detail: "summary"`. `root_format_metadata` holds the
|
||||
observed `header` and `descriptor` (its `kind` is the app or bootloader role)
|
||||
and the derived `chip`, `checksum`, `appended_sha256` and
|
||||
`undescribed_trailing` ranges. Segment rows (`segment-NN.bin`) carry
|
||||
`load_addr`, `classification` (`mapped`, `loaded`, `not-loaded` or `unknown`,
|
||||
per ESP-IDF `should_map`/`should_load`; RAM ranges are not checked) and
|
||||
`source_location`; select them with `inspect_analysis_view` and
|
||||
`logical_paths`. Signature blocks are `unverified`; `public_key_sha256`
|
||||
matches eFuse key digests.
|
||||
|
||||
Unknown `source_coverage` means truncation or undescribed trailing bytes, such
|
||||
as the flash segments of a `--ram-only-header` image; do not infer absence
|
||||
from it. Erased flash is described. Integrity mismatches are facts, not
|
||||
refusals. ESP8266 images are refused, and flash-encrypted images are not
|
||||
recognized. Segments are not disassembled, and RISC-V or Xtensa segments
|
||||
cannot be opened as raw-binary. See
|
||||
[ESP images](https://github.com/morluto/rea/blob/main/docs/esp-image.md).
|
||||
|
||||
## ESP flash images
|
||||
|
||||
A whole-flash dump opens as `esp-flash-image`. `root_format_metadata` holds
|
||||
`partition_table` (its `method`: `default`, `caller` or a derived
|
||||
`scan`), `bootloader` and `apps` ESP image facts, the `otadata` records,
|
||||
`partitions_outside_dump` and `boot_selection`. Every table has a
|
||||
`boot_selection`: its `basis` names the bootloader's branch (`no-otadata`
|
||||
without otadata, `none` when the bootloader rejects the table, `unknown`
|
||||
when a record is missing), and it lists its `assumptions`;
|
||||
`running_partition` is always `unknown`. Each image names its entry
|
||||
`path`. With `detail: "summary"` the root kind is `esp-flash-image-summary`
|
||||
and each image is a compact `summary` (chip, entry point, segment count,
|
||||
classified load ranges, signature block count, descriptor identity,
|
||||
integrity); complete detail keeps every segment. Partition rows
|
||||
(`partitions/<label>.bin`) carry the declared `type`, `subtype`, names,
|
||||
`flash_address`, `size` and flags; `encrypted` is a storage flag, not
|
||||
proof of ciphertext. `written_offset`/`written_end` bound the bytes that
|
||||
are not erased: an app slot whose facts say `erased` can still hold an older
|
||||
image, and a limitation says so. For a dump read from another flash address, call
|
||||
`open_binary` with `format: "esp-flash-image"` and `flash_offset`; an
|
||||
`esp-image` whose `flash_dump` is set names that offset in a next probe; a
|
||||
relocated table can be named with `partition_table_offset`. No table found
|
||||
means unknown, often flash encryption. Extraction writes `nvs_keys` as
|
||||
ordinary data. See
|
||||
[ESP flash images](https://github.com/morluto/rea/blob/main/docs/esp-flash-image.md).
|
||||
|
||||
## Bun standalone executables
|
||||
|
||||
A `bun build --compile` executable stores its module graph in Mach-O
|
||||
`__BUN,__bun`, a PE or ELF `.bun` section, or an appended trailer.
|
||||
`open_binary` then `inspect_artifact` reports root format `bun-standalone` with
|
||||
the carrier, structural layout, module count, entry point, raw flags and
|
||||
compile exec-argv; each module keeps its `/$bunfs/...` source name, loader,
|
||||
encoding, sizes and SHA-256, with sourcemap, bytecode and module-info regions as
|
||||
separate children. Use summary root metadata `entry_point_logical_path` as the
|
||||
exact inventory `logical_paths` selector, then read the selected content node by
|
||||
its artifact ID; the reader owns virtual-name normalization and indexed fallback. `extract-artifact` writes stored contents through the safe
|
||||
output tree; route extracted JavaScript to `analyze_javascript_application`.
|
||||
Layout labels describe record shapes, not a detected compiler version; unknown
|
||||
or ambiguous layouts are unsupported. Bytecode is retained, not decoded, and the
|
||||
executable is never run. See
|
||||
[Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
|
||||
## Deno compile executables
|
||||
|
||||
A `deno compile` executable stores its application in a Mach-O `__SUI,d3n0l4nd`
|
||||
section, an Intel Mach-O `<~sui-data~>` trailer, an ELF `SUI` note of type
|
||||
`0x53554901`, or a PE `RT_RCDATA` resource named `D3N0L4ND`.
|
||||
`open_binary` then `inspect_artifact` reports root format `deno-standalone`
|
||||
with the carrier, entry point, recorded options, and one occurrence per virtual
|
||||
file, including size and SHA-256. Transpiled output, CJS export analysis, and
|
||||
source maps are separate occurrences with the file's `index` and their own
|
||||
`role`. They normally use a `.deno-*` suffix on the file path and move to an
|
||||
indexed path on a collision, so use the reported `logical_path`. Symlink targets
|
||||
stay recorded path parts. A virtual filesystem that cannot be listed leaves
|
||||
coverage unknown.
|
||||
Environment-file values and CA certificate bytes are retained in local Evidence
|
||||
with their counts and byte lengths. The npm snapshot length is reported and not
|
||||
decoded. `extract-artifact` writes
|
||||
stored files through the safe output tree; route extracted JavaScript to
|
||||
`analyze_javascript_application`. Other layouts are unsupported, and the
|
||||
executable is never run. See
|
||||
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
|
||||
@@ -1,58 +1,4 @@
|
||||
# Runtime observation
|
||||
|
||||
## Native runtime questions
|
||||
|
||||
Start from the requested observation. State the selected target, input, expected
|
||||
result, failure signal and stopping condition before a run. Reuse static Evidence
|
||||
to choose a function or hypothesis; `trace_native_values` follows recovered
|
||||
static value dependencies and does not measure runtime values. Prefer static
|
||||
inspection or existing capture comparison when it already answers the question.
|
||||
|
||||
| Question | Existing workflow | Host and effects |
|
||||
| -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Which selected native entry is reached, with which integer argument registers? | `observe_native_calls` on the active Mach-O, with exact function/Objective-C breakpoints and a duration/event bound | macOS with LLDB and required debugger access. Launches a new owned process, stops at entries, records and continues; target execution may write files, show UI or use the network. It never attaches to an existing process. Inspect termination and partial coverage after the run. |
|
||||
| What does caller-authored instrumentation observe? | `list_frida_devices` / `list_frida_processes` when discovery is needed; `start_frida_session`, `load_frida_script`, then status/unload/close | Requires compatible installed Frida bindings and selected device access; the real REA verifier covers Linux. Attach selects an existing PID; spawn selects a program and starts it paused until `resume_frida_session`. Scripts execute in the target and can change memory, files, network activity and behavior. This is not passive inspection. |
|
||||
| Which debugger state or controlled execution explains the behavior? | `start_gdb_session` and `gdb_console`, or `start_rizin_debug_session` and `rizin_debug_command` | Persistent debugger workflows are Linux-only. GDB starts a debugger; console commands choose/load/attach/run targets. Rizin debug start launches the selected target and requires a durable `inferior_tty`. Both consoles are unrestricted and can change target memory/registers, run shell commands, write files and use the network. |
|
||||
| How do terminal output, process lifetime and selected files differ between runs? | `capture_process_scenario`, then `compare_process_captures` with compatible recorded inputs | New captures require Linux or macOS and a working native PTY backend; macOS also needs `xcrun`/Swift for ownership inspection. Capture launches an owned command with ordinary user permissions. Observation paths select snapshots, not a filesystem sandbox. Comparing retained captures does not execute targets and is also available on Windows. |
|
||||
| What does an existing desktop window show, or what changes after a declared UI action? | `observe_native_ui`; use `capture_native_ui_scenario` only for requested actions | macOS 14+ with required Screen Recording/Accessibility access. Select an existing PID/window. Observation does not launch or own the app; scenario actions can change its data or trigger network activity, and leave its state as-is. |
|
||||
|
||||
Close each persistent session with its matching tool. `close_frida_session`
|
||||
unloads scripts and detaches; it does not kill the target. A still-paused spawn
|
||||
is resumed only after instrumentation cleanup is confirmed. GDB and Rizin close
|
||||
can detach a live inferior and let it continue only after verifying its durable
|
||||
output terminal. When disposition or cleanup is unresolved, preserve the owned
|
||||
session and reported recovery; do not assume that closing the debugger terminated
|
||||
the target. Process capture and LLDB call observation instead own their launched
|
||||
processes and report their termination/cleanup outcome. Cancellation is not proof
|
||||
that cleanup completed.
|
||||
|
||||
Unrestricted GDB/Rizin commands and relevant start/close operations retire an
|
||||
active binary-analysis session in the same MCP server. Close an active Ghidra
|
||||
session explicitly first; after target effects settle, reopen the artifact before
|
||||
new static analysis. A Cutter bridge command is likewise unrestricted and may
|
||||
affect its GUI/target; it is not a passive native observation substitute.
|
||||
|
||||
To test a static hypothesis across two builds, retain compatible static
|
||||
comparison Evidence and `compare_process_captures` Evidence. Supply explicit
|
||||
finding/dimension mappings to `correlate_static_and_runtime`. It does not ingest
|
||||
arbitrary Frida, LLDB or debugger traces, auto-match names, or prove causality
|
||||
from cochange. Preserve those traces as their own Evidence and leave unobserved
|
||||
behavior unknown. Record truncation, unmatched breakpoints, changed target
|
||||
identity and cleanup uncertainty as limitations, rather than treating no retained
|
||||
event as proof that the behavior never occurs.
|
||||
|
||||
For same-session MCP comparisons, pass each capture inline or reuse it as
|
||||
`{"kind":"retained-evidence","evidence_id":"ev_..."}`. Default comparison
|
||||
keeps capture-local timing as provenance without treating scheduler or observer
|
||||
latency as a behavior change; use an exact `trace_spec` event when timing is part
|
||||
of the analyst question.
|
||||
|
||||
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/guides/runtime/frida-instrumentation.md),
|
||||
[GDB/Rizin/Cutter](https://github.com/morluto/rea/blob/main/docs/reverse-engineering-tools.md),
|
||||
[process capture](https://github.com/morluto/rea/blob/main/docs/guides/runtime/process-capture.md),
|
||||
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/guides/native/native-investigation.md#native-desktop-observation).
|
||||
|
||||
## Browser and Electron
|
||||
# Browser and Electron runtime observation
|
||||
|
||||
Supply the existing browser or Electron instance's literal-port loopback CDP
|
||||
endpoint in the request. No separate permission configuration or per-call
|
||||
@@ -92,8 +38,6 @@ Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
|
||||
browser and profile. Check `browser.cleanup` for the outcome.
|
||||
Request only the DOM, accessibility, or network details needed for the question.
|
||||
The default final capture contains the URL; step outcomes are reported separately.
|
||||
For multi-step flows with trees or events, select `detail: "summary"` and page
|
||||
exact steps, transactions, text slices or events with `inspect_analysis_view` kind `browser-scenario`.
|
||||
Add a wait action for the expected page state before capturing it. Report that
|
||||
REA performed the declared actions.
|
||||
|
||||
@@ -136,5 +80,5 @@ Inspect native `.saz` snapshots with `inspect_web_network_capture` and
|
||||
traffic analysis, connect Everywhere's separate MCP server and use its
|
||||
advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or
|
||||
`AgentCalls`). Preserve existing sessions and filters when inspecting a
|
||||
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md)
|
||||
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md)
|
||||
for the workflow and Telerik's public setup and traffic-debugging skills.
|
||||
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
|
||||
"name": "rea",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"owner": {
|
||||
"name": "morluto",
|
||||
"url": "https://github.com/morluto"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "rea",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"source": {
|
||||
"source": "npm",
|
||||
"package": "rea-agents",
|
||||
"version": "6.8.0"
|
||||
},
|
||||
"category": "development"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
{
|
||||
"name": "rea",
|
||||
"version": "6.8.0",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"author": {
|
||||
"name": "morluto",
|
||||
"url": "https://github.com/morluto"
|
||||
},
|
||||
"homepage": "https://rea.tools/",
|
||||
"repository": "https://github.com/morluto/rea",
|
||||
"license": "MIT",
|
||||
"keywords": [
|
||||
"reverse-engineering",
|
||||
"binary-analysis",
|
||||
"decompiler",
|
||||
"mcp-server"
|
||||
],
|
||||
"skills": "./skills",
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "node",
|
||||
"args": [
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/plugin-mcp-launcher.mjs",
|
||||
"6.8.0"
|
||||
]
|
||||
}
|
||||
},
|
||||
"icon": "./assets/rea-plugin.png"
|
||||
}
|
||||
@@ -1,13 +0,0 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "node",
|
||||
"args": [
|
||||
"./scripts/plugin-mcp-launcher.mjs",
|
||||
"6.8.0"
|
||||
],
|
||||
"cwd": ".",
|
||||
"startup_timeout_sec": 30
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,40 +0,0 @@
|
||||
{
|
||||
"name": "rea",
|
||||
"version": "6.8.0",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"author": {
|
||||
"name": "morluto",
|
||||
"url": "https://github.com/morluto"
|
||||
},
|
||||
"homepage": "https://rea.tools/",
|
||||
"repository": "https://github.com/morluto/rea",
|
||||
"license": "MIT",
|
||||
"keywords": [
|
||||
"reverse-engineering",
|
||||
"binary-analysis",
|
||||
"decompiler",
|
||||
"mcp-server"
|
||||
],
|
||||
"skills": "./skills",
|
||||
"mcpServers": "./.codex-plugin/mcp.json",
|
||||
"interface": {
|
||||
"displayName": "REA",
|
||||
"shortDescription": "Reverse engineer anything",
|
||||
"longDescription": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"developerName": "morluto",
|
||||
"category": "Developer Tools",
|
||||
"capabilities": [
|
||||
"Interactive",
|
||||
"Read",
|
||||
"Write"
|
||||
],
|
||||
"websiteURL": "https://rea.tools/",
|
||||
"defaultPrompt": [
|
||||
"Use REA to explain how this binary implements a feature.",
|
||||
"Use REA to compare two versions of this application.",
|
||||
"Use REA to recover the sources from this JavaScript bundle."
|
||||
],
|
||||
"composerIcon": "./assets/rea-plugin.png",
|
||||
"logo": "./assets/rea-plugin.png"
|
||||
}
|
||||
}
|
||||
@@ -1,12 +0,0 @@
|
||||
# Review changes to security-sensitive automation and dependency inputs.
|
||||
/.github/CODEOWNERS @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/dependabot.yml @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/workflows/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/.github/actions/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/verify-tools/skills/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/verify-tools/skills/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/scripts/fixtures/http3-lab-requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/website/requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
/third_party/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
|
||||
@@ -27,7 +27,7 @@ body:
|
||||
attributes:
|
||||
label: Documentation location
|
||||
description: Link the page, section, command, API, skill, or inline comment.
|
||||
placeholder: README.md#quick-start or docs/reference/mcp-contracts.md
|
||||
placeholder: README.md#quick-start or docs/mcp-contracts.md
|
||||
validations:
|
||||
required: true
|
||||
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
name: REA production code
|
||||
|
||||
# Match the existing default-setup scan's local-input threat model.
|
||||
threat-models: [local]
|
||||
|
||||
# Retain root traversal, including future production roots. Hidden directories
|
||||
# need their own root; use a directory so Python can share this configuration.
|
||||
paths:
|
||||
- .
|
||||
- docs/.vitepress
|
||||
|
||||
# Keep the default query suite and all production roots, including bridge/,
|
||||
# install.sh, runtime scripts, release controllers, and GitHub Actions.
|
||||
# Exclude test inputs at extraction time so they cannot taint production callers.
|
||||
paths-ignore:
|
||||
- tests
|
||||
- scripts/fixtures
|
||||
- "**/*.test.*"
|
||||
- "**/*.fixture.*"
|
||||
- "**/test_*.py"
|
||||
- "**/*_test.py"
|
||||
@@ -1,33 +0,0 @@
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: npm
|
||||
directories:
|
||||
- /
|
||||
- /scripts/verify-tools/skills
|
||||
schedule:
|
||||
interval: weekly
|
||||
cooldown:
|
||||
default-days: 7
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
npm-minor-and-patch:
|
||||
update-types:
|
||||
- minor
|
||||
- patch
|
||||
|
||||
- package-ecosystem: pip
|
||||
directory: /website
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
|
||||
- package-ecosystem: github-actions
|
||||
directory: /
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
actions-minor-and-patch:
|
||||
update-types:
|
||||
- minor
|
||||
- patch
|
||||
@@ -36,9 +36,6 @@ unchecked. Maintainers can help resolve open questions.
|
||||
Use "none" or "not applicable" only when established. -->
|
||||
|
||||
- Semantic owner and earliest changed stage:
|
||||
<!-- Name the producer of parsed facts or resources and its affected consumers.
|
||||
Summarize superseded paths removed and any distinct policy, lifecycle,
|
||||
lazy-loading, or compatibility responsibility retained. -->
|
||||
- Contribution scope fit and linked maintainer decision:
|
||||
- CLI and MCP/tool-catalog contract:
|
||||
- Provider, bridge, target-format, or platform compatibility:
|
||||
@@ -71,10 +68,6 @@ source, link, or rendered-document checks. -->
|
||||
- User-visible CLI/MCP output (if applicable):
|
||||
- Remaining proof gaps:
|
||||
|
||||
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#boundary-regressions)
|
||||
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#lifecycle-regressions)
|
||||
covered by the evidence above, and explain relevant gaps.
|
||||
|
||||
For evidence-bearing changes:
|
||||
|
||||
- [ ] Observed, derived, and inferred claims remain distinguishable.
|
||||
@@ -99,15 +92,14 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
|
||||
its new behavior and migration, and why compatibility/deprecation cannot
|
||||
preserve it. Every release increments minor, including breaking changes;
|
||||
keep breaking markers and migration notes so callers can assess compatibility.
|
||||
See docs/development/releasing.md. -->
|
||||
See docs/releasing.md. -->
|
||||
- Real Hopper/Ghidra, browser, or OS coverage:
|
||||
- Package or release metadata impact:
|
||||
- Security, privacy, process, or containment review:
|
||||
|
||||
## Author readiness
|
||||
|
||||
<!-- The human owner completes these declarations before requesting review,
|
||||
unless the PR author is exempt as listed in CONTRIBUTING.md.
|
||||
<!-- The human owner completes these declarations before requesting review.
|
||||
The PR author is the owner by default. A different owner must explicitly accept
|
||||
ownership on this PR.
|
||||
AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA.
|
||||
|
||||
+21
-99
@@ -61,9 +61,6 @@ jobs:
|
||||
qualification_run_id: ${{ steps.qualification.outputs.run_id }}
|
||||
code: ${{ steps.plan.outputs.code }}
|
||||
reverse_tools: ${{ steps.plan.outputs.reverse_tools }}
|
||||
python: ${{ steps.plan.outputs.python }}
|
||||
packet_capture: ${{ steps.plan.outputs.packet_capture }}
|
||||
signal_recording: ${{ steps.plan.outputs.signal_recording }}
|
||||
sqlite: ${{ steps.plan.outputs.sqlite }}
|
||||
runtime: ${{ steps.plan.outputs.runtime }}
|
||||
package: ${{ steps.plan.outputs.package }}
|
||||
@@ -147,9 +144,6 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Validate source architecture before dependency installation
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
run: node scripts/verify/architecture-guards.mjs
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
@@ -270,27 +264,6 @@ jobs:
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
python:
|
||||
name: Python producer tests
|
||||
needs: changes
|
||||
if: needs.changes.outputs.python == 'true' && needs.changes.outputs.tier == 'fast'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
# Analysis-efficiency checks import compiled process modules.
|
||||
- name: Reuse portable build outputs across runs
|
||||
uses: ./.github/actions/portable-build-cache
|
||||
- run: npm run test:prepare
|
||||
- run: npm run test:python
|
||||
|
||||
test-source:
|
||||
name: Source tests
|
||||
needs: changes
|
||||
@@ -418,9 +391,7 @@ jobs:
|
||||
needs.changes.outputs.web_network == 'true' ||
|
||||
needs.changes.outputs.web_runtime == 'true' ||
|
||||
needs.changes.outputs.web_source_map == 'true' ||
|
||||
needs.changes.outputs.agentenv == 'true' ||
|
||||
needs.changes.outputs.packet_capture == 'true' ||
|
||||
needs.changes.outputs.signal_recording == 'true'
|
||||
needs.changes.outputs.agentenv == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
@@ -544,7 +515,7 @@ jobs:
|
||||
--coverage.reporter=text
|
||||
--coverage.reporter=json-summary
|
||||
--coverage.reporter=lcov
|
||||
--reporter=dot
|
||||
--reporter=default
|
||||
--reporter=junit
|
||||
--reporter=json
|
||||
--reporter=./scripts/ci/test-timing-reporter.mjs
|
||||
@@ -559,7 +530,7 @@ jobs:
|
||||
needs.test-source.result == 'success' || needs.test-source.result == 'failure')
|
||||
run: >-
|
||||
npm exec -- vitest --merge-reports
|
||||
--reporter=dot
|
||||
--reporter=default
|
||||
--reporter=junit
|
||||
--reporter=json
|
||||
--reporter=./scripts/ci/test-timing-reporter.mjs
|
||||
@@ -779,23 +750,9 @@ jobs:
|
||||
npm run verify:legacy-simulator
|
||||
npm run verify:apple-dispatch
|
||||
npm run verify:interface-builder
|
||||
- name: Verify portable iNES CLI, MCP and source boundaries
|
||||
run: >-
|
||||
npm run test:focused --
|
||||
tests/acceptance/inesContainer.test.ts
|
||||
tests/boundary/filesystem/inesArtifact.test.ts
|
||||
tests/boundary/filesystem/inesCancellation.test.ts
|
||||
- name: Run native Apple and process boundary regressions
|
||||
run: >-
|
||||
npm run test:focused --
|
||||
tests/boundary/providers/hopper/hopperMacLaunchArguments.test.ts
|
||||
tests/acceptance/providers/hopperStartupRecovery.test.ts
|
||||
tests/boundary/providers/hopper/hopperClient.helperFailure.test.ts
|
||||
tests/boundary/providers/hopper/bridgeBootstrapCleanup.test.ts
|
||||
tests/boundary/providers/hopper/bridgeTargetLease.integration.test.ts
|
||||
tests/acceptance/artifactDmgAttachmentRecoveryCli.test.ts
|
||||
tests/boundary/providers/native/nativeDmgArtifactReader.test.ts
|
||||
tests/process-global/dmgCancellationReason.test.ts
|
||||
tests/boundary/filesystem/installScript.test.ts
|
||||
tests/boundary/filesystem/nibNativeHierarchy.test.ts
|
||||
tests/boundary/filesystem/interfaceBuilderBundle.test.ts
|
||||
@@ -805,13 +762,9 @@ jobs:
|
||||
tests/boundary/cli/nativeDemangleCli.test.ts
|
||||
tests/boundary/cli/nativeDispatchPropertyAttributes.test.ts
|
||||
tests/boundary/mcp/nativeDemangleMcp.test.ts
|
||||
tests/boundary/mcp/nativeMachoSummaryMcp.test.ts
|
||||
tests/boundary/providers/native/nativeSignatureTargetBinding.test.ts
|
||||
tests/boundary/providers/native/nativeUiOffscreenWindow.test.ts
|
||||
tests/boundary/process/nativeUiVerifierLane.test.ts
|
||||
tests/boundary/cli/nativeSignatureCli.test.ts
|
||||
tests/boundary/cli/nativeMachoArm64_32Cli.test.ts
|
||||
tests/boundary/cli/nativeMachoEncryptionCli.test.ts
|
||||
tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts
|
||||
src/process/capture/FilesystemSnapshot.test.ts
|
||||
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
|
||||
@@ -836,7 +789,7 @@ jobs:
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- name: Verify static CLI/MCP journeys and synthetic report conformance
|
||||
- name: Run the public CLI/MCP reconstruction journey
|
||||
env:
|
||||
REA_READINESS_REPORT_PATH: readiness-report.json
|
||||
run: |
|
||||
@@ -899,32 +852,12 @@ jobs:
|
||||
# The matching native artifact establishes real Job Object and private
|
||||
# runtime boundaries for these fixtures. Ghidra and Java are not needed;
|
||||
# real engine claims remain in the separate controlled-provider workflow.
|
||||
# Run native proofs before the broad curated suite so an unrelated
|
||||
# fixture failure cannot hide the real control and recovery checks.
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-process-boundary-fixture
|
||||
path: .cache/windows-process-boundary/
|
||||
# Passing the fixture adds the breakaway, caller/child token, environment
|
||||
# semantics and ancestor reparse observations, which the lane skips
|
||||
# entirely when it is absent.
|
||||
- run: >-
|
||||
npm run verify:windows-native -- .
|
||||
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-native-inspection-fixture
|
||||
path: .cache/windows-native-inspection-fixture/
|
||||
- name: Verify native inspection failure ownership and recovery
|
||||
run: node scripts/verify-windows-native-inspection.mjs
|
||||
- name: Run curated Windows capability tests
|
||||
run: >-
|
||||
npx vitest run --coverage.enabled=false
|
||||
src/process/capture/FilesystemSnapshot.test.ts
|
||||
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
|
||||
tests/boundary/filesystem/typescriptCompiler.test.ts
|
||||
tests/boundary/filesystem/releaseProposal.test.ts
|
||||
tests/acceptance/byteEntropyWindowsIdentity.test.ts
|
||||
tests/boundary/filesystem/binaryTarget.test.ts
|
||||
src/contracts/sessionPathInputs.test.ts
|
||||
tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts
|
||||
@@ -934,7 +867,6 @@ jobs:
|
||||
tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts
|
||||
tests/boundary/filesystem/setupUnavailableProvider.test.ts
|
||||
tests/boundary/filesystem/windowsNpxRegistration.test.ts
|
||||
tests/boundary/filesystem/claudeDesktopSkillUninstallWindows.test.ts
|
||||
tests/boundary/filesystem/clientConfigurationEnvironment.test.ts
|
||||
tests/boundary/filesystem/clientConfigurationLayeredEnvironment.test.ts
|
||||
tests/acceptance/setup/registrationRuntimeEnvironmentCli.test.ts
|
||||
@@ -946,7 +878,6 @@ jobs:
|
||||
tests/boundary/process/packageNpmInvocation.test.ts
|
||||
tests/boundary/process/packedBridgeVerification.test.ts
|
||||
tests/boundary/process/windowsOutputCapture.test.ts
|
||||
tests/boundary/process/windowsExecutableSelection.test.ts
|
||||
tests/acceptance/webModuleCli.test.ts
|
||||
tests/acceptance/webSourceLocationCli.test.ts
|
||||
tests/boundary/cli/referenceSourceImportStatus.test.ts
|
||||
@@ -971,6 +902,22 @@ jobs:
|
||||
tests/process-global/runnerCancellation.test.ts
|
||||
tests/process-global/windowsOwnedCommand.test.ts
|
||||
tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-process-boundary-fixture
|
||||
path: .cache/windows-process-boundary/
|
||||
# Passing the fixture adds the breakaway, caller/child token, environment
|
||||
# semantics and ancestor reparse observations, which the lane skips
|
||||
# entirely when it is absent.
|
||||
- run: >-
|
||||
npm run verify:windows-native -- .
|
||||
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-native-inspection-fixture
|
||||
path: .cache/windows-native-inspection-fixture/
|
||||
- name: Verify native inspection failure ownership and recovery
|
||||
run: node scripts/verify-windows-native-inspection.mjs
|
||||
- name: Verify real Node Inspector discovery and observation
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
@@ -1002,7 +949,7 @@ jobs:
|
||||
retention-days: 30
|
||||
|
||||
docs:
|
||||
name: Documentation
|
||||
name: Docs Generation
|
||||
needs: changes
|
||||
if: needs.changes.outputs.docs == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
@@ -1017,8 +964,6 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- run: npm run docs:check
|
||||
- name: Verify published routes, source links and local anchors
|
||||
run: npm run verify:docs:routes
|
||||
- name: Generation leaves authored files unchanged
|
||||
run: git diff --exit-code
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
@@ -1131,26 +1076,6 @@ jobs:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
packet_capture:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.packet_capture == 'true'
|
||||
secrets:
|
||||
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
|
||||
uses: ./.github/workflows/real-packet-capture.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
signal_recording:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.signal_recording == 'true'
|
||||
secrets:
|
||||
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
|
||||
uses: ./.github/workflows/real-signal-recording.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
nativeaot:
|
||||
needs: [changes, static]
|
||||
if: needs.changes.outputs.nativeaot == 'true'
|
||||
@@ -1177,9 +1102,6 @@ jobs:
|
||||
static-deep,
|
||||
build,
|
||||
reverse-tools-quality,
|
||||
python,
|
||||
packet_capture,
|
||||
signal_recording,
|
||||
test-shard,
|
||||
test-source,
|
||||
sqlite-runtime,
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
name: CodeQL production
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
schedule:
|
||||
- cron: "37 5 * * 1"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: codeql-production-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
# A repository administrator must disable default setup before enabling
|
||||
# this workflow; default setup rejects advanced CodeQL result uploads.
|
||||
if: vars.REA_CODEQL_ADVANCED == 'true'
|
||||
name: Analyze production (${{ matrix.language }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write
|
||||
strategy:
|
||||
fail-fast: false
|
||||
max-parallel: 1
|
||||
matrix:
|
||||
language: [javascript-typescript, python, actions]
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Initialize production analysis
|
||||
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
build-mode: none
|
||||
config-file: .github/codeql/production.yml
|
||||
threads: 2
|
||||
ram: 4096
|
||||
- name: Analyze production code
|
||||
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
|
||||
with:
|
||||
category: /language:${{ matrix.language }}
|
||||
output: ${{ runner.temp }}/codeql-production-results
|
||||
threads: 2
|
||||
ram: 4096
|
||||
- name: Retain complete analysis evidence
|
||||
if: always()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: codeql-production-${{ matrix.language }}
|
||||
path: ${{ runner.temp }}/codeql-production-results
|
||||
if-no-files-found: error
|
||||
retention-days: 7
|
||||
@@ -25,7 +25,7 @@ jobs:
|
||||
node-version-file: .nvmrc
|
||||
- name: Validate the SDK pin against the provider's verified release
|
||||
run: node scripts/verify-ghidra-bridge-sources.mjs --check-sdk-pin >> "$GITHUB_ENV"
|
||||
# The bridge probe imports only source-owned ARM, Thumb and MIPS bytes and never
|
||||
# The bridge probe imports only source-owned ARM/Thumb bytes and never
|
||||
# executes target code. All SDK/project state stays in runner storage.
|
||||
- name: Acquire checksum-verified Ghidra SDK in isolated runner storage
|
||||
shell: bash
|
||||
@@ -46,7 +46,7 @@ jobs:
|
||||
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
|
||||
test -n "${!java_home_variable}"
|
||||
JAVA_HOME="${!java_home_variable}" npm run verify:ghidra:bridge-sources -- --verified-sdk
|
||||
- name: Inspect real ARM, Thumb and MIPS instruction context through the bridge
|
||||
- name: Inspect real ARM and Thumb instruction context through the bridge
|
||||
shell: bash
|
||||
run: |
|
||||
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
|
||||
@@ -58,79 +58,7 @@ jobs:
|
||||
-import "${probe_root}/arm-thumb.bin" -loader BinaryLoader -processor ARM:LE:32:v7 \
|
||||
-noanalysis -readOnly -deleteProject \
|
||||
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
-postScript ReaInstructionContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
-postScript ReaArmContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
|
||||
cat "${probe_root}/output.log"
|
||||
grep -F 'REA_ARM_CONTEXT_OK arm=0 thumb=1 unavailable=null (injected provider value)' "${probe_root}/output.log"
|
||||
mkdir -p "${probe_root}/mips-project"
|
||||
# MIPS32 addiu v0,a0,3; MIPS16 li v0,90. These bytes are never executed.
|
||||
printf '\003\000\202\044\132\152\000\000' > "${probe_root}/mips.bin"
|
||||
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/mips-project" MipsContext \
|
||||
-import "${probe_root}/mips.bin" -loader BinaryLoader -processor MIPS:LE:32:default \
|
||||
-noanalysis -readOnly -deleteProject \
|
||||
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
-postScript ReaInstructionContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" mips \
|
||||
> "${probe_root}/mips-output.log" 2>&1 || { cat "${probe_root}/mips-output.log"; exit 1; }
|
||||
cat "${probe_root}/mips-output.log"
|
||||
grep -F 'REA_MIPS_CONTEXT_OK standard=0 mips16=1' "${probe_root}/mips-output.log"
|
||||
# Each probe imports source-owned bytes at 0x1000, away from the Cortex-M
|
||||
# vector symbols that ARMCortex.pspec defines at 0, and decodes them from
|
||||
# the language's default context, so M-profile Thumb state must hold at
|
||||
# the first byte. The bytes are never executed.
|
||||
- name: Decode M-profile ARM and Xtensa bytes in their default context
|
||||
shell: bash
|
||||
run: |
|
||||
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
|
||||
export JAVA_HOME="${!java_home_variable}"
|
||||
probe_root="${RUNNER_TEMP}/rea-decode"
|
||||
mkdir -p "${probe_root}"
|
||||
# mrs r0, msp; bx lr
|
||||
printf '\357\363\010\200\160\107' > "${probe_root}/cortex.bin"
|
||||
# mrs r0, msplim (ARMv8-M only); bx lr
|
||||
printf '\357\363\012\200\160\107' > "${probe_root}/v8m.bin"
|
||||
# entry a1, 32; retw.n
|
||||
printf '\066\101\000\035\360' > "${probe_root}/xtensa.bin"
|
||||
decode() {
|
||||
local name="$1" language="$2"
|
||||
shift 2
|
||||
mkdir -p "${probe_root}/${name}-project"
|
||||
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/${name}-project" Decode \
|
||||
-import "${probe_root}/${name}.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
|
||||
-processor "${language}" -noanalysis -readOnly -deleteProject \
|
||||
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra" \
|
||||
-postScript ReaDecodeProbe.java "${language}" "$@" \
|
||||
> "${probe_root}/${name}.log" 2>&1 || { cat "${probe_root}/${name}.log"; exit 1; }
|
||||
cat "${probe_root}/${name}.log"
|
||||
grep -F "REA_DECODE_OK ${language} $*" "${probe_root}/${name}.log"
|
||||
}
|
||||
decode cortex ARM:LE:32:Cortex mrs/4/msp bx/2/lr
|
||||
decode v8m ARM:LE:32:v8-m mrs/4/msplim bx/2/lr
|
||||
decode xtensa Xtensa:LE:32:default entry/3/a1 retw.n/2
|
||||
# A call through a writable slot keeps Ghidra's recorded target as an
|
||||
# observed callee while callee coverage stays partial; a direct call is
|
||||
# the complete control. The source-owned x86-64 bytes are never executed.
|
||||
- name: Keep computed-call targets non-exhaustive in callee coverage
|
||||
shell: bash
|
||||
run: |
|
||||
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
|
||||
export JAVA_HOME="${!java_home_variable}"
|
||||
probe_root="${RUNNER_TEMP}/rea-computed-call"
|
||||
mkdir -p "${probe_root}/project"
|
||||
{
|
||||
printf '\270\001\000\000\000\303\314\314' # 0x1000: mov eax, 1; ret
|
||||
printf '\270\002\000\000\000\303\314\314' # 0x1008: mov eax, 2; ret
|
||||
printf '\377\025\052\000\000\000\303\314' # 0x1010: call [rip + 0x2a]; ret
|
||||
printf '\377\320\303\314\314\314\314\314' # 0x1018: call rax; ret
|
||||
printf '\350\333\377\377\377\303\314\314' # 0x1020: call 0x1000; ret
|
||||
printf '\314%.0s' $(seq 24) # 0x1028: int3 padding
|
||||
printf '\000\020\000\000\000\000\000\000' # 0x1040: slot = 0x1000
|
||||
} > "${probe_root}/computed-call.bin"
|
||||
test "$(wc -c < "${probe_root}/computed-call.bin")" -eq 72
|
||||
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/project" ComputedCall \
|
||||
-import "${probe_root}/computed-call.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
|
||||
-processor x86:LE:64:default -noanalysis -readOnly -deleteProject \
|
||||
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
-postScript ReaComputedCallCoverageProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
|
||||
cat "${probe_root}/output.log"
|
||||
grep -F 'REA_COMPUTED_CALL_COVERAGE_OK slot=observed-complete-discovery-unknown register=partial direct=complete callers=discovery-unknown' "${probe_root}/output.log"
|
||||
@@ -27,4 +27,4 @@ jobs:
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- run: npm run verify:docs:routes
|
||||
- run: npm run docs:build
|
||||
@@ -54,11 +54,10 @@ jobs:
|
||||
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
# The pinned framework builds its gateway and service-db images only when pulling them from
|
||||
# ghcr.io fails, from public ECR base images; public ECR caps anonymous pulls by data per source
|
||||
# IP and hosted runners share IPs, so such a build can fail here for hours. mirror.gcr.io serves
|
||||
# the same Docker official images under the same digests, so each base image those Dockerfiles
|
||||
# name is pulled there and named as its Dockerfile names it, as the framework's own CI does.
|
||||
# Public ECR caps anonymous pulls by data per source IP and hosted runners share IPs, so the
|
||||
# framework's image builds can fail here for hours. mirror.gcr.io serves the same Docker
|
||||
# official images under the same digests, so each base image the pinned framework builds from
|
||||
# is pulled there and named as its Dockerfile names it, as the framework's own CI does.
|
||||
# Digest-pinned names are set in containerd, whose images Docker reads only on the containerd
|
||||
# image store; switching stores restarts dockerd, so it happens before anything uses Docker.
|
||||
- name: Use the containerd image store
|
||||
@@ -73,16 +72,16 @@ jobs:
|
||||
run: |
|
||||
runner_root="${RUNNER_TEMP}/rea-agentenv-runner"
|
||||
python -m venv "${runner_root}"
|
||||
"${runner_root}/bin/python" -m pip install --require-virtualenv --no-cache-dir 'agentenv-framework==0.9.1431'
|
||||
"${runner_root}/bin/python" -m pip install --require-virtualenv --no-cache-dir 'agentenv-framework==0.9.1334'
|
||||
"${runner_root}/bin/agent-env" --version
|
||||
docker version --format '{{.Server.Version}}'
|
||||
echo "REA_AGENT_ENV_COMMAND=${runner_root}/bin/agent-env" >> "${GITHUB_ENV}"
|
||||
echo "REA_DOCKER_COMMAND=$(command -v docker)" >> "${GITHUB_ENV}"
|
||||
- name: Pull the gateway and service-db public ECR base images from mirror.gcr.io
|
||||
- name: Pull the pinned framework's public ECR base images from mirror.gcr.io
|
||||
shell: bash
|
||||
run: |
|
||||
package="$("${REA_AGENT_ENV_COMMAND%/agent-env}/python" -c 'import agent_env, pathlib; print(pathlib.Path(agent_env.__file__).parent)')"
|
||||
grep -rhoE '^FROM +public\.ecr\.aws/docker/library/[^ ]+' --include='Dockerfile*' "${package}/env/gateway" "${package}/env/envs/service_db" | awk '{print $2}' | sort -u |
|
||||
grep -rhoE '^FROM +public\.ecr\.aws/docker/library/[^ ]+' --include='Dockerfile*' "${package}" | awk '{print $2}' | sort -u |
|
||||
while read -r image; do
|
||||
name=${image#public.ecr.aws/docker/library/}
|
||||
source=mirror.gcr.io/library/$name
|
||||
|
||||
@@ -1,63 +0,0 @@
|
||||
name: Real Android metadata bridge
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ["**"]
|
||||
paths:
|
||||
- .github/workflows/real-android-metadata.yml
|
||||
- bridge/android/**
|
||||
- src/android/JadxRelease.ts
|
||||
- scripts/fixtures/android/**
|
||||
- tests/conformance/android/**
|
||||
- tests/fixtures/android/static-initializers/**
|
||||
- tests/fixtures/android/direct-invokes/**
|
||||
- tests/fixtures/android/external-invokes/**
|
||||
- .nvmrc
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: android-metadata-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
android-metadata:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Fetch checksum-pinned engine and public APK fixture
|
||||
run: node scripts/fixtures/android/fetch.mjs "$RUNNER_TEMP/rea-android"
|
||||
- name: Compile and exercise the real metadata bridge
|
||||
shell: bash
|
||||
run: |
|
||||
export JAVA_HOME="$JAVA_HOME_21_X64"
|
||||
android_root="$RUNNER_TEMP/rea-android"
|
||||
classes="$android_root/classes"
|
||||
jar="$android_root/jadx-headless-mcp-0.7.1-all.jar"
|
||||
mkdir -p "$classes"
|
||||
"$JAVA_HOME/bin/javac" -J-Xmx512m -J-XX:ActiveProcessorCount=2 \
|
||||
-classpath "$jar" -d "$classes" \
|
||||
bridge/android/ReaJadxBridge.java \
|
||||
tests/conformance/android/ReaJadxMetadataRegression.java \
|
||||
tests/conformance/android/ReaJadxExternalReferencesRegression.java
|
||||
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
|
||||
-classpath "$classes:$jar" ReaJadxMetadataRegression \
|
||||
"$android_root/ApiDemos-debug.apk" \
|
||||
"$GITHUB_WORKSPACE/tests/fixtures/android/static-initializers" \
|
||||
"$classes" "$GITHUB_WORKSPACE/tests/fixtures/android/direct-invokes" \
|
||||
| tee "$android_root/metadata.log"
|
||||
grep -F 'RAW_CLASS_SELECTOR_REGRESSION_PASS nested=true metadata=true references=true' \
|
||||
"$android_root/metadata.log"
|
||||
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
|
||||
-classpath "$classes:$jar" ReaJadxExternalReferencesRegression \
|
||||
"$GITHUB_WORKSPACE/tests/fixtures/android/external-invokes" "$classes" \
|
||||
| tee "$android_root/external-references.log"
|
||||
grep -F 'EXTERNAL_REFERENCE_REGRESSION_PASS prototypes=2 using_nodes=3 sites=4' \
|
||||
"$android_root/external-references.log"
|
||||
@@ -55,12 +55,6 @@ jobs:
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Install AArch64 and ELF32 cross toolchains for source-owned fixtures
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --yes --no-install-recommends \
|
||||
gcc-aarch64-linux-gnu gcc-i686-linux-gnu gcc-arm-linux-gnueabi gcc-mips-linux-gnu gcc-mipsel-linux-gnu \
|
||||
libc6-dev-arm64-cross libc6-dev-i386-cross libc6-dev-armel-cross libc6-dev-mips-cross libc6-dev-mipsel-cross
|
||||
- name: Acquire exact upstream libraries in an isolated runner environment
|
||||
shell: bash
|
||||
run: |
|
||||
|
||||
@@ -48,13 +48,7 @@ jobs:
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
# Own a fresh bus so Chromium never depends on an inherited session address.
|
||||
- run: dbus-run-session -- npm run verify:browser
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
# The preceding verifier builds the runtime used by this request journey.
|
||||
- run: dbus-run-session -- node scripts/verify-browser-request.mjs
|
||||
- run: npm run verify:browser
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
@@ -1,35 +1,39 @@
|
||||
name: Real packet capture
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/captures/**"
|
||||
- "tests/boundary/cli/captureCleanup.test.ts"
|
||||
- "tests/boundary/mcp/packetCaptureMcp.test.ts"
|
||||
- "src/captures/**"
|
||||
- "src/pcap/**"
|
||||
- "src/domain/pcap/**"
|
||||
- "src/domain/analysisView/pcapCaptureView.ts"
|
||||
- "src/application/pcap/**"
|
||||
- "src/contracts/pcap/**"
|
||||
- "src/server/registerPcapTools.ts"
|
||||
- "tests/boundary/pcap/**"
|
||||
- "src/domain/captures/**"
|
||||
- "src/application/captures/**"
|
||||
- "src/contracts/captures/**"
|
||||
- "src/composition/captures.ts"
|
||||
- "src/cli/captureCommands.ts"
|
||||
- "src/server/registerCaptureTools.ts"
|
||||
- "scripts/verify-packet-capture.mjs"
|
||||
- ".github/workflows/real-packet-capture.yml"
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
concurrency:
|
||||
group: real-packet-capture-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
tshark:
|
||||
runs-on: ubuntu-24.04
|
||||
@@ -44,18 +48,8 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run build:cached
|
||||
- name: Run focused tests and require Linux resource and cleanup cases
|
||||
run: |
|
||||
report=".cache/rea-packet-required-tests.json"
|
||||
mkdir -p .cache
|
||||
rm -f "$report"
|
||||
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
|
||||
tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts tests/acceptance/pcapVerifier.test.ts
|
||||
node scripts/ci/require-provider-tests.mjs packet "$report"
|
||||
- run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
|
||||
- run: node scripts/verify-packet-capture.mjs
|
||||
env:
|
||||
REA_TSHARK_COMMAND: /usr/bin/tshark
|
||||
|
||||
@@ -1,35 +1,32 @@
|
||||
name: Real signal recording
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/signals/**"
|
||||
- "tests/boundary/cli/signalCleanup.test.ts"
|
||||
- "tests/boundary/mcp/signalRecordingMcp.test.ts"
|
||||
- "src/signals/**"
|
||||
- "src/domain/signals/**"
|
||||
- "src/application/signals/**"
|
||||
- "src/contracts/signals/**"
|
||||
- "src/composition/signals.ts"
|
||||
- "src/cli/signalCommands.ts"
|
||||
- "src/server/registerSignalTools.ts"
|
||||
- "scripts/verify-signal-recording.mjs"
|
||||
- ".github/workflows/real-signal-recording.yml"
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
concurrency:
|
||||
group: real-signal-recording-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
sigrok:
|
||||
runs-on: ubuntu-24.04
|
||||
@@ -44,18 +41,8 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run build:cached
|
||||
- name: Run focused tests and require Linux resource and cleanup cases
|
||||
run: |
|
||||
report=".cache/rea-signal-required-tests.json"
|
||||
mkdir -p .cache
|
||||
rm -f "$report"
|
||||
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
|
||||
tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
|
||||
node scripts/ci/require-provider-tests.mjs signal "$report"
|
||||
- run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
|
||||
- run: node scripts/verify-signal-recording.mjs
|
||||
env:
|
||||
REA_SIGROK_COMMAND: /usr/bin/sigrok-cli
|
||||
@@ -44,8 +44,7 @@ jobs:
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
# Own a fresh bus so Chromium never depends on an inherited session address.
|
||||
- run: dbus-run-session -- npm run verify:browser:runtime
|
||||
- run: env -u DBUS_SESSION_BUS_ADDRESS npm run verify:browser:runtime
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
@@ -46,12 +46,8 @@ jobs:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Install the pinned compiler into an isolated fixture directory
|
||||
run: npm install --prefix "$RUNNER_TEMP/rea-web-source-map-compiler" --no-audit --no-fund --package-lock=false [email protected]
|
||||
# Own a fresh bus so Chromium never depends on an inherited session address.
|
||||
- run: dbus-run-session -- npm run verify:browser:source-maps
|
||||
- run: npm run verify:browser:source-maps
|
||||
env:
|
||||
# Retain browser launch argv/PID/stderr in the verifier's captured
|
||||
# CLI failure diagnostics without changing its deadline or retrying.
|
||||
DEBUG: pw:browser
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
REA_WEB_SOURCE_MAP_COMPILER: ${{ runner.temp }}/rea-web-source-map-compiler/node_modules/esbuild/lib/main.js
|
||||
@@ -47,63 +47,19 @@ jobs:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
run: |
|
||||
# Filter at the repository connection, without a search-index dependency.
|
||||
pages=$(gh api graphql --paginate --slurp \
|
||||
-f owner="${GH_REPO%/*}" -f name="${GH_REPO#*/}" \
|
||||
-f query='query($owner:String!,$name:String!,$endCursor:String) {
|
||||
repository(owner:$owner,name:$name) {
|
||||
nameWithOwner
|
||||
pullRequests(first:100,after:$endCursor,states:OPEN,baseRefName:"main",labels:["rea:release-notes-finalized"]) {
|
||||
nodes { number headRefName headRepository { nameWithOwner } }
|
||||
pageInfo { hasNextPage endCursor }
|
||||
}
|
||||
}
|
||||
}')
|
||||
finalized=$(jq -er '
|
||||
if type != "array" or length == 0 then error("Missing GitHub pages") else . end |
|
||||
if .[-1].data.repository.pullRequests.pageInfo.hasNextPage != false
|
||||
then error("Incomplete GitHub pagination") else . end |
|
||||
map(
|
||||
if (.errors // [] | length) != 0 then error("GitHub query failed") else . end |
|
||||
.data.repository |
|
||||
if (.nameWithOwner | type) != "string" or (.pullRequests.nodes | type) != "array" or
|
||||
(.pullRequests.pageInfo.hasNextPage | type) != "boolean"
|
||||
then error("Invalid GitHub repository connection") else . end |
|
||||
.nameWithOwner as $repo | .pullRequests.nodes[] |
|
||||
if (.number | type) != "number" or (.headRefName | type) != "string" or (has("headRepository") | not) or
|
||||
(.headRepository != null and (.headRepository.nameWithOwner | type) != "string")
|
||||
then error("Invalid GitHub pull request") else . end |
|
||||
select(.headRepository.nameWithOwner == $repo) |
|
||||
select(.headRefName | startswith("release-please--branches--main--")) | .number
|
||||
) | map(tostring) | join(" ")' <<< "$pages")
|
||||
finalized=$(gh api --paginate "repos/${GH_REPO}/pulls?state=open&base=main&per_page=100" \
|
||||
--jq '.[] | select(.head.repo.full_name == .base.repo.full_name) | select(.head.ref | startswith("release-please--branches--main--")) | select(any(.labels[]; .name == "rea:release-notes-finalized")) | .number')
|
||||
if [[ -n "${finalized}" ]]; then
|
||||
echo "Finalized release notes on PR ${finalized}; automatic proposal refresh is paused." >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "frozen=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Check out proposal controller and complete Git history
|
||||
if: steps.editorial.outputs.frozen != 'true'
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
|
||||
if: steps.editorial.outputs.frozen != 'true'
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
|
||||
- name: Install proposal controller dependencies
|
||||
if: steps.editorial.outputs.frozen != 'true'
|
||||
run: npm ci
|
||||
|
||||
- name: Update release proposal
|
||||
if: steps.editorial.outputs.frozen != 'true'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
SOURCE_SHA: ${{ github.sha }}
|
||||
run: node scripts/release-proposal.mjs --target-branch main --source-sha "$SOURCE_SHA"
|
||||
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
target-branch: main
|
||||
skip-github-release: true
|
||||
|
||||
merged-release:
|
||||
name: Release reviewed main merge
|
||||
@@ -276,25 +232,15 @@ jobs:
|
||||
node scripts/release-notes.mjs publication --repository "$GH_REPO" \
|
||||
--source "$SOURCE_SHA"
|
||||
|
||||
- name: Prepare checkpoint release proposal
|
||||
if: inputs.phase == 'prepare'
|
||||
id: proposal
|
||||
env:
|
||||
# A dedicated credential starts PR CI automatically; the built-in
|
||||
# token creates approval-required PR runs for maintainers to approve.
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
RELEASE_BRANCH: ${{ inputs.release_branch }}
|
||||
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
|
||||
run: node scripts/release-proposal.mjs --target-branch "$RELEASE_BRANCH" --source-sha "$SOURCE_SHA"
|
||||
|
||||
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
|
||||
if: inputs.phase == 'publish'
|
||||
id: release
|
||||
with:
|
||||
# A dedicated credential starts PR CI automatically; the built-in
|
||||
# token creates approval-required PR runs for maintainers to approve.
|
||||
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
target-branch: ${{ inputs.release_branch }}
|
||||
skip-github-release: false
|
||||
skip-github-pull-request: true
|
||||
skip-github-release: ${{ inputs.phase == 'prepare' }}
|
||||
skip-github-pull-request: ${{ inputs.phase == 'publish' }}
|
||||
|
||||
- name: Require a merged release PR for publication
|
||||
if: inputs.phase == 'publish' && steps.release.outputs.release_created != 'true'
|
||||
@@ -314,26 +260,26 @@ jobs:
|
||||
fi
|
||||
|
||||
- name: Check out release pull request
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ fromJSON(steps.proposal.outputs.pr).headBranchName }}
|
||||
ref: ${{ fromJSON(steps.release.outputs.pr).headBranchName }}
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Node.js for generated documentation
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
|
||||
- name: Install dependencies
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm ci
|
||||
|
||||
- name: Validate prepared release version and migration notes
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
env:
|
||||
RELEASE_BRANCH: ${{ inputs.release_branch }}
|
||||
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
|
||||
@@ -343,11 +289,11 @@ jobs:
|
||||
--source-sha "$SOURCE_SHA"
|
||||
|
||||
- name: Regenerate release documentation
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm run docs:generate
|
||||
|
||||
- name: Validate generated release documentation
|
||||
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm run docs:check && git diff --exit-code
|
||||
|
||||
qualification:
|
||||
|
||||
@@ -17,10 +17,10 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
# Exercise unavailable-runtime diagnostics and rolling supported releases.
|
||||
# Security floors prevent fallback to an older hosted-toolcache patch.
|
||||
# Exercise unavailable-runtime diagnostics, the first supported release,
|
||||
# and new patches on both supported lines without hand-editing versions.
|
||||
# The repository-pinned Node 24 version also runs the complete test suite.
|
||||
node: ["^22.23.4", "24.22.1", "^24.22.1", "^26.12.1"]
|
||||
node: ["22.x", "24.22.1", "24.x", "26.x"]
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
|
||||
@@ -21,10 +21,6 @@ jobs:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Set up Node.js for the website Worker checks
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
|
||||
@@ -54,10 +54,6 @@ jobs:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Set up Node.js for the website Worker checks
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
@@ -79,21 +75,10 @@ jobs:
|
||||
PY
|
||||
- name: Configure Pages
|
||||
uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
|
||||
# upload-pages-artifact excludes dot directories, including .well-known.
|
||||
- name: Package the website including agent discovery
|
||||
run: |
|
||||
tar --dereference --hard-dereference --directory website/_site \
|
||||
--exclude=.git --exclude=.github -cf "$RUNNER_TEMP/artifact.tar" .
|
||||
tar -tf "$RUNNER_TEMP/artifact.tar" \
|
||||
./.well-known/ard.json ./.well-known/ai-catalog.json \
|
||||
./.well-known/agent-skills/index.json > /dev/null
|
||||
- name: Upload the website
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0
|
||||
with:
|
||||
name: github-pages
|
||||
path: ${{ runner.temp }}/artifact.tar
|
||||
retention-days: 1
|
||||
if-no-files-found: error
|
||||
path: website/_site
|
||||
|
||||
deploy:
|
||||
needs: prepare
|
||||
@@ -145,24 +130,14 @@ jobs:
|
||||
PUBLISH_TARGET: ${{ inputs.target }}
|
||||
run: |
|
||||
python3 -u - <<'PY'
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from urllib.error import URLError
|
||||
from urllib.parse import urljoin, urlsplit
|
||||
from urllib.parse import urljoin
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
def open_public(url, accept=None):
|
||||
headers = {
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
}
|
||||
if accept:
|
||||
headers["Accept"] = accept
|
||||
return urlopen(Request(url, headers=headers), timeout=10)
|
||||
|
||||
expected = os.environ["GITHUB_SHA"]
|
||||
pages_url = os.environ["PAGES_URL"]
|
||||
if not pages_url:
|
||||
@@ -178,7 +153,11 @@ jobs:
|
||||
for name, base in list(pending.items()):
|
||||
url = urljoin(base, "deployment.json") + "?commit=" + expected
|
||||
try:
|
||||
with open_public(url) as response:
|
||||
request = Request(url, headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
})
|
||||
with urlopen(request, timeout=10) as response:
|
||||
actual = json.load(response).get("commit")
|
||||
if actual == expected:
|
||||
print(f"{name} serves {expected}: {base}")
|
||||
@@ -193,27 +172,6 @@ jobs:
|
||||
time.sleep(5)
|
||||
if pending:
|
||||
sys.exit("Could not verify published revision on: " + ", ".join(pending))
|
||||
for name, base in hosts.items():
|
||||
with open_public(urljoin(base, ".well-known/agent-skills/index.json")) as response:
|
||||
skill, = json.load(response)["skills"]
|
||||
# Fetch from each selected host, including the /rea/ mirror.
|
||||
artifact_url = urljoin(base, urlsplit(skill["url"]).path.lstrip("/"))
|
||||
with open_public(artifact_url) as response:
|
||||
digest = "sha256:" + hashlib.sha256(response.read()).hexdigest()
|
||||
if digest != skill["digest"]:
|
||||
sys.exit(name + " serves a skill archive with the wrong digest")
|
||||
for manifest in ("ard.json", "ai-catalog.json"):
|
||||
with open_public(urljoin(base, ".well-known/" + manifest)) as response:
|
||||
if not json.load(response).get("entries"):
|
||||
sys.exit(name + " serves an empty discovery manifest")
|
||||
if name == "Cloudflare":
|
||||
with open_public(base, accept="text/markdown") as response:
|
||||
vary = {value.strip().lower() for value in response.headers.get("Vary", "").split(",")}
|
||||
if response.headers.get_content_type() != "text/markdown" or "accept" not in vary:
|
||||
sys.exit("Cloudflare did not negotiate Markdown with Vary: Accept")
|
||||
if 'rel="ard"' not in response.headers.get("Link", ""):
|
||||
sys.exit("Cloudflare is missing its discovery Link header")
|
||||
print(name + " serves the discovery manifests and verified skill archive")
|
||||
with open(os.environ["GITHUB_STEP_SUMMARY"], "a", encoding="utf-8") as summary:
|
||||
links = " and ".join(f"[{name}]({url})" for name, url in hosts.items())
|
||||
summary.write(f"Published commit `{expected}` to {links}.\n")
|
||||
|
||||
+1
-1
@@ -50,7 +50,7 @@ Thumbs.db
|
||||
|
||||
# Local reference sources and analysis artifacts
|
||||
/_reference/
|
||||
/experiments/analysis-efficiency/
|
||||
/experiments/analysis-efficiency/results/
|
||||
|
||||
# Generated website download
|
||||
/website/public/examples/notes-example.zip
|
||||
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"threshold": 4,
|
||||
"threshold": 3.5,
|
||||
"reporters": ["consoleFull"],
|
||||
"ignore": [
|
||||
"node_modules/**",
|
||||
|
||||
+1
-4
@@ -10,10 +10,7 @@
|
||||
"node_modules/**",
|
||||
"CHANGELOG.md",
|
||||
"package-lock.json",
|
||||
".claude-plugin/plugin.json",
|
||||
".codex-plugin/plugin.json",
|
||||
"_reference/**",
|
||||
"third_party/**",
|
||||
".codex-plugin/mcp.json"
|
||||
"third_party/**"
|
||||
]
|
||||
}
|
||||
+3
-95
@@ -1,13 +1,7 @@
|
||||
{
|
||||
"$schema": "./node_modules/oxlint/configuration_schema.json",
|
||||
"plugins": ["typescript", "import", "unicorn", "oxc", "vitest"],
|
||||
"plugins": ["typescript", "import", "unicorn", "oxc"],
|
||||
"rules": {
|
||||
"rea/no-buffer-deep-equality": "error",
|
||||
"rea/cancellation-cause": "error",
|
||||
"no-async-promise-executor": "error",
|
||||
"no-unreachable": "error",
|
||||
"no-unsafe-optional-chaining": "error",
|
||||
"no-useless-escape": ["warn", { "allowRegexCharacters": ["-"] }],
|
||||
"complexity": ["warn", 20],
|
||||
"max-depth": ["warn", 4],
|
||||
"max-lines": [
|
||||
@@ -29,38 +23,7 @@
|
||||
"max-params": ["warn", 4],
|
||||
"no-var": "warn",
|
||||
"no-var-requires": "error",
|
||||
"typescript/no-import-type-side-effects": "error",
|
||||
"vitest/no-standalone-expect": "off",
|
||||
"vitest/no-conditional-expect": "off",
|
||||
"vitest/expect-expect": "off",
|
||||
"vitest/no-disabled-tests": "off",
|
||||
"vitest/require-mock-type-parameters": "off",
|
||||
"vitest/require-to-throw-message": "off",
|
||||
"import/no-duplicates": [
|
||||
"warn",
|
||||
{
|
||||
"preferInline": false
|
||||
}
|
||||
],
|
||||
"vitest/valid-expect": [
|
||||
"warn",
|
||||
{
|
||||
"maxArgs": 2
|
||||
}
|
||||
],
|
||||
"rea/no-evidence-json-preadmission": "error",
|
||||
"rea/no-positional-tool-contract": "error",
|
||||
"rea/no-fixture-root-cleanup": "error",
|
||||
"typescript/no-floating-promises": "error",
|
||||
"typescript/no-misused-promises": "warn",
|
||||
"typescript/return-await": ["error", "error-handling-correctness-only"],
|
||||
"typescript/switch-exhaustiveness-check": [
|
||||
"error",
|
||||
{
|
||||
"considerDefaultExhaustiveForUnions": true
|
||||
}
|
||||
],
|
||||
"rea/no-shared-module-mock": "error"
|
||||
"typescript/no-import-type-side-effects": "error"
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
@@ -73,12 +36,6 @@
|
||||
"no-var-requires": "off"
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": ["src/**/*.ts"],
|
||||
"rules": {
|
||||
"typescript/no-misused-promises": "error"
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": ["tests/**/*.ts", "src/**/*.test.ts", "src/**/*.fixture.ts"],
|
||||
"rules": {
|
||||
@@ -90,8 +47,7 @@
|
||||
"skipBlankLines": true,
|
||||
"skipComments": true
|
||||
}
|
||||
],
|
||||
"typescript/no-misused-promises": "warn"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
@@ -135,45 +91,6 @@
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": ["tests/**/*.test.ts", "src/**/*.test.ts"],
|
||||
"rules": {
|
||||
"vitest/valid-expect": [
|
||||
"warn",
|
||||
{
|
||||
"maxArgs": 2
|
||||
}
|
||||
],
|
||||
"vitest/valid-expect-in-promise": "error",
|
||||
"vitest/no-focused-tests": "error"
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": [
|
||||
"scripts/lib/http3-capture-assertions.test.mjs",
|
||||
"scripts/lib/http3-verifier-lifecycle.test.mjs"
|
||||
],
|
||||
"rules": {
|
||||
"typescript/no-floating-promises": [
|
||||
"error",
|
||||
{
|
||||
"allowForKnownSafeCalls": [
|
||||
{
|
||||
"from": "package",
|
||||
"package": "@types/node",
|
||||
"name": "test"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": ["scripts/fixtures/javascript-recovery/code-split/entry.mjs"],
|
||||
"rules": {
|
||||
"typescript/no-floating-promises": "off"
|
||||
}
|
||||
}
|
||||
],
|
||||
"env": {
|
||||
@@ -192,14 +109,5 @@
|
||||
".devin",
|
||||
".factory",
|
||||
"third_party"
|
||||
],
|
||||
"options": {
|
||||
"reportUnusedDisableDirectives": "error"
|
||||
},
|
||||
"jsPlugins": [
|
||||
{
|
||||
"name": "rea",
|
||||
"specifier": "./scripts/lint/index.mjs"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "6.8.0"
|
||||
".": "6.7.0"
|
||||
}
|
||||
@@ -14,13 +14,13 @@ Preserve caller-selected inputs and local evidence, including output, URLs, path
|
||||
|
||||
## Architecture and Cleanup
|
||||
|
||||
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture/architecture.mermaid](docs/architecture/architecture.mermaid) when changing composition.
|
||||
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture.mermaid](docs/architecture.mermaid) when changing composition.
|
||||
|
||||
Give each interpretation and resource a clear owner. Consumers must use that owner's parsed facts rather than reinterpret raw source. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
|
||||
|
||||
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
|
||||
|
||||
For lifecycle changes, follow [ownership and completion invariants](docs/development/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/development/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
|
||||
For lifecycle changes, follow [ownership and completion invariants](docs/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
|
||||
|
||||
Before adding complexity for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode, likelihood, and complexity cost to the user and get their buy-in. Do not silently expand scope for theoretical robustness.
|
||||
|
||||
@@ -28,10 +28,6 @@ Assess brittleness in touched code during implementation, debugging, review, and
|
||||
|
||||
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
|
||||
|
||||
After confirming a bug, search sibling adapters, consumers and similar workflows
|
||||
for the same failure. Verify suspected matches, record working comparisons, and
|
||||
group findings by root cause instead of filing duplicate reports.
|
||||
|
||||
When addressing a review thread, inspect the producing boundary and affected consumers for brittleness. Complete directly related refactoring in the same change. Before resolving the thread, explain the root cause, structural fix, validation, and any remaining cleanup.
|
||||
|
||||
Parse unknown boundary values and model expected failures with the tagged error algebra and `Result`. Preserve partial facts and meaningful failure reasons. Keep identity and lookup values separate from display formatting; preserve source values and explicit unknowns when normalization loses information. Absolute paths, file URLs, and HTTP paths have different semantics.
|
||||
@@ -57,23 +53,14 @@ Optimize in this order:
|
||||
3. CPU and memory. Avoid unnecessary work and copies; bound concurrency,
|
||||
retained state and serialization growth.
|
||||
|
||||
Reserve performance measurements for changes that target performance or have a
|
||||
concrete regression concern. Routine cleanup uses correctness checks; changes
|
||||
to helper calls, imports, or representations alone do not require synthetic
|
||||
benchmarks. When measurement is warranted, measure representative completed
|
||||
tasks and compare equivalent workflows before and after where available. Report
|
||||
token usage, elapsed time, CPU time and peak process-family memory. Separate cold
|
||||
For changes affecting these costs, measure representative completed tasks and
|
||||
compare equivalent workflows before and after where available. Report token
|
||||
usage, elapsed time, CPU time and peak process-family memory. Separate cold
|
||||
and warm runs, label estimates, and state unmeasured costs. Fewer bytes alone
|
||||
do not establish fewer tokens or faster analysis.
|
||||
|
||||
## Development
|
||||
|
||||
Documentation ownership and authoring conventions live in
|
||||
[docs/development/documentation.md](docs/development/documentation.md). A feature
|
||||
uses `docs/specs/<domain>/<feature-slug>/spec.md`, one `plan.md` and `tasks.md`;
|
||||
individual architectural decisions use `docs/adr/<domain>/<decision-slug>.md`.
|
||||
Consult the [documentation directory](docs/README.md) before adding a guide.
|
||||
|
||||
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`. Prefix `PATH="/usr/local/bin:$PATH"` before `node` and `npm`.
|
||||
|
||||
- `npm ci`: install locked dependencies.
|
||||
@@ -85,24 +72,15 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
|
||||
|
||||
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
|
||||
|
||||
When changing installers or setup recovery, follow
|
||||
[Checking installers and publisher downloads](docs/development/testing.md#checking-installers-and-publisher-downloads).
|
||||
|
||||
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/development/testing.md#developer-commands).
|
||||
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/testing.md#developer-commands).
|
||||
|
||||
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
|
||||
|
||||
## Tool and Test Changes
|
||||
|
||||
For tool design, follow [docs/development/tool-design.md](docs/development/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
|
||||
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
|
||||
|
||||
For user- or agent-facing tool problems, prioritize fixing the owning behavior,
|
||||
contract, and actionable result or recovery at the point of use. Document the
|
||||
result in product docs when explanation is needed. Add skill guidance only for
|
||||
workflow judgment or context that the tool cannot reasonably convey; do not use
|
||||
a skill as the primary repair for a discoverability or contract defect.
|
||||
|
||||
For test changes or provider claims, use [docs/development/testing.md](docs/development/testing.md). Select applicable [boundary regressions](docs/development/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
|
||||
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
|
||||
|
||||
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
|
||||
|
||||
@@ -130,12 +108,11 @@ distinguish source-derived conclusions, proposed checks, and unavailable coverag
|
||||
Address actionable automated review findings and explain disagreements.
|
||||
|
||||
Leave the template's **Author readiness** declarations unchecked for the human
|
||||
owner to complete. Never attest to human review on their behalf. Draft status
|
||||
is optional; preserve the PR's current draft status unless the user asks to change
|
||||
it. Later pushes do not invalidate the readiness declaration or require another
|
||||
human sign-off. Keep the PR body and affected validation current as work changes.
|
||||
The readiness check validates the filled declaration or the author exemption
|
||||
listed in CONTRIBUTING.md; CI and automated review results do not establish that
|
||||
owner to complete. Never attest to human review on their behalf. Keep the PR in
|
||||
draft until the human declares the PR initially ready. Later pushes do not
|
||||
invalidate that declaration or require another human sign-off. Keep the PR body
|
||||
and affected validation current as work changes. The readiness check validates
|
||||
the filled declaration; CI and automated review results do not establish that
|
||||
human review occurred, maintainer scope approval, or merge permission.
|
||||
|
||||
<!-- BEGIN:turborepo-agent-rules -->
|
||||
|
||||
+10
-112
@@ -1,107 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## [6.8.0](https://github.com/morluto/rea/compare/rea-agents-6.7.0...rea-agents-6.8.0) (2026-10-25)
|
||||
|
||||
### Highlights
|
||||
|
||||
- Build local agent environments from network captures, browser scenarios, or a prompt, then serve derived reads and evidence-gated writes and compare them with fresh application captures.
|
||||
- Inspect more shipped formats, including GPU code objects, BEAM, Python bytecode, Unity IL2CPP metadata, Godot PCK, PyInstaller, Node.js SEA, devicetree, and additional PE/ELF tables.
|
||||
- Keep large investigations focused with retained summaries, selected fact queries, configurable recovery limits, and row-oriented CLI exports. Complete Evidence remains available.
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
- **Function analysis:** `analyze_function` now defaults to a retained summary. Set `detail: "complete"` for the previous inline dossier; the CLI retains its complete default and requires `--evidence_output` when selecting a summary. [#5660](https://github.com/morluto/rea/pull/5660). Thanks @CryptoJones, @gensokyo-dev, @RishTheFish21.
|
||||
- **Accessibility:** Accept JSON numbers and booleans in node `value`, alongside strings and null. Update consumers that previously assumed text-only values. [cc90771](https://github.com/morluto/rea/commit/cc90771af14f045538651615147d8bf86b576549), [#6759](https://github.com/morluto/rea/pull/6759), [#6467](https://github.com/morluto/rea/issues/6467). Thanks @coygeek, @gensokyo-dev, @mvanhorn.
|
||||
- **JSON Lines:** `--format jsonl` now emits object-collection rows followed by a `$rea: "jsonl/v1"`, `type: "metadata"` record; scalar selections and unsupported reports are refused. Parse control records separately, select a single supported collection when necessary, and use `--format json` for the previous whole-document output. See [JSON Lines framing](https://github.com/morluto/rea/blob/rea-agents-6.8.0/docs/cli.md#json-lines-record-output). [#5629](https://github.com/morluto/rea/pull/5629), [#5487](https://github.com/morluto/rea/issues/5487). Thanks @gensokyo-dev, @jayykernel, @vijaykiran06.
|
||||
- **Interface Builder:** Keyed-archive graph identities use decimal table UIDs instead of authored `objectID`/`object-id`. Treat authored IDs as attributes and update persisted graph lookups; ibtool and native NIBArchive identities keep their source semantics. [6fbe67d](https://github.com/morluto/rea/commit/6fbe67d66e3d716db69929503a3a232a7ada0860).
|
||||
- **Frida:** `instrument-with-frida` JSON returns the canonical Evidence directly. Read `normalized_result.cleanup_error` instead of the former top-level wrapper field. [48f700e](https://github.com/morluto/rea/commit/48f700ee35e0988ed973e5486dc9db33f2a5d944).
|
||||
- **Reconstruction proof:** ID-array proof records no longer establish verified or ready ledgers. Supply implementation/revision bindings and individual claim outcomes; handle inapplicable, unavailable, and failed proof diagnostics. [f8b65e7](https://github.com/morluto/rea/commit/f8b65e78723b1e2169f94dd1226198ea3139707e).
|
||||
- **Mach-O imports:** Handle a null import `name` when identity is ambiguous; use `raw_name` and `name_status` to distinguish observed expressions from exact symbols. [5d0d243](https://github.com/morluto/rea/commit/5d0d243a45c9132e1aff75d454ab53d86a74cc31), [169512a](https://github.com/morluto/rea/commit/169512a35bb015ff209168fb8bc1c9b31a2c678b).
|
||||
- **Node.js:** Use a runtime satisfying `^22.23.4 || ^24.22.1 || >=26.12.1`; Node 23 and 25 are unsupported. Upgrade Node before installing or updating REA; the installer does not upgrade it. [#6315](https://github.com/morluto/rea/pull/6315), [#6283](https://github.com/morluto/rea/issues/6283). Thanks @GamerDubz, @gensokyo-dev, @ShikamaruNaaraa.
|
||||
- **Browser storage:** Historical hashed storage remains readable, but comparisons stay unknown until both captures are recaptured in the new representation. [#6341](https://github.com/morluto/rea/pull/6341), [#6289](https://github.com/morluto/rea/issues/6289). Thanks @coygeek, @gongfpp.
|
||||
- **Environment serving:** `environment-serve` serves derived state by default, including write tools whose effects are confirmed by recorded evidence. Use `--replay-only` for stateless replay of manifest tools. Read stop-summary `state` instead of `state_evidence_id`. [#5351](https://github.com/morluto/rea/pull/5351), [#6158](https://github.com/morluto/rea/pull/6158). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **MCP schemas:** Input advertisements default to compact. Set `REA_MCP_INPUT_SCHEMA_PROFILE=full` for complete advertised guidance and nested constraints. [#5975](https://github.com/morluto/rea/pull/5975). Thanks @gensokyo-dev.
|
||||
- **Browser console arguments:** Accept null `argument_types` and `text_capture.status: "unavailable"` when the protocol argument inventory is omitted or malformed; preserve that uncertainty. [#6016](https://github.com/morluto/rea/pull/6016), [#5934](https://github.com/morluto/rea/issues/5934). Thanks @coygeek, @kaoru0822-kitauji.
|
||||
- **Reconstruction summaries:** `summary.by_evidence_authority` is keyed by Evidence authority; read the former source-state counts from required `summary.by_source_state`. Regenerate ledgers whose closure digests depended on the old projection. [#5875](https://github.com/morluto/rea/pull/5875), [#5838](https://github.com/morluto/rea/issues/5838). Thanks @coygeek, @kaoru0822-kitauji.
|
||||
- **Swift bindings:** Conformance `type_name` and `protocol_name` may be null; consume typed `type_binding`/`protocol_binding` evidence. Replace the former Objective-C-specific import-binding type/schema with `MachoImportBinding`. [d0067ba](https://github.com/morluto/rea/commit/d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f).
|
||||
- **WebSocket frames:** Accept null opcodes and preserve unknown frame facts; numeric opcodes are restricted to the four-bit protocol range. [0bc2b42](https://github.com/morluto/rea/commit/0bc2b42001f684beffcf47b70e507443b1a5d320).
|
||||
- **Resource-only inspection:** Evidence subjects and executable digests may be null. Use `resource_root` and each resource's reported provenance instead of requiring an executable target. [6a6adf7](https://github.com/morluto/rea/commit/6a6adf703fa44100643947d20fbc670f86260b1b).
|
||||
- **Browser timestamps:** Console and session-timeline timestamps may be null when the protocol supplies no finite/comparable time. Keep unknown time explicit in sorting and comparisons. [690fc51](https://github.com/morluto/rea/commit/690fc518540bae982038a74f47ef7781201785aa), [2a0410f](https://github.com/morluto/rea/commit/2a0410f1717b3126e8a95063d77dea66a597010a).
|
||||
- **Reference sources:** Read failed root enumeration from `rootFailure` rather than a synthetic failed child at `"."`; preserve the top-level source-graph limitation. [6296fdf](https://github.com/morluto/rea/commit/6296fdf560f1ac4aa55985558edaa98300aacdbc).
|
||||
- **Filesystem capture:** Accept `entry_unavailable` and `identity_changed` enumeration omission reasons in addition to the previous reason literals. [c85b44e](https://github.com/morluto/rea/commit/c85b44e1b14851c54e65327a4807f79ad5cd5a61). Thanks @rudrakshagarwal4200-ai.
|
||||
- **APK locales:** Replace numeric legacy selectors such as `es-r419` with `es-419` or `b+es+419`. Two-letter regions remain supported. [29910d1](https://github.com/morluto/rea/commit/29910d16b98fe4b607a7aa15928789a73614b0b9). Thanks @mertkayacs.
|
||||
- **XML plist scalars:** Recognize `$plist_type: "unknown"` and read `element`, `literal`, and `reason`; unresolved XML scalars use this wrapper instead of the former real/null unknown marker or malformed scalar prefixes. Supported extended UTC years preserve their timestamp. [3537e58](https://github.com/morluto/rea/commit/3537e58c1ab29309f06eabf50bc52adba9817a0c).
|
||||
|
||||
### Changes
|
||||
|
||||
- **Binary layouts:** Inspect ELF32 i386, little-endian ARM, and little- or big-endian MIPS `EXEC`/`DYN`/`REL` layouts on Linux x64 using a caller-supplied pwntools environment. [#7052](https://github.com/morluto/rea/pull/7052), [#6934](https://github.com/morluto/rea/issues/6934). Thanks @roblennon, @salmanalmutairi1.
|
||||
- **Agent environments:** Project network captures and Evidence files into manifests, preserve template path pairs and positions, export AgentEnv bundles, and serve manifests as replay environments. [#4954](https://github.com/morluto/rea/pull/4954), [#5293](https://github.com/morluto/rea/pull/5293), [#5664](https://github.com/morluto/rea/pull/5664), [#4999](https://github.com/morluto/rea/pull/4999), [#4975](https://github.com/morluto/rea/pull/4975). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Capture to environment:** Build environments from browser scenarios or a prompt, extract the initial observed state, and compare the environment with application probe captures. [#5182](https://github.com/morluto/rea/pull/5182), [#6946](https://github.com/morluto/rea/pull/6946), [#5319](https://github.com/morluto/rea/pull/5319), [#5699](https://github.com/morluto/rea/pull/5699). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **State-backed tools:** Render read filters, search, sorting and paging from state; apply confirmed write effects with broader state targets and recording-based write admission. [#5352](https://github.com/morluto/rea/pull/5352), [293832b](https://github.com/morluto/rea/commit/293832bafc9a054914ec514fe906dd4936c7a372), [#5800](https://github.com/morluto/rea/pull/5800), [#5787](https://github.com/morluto/rea/pull/5787), [#5600](https://github.com/morluto/rea/pull/5600), [5375ff0](https://github.com/morluto/rea/commit/5375ff0f9aadb4fd0d46333815aa0431e3df1010), [#6169](https://github.com/morluto/rea/pull/6169), [#6385](https://github.com/morluto/rea/pull/6385), [#6897](https://github.com/morluto/rea/pull/6897). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **State learning:** Model application-assigned values, preserve view-specific clock spellings, learn write clocks from scenario timing, derive fields, and learn creates across sorted arrays and referenced entities. [7ec9b6e](https://github.com/morluto/rea/commit/7ec9b6e56e1b3762285dca881db6d900a389ab9c), [#6136](https://github.com/morluto/rea/pull/6136), [957bcfa](https://github.com/morluto/rea/commit/957bcfa149c1a0593bb0aa2466cc56d1d4f62e7d), [#6138](https://github.com/morluto/rea/pull/6138), [#6967](https://github.com/morluto/rea/pull/6967), [#7025](https://github.com/morluto/rea/pull/7025), [#7034](https://github.com/morluto/rea/pull/7034). Thanks @earakely-scale.
|
||||
- **Recorded writes:** Serve creates whose compatible read-backs add omitted fields or express timestamps at a different precision, and whole-object updates that regenerate observed slugs or modification times. Conflicting observations still withhold writes. State views and bundles carrying the new read-back data need REA 6.8.0 or later to serve. [#7080](https://github.com/morluto/rea/pull/7080). Thanks @earakely-scale.
|
||||
- **Native investigations:** Page native inventory summaries, match functions across builds, export pseudocode to resumable directories, and select objdump disassembly by symbol. [#5378](https://github.com/morluto/rea/pull/5378), [#5390](https://github.com/morluto/rea/pull/5390), [#6170](https://github.com/morluto/rea/pull/6170), [#5171](https://github.com/morluto/rea/pull/5171), [#4249](https://github.com/morluto/rea/issues/4249), [#5343](https://github.com/morluto/rea/issues/5343), [#5159](https://github.com/morluto/rea/issues/5159). Thanks @coygeek, @CryptoJones, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Engine workflows:** Bound a single Ghidra decompilation without closing its session, analyze GameCube/Wii DOL executables, and inspect static CUDA/HIP code objects. [#5662](https://github.com/morluto/rea/pull/5662), [#6240](https://github.com/morluto/rea/pull/6240), [#5813](https://github.com/morluto/rea/pull/5813), [#5340](https://github.com/morluto/rea/issues/5340), [#5345](https://github.com/morluto/rea/issues/5345). Thanks @arielfikru, @coygeek, @gensokyo-dev, @RishTheFish21.
|
||||
- **PE/ELF:** Inspect export tables without an engine, PE imports/delay-loads, Authenticode certificates, TLS callbacks/load configuration, ELF notes, Linux kernel module metadata, and Zstandard section-name tables. [#6278](https://github.com/morluto/rea/pull/6278), [#6509](https://github.com/morluto/rea/pull/6509), [#4830](https://github.com/morluto/rea/pull/4830), [#7008](https://github.com/morluto/rea/pull/7008), [#4837](https://github.com/morluto/rea/pull/4837), [#7010](https://github.com/morluto/rea/pull/7010), [#6987](https://github.com/morluto/rea/pull/6987), [#4057](https://github.com/morluto/rea/issues/4057). Thanks @gensokyo-dev, @kaoru0822-kitauji, @mvanhorn, @Rinkia, @ShuaixinHuang.
|
||||
- **Apple/native metadata:** Inspect IPS crash reports, decode Swift nominal declarations, and select universal Mach-O slices in Go/Rust metadata readers. [#6511](https://github.com/morluto/rea/pull/6511), [#6257](https://github.com/morluto/rea/pull/6257), [#6219](https://github.com/morluto/rea/pull/6219), [#5341](https://github.com/morluto/rea/issues/5341), [#4102](https://github.com/morluto/rea/issues/4102). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @mvanhorn, @Rinkia.
|
||||
- **Packaged applications:** Triage desktop installation directories and inspect/extract Godot PCK, PyInstaller archives, and Node.js single-executable application blobs. [#5207](https://github.com/morluto/rea/pull/5207), [#3634](https://github.com/morluto/rea/pull/3634), [afa5da7](https://github.com/morluto/rea/commit/afa5da745e0b139e68029a96b776ae30e48ea2b2), [#2433](https://github.com/morluto/rea/pull/2433), [#7000](https://github.com/morluto/rea/pull/7000), [#4399](https://github.com/morluto/rea/issues/4399). Thanks @0xthearchitect, @coygeek, @gensokyo-dev, @mvanhorn.
|
||||
- **Unity and managed code:** Inventory IL2CPP images/types/methods and metadata versions 24/27; read member names and literals, link native methods and generic instances, and export Ghidra seeds. Project managed call/field-access edges and page managed member summaries. [#3807](https://github.com/morluto/rea/pull/3807), [#5326](https://github.com/morluto/rea/pull/5326), [52d1a72](https://github.com/morluto/rea/commit/52d1a72af047a73b787b8935b032786a34966036), [#5327](https://github.com/morluto/rea/pull/5327), [#5785](https://github.com/morluto/rea/pull/5785), [#6260](https://github.com/morluto/rea/pull/6260), [#5167](https://github.com/morluto/rea/pull/5167), [#3784](https://github.com/morluto/rea/issues/3784), [#5112](https://github.com/morluto/rea/issues/5112), [#4056](https://github.com/morluto/rea/issues/4056), [#5148](https://github.com/morluto/rea/issues/5148). Thanks @8rulerstar, @alexlanghart, @elricfd, @gensokyo-dev, @jaelliot, @kaoru0822-kitauji, @Rinkia.
|
||||
- **Android:** Inspect APK signing declarations and structured manifest components; report static-final DEX field values. [e93e3b1](https://github.com/morluto/rea/commit/e93e3b1e7d6e7e4f49c2de24b92452ec29e853f8), [#4158](https://github.com/morluto/rea/pull/4158), [#5025](https://github.com/morluto/rea/pull/5025), [#6981](https://github.com/morluto/rea/pull/6981), [#4537](https://github.com/morluto/rea/issues/4537), [#6397](https://github.com/morluto/rea/issues/6397). Thanks @Abelo9996, @BaaRai, @FerroLx, @gensokyo-dev, @sserrgo12.
|
||||
- **Bytecode and firmware:** Inspect and query CPython pyc and BEAM facts, inspect flattened devicetrees, and attribute MMIO peripheral regions in native disassembly. [#4826](https://github.com/morluto/rea/pull/4826), [#6928](https://github.com/morluto/rea/pull/6928), [#4833](https://github.com/morluto/rea/pull/4833), [#6943](https://github.com/morluto/rea/pull/6943), [#7009](https://github.com/morluto/rea/pull/7009), [#5348](https://github.com/morluto/rea/pull/5348), [#6922](https://github.com/morluto/rea/issues/6922), [#6938](https://github.com/morluto/rea/issues/6938). Thanks @gensokyo-dev, @liziye627-design, @mvanhorn.
|
||||
- **JavaScript:** Retain queryable static route inventories, link assignment values to writes, and accept source selections as a JSON array or file. [#6019](https://github.com/morluto/rea/pull/6019), [#6206](https://github.com/morluto/rea/pull/6206), [#5029](https://github.com/morluto/rea/pull/5029), [#4772](https://github.com/morluto/rea/issues/4772), [#6193](https://github.com/morluto/rea/issues/6193). Thanks @gensokyo-dev, @IvenKooLab, @jacobmalibiran, @JoshOsullivan-au, @kaoru0822-kitauji, @KJLavender.
|
||||
- **Data inspection:** Decode selected protobuf/gRPC bodies, inspect GGUF/safetensors declarations, and produce byte-entropy profiles. [#5031](https://github.com/morluto/rea/pull/5031), [#5603](https://github.com/morluto/rea/pull/5603), [#2331](https://github.com/morluto/rea/pull/2331), [#4986](https://github.com/morluto/rea/issues/4986), [#3323](https://github.com/morluto/rea/issues/3323). Thanks @AayushJB03, @gensokyo-dev, @kaoru0822-kitauji, @KJLavender, @michaelkillgta, @mvanhorn.
|
||||
- **CLI input/output:** Accept structured request documents on stdin and expose version/runtime output formats. Row-oriented JSONL exports use the migration above. [#7040](https://github.com/morluto/rea/pull/7040), [#5486](https://github.com/morluto/rea/issues/5486), [#6530](https://github.com/morluto/rea/pull/6530), [169512a](https://github.com/morluto/rea/commit/169512a35bb015ff209168fb8bc1c9b31a2c678b), [#5483](https://github.com/morluto/rea/issues/5483). Thanks @Abelo9996, @ahmed-husssain, @CassieFox23, @DeryFerd, @eco-null, @ezrabrilliant, @GamerDubz, @gensokyo-dev, @gongfpp, @hifzabuildsai, @jayykernel, @mhamzasakhi, @pablodiazjorge, @Plastic-time, @raidnouredden-ux, @SandstromPL, @ShuaixinHuang, @Sriram-PR, @XiaoBinGan.
|
||||
- **MCP and setup:** Add opt-in Streamable HTTP and FreeBuff setup registration. FreeBuff support first appears in 6.8.0; installed native FreeBuff loading and skill discovery remain unverified. [#4955](https://github.com/morluto/rea/pull/4955), [#2736](https://github.com/morluto/rea/pull/2736), [#3243](https://github.com/morluto/rea/issues/3243). Thanks @0xLLVEL, @codebuff-team, @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Recovery controls:** Adjust per-call WASM/SQLite/firmware deadlines, MCP startup budgets, regex matching deadlines/worker heap, capture decoder heap, ASAR header admission, Apple dispatch input, and the bridge request ceiling. MCP runtime limit recovery reports actionable settings. [#5798](https://github.com/morluto/rea/pull/5798), [#5818](https://github.com/morluto/rea/pull/5818), [aa53c06](https://github.com/morluto/rea/commit/aa53c065387c58b4189d82c47f1f3445fc505e30), [#5863](https://github.com/morluto/rea/pull/5863), [#6304](https://github.com/morluto/rea/pull/6304), [#6267](https://github.com/morluto/rea/pull/6267), [#6301](https://github.com/morluto/rea/pull/6301), [#6299](https://github.com/morluto/rea/pull/6299), [#6209](https://github.com/morluto/rea/pull/6209), [#6233](https://github.com/morluto/rea/pull/6233), [#6376](https://github.com/morluto/rea/pull/6376), [#7049](https://github.com/morluto/rea/pull/7049), [#5797](https://github.com/morluto/rea/issues/5797), [#5812](https://github.com/morluto/rea/issues/5812), [#5851](https://github.com/morluto/rea/issues/5851), [#6176](https://github.com/morluto/rea/issues/6176), [#3917](https://github.com/morluto/rea/issues/3917). Thanks @gensokyo-dev, @jaychou813, @kaoru0822-kitauji.
|
||||
- **APK output:** Set `string_tables_only` or CLI `--string-tables-only` to omit duplicate primary strings and avoid their output-capacity charge. [7c458f0](https://github.com/morluto/rea/commit/7c458f04b9155707c8ba406f8b455b10016f1b99), [#4990](https://github.com/morluto/rea/pull/4990), [#4982](https://github.com/morluto/rea/issues/4982). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Retained analysis:** Reuse successful validation of authenticated immutable native/imported-JavaScript parents and captures. Query retained Go build identity, Rust dependencies, JavaScript source-to-bundle mappings, and native export facts without repeating full output. [8f7d12e](https://github.com/morluto/rea/commit/8f7d12e30d63f3d3b4cfe2a90336362841438f56), [#4962](https://github.com/morluto/rea/pull/4962), [#5349](https://github.com/morluto/rea/pull/5349), [#5524](https://github.com/morluto/rea/pull/5524), [#5815](https://github.com/morluto/rea/pull/5815), [#6128](https://github.com/morluto/rea/pull/6128), [#6907](https://github.com/morluto/rea/pull/6907), [#4810](https://github.com/morluto/rea/issues/4810), [#5499](https://github.com/morluto/rea/issues/5499), [#5802](https://github.com/morluto/rea/issues/5802), [#6112](https://github.com/morluto/rea/issues/6112), [#6900](https://github.com/morluto/rea/issues/6900). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Compact results:** Use retained Frida/GDB/objdump summaries, omit redundant Ghidra raw results, and defer the analysis runtime during MCP catalog discovery. [#5661](https://github.com/morluto/rea/pull/5661), [#5614](https://github.com/morluto/rea/pull/5614), [#5355](https://github.com/morluto/rea/pull/5355), [#4424](https://github.com/morluto/rea/pull/4424), [#5640](https://github.com/morluto/rea/issues/5640), [#5584](https://github.com/morluto/rea/issues/5584), [#5337](https://github.com/morluto/rea/issues/5337). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @leeguooooo.
|
||||
- **Analysis work:** Reuse Hopper literal-string records, JavaScript workers and captured prefix selections, Interface Builder validation and plist scalar schemas; bound stable reads to one snapshot buffer and avoid decoding omitted GGUF strings. [#5017](https://github.com/morluto/rea/pull/5017), [#5819](https://github.com/morluto/rea/pull/5819), [31de177](https://github.com/morluto/rea/commit/31de177404461d122761d8243abe4fd24564f4da), [#5885](https://github.com/morluto/rea/pull/5885), [85e38ac](https://github.com/morluto/rea/commit/85e38ac199dff15e8c43343d9170db7e0e46d284), [#6966](https://github.com/morluto/rea/pull/6966), [#5781](https://github.com/morluto/rea/pull/5781), [#5010](https://github.com/morluto/rea/issues/5010), [#5855](https://github.com/morluto/rea/issues/5855), [#6954](https://github.com/morluto/rea/issues/6954), [#5774](https://github.com/morluto/rea/issues/5774). Thanks @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Native scheduling:** Batch modern IDA callee lookups, schedule decompilation reads on demand, index NativeAOT type closure, and retain call-path ancestry once per visited node. [0c6fed9](https://github.com/morluto/rea/commit/0c6fed90b74c75a9fd23d1ec1b510a7274f85887), [06fe7ee](https://github.com/morluto/rea/commit/06fe7eecbc8c387414b4c1aa0d88ad6314f54f0c), [ce61680](https://github.com/morluto/rea/commit/ce61680c04c95a9cdfad5335632a99542af26d37), [5e4ec0f](https://github.com/morluto/rea/commit/5e4ec0fa1b678aa3bf20704ec9110224ba8c1676).
|
||||
- **Selected processing:** Prepare CLI positional schemas only for the selected command, separate entropy reads from report resolution, prune typed-array reference escapes, index state-write cutoffs, decode only selected replay query values, and skip payload text during WAT framing. [315b234](https://github.com/morluto/rea/commit/315b23403f0819967878cee418c3f9cb4b1a69d9), [#6950](https://github.com/morluto/rea/pull/6950), [#5139](https://github.com/morluto/rea/pull/5139), [#5350](https://github.com/morluto/rea/pull/5350), [96ea32b](https://github.com/morluto/rea/commit/96ea32b7e4e28d3c9cd79a1de4b8e1ba3b4244e9), [70835d3](https://github.com/morluto/rea/commit/70835d3d5f030deadf606e1c79c70a60883e50c9), [#5766](https://github.com/morluto/rea/pull/5766), [#6947](https://github.com/morluto/rea/issues/6947), [#5760](https://github.com/morluto/rea/issues/5760). Thanks @galleonli, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
|
||||
### Fixes
|
||||
|
||||
- **IPS report fields:** Preserve the recorded terminating process and PID, termination flags, parent PID, and build/platform metadata. [#7079](https://github.com/morluto/rea/pull/7079), [#7078](https://github.com/morluto/rea/issues/7078). Thanks @rodricksz4h5.
|
||||
- **Ghidra language selection:** Accept stock language IDs containing `LEBE` or slash-separated processor names, and report conflicting DOL language overrides as configuration errors naming `REA_GHIDRA_LANGUAGE_ID`. [#7070](https://github.com/morluto/rea/pull/7070), [#7072](https://github.com/morluto/rea/issues/7072), [#7081](https://github.com/morluto/rea/pull/7081), [#7082](https://github.com/morluto/rea/issues/7082). Thanks @Adolanium.
|
||||
- **Setup and Hopper:** Preserve the Linux embedding runtime, supported x86_64 host restriction, readiness-based provider choices, orderly demo-helper shutdown, and installed apt dependency versions. Keep cleanup and skill publication ownership through retries. [#5041](https://github.com/morluto/rea/pull/5041), [#3422](https://github.com/morluto/rea/pull/3422), [f5d1012](https://github.com/morluto/rea/commit/f5d1012fabfe99af82523c3078ce9d93eb3780c1), [#4966](https://github.com/morluto/rea/pull/4966), [#6974](https://github.com/morluto/rea/pull/6974), [211e353](https://github.com/morluto/rea/commit/211e3533b34f81ae0910ad4f53f2a3d3c8587d3e), [#6920](https://github.com/morluto/rea/pull/6920), [#5015](https://github.com/morluto/rea/issues/5015), [#3419](https://github.com/morluto/rea/issues/3419), [#4921](https://github.com/morluto/rea/issues/4921), [#6975](https://github.com/morluto/rea/issues/6975). Thanks @Abel-I-Akhigbe, @Adolanium, @DavidFB-creator, @DeryFerd, @gensokyo-dev, @kaoru0822-kitauji, @sx4im.
|
||||
- **Client setup:** Treat empty APPDATA as unset, distinguish MSIX Claude Desktop process views, preserve aliased Hermes server tables, reuse client dialect/alias bindings, and materialize the pinned package-runner cache before the first start. [#5126](https://github.com/morluto/rea/pull/5126), [#4903](https://github.com/morluto/rea/pull/4903), [#3785](https://github.com/morluto/rea/pull/3785), [b7eeded](https://github.com/morluto/rea/commit/b7eededf5af5061a834f93f57f2c888acf870344), [#7032](https://github.com/morluto/rea/pull/7032), [#5104](https://github.com/morluto/rea/issues/5104), [#4278](https://github.com/morluto/rea/issues/4278), [#3320](https://github.com/morluto/rea/issues/3320), [#7027](https://github.com/morluto/rea/issues/7027). Thanks @CmsChase, @coygeek, @myso-kr, @pablodiazjorge, @vaidehi-snk, @wxj006007.
|
||||
- **Target admission:** Preserve thin arm64_32 Mach-O support, distinguish JVM identity before FAT admission, validate FAT format/CPU identity, preserve original DOS header padding, and reject NULs in local objdump/rizin paths. [#5788](https://github.com/morluto/rea/pull/5788), [6843dbe](https://github.com/morluto/rea/commit/6843dbea9bce4709ee2e8bf46c38f0bbf640716b), [497037e](https://github.com/morluto/rea/commit/497037e6e3975cbb37cfd89ea0ad300faa2a4769), [#7013](https://github.com/morluto/rea/pull/7013), [#3600](https://github.com/morluto/rea/pull/3600), [#5713](https://github.com/morluto/rea/issues/5713), [#6510](https://github.com/morluto/rea/issues/6510). Thanks @coygeek, @dhruvac29, @kaoru0822-kitauji, @Lew1sWong, @qorvr.
|
||||
- **Mach-O and dylibs:** Preserve container framing, facts when dyld rejects a missing UUID, verified-signature snapshot identity, unsigned diagnostics, absolute paths inside the selected root, legacy platform rules, lazy-load payloads, and reasons for rejecting present dylibs. [cf3288d](https://github.com/morluto/rea/commit/cf3288d8cf7a09685acf89103b36c32359be56fa), [2b263a7](https://github.com/morluto/rea/commit/2b263a7fe9e9d4c185c7d6af6b4e67b21a73af30), [eb7acc7](https://github.com/morluto/rea/commit/eb7acc70bf06f119e8639e30a78081d5ba82b0bf), [1502cfb](https://github.com/morluto/rea/commit/1502cfbc8ca0d1d1d86eebf334018dfb418bbac0), [90062c0](https://github.com/morluto/rea/commit/90062c092228dea9d33fadc0d7e1b968105b8b2c), [#5650](https://github.com/morluto/rea/pull/5650), [3b67b36](https://github.com/morluto/rea/commit/3b67b362aa2cd79eac5af57d0965e81a7d0b8c2b), [#5551](https://github.com/morluto/rea/issues/5551). Thanks @coygeek.
|
||||
- **Native facts:** Admit direct dispatch-metadata providers; preserve record-local Objective-C omissions, snapshot profiles, class-import binding identity, stripped class labels, version-like names, and qualified Swift protocol identities. [50b6975](https://github.com/morluto/rea/commit/50b6975d861069e2c6cfa202780621449f863802), [9eec57b](https://github.com/morluto/rea/commit/9eec57b469087fe9bff3d29dcc02f47074159e91), [0ba4d6f](https://github.com/morluto/rea/commit/0ba4d6f43ad610dacc861a1548be1a5c4712cd61), [4d7de00](https://github.com/morluto/rea/commit/4d7de00d4cc9a95ec2b7a3123b04e01db2daafa0), [22c84ce](https://github.com/morluto/rea/commit/22c84ce64aa6c7601aa81af86d4117b5a4c2feac), [4852c83](https://github.com/morluto/rea/commit/4852c830af1e3b9df437235b7a8fa8970a6d1c59), [aa9043f](https://github.com/morluto/rea/commit/aa9043f6b61c453e4735795cfe6a559bf6b6cc16).
|
||||
- **Apple resources:** Preserve independent plist scalars, processing instructions, nonstandard table widths, unsupported NIB connector coverage, and strict IPS JSON surrogate validation; classify Interface Builder objects by their declared superclass chains. [fd728f2](https://github.com/morluto/rea/commit/fd728f271e7de7bcfa63890b573715720d5f948c), [#5806](https://github.com/morluto/rea/pull/5806), [4526862](https://github.com/morluto/rea/commit/4526862728af01f1135714079c0d8a8c754eb364), [14cd78c](https://github.com/morluto/rea/commit/14cd78c3224edcf79fca152915077112f547b9ed), [#6990](https://github.com/morluto/rea/pull/6990), [#5805](https://github.com/morluto/rea/issues/5805), [#5228](https://github.com/morluto/rea/pull/5228), [#5218](https://github.com/morluto/rea/issues/5218). Thanks @coygeek, @DeryFerd, @kaoru0822-kitauji, @Mangolycheematcha, @mhamzasakhi, @Shreyash021104.
|
||||
- **Archives and files:** Complete positional ZIP reads, preserve bounded ASAR signatures, report stable symlink loops as resolution failures, retain target applicability, and report unknown traversal coverage. Retain reported integrity contradictions for members present in only one compared artifact. [#4414](https://github.com/morluto/rea/pull/4414), [e8da7e1](https://github.com/morluto/rea/commit/e8da7e1edd075feec0e6beedec93d5645fed8125), [#5571](https://github.com/morluto/rea/pull/5571), [#5763](https://github.com/morluto/rea/pull/5763), [#5867](https://github.com/morluto/rea/pull/5867), [#5521](https://github.com/morluto/rea/issues/5521), [#5731](https://github.com/morluto/rea/issues/5731), [#5837](https://github.com/morluto/rea/issues/5837), [#7076](https://github.com/morluto/rea/pull/7076). Thanks @Afloat16, @coygeek, @jdotc1, @kaoru0822-kitauji, @WenyuChiou.
|
||||
- **Extraction and file ownership:** Restrict destination-case recovery to owned collisions, retain physical-read failures through late cancellation/cleanup, preserve cleanup capabilities for retry, and retain independent firmware extraction failures. [932fbb7](https://github.com/morluto/rea/commit/932fbb7d84a9a995b9aee3c1be7da05a8c4d7bbb), [9499667](https://github.com/morluto/rea/commit/949966712e68d8857a50a165ecddf04c2d7e6d3b), [56679dd](https://github.com/morluto/rea/commit/56679dd2ae31dda58e181f8bf99775014f7bb82b), [b3c912a](https://github.com/morluto/rea/commit/b3c912ab79ad147a37b6cfbe1552a1a5eecec629), [7298bd0](https://github.com/morluto/rea/commit/7298bd08d134860b84b4073f6703021916a5a4c5).
|
||||
- **Processes and cancellation:** Retain partial samples and shutdown gaps, unresolved descendants and caller abort reasons; bind cleanup errors to their owners, propagate comparison contradictions, and report no normal exit code for signalled targets. [ba866a2](https://github.com/morluto/rea/commit/ba866a26ecbe179a717e746cbc0d1605ae616ad5), [#5270](https://github.com/morluto/rea/pull/5270), [#6144](https://github.com/morluto/rea/pull/6144), [a721c42](https://github.com/morluto/rea/commit/a721c42c07dc674c32ab939627355b8a1bee97a5), [#6298](https://github.com/morluto/rea/pull/6298), [#6486](https://github.com/morluto/rea/pull/6486), [#5935](https://github.com/morluto/rea/issues/5935), [#5944](https://github.com/morluto/rea/issues/5944), [#6100](https://github.com/morluto/rea/issues/6100), [#6101](https://github.com/morluto/rea/issues/6101), [#6110](https://github.com/morluto/rea/issues/6110), [#6113](https://github.com/morluto/rea/issues/6113), [#6121](https://github.com/morluto/rea/issues/6121), [#6131](https://github.com/morluto/rea/issues/6131), [#6132](https://github.com/morluto/rea/issues/6132), [#6218](https://github.com/morluto/rea/issues/6218), [#6445](https://github.com/morluto/rea/issues/6445). Thanks @agorevski, @coygeek, @gongfpp, @kwakhyun, @mhamzasakhi.
|
||||
- **Provider cleanup:** Preserve Windows native handles until close is confirmed, native/Inspector/Electron cleanup ownership, Kuna probe identity and cleanup, IDA registration configuration, and Hopper preparation ownership. [#5772](https://github.com/morluto/rea/pull/5772), [87ab054](https://github.com/morluto/rea/commit/87ab054ea384b3fcbae2998b1f94685edcd5f624), [ff78165](https://github.com/morluto/rea/commit/ff78165978de9dbcf9fa16c2acf74bf85a69dc44), [04ae07a](https://github.com/morluto/rea/commit/04ae07ab8c9ae1a6697b6ad15489bbde515da8b5), [dd0015a](https://github.com/morluto/rea/commit/dd0015aaf156c5701ef9598bcc6c885e6b8da803), [b231c28](https://github.com/morluto/rea/commit/b231c28cf64c2f799ada99343f8f21e0e6acb15e), [#4505](https://github.com/morluto/rea/issues/4505), [#5616](https://github.com/morluto/rea/issues/5616).
|
||||
- **Provider diagnostics:** Give Ghidra restart recovery, stable Homebrew aliases, unsupported-host results for Linux-only integrations, correct IDA regex/readiness handling, and clear diagnostics when Windows refuses a verified native addon. [#5356](https://github.com/morluto/rea/pull/5356), [7c6142e](https://github.com/morluto/rea/commit/7c6142efdc19a42c148d0e8c96b69d18e63c7f87), [#5793](https://github.com/morluto/rea/pull/5793), [#4862](https://github.com/morluto/rea/pull/4862), [99ccfb6](https://github.com/morluto/rea/commit/99ccfb6e709ac4bc0afe176315280abde9ebcb9f), [#6832](https://github.com/morluto/rea/pull/6832), [#5730](https://github.com/morluto/rea/issues/5730), [#4860](https://github.com/morluto/rea/issues/4860), [#5723](https://github.com/morluto/rea/issues/5723). Thanks @coygeek, @dark13b, @gensokyo-dev, @jgt87, @jpmorris41488, @Lew1sWong.
|
||||
- **MCP results:** Preserve protocol envelopes on oversized errors, enforce and advertise absolute Evidence paths, retain typed output-filter errors and selected JSON paths, and give client/provider timeout recovery. Compact advertisements now normalize reported enum and union shapes for strict gateway translation. [562dab0](https://github.com/morluto/rea/commit/562dab00c5cbf76a9b2695bad93792eccdc9a49d), [3cf6206](https://github.com/morluto/rea/commit/3cf6206614e65b5ebb6284a5709090dfb8f9aaee), [0e83d92](https://github.com/morluto/rea/commit/0e83d92c583ae36afe00b2c23dd2a0f1d025cbca), [4714c7e](https://github.com/morluto/rea/commit/4714c7eaa3d1d8b6a5df0aff586e844b9d9ab06d), [1bfa60a](https://github.com/morluto/rea/commit/1bfa60ac20e1a344bcf3132c08d15b6860b05c61), [#5608](https://github.com/morluto/rea/pull/5608), [#3806](https://github.com/morluto/rea/pull/3806), [#5598](https://github.com/morluto/rea/issues/5598), [#3800](https://github.com/morluto/rea/issues/3800). Thanks @gensokyo-dev, @Hahaa909, @wang1xiang.
|
||||
- **Browser lifecycles:** Retain historical outcomes through cleanup, stop late navigation before session release, stop expired work, observe aborted fetch settlement, and mark later scenario actions skipped after failure or cancellation. [2e5171e](https://github.com/morluto/rea/commit/2e5171e391b83f8a467d0861fd452c1f78621fd8), [599fdac](https://github.com/morluto/rea/commit/599fdac029912f96c511f859660483405d2de26f), [b2fad36](https://github.com/morluto/rea/commit/b2fad362f2b37f81dc9a6f784f4521e4ca24d41e), [13a8640](https://github.com/morluto/rea/commit/13a864020414577006dce58d7fdaa877ddd0cb52), [#6023](https://github.com/morluto/rea/pull/6023), [#5943](https://github.com/morluto/rea/issues/5943). Thanks @coygeek, @kaoru0822-kitauji.
|
||||
- **Browser comparisons:** Preserve frame identity, default origin scope, unknown body/shape coverage, correspondence and uncertainty, and independent retained-observation budgets. [#4040](https://github.com/morluto/rea/pull/4040), [#4413](https://github.com/morluto/rea/pull/4413), [#4476](https://github.com/morluto/rea/pull/4476), [a0c22ff](https://github.com/morluto/rea/commit/a0c22ff0125b8af7612db54d3708413e2bfecb25), [a0fcc60](https://github.com/morluto/rea/commit/a0fcc60d89eb0cc09b08181eb9f961acfd07d8e9), [#3878](https://github.com/morluto/rea/issues/3878), [#3910](https://github.com/morluto/rea/issues/3910), [#3941](https://github.com/morluto/rea/issues/3941), [#4385](https://github.com/morluto/rea/issues/4385), [#4477](https://github.com/morluto/rea/issues/4477). Thanks @Adolanium, @coygeek, @eco-null, @gensokyo-dev, @kaoru0822-kitauji, @XY-code1.
|
||||
- **Electron observations:** Preserve listener callback ownership and count registrations by source callsite; block trash calls in the active hook, retain window handles through cancelled selection, and preserve cancellation during finalization. [dfe8378](https://github.com/morluto/rea/commit/dfe8378cf5a727e3d93d47a5986233175c6ef898), [5064007](https://github.com/morluto/rea/commit/506400748eb1d6c674f0cf51330073929c45f939), [816a9dc](https://github.com/morluto/rea/commit/816a9dc98886e6d824cd8ae88b68731b161d5dda), [1707d17](https://github.com/morluto/rea/commit/1707d171d750955bf6387e028e378183b360ffe3), [4e2a566](https://github.com/morluto/rea/commit/4e2a566d6c7e053ecbbee1b684311ed31ea163bf).
|
||||
- **JavaScript source maps:** Keep unmapped and missing mappings unknown, preserve partial indexed-map facts and section paths, propagate preparation capability errors, and retain observed worker failures through cancellation. [a4dbb36](https://github.com/morluto/rea/commit/a4dbb36d8ea83e9f94dd289ac1a2d220d2c72c9b), [2b2e362](https://github.com/morluto/rea/commit/2b2e362a00fc5ba2f93fa6c62e2d8780072d1b4b), [498a667](https://github.com/morluto/rea/commit/498a667234b0140c1cfa43c819b35454ab11d14e), [#6149](https://github.com/morluto/rea/pull/6149), [#6214](https://github.com/morluto/rea/pull/6214), [f01dcca](https://github.com/morluto/rea/commit/f01dccab77f712fb4ac936d0a6e92a70d12f0789), [#6082](https://github.com/morluto/rea/issues/6082), [#6061](https://github.com/morluto/rea/issues/6061). Thanks @coygeek, @diyasharma05, @kaoru0822-kitauji.
|
||||
- **JavaScript semantics:** Preserve overflowing timer uncertainty, unavailable-scope comparisons, directory-form URL/ESM path intent, and artifact confinement. Replaced globals lose built-in effects; erased imports no longer become runtime bindings. Treat a known truthy `fork()` `silent` option as piped stdio unless a string/array `stdio` override takes precedence; unresolved options stay dynamic. [#3456](https://github.com/morluto/rea/pull/3456), [8d198d6](https://github.com/morluto/rea/commit/8d198d6ec707402d75276f516e0090d71767b7e4), [#4646](https://github.com/morluto/rea/pull/4646), [#2700](https://github.com/morluto/rea/pull/2700), [#4565](https://github.com/morluto/rea/pull/4565), [#6994](https://github.com/morluto/rea/pull/6994), [#3400](https://github.com/morluto/rea/issues/3400), [#3882](https://github.com/morluto/rea/issues/3882), [#3367](https://github.com/morluto/rea/issues/3367), [#7083](https://github.com/morluto/rea/pull/7083), [#7084](https://github.com/morluto/rea/issues/7084). Thanks @Adolanium, @amyanger, @coygeek, @gensokyo-dev, @HawkForge, @rohitg00, @viniciusdsandrade.
|
||||
- **JavaScript routes and imports:** Treat path data as route candidates, preserve route flag and lexical RPC identities, package-main trailing-slash selection, catch aliases, and factory parameter ownership. [#6360](https://github.com/morluto/rea/pull/6360), [38ecc02](https://github.com/morluto/rea/commit/38ecc0248e1993d25a2f7dd350024484cce797bb), [#4219](https://github.com/morluto/rea/pull/4219), [21097b1](https://github.com/morluto/rea/commit/21097b192b1ab94647e3446b1a2a4dd00e924d5e), [f774dc4](https://github.com/morluto/rea/commit/f774dc420b3befe2b8e68971b4116da2d62fc800), [#6349](https://github.com/morluto/rea/issues/6349), [#4189](https://github.com/morluto/rea/issues/4189). Thanks @coygeek, @hemv-857, @kaoru0822-kitauji, @Nikhi00718.
|
||||
- **Managed/IL2CPP:** Preserve extension-only scope uncertainty and type discovery after member limits, nested names, ReadyToRun/declaration facts, selected type scope, valid shared-suffix strings, and truncated IL2CPP metadata prefixes. [#5674](https://github.com/morluto/rea/pull/5674), [#5312](https://github.com/morluto/rea/pull/5312), [87fa1b6](https://github.com/morluto/rea/commit/87fa1b613d2b6bda1722855a00f1390f27731b64), [bc2d494](https://github.com/morluto/rea/commit/bc2d494030efd1ccabaeaa0bd49226aa87c378a2), [1f0eb22](https://github.com/morluto/rea/commit/1f0eb228e50908ee2a2aaa72d329178ae1a2cdbd), [4c5fae5](https://github.com/morluto/rea/commit/4c5fae5179dd54c4a0d427cd2a115b9c887e3841), [b3aadaf](https://github.com/morluto/rea/commit/b3aadaf474e59b33e5ff465236008a67eccefb87), [#5467](https://github.com/morluto/rea/issues/5467), [#5146](https://github.com/morluto/rea/issues/5146). Thanks @coygeek, @eco-null, @elricfd, @umi008.
|
||||
- **Format boundaries:** Preserve independent facts across malformed inputs in model, GPU, WASM, Rust, Android, and SQLite inspection; report optional metadata warnings without discarding core results. [eb41f35](https://github.com/morluto/rea/commit/eb41f358cf83d5626f09de68332f9718f9aa6482), [#5789](https://github.com/morluto/rea/pull/5789), [00a9e95](https://github.com/morluto/rea/commit/00a9e95de91a4bd4d2a41f8aee704a53fa575750), [fc63fe8](https://github.com/morluto/rea/commit/fc63fe8682654086261540dcf3c59afc36c7d58f), [39a91d4](https://github.com/morluto/rea/commit/39a91d4db96ca13224c25936330ea84fe78970e5), [6c6e2ad](https://github.com/morluto/rea/commit/6c6e2ad62b20f810b54f5a567e66fc55241514b2), [929a0a7](https://github.com/morluto/rea/commit/929a0a76c870d9991e015dde58308fd9df0280b8), [d137a4c](https://github.com/morluto/rea/commit/d137a4cdde0049de967f1d5cd429ed9e1f7d0a10), [ef475fc](https://github.com/morluto/rea/commit/ef475fcf8b0601c61052b7091568ba574546417d), [#5749](https://github.com/morluto/rea/issues/5749). Thanks @coygeek, @gensokyo-dev.
|
||||
- **Capture decoding:** Preserve malformed protobuf trailer facts and wire/schema identity, HTTP/2 URI and UTF-8 failure facts, WebMCP property coverage, and offline capture availability without a browser provider. Report invalid record ordinals as selection errors with actual bounds where available, and trim surrounding SP/HTAB from gRPC-web trailer values while preserving internal whitespace. [9cf7a7c](https://github.com/morluto/rea/commit/9cf7a7cb0d3c72a1121fde280cb11bf911af8055), [b3c3b21](https://github.com/morluto/rea/commit/b3c3b21e43cab4e71b460d08829d8c6e4e9b9c21), [4cb1f1d](https://github.com/morluto/rea/commit/4cb1f1d820df4498ee464bcf72a5f47d40cf8dab), [51e5938](https://github.com/morluto/rea/commit/51e593877642ab42fb5bf25812f4a4110ee7ac26), [d23be5c](https://github.com/morluto/rea/commit/d23be5c0898c39c9f0c54e2e0b43ee746d92d316), [#6725](https://github.com/morluto/rea/pull/6725), [#3947](https://github.com/morluto/rea/issues/3947), [#7067](https://github.com/morluto/rea/pull/7067), [#7068](https://github.com/morluto/rea/issues/7068), [#7069](https://github.com/morluto/rea/pull/7069), [#7071](https://github.com/morluto/rea/issues/7071). Thanks @Adolanium, @coygeek, @Sriram-PR.
|
||||
- **Evidence and errors:** Authenticate ledger Evidence before committing; preserve parent-schema diagnostics, canonical correction details and owned cleanup outcomes; identify the rejected function-Evidence side/kind. Fixed allowances now name their actual limits. [0407bbf](https://github.com/morluto/rea/commit/0407bbfbe65e8fa475e8bb0c78c30170e2c2f55e), [#6361](https://github.com/morluto/rea/pull/6361), [1629266](https://github.com/morluto/rea/commit/1629266e3c08c90fed6bba5383528af7986a12b7), [#6969](https://github.com/morluto/rea/pull/6969), [#5860](https://github.com/morluto/rea/pull/5860), [#6034](https://github.com/morluto/rea/issues/6034), [#5932](https://github.com/morluto/rea/issues/5932), [#5847](https://github.com/morluto/rea/issues/5847). Thanks @AyoubElleuch, @coygeek, @dhruvac29, @kaoru0822-kitauji.
|
||||
- **ADB and firmware recovery:** Report an unparsed push checksum as unverified, normalize absent installer-package sentinels, describe extraction-relative Unblob links, and avoid recommending a budget increase at its fixed ceiling. [#6010](https://github.com/morluto/rea/pull/6010), [#4073](https://github.com/morluto/rea/pull/4073), [#4573](https://github.com/morluto/rea/pull/4573), [#6971](https://github.com/morluto/rea/pull/6971), [#5924](https://github.com/morluto/rea/issues/5924), [#4175](https://github.com/morluto/rea/issues/4175). Thanks @coygeek, @Ganesh2325, @kaoru0822-kitauji, @Rinkia, @XiaoBinGan, @yankikucuk.
|
||||
- **CLI formatting:** Share Markdown report formatting, exclude inherited row values, and preserve arrays of empty objects as JSON. Render Markdown table headers and cells literally while keeping intraword underscores readable. [b784912](https://github.com/morluto/rea/commit/b784912811fd10ac9dc425696dc0e9af632afa34), [#6984](https://github.com/morluto/rea/pull/6984), [#7058](https://github.com/morluto/rea/pull/7058), [#7057](https://github.com/morluto/rea/issues/7057), [#7073](https://github.com/morluto/rea/pull/7073). Thanks @Plastic-time, @ShuaixinHuang, @viniciusdsandrade.
|
||||
- **Format identity:** Preserve Python marshal scalar/decode limits, empty optional BEAM chunks, PE resource branch identities and certificate alignment, unsupported compressed ELF notes, and codec-framed GPU bundle extents. [0603223](https://github.com/morluto/rea/commit/06032237c6755e3dd5f9a8db732c89958b238cc7), [c46064c](https://github.com/morluto/rea/commit/c46064c3f6eaba121c5de7b17390bfdecdd132ef), [66671e8](https://github.com/morluto/rea/commit/66671e874888e98f7e2b27e68a7c6fb320bbb827), [03077d5](https://github.com/morluto/rea/commit/03077d54afc49613db4e38f70c506bc60f5223aa), [#6721](https://github.com/morluto/rea/pull/6721), [9fb8c11](https://github.com/morluto/rea/commit/9fb8c114a03ad60e57c11586967c6df845a3f97e), [f6a1e2c](https://github.com/morluto/rea/commit/f6a1e2c892dc23c7583c3d03af9edb01632ff5a5), [#6694](https://github.com/morluto/rea/issues/6694). Thanks @coygeek, @Deepak-Kambala.
|
||||
- **WASM and APK metadata:** Preserve ambiguous WASM identity and partial facts, glue URL query/fragment text across line terminators, APK signer identity, and raw issuer identity when comparison rules are unknown. [af8f0ba](https://github.com/morluto/rea/commit/af8f0badf64fd3546392b6aa4625777904b2f051), [#6802](https://github.com/morluto/rea/pull/6802), [f783f0f](https://github.com/morluto/rea/commit/f783f0f1e82472cdddd4de2156d974f729e332a6), [115a78f](https://github.com/morluto/rea/commit/115a78fe241276c34917b496621954c5f29bfc15), [#6639](https://github.com/morluto/rea/issues/6639). Thanks @coygeek, @gensokyo-dev, @gongfpp.
|
||||
- **Source identity:** Preserve Next route spelling, module grammar and loader request identity, directory package-main behavior, accessor effects, exact source targets, and resolver-discovery versus module-loading distinctions. [ddd7ac7](https://github.com/morluto/rea/commit/ddd7ac7b846b48191ead1f7dcd0e3bd735468fa8), [9974512](https://github.com/morluto/rea/commit/99745126009b15060d71c38809418858a07d5afa), [f2e5fd1](https://github.com/morluto/rea/commit/f2e5fd148a85fa737302dd194a33172c4561093e), [baf1678](https://github.com/morluto/rea/commit/baf16784af3592534687052431c145daef55cfd1), [6e324b2](https://github.com/morluto/rea/commit/6e324b224405f2befd953234f6dc210cf72ad50a), [75daf0c](https://github.com/morluto/rea/commit/75daf0c7df108f04ab41f34aec1a31d4a7ceb322), [f91b7f8](https://github.com/morluto/rea/commit/f91b7f8009c9af27a5e0148e3b07d20bfe3b7a2a).
|
||||
- **Graphs and selected facts:** Preserve JavaScript graph omissions/version facts, Electron native-loader facts, web JSON identity/coverage, module-path selection, feature-trace qualifications, selected native paths, endpoint context, and mixed Swift demangler outcomes. [30db499](https://github.com/morluto/rea/commit/30db499db310f073d9bf3d3acfd77985e1cb7e2e), [5a7bddb](https://github.com/morluto/rea/commit/5a7bddb3aeb0345854e76b79d529aed1c40e2054), [79805f2](https://github.com/morluto/rea/commit/79805f244a1ec0850f3109929168a9c8ebb1b908), [f719dc4](https://github.com/morluto/rea/commit/f719dc46c51a55b3926eec9659d06e7dfb95132b), [7784fdb](https://github.com/morluto/rea/commit/7784fdbad78f8b384d513f0e7dae8e983cd48ef8), [76500f1](https://github.com/morluto/rea/commit/76500f12b54e07b1ea700a403b2593890d63b42a), [676d29f](https://github.com/morluto/rea/commit/676d29f9eef794cea668f5f09f0ff510c848a513), [edf7b3d](https://github.com/morluto/rea/commit/edf7b3d32c3248f221e27f7ce5a3d34e8590285b), [93de7bc](https://github.com/morluto/rea/commit/93de7bc7e324a761162a31625b838ba0d4e3824d).
|
||||
- **Managed boundaries:** Honor native platform symbol lookup, consume resolved IL2CPP ELF declarations, and attribute malformed comparison inputs to the supplied argument. [64fa1db](https://github.com/morluto/rea/commit/64fa1db1309218283df19f3ee4dba61f5fbd2a67), [87745af](https://github.com/morluto/rea/commit/87745af52ab0ccd3a172c2534c516d41b0095035), [96ad94f](https://github.com/morluto/rea/commit/96ad94f69eda5d68c7335ff775b6927b1376fd2b).
|
||||
- **Provider and worker recovery:** Preserve output loss after successful command exit, physical refusal diagnostics, bounded worker control framing, JavaScript worker-phase/budget attribution, admitted ambiguity choices, and selected EVM limiter availability. Scope objdump debug-link limitations to DWARF requests and keep process-capture preparation nonexecuting. [e639edf](https://github.com/morluto/rea/commit/e639edf6204c05285d96840478926592daf95876), [171b33b](https://github.com/morluto/rea/commit/171b33b2c4defc6cbea49545bedf4272b72d1ea2), [34563fc](https://github.com/morluto/rea/commit/34563fcc9bd92af30668820dcac5635e27181094), [3f9357b](https://github.com/morluto/rea/commit/3f9357b5990d166a38440f56f439452266650093), [cec8f1e](https://github.com/morluto/rea/commit/cec8f1e8f305d6332272457a6ee419d3c5fb6359), [a74d859](https://github.com/morluto/rea/commit/a74d859a19122fb8f1282b3632d320997b1caae0), [c284168](https://github.com/morluto/rea/commit/c284168c4b1cd214ce7f504d43da28528d82d603), [4fb936f](https://github.com/morluto/rea/commit/4fb936fe86bb7a07f94f819ae8510301f2ef69c9).
|
||||
- **Setup and limits:** Retain unknown environment-budget facts, selected-skill/maintenance uncertainty, Hopper URL admission and bookmark verification, and listener failure recovery that matches the system cause. [a718037](https://github.com/morluto/rea/commit/a7180370332350bc0f998fbd643888221759570e), [d1fbebb](https://github.com/morluto/rea/commit/d1fbebb578538dc94fd5e376e6e1959218efa835), [60accad](https://github.com/morluto/rea/commit/60accad2de8f2862fe91901502fe26cc5390fb36), [fd7da77](https://github.com/morluto/rea/commit/fd7da77fc02dcfaf7d1bc9f2fb240d71cdd27c59), [0c440bf](https://github.com/morluto/rea/commit/0c440bfc07f3cb343e2e836d27cccaed539cfe9c).
|
||||
- **State and Evidence:** Retain export tarball ownership and SQLite primary outcomes through cleanup retry, count emitted SQLite schema names across encodings, scope display refresh to matching snapshot bindings, and report retained-result validation failures consistently. Reconstruction uses one effective completion scope and kind-based labels when names are blank. [ab58edc](https://github.com/morluto/rea/commit/ab58edcdf50b4bd687900ee2dc362ecc5c3dce15), [1006714](https://github.com/morluto/rea/commit/100671427105791a5c5913a918b63a394f6bc83e), [a47788a](https://github.com/morluto/rea/commit/a47788a6a4abb067273f954275a9d0a269adf1e7), [8aa40e9](https://github.com/morluto/rea/commit/8aa40e9a61155cff8b6f3c138c601bf805b20653), [b8c1b45](https://github.com/morluto/rea/commit/b8c1b45316882beeab4a9de49381d94e131fab0f), [a35b933](https://github.com/morluto/rea/commit/a35b9335a0124674ee5e3abe8c075f1fff6b4fa6), [#6137](https://github.com/morluto/rea/pull/6137), [#6126](https://github.com/morluto/rea/issues/6126). Thanks @carlogiovannineri-rgb, @mohithhhh.
|
||||
- **Capture coverage:** Name browser bundle budgets and deadline recovery, separate domain readiness from worker attribution, attribute rendered-frame omissions to their producing stage, share offline-capture host admission with discovery, preserve selected signal-file semantics, and retain short Flutter GNU build IDs. [#6036](https://github.com/morluto/rea/pull/6036), [ba78133](https://github.com/morluto/rea/commit/ba781330d6d28b22d15498e63c144fb40e086156), [5934387](https://github.com/morluto/rea/commit/5934387f4185129efe9dc8b7338d8bc3fa5b25bd), [ad629b2](https://github.com/morluto/rea/commit/ad629b2e31c3a9495b27179360b508fa021fe86a), [928ba76](https://github.com/morluto/rea/commit/928ba76882bfd216dde8ed79118789b2c3aad4f9), [678c4cf](https://github.com/morluto/rea/commit/678c4cfa68e2fee12f620075e12dc87d9b1195b0).
|
||||
- **Additional boundary facts:** Retain parser format/query occurrence identity and independent malformed-input facts, unresolved nested Swift identifiers, and completed signing-capture coverage; describe crash documents without assuming a provider. [5f702dd](https://github.com/morluto/rea/commit/5f702dd6ccd9e8e505e224854b91678439c3b41f), [99d2d7a](https://github.com/morluto/rea/commit/99d2d7ad22bcbd780919760008aba6a88e7d1077), [90e433d](https://github.com/morluto/rea/commit/90e433df06b3f83417176b8db1b0a905f9df1d3a), [#5548](https://github.com/morluto/rea/pull/5548), [ee12d51](https://github.com/morluto/rea/commit/ee12d517a25d34853d40e31fc039c7d1576c60e1), [#5538](https://github.com/morluto/rea/issues/5538). Thanks @coygeek, @ShuaixinHuang.
|
||||
- **Documentation:** Add the Romanian README. [#7026](https://github.com/morluto/rea/pull/7026). Thanks @An3treiu.
|
||||
|
||||
Full contribution record: [1740 commits and 497 merged PRs](https://github.com/morluto/rea/blob/rea-agents-6.8.0/docs/releases/6.8.0.contributions.json).
|
||||
|
||||
## [6.7.0](https://github.com/morluto/rea/compare/rea-agents-6.6.0...rea-agents-6.7.0) (2026-10-18)
|
||||
|
||||
### Highlights
|
||||
@@ -132,8 +30,8 @@ Full contribution record: [4 commits and 1 merged PR](https://github.com/morluto
|
||||
- **Kuna analysis:** Use a caller-installed Kuna for procedure listings, pseudocode, assembly, call relationships, exact-address xrefs and cached string searches. Project exports retain source spans and digests; repeated requests reuse completed work. Recorded real-provider coverage uses Kuna 1.833 on Linux; Windows, PE and universal Mach-O are unsupported. See [the provider guide](docs/kuna-provider.md). ([#3832](https://github.com/morluto/rea/pull/3832), [#3854](https://github.com/morluto/rea/pull/3854), [#4071](https://github.com/morluto/rea/pull/4071), [#4095](https://github.com/morluto/rea/pull/4095)). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Retained JavaScript facts:** Query exact application paths, selected properties and filtered module references through `inspect_analysis_view`, with paging and complete row Evidence. These views reuse completed analysis and preserve source selection and unknown coverage. ([#4265](https://github.com/morluto/rea/pull/4265), [morluto/rea#4190](https://github.com/morluto/rea/issues/4190), [#4097](https://github.com/morluto/rea/pull/4097), [morluto/rea#4092](https://github.com/morluto/rea/issues/4092)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **APK locale tables:** Request several resource locales in one decode with repeatable CLI `--locales` or MCP `locales`. Each table reports its status and coverage; existing single-locale fields remain available. ([#4804](https://github.com/morluto/rea/pull/4804), [morluto/rea#4777](https://github.com/morluto/rea/issues/4777)). Thanks @abhay-2108, @N0zoM1z0.
|
||||
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/providers/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
|
||||
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/providers/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
|
||||
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
|
||||
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
|
||||
- **JADX deadlines:** Set `REA_JADX_OPERATION_TIMEOUT_MS` to extend APK loading and upstream operation deadlines together. Doctor reports the effective value and setup forwards an explicit selection. ([#4466](https://github.com/morluto/rea/pull/4466), [morluto/rea#3918](https://github.com/morluto/rea/issues/3918)). Thanks @jaychou813, @SandstromPL.
|
||||
- **Electron window options:** Resolve stable local constant `BrowserWindow` options while retaining source provenance. Relevant writes and escapes keep the result unknown. ([#4400](https://github.com/morluto/rea/pull/4400), [morluto/rea#4366](https://github.com/morluto/rea/issues/4366)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Language editions:** Add the Greek website edition and Malayalam documentation, with Korean typography and translation corrections across the site. ([#3520](https://github.com/morluto/rea/pull/3520), [#4792](https://github.com/morluto/rea/pull/4792), [#3821](https://github.com/morluto/rea/pull/3821), [morluto/rea#3820](https://github.com/morluto/rea/issues/3820), [#3799](https://github.com/morluto/rea/pull/3799), [morluto/rea#3798](https://github.com/morluto/rea/issues/3798), [#4344](https://github.com/morluto/rea/pull/4344), [#3856](https://github.com/morluto/rea/pull/3856), [#4166](https://github.com/morluto/rea/pull/4166)). Thanks @gensokyo-dev, @hi2242, @kalayciburak, @kaoru0822-kitauji, @kraken-503, @kukinghan, @N0zoM1z0, @yeseong0412.
|
||||
@@ -180,7 +78,7 @@ Full contribution record: [4 commits and 1 merged PR](https://github.com/morluto
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
- **Saved Evidence:** Evidence and analysis snapshots now require explicit nullable `relationship_coverage` in their identity commitments. Earlier records whose digest omitted null coverage are incompatible even when the serialized field is explicitly null. Artifact inventories require scoped coverage and explicit `integrity_contradictions`; obsolete `coverage_status` and prose-based coverage inference are removed. Regenerate old Evidence/snapshots and inventory observations with this release: saved files are not migrated and omitted facts are not filled with guessed defaults. ([db8128d](https://github.com/morluto/rea/commit/db8128dc0d1c4f731f3d19877fa00d25cf17f657), [0405c9d](https://github.com/morluto/rea/commit/0405c9d361de6d904dfa16cc0b6d8350407f2c9b), [a9588be](https://github.com/morluto/rea/commit/a9588be4ebdd20eeda975cb63ffcdb1cd868b340)).
|
||||
- **Saved Evidence:** Evidence and analysis snapshots now require explicit nullable `relationship_coverage` in their identity commitments. Artifact inventories require scoped coverage and explicit `integrity_contradictions`; obsolete `coverage_status` and prose-based coverage inference are removed. Regenerate old Evidence/snapshots and inventory observations with this release: saved files are not migrated and omitted facts are not filled with guessed defaults. ([db8128d](https://github.com/morluto/rea/commit/db8128dc0d1c4f731f3d19877fa00d25cf17f657), [0405c9d](https://github.com/morluto/rea/commit/0405c9d361de6d904dfa16cc0b6d8350407f2c9b), [a9588be](https://github.com/morluto/rea/commit/a9588be4ebdd20eeda975cb63ffcdb1cd868b340)).
|
||||
- **Browser capture results:** Consumers must retain resolved capture limits, event families and omission facts. Set desired budgets through the supported capture options; omitted caller inputs still select defaults, but stored observations must report the resolved values. Admitted content reads queue behind bounded concurrency. ([57a23f0](https://github.com/morluto/rea/commit/57a23f0a2da0fdc3835e203718ded8acea5a34e6)).
|
||||
- **Android version codes:** `inspect_adb_package.version_code` is a decimal string rather than a JSON number. Read it as a string, or use `BigInt` for numeric comparison. ([#3485](https://github.com/morluto/rea/pull/3485), [morluto/rea#3368](https://github.com/morluto/rea/issues/3368), [7666a48](https://github.com/morluto/rea/commit/7666a482b69dc7cabdbac3bbaf89a43e32300192)). Thanks @coygeek, @kaoru0822-kitauji.
|
||||
- **Native UI failures:** Failed or cancelled native UI scenarios now return typed errors with the collected report in `partial_observation`. Handle the error path and read that report instead of expecting a successful result containing failed steps. ([d909919](https://github.com/morluto/rea/commit/d9099190ab3a819bd3a9fd62a8714cef398016d5)).
|
||||
@@ -192,7 +90,7 @@ Full contribution record: [571 commits and 236 merged PRs](https://github.com/mo
|
||||
|
||||
## [6.5.0](https://github.com/morluto/rea/compare/rea-agents-6.4.0...rea-agents-6.5.0) (2026-10-13)
|
||||
|
||||
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/development/testing.md).
|
||||
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/testing.md).
|
||||
|
||||
### Highlights
|
||||
|
||||
@@ -220,7 +118,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
|
||||
- **HarmonyOS packages:** inventory HAP/HSP/App Pack structures without executing code. The recorded real workflow covers one pinned HAP through repository-built CLI/MCP; installed-package and real HSP/App Pack coverage remain unverified. ([#1447](https://github.com/morluto/rea/pull/1447)). Thanks @BaaRai.
|
||||
- **Ghidra raw inputs and references:** compatible Linux/macOS Ghidra profiles accept caller-selected raw architecture/entry information and expose typed incoming references at exact addresses. Windows Ghidra P0 admits the raw NMOS 6502 profile described below; other raw targets and the incoming-reference facet remain outside Windows P0. ([#1868](https://github.com/morluto/rea/pull/1868), [issue morluto/rea#1836](https://github.com/morluto/rea/issues/1836), [#1715](https://github.com/morluto/rea/pull/1715), [issue morluto/rea#1245](https://github.com/morluto/rea/issues/1245)). Thanks @bugstan, @casesolved-co-uk, @mvanhorn, @N0zoM1z0.
|
||||
- **Ghidra MIPS:** Add bounded MIPS32r2/o32 relocatable objects, soft-float integer-register storage, admitted MIPS16 shared ELF, and a narrowly admitted recordless legacy MIPS32 executable profile. The latter pins an endian-matched 32-bit language and records inferred o32 as an analysis assumption. Admission follows each documented ABI/flag/relocation boundary. ([#1949](https://github.com/morluto/rea/pull/1949), [issue morluto/rea#1937](https://github.com/morluto/rea/issues/1937), [#1950](https://github.com/morluto/rea/pull/1950), [issue morluto/rea#1938](https://github.com/morluto/rea/issues/1938), [#1991](https://github.com/morluto/rea/pull/1991), [issue morluto/rea#1851](https://github.com/morluto/rea/issues/1851), [#1898](https://github.com/morluto/rea/pull/1898), [issue morluto/rea#1620](https://github.com/morluto/rea/issues/1620)). Thanks @AurelioB, @BaaRai, @bqw371, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/providers/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
|
||||
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
|
||||
- **PSP static ELF profile:** the Ghidra profile is Linux x64, bounded little-endian ELF32 `ET_EXEC` with the documented PSP/ABI declarations and a compatible caller-installed Allegrex extension. PRX, shared/relocatable images, PBP/ISO/CSO and runtime emulation are outside scope. Historical extension experiments are separate from public-provider verification; real public Ghidra PSP coverage remains unverified. ([#1599](https://github.com/morluto/rea/pull/1599)). Thanks @jaelliot.
|
||||
- **Ghidra analysis settings:** apply validated analyzer overrides before analysis while retaining the selected analysis profile. ([#1913](https://github.com/morluto/rea/pull/1913)). Thanks @akushonkamen, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Attached native IDA MCP:** add an optional native bridge alongside existing legacy/headless profiles. Recorded native runs use Linux IDA Free 9.4 with a navigation correction and Windows IDA Pro 9.3 with a loopback-listener change; stock native switching, native macOS and Windows IDA Free remain unverified. Attached databases remain read-only and are never saved or closed. ([#1544](https://github.com/morluto/rea/pull/1544)). Thanks @imissapixel, @N0zoM1z0.
|
||||
@@ -292,7 +190,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
|
||||
- **Source maps:** original-source and mapping rows preserve a null source name when unavailable. Handle null explicitly instead of expecting an invented unknown-source string; new index fields remain optional when reading older Evidence. ([ee50ed1](https://github.com/morluto/rea/commit/ee50ed121e38cc6e1a2ff0dc8b789f15f71c5402)).
|
||||
- **Streamed filtered JSON:** `--filter-output` now emits JSON `null` instead of `{}` when the selected value is null; consumers must distinguish a present null from a missing selection. Use `--full-output` when an object envelope is required. ([#2333](https://github.com/morluto/rea/pull/2333)). Thanks @Jackmeson1.
|
||||
- **Call graphs:** `get_call_graph.normalized_result` is now `{ graph, limitations }`; read traversal-depth groups under `graph` and retain relationship-specific caveats. Contract/fixture validation does not establish new Ghidra or IDA engine coverage. ([5dfb9d0](https://github.com/morluto/rea/commit/5dfb9d050c2cc44a61b183dae765bef4d22a53b7)).
|
||||
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/reference/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
|
||||
- **Provider admission:** MIPS targets with unresolved GNU ABI attributes are refused. Supply supported ABI declarations. Saved Hopper databases on macOS are unavailable because the launcher cannot run the analysis bridge when opening a database; select an executable target instead. Database support requires a script-capable database launch route. ([d4bd65c](https://github.com/morluto/rea/commit/d4bd65c53c0254d0195cc7ab8da163dcadfd3b9b)).
|
||||
- **Segment extents:** `binary_overview` segment length may be null when Hopper coordinates cannot establish a finite extent in one address space. Preserve unknown rather than converting it to zero or a saturated value. ([36db19c](https://github.com/morluto/rea/commit/36db19c59)).
|
||||
@@ -372,7 +270,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/guides/runtime/process-capture.md) and [browser scenario contract](docs/guides/web/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
|
||||
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/process-capture.md) and [browser scenario contract](docs/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
|
||||
* **browser:** JSON shape paths now contain typed property and array-element segments. WebMCP input_schema_shape is replaced by input_schema and input_schema_sha256, retaining declared examples and defaults.
|
||||
* **process:** Filesystem digest omission entries now require system_code; budget and changed-file omissions use null, and unavailable-file omissions preserve the operating-system error code.
|
||||
* **contracts:** Provider analysis outputs must explicitly supply metadata previously repaired with defaults; unsupported nullable facets must be reported as null.
|
||||
@@ -540,7 +438,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
|
||||
Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
|
||||
|
||||
* **MCP / Evidence / snapshots / artifacts:** Success results are direct canonical Evidence; read `normalized_result` rather than the old `result`/`evidence` wrapper. Evidence requires nullable `analysis_profile`, session provider aliases are removed, snapshots require complete profile-bound query records, and artifact roles/permissions belong to occurrences. Recreate incompatible saved data; adding fields without recomputing their semantic identity is invalid. ([63bf914](https://github.com/morluto/rea/commit/63bf91405cf97b114a84f337fb1272d98a267999))
|
||||
* **Process captures:** Legacy v3 conversion is removed. Explicit executable identity and `event_journal` are required; retain old captures and recapture when current comparisons are needed instead of inventing launch identity or event order. ([2a77469](https://github.com/morluto/rea/commit/2a7746998d47446c4db6d4f524e0f183d6803f3f))
|
||||
@@ -2683,7 +2581,7 @@ Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before u
|
||||
* enforce conventional pull request titles ([3a21f16](https://github.com/morluto/rea/commit/3a21f16c1832430765083636511a8c12dbccf2dd))
|
||||
* verify installed package with Linux Hopper ([4f97f69](https://github.com/morluto/rea/commit/4f97f699e4d5471557d1ce0ff91e144065d06a61))
|
||||
|
||||
## [0.3.0](https://github.com/morluto/rea/compare/rea-0.2.1...rea-agents-0.3.0) (2026-07-12)
|
||||
## [0.3.0](https://github.com/morluto/rea/compare/rea-agents-0.2.1...rea-agents-0.3.0) (2026-07-12)
|
||||
|
||||
|
||||
### Features
|
||||
@@ -2736,7 +2634,7 @@ Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before u
|
||||
* redesign and localize README ([051b6db](https://github.com/morluto/rea/commit/051b6db65687fd14826b6462cd623feb2504f123))
|
||||
* redesign and localize README ([910eda6](https://github.com/morluto/rea/commit/910eda6f323ba1151b0771fe5de6c60a91e3d46d))
|
||||
|
||||
## [0.2.0](https://github.com/morluto/rea/releases/tag/rea-0.2.0) (2026-07-12)
|
||||
## [0.2.0](https://github.com/morluto/rea/compare/rea-0.1.0...rea-0.2.0) (2026-07-12)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
+20
-88
@@ -10,32 +10,13 @@ and maintenance cost. Keep cosmetic polish with useful fixes; confusing or
|
||||
incorrect user instructions warrant standalone corrections. Collect other
|
||||
cosmetic-only suggestions in an issue rather than separate PRs.
|
||||
|
||||
For selected architectural owners and dependency paths, use the
|
||||
[architecture map](docs/architecture/architecture.mermaid). It is not a complete directory or
|
||||
import inventory. For current provider wiring, start with
|
||||
[`src/composition/binary.ts`](src/composition/binary.ts) for deep candidates and
|
||||
the other `src/composition/` factories for tool services, including
|
||||
[`gpu.ts`](src/composition/gpu.ts) and [`model.ts`](src/composition/model.ts).
|
||||
Live Frida, Cutter, GDB and Rizin construction also uses shared composition
|
||||
factories. For service ownership, availability and catalog declarations, follow
|
||||
the [capability composition guide](docs/capability-composition.md).
|
||||
Production factories construct fresh providers without starting engines or
|
||||
acquiring targets; optional adapter failures must preserve successful peers. Run
|
||||
For capability organization and provider composition, use the
|
||||
[architecture map](docs/architecture.mermaid). Production factories in
|
||||
`src/composition/` construct fresh providers without starting engines or acquiring
|
||||
targets; optional adapter failures must preserve successful peers. Run
|
||||
`npm run verify:test-discovery` after adding or moving tests. Resolved import
|
||||
boundaries are checked by `npm run verify:module-boundaries`, included in lint.
|
||||
|
||||
Before retiring an alias or facade, trace runtime registration, dynamic imports,
|
||||
JavaScript verifiers, subprocess scripts, and packaged entrypoints as well as
|
||||
TypeScript references. Tests may be the last consumers of a superseded internal
|
||||
path; migrate their distinct assertions to the current production path before
|
||||
removing it. Preserve validation, lazy loading, caller policy, and resource
|
||||
acquisition and cleanup supplied by the old boundary. Published APIs, persisted
|
||||
formats, and independently versioned bridges require an explicit compatibility
|
||||
decision; an internal representation alone does not establish that obligation.
|
||||
See [interpreting boundary facts once](docs/development/tool-design.md#interpret-boundary-facts-once)
|
||||
for admission ownership and the distinction between repeated validation and
|
||||
independent checks.
|
||||
|
||||
For analysis workflows and agent interfaces, follow the
|
||||
[performance priorities](AGENTS.md#performance). Explain the selected output
|
||||
scope, reuse/cache decisions, and relevant before/after measurements when these
|
||||
@@ -44,11 +25,9 @@ Use the [analysis optimization skill](.agents/skills/rea-analysis-optimization/S
|
||||
to locate provider and shared-workflow bottlenecks, preserve Evidence, and
|
||||
compare equivalent real tasks before claiming an improvement.
|
||||
|
||||
When adding or changing an MCP tool, follow the [tool design guide](docs/development/tool-design.md) and preserve the canonical contracts and generated catalog.
|
||||
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
|
||||
When adding or changing an admission ceiling, budget, or deadline, follow
|
||||
[analysis settings, limits, and recovery](docs/design/limits-and-recovery.md)
|
||||
for ownership, operation snapshots, provider integration, and the steps for
|
||||
adding a runtime setting.
|
||||
[admission limits and recovery](docs/design/limits-and-recovery.md).
|
||||
|
||||
## Reporting issues
|
||||
|
||||
@@ -93,8 +72,7 @@ Before requesting maintainer review, the human owner must:
|
||||
passing. Follow the [verification requirements](#development-feedback-and-pr-verification)
|
||||
for required checks and real-provider coverage.
|
||||
- Complete the **Author readiness** declaration in the PR template, explicitly
|
||||
stating that the PR is ready for initial review, unless the PR author is
|
||||
exempt from the declaration as listed below.
|
||||
stating that the PR is ready for initial review.
|
||||
|
||||
Keep the PR description concise and current. Explain the problem, observable
|
||||
outcome, approach, and useful evidence; link long logs or put them in optional
|
||||
@@ -103,11 +81,7 @@ Human ownership and review remain required when CI or an automated reviewer
|
||||
reports success. Readiness requests maintainer review; it does not grant scope
|
||||
approval or permission to merge.
|
||||
|
||||
Draft status is optional. Draft PRs may request early feedback with open questions
|
||||
and incomplete checks. Agents preserve the current draft status unless asked to
|
||||
change it; missing readiness declarations or later updates do not require a PR
|
||||
to become a draft.
|
||||
|
||||
Draft PRs may request early feedback with open questions and incomplete checks.
|
||||
Leave the readiness declaration unchecked until the human owner has completed
|
||||
it. The declaration records initial readiness; later pushes do not invalidate
|
||||
it or require another human sign-off. Keep the PR description and affected
|
||||
@@ -115,12 +89,9 @@ validation current as work changes. AI agents must leave human declarations to
|
||||
the human owner and must not attest to a human review on their behalf.
|
||||
|
||||
The `PR readiness` check reports pending for drafts and requires a named human
|
||||
owner and all three template declarations for non-drafts. PRs authored by
|
||||
@morluto, @kaoru0822-kitauji, @N0zoM1z0, or @gensokyo-dev are exempt from
|
||||
the declaration; their human ownership, review, and evidence responsibilities
|
||||
above still apply. The check identifies authors by GitHub account ID. It checks
|
||||
that the declaration is filled, not whether the code or evidence is correct.
|
||||
Existing PRs can copy the **Author readiness** section from the current template.
|
||||
owner and all three template declarations for non-drafts. It checks that the
|
||||
declaration is filled, not whether the code or evidence is correct. Existing PRs
|
||||
can copy the **Author readiness** section from the current template.
|
||||
Open PRs must have distinct head commits because GitHub shares commit statuses.
|
||||
|
||||
Maintainers enable merge enforcement after the workflow is merged: run
|
||||
@@ -130,19 +101,6 @@ status checks alongside its existing checks. Deploy the workflow to each
|
||||
protected target branch before requiring it there. The workflow reads PR
|
||||
metadata using trusted default-branch code and writes only a commit status.
|
||||
|
||||
When an open PR may overlap another open change or a recently landed fix, note
|
||||
the related PR, issue, or commit in the description or a comment as soon as the
|
||||
overlap is known. Prefer one clear pointer over repeated status updates.
|
||||
|
||||
If an overlapping change merges while a PR awaits review, link that commit on
|
||||
the open PR and state what remains useful. Rebase or otherwise align with the
|
||||
current base when the remaining work is still wanted, narrow the PR to the
|
||||
uncovered portion, or close it when the change is fully superseded. Keep the
|
||||
description and validation current after that update. Maintainers may also note
|
||||
the expected disposition (rebase, narrow, or close) when they land or recognize
|
||||
an overlapping fix. Readiness still does not grant merge permission or a review
|
||||
date.
|
||||
|
||||
Maintainers may return a PR to the author or close it when the problem is
|
||||
unsupported, validation is missing, or the owner cannot explain the change.
|
||||
Contributors are responsible for completing this work before requesting review.
|
||||
@@ -247,9 +205,7 @@ guarantee.
|
||||
Use the development toolchain pinned in `.nvmrc` and
|
||||
`package.json#packageManager`. Run `nvm use` before installing dependencies;
|
||||
installed-package runtime support is documented in
|
||||
[installation](docs/getting-started/installation.md#start-setup).
|
||||
Use Node.js and npm for source builds and verifier scripts; the installed
|
||||
package's Bun runtime support does not extend to those development scripts.
|
||||
[installation](docs/installation.md#start-setup).
|
||||
|
||||
```bash
|
||||
npm ci
|
||||
@@ -264,15 +220,6 @@ catalogs, documentation projections, or managed verification evidence. Turbo
|
||||
caches deterministic builds and static checks across Git worktrees. After a package, lockfile, or managed-skill version change, run
|
||||
`npm run metadata:generate` before building.
|
||||
|
||||
Dependabot proposes updates for the npm lockfiles, website Python requirements,
|
||||
and GitHub Actions. Review dependency changes and run the checks relevant to
|
||||
their consumers. For dependencies maintained outside Dependabot, including
|
||||
`third_party/` and the HTTP/3 lab's pinned requirements, the maintainer should
|
||||
review upstream security advisories when updating or using them. Preserve source
|
||||
and version provenance, update license notices when needed, and report which
|
||||
verification was actually repeated. The lab requirements describe a recorded
|
||||
environment; a dependency update does not verify the revised environment.
|
||||
|
||||
Repository invariants live in [AGENTS.md](AGENTS.md). Keep provider-specific
|
||||
protocols in adapters and shared CLI/MCP workflows in the application layer.
|
||||
|
||||
@@ -299,23 +246,16 @@ The general source suite (`npm run test:fast`, `npm run test:changed`, and
|
||||
On a Windows checkout, some source tests fail because fixtures use POSIX paths,
|
||||
`#!/bin/sh` tool stubs, or POSIX rename semantics. Compare failures against the
|
||||
base revision, or run the suite on Linux or WSL, before attributing them to a
|
||||
change. The embedded-artifact parent-swap tests exercise the selected-path
|
||||
replacement when the rename succeeds; if Windows refuses it while the admitted
|
||||
file is open, they instead verify the refusal and cleanup. See
|
||||
[testing](docs/development/testing.md#developer-commands) for command scopes
|
||||
change. See [testing](docs/testing.md#developer-commands) for command scopes
|
||||
and lane selection.
|
||||
|
||||
`git submodule update --init --recursive` fails at the pinned Unblob submodule
|
||||
on Windows and can leave later submodules unchecked out. Apply the
|
||||
[Unblob checkout workaround](docs/guides/firmware/firmware-analysis.md#windows-submodule-checkout),
|
||||
[Unblob checkout workaround](docs/firmware-analysis.md#windows-submodule-checkout),
|
||||
then rerun the command.
|
||||
|
||||
## Documentation website
|
||||
|
||||
Use the [documentation directory](docs/README.md) and
|
||||
[authoring guide](docs/development/documentation.md) for document ownership,
|
||||
domain-based feature specifications and architectural decision records.
|
||||
|
||||
The VitePress site uses the Markdown files in `docs/`. Run `npm run docs:dev`
|
||||
for live editing, `npm run docs:build` to check the production build and
|
||||
links, and `npm run docs:preview` to preview that build at `/rea/`.
|
||||
@@ -369,7 +309,7 @@ npm run check:fast
|
||||
`test:local` runs source tests without building. `check:changed` combines cached
|
||||
static checks with source tests affected since the merge base with `origin/main`.
|
||||
Import-graph selection can miss runtime registration, bridges, and generated data;
|
||||
select relevant boundary tests explicitly. See [testing](docs/development/testing.md#developer-commands)
|
||||
select relevant boundary tests explicitly. See [testing](docs/testing.md#developer-commands)
|
||||
for commands, test placement, pruning, and real-provider prerequisites.
|
||||
|
||||
Before requesting review, run relevant tests and `npm run check`; add
|
||||
@@ -420,17 +360,9 @@ Real-provider execution is uncached.
|
||||
### Temporary files and diagnostics
|
||||
|
||||
Tests use `createTestTempDirectory` from `tests/fixtures/temporaryDirectory.ts`
|
||||
for exact-path, awaited cleanup, including failures and timeouts. It registers
|
||||
deletion with Vitest's `onTestFinished`; do not also add its result to a roots
|
||||
array or an `afterEach` removal. `createTestWorkspace` from
|
||||
`tests/support/workspace/workspaceFixture.ts` leaves cleanup with its caller,
|
||||
which must await `removeTestWorkspace`. Keep explicit resource teardown before
|
||||
directory deletion, using the case's `finally` or `afterEach`; Vitest runs
|
||||
`afterEach` before `onTestFinished` and awaits finish hooks in reverse
|
||||
registration order.
|
||||
for exact-path, awaited cleanup, including failures and timeouts.
|
||||
`npm run verify:test-temp-hygiene` runs the suite under a fresh `TMPDIR` and
|
||||
rejects remaining REA-owned paths. It cannot detect duplicate deletion owners or
|
||||
prove resource-close ordering. Never glob-clean shared `/tmp/rea-*` content.
|
||||
rejects remaining REA-owned paths. Never glob-clean shared `/tmp/rea-*` content.
|
||||
|
||||
`REA_LOG_LEVEL` selects `trace`, `debug`, `info`, `warn`, `error`, `fatal`, or
|
||||
`silent`. Logs go to stderr: MCP defaults to `info`, and one-shot CLI logging
|
||||
@@ -444,8 +376,8 @@ Run `npm run verify:hopper` on a supported macOS host or
|
||||
`npm run verify:hopper:linux` on the supported Linux demo runner. These build
|
||||
native fixtures and verify the installed engine through CLI/MCP; package or
|
||||
simulated-provider checks do not establish engine behavior. See
|
||||
[real-toolchain verification](docs/development/testing.md#end-to-end-integration-and-golden-evidence)
|
||||
for prerequisites and scope, and [Hopper setup](docs/getting-started/installation.md#hopper)
|
||||
[real-toolchain verification](docs/testing.md#end-to-end-integration-and-golden-evidence)
|
||||
for prerequisites and scope, and [Hopper setup](docs/installation.md#hopper)
|
||||
for supported hosts and unattended startup.
|
||||
|
||||
Use the default `build/conformance/manifest.json` unless explicitly verifying
|
||||
@@ -454,7 +386,7 @@ Generated fixtures and manifests remain ignored.
|
||||
|
||||
## Maintainer release checklist
|
||||
|
||||
Use the [release guide](docs/development/releasing.md). Main pushes automatically refresh a
|
||||
Use the [release guide](docs/releasing.md). Main pushes automatically refresh a
|
||||
Release Please PR with the next version and changelog. Review its migration
|
||||
notes, wait for the current CI checks to pass, and merge it into main to publish.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md)
|
||||
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md) · [Română](docs/readme/README_ro.md) · [Български](docs/readme/README_bg.md)
|
||||
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md)
|
||||
|
||||
# REA: Reverse Engineer Anything
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
**See a feature you like. Understand how it works, down to the binary level.**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](docs/reference/mcp-contracts.md#generated-catalog)
|
||||
[](docs/mcp-contracts.md#generated-catalog)
|
||||
[](#what-you-can-analyze)
|
||||
[](https://skills.sh/morluto/rea/reverse-engineer-anything)
|
||||
[](LICENSE)
|
||||
@@ -20,9 +20,7 @@
|
||||
|
||||
**[Website](https://rea.tools/) · [Guides](https://rea.tools/guides/) · [Showcases](https://rea.tools/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Join the REA community on Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [Build RL environments](#build-rl-environments) · [FAQ](#faq) · [Documentation](#documentation)
|
||||
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [FAQ](#faq) · [Documentation](#documentation)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -64,7 +62,7 @@ See a feature in an app that you want in your own product? Ask your agent to inv
|
||||
|
||||
REA connects your agent to tools for inspecting native binaries, JavaScript and Electron apps, .NET assemblies, and websites. You can also use the same tools from your terminal. Analysis runs locally, and results include the evidence and limitations behind each conclusion.
|
||||
|
||||
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, IDA or Kuna installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
|
||||
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, or IDA installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
|
||||
|
||||
> **[Visit the REA website](https://rea.tools/)** for setup instructions, illustrated guides, and real case studies.
|
||||
|
||||
@@ -72,7 +70,7 @@ Setup registers REA with your agent and installs matching workflow instructions.
|
||||
|
||||
On Linux, Ghidra, JADX, firmware tool launches and static JavaScript/Electron
|
||||
analysis require a procps-compatible `ps` on REA's `PATH` for process ownership
|
||||
checks. Recovery steps are in `docs/getting-started/installation.md` under
|
||||
checks. Recovery steps are in `docs/installation.md` under
|
||||
"Linux owned-process prerequisite".
|
||||
|
||||
### Set up your agent
|
||||
@@ -85,15 +83,12 @@ npx rea-agents setup
|
||||
|
||||
Choose your agents, review the proposed changes, and approve them. Setup adds
|
||||
REA's MCP server and matching workflow instructions, with backups of existing
|
||||
configuration. Restart your agent afterward. Setup records the selected runtime and REA
|
||||
entry script directly. If acquired through a package runner, keep its cache entry
|
||||
available. Run package acquisition from a trusted directory outside the target
|
||||
project. Installation and recovery details are in the setup guide below.
|
||||
configuration. Restart your agent afterward.
|
||||
|
||||
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
|
||||
[other agents](docs/getting-started/installation.md#supported-agents). See
|
||||
[installation and setup](docs/getting-started/installation.md) for provider configuration and
|
||||
manual MCP registration and Claude Code/Codex plugin installation.
|
||||
[other agents](docs/installation.md#supported-agents). See
|
||||
[installation and setup](docs/installation.md) for provider configuration and
|
||||
manual MCP registration.
|
||||
|
||||
MCP uses compact input schemas by default to reduce context overhead. Set
|
||||
`REA_MCP_INPUT_SCHEMA_PROFILE=full` in your MCP registration when you need the
|
||||
@@ -101,12 +96,6 @@ complete advertised field guidance and nested validation details. Both profiles
|
||||
keep the full tool set and canonical server validation. See
|
||||
"Compact input schema profile" in the MCP contracts guide.
|
||||
|
||||
Complete output schemas are advertised by default. If your client caps the
|
||||
complete `tools/list` frame, set `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact` to
|
||||
bound each advertised output schema, or `off` to omit output schemas entirely.
|
||||
Both settings keep canonical server-side validation and complete structured
|
||||
results. See "Output schemas" in the MCP contracts guide.
|
||||
|
||||
### Ask your agent
|
||||
|
||||
```text
|
||||
@@ -135,7 +124,7 @@ rea --help
|
||||
```
|
||||
|
||||
For native analysis, configure a provider first. See the
|
||||
[CLI and Evidence guide](docs/getting-started/cli.md) for native commands, provider selection,
|
||||
[CLI and Evidence guide](docs/cli.md) for native commands, provider selection,
|
||||
snapshots and scripting.
|
||||
|
||||
### Update REA
|
||||
@@ -191,47 +180,41 @@ CLI commands use the same workflows.
|
||||
REA requires Node.js 22.x (>=22.23.4), 24.x (>=24.22.1), or 26.12.1+, plus npm.
|
||||
Additional tools and host support depend on the target:
|
||||
|
||||
| Target | What REA returns | Requirements and guide |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/guides/native/native-investigation.md) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/guides/native/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
|
||||
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
|
||||
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
|
||||
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
|
||||
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
|
||||
| Windows shortcuts | Shell Link target, arguments, working dir, ExtraData and Tracker droid facts | Built-in static LNK inspection; [Windows shortcuts](docs/guides/native/windows-shortcut.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
|
||||
| Binary hardening | PIE, NX, RELRO, CFG and related header flags for ELF, PE and Mach-O | Built-in static header inspection; [binary hardening](docs/binary-hardening.md) |
|
||||
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
|
||||
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/guides/evm/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/guides/native/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/guides/javascript/javascript-application-workflows.md) |
|
||||
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/guides/web/browser-observation.md) |
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/guides/web/web-network-captures.md) |
|
||||
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/guides/managed/managed-code-analysis.md) |
|
||||
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/guides/android/android-analysis.md) |
|
||||
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
|
||||
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/guides/android/adb-device-analysis.md) |
|
||||
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/providers/jeb-analysis.md) |
|
||||
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/guides/android/apktool-resource-analysis.md) |
|
||||
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/guides/android/flutter-analysis.md) |
|
||||
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/guides/firmware/firmware-analysis.md) |
|
||||
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/guides/data/packet-captures.md) |
|
||||
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/guides/data/pcap-analysis.md) |
|
||||
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/guides/signals/signal-recordings.md) |
|
||||
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/guides/javascript/javascript-artifact-reconstruction.md), [Apple applications](docs/guides/apple/apple-application-analysis.md) |
|
||||
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/guides/runtime/process-capture.md) |
|
||||
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
|
||||
| Deno compile binaries | Virtual files, entry point, recorded options, npm snapshot length | None (static only); [Deno compile executables](docs/deno-standalone.md) |
|
||||
| Target | What REA returns | Requirements and guide |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
|
||||
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
|
||||
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
|
||||
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
|
||||
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
|
||||
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
|
||||
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
|
||||
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
|
||||
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
|
||||
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
|
||||
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
|
||||
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
|
||||
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
|
||||
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
|
||||
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
|
||||
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
|
||||
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
|
||||
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
|
||||
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
|
||||
|
||||
Static JavaScript and .NET inspection read the supplied files without running
|
||||
the application. Runtime capture runs or interacts with the selected target
|
||||
@@ -245,25 +228,20 @@ or update either provider.
|
||||
<a id="choosing-a-deep-analysis-provider"></a>
|
||||
|
||||
Native formats and host support vary by provider. See
|
||||
[Hopper and Ghidra setup](docs/getting-started/installation.md#hopper), the
|
||||
[IDA guide](docs/providers/ida-provider.md) (including an optional native MCP bridge), the
|
||||
[Hopper and Ghidra setup](docs/installation.md#hopper), the
|
||||
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), the
|
||||
[Kuna decompiler guide](docs/kuna-provider.md), and
|
||||
[experimental Windows Ghidra support](docs/providers/windows-ghidra-p0.md).
|
||||
Ghidra also supports [16-bit DOS analysis](docs/providers/ghidra-dos.md).
|
||||
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
|
||||
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
|
||||
For large binaries, raise the provider's startup deadline with
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS` or `REA_HOPPER_STARTUP_TIMEOUT_MS`.
|
||||
For provider selection, see the [CLI guide](docs/getting-started/cli.md#choose-a-provider).
|
||||
Check [release availability](docs/getting-started/installation.md#released-package-and-main)
|
||||
For provider selection, see the [CLI guide](docs/cli.md#choose-a-provider).
|
||||
Check [release availability](docs/installation.md#released-package-and-main)
|
||||
for features added since the latest npm release.
|
||||
|
||||
## Showcases
|
||||
|
||||
<a href="https://rea.tools/showcase/">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/rea-showcases-dark.svg" />
|
||||
<img alt="Illustrations of the DX-Ball sound-pan, Notion clipboard-bridge, and TH04 bullet-ring showcases" src="docs/assets/rea-showcases.svg" />
|
||||
</picture>
|
||||
</a>
|
||||
[](https://rea.tools/showcase/)
|
||||
|
||||
### DX-Ball: reconstruct a sound-pan calculation
|
||||
|
||||
@@ -290,78 +268,28 @@ historical compiler output.
|
||||
[Read the case study](https://rea.tools/showcase/th04/) ·
|
||||
[Reconstruction repository](https://github.com/N0zoM1z0/th04)
|
||||
|
||||
### Build RL environments with REA
|
||||
|
||||
Record a web app session, turn its API into tools, and export an environment an AI agent can use. Run it with AgentEnv, then add tasks and reward checks for RL training or evaluation.
|
||||
|
||||
[Read the guide](https://rea.tools/guides/rl-environments/) ·
|
||||
[Watch the video](https://x.com/reatools/status/2112574822584918075?s=20)
|
||||
|
||||
If you've used REA on something interesting, we'd love to see it. Share your
|
||||
case in an [issue](https://github.com/morluto/rea/issues) or a
|
||||
[pull request](https://github.com/morluto/rea/pulls), including the target,
|
||||
your question, how REA helped, and what you found.
|
||||
|
||||
## Build RL environments
|
||||
|
||||
Give an AI agent a local environment to practice tasks such as finding an issue or creating a folder, without running the original app. Use it for reinforcement learning (RL) or to test how well an agent completes tasks.
|
||||
|
||||
REA turns recorded API requests and responses into tools the agent can call. The environment uses the recorded data and supports changes only when the recording confirms their effects.
|
||||
|
||||
With REA set up in your agent, replace the example URL and folder tasks below with your own:
|
||||
|
||||
```text
|
||||
Use REA to build an RL environment from my test web app at
|
||||
http://127.0.0.1:3000.
|
||||
Cover listing, creating, renaming, and deleting folders. You may create and
|
||||
remove test folders. Leave existing data unchanged.
|
||||
Save it in a folder named my-env. Check which operations work and tell me
|
||||
how to run it.
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary>Use the command line instead</summary>
|
||||
|
||||
Use these commands if you already have a file containing the app’s recorded browser requests and responses. Running the result also requires AgentEnv and Docker.
|
||||
|
||||
```sh
|
||||
rea environment-build ./session.har ./my-env --name my-env --json
|
||||
agent-env run ./my-env
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
REA exports a runnable environment with a basic test to check that it starts and responds. Use it with your RL framework to define task goals, add reward checks, and run training.
|
||||
|
||||
[Build an environment](https://rea.tools/guides/rl-environments/) ·
|
||||
[Environment reference](docs/agent-environments.md) ·
|
||||
[Watch the demo on X](https://x.com/reatools/status/2112574822584918075?s=20)
|
||||
|
||||
### Desktop app environments
|
||||
|
||||
REA can also build an environment of a Linux desktop app, which an agent uses through screenshots, the mouse and the keyboard. From a one-line request, such as "Build me a Gnote environment where the agent adds olive oil to the shopping list note", your agent installs the app on a local build computer, writes the task, exports the environment and checks the task in a fresh computer. The machine that builds it needs Linux on amd64 or arm64, Docker and the agentenv framework version the guide pins.
|
||||
|
||||
[Set up desktop app environments](docs/desktop-build-computer.md)
|
||||
|
||||
## FAQ
|
||||
|
||||
<details>
|
||||
<summary><strong>Which agents can use REA?</strong></summary>
|
||||
|
||||
Any agent that supports local MCP servers. Setup configures the
|
||||
[supported agents](docs/getting-started/installation.md#supported-agents); other clients can use
|
||||
[manual MCP registration](docs/getting-started/installation.md#mcp-registry).
|
||||
[supported agents](docs/installation.md#supported-agents); other clients can use
|
||||
[manual MCP registration](docs/installation.md#mcp-registry).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Do I need a native analysis engine?</strong></summary>
|
||||
<summary><strong>Do I need Hopper, Ghidra or IDA?</strong></summary>
|
||||
|
||||
Deep native analysis uses Hopper, Ghidra, IDA or Kuna, depending on the target
|
||||
and operation. Static JavaScript and .NET inspection work without a native
|
||||
analysis engine. Setup can install Hopper after approval; Ghidra, IDA and Kuna
|
||||
use your existing installations. See [provider setup](docs/getting-started/installation.md#hopper)
|
||||
for the options and their supported targets.
|
||||
Deep native analysis uses one of them. Static JavaScript and .NET inspection
|
||||
work without a native analysis engine. Setup can install Hopper after approval;
|
||||
Ghidra and IDA use your existing installations. See [provider setup](docs/installation.md#hopper).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -372,7 +300,7 @@ REA starts Hopper when an operation needs it. On macOS, a first-run dialog may
|
||||
ask you to choose demo mode or activate your license. Complete that prompt in the
|
||||
same user session before your first CLI/MCP operation or unattended job. Open
|
||||
Hopper manually if needed. See
|
||||
[Hopper startup and troubleshooting](docs/getting-started/installation.md#launcher-paths-and-troubleshooting).
|
||||
[Hopper startup and troubleshooting](docs/installation.md#launcher-paths-and-troubleshooting).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -381,7 +309,7 @@ Hopper manually if needed. See
|
||||
|
||||
The skill supplies investigation instructions for your agent. Use `npx rea-agents setup`
|
||||
to register REA's MCP server and install the matching instructions, then restart
|
||||
your agent. See [skill-only installation](docs/getting-started/installation.md#skill-only-installation).
|
||||
your agent. See [skill-only installation](docs/installation.md#skill-only-installation).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -430,18 +358,18 @@ with your REA version, target type, steps to reproduce and error output.
|
||||
Start with the website's [worked guides](https://rea.tools/guides/).
|
||||
For exact options, prerequisites and result contracts:
|
||||
|
||||
- [Installation and setup](docs/getting-started/installation.md): agent registration, provider configuration, updates and uninstall.
|
||||
- [Readiness and troubleshooting](docs/getting-started/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
|
||||
- [CLI and Evidence](docs/getting-started/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
|
||||
- [MCP contracts](docs/reference/mcp-contracts.md) and [agent prompts](docs/getting-started/mcp-prompts.md): tool results, sessions and guided investigations.
|
||||
- [Tool catalog](docs/reference/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
|
||||
- [Roadmap](docs/development/roadmap.md): planned work and capability trackers.
|
||||
- [Installation and setup](docs/installation.md): agent registration, provider configuration, updates and uninstall.
|
||||
- [Readiness and troubleshooting](docs/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
|
||||
- [CLI and Evidence](docs/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
|
||||
- [MCP contracts](docs/mcp-contracts.md) and [agent prompts](docs/mcp-prompts.md): tool results, sessions and guided investigations.
|
||||
- [Tool catalog](docs/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
|
||||
- [Roadmap](docs/roadmap.md): planned work and capability trackers.
|
||||
|
||||
Report vulnerabilities through [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Star history
|
||||
|
||||
🎉 **400,000 GitHub stars — thank you!**
|
||||
🎉 **300,000 GitHub stars — thank you!**
|
||||
|
||||
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
|
||||
|
||||
@@ -470,8 +398,8 @@ report a bug or suggest a feature, or [send a pull request](https://github.com/m
|
||||
to improve the code or docs.
|
||||
|
||||
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
|
||||
[testing](docs/development/testing.md) for verification lanes, and the
|
||||
[architecture map](docs/architecture/architecture.mermaid) for selected layers and dependency paths.
|
||||
[testing](docs/testing.md) for verification lanes, and the
|
||||
[architecture map](docs/architecture.mermaid) for the project structure.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
+28
-112
@@ -4,124 +4,13 @@
|
||||
|
||||
Security fixes are provided for the latest published version of REA. Upgrade to the latest release before reporting a problem that may already have been resolved.
|
||||
|
||||
## Security boundary
|
||||
|
||||
REA is a local reverse-engineering and automation tool. CLI and MCP use the
|
||||
same application workflows and run with the host process's permissions.
|
||||
REA is not a general-purpose sandbox or a multi-tenant authorization service.
|
||||
|
||||
### Threat model
|
||||
|
||||
- **Trusted authority:** the operating-system account running REA, the caller
|
||||
or agent host that selects operations, and configured analysis providers,
|
||||
their executables and extensions, and service endpoints. An MCP client can
|
||||
invoke the operations exposed to it; its host must enforce the user's tool
|
||||
permissions and approvals.
|
||||
- **Untrusted data:** target binaries, archives, source files, project contents,
|
||||
captures, web pages and returned target content can be malicious. Selecting
|
||||
an input for static inspection does not authorize commands embedded in that
|
||||
input. Agents must treat target text and Evidence as data rather
|
||||
than instructions granting additional authority.
|
||||
- **Operation-specific effects:** static inspection parses target data and may
|
||||
launch an analysis engine. Runtime, debugger, instrumentation, browser,
|
||||
desktop and environment workflows can deliberately execute code, attach to
|
||||
processes, make network requests, write files or interact with applications
|
||||
and devices. Review the selected operation's effects and limitations; static
|
||||
inspection does not authorize launching or interacting with the target
|
||||
application. See
|
||||
[runtime and network responsibilities](DISCLAIMER.md#runtime-network-and-device-operations).
|
||||
|
||||
Standard client registrations use a dedicated stdio process. Opt-in
|
||||
[HTTP MCP](docs/reference/mcp-contracts.md#transports) uses plain HTTP without
|
||||
TLS or client authentication and shares provider/active-target state across
|
||||
protocol sessions. Its session ID is not a user-authentication or isolation
|
||||
boundary. Keep this endpoint within a trusted local or isolated environment.
|
||||
|
||||
### Intended security properties
|
||||
|
||||
Static JavaScript and .NET inspection do not run the target application.
|
||||
Unexpected execution of input code through a data-only analysis path is a
|
||||
security concern. Analysis engines themselves are executable software trusted
|
||||
with the host account's permissions; parsing an input in an external engine
|
||||
does not isolate that engine from the host.
|
||||
|
||||
REA authenticates its managed Hopper and Ghidra bridges with per-session random
|
||||
capability tokens transferred through private session files, rather than
|
||||
process arguments or environment variables. POSIX uses current-user Unix
|
||||
sockets. Windows Ghidra uses authenticated loopback TCP and a private NTFS
|
||||
runtime whose DACL admits only the current user and SYSTEM. These controls
|
||||
protect the managed session boundary, not every external provider or debugger
|
||||
endpoint. See [Windows Ghidra transport and lifecycle](docs/providers/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
|
||||
|
||||
Provider ownership and mutation rules remain specific to each workflow:
|
||||
|
||||
- Ghidra's working projects are temporary and owned by REA. Opt-in
|
||||
[checkpoints](docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program)
|
||||
retain REA-owned Program state. A
|
||||
[caller-selected existing project](docs/getting-started/installation.md#reuse-an-existing-ghidra-project)
|
||||
is fingerprinted and copied into an owned runtime for read-only processing;
|
||||
Ghidra does not open or save the original database through that workflow.
|
||||
- [IDA analysis](docs/providers/ida-provider.md) is read-only. An attached GUI
|
||||
database is never saved or closed by REA; owned headless sessions have a
|
||||
separate lifecycle.
|
||||
- Cleanup must stay within resources REA owns. Failed or unverified cleanup
|
||||
can retain a workspace or session for retry; closing a session
|
||||
does not imply that every investigation artifact has been erased.
|
||||
|
||||
`rea setup` shows planned configuration and installation changes and requires
|
||||
approval before applying them. Ghidra, Java and IDA are user-managed. Approved
|
||||
Linux Hopper installation authenticates the package's detached PGP signature
|
||||
against the pinned publisher identity and checks the signed launcher against
|
||||
the selected release's digest. The release feed's display version is separate
|
||||
from that authenticated identity. See [installation and publisher verification](docs/getting-started/installation.md#hopper).
|
||||
|
||||
### Limits and caller responsibilities
|
||||
|
||||
- **Host isolation:** private runtimes and capability tokens do not protect
|
||||
against malicious software already running as the same operating-system
|
||||
user, or a compromised trusted client or provider. Use an appropriately
|
||||
isolated account, VM or restricted environment when handling hostile targets.
|
||||
- **Resource bounds:** timeouts, cancellation, streaming and size limits are
|
||||
operation-specific. They are not a universal CPU, memory, disk or wall-time
|
||||
limit on every target and third-party tool. Enforce hard host resource limits
|
||||
externally when needed. Failures of a documented boundary or cancellation
|
||||
control still warrant investigation.
|
||||
- **Data and network exposure:** Evidence, caches, checkpoints, exports and
|
||||
captures can retain private target data, paths, URLs and memory contents.
|
||||
Local execution does not make these outputs safe to publish. Requested
|
||||
network operations, agent hosts and provider features such as
|
||||
[IDA Free's cloud decompiler](docs/providers/ida-provider.md) can communicate
|
||||
outside the host. Review their data handling and secure any exposed endpoint.
|
||||
- **Interpretation:** analysis, reconstruction and generated environments can
|
||||
be incomplete or incorrect. Identity checks, signatures and successful
|
||||
analysis do not establish that a target, provider or generated program is
|
||||
safe to execute. Review the Evidence and proposed actions before relying on
|
||||
them.
|
||||
|
||||
### Vulnerability scope
|
||||
|
||||
Please report suspected violations of these boundaries privately. Examples
|
||||
include unintended code execution during data-only inspection, escaping a
|
||||
documented extraction/output boundary, bypassing bridge authentication or
|
||||
target/session identity checks, unauthorized effects on an unrelated resource,
|
||||
and disclosure of transport credentials or explicitly marked sensitive values.
|
||||
An attacker-triggered bypass of a documented resource or cancellation control
|
||||
can also be security-relevant without code execution.
|
||||
|
||||
Incorrect analysis, unsupported formats, crashes, hangs and high resource use
|
||||
without a security-boundary impact are generally ordinary bug reports. Effects
|
||||
explicitly requested through an execution or interaction workflow are not by
|
||||
themselves a vulnerability. A dependency advisory needs its actual REA exposure
|
||||
assessed; a complete exploit or diagnosis is not required to report a concern.
|
||||
If you are unsure whether a boundary is affected, use the private channel below.
|
||||
|
||||
## Reporting a vulnerability
|
||||
|
||||
Do not disclose vulnerabilities, proof-of-concept exploits, credentials,
|
||||
private binaries, Hopper documents, or private Ghidra projects in a public
|
||||
issue.
|
||||
|
||||
Use **Report a vulnerability** in the repository's **Security** tab as the preferred way to submit the report privately. If GitHub private reporting is unavailable to you, email [[email protected]](mailto:[email protected]). Do not open a public issue to request a private contact channel.
|
||||
Use **Report a vulnerability** in the repository's **Security** tab to submit the report privately. Do not open a public issue to request a private contact channel.
|
||||
|
||||
Include the affected REA version, operating system/distribution, selected
|
||||
provider and version, impact, reproduction conditions, and any suggested
|
||||
@@ -132,3 +21,30 @@ unrelated local paths from logs or examples.
|
||||
The maintainer will acknowledge the report, assess its scope, and coordinate
|
||||
remediation and disclosure timing with the reporter. Please allow time for
|
||||
provider-dependent behavior to be reproduced safely.
|
||||
|
||||
## Security boundary
|
||||
|
||||
REA authenticates its Hopper and Ghidra bridges with per-session random
|
||||
capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses
|
||||
authenticated loopback TCP with a private NTFS runtime whose DACL admits only
|
||||
the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
|
||||
Tokens are transferred through private session descriptors rather than process
|
||||
arguments or environment variables.
|
||||
This is not a sandbox and does not protect against malicious processes already
|
||||
running as the same operating-system user. Opening an untrusted binary
|
||||
delegates parsing and analysis to the selected local provider with that user's
|
||||
permissions.
|
||||
|
||||
Ghidra sessions use a packaged Java `HeadlessScript`, an isolated temporary
|
||||
project, and private home, cache, and runtime directories. REA imports only the
|
||||
requested target, bounds startup and protocol messages, requests bounded CPU
|
||||
and heap settings, and deletes the temporary project when the session closes.
|
||||
These controls limit accidental persistence and resource use; they do not make
|
||||
Ghidra a sandbox. REA does not open or modify user-owned Ghidra projects.
|
||||
|
||||
`GHIDRA_INSTALL_DIR` and optional `JAVA_HOME` identify user-managed
|
||||
installations. Setup can add those values to detected client configuration only
|
||||
after showing its plan and receiving approval. REA does not download, install,
|
||||
upgrade, or modify Ghidra or Java.
|
||||
|
||||
On Linux, REA accepts release metadata only from Hopper's HTTPS endpoint, restricts package URLs to Hopper's public download origin, bounds the package size, and compares the downloaded bytes with Hopper's published checksum before requesting installation. The published SHA-1 value is a corruption check, not a modern package signature; HTTPS origin validation remains part of the trust boundary. Dependency installation is delegated without shell evaluation to `apt-get`, `dnf`, or `pacman`, directly as root or through `pkexec`. REA never invokes `sudo`.
|
||||
+21
-23
@@ -16,8 +16,6 @@ notices supplied with the software you actually obtain or redistribute.
|
||||
| Material | Use and licensing records |
|
||||
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| npm dependencies | [Complete inventory](licenses/npm-dependencies.txt) of every entry in `package-lock.json`, including transitive, optional platform, development, and website dependencies. [Runtime notices](licenses/npm-runtime-notices.txt) preserve the copyright, license, and NOTICE documents from integrity-verified release archives, plus the MIT notice for Rolldown's generated runtime helpers. Development dependencies are inventoried but are not all distributed in REA's npm package. |
|
||||
| pefile ordinal tables | Import-hash ordinal names are derived from [pefile 2024.8.26](https://github.com/erocarrera/pefile/tree/4b3b1e2e568a88d4f1897d694d684f23d9e270c4), copyright 2004–2024 Ero Carrera. Preserve the [MIT license](licenses/pefile-2024.8.26-LICENSE.txt). The Python package is not a runtime dependency. |
|
||||
| UF2 family names | UF2 family IDs and short names are copied from [microsoft/uf2 `utils/uf2families.json`](https://github.com/microsoft/uf2/blob/cf3bd7f9c274b949528f8cd6a19eef2502a687d2/utils/uf2families.json), copyright Microsoft Corporation. Preserve the [MIT license](licenses/microsoft-uf2-MIT.txt). |
|
||||
| NativeAOT parser | REA's derived parser uses the MIT-licensed [ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot/tree/effeb734fc570c32650f88b159608979dc7b423e) work of Washi and the NativeAOT layout definitions in [dotnet/runtime](https://github.com/dotnet/runtime/tree/a3b2d40328be0be3f8dd43f0e5cc925b8827b044). Preserve [Washi's MIT notice](licenses/nativeaot-parser-MIT.txt) and the [.NET Foundation MIT notice](licenses/dotnet-runtime-MIT.txt). |
|
||||
| EVM analysis | `[email protected]`, MIT, copyright 2023 Maxim Andreev. The published npm archive omits its license document; REA retains the [original upstream license](third_party/evmole/LICENSE) and [provenance](third_party/evmole/README.md). It is also included in the runtime notices. |
|
||||
| Windows native addon | The release addon is compiled with MinGW-w64 and statically links GCC runtime libraries. Preserve the [MinGW-w64 runtime notices](licenses/MinGW-w64-runtime.txt), [GPLv3 text](licenses/GPL-3.0.txt), and [GCC Runtime Library Exception 3.1](licenses/GCC-Runtime-Library-Exception-3.1.txt). The exception governs eligible compiled combinations; it does not relicense the runtime itself under MIT. The Node-API build headers are from Node 22.19.0; its [original license and component notices](licenses/Node-22.19.0-LICENSE.txt) are retained. Node itself is supplied by the user. Snapshot provenance and release-toolchain review requirements are in [licenses/README.md](licenses/README.md). |
|
||||
@@ -54,27 +52,27 @@ REA's adapters do not grant, replace, or waive these licenses. The upstream
|
||||
executables and their dependency graphs are supplied separately. Git submodules
|
||||
pin inspection sources; they are not included in the npm package.
|
||||
|
||||
| Project | License and source | REA relationship |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/getting-started/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
|
||||
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
|
||||
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
|
||||
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
|
||||
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
|
||||
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
|
||||
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
|
||||
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
|
||||
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
|
||||
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
|
||||
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
|
||||
| Project | License and source | REA relationship |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra 12.1.4. Copies of its [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.833/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.833/NOTICE) | Bring your own `kuna`; REA verified 1.833 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
|
||||
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
|
||||
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
|
||||
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
|
||||
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
|
||||
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
|
||||
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
|
||||
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
|
||||
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
|
||||
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
|
||||
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
|
||||
|
||||
Ghidra's original notice credits software developed at the National Security
|
||||
Agency. It explains that US-government-created portions are not subject to US
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
# Plugin icon
|
||||
|
||||
`rea-plugin.png` is a square adaptation of the left panel of REA's GitHub
|
||||
social preview, retrieved on 2026-10-28:
|
||||
|
||||
https://repository-images.githubusercontent.com/1209966933/1a3fef9a-be4b-41f4-8830-a94c0e8288cc
|
||||
|
||||
The image was adapted with imagegen, preserving the cyan REA lettering,
|
||||
pixel typography, orange rule, and taglines. Both plugin manifests reference
|
||||
this shared asset; only the PNG is included in the npm package.
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 1.1 MiB |
@@ -1,516 +0,0 @@
|
||||
"""Run REA inside an agentenv build computer for REA on the host: capture the app, draft its task and export it.
|
||||
|
||||
REA sends this file's source through the computer's ``computer-exec/v1`` route as
|
||||
``python3 -I -c <source> <command> <JSON arguments>``. JSON that would exceed the bytes Linux
|
||||
passes as one string travels as several arguments, which this file joins before parsing. Each command prints
|
||||
one JSON line and exits. ``start``
|
||||
leaves the capture running on its own: its stdin is a FIFO it holds open for reading and writing, so it never
|
||||
sees the end of its input, and ``mark`` and ``stop`` write their requests there and wait for the capture's own
|
||||
acknowledgement on its progress log. A small shell around the capture writes its exit status to a file, the
|
||||
mark a later command reads. REA runs as the session user in the home, with no ``AGENTENV_*`` variable, so
|
||||
neither REA nor the app it launches holds the computer's control token, and with ``APT_CONFIG`` naming the
|
||||
package lists ``start`` downloads as that user, so the capture finds the package's sources.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
POLL_S = 0.2
|
||||
TAIL_BYTES = 4096
|
||||
|
||||
|
||||
def _say(value: dict) -> int:
|
||||
print(json.dumps(value), flush=True)
|
||||
return 0
|
||||
|
||||
|
||||
def _tail(path: str) -> str:
|
||||
try:
|
||||
with open(path, "rb") as handle:
|
||||
handle.seek(max(0, os.path.getsize(path) - TAIL_BYTES))
|
||||
return handle.read().decode("utf-8", "replace")
|
||||
except OSError:
|
||||
return ""
|
||||
|
||||
|
||||
def _environment(work: str) -> dict:
|
||||
environment = {name: value for name, value in os.environ.items() if not name.startswith("AGENTENV_")}
|
||||
environment["APT_CONFIG"] = os.path.join(work, "apt", "apt.conf")
|
||||
return environment
|
||||
|
||||
|
||||
def _apt(work: str, timeout: float) -> dict:
|
||||
"""Download the package lists into the work folder as the session user, until one update succeeds."""
|
||||
root = os.path.join(work, "apt")
|
||||
lists = os.path.join(root, "lists")
|
||||
updated = os.path.join(root, "updated")
|
||||
os.makedirs(os.path.join(lists, "partial"), exist_ok=True)
|
||||
os.makedirs(os.path.join(root, "cache", "archives", "partial"), exist_ok=True)
|
||||
with open(os.path.join(root, "apt.conf"), "w", encoding="utf-8") as conf:
|
||||
conf.write(
|
||||
f'Dir::State::Lists "{lists}";\n'
|
||||
f'Dir::Cache "{root}/cache";\n'
|
||||
'Debug::NoLocking "true";\n'
|
||||
'APT::Update::Error-Mode "any";\n'
|
||||
)
|
||||
# apt keeps lists compressed as it fetched them, such as *_Packages.lz4 on Ubuntu.
|
||||
count = lambda: len([name for name in os.listdir(lists) if re.search(r"_Packages(\.\w+)?$", name)]) # noqa: E731
|
||||
# A marker left by an update that fetched nothing is not a success; download again.
|
||||
if os.path.exists(updated) and count() > 0:
|
||||
return {"lists": count(), "updated": False}
|
||||
log = os.path.join(root, "update.log")
|
||||
try:
|
||||
with open(log, "wb") as output:
|
||||
code = subprocess.run(["apt-get", "update"], env=_environment(work), stdin=subprocess.DEVNULL,
|
||||
stdout=output, stderr=subprocess.STDOUT, timeout=timeout).returncode
|
||||
except (OSError, subprocess.TimeoutExpired) as error:
|
||||
return {"lists": count(), "updated": True, "error": f"{type(error).__name__}: {error}"}
|
||||
lists_fetched = count()
|
||||
if code == 0 and lists_fetched > 0:
|
||||
# Only a whole update that fetched lists is kept, so a failed or empty one runs again.
|
||||
open(updated, "w").close()
|
||||
return {"lists": lists_fetched, "updated": True, "exit_code": code}
|
||||
if code == 0:
|
||||
return {"lists": lists_fetched, "updated": True, "exit_code": code,
|
||||
"error": "apt-get update exited 0 without a package list"}
|
||||
return {"lists": lists_fetched, "updated": True, "exit_code": code,
|
||||
"error": f"apt-get update exited {code}: {_tail(log)[-1500:].strip()}"}
|
||||
|
||||
|
||||
def _exit_code(capture: str):
|
||||
try:
|
||||
with open(os.path.join(capture, "exit"), encoding="utf-8") as handle:
|
||||
return int(handle.read().strip() or "-1")
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _ended(capture: str, code: int) -> dict:
|
||||
"""A capture that has exited: its status, the end of its progress log and its output's size, which REA reads
|
||||
whole for the Evidence or failure it ended with, whichever step was waiting on it."""
|
||||
return {"exited": code, "progress": _tail(os.path.join(capture, "progress.log")),
|
||||
"output_bytes": os.path.getsize(os.path.join(capture, "capture.out"))}
|
||||
|
||||
|
||||
def _event_line(path: str, offset: int, kinds: set, name):
|
||||
"""The first progress line after ``offset`` whose event is of one of ``kinds``, for ``name`` when given."""
|
||||
try:
|
||||
with open(path, "rb") as handle:
|
||||
handle.seek(offset)
|
||||
lines = handle.read().decode("utf-8", "replace").splitlines()
|
||||
except OSError:
|
||||
return None
|
||||
for line in lines:
|
||||
try:
|
||||
event = json.loads(line)["rea_progress"]["event"]
|
||||
except (ValueError, KeyError, TypeError):
|
||||
continue
|
||||
if isinstance(event, dict) and event.get("kind") in kinds and (name is None or event.get("name") == name):
|
||||
return line
|
||||
return None
|
||||
|
||||
|
||||
def _wait(capture: str, offset: int, kinds: set, name, timeout: float) -> dict:
|
||||
"""Wait for the capture's event, or for it to exit, or for time to run out."""
|
||||
log = os.path.join(capture, "progress.log")
|
||||
deadline = time.monotonic() + timeout
|
||||
while True:
|
||||
line = _event_line(log, offset, kinds, name)
|
||||
if line is not None:
|
||||
return {"line": line}
|
||||
code = _exit_code(capture)
|
||||
if code is not None:
|
||||
return _ended(capture, code)
|
||||
if time.monotonic() > deadline:
|
||||
return {"timed_out": True, "progress": _tail(log)}
|
||||
time.sleep(POLL_S)
|
||||
|
||||
|
||||
def _within(path: str, folder: str) -> bool:
|
||||
return path == folder or path.startswith(folder + os.sep)
|
||||
|
||||
|
||||
def _state_paths(values: list, work: str):
|
||||
"""Each state path as an absolute path strictly under the home and outside REA's work folder, both as named
|
||||
and with its links resolved, or the refusal of the first that is not. The capture resolves the links again
|
||||
and retains what they lead to, so a link out of the home is refused here. Each passes on as named, so the
|
||||
capture keeps what its spelling means, such as a hidden settings folder."""
|
||||
home = os.path.normpath(os.path.expanduser("~"))
|
||||
real_home, real_work = os.path.realpath(home), os.path.realpath(work)
|
||||
spellings = {home, real_home}
|
||||
paths = []
|
||||
for value in values:
|
||||
relative = value[2:] if value.startswith("~/") else value
|
||||
path = os.path.normpath(os.path.join(home, relative))
|
||||
if ".." in relative.split("/") or not any(path.startswith(spelling + os.sep) for spelling in spellings):
|
||||
return None, f"The state path {value} is not under the computer's home, {home}."
|
||||
# Every link on the way is followed, a dangling one too; a path that does not exist yet resolves
|
||||
# through its nearest existing folder.
|
||||
resolved = os.path.realpath(path)
|
||||
if not resolved.startswith(real_home + os.sep):
|
||||
return None, f"The state path {value} leads through a link to {resolved}, which is not under the computer's home, {home}."
|
||||
if _within(path, work) or _within(resolved, real_work):
|
||||
return None, f"The state path {value} is in REA's work folder {work}, which every capture leaves out."
|
||||
paths.append(path)
|
||||
return paths, None
|
||||
|
||||
|
||||
def start(arguments: dict) -> int:
|
||||
work, capture = os.path.abspath(arguments["work"]), os.path.abspath(arguments["capture"])
|
||||
rea = shutil.which("rea")
|
||||
if rea is None:
|
||||
return _say({"error": "REA is not installed in this computer; start the computer from a request, which installs it."})
|
||||
state_paths, refusal = _state_paths(arguments.get("state_paths", []), work)
|
||||
if refusal is not None:
|
||||
return _say({"refused": refusal})
|
||||
if os.path.exists(capture):
|
||||
return _say({"error": f"the capture folder {capture} is already used"})
|
||||
apt = _apt(work, arguments.get("apt_timeout_s", 600))
|
||||
try:
|
||||
os.makedirs(capture)
|
||||
except FileExistsError:
|
||||
return _say({"error": f"a stop claimed {capture} before the capture launched", "apt": apt})
|
||||
control = os.path.join(capture, "control")
|
||||
os.mkfifo(control, 0o600)
|
||||
argv = [rea, "capture-desktop-application", "--desktop-entry", arguments["desktop_entry"],
|
||||
"--output", os.path.join(capture, "capture"), "--exclude", work, "--json"]
|
||||
for value in arguments.get("arguments", []):
|
||||
argv += ["--argument", value]
|
||||
for path in state_paths:
|
||||
argv += ["--state-path", path]
|
||||
if arguments.get("package_file"):
|
||||
argv += ["--package-file", arguments["package_file"]]
|
||||
# Reading and writing, so the capture never reads the end of its input between requests.
|
||||
reader = os.open(control, os.O_RDWR)
|
||||
process = None
|
||||
try:
|
||||
try:
|
||||
with open(os.path.join(capture, "capture.out"), "wb") as out, \
|
||||
open(os.path.join(capture, "progress.log"), "wb") as progress:
|
||||
process = subprocess.Popen(
|
||||
["sh", "-c", 'done=$1; shift; "$@"; code=$?; printf "%s" "$code" > "$done.tmp" && mv "$done.tmp" "$done"',
|
||||
"sh", os.path.join(capture, "exit"), *argv],
|
||||
cwd=os.path.expanduser("~"), env=_environment(work), stdin=reader, stdout=out, stderr=progress,
|
||||
start_new_session=True)
|
||||
except Exception as cause:
|
||||
# Nothing will write exit. Record one and drop the FIFO, so stop does not wait on it.
|
||||
try:
|
||||
with open(os.path.join(capture, "exit"), "w", encoding="utf-8") as handle:
|
||||
handle.write("1")
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
os.unlink(control)
|
||||
except OSError:
|
||||
pass
|
||||
return _say({"error": f"the capture did not start: {cause}"})
|
||||
finally:
|
||||
os.close(reader)
|
||||
if process is None:
|
||||
return _say({"error": "the capture did not start"})
|
||||
waited = _wait(capture, 0, {"window_open"}, None, arguments.get("timeout_s", 120))
|
||||
return _say({"pid": process.pid, "apt": apt, **waited})
|
||||
|
||||
|
||||
def _request(capture: str, line: str) -> bool:
|
||||
"""Write one request to the running capture; False when nothing reads its FIFO any more."""
|
||||
try:
|
||||
fd = os.open(os.path.join(capture, "control"), os.O_WRONLY | os.O_NONBLOCK)
|
||||
except OSError:
|
||||
return False
|
||||
try:
|
||||
os.write(fd, (line + "\n").encode("utf-8"))
|
||||
finally:
|
||||
os.close(fd)
|
||||
return True
|
||||
|
||||
|
||||
def mark(arguments: dict) -> int:
|
||||
"""Request a checkpoint and answer with the capture's acknowledgement line as it wrote it: the checkpoint's
|
||||
counts and up to ten of the files that changed, home-relative, with a count of the rest."""
|
||||
capture, name = os.path.abspath(arguments["capture"]), arguments["name"]
|
||||
code = _exit_code(capture)
|
||||
log = os.path.join(capture, "progress.log")
|
||||
if code is not None:
|
||||
return _say(_ended(capture, code))
|
||||
# A checkpoint already taken, whose answer was lost, is answered from the capture's own log.
|
||||
taken = _event_line(log, 0, {"checkpoint"}, name)
|
||||
if taken is not None:
|
||||
return _say({"line": taken})
|
||||
offset = os.path.getsize(log)
|
||||
if not _request(capture, f"checkpoint {name}"):
|
||||
return _say({"error": "the capture is not reading its requests"})
|
||||
return _say(_wait(capture, offset, {"checkpoint", "checkpoint_refused"}, name, arguments.get("timeout_s", 600)))
|
||||
|
||||
|
||||
def stop(arguments: dict) -> int:
|
||||
capture = os.path.abspath(arguments["capture"])
|
||||
if _exit_code(capture) is None and not os.path.exists(os.path.join(capture, "control")):
|
||||
# Never launched: claim the folder, so a start still on its way finds it used and launches nothing.
|
||||
os.makedirs(capture, exist_ok=True)
|
||||
return _say({"absent": True})
|
||||
if _exit_code(capture) is None:
|
||||
_request(capture, "stop")
|
||||
deadline = time.monotonic() + arguments.get("timeout_s", 600)
|
||||
while (code := _exit_code(capture)) is None and time.monotonic() < deadline:
|
||||
time.sleep(POLL_S)
|
||||
log = os.path.join(capture, "progress.log")
|
||||
if code is None:
|
||||
return _say({"timed_out": True, "progress": _tail(log)})
|
||||
output = os.path.join(capture, "capture.out")
|
||||
return _say({"exit_code": code, "progress": _tail(log), "output_bytes": os.path.getsize(output)})
|
||||
|
||||
|
||||
def run(arguments: dict) -> int:
|
||||
"""Run one REA command with its output in a file, after writing the files it reads."""
|
||||
rea = shutil.which("rea")
|
||||
if rea is None:
|
||||
return _say({"error": "REA is not installed in this computer"})
|
||||
os.makedirs(os.path.dirname(arguments["stdout"]), exist_ok=True)
|
||||
for path, text in arguments.get("files", {}).items():
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "w", encoding="utf-8") as handle:
|
||||
handle.write(text)
|
||||
with open(arguments["stdout"], "wb") as out:
|
||||
try:
|
||||
done = subprocess.run([rea, *arguments["argv"]], cwd=os.path.expanduser("~"),
|
||||
env=_environment(os.path.abspath(arguments["work"])), stdin=subprocess.DEVNULL, stdout=out,
|
||||
stderr=subprocess.PIPE, timeout=arguments.get("timeout_s", 1800))
|
||||
except subprocess.TimeoutExpired:
|
||||
return _say({"timed_out": True})
|
||||
return _say({"exit_code": done.returncode, "stdout_bytes": os.path.getsize(arguments["stdout"]),
|
||||
"stderr": done.stderr[-TAIL_BYTES:].decode("utf-8", "replace")})
|
||||
|
||||
|
||||
# The controller is sent with python -c, so a detached supervisor needs its
|
||||
# own source. It alone holds the child process handle and signals its group.
|
||||
LONG_RUN = r'''
|
||||
import json, os, signal, subprocess, sys, threading, time
|
||||
job = sys.argv[1]
|
||||
result = {}
|
||||
child = None
|
||||
drainer = None
|
||||
stderr_tail, stderr_errors = bytearray(), []
|
||||
|
||||
def drain(stream):
|
||||
try:
|
||||
while chunk := stream.read1(65536):
|
||||
stderr_tail.extend(chunk)
|
||||
del stderr_tail[:-4096]
|
||||
except OSError as error:
|
||||
stderr_errors.append(str(error))
|
||||
finally:
|
||||
try:
|
||||
stream.close()
|
||||
except OSError as error:
|
||||
stderr_errors.append(str(error))
|
||||
|
||||
def timed_out_marker():
|
||||
try:
|
||||
open(os.path.join(job, "timed_out"), "x").close()
|
||||
except FileExistsError:
|
||||
pass
|
||||
|
||||
try:
|
||||
request = json.loads("".join(sys.argv[2:]))
|
||||
if os.path.exists(os.path.join(job, "cancel")):
|
||||
result = {"absent": True}
|
||||
else:
|
||||
with open(request["stdout"], "wb") as out:
|
||||
child = subprocess.Popen(request["argv"], cwd=request["home"], stdin=subprocess.DEVNULL,
|
||||
stdout=out, stderr=subprocess.PIPE, start_new_session=True)
|
||||
drainer = threading.Thread(target=drain, args=(child.stderr,), daemon=True)
|
||||
drainer.start()
|
||||
deadline = time.monotonic() + request["timeout_s"]
|
||||
interrupted = False
|
||||
timed_out = False
|
||||
while child.poll() is None:
|
||||
timed_out = timed_out or time.monotonic() >= deadline
|
||||
if not interrupted and (timed_out or os.path.exists(os.path.join(job, "cancel"))):
|
||||
interrupted = True
|
||||
if timed_out:
|
||||
timed_out_marker()
|
||||
try:
|
||||
os.killpg(child.pid, signal.SIGINT)
|
||||
except OSError:
|
||||
pass
|
||||
time.sleep(0.2)
|
||||
# A descendant can still hold stderr after the leader exits. Do
|
||||
# not publish completion until that stream has actually closed.
|
||||
while drainer.is_alive():
|
||||
drainer.join(0.2)
|
||||
if time.monotonic() >= deadline:
|
||||
timed_out = True
|
||||
timed_out_marker()
|
||||
result = {"exit_code": child.returncode, "stdout_bytes": os.path.getsize(request["stdout"]),
|
||||
"timed_out": timed_out, "cancelled": interrupted and not timed_out,
|
||||
"stderr": bytes(stderr_tail).decode("utf-8", "replace")}
|
||||
if stderr_errors:
|
||||
result["error"] = f"The REA command's stderr could not be read: {stderr_errors[0]}"
|
||||
except Exception as error:
|
||||
result["error"] = f"The REA command supervisor failed: {error}"
|
||||
# Keep the child handle until it exits, including when cancellation arrives
|
||||
# after an unexpected supervisor error.
|
||||
while child is not None and child.poll() is None:
|
||||
if os.path.exists(os.path.join(job, "cancel")):
|
||||
try:
|
||||
os.killpg(child.pid, signal.SIGINT)
|
||||
except OSError:
|
||||
pass
|
||||
time.sleep(0.2)
|
||||
if drainer is not None and drainer.ident is not None:
|
||||
drainer.join()
|
||||
if child is not None:
|
||||
result.setdefault("exit_code", child.returncode)
|
||||
result.setdefault("stderr", bytes(stderr_tail).decode("utf-8", "replace"))
|
||||
try:
|
||||
result.setdefault("stdout_bytes", os.path.getsize(request["stdout"]))
|
||||
except OSError:
|
||||
pass
|
||||
if child is None or child.poll() is not None:
|
||||
temporary = os.path.join(job, "result.tmp")
|
||||
with open(temporary, "w", encoding="utf-8") as out:
|
||||
json.dump(result, out)
|
||||
os.replace(temporary, os.path.join(job, "result.json"))
|
||||
'''
|
||||
|
||||
|
||||
def run_begin(arguments: dict) -> int:
|
||||
"""Start one long REA command in its own capture-owned job folder."""
|
||||
rea = shutil.which("rea")
|
||||
if rea is None:
|
||||
return _say({"error": "REA is not installed in this computer"})
|
||||
job = os.path.abspath(arguments["job"])
|
||||
os.makedirs(os.path.dirname(job), exist_ok=True)
|
||||
try:
|
||||
os.mkdir(job, 0o700)
|
||||
except FileExistsError:
|
||||
return _say({"refused": "the command job folder is already used"})
|
||||
process = None
|
||||
try:
|
||||
with open(os.path.join(job, "owner"), "x", encoding="ascii") as out:
|
||||
out.write(arguments["token"])
|
||||
for path, text in arguments.get("files", {}).items():
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "w", encoding="utf-8") as handle:
|
||||
handle.write(text)
|
||||
os.makedirs(os.path.dirname(arguments["stdout"]), exist_ok=True)
|
||||
request = json.dumps({"argv": [rea, *arguments["argv"]], "stdout": arguments["stdout"],
|
||||
"home": os.path.expanduser("~"), "timeout_s": arguments["timeout_s"]})
|
||||
chunks = [request[i:i + 60000] for i in range(0, len(request), 60000)]
|
||||
with open(os.path.join(job, "supervisor.log"), "wb") as log:
|
||||
process = subprocess.Popen([sys.executable, "-I", "-c", LONG_RUN, job, *chunks], cwd=os.path.expanduser("~"),
|
||||
env=_environment(os.path.abspath(arguments["work"])), stdin=subprocess.DEVNULL,
|
||||
stdout=log, stderr=log, start_new_session=True)
|
||||
except (OSError, ValueError) as error:
|
||||
stage = "did not start" if process is None else "started but its setup did not finish"
|
||||
result = {"error": f"The REA command supervisor {stage}: {error}"}
|
||||
if process is None:
|
||||
temporary = os.path.join(job, "result.tmp")
|
||||
with open(temporary, "w", encoding="utf-8") as out:
|
||||
json.dump(result, out)
|
||||
os.replace(temporary, os.path.join(job, "result.json"))
|
||||
return _say(result)
|
||||
return _say({"started": True})
|
||||
|
||||
|
||||
def _run_result(job: str):
|
||||
try:
|
||||
with open(os.path.join(job, "result.json"), encoding="utf-8") as source:
|
||||
return json.load(source)
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
|
||||
|
||||
def _owns_run(job: str, token: str) -> bool:
|
||||
try:
|
||||
with open(os.path.join(job, "owner"), encoding="ascii") as owner:
|
||||
return owner.read() == token
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def run_poll(arguments: dict) -> int:
|
||||
job = os.path.abspath(arguments["job"])
|
||||
if not _owns_run(job, arguments["token"]):
|
||||
return _say({"error": "the command job ownership is not confirmed"})
|
||||
deadline = time.monotonic() + arguments.get("timeout_s", 30)
|
||||
while (result := _run_result(job)) is None and time.monotonic() < deadline:
|
||||
time.sleep(POLL_S)
|
||||
if result is None:
|
||||
pending = {"pending": True, "timed_out": os.path.exists(os.path.join(job, "timed_out"))}
|
||||
progress = _tail(os.path.join(job, "supervisor.log"))
|
||||
return _say({**pending, **({"progress": progress} if progress else {})})
|
||||
return _say({"finished": True, **result})
|
||||
|
||||
|
||||
def run_cancel(arguments: dict) -> int:
|
||||
job = os.path.abspath(arguments["job"])
|
||||
os.makedirs(os.path.dirname(job), exist_ok=True)
|
||||
try:
|
||||
os.mkdir(job, 0o700)
|
||||
except FileExistsError:
|
||||
pass
|
||||
else:
|
||||
# A late begin cannot launch into a folder cancellation claimed.
|
||||
with open(os.path.join(job, "owner"), "x", encoding="ascii") as owner:
|
||||
owner.write(arguments["token"])
|
||||
return _say({"absent": True})
|
||||
if not _owns_run(job, arguments["token"]):
|
||||
return _say({"error": "the command job ownership is not confirmed"})
|
||||
with open(os.path.join(job, "cancel"), "a", encoding="utf-8"):
|
||||
pass
|
||||
return run_poll(arguments)
|
||||
|
||||
|
||||
def listing(arguments: dict) -> int:
|
||||
"""Every regular file under a folder, by its path relative to the home, with its size and SHA-256.
|
||||
|
||||
The list goes to the file ``output`` names, outside the folder, and the answer gives its size: a large
|
||||
bundle's list outgrows the 1 MiB of stdout a command answers with, so REA reads it through ``data/get``.
|
||||
"""
|
||||
home = os.path.expanduser("~")
|
||||
files, skipped = [], []
|
||||
|
||||
def fail_walk(error: OSError) -> None:
|
||||
raise error
|
||||
|
||||
for directory, folders, names in os.walk(os.path.abspath(arguments["root"]), onerror=fail_walk):
|
||||
folders.sort()
|
||||
# os.walk does not follow directory symlinks; disclose them to the consumer.
|
||||
for name in folders[:]:
|
||||
path = os.path.join(directory, name)
|
||||
if os.path.islink(path):
|
||||
skipped.append(os.path.relpath(path, home))
|
||||
folders.remove(name)
|
||||
for name in sorted(names):
|
||||
path = os.path.join(directory, name)
|
||||
relative = os.path.relpath(path, home)
|
||||
if os.path.islink(path) or not os.path.isfile(path):
|
||||
skipped.append(relative)
|
||||
continue
|
||||
digest = hashlib.sha256()
|
||||
with open(path, "rb") as handle:
|
||||
for chunk in iter(lambda: handle.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
files.append({"path": relative, "size": os.path.getsize(path), "sha256": digest.hexdigest()})
|
||||
output = os.path.abspath(arguments["output"])
|
||||
os.makedirs(os.path.dirname(output), exist_ok=True)
|
||||
with open(output, "w", encoding="utf-8") as handle:
|
||||
json.dump({"files": files, "skipped": skipped}, handle)
|
||||
return _say({"listing_bytes": os.path.getsize(output)})
|
||||
|
||||
|
||||
COMMANDS = {"start": start, "mark": mark, "stop": stop, "run": run, "list": listing,
|
||||
"run-begin": run_begin, "run-poll": run_poll, "run-cancel": run_cancel}
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(COMMANDS[sys.argv[1]](json.loads("".join(sys.argv[2:]))))
|
||||
@@ -1,83 +0,0 @@
|
||||
"""Keep one agentenv computer up for REA until stdin closes or SIGTERM arrives.
|
||||
|
||||
REA runs ``computer.py <bundle folder> [task]`` with the agentenv runner's own interpreter, so ``agent_env``
|
||||
imports. It deploys the bundle's task, ``smoke`` by default, on this machine with
|
||||
``run_bundle(keep=True, emulate=False)``, prints one JSON line naming the computer's address, MCP URL, control
|
||||
token and sandbox id, or ``{"error": ...}``, and tears the computer down once its stdin closes or it gets SIGTERM
|
||||
or SIGINT. Everything else it or agentenv prints, a build's output included, goes to stderr. It exits 0 once
|
||||
the teardown left nothing up, 1 when it did, and 2 when no computer came up.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import threading
|
||||
|
||||
|
||||
def _protocol():
|
||||
"""A private copy of stdout for the JSON line; fd 1 then points at stderr, as do children that inherit it."""
|
||||
protocol = os.fdopen(os.dup(1), "w", encoding="utf-8")
|
||||
os.dup2(2, 1)
|
||||
sys.stdout = sys.stderr
|
||||
return protocol
|
||||
|
||||
|
||||
def _say(protocol, value: dict) -> None:
|
||||
protocol.write(json.dumps(value) + "\n")
|
||||
protocol.flush()
|
||||
|
||||
|
||||
def _computer(result):
|
||||
"""The run's deployed computer, or why there is none."""
|
||||
run = result.runs[0] if result.runs else None
|
||||
if run is None:
|
||||
return None, "the bundle ran no task"
|
||||
if run.error is not None or run.failed_steps:
|
||||
return None, f"the task failed: {run.error or run.failed_steps}"
|
||||
for env in run.context.deployed_envs:
|
||||
if env.environment_url and env.mcp_url and getattr(env, "sandbox_id", None):
|
||||
return env, None
|
||||
return None, "the task deployed no computer"
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
protocol = _protocol()
|
||||
bundle, task = argv[1], argv[2] if len(argv) > 2 else "smoke"
|
||||
try:
|
||||
from agent_env.bundle import run_bundle
|
||||
except ImportError as error:
|
||||
_say(protocol, {"error": f"agent_env does not import with {sys.executable}: {error}"})
|
||||
return 2
|
||||
try:
|
||||
result = run_bundle(bundle, tasks=[task], keep=True, emulate=False, sandbox="local",
|
||||
on_progress=lambda line: print(line, file=sys.stderr, flush=True))
|
||||
except BaseException as error: # noqa: BLE001 - a signal's RunInterrupted too; run_bundle tore its runs down
|
||||
_say(protocol, {"error": f"{type(error).__name__}: {error}"})
|
||||
return 2
|
||||
stop = threading.Event()
|
||||
for signum in (signal.SIGTERM, signal.SIGINT):
|
||||
signal.signal(signum, lambda *_: stop.set())
|
||||
env, problem = _computer(result)
|
||||
try:
|
||||
if env is None:
|
||||
_say(protocol, {"error": problem})
|
||||
else:
|
||||
_say(protocol, {"base_url": env.environment_url, "mcp_url": env.mcp_url,
|
||||
"control_token": env.control_token, "sandbox_id": env.sandbox_id})
|
||||
threading.Thread(target=lambda: (sys.stdin.buffer.read(), stop.set()), daemon=True).start()
|
||||
while not stop.wait(0.5):
|
||||
pass
|
||||
finally:
|
||||
torn_down = result.teardown()
|
||||
still_up = [sandbox for run in torn_down.runs if run.torn_down is not None for sandbox in run.torn_down.still_up]
|
||||
if still_up:
|
||||
print(f"still up after the teardown: {still_up}", file=sys.stderr, flush=True)
|
||||
return 1
|
||||
return 0 if env is not None else 2
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -1,121 +0,0 @@
|
||||
"""Run one solve of a bundle's task in a fresh agentenv computer for REA and print how it ended.
|
||||
|
||||
REA runs ``solve.py <bundle folder> <task> <model> <agent>`` with the agentenv runner's own interpreter, so
|
||||
``agent_env`` imports. It runs the task once on this machine with ``run_bundle``, the solver deployed as the agent
|
||||
and asked on the model, the endpoint taken from the environment or agentenv's config as a run takes it, and the
|
||||
computer torn down as the run ends. It prints one JSON line: the run's outcome, instance id, seconds, each check's
|
||||
result, the solver's turns and cost from its trajectory when that is a local file, and agentenv's version; or
|
||||
``{"error": ...}`` when the task could not run. Everything else, agentenv's progress included, goes to stderr. It exits
|
||||
0 after a run and 2 when none ran.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from importlib import metadata
|
||||
from pathlib import Path
|
||||
|
||||
#: The most characters of a check's description or a run's error REA keeps.
|
||||
MAX_TEXT = 400
|
||||
|
||||
|
||||
def _protocol():
|
||||
"""A private copy of stdout for the JSON line; fd 1 then points at stderr, as do children that inherit it."""
|
||||
protocol = os.fdopen(os.dup(1), "w", encoding="utf-8")
|
||||
os.dup2(2, 1)
|
||||
sys.stdout = sys.stderr
|
||||
return protocol
|
||||
|
||||
|
||||
def _say(protocol, value: dict) -> None:
|
||||
protocol.write(json.dumps(value) + "\n")
|
||||
protocol.flush()
|
||||
|
||||
|
||||
def _field(value, name: str):
|
||||
"""A field of agentenv's dataclasses, or of the dicts a stored context holds."""
|
||||
return value.get(name) if isinstance(value, dict) else getattr(value, name, None)
|
||||
|
||||
|
||||
def _trajectory_end(context) -> tuple:
|
||||
"""The solver's turns and cost from the last record of its local trajectory, or None for each."""
|
||||
for response in reversed(_field(context, "prompt_responses") or []):
|
||||
url = _field(response, "agent_trajectory_object_url") or ""
|
||||
if not url.startswith("file://"):
|
||||
continue
|
||||
try:
|
||||
records = json.loads(Path(url[len("file://"):]).read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return None, None
|
||||
last = records[-1] if isinstance(records, list) and records and isinstance(records[-1], dict) else {}
|
||||
turns, usd = last.get("num_turns"), last.get("total_cost_usd")
|
||||
return (turns if isinstance(turns, int) else None,
|
||||
float(usd) if isinstance(usd, (int, float)) and not isinstance(usd, bool) else None)
|
||||
return None, None
|
||||
|
||||
|
||||
def _checks(context, task: str) -> list:
|
||||
"""Each check's result as the task's verifier recorded it under the instance's verifications."""
|
||||
verifications = (_field(context, "metadata") or {}).get("verifications") or {}
|
||||
found = verifications.get(task) or next(iter(verifications.values()), {})
|
||||
results = found.get("results") if isinstance(found, dict) else None
|
||||
return [{"id": str(result.get("id")), "passed": result.get("result") is True,
|
||||
"description": str(result.get("description", ""))[:MAX_TEXT]}
|
||||
for result in results or [] if isinstance(result, dict)]
|
||||
|
||||
|
||||
def _version() -> str | None:
|
||||
try:
|
||||
return metadata.version("agentenv-framework")
|
||||
except metadata.PackageNotFoundError:
|
||||
return None
|
||||
|
||||
|
||||
def _cleanup(result) -> dict:
|
||||
"""Only a returned teardown report can confirm what agentenv released."""
|
||||
if result is None:
|
||||
return {"unknown": True, "sandboxes": []}
|
||||
unknown, sandboxes = False, []
|
||||
for run in result.runs:
|
||||
report = getattr(run, "torn_down", None)
|
||||
if report is None:
|
||||
unknown = True
|
||||
else:
|
||||
sandboxes.extend({"id": item.sandbox_id, "type": item.sandbox_type} for item in report.still_up)
|
||||
return {"unknown": unknown, "sandboxes": sandboxes}
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
protocol = _protocol()
|
||||
bundle, task, model, agent = argv[1:5]
|
||||
try:
|
||||
from agent_env.bundle import run_bundle
|
||||
except ImportError as error:
|
||||
_say(protocol, {"error": f"agent_env does not import with {sys.executable}: {error}",
|
||||
"cleanup": {"unknown": False, "sandboxes": []}})
|
||||
return 2
|
||||
try:
|
||||
result = run_bundle(bundle, tasks=[task], model=model, agent=agent, sandbox="local", emulate=False,
|
||||
on_progress=lambda line: print(line, file=sys.stderr, flush=True))
|
||||
except BaseException as error: # noqa: BLE001 - a signal's RunInterrupted too; run_bundle tore its run down
|
||||
_say(protocol, {"error": f"{type(error).__name__}: {error}"[:MAX_TEXT * 4],
|
||||
"cleanup": _cleanup(getattr(error, "result", None))})
|
||||
return 2
|
||||
run = result.runs[0] if result.runs else None
|
||||
if run is None:
|
||||
_say(protocol, {"error": "the bundle ran no task", "cleanup": _cleanup(result)})
|
||||
return 2
|
||||
turns, usd = _trajectory_end(run.context)
|
||||
error = getattr(run, "error", None)
|
||||
_say(protocol, {"outcome": run.outcome.value, "instance_id": run.instance_id,
|
||||
"seconds": round(float(run.duration), 1), "checks": _checks(run.context, task),
|
||||
"turns": turns, "usd": usd, "agentenv": _version(),
|
||||
"error": None if error is None else f"{type(error).__name__}: {error}"[:MAX_TEXT],
|
||||
"cleanup": _cleanup(result)})
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -1,63 +0,0 @@
|
||||
"""Recover local sandboxes under one stopped solve's private agentenv root."""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _docker(*args: str) -> str:
|
||||
command = os.environ.get("REA_DOCKER_COMMAND") or "docker"
|
||||
result = subprocess.run([command, *args], capture_output=True, text=True, check=False)
|
||||
if result.returncode:
|
||||
raise RuntimeError(f"Docker {' '.join(args[:2])} failed: {result.stderr[-1000:]}")
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
async def cleanup(root: Path) -> None:
|
||||
# A helper has already stopped. agentenv lists the sandboxes under the root
|
||||
# (AGENT_ENV_LOCAL_SANDBOX_DIR); anything else there remains owned for inspection.
|
||||
if not any(root.iterdir()):
|
||||
return
|
||||
from agent_env.providers.sandbox_providers.local_docker.provider import (
|
||||
LocalSandboxProvider, local_sandbox_ids, remove_local_work_dir,
|
||||
)
|
||||
for sandbox_id in local_sandbox_ids():
|
||||
await LocalSandboxProvider().terminate_sandbox(sandbox_id)
|
||||
# The pinned provider reports incomplete teardown and retains the folder.
|
||||
# Keep REA's independent Docker/Linux checks before releasing that owner.
|
||||
if _docker("ps", "-aq", "--filter", f"name=^agent-{sandbox_id}$"):
|
||||
raise RuntimeError(f"Sandbox {sandbox_id} still has its agent container")
|
||||
if _docker("ps", "-aq", "--filter", f"label=agentenv.sandbox={sandbox_id}"):
|
||||
raise RuntimeError(f"Sandbox {sandbox_id} still has labeled containers")
|
||||
if _docker("network", "ls", "-q", "--filter", f"label=agentenv.sandbox={sandbox_id}"):
|
||||
raise RuntimeError(f"Sandbox {sandbox_id} still has labeled networks")
|
||||
if sys.platform != "linux":
|
||||
raise RuntimeError(f"Cannot verify host processes for sandbox {sandbox_id} on {sys.platform}")
|
||||
from agent_env.providers.sandbox_providers.local_docker.provider import _marked_pids
|
||||
if _marked_pids(sandbox_id):
|
||||
raise RuntimeError(f"Sandbox {sandbox_id} still has marked host processes")
|
||||
if remove_local_work_dir(sandbox_id) is None:
|
||||
raise RuntimeError(f"Could not confirm removal of sandbox {sandbox_id}'s work folder")
|
||||
leftover = sorted(entry.name for entry in root.iterdir())
|
||||
if leftover:
|
||||
raise RuntimeError(f"Unexpected sandbox root entries: {', '.join(leftover)}")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
try:
|
||||
with contextlib.redirect_stdout(sys.stderr):
|
||||
asyncio.run(cleanup(Path(os.environ["AGENT_ENV_LOCAL_SANDBOX_DIR"])))
|
||||
print(json.dumps({"ok": True}))
|
||||
return 0
|
||||
except Exception as error:
|
||||
print(json.dumps({"ok": False, "error": f"{type(error).__name__}: {error}"}))
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,232 +0,0 @@
|
||||
"""Exercise the capture controller's filesystem inventory boundary."""
|
||||
|
||||
import json
|
||||
import io
|
||||
from contextlib import redirect_stdout
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from bridge.agentenv import capture
|
||||
|
||||
|
||||
CONTROLLER = Path(__file__).with_name("capture.py")
|
||||
|
||||
|
||||
class CaptureListingTest(unittest.TestCase):
|
||||
def run_listing(self, root, output):
|
||||
return subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
str(CONTROLLER),
|
||||
"list",
|
||||
json.dumps({"root": str(root), "output": str(output)}),
|
||||
],
|
||||
text=True,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
def test_directory_symlink_is_disclosed_without_following_it(self):
|
||||
with tempfile.TemporaryDirectory() as parent:
|
||||
parent = Path(parent)
|
||||
root = parent / "root"
|
||||
target = parent / "target"
|
||||
output = parent / "listing.json"
|
||||
root.mkdir()
|
||||
target.mkdir()
|
||||
(root / "present.txt").write_text("present")
|
||||
(target / "outside.txt").write_text("outside")
|
||||
try:
|
||||
(root / "linked").symlink_to(target, target_is_directory=True)
|
||||
except OSError as error:
|
||||
self.skipTest(f"directory symlinks unavailable: {error}")
|
||||
result = self.run_listing(root, output)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
listing = json.loads(output.read_text())
|
||||
self.assertEqual(
|
||||
[item["path"] for item in listing["files"]],
|
||||
[os.path.relpath(root / "present.txt", Path.home())],
|
||||
)
|
||||
self.assertEqual(
|
||||
listing["skipped"], [os.path.relpath(root / "linked", Path.home())]
|
||||
)
|
||||
|
||||
def test_missing_root_does_not_produce_an_empty_successful_inventory(self):
|
||||
with tempfile.TemporaryDirectory() as parent:
|
||||
parent = Path(parent)
|
||||
output = parent / "listing.json"
|
||||
result = self.run_listing(parent / "missing", output)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertFalse(output.exists())
|
||||
|
||||
|
||||
@unittest.skipIf(sys.platform == "win32", "desktop build computers use Linux")
|
||||
class LongRunTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
self.root = Path(self.tmp.name)
|
||||
self.bin = self.root / "bin"
|
||||
self.bin.mkdir()
|
||||
rea = self.bin / "rea"
|
||||
rea.write_text(
|
||||
f"#!{sys.executable}\n"
|
||||
"import os, signal, subprocess, sys, time\n"
|
||||
"signal.signal(signal.SIGINT, lambda *_: sys.exit(130))\n"
|
||||
"if len(sys.argv) > 1 and sys.argv[1] == 'wait':\n"
|
||||
" open(sys.argv[2], 'w').close()\n"
|
||||
" time.sleep(30)\n"
|
||||
"if len(sys.argv) > 1 and sys.argv[1] == 'ignore':\n"
|
||||
" signal.signal(signal.SIGINT, signal.SIG_IGN)\n"
|
||||
" open(sys.argv[2], 'w').close()\n"
|
||||
" while not os.path.exists(sys.argv[3]): time.sleep(0.02)\n"
|
||||
"if len(sys.argv) > 1 and sys.argv[1] == 'stderr':\n"
|
||||
" sys.stderr.write('x' * 100000 + 'TAIL')\n"
|
||||
"if len(sys.argv) > 1 and sys.argv[1] == 'hold':\n"
|
||||
" subprocess.Popen([sys.executable, '-c', 'import os,sys,time\\nwhile not os.path.exists(sys.argv[1]): time.sleep(0.02)', sys.argv[2]], stdout=subprocess.DEVNULL, stderr=sys.stderr, start_new_session=True)\n"
|
||||
" sys.exit(0)\n"
|
||||
"print(os.environ.get('AGENTENV_CONTROL_TOKEN', 'no token'))\n"
|
||||
)
|
||||
rea.chmod(0o755)
|
||||
self.environment = {**os.environ, "HOME": str(self.root), "PATH": f"{self.bin}{os.pathsep}{os.environ.get('PATH', '')}",
|
||||
"AGENTENV_CONTROL_TOKEN": "not-for-the-child"}
|
||||
self.job = self.root / "job"
|
||||
self.output = self.root / "out.json"
|
||||
|
||||
def command(self, name, **values):
|
||||
result = subprocess.run([sys.executable, str(CONTROLLER), name, json.dumps(values)],
|
||||
env=self.environment, text=True, capture_output=True, timeout=10, check=True)
|
||||
return json.loads(result.stdout)
|
||||
|
||||
def begin(self, argv, timeout=60):
|
||||
return self.command("run-begin", job=str(self.job), work=str(self.root), stdout=str(self.output),
|
||||
token="owner-token", argv=argv, files={}, timeout_s=timeout)
|
||||
|
||||
def test_runs_and_reports_its_output(self):
|
||||
self.assertEqual(self.begin([]), {"started": True})
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertEqual(result["exit_code"], 0)
|
||||
self.assertEqual(self.output.read_text().strip(), "no token")
|
||||
|
||||
def test_passes_large_command_without_persisting_its_arguments(self):
|
||||
self.assertEqual(self.begin(["done", *("x" * 4000 for _ in range(20))]), {"started": True})
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertEqual(result["exit_code"], 0)
|
||||
self.assertFalse((self.job / "request.json").exists())
|
||||
|
||||
def test_keeps_only_a_bounded_stderr_tail(self):
|
||||
self.assertEqual(self.begin(["stderr"]), {"started": True})
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertEqual(len(result["stderr"]), 4096)
|
||||
self.assertTrue(result["stderr"].endswith("TAIL"))
|
||||
|
||||
def test_cancels_a_running_child_and_confirms_exit(self):
|
||||
ready = self.root / "ready"
|
||||
self.assertEqual(self.begin(["wait", str(ready)]), {"started": True})
|
||||
import time
|
||||
deadline = time.monotonic() + 5
|
||||
while not ready.exists() and time.monotonic() < deadline:
|
||||
time.sleep(0.02)
|
||||
self.assertTrue(ready.exists())
|
||||
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertTrue(result["cancelled"])
|
||||
self.assertEqual(result["exit_code"], 130)
|
||||
|
||||
def test_cancel_before_late_start_claims_the_job(self):
|
||||
self.assertEqual(self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=1), {"absent": True})
|
||||
self.assertIn("already used", self.begin([])["refused"])
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_cancel_refuses_a_folder_without_the_matching_owner(self):
|
||||
self.job.mkdir()
|
||||
(self.job / "owner").write_text("someone-else")
|
||||
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=1)
|
||||
self.assertIn("ownership is not confirmed", result["error"])
|
||||
self.assertFalse((self.job / "cancel").exists())
|
||||
|
||||
def test_reports_supervisor_output_if_completion_is_unconfirmed(self):
|
||||
self.job.mkdir()
|
||||
(self.job / "owner").write_text("owner-token")
|
||||
(self.job / "supervisor.log").write_text("supervisor failed")
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=0)
|
||||
self.assertEqual(result["progress"], "supervisor failed")
|
||||
self.assertTrue(result["pending"])
|
||||
|
||||
def test_reports_a_timed_out_child_that_has_not_stopped(self):
|
||||
ready, release = self.root / "ready", self.root / "release"
|
||||
self.assertEqual(self.begin(["ignore", str(ready), str(release)], timeout=3), {"started": True})
|
||||
try:
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=4)
|
||||
self.assertTrue(result["pending"])
|
||||
self.assertTrue(result["timed_out"])
|
||||
self.assertTrue(ready.exists())
|
||||
finally:
|
||||
release.touch()
|
||||
self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
|
||||
def test_waits_for_an_inherited_stderr_pipe_after_the_child_exits(self):
|
||||
release = self.root / "release"
|
||||
self.assertEqual(self.begin(["hold", str(release)], timeout=3), {"started": True})
|
||||
try:
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=4)
|
||||
self.assertTrue(result["pending"])
|
||||
self.assertTrue(result["timed_out"])
|
||||
finally:
|
||||
release.touch()
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertTrue(result["finished"])
|
||||
|
||||
def test_failure_closing_supervisor_log_after_launch_does_not_claim_exit(self):
|
||||
ready = self.root / "ready"
|
||||
original_open = open
|
||||
original_popen = subprocess.Popen
|
||||
supervisors = []
|
||||
|
||||
class FailingClose:
|
||||
def __init__(self, file):
|
||||
self.file = file
|
||||
|
||||
def __enter__(self):
|
||||
return self.file
|
||||
|
||||
def __exit__(self, *args):
|
||||
self.file.close()
|
||||
raise OSError("log close failed")
|
||||
|
||||
def opening(path, *args, **kwargs):
|
||||
file = original_open(path, *args, **kwargs)
|
||||
return FailingClose(file) if str(path).endswith("supervisor.log") else file
|
||||
|
||||
def spawning(*args, **kwargs):
|
||||
process = original_popen(*args, **kwargs)
|
||||
supervisors.append(process)
|
||||
def cleanup():
|
||||
if process.poll() is None:
|
||||
(self.job / "cancel").touch()
|
||||
process.wait(timeout=5)
|
||||
self.addCleanup(cleanup)
|
||||
return process
|
||||
|
||||
arguments = {"job": str(self.job), "work": str(self.root), "stdout": str(self.output),
|
||||
"token": "owner-token", "argv": ["wait", str(ready)], "files": {}, "timeout_s": 60}
|
||||
with patch.dict(os.environ, self.environment, clear=True), patch("builtins.open", side_effect=opening), patch.object(capture.subprocess, "Popen", side_effect=spawning), redirect_stdout(io.StringIO()) as output:
|
||||
capture.run_begin(arguments)
|
||||
self.assertIn("started but its setup did not finish", json.loads(output.getvalue())["error"])
|
||||
import time
|
||||
deadline = time.monotonic() + 5
|
||||
while not ready.exists() and time.monotonic() < deadline:
|
||||
time.sleep(0.02)
|
||||
self.assertTrue(ready.exists())
|
||||
self.assertFalse((self.job / "result.json").exists())
|
||||
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertTrue(result["cancelled"])
|
||||
supervisors[0].wait(timeout=5)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,108 +0,0 @@
|
||||
"""Exercise cleanup decisions without a live agentenv provider."""
|
||||
import asyncio
|
||||
import tempfile
|
||||
import types
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from bridge.agentenv import solve_cleanup
|
||||
|
||||
|
||||
class SolveCleanupTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.folder = self.root / "agent-env-local-a1b2c3d4-abcdef12"
|
||||
self.folder.mkdir()
|
||||
self.terminated = []
|
||||
self.module = types.ModuleType("agent_env.providers.sandbox_providers.local_docker.provider")
|
||||
test = self
|
||||
|
||||
class Provider:
|
||||
async def terminate_sandbox(self, sandbox_id):
|
||||
test.terminated.append(sandbox_id)
|
||||
|
||||
self.module.LocalSandboxProvider = Provider
|
||||
self.module.local_sandbox_ids = self._listed
|
||||
self.module.remove_local_work_dir = lambda sandbox_id: self._remove(sandbox_id)
|
||||
self.module._marked_pids = lambda _sandbox_id: []
|
||||
self.patch_module = patch.dict("sys.modules", {self.module.__name__: self.module})
|
||||
self.patch_module.start()
|
||||
self.addCleanup(self.patch_module.stop)
|
||||
|
||||
def _listed(self):
|
||||
# As agentenv lists them: `agent-env-<id>-<suffix>` folders, whatever the id's shape.
|
||||
found = set()
|
||||
for entry in self.root.iterdir():
|
||||
sandbox_id, _, suffix = entry.name.removeprefix("agent-env-").rpartition("-")
|
||||
if entry.name.startswith("agent-env-") and sandbox_id and suffix and entry.is_dir():
|
||||
found.add(sandbox_id)
|
||||
return sorted(found)
|
||||
|
||||
def _remove(self, sandbox_id):
|
||||
folder = next(self.root.glob(f"agent-env-{sandbox_id}-*"))
|
||||
folder.rmdir()
|
||||
return folder
|
||||
|
||||
def run_cleanup(self):
|
||||
asyncio.run(solve_cleanup.cleanup(self.root))
|
||||
|
||||
def test_terminates_owned_sandbox_before_removing_its_folder(self):
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
self.run_cleanup()
|
||||
self.assertEqual(self.terminated, ["local-a1b2c3d4"])
|
||||
self.assertFalse(self.folder.exists())
|
||||
|
||||
def test_retains_folder_when_docker_reports_container_then_retries(self):
|
||||
with patch.object(solve_cleanup, "_docker", return_value="still-running"), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
with self.assertRaisesRegex(RuntimeError, "still has its agent container"):
|
||||
self.run_cleanup()
|
||||
self.assertTrue(self.folder.exists())
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
self.run_cleanup()
|
||||
self.assertFalse(self.folder.exists())
|
||||
self.assertEqual(len(self.terminated), 2)
|
||||
|
||||
def test_partial_cleanup_and_late_folder_are_retried(self):
|
||||
later = self.root / "agent-env-local-f1223344-abcdef12"
|
||||
later.mkdir()
|
||||
def docker(*args):
|
||||
return "still-running" if "local-f1223344" in " ".join(args) else ""
|
||||
with patch.object(solve_cleanup, "_docker", side_effect=docker), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
with self.assertRaisesRegex(RuntimeError, "still has its agent container"):
|
||||
self.run_cleanup()
|
||||
self.assertTrue(later.exists())
|
||||
self.assertFalse(self.folder.exists())
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
self.run_cleanup()
|
||||
self.assertEqual(list(self.root.iterdir()), [])
|
||||
|
||||
def test_macos_keeps_folder_when_host_process_cleanup_cannot_be_verified(self):
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "darwin"):
|
||||
with self.assertRaisesRegex(RuntimeError, "Cannot verify host processes"):
|
||||
self.run_cleanup()
|
||||
self.assertTrue(self.folder.exists())
|
||||
|
||||
def test_cleans_up_each_sandbox_agentenv_lists_whatever_its_id_looks_like(self):
|
||||
self.folder.rmdir()
|
||||
listed = self.root / "agent-env-local-a1b2c3d4e5-abcdef12"
|
||||
listed.mkdir()
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
self.run_cleanup()
|
||||
self.assertEqual(self.terminated, ["local-a1b2c3d4e5"])
|
||||
self.assertFalse(listed.exists())
|
||||
|
||||
def test_unexpected_entry_remains_owned(self):
|
||||
self.folder.rmdir()
|
||||
(self.root / "unexpected").mkdir()
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
with self.assertRaisesRegex(RuntimeError, "Unexpected sandbox root entries: unexpected"):
|
||||
self.run_cleanup()
|
||||
self.assertEqual(self.terminated, [])
|
||||
self.assertTrue((self.root / "unexpected").exists())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,35 +1,20 @@
|
||||
import com.atxx.jhmcp.*;
|
||||
import com.google.gson.*;
|
||||
import com.android.tools.smali.dexlib2.Opcodes;
|
||||
import com.android.tools.smali.dexlib2.dexbacked.DexBackedDexFile;
|
||||
import com.android.tools.smali.dexlib2.dexbacked.DexBackedClassDef;
|
||||
import com.android.tools.smali.dexlib2.dexbacked.DexBackedMethod;
|
||||
import io.modelcontextprotocol.kotlin.sdk.server.*;
|
||||
import io.modelcontextprotocol.kotlin.sdk.types.*;
|
||||
import jadx.api.*;
|
||||
import jadx.api.plugins.input.data.annotations.EncodedValue;
|
||||
import jadx.api.plugins.input.data.attributes.JadxAttrType;
|
||||
import jadx.api.plugins.input.data.ICodeReader;
|
||||
import jadx.api.plugins.input.data.IMethodRef;
|
||||
import jadx.api.plugins.input.insns.InsnIndexType;
|
||||
import jadx.api.plugins.input.insns.Opcode;
|
||||
import jadx.core.codegen.TypeGen;
|
||||
import jadx.core.dex.info.MethodInfo;
|
||||
import jadx.core.dex.instructions.args.ArgType;
|
||||
import jadx.core.dex.nodes.*;
|
||||
import jadx.core.xmlgen.BinaryXMLParser;
|
||||
import java.io.PrintStream;
|
||||
import java.io.StringReader;
|
||||
import java.io.InputStream;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.CountDownLatch;
|
||||
import java.util.stream.Collectors;
|
||||
import java.util.zip.CRC32;
|
||||
import java.util.zip.ZipEntry;
|
||||
import java.util.zip.ZipFile;
|
||||
import kotlin.Unit;
|
||||
import kotlin.ResultKt;
|
||||
import kotlin.coroutines.Continuation;
|
||||
@@ -52,25 +37,16 @@ class ReaJadxBridge {
|
||||
private Map<String, List<ClassSnapshot>> classes = Map.of();
|
||||
private JadxSession manifestSession;
|
||||
private ManifestSnapshot manifestSnapshot;
|
||||
private ExternalInventory externalInventory;
|
||||
|
||||
private record MethodSnapshot(String name, String rawName, String shortId, List<String> bodyAbsenceMarkers,
|
||||
Map<String, Object> summary) {}
|
||||
private record ClassSnapshot(JavaClass node, List<MethodSnapshot> methods, Map<String, Object> summary,
|
||||
Map<String, Object> summaryWithValues) {}
|
||||
private record ClassSnapshot(JavaClass node, List<MethodSnapshot> methods, Map<String, Object> summary) {}
|
||||
private record ManifestSnapshot(String text, Map<String, Object> summary) {}
|
||||
private record ExternalMethod(MethodInfo info, Map<String, Object> summary) {}
|
||||
private record ExternalInventory(JadxSession session, String className, String methodName,
|
||||
List<ExternalMethod> methods) {}
|
||||
|
||||
private static final class InputFailure extends IllegalArgumentException {
|
||||
final String field;
|
||||
InputFailure(String field, String message) { super(message); this.field = field; }
|
||||
}
|
||||
private static final class ReferenceLimit extends IllegalArgumentException {
|
||||
final String stage;
|
||||
ReferenceLimit(String stage) { super("External reference representation limit"); this.stage = stage; }
|
||||
}
|
||||
|
||||
private ReaJadxBridge(SessionHolder holder) { this.holder = holder; }
|
||||
|
||||
@@ -166,14 +142,6 @@ class ReaJadxBridge {
|
||||
return value.getAsString();
|
||||
}
|
||||
|
||||
private static boolean flag(JsonObject args, String name) {
|
||||
JsonElement value = args.get(name);
|
||||
if (value == null) return false;
|
||||
if (!value.isJsonPrimitive() || !value.getAsJsonPrimitive().isBoolean())
|
||||
throw new IllegalArgumentException(name + " must be a boolean");
|
||||
return value.getAsBoolean();
|
||||
}
|
||||
|
||||
private static int integer(JsonObject args, String name, int fallback) {
|
||||
JsonElement value = args.get(name);
|
||||
if (value == null) return fallback;
|
||||
@@ -197,206 +165,6 @@ class ReaJadxBridge {
|
||||
private void clearManifest() {
|
||||
manifestSession = null;
|
||||
manifestSnapshot = null;
|
||||
externalInventory = null;
|
||||
}
|
||||
|
||||
// One bounded DEX at a time, beside the already admitted analysis JVM. The
|
||||
// response allowance accounts for worst-case JSON escaping before retention.
|
||||
private static final int NATIVE_DEX_BYTES = 64 * 1024 * 1024;
|
||||
private static final long NATIVE_TOTAL_DEX_BYTES = 256L * 1024 * 1024;
|
||||
// This JSON is escaped again inside an 8 MiB MCP frame.
|
||||
private static final long NATIVE_JSON_BYTES = 4L * 1024 * 1024;
|
||||
|
||||
/** The bundled decoder allocates its declared UTF-16 length before validating bytes. */
|
||||
private static final class NativeDex extends DexBackedDexFile {
|
||||
private OptionalIndexedSection<String> boundedStrings;
|
||||
NativeDex(byte[] bytes) { super(Opcodes.getDefault(), bytes); }
|
||||
|
||||
@Override public OptionalIndexedSection<String> getStringSection() {
|
||||
if (boundedStrings == null) {
|
||||
var source = super.getStringSection();
|
||||
boundedStrings = new OptionalIndexedSection<>() {
|
||||
@Override public int size() { return source.size(); }
|
||||
@Override public int getOffset(int index) { return source.getOffset(index); }
|
||||
@Override public String getOptional(int index) { return index == -1 ? null : get(index); }
|
||||
@Override public String get(int index) {
|
||||
int offset = getBuffer().readSmallUint(source.getOffset(index));
|
||||
int units = getDataBuffer().readerAt(offset).readSmallUleb128();
|
||||
if (units > NATIVE_JSON_BYTES / 6
|
||||
|| units > getDataBuffer().getBuf().length - offset)
|
||||
throw new IllegalArgumentException("DEX string exceeds the native decoding allowance");
|
||||
return source.get(index);
|
||||
}
|
||||
};
|
||||
}
|
||||
return boundedStrings;
|
||||
}
|
||||
}
|
||||
|
||||
private static void checkNativeInterrupted() {
|
||||
if (Thread.currentThread().isInterrupted())
|
||||
throw new java.util.concurrent.CancellationException("Native declaration inventory interrupted");
|
||||
}
|
||||
|
||||
private static final class NativeInventory {
|
||||
final List<Map<String, Object>> methods = new ArrayList<>();
|
||||
final List<String> unknowns = new ArrayList<>();
|
||||
long retainedBytes = 1024;
|
||||
int dexEntries;
|
||||
int examinedDexEntries;
|
||||
int examinedClasses;
|
||||
int examinedMethods;
|
||||
boolean partial;
|
||||
boolean diagnosticLimit;
|
||||
|
||||
void unknown(String reason) {
|
||||
partial = true;
|
||||
long bytes = 64L + reason.length() * 6L;
|
||||
if (unknowns.size() < 16 && retainedBytes + bytes <= NATIVE_JSON_BYTES - 4096) {
|
||||
unknowns.add(reason);
|
||||
retainedBytes += bytes;
|
||||
} else if (!diagnosticLimit) {
|
||||
diagnosticLimit = true;
|
||||
unknowns.add("Additional native-inventory limitations exceed the retained diagnostic allowance.");
|
||||
}
|
||||
}
|
||||
|
||||
boolean retain(Map<String, Object> method, long bytes) {
|
||||
if (retainedBytes + bytes > NATIVE_JSON_BYTES - 4096) {
|
||||
unknown("Native declarations exceed the 4 MiB retained JSON allowance; remaining declarations are unknown.");
|
||||
return false;
|
||||
}
|
||||
retainedBytes += bytes;
|
||||
methods.add(method);
|
||||
return true;
|
||||
}
|
||||
|
||||
Map<String, Object> result() {
|
||||
return Map.of("scope", "apk-root-dex-declarations", "methods", List.copyOf(methods),
|
||||
"coverage", Map.of("status", partial ? "partial" : "complete",
|
||||
"dex_entries", dexEntries, "examined_dex_entries", examinedDexEntries,
|
||||
"examined_classes", examinedClasses, "examined_methods", examinedMethods),
|
||||
"unknowns", List.copyOf(unknowns));
|
||||
}
|
||||
}
|
||||
|
||||
/** Match Android's standard root multidex carriers, without a basename guess. */
|
||||
private static boolean nativeDexEntry(String name) {
|
||||
return name.equals("classes.dex") || name.matches("classes[0-9]+\\.dex");
|
||||
}
|
||||
|
||||
private static byte[] readNativeDex(ZipFile archive, ZipEntry entry) throws IOException {
|
||||
long size = entry.getSize();
|
||||
if (size < 0 || size > NATIVE_DEX_BYTES)
|
||||
throw new IllegalArgumentException("DEX exceeds the 64 MiB single-member read allowance");
|
||||
byte[] bytes = new byte[(int) size];
|
||||
CRC32 crc = new CRC32();
|
||||
try (InputStream input = archive.getInputStream(entry)) {
|
||||
int offset = 0;
|
||||
while (offset < bytes.length) {
|
||||
checkNativeInterrupted();
|
||||
int read = input.read(bytes, offset, Math.min(65536, bytes.length - offset));
|
||||
if (read < 0) throw new IOException("DEX is shorter than its ZIP declaration");
|
||||
if (read == 0) continue;
|
||||
crc.update(bytes, offset, read);
|
||||
offset += read;
|
||||
}
|
||||
if (input.read() != -1) throw new IOException("DEX exceeds its ZIP declaration");
|
||||
}
|
||||
if (entry.getCrc() < 0 || entry.getCrc() != crc.getValue())
|
||||
throw new IOException("DEX does not match its ZIP CRC32 declaration");
|
||||
return bytes;
|
||||
}
|
||||
|
||||
/** Raw DEX ownership precedes JADX's duplicate-class selection and aliases. */
|
||||
static Map<String, Object> nativeDeclarations(Path apk) {
|
||||
NativeInventory inventory = new NativeInventory();
|
||||
try (ZipFile archive = new ZipFile(apk.toFile())) {
|
||||
Map<String, Integer> names = new HashMap<>();
|
||||
var directory = archive.entries();
|
||||
while (directory.hasMoreElements()) {
|
||||
checkNativeInterrupted();
|
||||
ZipEntry entry = directory.nextElement();
|
||||
if (!entry.isDirectory() && nativeDexEntry(entry.getName())) {
|
||||
inventory.dexEntries++;
|
||||
names.merge(entry.getName(), 1, Integer::sum);
|
||||
}
|
||||
}
|
||||
long totalBytes = 0;
|
||||
int entryIndex = -1;
|
||||
directory = archive.entries();
|
||||
while (directory.hasMoreElements()) {
|
||||
checkNativeInterrupted();
|
||||
ZipEntry entry = directory.nextElement();
|
||||
entryIndex++;
|
||||
String path = entry.getName();
|
||||
if (entry.isDirectory() || !nativeDexEntry(path)) continue;
|
||||
if (names.get(path) != 1) {
|
||||
inventory.unknown("Duplicate root DEX entry name cannot be selected uniquely by the ZIP reader: " + path);
|
||||
continue;
|
||||
}
|
||||
if (entry.getSize() < 0 || entry.getSize() > NATIVE_DEX_BYTES
|
||||
|| entry.getSize() > NATIVE_TOTAL_DEX_BYTES - totalBytes) {
|
||||
inventory.unknown("Root DEX member exceeds the 64 MiB member or 256 MiB total read allowance: " + path);
|
||||
continue;
|
||||
}
|
||||
totalBytes += entry.getSize();
|
||||
try {
|
||||
DexBackedDexFile dex = new NativeDex(readNativeDex(archive, entry));
|
||||
var classes = dex.getClassSection();
|
||||
for (int classIndex = 0; classIndex < classes.size(); classIndex++) {
|
||||
checkNativeInterrupted();
|
||||
DexBackedClassDef cls = classes.get(classIndex);
|
||||
String type = cls.getType();
|
||||
if (!type.startsWith("L") || !type.endsWith(";"))
|
||||
throw new IllegalArgumentException("Class definition has no reference type");
|
||||
String className = type.substring(1, type.length() - 1).replace('/', '.');
|
||||
// Passing false retains duplicate encoded method references too.
|
||||
for (Iterable<? extends DexBackedMethod> group : List.of(
|
||||
cls.getDirectMethods(false), cls.getVirtualMethods(false))) {
|
||||
for (DexBackedMethod method : group) {
|
||||
checkNativeInterrupted();
|
||||
inventory.examinedMethods++;
|
||||
if ((method.getAccessFlags() & 0x100) == 0) continue;
|
||||
String methodName = method.getName();
|
||||
String returnType = method.getReturnType();
|
||||
long characters = 2L + returnType.length();
|
||||
for (CharSequence parameter : method.getParameterTypes()) {
|
||||
characters += parameter.length();
|
||||
if (characters * 6L > NATIVE_JSON_BYTES) break;
|
||||
}
|
||||
long bytes = 512L + 6L * (className.length() + methodName.length()
|
||||
+ path.length() + characters);
|
||||
if (inventory.retainedBytes + bytes > NATIVE_JSON_BYTES - 4096) {
|
||||
inventory.unknown("Native declarations exceed the 4 MiB retained JSON allowance; remaining declarations are unknown.");
|
||||
return inventory.result();
|
||||
}
|
||||
StringBuilder descriptor = new StringBuilder((int) characters).append('(');
|
||||
for (CharSequence parameter : method.getParameterTypes()) descriptor.append(parameter);
|
||||
descriptor.append(')').append(returnType);
|
||||
Map<String, Object> row = Map.of("source_index", inventory.methods.size(),
|
||||
"class_name", className, "method_name", methodName, "descriptor", descriptor.toString(),
|
||||
"dex_entry", path, "dex_entry_index", entryIndex, "class_index", classIndex,
|
||||
"method_index", method.getMethodIndex());
|
||||
if (!inventory.retain(row, bytes)) return inventory.result();
|
||||
}
|
||||
}
|
||||
inventory.examinedClasses++;
|
||||
}
|
||||
inventory.examinedDexEntries++;
|
||||
} catch (IOException | RuntimeException failure) {
|
||||
if (failure instanceof java.util.concurrent.CancellationException) throw (RuntimeException) failure;
|
||||
inventory.unknown("Root DEX native declarations could not be fully read from " + path + ": "
|
||||
+ failure.getClass().getSimpleName()
|
||||
+ (failure.getMessage() == null ? "" : " (" + failure.getMessage() + ")"));
|
||||
}
|
||||
}
|
||||
} catch (IOException failure) {
|
||||
// A failed archive close must fail the operation so the provider retires
|
||||
// its owned JVM; do not cache a result with lost file-resource ownership.
|
||||
throw new IllegalArgumentException("APK ZIP native declaration inventory could not finish its reader", failure);
|
||||
}
|
||||
return inventory.result();
|
||||
}
|
||||
|
||||
/** Original archive identity, never a deobfuscated basename or name suffix. */
|
||||
@@ -411,7 +179,10 @@ class ReaJadxBridge {
|
||||
selected = resource;
|
||||
}
|
||||
if (selected == null) throw new IllegalArgumentException("Root AndroidManifest.xml not found");
|
||||
String xml = manifestText(selected);
|
||||
var content = selected.loadContent();
|
||||
if (content == null || content.getDataType() != jadx.core.xmlgen.ResContainer.DataType.TEXT)
|
||||
throw new IllegalArgumentException("Root AndroidManifest.xml did not decode as text");
|
||||
String xml = content.getText().getCodeStr();
|
||||
ManifestSnapshot snapshot = new ManifestSnapshot(xml, manifestSummary(xml));
|
||||
manifestSnapshot = snapshot;
|
||||
manifestSession = current;
|
||||
@@ -419,23 +190,6 @@ class ReaJadxBridge {
|
||||
return manifestSnapshot;
|
||||
}
|
||||
|
||||
/** Decode with resource names but without JADX's class alias substitution. */
|
||||
static String manifestText(ResourceFile resource) {
|
||||
var decompiler = resource.getDecompiler();
|
||||
// BinaryXMLParser resolves android:name against its root's class identities
|
||||
// and replaces raw '$' and relative names with Java-style aliases. A fresh
|
||||
// root owns no classes, while its resource-name table preserves reference
|
||||
// rendering from the loaded target.
|
||||
RootNode root = new RootNode(decompiler.getArgs());
|
||||
root.getConstValues().setResourcesNames(decompiler.getRoot().getConstValues().getResourcesNames());
|
||||
try {
|
||||
return ResourcesLoader.decodeStream(resource,
|
||||
(size, input) -> new BinaryXMLParser(root).parse(input).getCodeStr());
|
||||
} catch (jadx.core.utils.exceptions.JadxException failure) {
|
||||
throw new IllegalArgumentException("Root AndroidManifest.xml did not decode as text", failure);
|
||||
}
|
||||
}
|
||||
|
||||
/** One parsed XML owner supplies namespace-aware values alongside the exact decoded text. */
|
||||
static Map<String, Object> manifestSummary(String xml) {
|
||||
Element root;
|
||||
@@ -479,7 +233,7 @@ class ReaJadxBridge {
|
||||
return Collections.unmodifiableMap(summary);
|
||||
}
|
||||
|
||||
private void indexClasses() {
|
||||
private ClassSnapshot selected(JsonObject args) {
|
||||
JadxSession current = session();
|
||||
if (indexedSession != current) {
|
||||
Map<String, List<ClassSnapshot>> index = new LinkedHashMap<>();
|
||||
@@ -505,60 +259,27 @@ class ReaJadxBridge {
|
||||
.map(Object::toString).collect(Collectors.joining(", "))
|
||||
+ "): " + method.getReturnType().toString();
|
||||
return new MethodSnapshot(name, rawName, shortId, List.copyOf(bodyAbsenceMarkers), Map.of(
|
||||
"name", name, "raw_name", rawName, "signature", signature,
|
||||
"name", name, "signature", signature,
|
||||
"dex_descriptor", descriptor,
|
||||
"is_constructor", method.isConstructor(), "def_pos", 0));
|
||||
}).toList();
|
||||
List<FieldNode> orderedFields = new ArrayList<>(node.getFields());
|
||||
orderedFields.sort(Comparator.comparing(FieldNode::getAlias));
|
||||
List<Map<String, Object>> fields = new ArrayList<>();
|
||||
List<Map<String, Object>> valuedFields = new ArrayList<>();
|
||||
boolean anyValue = false;
|
||||
for (FieldNode field : orderedFields) {
|
||||
// Raw DEX identity for mapping lookups; the display type can carry generics.
|
||||
Map<String, Object> entry = Map.of("name", field.getAlias(),
|
||||
"raw_name", field.getFieldInfo().getName(),
|
||||
"type", field.getType().toString(),
|
||||
"dex_type", TypeGen.signature(field.getFieldInfo().getType()), "def_pos", 0);
|
||||
fields.add(entry);
|
||||
Map<String, Object> value = dexStaticValue(field);
|
||||
if (value == null) valuedFields.add(entry);
|
||||
else {
|
||||
anyValue = true;
|
||||
Map<String, Object> valued = new LinkedHashMap<>(entry);
|
||||
valued.put("dex_static_value", value);
|
||||
valuedFields.add(Collections.unmodifiableMap(valued));
|
||||
}
|
||||
}
|
||||
List<Map<String, Object>> fields = node.getFields().stream()
|
||||
.sorted(Comparator.comparing(FieldNode::getAlias))
|
||||
.map(field -> Map.<String, Object>of("name", field.getAlias(),
|
||||
"type", field.getType().toString(), "def_pos", 0)).toList();
|
||||
List<String> inner = node.getInnerClasses().stream().map(ClassNode::getFullName).toList();
|
||||
List<String> innerRaw = node.getInnerClasses().stream().map(ClassNode::getRawName).toList();
|
||||
Map<String, Object> summary = Map.of(
|
||||
"full_name", cls.getFullName(), "raw_name", cls.getRawName(), "name", cls.getName(),
|
||||
"full_name", cls.getFullName(), "name", cls.getName(),
|
||||
"method_count", methods.size(), "field_count", fields.size(),
|
||||
"inner_class_count", inner.size(), "inner_classes", inner, "inner_class_raw_names", innerRaw,
|
||||
"methods", methods.stream().map(MethodSnapshot::summary).toList(), "fields", List.copyOf(fields));
|
||||
// DEX static values are opt-in, so method and reference lookups keep the member-only reply.
|
||||
Map<String, Object> summaryWithValues = summary;
|
||||
if (anyValue) {
|
||||
Map<String, Object> copy = new LinkedHashMap<>(summary);
|
||||
copy.put("fields", List.copyOf(valuedFields));
|
||||
summaryWithValues = Collections.unmodifiableMap(copy);
|
||||
}
|
||||
ClassSnapshot snapshot = new ClassSnapshot(cls, methods, summary, summaryWithValues);
|
||||
index.computeIfAbsent(cls.getFullName(), ignored -> new ArrayList<>()).add(snapshot);
|
||||
// Keep the binary identity supplied by JADX; '$' can also be a literal
|
||||
// name character, so replacing separators would invent lookup aliases.
|
||||
if (!cls.getRawName().equals(cls.getFullName()))
|
||||
index.computeIfAbsent(cls.getRawName(), ignored -> new ArrayList<>()).add(snapshot);
|
||||
"inner_class_count", inner.size(), "inner_classes", inner,
|
||||
"methods", methods.stream().map(MethodSnapshot::summary).toList(), "fields", fields);
|
||||
index.computeIfAbsent(cls.getFullName(), ignored -> new ArrayList<>())
|
||||
.add(new ClassSnapshot(cls, methods, summary));
|
||||
}
|
||||
}
|
||||
classes = Collections.unmodifiableMap(index);
|
||||
indexedSession = current;
|
||||
}
|
||||
}
|
||||
|
||||
private ClassSnapshot selected(JsonObject args) {
|
||||
indexClasses();
|
||||
String name = string(args, "class_name");
|
||||
List<ClassSnapshot> candidates = classes.get(name);
|
||||
if (candidates == null) throw new InputFailure("class_name", "Exact class not found: " + name
|
||||
@@ -568,28 +289,6 @@ class ReaJadxBridge {
|
||||
return candidates.get(0);
|
||||
}
|
||||
|
||||
/** A static final field's class-definition static value; a static initializer can still assign it. */
|
||||
private static Map<String, Object> dexStaticValue(FieldNode field) {
|
||||
if (!field.getAccessFlags().isStatic() || !field.getAccessFlags().isFinal()) return null;
|
||||
EncodedValue encoded = field.get(JadxAttrType.CONSTANT_VALUE);
|
||||
if (encoded == null) return null;
|
||||
Object value = encoded.getValue();
|
||||
// JADX also attaches the shared null to static finals without a value, so null is never reported.
|
||||
// Wide and floating values keep their Java text instead of a lossy JSON number.
|
||||
return switch (encoded.getType()) {
|
||||
case ENCODED_BOOLEAN -> Map.of("type", "boolean", "value", value);
|
||||
case ENCODED_BYTE -> Map.of("type", "byte", "value", value);
|
||||
case ENCODED_SHORT -> Map.of("type", "short", "value", value);
|
||||
case ENCODED_CHAR -> Map.of("type", "char", "value", (int) (Character) value);
|
||||
case ENCODED_INT -> Map.of("type", "int", "value", value);
|
||||
case ENCODED_LONG -> Map.of("type", "long", "value", value.toString());
|
||||
case ENCODED_FLOAT -> Map.of("type", "float", "value", value.toString());
|
||||
case ENCODED_DOUBLE -> Map.of("type", "double", "value", value.toString());
|
||||
case ENCODED_STRING -> Map.of("type", "string", "value", value);
|
||||
default -> null;
|
||||
};
|
||||
}
|
||||
|
||||
private List<MethodSnapshot> methods(ClassSnapshot cls, JsonObject args) {
|
||||
String name = string(args, "method_name");
|
||||
List<MethodSnapshot> result = cls.methods().stream().filter(method -> method.name().equals(name)).toList();
|
||||
@@ -658,18 +357,12 @@ class ReaJadxBridge {
|
||||
// JSON-RPC/CallToolResult envelope and generated numeric request id, and
|
||||
// account for JSON text being escaped again inside TextContent.
|
||||
private static final int MAX_XREF_TEXT_BYTES = 8 * 1024 * 1024 - 1024;
|
||||
// Leave room for the prototype reply, reference reply, retained raw calls,
|
||||
// normalized Evidence and their text/structured MCP copies, not just one frame.
|
||||
private static final int MAX_EXTERNAL_XREF_BYTES = MAX_XREF_TEXT_BYTES / 8;
|
||||
private static int escapedJsonBytes(Object value) {
|
||||
return JSON.toJson(JSON.toJson(value)).getBytes(StandardCharsets.UTF_8).length;
|
||||
}
|
||||
private static final class InvokeBudget {
|
||||
long remaining;
|
||||
InvokeBudget(long remaining) { this.remaining = Math.max(0, remaining); }
|
||||
boolean fitsText(long characters) {
|
||||
return remaining >= 256 && characters <= (remaining - 256) / 12;
|
||||
}
|
||||
boolean admit(Map<String, Object> site) {
|
||||
// Check a conservative two-layer escape bound before serialization.
|
||||
long bound = 256;
|
||||
@@ -684,55 +377,26 @@ class ReaJadxBridge {
|
||||
}
|
||||
private record DirectInvokes(List<Map<String, Object>> sites, boolean exhaustive) {}
|
||||
|
||||
@FunctionalInterface private interface InvokeVisitor {
|
||||
void accept(MethodInfo target, int offset, int index, Opcode opcode);
|
||||
}
|
||||
|
||||
/** Decode raw invoke identity once at the shared bridge boundary, without generating source. */
|
||||
private static void visitInvokes(MethodNode node, InvokeVisitor visitor) {
|
||||
checkNativeInterrupted();
|
||||
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
|
||||
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
|
||||
MethodNode node = source.getMethodNode();
|
||||
ICodeReader code = node.getCodeReader();
|
||||
if (code == null) return;
|
||||
if (code == null) return null;
|
||||
List<Map<String, Object>> sites = new ArrayList<>();
|
||||
boolean[] exhaustive = {true};
|
||||
code.visitInstructions(insn -> {
|
||||
checkNativeInterrupted();
|
||||
insn.decode();
|
||||
if (insn.getIndexType() != InsnIndexType.METHOD_REF) return;
|
||||
Opcode opcode = insn.getOpcode();
|
||||
if (!opcode.name().startsWith("INVOKE_")) return;
|
||||
IMethodRef ref = insn.getIndexAsMethod();
|
||||
ref.load();
|
||||
visitor.accept(MethodInfo.fromRef(node.root(), ref), insn.getOffset(), insn.getIndex(), opcode);
|
||||
});
|
||||
}
|
||||
|
||||
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
|
||||
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
|
||||
return directInvokes(source.getMethodNode(), target, budget);
|
||||
}
|
||||
|
||||
private static void visitDirectInvokes(MethodNode node, MethodInfo target, InvokeVisitor visitor) {
|
||||
visitInvokes(node, (referenced, offset, index, opcode) -> {
|
||||
if (referenced.getRawFullId().equals(target.getRawFullId()))
|
||||
visitor.accept(referenced, offset, index, opcode);
|
||||
});
|
||||
}
|
||||
|
||||
private static boolean referencesMethod(MethodNode node, MethodInfo target) {
|
||||
boolean[] found = {false};
|
||||
visitDirectInvokes(node, target, (referenced, offset, index, opcode) -> found[0] = true);
|
||||
return found[0];
|
||||
}
|
||||
|
||||
private static DirectInvokes directInvokes(MethodNode node, MethodInfo target, InvokeBudget budget) {
|
||||
if (node.getCodeReader() == null) return null;
|
||||
List<Map<String, Object>> sites = new ArrayList<>();
|
||||
boolean[] exhaustive = {true};
|
||||
visitDirectInvokes(node, target, (referenced, offset, index, opcode) -> {
|
||||
if (!MethodInfo.fromRef(node.root(), ref).getRawFullId().equals(target.getRawFullId())) return;
|
||||
Map<String, Object> site = Map.of(
|
||||
"dex_entry", node.getInputFileName(),
|
||||
"source_raw_method_id", node.getMethodInfo().getRawFullId(),
|
||||
"dex_code_unit_offset", offset,
|
||||
"dex_method_index", index,
|
||||
"dex_code_unit_offset", insn.getOffset(),
|
||||
"dex_method_index", insn.getIndex(),
|
||||
"opcode", opcode.name());
|
||||
if (budget.admit(site)) sites.add(site);
|
||||
else exhaustive[0] = false;
|
||||
@@ -740,143 +404,6 @@ class ReaJadxBridge {
|
||||
return new DirectInvokes(sites, exhaustive[0]);
|
||||
}
|
||||
|
||||
private List<ClassSnapshot> sourceClasses() {
|
||||
indexClasses();
|
||||
Set<ClassSnapshot> seen = Collections.newSetFromMap(new IdentityHashMap<>());
|
||||
List<ClassSnapshot> result = new ArrayList<>();
|
||||
for (List<ClassSnapshot> group : classes.values())
|
||||
for (ClassSnapshot source : group) if (seen.add(source)) result.add(source);
|
||||
result.sort(Comparator.comparing(source -> source.node().getRawName()));
|
||||
return result;
|
||||
}
|
||||
|
||||
private static long displayTypeCharacters(ArgType type) {
|
||||
if (type.isArray())
|
||||
return 2L * type.getArrayDimension() + displayTypeCharacters(type.getArrayRootElement());
|
||||
if (type.isPrimitive()) return type.getPrimitiveType().getLongName().length();
|
||||
if (type.isObject() && !type.isGeneric() && !type.isGenericType()) return type.getObject().length();
|
||||
throw new IllegalArgumentException("External prototype contains a non-DEX type");
|
||||
}
|
||||
|
||||
private static Map<String, Object> externalMethodSummary(MethodInfo info, InvokeBudget budget) {
|
||||
String name = info.getName();
|
||||
long signatureCharacters = name.length() + 4L + displayTypeCharacters(info.getReturnType())
|
||||
+ 2L * Math.max(0, info.getArgsCount() - 1);
|
||||
for (ArgType type : info.getArgumentsTypes()) signatureCharacters += displayTypeCharacters(type);
|
||||
long descriptorCharacters = info.getShortId().length() - (long) name.length();
|
||||
// Use parsed type facts and the existing short-id length before copying/joining strings.
|
||||
if (!budget.fitsText(2L * name.length() + descriptorCharacters + signatureCharacters))
|
||||
throw new ReferenceLimit("method_inventory");
|
||||
String descriptor = info.getShortId().substring(name.length());
|
||||
String signature = name + "(" + info.getArgumentsTypes().stream()
|
||||
.map(Object::toString).collect(Collectors.joining(", "))
|
||||
+ "): " + info.getReturnType();
|
||||
Map<String, Object> summary = Map.of("name", name, "raw_name", name, "signature", signature,
|
||||
"dex_descriptor", descriptor, "is_constructor", info.isConstructor(), "def_pos", 0);
|
||||
if (!budget.admit(summary)) throw new ReferenceLimit("method_inventory");
|
||||
return summary;
|
||||
}
|
||||
|
||||
/** Only directly referenced prototypes, not invented definitions or an external classpath. */
|
||||
private List<ExternalMethod> externalMethods(JsonObject args) {
|
||||
String className = string(args, "class_name");
|
||||
String methodName = string(args, "method_name");
|
||||
JadxSession current = session();
|
||||
if (externalInventory != null && externalInventory.session() == current
|
||||
&& externalInventory.className().equals(className) && externalInventory.methodName().equals(methodName))
|
||||
return externalInventory.methods();
|
||||
// Retain only the latest bounded selection, never results from a replaced APK/session.
|
||||
externalInventory = null;
|
||||
SortedMap<String, ExternalMethod> methods = new TreeMap<>();
|
||||
boolean[] observedClass = {false};
|
||||
InvokeBudget budget = new InvokeBudget(MAX_EXTERNAL_XREF_BYTES
|
||||
- 4096L - 12L * (className.length() + (long) methodName.length()));
|
||||
for (ClassSnapshot source : sourceClasses()) {
|
||||
for (MethodSnapshot selected : source.methods()) {
|
||||
visitInvokes(method(source, selected).getMethodNode(), (info, offset, index, opcode) -> {
|
||||
if (!info.getDeclClass().getRawName().equals(className)) return;
|
||||
observedClass[0] = true;
|
||||
if (!info.getName().equals(methodName) || methods.containsKey(info.getShortId())) return;
|
||||
methods.put(info.getShortId(), new ExternalMethod(info, externalMethodSummary(info, budget)));
|
||||
});
|
||||
}
|
||||
}
|
||||
if (!observedClass[0]) throw new InputFailure("class_name",
|
||||
"No APK definition or direct DEX invoke references the exact class: " + className);
|
||||
if (methods.isEmpty()) throw new InputFailure("method_name",
|
||||
"No direct DEX invoke references " + className + "." + methodName);
|
||||
List<ExternalMethod> result = List.copyOf(methods.values());
|
||||
externalInventory = new ExternalInventory(current, className, methodName, result);
|
||||
return result;
|
||||
}
|
||||
|
||||
private Map<String, Object> externalSummary(JsonObject args) {
|
||||
List<ExternalMethod> methods = externalMethods(args);
|
||||
return Map.of("class_name", string(args, "class_name"), "reference_target", "external",
|
||||
"observed_method_count", methods.size(),
|
||||
"prototypes", methods.stream().map(ExternalMethod::summary).toList());
|
||||
}
|
||||
|
||||
private record ExternalCaller(MethodNode node, Map<String, Object> row) {}
|
||||
|
||||
/** Reserve the complete using-node inventory before optional instruction positions. */
|
||||
private List<ExternalCaller> externalCallers(MethodInfo target, InvokeBudget budget) {
|
||||
List<ExternalCaller> callers = new ArrayList<>();
|
||||
for (ClassSnapshot source : sourceClasses()) {
|
||||
String className = String.valueOf(source.summary().get("full_name"));
|
||||
Map<String, Integer> overloads = new HashMap<>();
|
||||
for (MethodSnapshot candidate : source.methods()) {
|
||||
int overload = overloads.merge(candidate.name(), 1, Integer::sum) - 1;
|
||||
MethodNode node = method(source, candidate).getMethodNode();
|
||||
if (!referencesMethod(node, target)) continue;
|
||||
if (classes.get(className).size() != 1)
|
||||
throw new IllegalArgumentException("External caller has an ambiguous APK class identity: " + className);
|
||||
Map<String, Object> row = new LinkedHashMap<>(Map.of(
|
||||
"kind", "method", "name", candidate.name(), "full_name", className + "." + candidate.name(),
|
||||
"containing_class", className, "top_class", source.node().getTopParentClass().getFullName(),
|
||||
"def_pos", 0, "source_selector", Map.of("class_name", className,
|
||||
"method_name", candidate.name(), "overload_index", overload),
|
||||
"direct_invokes", List.of(), "direct_invokes_exhaustive", false));
|
||||
if (!budget.admit(row)) throw new ReferenceLimit("caller_inventory");
|
||||
callers.add(new ExternalCaller(node, row));
|
||||
}
|
||||
}
|
||||
return List.copyOf(callers);
|
||||
}
|
||||
|
||||
private Map<String, Object> externalReferences(JsonObject args) {
|
||||
List<ExternalMethod> methods = externalMethods(args);
|
||||
if (methods.size() != 1 && !args.has("overload_index"))
|
||||
throw new InputFailure("overload_index", "Select one of " + methods.size() + " observed external prototypes");
|
||||
int index = integer(args, "overload_index", 0);
|
||||
if (index >= methods.size()) throw new InputFailure("overload_index", "External prototype index out of range: " + index);
|
||||
ExternalMethod method = methods.get(index);
|
||||
MethodInfo target = method.info();
|
||||
Map<String, Object> selected = method.summary();
|
||||
InvokeBudget budget = new InvokeBudget(MAX_EXTERNAL_XREF_BYTES
|
||||
- 4096L - escapedJsonBytes(selected));
|
||||
List<ExternalCaller> callers = externalCallers(target, budget);
|
||||
List<Map<String, Object>> items = new ArrayList<>();
|
||||
for (ExternalCaller caller : callers) {
|
||||
DirectInvokes invokes = directInvokes(caller.node(), target, budget);
|
||||
caller.row().put("direct_invokes", invokes.sites());
|
||||
caller.row().put("direct_invokes_exhaustive", invokes.exhaustive());
|
||||
items.add(caller.row());
|
||||
}
|
||||
Map<String, Object> result = new LinkedHashMap<>();
|
||||
result.put("target", string(args, "class_name") + "." + target.getName());
|
||||
result.put("reference_target", "external");
|
||||
result.put("reference_scope", "direct_dex_invokes");
|
||||
result.put("dex_descriptor", selected.get("dex_descriptor"));
|
||||
result.put("reported_signature", selected.get("signature"));
|
||||
result.put("overload_index", index);
|
||||
result.put("total", items.size());
|
||||
result.put("count", items.size());
|
||||
result.put("resolve_line", false);
|
||||
result.put("items", items);
|
||||
return result;
|
||||
}
|
||||
|
||||
private CallToolResult handle(String name, kotlinx.serialization.json.JsonObject input) {
|
||||
JsonObject args = JsonParser.parseString(input.toString()).getAsJsonObject();
|
||||
if (name.equals("rea_jvm_status")) return json(Map.of(
|
||||
@@ -885,8 +412,6 @@ class ReaJadxBridge {
|
||||
"available_processors", Runtime.getRuntime().availableProcessors(),
|
||||
"java_version", System.getProperty("java.version"),
|
||||
"metadata_scope", "parsed_members"));
|
||||
if (name.equals("rea_native_declarations"))
|
||||
return json(nativeDeclarations(Path.of(string(args, "path"))));
|
||||
JadxSession current = session();
|
||||
if (name.equals("get_app_info")) return json(manifest(current).summary());
|
||||
if (name.equals("get_android_manifest"))
|
||||
@@ -913,15 +438,8 @@ class ReaJadxBridge {
|
||||
"scope", "top_level", "offset", offset, "limit", limit,
|
||||
"items", items));
|
||||
}
|
||||
indexClasses();
|
||||
if (flag(args, "reference_target") && !classes.containsKey(string(args, "class_name"))) {
|
||||
if (name.equals("get_class_summary")) return json(externalSummary(args));
|
||||
if (name.equals("get_xrefs_to_method")) return json(externalReferences(args));
|
||||
throw new InputFailure("class_name", "External prototypes are only available to method reference queries");
|
||||
}
|
||||
ClassSnapshot cls = selected(args);
|
||||
if (name.equals("get_class_summary"))
|
||||
return json(flag(args, "dex_static_values") ? cls.summaryWithValues() : cls.summary());
|
||||
if (name.equals("get_class_summary")) return json(cls.summary());
|
||||
if (name.equals("get_method_body")) {
|
||||
List<MethodSnapshot> overloads = methods(cls, args);
|
||||
int index = integer(args, "overload_index", 0);
|
||||
@@ -936,7 +454,7 @@ class ReaJadxBridge {
|
||||
"markers", code.getMarkers(), "body", code.getText()));
|
||||
}
|
||||
List<JavaNode> uses;
|
||||
String target = (String) cls.summary().get("full_name");
|
||||
String target = string(args, "class_name");
|
||||
MethodSnapshot selectedMethod = null;
|
||||
int overloadIndex = 0;
|
||||
if (name.equals("get_xrefs_to_class")) uses = cls.node().getUseIn();
|
||||
@@ -992,9 +510,6 @@ class ReaJadxBridge {
|
||||
try { return bridge.handle(name, request.getArguments()); }
|
||||
catch (InputFailure error) { return text(JSON.toJson(Map.of(
|
||||
"kind", "invalid-input", "field", error.field, "message", error.getMessage())), true); }
|
||||
catch (ReferenceLimit error) { return text(JSON.toJson(Map.of(
|
||||
"kind", "reference-output-limit", "stage", error.stage,
|
||||
"limit_bytes", MAX_EXTERNAL_XREF_BYTES)), true); }
|
||||
catch (IllegalArgumentException error) { return text(error.getMessage(), true); }
|
||||
});
|
||||
}
|
||||
@@ -1007,7 +522,7 @@ class ReaJadxBridge {
|
||||
SessionHolder holder = new SessionHolder(defaults);
|
||||
ServerCapabilities capabilities = new ServerCapabilities(
|
||||
new ServerCapabilities.Tools(null), null, null, null, null, null, null, Map.of());
|
||||
Server server = new Server(new Implementation("rea-jadx-bridge", "10", null, null, null),
|
||||
Server server = new Server(new Implementation("rea-jadx-bridge", "5", null, null, null),
|
||||
new ServerOptions(capabilities), "", ignored -> Unit.INSTANCE);
|
||||
SessionToolsKt.registerSessionTools(server, holder);
|
||||
ClassToolsKt.registerClassTools(server, holder);
|
||||
@@ -1027,11 +542,6 @@ class ReaJadxBridge {
|
||||
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of()), List.of(), null),
|
||||
"Observed worker JVM and metadata scope", null, null, null, null, null, null);
|
||||
server.addTool(diagnostics, (connection, request, continuation) -> bridge.handle("rea_jvm_status", request.getArguments()));
|
||||
registerBridgeTool(server, bridge, "rea_native_declarations",
|
||||
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of("path",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of("type",
|
||||
kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))))), List.of("path"), null),
|
||||
"Original ACC_NATIVE declarations from the admitted APK's standard root DEX members, without loading or decompiling the APK.");
|
||||
for (String name : List.of("get_app_info", "get_android_manifest")) {
|
||||
String description = name.equals("get_app_info")
|
||||
? "Namespace-aware summary of the exact root AndroidManifest.xml resource."
|
||||
@@ -1054,18 +564,9 @@ class ReaJadxBridge {
|
||||
if (name.equals("list_classes")) properties.put("query",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))));
|
||||
if (name.equals("get_class_summary")) properties.put("dex_static_values",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("boolean"))));
|
||||
if (name.equals("get_class_summary") || name.equals("get_xrefs_to_method"))
|
||||
properties.put("reference_target", new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("boolean"))));
|
||||
if (name.equals("get_class_summary")) properties.put("method_name",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))));
|
||||
String description = switch (name) {
|
||||
case "list_classes" -> "Pages of case-sensitive matching top-level class names; offset and next_offset are inventory positions, limit bounds matching rows. Omitted or empty query inventories all names.";
|
||||
case "get_class_summary" -> "Parsed member metadata for one exact class name; no source generation. Set dex_static_values to include encoded static final field values.";
|
||||
case "get_class_summary" -> "Parsed member metadata for one exact class name; no source generation.";
|
||||
case "get_method_body" -> "Source for an exact class and stable same-name overload index; static initializers use smali to preserve folded field initialization.";
|
||||
default -> "Complete static using-node inventory for an exact class or selected method overload; one node may contain multiple invoke instructions, and source lines are unresolved.";
|
||||
};
|
||||
|
||||
+192
-735
File diff suppressed because it is too large.
Load diff
@@ -1,193 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.database.mem.FileBytes;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.address.AddressSpace;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.mem.Memory;
|
||||
import ghidra.program.model.mem.MemoryBlock;
|
||||
import ghidra.program.model.mem.MemoryBlockSourceInfo;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.LinkOption;
|
||||
import java.nio.file.Path;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* Rebuild a GameCube/Wii DOL memory map from the validated load ranges REA's TypeScript
|
||||
* admission wrote. BinaryLoader maps the whole file once at address 0; this
|
||||
* script replaces that block with text, data, and BSS blocks at their load addresses.
|
||||
*/
|
||||
public final class ReaGhidraPrepareDol extends GhidraScript {
|
||||
// Canonical v2 header, 18 section rows, 12 gap rows and entry, using uint32 hex.
|
||||
private static final int MAX_PLAN_BYTES = 13 + 18 * 49 + 12 * 30 + 17;
|
||||
private static final class Section {
|
||||
final String kind;
|
||||
final int index;
|
||||
final long fileOffset;
|
||||
final long address;
|
||||
final long size;
|
||||
|
||||
Section(String kind, int index, long fileOffset, long address, long size) {
|
||||
this.kind = kind;
|
||||
this.index = index;
|
||||
this.fileOffset = fileOffset;
|
||||
this.address = address;
|
||||
this.size = size;
|
||||
}
|
||||
}
|
||||
|
||||
private record BssGap(long address, long size) {}
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] arguments = getScriptArgs();
|
||||
if (arguments.length != 1) {
|
||||
throw new IllegalArgumentException(
|
||||
"GameCube DOL preparation requires exactly one section table path"
|
||||
);
|
||||
}
|
||||
if (!currentProgram.getExecutableFormat().equals("Raw Binary") ||
|
||||
!currentProgram.getLanguageID().getIdAsString().startsWith("PowerPC:BE:32:")) {
|
||||
throw new IllegalArgumentException(
|
||||
"GameCube DOL preparation requires a BinaryLoader import in big-endian 32-bit PowerPC"
|
||||
);
|
||||
}
|
||||
Path tablePath = Path.of(arguments[0]);
|
||||
if (!Files.isRegularFile(tablePath, LinkOption.NOFOLLOW_LINKS)) {
|
||||
throw new IllegalArgumentException("GameCube DOL section table is not a regular file");
|
||||
}
|
||||
if (Files.size(tablePath) > MAX_PLAN_BYTES) {
|
||||
throw new IllegalArgumentException("GameCube DOL plan exceeds the canonical format's byte bound");
|
||||
}
|
||||
|
||||
List<Section> sections = new ArrayList<>();
|
||||
List<BssGap> bssGaps = new ArrayList<>();
|
||||
int expectedSections = -1;
|
||||
int expectedGaps = -1;
|
||||
long entry = -1;
|
||||
for (String line : Files.readAllLines(tablePath)) {
|
||||
String[] fields = line.trim().split("\\s+");
|
||||
switch (fields[0]) {
|
||||
case "plan" -> {
|
||||
requireFields(fields, 4, line);
|
||||
if (expectedSections >= 0 || !fields[1].equals("2")) {
|
||||
throw new IllegalArgumentException("Duplicate or unsupported GameCube DOL plan header");
|
||||
}
|
||||
expectedSections = Integer.parseInt(fields[2]);
|
||||
expectedGaps = Integer.parseInt(fields[3]);
|
||||
// A DOL has 18 slots, including at most 11 data slots and 12 BSS gaps.
|
||||
if (expectedSections < 1 || expectedSections > 18 || expectedGaps < 0 || expectedGaps > 12) {
|
||||
throw new IllegalArgumentException("GameCube DOL plan counts exceed the format's slot bounds");
|
||||
}
|
||||
}
|
||||
case "section" -> {
|
||||
requireFields(fields, 6, line);
|
||||
if (expectedSections < 0 || sections.size() >= expectedSections) {
|
||||
throw new IllegalArgumentException("GameCube DOL section exceeds its declared count");
|
||||
}
|
||||
sections.add(new Section(
|
||||
fields[1],
|
||||
Integer.parseInt(fields[2]),
|
||||
Long.decode(fields[3]),
|
||||
Long.decode(fields[4]),
|
||||
Long.decode(fields[5])
|
||||
));
|
||||
}
|
||||
case "bss-gap" -> {
|
||||
requireFields(fields, 3, line);
|
||||
if (expectedGaps < 0 || bssGaps.size() >= expectedGaps) {
|
||||
throw new IllegalArgumentException("GameCube DOL BSS gap exceeds its declared count");
|
||||
}
|
||||
long address = Long.decode(fields[1]);
|
||||
long size = Long.decode(fields[2]);
|
||||
if (address < 0 || size <= 0) {
|
||||
throw new IllegalArgumentException("GameCube DOL BSS gap is not a positive range");
|
||||
}
|
||||
bssGaps.add(new BssGap(address, size));
|
||||
}
|
||||
case "entry" -> {
|
||||
requireFields(fields, 2, line);
|
||||
entry = Long.decode(fields[1]);
|
||||
}
|
||||
default -> throw new IllegalArgumentException("Unknown GameCube DOL table line: " + line);
|
||||
}
|
||||
}
|
||||
if (expectedSections != sections.size() || expectedGaps != bssGaps.size() || entry < 0) {
|
||||
throw new IllegalArgumentException("GameCube DOL section table is incomplete");
|
||||
}
|
||||
|
||||
Memory memory = currentProgram.getMemory();
|
||||
List<FileBytes> files = memory.getAllFileBytes();
|
||||
if (files.size() != 1) {
|
||||
throw new IllegalStateException("GameCube DOL preparation requires exactly one imported file image");
|
||||
}
|
||||
FileBytes fileBytes = files.get(0);
|
||||
AddressSpace space = currentProgram.getAddressFactory().getDefaultAddressSpace();
|
||||
|
||||
// Remove the whole-file block before creating section blocks, so no two blocks claim one address.
|
||||
for (MemoryBlock block : memory.getBlocks()) {
|
||||
for (MemoryBlockSourceInfo source : block.getSourceInfos()) {
|
||||
if (source.getFileBytes().isPresent() && source.getFileBytes().get().equals(fileBytes)) {
|
||||
memory.removeBlock(block, monitor);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (Section section : sections) {
|
||||
if (section.fileOffset + section.size > fileBytes.getSize()) {
|
||||
throw new IllegalArgumentException(
|
||||
"GameCube DOL " + section.kind + " " + section.index + " exceeds the imported file"
|
||||
);
|
||||
}
|
||||
boolean text = section.kind.equals("text");
|
||||
if (!text && !section.kind.equals("data")) {
|
||||
throw new IllegalArgumentException("Unknown GameCube DOL section kind: " + section.kind);
|
||||
}
|
||||
MemoryBlock block = memory.createInitializedBlock(
|
||||
"." + section.kind + section.index,
|
||||
space.getAddress(section.address),
|
||||
fileBytes,
|
||||
section.fileOffset,
|
||||
section.size,
|
||||
false
|
||||
);
|
||||
block.setRead(true);
|
||||
block.setWrite(!text);
|
||||
block.setExecute(text);
|
||||
}
|
||||
|
||||
// Apply the domain owner's exact uninitialized ranges; do not reinterpret BSS here.
|
||||
int gapIndex = 0;
|
||||
for (BssGap gap : bssGaps) {
|
||||
MemoryBlock block = memory.createUninitializedBlock(
|
||||
".bss" + gapIndex++,
|
||||
space.getAddress(gap.address()),
|
||||
gap.size(),
|
||||
false
|
||||
);
|
||||
block.setRead(true);
|
||||
block.setWrite(true);
|
||||
block.setExecute(false);
|
||||
}
|
||||
|
||||
Address entryAddress = space.getAddress(entry);
|
||||
MemoryBlock entryBlock = memory.getBlock(entryAddress);
|
||||
if (entryBlock == null || !entryBlock.isExecute()) {
|
||||
throw new IllegalArgumentException("GameCube DOL entry must lie in an executable section");
|
||||
}
|
||||
addEntryPoint(entryAddress);
|
||||
if (!disassemble(entryAddress) || createFunction(entryAddress, "__start") == null) {
|
||||
throw new IllegalStateException("GameCube DOL entry cannot be decoded as a function");
|
||||
}
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setBoolean("REA raw binary prepared", true);
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setBoolean("REA GameCube DOL prepared", true);
|
||||
}
|
||||
|
||||
private static void requireFields(String[] fields, int count, String line) {
|
||||
if (fields.length != count) {
|
||||
throw new IllegalArgumentException("Malformed GameCube DOL table line: " + line);
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@@ -1,283 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.database.SpecExtension;
|
||||
import ghidra.program.model.data.DataType;
|
||||
import ghidra.program.model.data.DoubleDataType;
|
||||
import ghidra.program.model.data.Float16DataType;
|
||||
import ghidra.program.model.data.FloatDataType;
|
||||
import ghidra.program.model.data.FunctionDefinition;
|
||||
import ghidra.program.model.data.IntegerDataType;
|
||||
import ghidra.program.model.data.StructureDataType;
|
||||
import ghidra.program.model.lang.CompilerSpec;
|
||||
import ghidra.program.model.lang.Language;
|
||||
import ghidra.program.model.lang.PrototypeModel;
|
||||
import ghidra.program.model.lang.Register;
|
||||
import ghidra.program.model.listing.Function;
|
||||
import ghidra.program.model.listing.Parameter;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.listing.VariableStorage;
|
||||
import ghidra.program.model.pcode.Varnode;
|
||||
import ghidra.program.model.pcode.XmlEncode;
|
||||
import java.io.StringReader;
|
||||
import java.io.StringWriter;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.regex.Pattern;
|
||||
import javax.xml.parsers.DocumentBuilderFactory;
|
||||
import javax.xml.transform.OutputKeys;
|
||||
import javax.xml.transform.TransformerFactory;
|
||||
import javax.xml.transform.dom.DOMSource;
|
||||
import javax.xml.transform.stream.StreamResult;
|
||||
import org.w3c.dom.Document;
|
||||
import org.w3c.dom.Element;
|
||||
import org.w3c.dom.Node;
|
||||
import org.w3c.dom.NodeList;
|
||||
import org.xml.sax.InputSource;
|
||||
|
||||
/**
|
||||
* Apply the committed RISC-V calling-convention policy after analysis. Ghidra's generic RV32
|
||||
* compiler specification models ilp32d. Each policy derives a program-specific prototype from the
|
||||
* program's own default prototype, as its {@link Policy} row and docs/providers/ghidra-riscv.md state. The
|
||||
* script installs the prototype as a specification extension and assigns it, with dynamic
|
||||
* storage, to every non-thunk function, external function and function definition.
|
||||
*/
|
||||
public final class ReaGhidraRiscvCallingConvention extends GhidraScript {
|
||||
static final String REPORT_KEY = "REA RISC-V calling convention";
|
||||
/** RISC-V floating-point registers by ABI name: ft0-ft11, fs0-fs11 and fa0-fa7. */
|
||||
private static final Pattern FLOAT_REGISTER = Pattern.compile("f[ast]\\d+");
|
||||
|
||||
/**
|
||||
* What a policy changes in the default ilp32d prototype. Every policy also passes values of 9
|
||||
* bytes or more, such as {@code long double}, by reference.
|
||||
*
|
||||
* @param floatEntryBytes the largest value a floating-point register may pass or return; 0
|
||||
* removes those entries and every rule that assigns floating-point storage
|
||||
* @param preservesFloatRegisters whether fs0-fs11 stay preserved across calls
|
||||
*/
|
||||
private record Policy(
|
||||
String id, String prototype, int floatEntryBytes, boolean preservesFloatRegisters) {}
|
||||
|
||||
private static final List<Policy> POLICIES =
|
||||
List.of(new Policy("ilp32-prototype-v1", "__rea_ilp32", 0, false));
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] arguments = getScriptArgs();
|
||||
Policy policy =
|
||||
POLICIES.stream()
|
||||
.filter(candidate -> arguments.length == 1 && candidate.id().equals(arguments[0]))
|
||||
.findFirst()
|
||||
.orElseThrow(
|
||||
() ->
|
||||
new IllegalArgumentException(
|
||||
"REA RISC-V calling convention requires one of the policies "
|
||||
+ POLICIES.stream().map(Policy::id).toList()));
|
||||
Language language = currentProgram.getLanguage();
|
||||
if (!language.getProcessor().toString().equals("RISCV")
|
||||
|| currentProgram.getDefaultPointerSize() != 4)
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.id() + " requires a 32-bit RISC-V program, not "
|
||||
+ language.getLanguageID());
|
||||
CompilerSpec spec = currentProgram.getCompilerSpec();
|
||||
new SpecExtension(currentProgram)
|
||||
.addReplaceCompilerSpecExtension(
|
||||
derive(spec.getDefaultCallingConvention(), spec, language, policy), monitor);
|
||||
PrototypeModel model =
|
||||
currentProgram.getCompilerSpec().getCallingConvention(policy.prototype());
|
||||
if (model == null)
|
||||
throw new IllegalStateException("REA " + policy.prototype() + " was not installed");
|
||||
selfCheck(model, policy, language);
|
||||
int functions = 0;
|
||||
List<Function> callables = new ArrayList<>();
|
||||
currentProgram.getFunctionManager().getFunctions(true).forEach(callables::add);
|
||||
currentProgram.getFunctionManager().getExternalFunctions().forEach(callables::add);
|
||||
for (Function function : callables) {
|
||||
// A thunk's signature is its target's; the target is in this list.
|
||||
if (function.isThunk()) continue;
|
||||
if (function.hasCustomVariableStorage()) function.setCustomVariableStorage(false);
|
||||
function.setCallingConvention(policy.prototype());
|
||||
if (policy.floatEntryBytes() == 0) {
|
||||
List<VariableStorage> storage = new ArrayList<>();
|
||||
storage.add(function.getReturn().getVariableStorage());
|
||||
for (Parameter parameter : function.getParameters())
|
||||
storage.add(parameter.getVariableStorage());
|
||||
// An untyped function has no assigned storage yet; the decompiler applies the
|
||||
// prototype when it recovers one.
|
||||
requireIntegerStorage(
|
||||
storage.toArray(VariableStorage[]::new), function.getName(true), true);
|
||||
}
|
||||
functions += 1;
|
||||
}
|
||||
int definitions = 0;
|
||||
Iterator<DataType> types = currentProgram.getDataTypeManager().getAllDataTypes();
|
||||
while (types.hasNext()) {
|
||||
if (!(types.next() instanceof FunctionDefinition definition)) continue;
|
||||
definition.setCallingConvention(policy.prototype());
|
||||
definitions += 1;
|
||||
}
|
||||
// Written last: any failure above leaves no report, and REA refuses the session.
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setString(
|
||||
REPORT_KEY,
|
||||
"{\"policy\":\"" + policy.id() + "\",\"functions\":" + functions
|
||||
+ ",\"definitions\":" + definitions + "}");
|
||||
println(
|
||||
"REA RISC-V " + policy.prototype() + ": functions=" + functions
|
||||
+ " definitions=" + definitions);
|
||||
}
|
||||
|
||||
/** The default ilp32d prototype changed as the policy states. */
|
||||
private static String derive(
|
||||
PrototypeModel base, CompilerSpec spec, Language language, Policy policy) throws Exception {
|
||||
XmlEncode encoder = new XmlEncode();
|
||||
base.encode(encoder, spec.getPcodeInjectLibrary());
|
||||
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
|
||||
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
|
||||
Document document =
|
||||
factory.newDocumentBuilder().parse(new InputSource(new StringReader(encoder.toString())));
|
||||
Element root = document.getDocumentElement();
|
||||
if (!root.getTagName().equals("prototype"))
|
||||
throw new IllegalStateException("Unexpected default prototype encoding " + root.getTagName());
|
||||
root.setAttribute("name", policy.prototype());
|
||||
List<Element> floatEntries = new ArrayList<>();
|
||||
NodeList entries = root.getElementsByTagName("pentry");
|
||||
for (int index = 0; index < entries.getLength(); index += 1) {
|
||||
Element entry = (Element) entries.item(index);
|
||||
if (entry.getAttribute("storage").equals("float")) floatEntries.add(entry);
|
||||
}
|
||||
if (floatEntries.isEmpty())
|
||||
throw new IllegalStateException("Default prototype has no floating-point entries");
|
||||
if (policy.floatEntryBytes() == 0) {
|
||||
List<Node> removed = new ArrayList<>(floatEntries);
|
||||
NodeList rules = root.getElementsByTagName("rule");
|
||||
for (int index = 0; index < rules.getLength(); index += 1) {
|
||||
Element rule = (Element) rules.item(index);
|
||||
if (usesFloatStorage(rule)) removed.add(rule);
|
||||
}
|
||||
if (removed.size() == floatEntries.size())
|
||||
throw new IllegalStateException("Default prototype has no floating-point rules to remove");
|
||||
for (Node node : removed) node.getParentNode().removeChild(node);
|
||||
} else
|
||||
for (Element entry : floatEntries)
|
||||
entry.setAttribute("maxsize", Integer.toString(policy.floatEntryBytes()));
|
||||
// Values over 2 x XLEN go by reference. Returns need no rule: a value that the return
|
||||
// registers cannot hold already becomes a caller buffer passed in a0.
|
||||
NodeList inputs = root.getElementsByTagName("input");
|
||||
if (inputs.getLength() != 1)
|
||||
throw new IllegalStateException("Default prototype has no single input list");
|
||||
Element input = (Element) inputs.item(0);
|
||||
Element last = null;
|
||||
for (Node child = input.getFirstChild(); child != null; child = child.getNextSibling())
|
||||
if (child instanceof Element element && element.getTagName().equals("rule")) last = element;
|
||||
if (last == null || !joinsAnyValue(last))
|
||||
throw new IllegalStateException("Default prototype's input list does not end with a join");
|
||||
Element byReference = document.createElement("rule");
|
||||
Element datatype = document.createElement("datatype");
|
||||
datatype.setAttribute("name", "any");
|
||||
datatype.setAttribute("minsize", "9");
|
||||
byReference.appendChild(datatype);
|
||||
byReference.appendChild(document.createElement("convert_to_ptr"));
|
||||
input.insertBefore(byReference, last);
|
||||
if (!policy.preservesFloatRegisters()) {
|
||||
NodeList killedLists = root.getElementsByTagName("killedbycall");
|
||||
Element killed = killedLists.getLength() == 0 ? null : (Element) killedLists.item(0);
|
||||
if (killed == null) {
|
||||
killed = document.createElement("killedbycall");
|
||||
root.appendChild(killed);
|
||||
}
|
||||
List<Element> moved = new ArrayList<>();
|
||||
NodeList preservedLists = root.getElementsByTagName("unaffected");
|
||||
for (int list = 0; list < preservedLists.getLength(); list += 1) {
|
||||
NodeList preserved = preservedLists.item(list).getChildNodes();
|
||||
for (int index = 0; index < preserved.getLength(); index += 1) {
|
||||
if (preserved.item(index) instanceof Element varnode
|
||||
&& isFloatRegister(varnode, language))
|
||||
moved.add(varnode);
|
||||
}
|
||||
}
|
||||
for (Element varnode : moved) {
|
||||
varnode.getParentNode().removeChild(varnode);
|
||||
killed.appendChild(varnode);
|
||||
}
|
||||
}
|
||||
StringWriter xml = new StringWriter();
|
||||
var transformer = TransformerFactory.newInstance().newTransformer();
|
||||
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
|
||||
transformer.transform(new DOMSource(document), new StreamResult(xml));
|
||||
return xml.toString();
|
||||
}
|
||||
|
||||
/** The installed prototype places each class of value as its policy states. */
|
||||
private void selfCheck(PrototypeModel model, Policy policy, Language language) {
|
||||
StructureDataType pair = new StructureDataType("rea_float_pair", 0);
|
||||
pair.add(FloatDataType.dataType);
|
||||
pair.add(FloatDataType.dataType);
|
||||
VariableStorage[] storage =
|
||||
model.getStorageLocations(
|
||||
currentProgram,
|
||||
new DataType[] {
|
||||
DoubleDataType.dataType, FloatDataType.dataType, DoubleDataType.dataType,
|
||||
IntegerDataType.dataType, pair, Float16DataType.dataType
|
||||
},
|
||||
false);
|
||||
VariableStorage quad = storage[storage.length - 1];
|
||||
if (quad.size() != currentProgram.getDefaultPointerSize() || quad.getRegister() == null)
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.prototype() + " does not pass a 16-byte value by reference: " + quad);
|
||||
if (policy.floatEntryBytes() == 0)
|
||||
requireIntegerStorage(storage, policy.prototype() + " self-check", false);
|
||||
if (policy.preservesFloatRegisters()) return;
|
||||
for (Varnode preserved : model.getUnaffectedList()) {
|
||||
Register register = language.getRegister(preserved.getAddress(), preserved.getSize());
|
||||
if (register != null && FLOAT_REGISTER.matcher(register.getName()).matches())
|
||||
throw new IllegalStateException(
|
||||
"REA " + policy.prototype() + " still preserves " + register.getName());
|
||||
}
|
||||
}
|
||||
|
||||
private static boolean joinsAnyValue(Element rule) {
|
||||
List<Element> children = new ArrayList<>();
|
||||
for (Node child = rule.getFirstChild(); child != null; child = child.getNextSibling())
|
||||
if (child instanceof Element element) children.add(element);
|
||||
return children.size() == 2
|
||||
&& children.get(0).getTagName().equals("datatype")
|
||||
&& children.get(0).getAttribute("name").equals("any")
|
||||
&& children.get(1).getTagName().equals("join");
|
||||
}
|
||||
|
||||
/** Whether a rule's action, at any depth, assigns floating-point storage. */
|
||||
private static boolean usesFloatStorage(Element element) {
|
||||
NodeList children = element.getChildNodes();
|
||||
for (int index = 0; index < children.getLength(); index += 1) {
|
||||
if (!(children.item(index) instanceof Element child)) continue;
|
||||
if (child.getAttribute("storage").equals("float") || usesFloatStorage(child)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private static boolean isFloatRegister(Element varnode, Language language) {
|
||||
if (!varnode.getTagName().equals("varnode")
|
||||
|| !varnode.getAttribute("space").equals("register"))
|
||||
return false;
|
||||
Register register =
|
||||
language.getRegister(
|
||||
language.getAddressFactory().getRegisterSpace()
|
||||
.getAddress(Long.decode(varnode.getAttribute("offset"))),
|
||||
Integer.parseInt(varnode.getAttribute("size")));
|
||||
return register != null && FLOAT_REGISTER.matcher(register.getName()).matches();
|
||||
}
|
||||
|
||||
private static void requireIntegerStorage(
|
||||
VariableStorage[] storage, String owner, boolean allowUnassigned) {
|
||||
for (VariableStorage item : storage) {
|
||||
if (item.isBadStorage() || (!allowUnassigned && item.isUnassignedStorage()))
|
||||
throw new IllegalStateException("REA RISC-V storage unresolved for " + owner);
|
||||
List<Register> registers = item.getRegisters();
|
||||
if (registers == null) continue; // stack-only storage
|
||||
for (Register register : registers) {
|
||||
if (FLOAT_REGISTER.matcher(register.getName()).matches())
|
||||
throw new IllegalStateException(
|
||||
"REA RISC-V storage still uses " + register.getName() + " for " + owner);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+40
-204
@@ -22,9 +22,6 @@ HOPPER_SYMBOL_NAME_UTF16_UNITS = 1024
|
||||
# that does not fit is null, not an older descriptor path.
|
||||
# 536870887 is Node's 536870888-character string ceiling minus the framing LF.
|
||||
# The ceiling is the encoded payload; JSON expansion is measured before it.
|
||||
# Encoded JSON allowance, in bytes, shared by one instruction list, one
|
||||
# native-caller list, and one caller-scan page. This release has no runtime override.
|
||||
CALLER_SCAN_ALLOWANCE_BYTES = 512 * 1024
|
||||
_BRIDGE_MAX_REQUEST_BYTES_SETTING = "REA_BRIDGE_MAX_REQUEST_BYTES"
|
||||
DEFAULT_BRIDGE_MAX_REQUEST_BYTES = 16 * 1024 * 1024
|
||||
MINIMUM_BRIDGE_MAX_REQUEST_BYTES = 7464
|
||||
@@ -235,9 +232,6 @@ EXHAUSTIVE_ANALYSIS_METHODS = frozenset(
|
||||
"procedure_assembly",
|
||||
"procedure_callees",
|
||||
"procedure_callers",
|
||||
"rea_caller_scan_page",
|
||||
"rea_caller_scan_target",
|
||||
"rea_caller_base",
|
||||
"procedure_info",
|
||||
"procedure_pseudo_code",
|
||||
"procedure_references",
|
||||
@@ -330,27 +324,20 @@ def _name_matches(document, address, name):
|
||||
return observed == name or (not name and observed is None)
|
||||
|
||||
|
||||
def _bookmark_api(document, name):
|
||||
method = getattr(document, name, None)
|
||||
if not callable(method):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose %s" % name
|
||||
)
|
||||
return method
|
||||
|
||||
|
||||
def _bookmark_name(document, address):
|
||||
return _bookmark_api(document, "getBookmarkName")(address)
|
||||
getter = getattr(document, "getBookmarkName", None)
|
||||
if not callable(getter):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose getBookmarkName"
|
||||
)
|
||||
return getter(address)
|
||||
|
||||
|
||||
def _address_selection(document, value=None):
|
||||
"""Own the coordinate and whether it was selected by symbol or address."""
|
||||
def _address(document, value=None):
|
||||
"""Keep explicit hexadecimal coordinates distinct from symbol names."""
|
||||
if value is None:
|
||||
return document.getCurrentAddress(), "cursor"
|
||||
return document.getCurrentAddress()
|
||||
_api_text(value, "Address")
|
||||
if value.startswith("file-offset:"):
|
||||
raise InvalidRequestError("Hopper does not support file-offset: selectors; select Ghidra for source-file coordinates or use a Hopper analysis address")
|
||||
source = "address"
|
||||
if value.lower().startswith("0x"):
|
||||
try:
|
||||
address = int(value, 16)
|
||||
@@ -361,8 +348,6 @@ def _address_selection(document, value=None):
|
||||
# Prefer an observed symbol, retaining bare-hex compatibility only
|
||||
# when Hopper has no symbol with the supplied name.
|
||||
address = document.getAddressForName(value)
|
||||
if address not in BAD_ADDRESSES:
|
||||
source = "symbol"
|
||||
if address in BAD_ADDRESSES:
|
||||
try:
|
||||
address = int(value, 16)
|
||||
@@ -370,11 +355,7 @@ def _address_selection(document, value=None):
|
||||
raise InvalidRequestError("Unknown Hopper address or name: %s" % value) from error
|
||||
if not 0 <= address < 0xFFFFFFFFFFFFFFFF:
|
||||
raise InvalidRequestError("Address is outside Hopper's usable unsigned 64-bit range: %s" % value)
|
||||
return address, source
|
||||
|
||||
|
||||
def _address(document, value=None):
|
||||
return _address_selection(document, value)[0]
|
||||
return address
|
||||
|
||||
|
||||
def _segment(document, address):
|
||||
@@ -385,15 +366,10 @@ def _segment(document, address):
|
||||
|
||||
|
||||
def _procedure(document, value=None):
|
||||
address, source = _address_selection(document, value)
|
||||
address = document.getCurrentAddress() if value is None else _address(document, value)
|
||||
result = _segment(document, address).getProcedureAtAddress(address)
|
||||
if result is None:
|
||||
raise InvalidRequestError("No procedure exists at the requested address")
|
||||
if source == "symbol" and address != result.getEntryPoint():
|
||||
raise InvalidRequestError(
|
||||
"Symbol %s at %s is inside procedure %s at %s; select its entry name/address or use resolve_containing_procedure for explicit containment"
|
||||
% (value, _hex(address), _procedure_name(result), _hex(result.getEntryPoint()))
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
@@ -406,7 +382,6 @@ def _procedure_identity(procedure):
|
||||
return {
|
||||
"address": _hex(procedure.getEntryPoint()),
|
||||
"name": _procedure_name(procedure),
|
||||
"symbol": None,
|
||||
"classification": None,
|
||||
"body": {
|
||||
"available": False,
|
||||
@@ -491,30 +466,19 @@ def _instruction_addresses(procedure):
|
||||
return result
|
||||
|
||||
|
||||
def _iter_native_calls(procedure, direction):
|
||||
def _native_calls(procedure, direction):
|
||||
references = procedure.getAllCallees() if direction == "outgoing" else procedure.getAllCallers()
|
||||
kinds = {0: "none", 1: "unknown", 2: "direct", 3: "objective_c"}
|
||||
calls = {}
|
||||
unresolved = []
|
||||
for reference in references:
|
||||
source, target, kind = reference.fromAddress(), reference.toAddress(), reference.type()
|
||||
if source in BAD_ADDRESSES:
|
||||
yield None, None, None
|
||||
continue
|
||||
if target in BAD_ADDRESSES:
|
||||
yield source, None, {"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind}
|
||||
unresolved.append({"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind})
|
||||
continue
|
||||
yield source, target, {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
|
||||
|
||||
|
||||
def _native_calls(procedure, direction):
|
||||
calls = {}
|
||||
unresolved = []
|
||||
for source, target, detail in _iter_native_calls(procedure, direction):
|
||||
if detail is None:
|
||||
continue
|
||||
if target is None:
|
||||
unresolved.append(detail)
|
||||
else:
|
||||
calls[(source, target)] = detail
|
||||
calls[(source, target)] = {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
|
||||
return calls, sorted(unresolved, key=lambda item: int(item["address"], 16))
|
||||
|
||||
|
||||
@@ -564,136 +528,6 @@ def _procedure_map(document):
|
||||
return result
|
||||
|
||||
|
||||
def _caller_target_data(procedure):
|
||||
addresses = []
|
||||
seen = set()
|
||||
size = 0
|
||||
for block in procedure.basicBlockIterator():
|
||||
for address in _block_instruction_addresses(procedure, block):
|
||||
if address in seen:
|
||||
continue
|
||||
encoded = _hex(address)
|
||||
size += len(encoded) + 4
|
||||
if size > CALLER_SCAN_ALLOWANCE_BYTES:
|
||||
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": False}
|
||||
addresses.append(encoded)
|
||||
seen.add(address)
|
||||
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": True}
|
||||
|
||||
|
||||
def _caller_scan_target(document, params):
|
||||
return _caller_target_data(_procedure(document, params.get("procedure")))
|
||||
|
||||
|
||||
def _caller_base(document, params):
|
||||
procedure = _procedure(document, params.get("procedure"))
|
||||
callers = []
|
||||
size = 0
|
||||
complete = True
|
||||
for item in procedure.getAllCallerProcedures():
|
||||
address = _hex(item.getEntryPoint())
|
||||
size += len(address) + 4
|
||||
if size > CALLER_SCAN_ALLOWANCE_BYTES:
|
||||
complete = False
|
||||
break
|
||||
callers.append(address)
|
||||
return {
|
||||
**_caller_target_data(procedure),
|
||||
"callers": sorted(callers, key=lambda value: int(value, 16)),
|
||||
"callers_complete": complete,
|
||||
}
|
||||
|
||||
|
||||
def _caller_scan_page(document, params):
|
||||
"""Observe forward relationships in one bounded, stateless bridge request."""
|
||||
start = params.get("start")
|
||||
if isinstance(start, bool) or not isinstance(start, int) or start < 0:
|
||||
raise InvalidRequestError("Caller scan start must be a non-negative integer")
|
||||
segments = document.getSegmentsList()
|
||||
counts = [segment.getProcedureCount() for segment in segments]
|
||||
total = sum(counts)
|
||||
end = min(start + 64, total)
|
||||
if start > total:
|
||||
raise InvalidRequestError("Caller scan start exceeds the procedure inventory")
|
||||
items = []
|
||||
item_bytes = 0
|
||||
page_limit = CALLER_SCAN_ALLOWANCE_BYTES
|
||||
oversized = False
|
||||
identities = {}
|
||||
containing = {}
|
||||
|
||||
def identity(procedure):
|
||||
entry = procedure.getEntryPoint()
|
||||
if entry not in identities:
|
||||
identities[entry] = _procedure_identity(procedure)
|
||||
return identities[entry]
|
||||
|
||||
def owner(address):
|
||||
if address not in containing:
|
||||
procedure, _ = _containing_procedure(document, address)
|
||||
containing[address] = identity(procedure) if procedure is not None else None
|
||||
return containing[address]
|
||||
|
||||
segment_start = 0
|
||||
for segment, count in zip(segments, counts):
|
||||
begin, stop = max(start - segment_start, 0), min(end - segment_start, count)
|
||||
segment_start += count
|
||||
for index in range(begin, max(begin, stop)):
|
||||
candidate = segment.getProcedureAtIndex(index)
|
||||
item = {"caller": identity(candidate), "callees": [], "references": []}
|
||||
size = len(json.dumps(item, ensure_ascii=True).encode("utf-8")) + 1
|
||||
remaining = page_limit - item_bytes
|
||||
too_large = size > remaining
|
||||
if not too_large:
|
||||
for callee in candidate.getAllCalleeProcedures():
|
||||
address = _hex(callee.getEntryPoint())
|
||||
size += len(address) + 4
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
item["callees"].append(address)
|
||||
if not too_large:
|
||||
callee_entries = set(item["callees"])
|
||||
for source, destination, call in _iter_native_calls(candidate, "outgoing"):
|
||||
# Account for every native record, even when its target cannot
|
||||
# be associated with an observed procedure.
|
||||
size += 32
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
if destination is None:
|
||||
continue
|
||||
source_procedure = item["caller"] if _hex(source) == item["caller"]["address"] else owner(source)
|
||||
target_address = _hex(destination)
|
||||
if target_address in callee_entries:
|
||||
target_entry = target_address
|
||||
else:
|
||||
target_procedure = owner(destination)
|
||||
if target_procedure is None:
|
||||
continue
|
||||
target_entry = target_procedure["address"]
|
||||
edge = {
|
||||
"source_address": _hex(source), "target_address": _hex(destination),
|
||||
"source_procedure": "caller" if source_procedure is not None and source_procedure["address"] == item["caller"]["address"] else source_procedure,
|
||||
"target_procedure": target_entry,
|
||||
"call": call,
|
||||
}
|
||||
size += len(json.dumps(edge, ensure_ascii=True).encode("utf-8")) + 1
|
||||
if size > remaining:
|
||||
too_large = True
|
||||
break
|
||||
item["references"].append(edge)
|
||||
if too_large:
|
||||
oversized = not items
|
||||
break
|
||||
items.append(item)
|
||||
item_bytes += size
|
||||
else:
|
||||
continue
|
||||
break
|
||||
return {"start": start, "next": start + len(items), "total": total, "items": items, "oversized": oversized}
|
||||
|
||||
|
||||
def _string_record(segment, address, display):
|
||||
"""Read one native typed object, retaining its shortened display separately."""
|
||||
length = segment.getObjectLength(address)
|
||||
@@ -1034,15 +868,6 @@ def _dispatch(method, params):
|
||||
if method in EXHAUSTIVE_ANALYSIS_METHODS:
|
||||
_api().require_analysis_complete(document, method)
|
||||
|
||||
if method == "rea_caller_scan_target":
|
||||
return _caller_scan_target(document, params)
|
||||
|
||||
if method == "rea_caller_base":
|
||||
return _caller_base(document, params)
|
||||
|
||||
if method == "rea_caller_scan_page":
|
||||
return _caller_scan_page(document, params)
|
||||
|
||||
if method == "analyze_function":
|
||||
return _analyze_function(document, params)
|
||||
if method == "read_function_instructions":
|
||||
@@ -1227,7 +1052,6 @@ def _dispatch(method, params):
|
||||
length = sum(max(0, _basic_block_end(procedure, block) - block.getStartingAddress()) for block in blocks)
|
||||
return {
|
||||
"name": _procedure_name(procedure),
|
||||
"symbol": None,
|
||||
"entrypoint": _hex(procedure.getEntryPoint()),
|
||||
"basicblock_count": procedure.getBasicBlockCount(),
|
||||
"length": length,
|
||||
@@ -1268,24 +1092,39 @@ def _dispatch(method, params):
|
||||
observed = getter(address)
|
||||
return observed == comment or (comment == "" and observed is None)
|
||||
if method == "list_bookmarks":
|
||||
return [{"address": _hex(item), "name": _bookmark_name(document, item)} for item in _bookmark_api(document, "getBookmarks")()]
|
||||
return [{"address": _hex(item), "name": _bookmark_name(document, item)} for item in document.getBookmarks()]
|
||||
if method == "set_bookmark":
|
||||
address = _address(document, params.get("address"))
|
||||
_segment(document, address)
|
||||
name = params.get("name")
|
||||
if name is not None:
|
||||
_api_text(name, "Bookmark name")
|
||||
has_bookmark = _bookmark_api(document, "hasBookmarkAtAddress")
|
||||
has_bookmark = getattr(document, "hasBookmarkAtAddress", None)
|
||||
if not callable(has_bookmark):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose hasBookmarkAtAddress"
|
||||
)
|
||||
exists = has_bookmark(address)
|
||||
if name is None and exists:
|
||||
# Omission means preserve the existing bookmark and its name.
|
||||
return True
|
||||
_bookmark_api(document, "getBookmarkName")
|
||||
if not callable(getattr(document, "getBookmarkName", None)):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose getBookmarkName"
|
||||
)
|
||||
if name == "" and exists:
|
||||
rename = _bookmark_api(document, "renameBookmarkAtAddress")
|
||||
rename = getattr(document, "renameBookmarkAtAddress", None)
|
||||
if not callable(rename):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose renameBookmarkAtAddress"
|
||||
)
|
||||
rename(address, name)
|
||||
else:
|
||||
setter = _bookmark_api(document, "setBookmarkAtAddress")
|
||||
setter = getattr(document, "setBookmarkAtAddress", None)
|
||||
if not callable(setter):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose setBookmarkAtAddress"
|
||||
)
|
||||
setter(address, name)
|
||||
if not has_bookmark(address):
|
||||
return False
|
||||
@@ -1297,12 +1136,10 @@ def _dispatch(method, params):
|
||||
address = _address(document, params.get("address"))
|
||||
# Permit removing a legacy orphan bookmark, but do not report success
|
||||
# for an unmapped address with no bookmark to remove.
|
||||
has_bookmark = _bookmark_api(document, "hasBookmarkAtAddress")
|
||||
remove_bookmark = _bookmark_api(document, "removeBookmarkAtAddress")
|
||||
if not has_bookmark(address):
|
||||
if not document.hasBookmarkAtAddress(address):
|
||||
_segment(document, address)
|
||||
remove_bookmark(address)
|
||||
return not has_bookmark(address)
|
||||
document.removeBookmarkAtAddress(address)
|
||||
return not document.hasBookmarkAtAddress(address)
|
||||
raise InvalidRequestError("Unknown bridge method")
|
||||
|
||||
|
||||
@@ -1399,8 +1236,7 @@ def _serve_connection(connection):
|
||||
break
|
||||
except (ConnectionError, OSError):
|
||||
# A lost client cannot confirm document closure or revoke this run.
|
||||
# Keep the listener available for the owner to confirm cleanup.
|
||||
should_stop = False
|
||||
pass
|
||||
finally:
|
||||
try:
|
||||
file.close()
|
||||
|
||||
@@ -18,6 +18,7 @@ from mitmproxy import ctx, version
|
||||
from mitmproxy.io import tnetstring
|
||||
|
||||
PROFILE = "12.2.3"
|
||||
OUTPUT_BYTES = 96 * 1024 * 1024
|
||||
CREDENTIALS = {b"authorization", b"proxy-authorization", b"cookie", b"set-cookie"}
|
||||
|
||||
|
||||
@@ -203,7 +204,7 @@ def without_userinfo(value, base_scheme=None):
|
||||
raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
|
||||
|
||||
|
||||
def project_record(state, secrets, max_depth):
|
||||
def project_record(state, secrets):
|
||||
binaries, numbers, redactions = [], [], []
|
||||
byte_secrets = []
|
||||
for secret in secrets:
|
||||
@@ -250,8 +251,8 @@ def project_record(state, secrets, max_depth):
|
||||
del redactions[before[2]:]
|
||||
|
||||
def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
|
||||
if depth > max_depth:
|
||||
raise CaptureFailure("input-limit", f"Native capture exceeds max_nesting_depth ({max_depth} levels).", path)
|
||||
if depth > 64:
|
||||
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
|
||||
if credential:
|
||||
validate_omitted(value, path, depth)
|
||||
redactions.append({"pointer": path, "reason": "transport-credential"})
|
||||
@@ -336,12 +337,12 @@ def decode(request):
|
||||
except CaptureFailure:
|
||||
raise
|
||||
except RecursionError:
|
||||
raise CaptureFailure("unsupported", "Native parser reached Python's recursion limit at byte offset " + str(start) + ". max_nesting_depth does not raise that upstream limit.") from None
|
||||
raise CaptureFailure("input-limit", "Native parser exceeded its recursion budget at byte offset " + str(start) + ".") from None
|
||||
except (ValueError, IndexError, UnicodeError, EOFError):
|
||||
raise CaptureFailure("format", "Malformed native record at byte offset " + str(start) + ".") from None
|
||||
if not isinstance(state, dict):
|
||||
raise CaptureFailure("format", "Native flow record must be a dictionary at byte offset " + str(start) + ".")
|
||||
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"], request["budgets"]["depth"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
|
||||
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
|
||||
return {"decoder": {"id": "mitmproxy-native-tnetstring", "name": "REA offline native mitmproxy adapter", "version": PROFILE}, "container": {"reported": None, "numeric_literals": [], "redactions": [], "records_pointer": None}, "total_records": len(records), "records": records}
|
||||
|
||||
|
||||
@@ -349,8 +350,6 @@ def failure_reply(error):
|
||||
"""Keep producer constraints, resource exhaustion and adapter defects distinct."""
|
||||
if isinstance(error, CaptureFailure):
|
||||
return {"ok": False, "reason": error.reason, "message": str(error), "pointer": error.pointer}
|
||||
if isinstance(error, RecursionError):
|
||||
return {"ok": False, "reason": "unsupported", "message": "Native projection reached Python's recursion limit; max_nesting_depth does not raise that upstream limit.", "pointer": ""}
|
||||
if isinstance(error, MemoryError):
|
||||
return {"ok": False, "reason": "resource-limit", "message": "Native decoder exhausted its 768 MiB address-space budget.", "pointer": ""}
|
||||
return {"ok": False, "reason": "decoder", "message": "Offline native decoder failed unexpectedly: " + type(error).__name__ + ".", "pointer": ""}
|
||||
@@ -363,13 +362,9 @@ class OfflineCapture:
|
||||
def running(self):
|
||||
resource.setrlimit(resource.RLIMIT_AS, (768 * 1024 * 1024, 768 * 1024 * 1024))
|
||||
resource.setrlimit(resource.RLIMIT_CPU, (30, 30))
|
||||
resource.setrlimit(resource.RLIMIT_FSIZE, (OUTPUT_BYTES, OUTPUT_BYTES))
|
||||
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
|
||||
request = json.loads(Path(ctx.options.rea_request_path).read_text(encoding="utf-8"))
|
||||
budgets = request["budgets"]
|
||||
if any(type(budgets[key]) is not int or budgets[key] < 1 for key in ("inputBytes", "outputBytes", "depth")):
|
||||
raise ValueError("Invalid admitted capture budgets")
|
||||
output_bytes = budgets["outputBytes"]
|
||||
resource.setrlimit(resource.RLIMIT_FSIZE, (output_bytes, output_bytes))
|
||||
try:
|
||||
response = {"ok": True, "value": decode(request)}
|
||||
except Exception as error:
|
||||
@@ -379,7 +374,7 @@ class OfflineCapture:
|
||||
output = json.dumps(response, ensure_ascii=True, allow_nan=False, separators=(",", ":")).encode("ascii")
|
||||
except Exception as error:
|
||||
output = json.dumps(failure_reply(error), separators=(",", ":")).encode("ascii")
|
||||
if len(output) > output_bytes:
|
||||
if len(output) > OUTPUT_BYTES:
|
||||
output = b'{"ok":false,"reason":"limit","message":"Native reply exceeds the complete-evidence output budget.","pointer":""}'
|
||||
Path(request["reply_path"]).write_bytes(output)
|
||||
finally:
|
||||
|
||||
@@ -36,7 +36,7 @@ def location(value, scheme=None):
|
||||
state = {"response": {"headers": [[b"Location", value]]}}
|
||||
if scheme is not None:
|
||||
state["request"] = {"scheme": scheme}
|
||||
record = capture.project_record(state, [], 64)
|
||||
record = capture.project_record(state, [])
|
||||
sidecar = next(
|
||||
field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
|
||||
)
|
||||
@@ -75,11 +75,11 @@ class TransportUserinfoTests(unittest.TestCase):
|
||||
def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
|
||||
for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
|
||||
with self.subTest(value=value):
|
||||
record = capture.project_record({"request": {"path": value}}, [], 64)
|
||||
record = capture.project_record({"request": {"path": value}}, [])
|
||||
self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
|
||||
self.assertEqual(record["binary_fields"][0]["state"], "retained")
|
||||
self.assertEqual(record["redactions"], [])
|
||||
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [], 64)
|
||||
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
|
||||
self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
|
||||
self.assertEqual(record["binary_fields"][0]["state"], "redacted")
|
||||
|
||||
@@ -110,7 +110,7 @@ class TransportUserinfoTests(unittest.TestCase):
|
||||
"request": {"scheme": b"https"},
|
||||
"response": {"headers": [[b"Location", b"https:invoice@customer"]]},
|
||||
},
|
||||
}, [], 64)
|
||||
}, [])
|
||||
self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
|
||||
self.assertEqual(record["redactions"], [])
|
||||
|
||||
|
||||
@@ -1,39 +0,0 @@
|
||||
import Foundation
|
||||
import CoreFoundation
|
||||
|
||||
struct AccessibilityValue {
|
||||
let value: Any
|
||||
let gap: String?
|
||||
}
|
||||
|
||||
/** Preserve JSON scalars; report successful reads that JSON cannot represent faithfully. */
|
||||
func captureAccessibilityValue(_ source: Any?) -> AccessibilityValue {
|
||||
guard let source else { return AccessibilityValue(value: NSNull(), gap: nil) }
|
||||
if let text = source as? String { return AccessibilityValue(value: text, gap: nil) }
|
||||
if let number = source as? NSNumber {
|
||||
if CFGetTypeID(number) == CFBooleanGetTypeID() {
|
||||
return AccessibilityValue(value: number.boolValue, gap: nil)
|
||||
}
|
||||
let value = number.doubleValue
|
||||
let encoding = String(cString: number.objCType)
|
||||
let floating = encoding == "f" || encoding == "d"
|
||||
if number is NSDecimalNumber || !value.isFinite || (value == 0 && value.sign == .minus) || (!floating && abs(value) > 9_007_199_254_740_991) {
|
||||
return AccessibilityValue(value: NSNull(), gap: "AXValue number \(number.stringValue) cannot be represented faithfully as a JSON number")
|
||||
}
|
||||
return AccessibilityValue(value: number, gap: nil)
|
||||
}
|
||||
return AccessibilityValue(value: NSNull(), gap: "AXValue type \(type(of: source)) is not a supported JSON scalar")
|
||||
}
|
||||
|
||||
/** Keep an unavailable action list distinct from a successful empty list. */
|
||||
func captureActionNames(status: Int32, success: Int32, names: [String]?) -> (names: [String], error: Int32?) {
|
||||
guard status == success else { return ([], status) }
|
||||
return (names ?? [], nil)
|
||||
}
|
||||
|
||||
/** AX absence is distinct from a failed attribute read. */
|
||||
func captureAttributeValue<Value>(status: Int32, success: Int32, absent: [Int32], value: Value?) -> (value: Value?, error: Int32?) {
|
||||
if status == success { return (value, nil) }
|
||||
if absent.contains(status) { return (nil, nil) }
|
||||
return (nil, status)
|
||||
}
|
||||
@@ -18,10 +18,9 @@ struct Request: Decodable {
|
||||
struct Action: Decodable { var kind: String; var path: [Int]?; var direction: String?; var text: String? }
|
||||
struct BoundaryFailure: Error { var code: String; var message: String }
|
||||
func fail(_ code: String, _ message: String) throws -> Never { throw BoundaryFailure(code: code, message: message) }
|
||||
func attribute(_ element: AXUIElement, _ key: String) -> (value: CFTypeRef?, error: Int32?) {
|
||||
func attribute(_ element: AXUIElement, _ key: String) -> CFTypeRef? {
|
||||
var value: CFTypeRef?
|
||||
let status = AXUIElementCopyAttributeValue(element, key as CFString, &value)
|
||||
return captureAttributeValue(status: status.rawValue, success: AXError.success.rawValue, absent: [AXError.noValue.rawValue, AXError.attributeUnsupported.rawValue], value: value)
|
||||
return AXUIElementCopyAttributeValue(element, key as CFString, &value) == .success ? value : nil
|
||||
}
|
||||
func childCount(_ element: AXUIElement) -> ChildCount {
|
||||
var count = 0
|
||||
@@ -35,24 +34,19 @@ func children(_ element: AXUIElement, count: Int) -> ChildBatch<AXUIElement> {
|
||||
return value as? [AXUIElement]
|
||||
}
|
||||
}
|
||||
func windowBounds(_ element: AXUIElement) -> (bounds: CGRect?, error: Int32?) {
|
||||
let position = attribute(element, kAXPositionAttribute)
|
||||
let size = attribute(element, kAXSizeAttribute)
|
||||
if let error = position.error ?? size.error { return (nil, error) }
|
||||
guard let p = position.value, let s = size.value,
|
||||
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return (nil, nil) }
|
||||
var point = CGPoint.zero; var dimensions = CGSize.zero
|
||||
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &dimensions) else { return (nil, nil) }
|
||||
return (CGRect(origin: point, size: dimensions), nil)
|
||||
func text(_ element: AXUIElement, _ key: String) -> Any { (attribute(element, key) as? String) ?? NSNull() as Any }
|
||||
func windowBounds(_ element: AXUIElement) -> CGRect? {
|
||||
guard let p = attribute(element, kAXPositionAttribute), let s = attribute(element, kAXSizeAttribute),
|
||||
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return nil }
|
||||
var point = CGPoint.zero; var size = CGSize.zero
|
||||
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &size) else { return nil }
|
||||
return CGRect(origin: point, size: size)
|
||||
}
|
||||
func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard let app = NSRunningApplication(processIdentifier: request.pid) else {
|
||||
try fail("pid-not-application", "LaunchServices does not expose an application for the selected PID; this does not establish that the process is absent")
|
||||
guard let app = NSRunningApplication(processIdentifier: request.pid), let url = app.executableURL else {
|
||||
try fail("target-mismatch", "Selected PID does not expose an application executable URL")
|
||||
}
|
||||
guard let url = app.executableURL else {
|
||||
try fail("application-executable-unavailable", "The selected application does not expose an executable URL; select a PID with an identifiable application executable")
|
||||
}
|
||||
guard let canonicalPointer = realpath(url.path, nil) else { try fail("executable-path-unavailable", "Running executable path cannot be resolved") }
|
||||
guard let canonicalPointer = realpath(url.path, nil) else { try fail("target-mismatch", "Running executable path cannot be resolved") }
|
||||
defer { free(canonicalPointer) }
|
||||
let canonicalExecutable = String(cString: canonicalPointer)
|
||||
guard canonicalExecutable == request.executable else {
|
||||
@@ -68,10 +62,7 @@ func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard SHA256.hash(data: executableBytes).map({ String(format: "%02x", $0) }).joined() == request.sha256 else {
|
||||
try fail("target-mismatch", "Running executable bytes differ from the active target digest")
|
||||
}
|
||||
// .optionIncludingWindow omits a selected window while it is off screen, which
|
||||
// is not absence: select from every window, and let capture report what it can.
|
||||
guard let windows = (CGWindowListCopyWindowInfo([.optionAll], kCGNullWindowID) as? [[String: Any]])?
|
||||
.filter({ ($0[kCGWindowNumber as String] as? NSNumber)?.uint32Value == request.window_id }),
|
||||
guard let windows = CGWindowListCopyWindowInfo([.optionIncludingWindow], request.window_id) as? [[String: Any]],
|
||||
windows.count == 1, let window = windows.first,
|
||||
(window[kCGWindowOwnerPID as String] as? NSNumber)?.int32Value == request.pid,
|
||||
(window[kCGWindowLayer as String] as? NSNumber)?.intValue == 0,
|
||||
@@ -87,22 +78,11 @@ func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard AXIsProcessTrusted() else { try fail("accessibility-denied", "Grant Accessibility to the REA host in System Settings > Privacy & Security, then retry") }
|
||||
let application = AXUIElementCreateApplication(request.pid)
|
||||
AXUIElementSetMessagingTimeout(application, 2)
|
||||
let windowsAttribute = attribute(application, kAXWindowsAttribute)
|
||||
if let error = windowsAttribute.error {
|
||||
try fail("accessibility-unavailable", "Cannot read accessibility windows: AXError \(error)")
|
||||
guard let axWindows = attribute(application, kAXWindowsAttribute) as? [AXUIElement] else {
|
||||
try fail("accessibility-unavailable", "Selected application does not expose accessibility windows")
|
||||
}
|
||||
guard let axWindows = windowsAttribute.value as? [AXUIElement] else {
|
||||
try fail("accessibility-unavailable", "Accessibility windows are unavailable for the selected application")
|
||||
}
|
||||
let candidates = axWindows.map { (element: $0, observation: windowBounds($0)) }
|
||||
if let error = candidates.compactMap({ $0.observation.error }).first {
|
||||
try fail("accessibility-unavailable", "Cannot read accessibility window geometry: AXError \(error)")
|
||||
}
|
||||
guard candidates.allSatisfy({ $0.observation.bounds != nil }) else {
|
||||
try fail("accessibility-unavailable", "Accessibility window geometry is unavailable; the selected window cannot be identified")
|
||||
}
|
||||
let matches = candidates.filter { $0.observation.bounds == bounds }.map { $0.element }
|
||||
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(candidates.map { String(describing: $0.observation.bounds) })") }
|
||||
let matches = axWindows.filter { windowBounds($0) == bounds }
|
||||
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(axWindows.map { String(describing: windowBounds($0)) })") }
|
||||
selected = matches[0]
|
||||
}
|
||||
if let action = request.action, let root = selected {
|
||||
@@ -113,16 +93,13 @@ func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard let available = count.value else {
|
||||
try fail("element-count-unavailable", "Cannot validate selected accessibility path because child count failed with AXError \(count.error ?? -1)")
|
||||
}
|
||||
guard index >= 0 && index < available else { try fail("element-missing", "Selected accessibility path no longer exists") }
|
||||
let childStatus = AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild)
|
||||
guard childStatus == .success else { try fail("accessibility-unavailable", "Cannot read selected accessibility child: AXError \(childStatus.rawValue)") }
|
||||
guard let item = (selectedChild as? [AXUIElement])?.first else { try fail("accessibility-unavailable", "Selected accessibility child was not returned") }
|
||||
guard index >= 0 && index < available,
|
||||
AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild) == .success,
|
||||
let item = (selectedChild as? [AXUIElement])?.first else { try fail("element-missing", "Selected accessibility path no longer exists") }
|
||||
element = item
|
||||
}
|
||||
var owner: pid_t = 0
|
||||
let ownerStatus = AXUIElementGetPid(element, &owner)
|
||||
guard ownerStatus == .success else { try fail("accessibility-unavailable", "Cannot identify accessibility element owner: AXError \(ownerStatus.rawValue)") }
|
||||
guard owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
|
||||
guard AXUIElementGetPid(element, &owner) == .success && owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
|
||||
let outcome: AXError
|
||||
switch action.kind {
|
||||
case "click": outcome = AXUIElementPerformAction(element, kAXPressAction as CFString)
|
||||
@@ -139,18 +116,8 @@ func observe(_ request: Request) async throws -> [String: Any] {
|
||||
if nodes.count >= request.max_nodes { truncated = true; break }
|
||||
let childCountResult = childCount(element)
|
||||
var actions: CFArray?
|
||||
let actionStatus = AXUIElementCopyActionNames(element, &actions)
|
||||
let observedActions = captureActionNames(status: actionStatus.rawValue, success: AXError.success.rawValue, names: actions as? [String])
|
||||
if let error = observedActions.error { gaps.append("AX actions unavailable at path \(path): AXError \(error)"); truncated = true }
|
||||
let role = attribute(element, kAXRoleAttribute)
|
||||
let title = attribute(element, kAXTitleAttribute)
|
||||
let value = attribute(element, kAXValueAttribute)
|
||||
for (name, error) in [("role", role.error), ("title", title.error), ("value", value.error)] {
|
||||
if let error { gaps.append("AX \(name) unavailable at path \(path): AXError \(error)"); truncated = true }
|
||||
}
|
||||
let observedValue = captureAccessibilityValue(value.value)
|
||||
if let gap = observedValue.gap { gaps.append("\(gap) at path \(path)"); truncated = true }
|
||||
nodes.append(["path": path, "role": (role.value as? String) ?? NSNull() as Any, "title": (title.value as? String) ?? NSNull() as Any, "value": observedValue.value, "actions": observedActions.names, "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
|
||||
AXUIElementCopyActionNames(element, &actions)
|
||||
nodes.append(["path": path, "role": text(element, kAXRoleAttribute), "title": text(element, kAXTitleAttribute), "value": text(element, kAXValueAttribute), "actions": actions as? [String] ?? [], "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
|
||||
guard let totalChildren = childCountResult.value else {
|
||||
truncated = true
|
||||
gaps.append("AX child count unavailable at path \(path): AXError \(childCountResult.error ?? -1)")
|
||||
|
||||
@@ -25,17 +25,25 @@ ARGUMENT_REGISTERS = {
|
||||
"x86_64": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
|
||||
"x86_64h": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
|
||||
}
|
||||
MAX_SELECTOR_BYTES = 1024
|
||||
MAX_REPORTED_LOCATIONS = 64
|
||||
MAX_OTHER_STOPS = 256
|
||||
MAX_TRACE_BYTES = 8 * 1024 * 1024
|
||||
MAX_TRACE_FRAMES = 65_536
|
||||
MAX_OBSERVATION_JOURNAL_BYTES = MAX_TRACE_BYTES
|
||||
MAX_BREAKPOINT_LOCATION_BYTES = 8 * 1024 * 1024
|
||||
MAX_TARGET_OUTPUT_BYTES = 1024 * 1024
|
||||
OUTPUT_DRAIN_BYTES_PER_PASS = 1024 * 1024
|
||||
OUTPUT_EOF_WAIT_SECONDS = 1.0
|
||||
OUTPUT_READER_JOIN_SECONDS = 0.5
|
||||
LISTENER_NAME = "rea-native-call-observation"
|
||||
|
||||
|
||||
def _can_retain_trace(current_bytes, current_frames, event_bytes, event_frames, limits):
|
||||
def _can_retain_trace(current_bytes, current_frames, event_bytes, event_frames):
|
||||
"""Whether a complete event fits both aggregate trace budgets."""
|
||||
return (
|
||||
current_bytes + event_bytes <= limits["trace_bytes"]
|
||||
and current_frames + event_frames <= limits["trace_frames"]
|
||||
current_bytes + event_bytes <= MAX_TRACE_BYTES
|
||||
and current_frames + event_frames <= MAX_TRACE_FRAMES
|
||||
)
|
||||
|
||||
|
||||
@@ -64,7 +72,7 @@ def _fits_trace_budget(value, remaining_bytes):
|
||||
class _ObservationJournal:
|
||||
"""Flush a bounded JSONL prefix so observations survive tracer failure."""
|
||||
|
||||
def __init__(self, handle, max_bytes):
|
||||
def __init__(self, handle, max_bytes=MAX_OBSERVATION_JOURNAL_BYTES):
|
||||
self.handle = handle
|
||||
self.max_bytes = max_bytes
|
||||
self.bytes = 0
|
||||
@@ -236,20 +244,6 @@ def _hex(value):
|
||||
return "0x%x" % value
|
||||
|
||||
|
||||
def _source_location(address):
|
||||
"""Preserve available line-entry facets without acquiring source files."""
|
||||
entry = address.GetLineEntry()
|
||||
if not entry.IsValid():
|
||||
return None
|
||||
file_spec = entry.GetFileSpec()
|
||||
file_path = str(file_spec) if file_spec.IsValid() else None
|
||||
line = entry.GetLine() or None
|
||||
column = entry.GetColumn() or None
|
||||
if not file_path and line is None and column is None:
|
||||
return None
|
||||
return {"file_path": file_path or None, "line": line, "column": column}
|
||||
|
||||
|
||||
def _location(address, target):
|
||||
"""Load and file address, module and symbol of one code address."""
|
||||
module = address.GetModule()
|
||||
@@ -263,7 +257,6 @@ def _location(address, target):
|
||||
"module": module.GetFileSpec().GetFilename() if module.IsValid() else None,
|
||||
"module_path": str(module.GetFileSpec()) if module.IsValid() else None,
|
||||
"symbol": symbol.GetName() if symbol.IsValid() else None,
|
||||
"source_location": _source_location(address),
|
||||
}
|
||||
return location
|
||||
|
||||
@@ -364,7 +357,7 @@ def _record(context, thread, frame, index, sequence, location, remaining_bytes):
|
||||
error = lldb.SBError()
|
||||
text = process.ReadCStringFromMemory(
|
||||
values[names[1]],
|
||||
config["capture_limits"]["selector_bytes"],
|
||||
MAX_SELECTOR_BYTES,
|
||||
error,
|
||||
)
|
||||
selector = text if error.Success() and text else None
|
||||
@@ -397,16 +390,10 @@ def _record(context, thread, frame, index, sequence, location, remaining_bytes):
|
||||
_TRACE_CONTEXT = None
|
||||
|
||||
|
||||
def _resource_limit(context, limit):
|
||||
context["resource_limit"] = limit
|
||||
return "resource-limit"
|
||||
|
||||
|
||||
def _handle_breakpoint(context, frame, index):
|
||||
"""Read one hit while LLDB's breakpoint callback owns the stopped frame."""
|
||||
if not frame.IsValid() or frame.GetPC() == lldb.LLDB_INVALID_ADDRESS:
|
||||
raise RuntimeError("LLDB breakpoint callback did not supply a readable frame")
|
||||
limits = context["config"]["capture_limits"]
|
||||
thread = frame.GetThread()
|
||||
events = context["events"]
|
||||
location = _location(frame.GetPCAddress(), context["target"])
|
||||
@@ -423,50 +410,49 @@ def _handle_breakpoint(context, frame, index):
|
||||
context["retained_frames"],
|
||||
0,
|
||||
event_frame_count,
|
||||
limits,
|
||||
):
|
||||
return _resource_limit(context, "trace_frames")
|
||||
return "resource-limit"
|
||||
try:
|
||||
remaining_bytes = limits["trace_bytes"] - context["retained_trace_bytes"]
|
||||
remaining_bytes = MAX_TRACE_BYTES - context["retained_trace_bytes"]
|
||||
record = _record(
|
||||
context, thread, frame, index, len(events), location, remaining_bytes
|
||||
)
|
||||
except ValueError:
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
return "resource-limit"
|
||||
if record is not None:
|
||||
if not _fits_trace_budget(record, remaining_bytes):
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
return "resource-limit"
|
||||
record_bytes = len(json.dumps(record).encode("utf-8"))
|
||||
if context["retained_trace_bytes"] + record_bytes > limits["trace_bytes"]:
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
if context["retained_trace_bytes"] + record_bytes > MAX_TRACE_BYTES:
|
||||
return "resource-limit"
|
||||
for depth in range(1, frame_count + 1):
|
||||
try:
|
||||
location = _location(
|
||||
thread.GetFrameAtIndex(depth).GetPCAddress(), context["target"]
|
||||
)
|
||||
except ValueError:
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
return "resource-limit"
|
||||
if (
|
||||
_estimated_json_size(location) + record_bytes + 1
|
||||
> limits["trace_bytes"] - context["retained_trace_bytes"]
|
||||
> MAX_TRACE_BYTES - context["retained_trace_bytes"]
|
||||
):
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
return "resource-limit"
|
||||
location_bytes = len(json.dumps(location).encode("utf-8"))
|
||||
addition = location_bytes + (1 if record["backtrace"] else 0)
|
||||
if (
|
||||
context["retained_trace_bytes"]
|
||||
+ record_bytes
|
||||
+ addition
|
||||
> limits["trace_bytes"]
|
||||
> MAX_TRACE_BYTES
|
||||
):
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
return "resource-limit"
|
||||
record["backtrace"].append(location)
|
||||
record_bytes += addition
|
||||
journal = context.get("observation_journal")
|
||||
if journal is not None and not journal.append(
|
||||
{"kind": "event", "event": record}
|
||||
):
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
return "resource-limit"
|
||||
events.append(record)
|
||||
context["retained_frames"] += event_frame_count
|
||||
context["retained_trace_bytes"] += record_bytes
|
||||
@@ -498,9 +484,9 @@ def _handle_stop(context, other_stops):
|
||||
if journal is not None and not journal.append(
|
||||
{"kind": "other-stop", "reason": stop_description}
|
||||
):
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
return "resource-limit"
|
||||
other_stops.append(stop_description)
|
||||
if len(other_stops) >= context["config"]["capture_limits"]["other_stops"]:
|
||||
if len(other_stops) >= MAX_OTHER_STOPS:
|
||||
return "stop-limit"
|
||||
return None
|
||||
|
||||
@@ -519,19 +505,20 @@ def _breakpoint_report(target, created, config):
|
||||
if not _wanted(config["breakpoints"][index], location["symbol"] or ""):
|
||||
continue
|
||||
count += 1
|
||||
remaining_bytes = (
|
||||
config["capture_limits"]["breakpoint_location_bytes"] - retained_location_bytes
|
||||
)
|
||||
if not _fits_trace_budget(location, remaining_bytes):
|
||||
locations_truncated = True
|
||||
continue
|
||||
location_bytes = len(json.dumps(location).encode("utf-8"))
|
||||
addition = location_bytes + (1 if locations else 0)
|
||||
if addition > remaining_bytes:
|
||||
locations_truncated = True
|
||||
continue
|
||||
locations.append(location)
|
||||
retained_location_bytes += addition
|
||||
if len(locations) < MAX_REPORTED_LOCATIONS:
|
||||
remaining_bytes = (
|
||||
MAX_BREAKPOINT_LOCATION_BYTES - retained_location_bytes
|
||||
)
|
||||
if not _fits_trace_budget(location, remaining_bytes):
|
||||
locations_truncated = True
|
||||
continue
|
||||
location_bytes = len(json.dumps(location).encode("utf-8"))
|
||||
addition = location_bytes + (1 if locations else 0)
|
||||
if addition > remaining_bytes:
|
||||
locations_truncated = True
|
||||
continue
|
||||
locations.append(location)
|
||||
retained_location_bytes += addition
|
||||
report.append({"index": index, "location_count": count, "locations": locations})
|
||||
return report, locations_truncated
|
||||
|
||||
@@ -651,7 +638,7 @@ def trace(debugger, config):
|
||||
for breakpoint in breakpoints:
|
||||
breakpoint_index[breakpoint.GetID()] = index
|
||||
listener = lldb.SBListener(LISTENER_NAME)
|
||||
output_budget = config["capture_limits"]["output_bytes"]
|
||||
output_budget = config.get("max_output_bytes", MAX_TARGET_OUTPUT_BYTES)
|
||||
stdout = _OutputPipe(
|
||||
config["stdout_path"], config["stdout_capture_path"], output_budget
|
||||
)
|
||||
@@ -671,19 +658,21 @@ def trace(debugger, config):
|
||||
observation_path = config.get("observation_path")
|
||||
if observation_path:
|
||||
observation_journal = _ObservationJournal(
|
||||
open(observation_path, "wb"),
|
||||
config["capture_limits"]["trace_bytes"],
|
||||
open(observation_path, "wb")
|
||||
)
|
||||
process = target.Launch(_launch_info(target, config, listener), error)
|
||||
process_created = process.IsValid()
|
||||
if not error.Success() or not process_created:
|
||||
if not error.Success() or not process.IsValid():
|
||||
exit_observed = _retire_failed_launch(
|
||||
process, config, listener, stdout, stderr
|
||||
)
|
||||
return {
|
||||
"status": "launch-error",
|
||||
"error": error.GetCString(),
|
||||
"target_process_created": process_created,
|
||||
**(
|
||||
{"target_process_created": True}
|
||||
if process.IsValid()
|
||||
else {}
|
||||
),
|
||||
**(
|
||||
{"target_process_exit_observed": True}
|
||||
if exit_observed
|
||||
@@ -863,7 +852,6 @@ def trace(debugger, config):
|
||||
"breakpoint_locations_truncated": breakpoint_locations_truncated,
|
||||
"events": events,
|
||||
"resource_limit_reached": outcome == "resource-limit",
|
||||
"resource_limit": context.get("resource_limit"),
|
||||
"target_output": {
|
||||
"stdout_bytes": stdout.bytes,
|
||||
"stderr_bytes": stderr.bytes,
|
||||
|
||||
@@ -25,113 +25,6 @@ spec.loader.exec_module(tracer)
|
||||
|
||||
|
||||
class NativeTraceBudgetTests(unittest.TestCase):
|
||||
def test_source_coordinates_preserve_partial_metadata_without_reading_source(self):
|
||||
def address(valid=True, file_valid=True, line=23, column=7):
|
||||
class SourceFileSpec:
|
||||
def IsValid(self):
|
||||
return file_valid
|
||||
|
||||
def __str__(self):
|
||||
return "/missing/é space/fixture.pas"
|
||||
|
||||
file_spec = SourceFileSpec()
|
||||
entry = types.SimpleNamespace(
|
||||
IsValid=lambda: valid,
|
||||
GetFileSpec=lambda: file_spec,
|
||||
GetLine=lambda: line,
|
||||
GetColumn=lambda: column,
|
||||
)
|
||||
return types.SimpleNamespace(GetLineEntry=lambda: entry)
|
||||
|
||||
self.assertEqual(
|
||||
tracer._source_location(address()),
|
||||
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": 7},
|
||||
)
|
||||
self.assertEqual(
|
||||
tracer._source_location(address(column=0)),
|
||||
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": None},
|
||||
)
|
||||
self.assertEqual(
|
||||
tracer._source_location(address(file_valid=False)),
|
||||
{"file_path": None, "line": 23, "column": 7},
|
||||
)
|
||||
self.assertEqual(
|
||||
tracer._source_location(address(line=0)),
|
||||
{"file_path": "/missing/é space/fixture.pas", "line": None, "column": 7},
|
||||
)
|
||||
self.assertIsNone(tracer._source_location(address(valid=False)))
|
||||
self.assertIsNone(
|
||||
tracer._source_location(address(file_valid=False, line=0, column=0))
|
||||
)
|
||||
|
||||
def test_invalid_launch_process_reports_explicit_no_creation(self):
|
||||
class InvalidProcess:
|
||||
def IsValid(self):
|
||||
return False
|
||||
|
||||
def Kill(self):
|
||||
raise AssertionError("an invalid process cannot own target cleanup")
|
||||
|
||||
class LaunchError:
|
||||
def Success(self):
|
||||
return False
|
||||
|
||||
def GetCString(self):
|
||||
return "No such file or directory: fixture/absent"
|
||||
|
||||
class Target:
|
||||
def IsValid(self):
|
||||
return True
|
||||
|
||||
def Launch(self, _launch_info, _error):
|
||||
return InvalidProcess()
|
||||
|
||||
class Debugger:
|
||||
def SetAsync(self, _enabled):
|
||||
pass
|
||||
|
||||
def GetInstanceName(self):
|
||||
return "fixture"
|
||||
|
||||
def CreateTarget(self, *_arguments):
|
||||
return Target()
|
||||
|
||||
class Pipe:
|
||||
def __init__(self, *_arguments):
|
||||
self.closed = False
|
||||
pipes.append(self)
|
||||
|
||||
def start(self):
|
||||
pass
|
||||
|
||||
def close(self, expected_exit=False):
|
||||
self.closed = True
|
||||
|
||||
pipes = []
|
||||
debugger_api = types.SimpleNamespace(SetInternalVariable=lambda *_arguments: None)
|
||||
with patch.object(tracer.lldb, "SBDebugger", debugger_api, create=True), patch.object(
|
||||
tracer.lldb, "SBError", LaunchError, create=True
|
||||
), patch.object(tracer.lldb, "SBListener", lambda _name: object(), create=True), patch.object(
|
||||
tracer, "_hash_file", return_value="expected"
|
||||
), patch.object(tracer, "_file_identity", return_value=(1, 2)), patch.object(
|
||||
tracer, "_module_identity", return_value=("fixture", (1, 2), "fixture-uuid")
|
||||
), patch.object(tracer, "_launch_info", return_value=object()), patch.object(
|
||||
tracer, "_OutputPipe", Pipe
|
||||
):
|
||||
result = tracer.trace(Debugger(), {
|
||||
"executable": "fixture", "architecture": "arm64", "expected_sha256": "expected",
|
||||
"breakpoints": [], "working_directory": "fixture/absent",
|
||||
"capture_limits": {"output_bytes": 1024},
|
||||
"stdout_path": "stdout", "stdout_capture_path": "stdout-capture",
|
||||
"stderr_path": "stderr", "stderr_capture_path": "stderr-capture",
|
||||
})
|
||||
self.assertEqual(result["status"], "launch-error")
|
||||
self.assertEqual(result["error"], "No such file or directory: fixture/absent")
|
||||
self.assertIs(result["target_process_created"], False)
|
||||
self.assertNotIn("target_process_exit_observed", result)
|
||||
self.assertEqual(len(pipes), 2)
|
||||
self.assertTrue(all(pipe.closed for pipe in pipes))
|
||||
|
||||
def test_failed_launch_does_not_infer_exit_from_a_failed_kill(self):
|
||||
class Process:
|
||||
def IsValid(self):
|
||||
@@ -321,44 +214,37 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
return super().flush()
|
||||
|
||||
stream = CountingStream()
|
||||
journal = tracer._ObservationJournal(stream, max_bytes=4096)
|
||||
journal = tracer._ObservationJournal(stream)
|
||||
journal.append({"kind": "event", "event": {"sequence": 0}})
|
||||
journal.append({"kind": "other-stop", "reason": "signal"})
|
||||
self.assertEqual((stream.writes, stream.flushes), (2, 2))
|
||||
self.assertEqual(journal.bytes, len(stream.getvalue()))
|
||||
|
||||
def test_aggregate_trace_budget_uses_the_selected_admission(self):
|
||||
limits = {"trace_bytes": 100, "trace_frames": 10}
|
||||
self.assertTrue(tracer._can_retain_trace(90, 8, 10, 2, limits))
|
||||
self.assertFalse(tracer._can_retain_trace(90, 8, 11, 2, limits))
|
||||
self.assertFalse(tracer._can_retain_trace(90, 8, 10, 3, limits))
|
||||
raised = {"trace_bytes": 101, "trace_frames": 11}
|
||||
self.assertTrue(tracer._can_retain_trace(90, 8, 11, 3, raised))
|
||||
|
||||
def test_breakpoint_reports_exact_exhausted_budget_and_accepts_more_room(self):
|
||||
frame = types.SimpleNamespace(
|
||||
IsValid=lambda: True, GetPC=lambda: 1,
|
||||
GetThread=lambda: types.SimpleNamespace(GetNumFrames=lambda: 1),
|
||||
GetPCAddress=lambda: object(),
|
||||
def test_aggregate_trace_budget_admits_exact_boundary_and_rejects_next_record(self):
|
||||
self.assertTrue(
|
||||
tracer._can_retain_trace(
|
||||
tracer.MAX_TRACE_BYTES - 10,
|
||||
tracer.MAX_TRACE_FRAMES - 2,
|
||||
10,
|
||||
2,
|
||||
)
|
||||
)
|
||||
self.assertFalse(
|
||||
tracer._can_retain_trace(
|
||||
tracer.MAX_TRACE_BYTES - 10,
|
||||
tracer.MAX_TRACE_FRAMES - 2,
|
||||
11,
|
||||
2,
|
||||
)
|
||||
)
|
||||
self.assertFalse(
|
||||
tracer._can_retain_trace(
|
||||
tracer.MAX_TRACE_BYTES - 10,
|
||||
tracer.MAX_TRACE_FRAMES - 2,
|
||||
10,
|
||||
3,
|
||||
)
|
||||
)
|
||||
context = {
|
||||
"target": object(), "events": [], "retained_frames": 1,
|
||||
"retained_trace_bytes": 0,
|
||||
"config": {"breakpoints": [{}], "backtrace_frames": 0, "max_events": 10,
|
||||
"capture_limits": {"trace_bytes": 4096, "trace_frames": 1}},
|
||||
}
|
||||
with patch.object(tracer, "_location", return_value={"symbol": "main"}), patch.object(
|
||||
tracer, "_wanted", return_value=True
|
||||
), patch.object(tracer, "_record", return_value={"backtrace": []}):
|
||||
self.assertEqual(tracer._handle_breakpoint(context, frame, 0), "resource-limit")
|
||||
self.assertEqual(context["resource_limit"], "trace_frames")
|
||||
context["config"]["capture_limits"] = {"trace_bytes": 8, "trace_frames": 2}
|
||||
self.assertEqual(tracer._handle_breakpoint(context, frame, 0), "resource-limit")
|
||||
self.assertEqual(context["resource_limit"], "trace_bytes")
|
||||
self.assertEqual(context["events"], [])
|
||||
context["config"]["capture_limits"]["trace_bytes"] = 4096
|
||||
self.assertIsNone(tracer._handle_breakpoint(context, frame, 0))
|
||||
self.assertEqual(context["events"], [{"backtrace": []}])
|
||||
|
||||
def test_oversized_producer_string_is_rejected_before_serialization(self):
|
||||
self.assertTrue(tracer._fits_trace_budget("method", 1024))
|
||||
@@ -437,9 +323,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
return FileSpec()
|
||||
|
||||
class Address:
|
||||
def GetLineEntry(self):
|
||||
return types.SimpleNamespace(IsValid=lambda: False)
|
||||
|
||||
def GetModule(self):
|
||||
return Module()
|
||||
|
||||
@@ -467,7 +350,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
object(),
|
||||
[[Breakpoint()]],
|
||||
{
|
||||
"capture_limits": {"breakpoint_location_bytes": 4096},
|
||||
"breakpoints": [
|
||||
{
|
||||
"kind": "objc-method",
|
||||
@@ -481,34 +363,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
self.assertEqual(report[0]["locations"], [])
|
||||
self.assertFalse(locations_truncated)
|
||||
|
||||
def test_breakpoint_report_keeps_all_locations_under_aggregate_budget(self):
|
||||
class Location:
|
||||
def __init__(self, index):
|
||||
self.index = index
|
||||
|
||||
def GetAddress(self):
|
||||
return self.index
|
||||
|
||||
class Breakpoint:
|
||||
def GetNumLocations(self):
|
||||
return 65
|
||||
|
||||
def GetLocationAtIndex(self, index):
|
||||
return Location(index)
|
||||
|
||||
with patch.object(
|
||||
tracer,
|
||||
"_location",
|
||||
side_effect=lambda address, _target: {"address": address, "symbol": None},
|
||||
), patch.object(tracer, "_wanted", return_value=True):
|
||||
report, locations_truncated = tracer._breakpoint_report(
|
||||
object(), [[Breakpoint()]], {"breakpoints": [{}], "capture_limits": {"breakpoint_location_bytes": 4096}}
|
||||
)
|
||||
|
||||
self.assertEqual(report[0]["location_count"], 65)
|
||||
self.assertEqual(len(report[0]["locations"]), 65)
|
||||
self.assertFalse(locations_truncated)
|
||||
|
||||
def test_breakpoint_report_bounds_aggregate_location_json(self):
|
||||
tracer.lldb.LLDB_INVALID_ADDRESS = -1
|
||||
symbol_name = (
|
||||
@@ -542,24 +396,7 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
def GetFileSpec(self):
|
||||
return FileSpec()
|
||||
|
||||
class SourceFileSpec:
|
||||
def IsValid(self):
|
||||
return True
|
||||
|
||||
def __str__(self):
|
||||
return source_path
|
||||
|
||||
class Address:
|
||||
def GetLineEntry(self):
|
||||
if source_path is None:
|
||||
return types.SimpleNamespace(IsValid=lambda: False)
|
||||
return types.SimpleNamespace(
|
||||
IsValid=lambda: True,
|
||||
GetFileSpec=SourceFileSpec,
|
||||
GetLine=lambda: 15,
|
||||
GetColumn=lambda: 3,
|
||||
)
|
||||
|
||||
def GetModule(self):
|
||||
return Module()
|
||||
|
||||
@@ -583,39 +420,22 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
def GetLocationAtIndex(self, index):
|
||||
return Location()
|
||||
|
||||
breakpoint_count = 30
|
||||
breakpoint_count = 1000
|
||||
created = [[Breakpoint()] for _ in range(breakpoint_count)]
|
||||
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count,
|
||||
"capture_limits": {"breakpoint_location_bytes": 512 * 1024}}
|
||||
for source_path in (None, "/tmp/" + "源🚀" * 3000 + "/calls.pas"):
|
||||
with self.subTest(source_path_present=source_path is not None):
|
||||
report, locations_truncated = tracer._breakpoint_report(
|
||||
object(), created, config
|
||||
)
|
||||
retained_bytes = sum(
|
||||
len(json.dumps(location).encode("utf-8"))
|
||||
+ (1 if index > 0 else 0)
|
||||
for row in report
|
||||
for index, location in enumerate(row["locations"])
|
||||
)
|
||||
self.assertTrue(locations_truncated)
|
||||
self.assertEqual(
|
||||
sum(row["location_count"] for row in report), 1920
|
||||
)
|
||||
self.assertLess(
|
||||
sum(len(row["locations"]) for row in report), 1920
|
||||
)
|
||||
self.assertLessEqual(
|
||||
retained_bytes, 512 * 1024
|
||||
)
|
||||
if source_path is not None:
|
||||
self.assertTrue(
|
||||
any(
|
||||
location["source_location"]["file_path"] == source_path
|
||||
for row in report
|
||||
for location in row["locations"]
|
||||
)
|
||||
)
|
||||
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count}
|
||||
report, locations_truncated = tracer._breakpoint_report(
|
||||
object(), created, config
|
||||
)
|
||||
retained_bytes = sum(
|
||||
len(json.dumps(location).encode("utf-8"))
|
||||
+ (1 if index > 0 else 0)
|
||||
for row in report
|
||||
for index, location in enumerate(row["locations"])
|
||||
)
|
||||
self.assertTrue(locations_truncated)
|
||||
self.assertEqual(sum(row["location_count"] for row in report), 64_000)
|
||||
self.assertLess(sum(len(row["locations"]) for row in report), 64_000)
|
||||
self.assertLessEqual(retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -78,9 +78,14 @@ public final class NativeAotExtension {
|
||||
new FrozenObjectAnnotator(program, manager).analyze(located.directory(), scan.getPointerLocations(), monitor, log);
|
||||
}
|
||||
monitor.checkCancelled();
|
||||
report.addProperty("method_tables", manager.getMethodTableCount());
|
||||
JsonObject coverage = auditFrozenObjects(program, scan.getPointerLocations(), monitor);
|
||||
report.add("coverage", coverage);
|
||||
if (coverage.get("frozen_objects_annotated").getAsInt() != coverage.get("frozen_object_candidates").getAsInt())
|
||||
diagnostics.add("Some frozen-object candidates lack committed instance annotations; inspect coverage counts and Ghidra diagnostics.");
|
||||
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
|
||||
report.add("reason", JsonNull.INSTANCE);
|
||||
report.add("derived_memory", derivedRange.deepCopy());
|
||||
JsonArray tables = new JsonArray();
|
||||
for (var mt : manager.getMethodTables()) {
|
||||
monitor.checkCancelled();
|
||||
@@ -89,13 +94,6 @@ public final class NativeAotExtension {
|
||||
identity.addProperty("type", mt.getMTType().getPathName());
|
||||
tables.add(identity);
|
||||
}
|
||||
// Publish recovery facts only after every fallible recovery and audit step.
|
||||
// A failed transaction is rolled back, so it cannot report committed metadata.
|
||||
report.addProperty("method_tables", manager.getMethodTableCount());
|
||||
report.add("coverage", coverage);
|
||||
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
|
||||
report.add("reason", JsonNull.INSTANCE);
|
||||
report.add("derived_memory", derivedRange.deepCopy());
|
||||
report.add("types", tables);
|
||||
commit = true;
|
||||
return finish(log);
|
||||
|
||||
@@ -15,8 +15,6 @@ struct ProcessIdentityObservation: Encodable {
|
||||
let effective_uid: UInt32?
|
||||
let lifecycle: String?
|
||||
let relation: ProcessRelationObservation?
|
||||
/** Display-only kernel accounting name (p_comm, at most MAXCOMLEN bytes). */
|
||||
let name: String?
|
||||
let reason: String?
|
||||
}
|
||||
|
||||
@@ -33,7 +31,6 @@ struct ProcessMetadata {
|
||||
let parentPid: Int32
|
||||
let processGroupId: Int32
|
||||
let pointerSize: Int
|
||||
let name: String
|
||||
}
|
||||
|
||||
struct ProcessSnapshot {
|
||||
@@ -95,8 +92,7 @@ func readProcessSnapshot(
|
||||
lifecycle: process.kp_proc.p_stat == SZOMB ? "zombie" : "non-zombie",
|
||||
parentPid: process.kp_eproc.e_ppid,
|
||||
processGroupId: process.kp_eproc.e_pgid,
|
||||
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4,
|
||||
name: processName(process)
|
||||
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4
|
||||
)
|
||||
)
|
||||
},
|
||||
@@ -107,13 +103,6 @@ func readProcessSnapshot(
|
||||
return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(ENOMEM)")
|
||||
}
|
||||
|
||||
/** Decode the NUL-terminated accounting name; invalid UTF-8 is replaced for display. */
|
||||
private func processName(_ process: kinfo_proc) -> String {
|
||||
withUnsafeBytes(of: process.kp_proc.p_comm) { bytes in
|
||||
String(decoding: bytes.prefix { $0 != 0 }, as: UTF8.self)
|
||||
}
|
||||
}
|
||||
|
||||
func readProcessIdentities(pids: [Int32]) -> [ProcessIdentityObservation] {
|
||||
readProcessIdentities(pids: pids, snapshot: readProcessSnapshot())
|
||||
}
|
||||
@@ -141,7 +130,6 @@ func readProcessIdentities(
|
||||
effective_uid: nil,
|
||||
lifecycle: nil,
|
||||
relation: nil,
|
||||
name: nil,
|
||||
reason: "not-in-snapshot"
|
||||
)
|
||||
}
|
||||
@@ -156,7 +144,6 @@ func readProcessIdentities(
|
||||
parent_pid: metadata.parentPid,
|
||||
process_group_id: metadata.processGroupId
|
||||
),
|
||||
name: metadata.name,
|
||||
reason: nil
|
||||
)
|
||||
}
|
||||
@@ -170,25 +157,12 @@ private func unavailableIdentity(pid: Int32, reason: String) -> ProcessIdentityO
|
||||
effective_uid: nil,
|
||||
lifecycle: nil,
|
||||
relation: nil,
|
||||
name: nil,
|
||||
reason: reason
|
||||
)
|
||||
}
|
||||
|
||||
enum ProcargsMalformedStage: String, CaseIterable {
|
||||
case header
|
||||
case argumentCount = "argument_count"
|
||||
case executablePath = "executable_path"
|
||||
case pointerSize = "pointer_size"
|
||||
case pathPadding = "path_padding"
|
||||
case arguments
|
||||
case terminator
|
||||
case trailingEntries = "trailing_entries"
|
||||
}
|
||||
|
||||
enum RunTokenReadError: Error {
|
||||
case invalidBuffer
|
||||
case invalidBufferAt(ProcargsMalformedStage)
|
||||
case duplicateToken
|
||||
case invalidTokenEncoding
|
||||
case environmentUnavailable
|
||||
@@ -228,22 +202,6 @@ func processRunTokenObservation(
|
||||
parse: Result<ParsedRunToken, RunTokenReadError>,
|
||||
codeSigningFlags: () -> UInt32?
|
||||
) -> RunTokenObservation {
|
||||
// Native inspection restrictions apply even when argv is incomplete or
|
||||
// overwritten, before parsing reaches the environment. Keep this fact at
|
||||
// the native boundary; cleanup still decides whether the process is related.
|
||||
func withheldEnvironment() -> RunTokenObservation? {
|
||||
guard let flags = codeSigningFlags() else { return nil }
|
||||
if (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func unavailableEnvironment(_ reason: String) -> RunTokenObservation {
|
||||
withheldEnvironment() ?? unavailable(pid: pid, reason: reason)
|
||||
}
|
||||
switch parse {
|
||||
case .success(let parsed):
|
||||
if let runId = parsed.runId {
|
||||
@@ -254,25 +212,38 @@ func processRunTokenObservation(
|
||||
if parsed.environmentEntryCount > 0 {
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
}
|
||||
if let withheld = withheldEnvironment() { return withheld }
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
case .failure(let failure):
|
||||
switch failure {
|
||||
case .environmentUnavailable:
|
||||
return unavailableEnvironment("environment_unavailable")
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "environment_unavailable")
|
||||
case .ambiguousEnvironmentBoundary:
|
||||
return unavailableEnvironment("ambiguous_environment_boundary")
|
||||
// These failures already establish that token bytes were available.
|
||||
if let flags = codeSigningFlags(), (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "ambiguous_environment_boundary")
|
||||
case .duplicateToken:
|
||||
return unavailable(pid: pid, reason: "duplicate_run_token")
|
||||
case .invalidTokenEncoding:
|
||||
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
|
||||
case .appleVectorUnavailable:
|
||||
return unavailableEnvironment("apple_vector_unavailable")
|
||||
return unavailable(pid: pid, reason: "apple_vector_unavailable")
|
||||
case .invalidBuffer:
|
||||
return unavailableEnvironment("malformed_procargs")
|
||||
case .invalidBufferAt(let stage):
|
||||
return unavailableEnvironment("malformed_procargs_\(stage.rawValue)")
|
||||
return unavailable(pid: pid, reason: "malformed_procargs")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -331,30 +302,27 @@ func parseProcessEnvironment(
|
||||
in bytes: [UInt8],
|
||||
pointerSize: Int = MemoryLayout<UnsafeRawPointer>.size
|
||||
) throws -> ParsedRunToken {
|
||||
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBufferAt(.header) }
|
||||
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBuffer }
|
||||
let argc = bytes.withUnsafeBytes { $0.loadUnaligned(as: Int32.self).littleEndian }
|
||||
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBufferAt(.argumentCount) }
|
||||
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBuffer }
|
||||
var offset = MemoryLayout<Int32>.size
|
||||
guard let executablePath = readString(in: bytes, offset: &offset) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.executablePath)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBufferAt(.pointerSize) }
|
||||
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBuffer }
|
||||
let pathRecordSize = Array("executable_path=".utf8).count + executablePath.count + 1
|
||||
let padding = (pointerSize - pathRecordSize % pointerSize) % pointerSize
|
||||
guard offset + padding <= bytes.count,
|
||||
bytes[offset..<(offset + padding)].allSatisfy({ $0 == 0 }) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.pathPadding)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
offset += padding
|
||||
let tokenPrefix = Array("REA_PROCESS_RUN_ID=".utf8)
|
||||
var tokenInArgument = false
|
||||
for _ in 0..<argc {
|
||||
guard let argument = readString(in: bytes, offset: &offset, allowEmpty: true) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.arguments)
|
||||
guard readString(in: bytes, offset: &offset, allowEmpty: true) != nil else {
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
tokenInArgument = tokenInArgument || argument.starts(with: tokenPrefix)
|
||||
}
|
||||
guard bytes.last == 0 else { throw RunTokenReadError.invalidBufferAt(.terminator) }
|
||||
guard bytes.last == 0 else { throw RunTokenReadError.invalidBuffer }
|
||||
if offset == bytes.count {
|
||||
throw RunTokenReadError.environmentUnavailable
|
||||
}
|
||||
@@ -372,7 +340,7 @@ func parseProcessEnvironment(
|
||||
continue
|
||||
}
|
||||
guard let entry = readString(in: bytes, offset: &offset) else {
|
||||
throw RunTokenReadError.invalidBufferAt(.trailingEntries)
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
trailingEntries.append(entry)
|
||||
}
|
||||
@@ -393,28 +361,22 @@ func parseProcessEnvironment(
|
||||
throw RunTokenReadError.appleVectorUnavailable
|
||||
}
|
||||
if trailingEntries[appleVectorStart...].contains(where: {
|
||||
$0.starts(with: tokenPrefix)
|
||||
$0.starts(with: Array("REA_PROCESS_RUN_ID=".utf8))
|
||||
}) {
|
||||
throw RunTokenReadError.ambiguousEnvironmentBoundary
|
||||
}
|
||||
let environmentEntries = trailingEntries[..<appleVectorStart]
|
||||
var found: String?
|
||||
for entry in environmentEntries {
|
||||
if entry.starts(with: tokenPrefix) {
|
||||
if entry.starts(with: Array("REA_PROCESS_RUN_ID=".utf8)) {
|
||||
guard found == nil else { throw RunTokenReadError.duplicateToken }
|
||||
let value = entry.dropFirst(tokenPrefix.count)
|
||||
let value = entry.dropFirst("REA_PROCESS_RUN_ID=".utf8.count)
|
||||
guard let decoded = String(bytes: value, encoding: .utf8) else {
|
||||
throw RunTokenReadError.invalidTokenEncoding
|
||||
}
|
||||
found = decoded
|
||||
}
|
||||
}
|
||||
// A changed process title can overwrite the original argument boundaries,
|
||||
// causing a real environment entry to be consumed as an argument. A real
|
||||
// argument can have this form too, so it proves neither ownership nor absence.
|
||||
if tokenInArgument {
|
||||
throw RunTokenReadError.ambiguousEnvironmentBoundary
|
||||
}
|
||||
return ParsedRunToken(runId: found, environmentEntryCount: environmentEntries.count)
|
||||
}
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ from pathlib import Path
|
||||
import resource
|
||||
import gdb
|
||||
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
|
||||
|
||||
class ContextFailure(Exception):
|
||||
@@ -78,7 +79,6 @@ def failure_exit(request_path, status, marker):
|
||||
def main():
|
||||
request_path = os.environ["REA_CORE_CONTEXT_REQUEST"]
|
||||
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
|
||||
output_bytes = request["producer_limits"]["output_bytes"]
|
||||
reserve = bytearray(1024 * 1024)
|
||||
try:
|
||||
reply = {"ok": True, "value": inspect(request)}
|
||||
@@ -94,8 +94,8 @@ def main():
|
||||
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
|
||||
try:
|
||||
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
|
||||
if len(encoded) > output_bytes:
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds producer_limits.output_bytes=" + str(output_bytes) + "."}).encode("utf-8")
|
||||
if len(encoded) > OUTPUT_BYTES:
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds the 64 MiB reply budget."}).encode("utf-8")
|
||||
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(descriptor, "wb") as handle:
|
||||
handle.write(encoded)
|
||||
|
||||
@@ -14,7 +14,7 @@ BRIDGE = Path(__file__).with_name("core_context.py")
|
||||
|
||||
|
||||
class CoreContextTests(unittest.TestCase):
|
||||
def inspect(self, failure=None, connection="core", symbols=None, architecture="x86_64-little-endian", objfile="recorded-executable", output_bytes=16384):
|
||||
def inspect(self, failure=None, connection="core", symbols=None, architecture="x86_64-little-endian", objfile="recorded-executable"):
|
||||
commands, reads, queries = [], [], []
|
||||
state = {"executable": None, "symbols": None, "connection": None}
|
||||
gdb = types.ModuleType("gdb")
|
||||
@@ -63,7 +63,7 @@ class CoreContextTests(unittest.TestCase):
|
||||
plugin.write_text("")
|
||||
request = stage / "request.json"
|
||||
reply = stage / "reply.json"
|
||||
request.write_text(json.dumps({"gdbinit": str(plugin), "architecture": architecture, "snapshot_path": str(root / "object.snapshot"), "reply_path": str(reply), "producer_limits": {"output_bytes": output_bytes}}))
|
||||
request.write_text(json.dumps({"gdbinit": str(plugin), "architecture": architecture, "snapshot_path": str(root / "object.snapshot"), "reply_path": str(reply)}))
|
||||
with patch.dict(sys.modules, modules), patch.dict(os.environ, REA_CORE_CONTEXT_REQUEST=str(request)):
|
||||
runpy.run_path(str(BRIDGE))
|
||||
result = json.loads(reply.read_text())
|
||||
@@ -104,15 +104,14 @@ class CoreContextTests(unittest.TestCase):
|
||||
self.assertEqual(reads, [])
|
||||
self.assertEqual(queries, [])
|
||||
|
||||
def test_selected_output_limit_reports_recovery_and_accepts_more_headroom(self):
|
||||
result, _, reads, queries = self.inspect(objfile="m" * 2048, output_bytes=1024)
|
||||
def test_output_limit_message_separates_the_words_and_names_the_limit(self):
|
||||
result, _, reads, queries = self.inspect(objfile="m" * (64 * 1024 * 1024))
|
||||
self.assertFalse(result["ok"])
|
||||
self.assertEqual(result["reason"], "output-limit")
|
||||
self.assertIn("producer_limits.output_bytes=1024", result["message"])
|
||||
self.assertNotIn("exceeds64", result["message"])
|
||||
self.assertEqual(result["message"], "Complete debugger context exceeds the 64 MiB reply budget.")
|
||||
self.assertEqual(reads, [])
|
||||
self.assertEqual(queries, ["maps"])
|
||||
recovered, _, _, _ = self.inspect(objfile="m" * 2048, output_bytes=4096)
|
||||
self.assertTrue(recovered["ok"], recovered)
|
||||
|
||||
def test_wrong_connection_or_remaining_symbols_never_queries_mappings(self):
|
||||
for options in [{"connection": "native"}, {"symbols": "unexpected-symbols"}]:
|
||||
|
||||
@@ -5,12 +5,14 @@ from pathlib import Path
|
||||
import resource
|
||||
import sys
|
||||
|
||||
def lower_resource_limits(policy):
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
|
||||
def lower_resource_limits():
|
||||
limits = {}
|
||||
for name, kind, maximum in (
|
||||
("address_space_bytes", resource.RLIMIT_AS, policy["address_space_bytes"]),
|
||||
("cpu_seconds", resource.RLIMIT_CPU, policy["cpu_seconds"]),
|
||||
("file_size_bytes", resource.RLIMIT_FSIZE, policy["file_bytes"]),
|
||||
("address_space_bytes", resource.RLIMIT_AS, 3 * 1024**3),
|
||||
("cpu_seconds", resource.RLIMIT_CPU, 30),
|
||||
("file_size_bytes", resource.RLIMIT_FSIZE, OUTPUT_BYTES),
|
||||
):
|
||||
soft, hard = resource.getrlimit(kind)
|
||||
finite = [maximum] + [value for value in (soft, hard) if value != resource.RLIM_INFINITY]
|
||||
@@ -23,11 +25,6 @@ def lower_resource_limits(policy):
|
||||
|
||||
def main(request_path, inspect, failure_type, profile):
|
||||
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
|
||||
policy = request["producer_limits"]
|
||||
for key in ("input_bytes", "output_bytes", "diagnostic_bytes", "address_space_bytes", "file_bytes", "cpu_seconds"):
|
||||
value = policy[key]
|
||||
if type(value) is not int or not 1 <= value <= 9007199254740991:
|
||||
raise ValueError("producer_limits." + key + " must be a positive safe integer")
|
||||
os.environ["PWNLIB_NOTERM"] = "1"
|
||||
os.environ["PWNLIB_CACHE_DIR"] = str(Path(request_path).parent / "cache")
|
||||
limits = None
|
||||
@@ -35,7 +32,7 @@ def main(request_path, inspect, failure_type, profile):
|
||||
# The outer exit-status contract also covers allocation failure during serialization.
|
||||
memory_reserve = bytearray(1024 * 1024)
|
||||
try:
|
||||
limits = lower_resource_limits(policy)
|
||||
limits = lower_resource_limits()
|
||||
descriptor = os.open(Path(request_path).parent / "limits.json", os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
|
||||
json.dump(limits, handle)
|
||||
@@ -54,8 +51,8 @@ def main(request_path, inspect, failure_type, profile):
|
||||
else:
|
||||
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
|
||||
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
|
||||
if len(encoded) > policy["output_bytes"]:
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete artifact result exceeds producer_limits.output_bytes=" + str(policy["output_bytes"]) + "."}).encode("utf-8")
|
||||
if len(encoded) > OUTPUT_BYTES:
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete artifact result exceeds the 64 MiB reply budget."}).encode("utf-8")
|
||||
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(descriptor, "wb") as handle:
|
||||
handle.write(encoded)
|
||||
|
||||
@@ -15,11 +15,6 @@ import sys
|
||||
PROFILE = "[email protected];[email protected];[email protected]"
|
||||
PACKAGES = {"pwntools": "4.15.0", "pyelftools": "0.33", "unicorn": "2.1.2"}
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
# (e_machine, ELF class bits, byte order) -> architecture pwntools must also report.
|
||||
PROFILES = {(62, 64, "little"): "amd64", (183, 64, "little"): "aarch64", (3, 32, "little"): "i386",
|
||||
(40, 32, "little"): "arm", (8, 32, "little"): "mips", (8, 32, "big"): "mips"}
|
||||
PROFILE_SCOPE = ("Layout profile requires ELF64 x86-64, ELF64 AArch64 little-endian, ELF32 i386, "
|
||||
"ELF32 ARM little-endian or ELF32 MIPS little- or big-endian objects.")
|
||||
|
||||
|
||||
class LayoutFailure(Exception):
|
||||
@@ -121,16 +116,13 @@ def inspect_elf(path, cache):
|
||||
raise LayoutFailure("format", "Selected object does not contain an ELF header.")
|
||||
if len(content) < 16:
|
||||
raise LayoutFailure("format", "Selected ELF identification is truncated.")
|
||||
if content[4] not in (1, 2) or content[5] not in (1, 2):
|
||||
raise LayoutFailure("unsupported-target", "Selected ELF declares an unknown class or byte order.")
|
||||
bits, header_size = (32, 52) if content[4] == 1 else (64, 64)
|
||||
order = "little" if content[5] == 1 else "big"
|
||||
if len(content) < header_size:
|
||||
raise LayoutFailure("format", f"Selected ELF{bits} header is truncated.")
|
||||
profile = (int.from_bytes(content[18:20], order), bits, order)
|
||||
if profile not in PROFILES:
|
||||
raise LayoutFailure("unsupported-target", PROFILE_SCOPE)
|
||||
if int.from_bytes(content[16:18], order) not in (1, 2, 3):
|
||||
if content[4] != 2 or content[5] != 1:
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires ELF64 little-endian objects.")
|
||||
if len(content) < 64:
|
||||
raise LayoutFailure("format", "Selected ELF64 header is truncated.")
|
||||
if int.from_bytes(content[18:20], "little") != 62:
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF objects.")
|
||||
if int.from_bytes(content[16:18], "little") not in (1, 2, 3):
|
||||
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
|
||||
from pwnlib.context import context
|
||||
with context.local(cache_dir=str(cache), log_level="warning"):
|
||||
@@ -140,11 +132,6 @@ def inspect_elf(path, cache):
|
||||
try:
|
||||
headers = ELFFile(io.BytesIO(content))
|
||||
section_count = headers.num_sections()
|
||||
if section_count:
|
||||
location(headers.header.e_shoff, section_count * headers.header.e_shentsize, len(content))
|
||||
segment_count = headers.num_segments()
|
||||
if segment_count:
|
||||
location(headers.header.e_phoff, segment_count * headers.header.e_phentsize, len(content))
|
||||
if headers.header.e_shstrndx == 0xffff and section_count == 0:
|
||||
raise LayoutFailure("format", "ELF extended section-name index requires a declared section-zero header.")
|
||||
raw_name_index = headers.header.e_shstrndx
|
||||
@@ -160,21 +147,9 @@ def inspect_elf(path, cache):
|
||||
if names_header.sh_type != "SHT_STRTAB":
|
||||
raise LayoutFailure("format", f"ELF section-name table {section_name_index} is not SHT_STRTAB.")
|
||||
location(names_header.sh_offset, names_header.sh_size, len(content))
|
||||
except (ELFError, ValueError, OverflowError, AssertionError, IndexError) as error:
|
||||
except ELFError as error:
|
||||
raise LayoutFailure("format", str(error)) from error
|
||||
class LayoutELF(ELF):
|
||||
def search(self, needle, writable=False, executable=False):
|
||||
# This adapter exposes structural facts, GOT/PLT and mitigation
|
||||
# indicators, not kernel config/version discovery. Upstream's
|
||||
# constructor searches virtual memory for those unrelated
|
||||
# heuristics, materializing NOBITS tails and interpreting
|
||||
# ordinary IKCFG_ST data as a compressed kernel config.
|
||||
return iter(())
|
||||
|
||||
def _populate_kernel_version(self):
|
||||
# Kernel banner decoding is outside this report too.
|
||||
return None
|
||||
|
||||
def _get_section_name(self, header):
|
||||
# SHN_UNDEF declares no names; upstream would otherwise read
|
||||
# strings at the inactive section-zero offset.
|
||||
@@ -193,8 +168,8 @@ def inspect_elf(path, cache):
|
||||
if type is None or section.header.sh_type == type)
|
||||
try:
|
||||
image = LayoutELF(str(path), checksec=False)
|
||||
if (image.arch, image.bits, image.endian) != (PROFILES[profile], bits, order):
|
||||
raise LayoutFailure("unsupported-target", PROFILE_SCOPE)
|
||||
if image.arch != "amd64" or image.bits != 64 or image.endian != "little":
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF64 little-endian objects.")
|
||||
if image.header.e_type not in ("ET_EXEC", "ET_DYN", "ET_REL"):
|
||||
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
|
||||
length = len(content)
|
||||
@@ -216,7 +191,7 @@ def inspect_elf(path, cache):
|
||||
})
|
||||
if h.sh_type in ("SHT_SYMTAB", "SHT_DYNSYM"):
|
||||
if h.sh_entsize < image.structs.Elf_Sym.sizeof():
|
||||
raise LayoutFailure("format", f"ELF symbol entry is smaller than the decoded Elf{bits}_Sym structure.")
|
||||
raise LayoutFailure("format", "ELF symbol entry is smaller than the decoded Elf64_Sym structure.")
|
||||
if h.sh_link >= len(all_sections) or all_sections[h.sh_link].header.sh_type != "SHT_STRTAB":
|
||||
raise LayoutFailure("format", f"ELF symbol table {index} links to invalid string table {h.sh_link}.")
|
||||
strings = all_sections[h.sh_link]
|
||||
@@ -319,7 +294,6 @@ def inspect_elf(path, cache):
|
||||
"Relocatable relocation targets with SHN_UNDEF or inactive SHT_NULL headers retain their reported section index/offset as unknown; they do not establish an associated target section.",
|
||||
"GOT/PLT are derived upstream convenience maps; PLT inference can emulate selected instructions in Unicorn. They do not establish target runtime behavior; aliases may collapse and warnings may indicate incomplete coverage. Their completeness is unknown.",
|
||||
"Mitigations are upstream static heuristics, not runtime protection. ET_DYN does not prove an executable; absent canary symbols do not prove every function unprotected.",
|
||||
"Kernel configuration/version discovery is not performed; these memory-content scans do not supply the reported structural, linkage or mitigation facts.",
|
||||
"This profile inspects ELF EXEC/DYN/REL layout only. It does not analyze recorded cores, resolve loaded libraries, launch the target as a host process or start a debugger.",
|
||||
"Symbol/relocation inventories reflect original section tables. A sectionless image can still report dynamic dependency names; missing tables do not prove absence of dynamic symbols or relocations.",
|
||||
*(["Absent section-name table: section-dependent RELRO/canary heuristics may be incomplete; protection coverage is unknown."] if all_sections and section_name_index == 0 else []),
|
||||
@@ -328,7 +302,7 @@ def inspect_elf(path, cache):
|
||||
}
|
||||
except LayoutFailure:
|
||||
raise
|
||||
except (ELFError, ValueError, OverflowError, AssertionError, IndexError) as error:
|
||||
except (ELFError, ValueError, AssertionError, IndexError) as error:
|
||||
raise LayoutFailure("format", "Selected ELF failed unchanged upstream structural parsing: " + str(error)) from error
|
||||
|
||||
|
||||
|
||||
@@ -88,18 +88,16 @@ def inspect_core(path, cache):
|
||||
segments, notes, padding, threads, signals = [], [], [], [], []
|
||||
for segment_index, segment in enumerate(image.iter_segments()):
|
||||
h = segment.header
|
||||
header_offset = header.e_phoff + segment_index * header.e_phentsize
|
||||
type_id = int.from_bytes(content[header_offset:header_offset + 4], "little")
|
||||
backing = type_id != 0 and h.p_filesz > 0
|
||||
backing = h.p_type != "PT_NULL" and h.p_filesz > 0
|
||||
if backing:
|
||||
location(h.p_offset, h.p_filesz, len(content))
|
||||
if type_id == 1 and h.p_filesz > h.p_memsz:
|
||||
if h.p_type == "PT_LOAD" and h.p_filesz > h.p_memsz:
|
||||
raise CoreFailure("format", "Recorded PT_LOAD file bytes exceed memory size.")
|
||||
segments.append({"index": segment_index, "type": h.p_type, "type_id": type_id,
|
||||
"header_location": location(header_offset, header.e_phentsize, len(content)),
|
||||
segments.append({"index": segment_index, "type": h.p_type,
|
||||
"header_location": location(header.e_phoff + segment_index * header.e_phentsize, header.e_phentsize, len(content)),
|
||||
"offset": hex(h.p_offset), "file_size": hex(h.p_filesz), "memory_size": hex(h.p_memsz),
|
||||
"virtual_address": hex(h.p_vaddr), "physical_address": hex(h.p_paddr), "alignment": hex(h.p_align), "flags": hex(h.p_flags), "file_backing": "file" if backing else "none"})
|
||||
if type_id != 4 or h.p_filesz == 0:
|
||||
if h.p_type != "PT_NOTE" or h.p_filesz == 0:
|
||||
continue
|
||||
position, end = h.p_offset, h.p_offset + h.p_filesz
|
||||
header_size = image.structs.Elf_Nhdr.sizeof()
|
||||
@@ -111,7 +109,6 @@ def inspect_core(path, cache):
|
||||
padding.append({"segment_index": segment_index, "location": location(position, len(raw), end), "bytes_base64": encoded(raw)})
|
||||
break
|
||||
note = struct_parse(image.structs.Elf_Nhdr, image.stream, stream_pos=position)
|
||||
note_type_id = int.from_bytes(content[position + 8:position + 12], "little")
|
||||
owner_start = position + header_size
|
||||
descriptor_start = owner_start + ((note.n_namesz + 3) // 4) * 4
|
||||
note_end = descriptor_start + ((note.n_descsz + 3) // 4) * 4
|
||||
@@ -122,10 +119,7 @@ def inspect_core(path, cache):
|
||||
display = owner.split(b"\0", 1)[0].decode("utf-8", "replace") if owner else None
|
||||
descriptor = content[descriptor_start:descriptor_start + note.n_descsz]
|
||||
note_index = len(notes)
|
||||
# pyelftools selects ET_CORE labels without owner context. Keep
|
||||
# those labels only for exact CORE owners; other kinds are opaque.
|
||||
note_type = note.n_type if owner == b"CORE\0" else note_type_id
|
||||
notes.append({"index": note_index, "segment_index": segment_index, "type": note_type, "type_id": note_type_id,
|
||||
notes.append({"index": note_index, "segment_index": segment_index, "type": note.n_type,
|
||||
"location": location(position, note_end - position, end),
|
||||
"owner_location": location(owner_start, len(owner), end), "owner_bytes_base64": encoded(owner), "owner_display": display,
|
||||
"descriptor_location": location(descriptor_start, len(descriptor), end), "descriptor_bytes_base64": encoded(descriptor)})
|
||||
@@ -133,7 +127,7 @@ def inspect_core(path, cache):
|
||||
# Only the initial Linux CORE ABI is interpreted. Other owners retain exact bytes.
|
||||
if owner != b"CORE\0":
|
||||
continue
|
||||
if note_type_id == 1:
|
||||
if note.n_type == "NT_PRSTATUS":
|
||||
if len(descriptor) < ctypes.sizeof(prstatus_type):
|
||||
raise CoreFailure("format", "Linux " + profile_label + " NT_PRSTATUS descriptor is truncated.")
|
||||
status = prstatus_type.from_buffer_copy(descriptor)
|
||||
@@ -142,7 +136,7 @@ def inspect_core(path, cache):
|
||||
relative = prstatus_type.pr_reg.offset + getattr(registers_type, name).offset
|
||||
registers.append({"name": name, "value": hex(getattr(status.pr_reg, name)), "location": location(descriptor_start + relative, ctypes.sizeof(kind), end)})
|
||||
threads.append({"note_index": note_index, "historical_pid": ctypes.c_int32(status.pr_pid).value, "recorded_current_signal": status.pr_cursig, "registers": registers})
|
||||
elif note_type_id == 0x53494749:
|
||||
elif note.n_type == "NT_SIGINFO":
|
||||
if len(descriptor) < ctypes.sizeof(elf_siginfo_64):
|
||||
raise CoreFailure("format", "Linux " + profile_label + " NT_SIGINFO descriptor is truncated.")
|
||||
signal = elf_siginfo_64.from_buffer_copy(descriptor)
|
||||
|
||||
+82
-88
@@ -1,18 +1,4 @@
|
||||
import { defineConfig } from "vitepress";
|
||||
import { dirname, relative, resolve, sep } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import {
|
||||
documentationRewrites,
|
||||
documentationSections,
|
||||
} from "./documentation.js";
|
||||
|
||||
const documentationRoot = fileURLToPath(new URL("..", import.meta.url));
|
||||
const rewrites: Record<string, string> = {
|
||||
...documentationRewrites,
|
||||
"README.md": "documentation.md",
|
||||
"specs/README.md": "specs/index.md",
|
||||
"adr/README.md": "adr/index.md",
|
||||
};
|
||||
|
||||
export default defineConfig({
|
||||
title: "REA",
|
||||
@@ -20,92 +6,100 @@ export default defineConfig({
|
||||
"Reverse engineer binaries, applications, and runtime behavior with one CLI and MCP server.",
|
||||
lang: "en-US",
|
||||
base: "/rea/",
|
||||
rewrites,
|
||||
markdown: {
|
||||
config(md) {
|
||||
// Repository links resolve from their physical source. Render them from
|
||||
// the stable public route instead, including links across categories.
|
||||
md.core.ruler.after("inline", "documentation-source-links", (state) => {
|
||||
const physicalPath =
|
||||
state.env.realPath ??
|
||||
(state.env.relativePath
|
||||
? resolve(documentationRoot, state.env.relativePath)
|
||||
: null);
|
||||
if (physicalPath === null) return;
|
||||
const renderedPath =
|
||||
state.env.path ??
|
||||
resolve(
|
||||
documentationRoot,
|
||||
rewrites[state.env.relativePath] ?? state.env.relativePath,
|
||||
);
|
||||
for (const block of state.tokens) {
|
||||
for (const token of block.children ?? []) {
|
||||
const attribute =
|
||||
token.type === "link_open"
|
||||
? "href"
|
||||
: token.type === "image"
|
||||
? "src"
|
||||
: null;
|
||||
if (attribute === null) continue;
|
||||
const destination = token.attrGet(attribute);
|
||||
if (
|
||||
!destination ||
|
||||
/^(?:[a-z][a-z\d+.-]*:|[/#])/iu.test(destination)
|
||||
)
|
||||
continue;
|
||||
const separator = destination.search(/[?#]/u);
|
||||
const path =
|
||||
separator === -1 ? destination : destination.slice(0, separator);
|
||||
const suffix = separator === -1 ? "" : destination.slice(separator);
|
||||
const source = relative(
|
||||
documentationRoot,
|
||||
resolve(dirname(physicalPath), path),
|
||||
)
|
||||
.split(sep)
|
||||
.join("/");
|
||||
const rendered = relative(
|
||||
dirname(renderedPath),
|
||||
resolve(
|
||||
documentationRoot,
|
||||
source.startsWith("public/")
|
||||
? source.slice("public/".length)
|
||||
: (rewrites[source] ?? source),
|
||||
),
|
||||
)
|
||||
.split(sep)
|
||||
.join("/");
|
||||
token.attrSet(attribute, rendered + suffix);
|
||||
}
|
||||
}
|
||||
});
|
||||
},
|
||||
},
|
||||
srcExclude: ["readme/**"],
|
||||
themeConfig: {
|
||||
nav: [
|
||||
{ text: "Get started", link: "/installation" },
|
||||
{ text: "Documentation", link: "/documentation" },
|
||||
{ text: "Reference", link: "/mcp-contracts" },
|
||||
{ text: "Roadmap", link: "/roadmap" },
|
||||
],
|
||||
sidebar: [
|
||||
{
|
||||
text: "Documentation directory",
|
||||
text: "Getting started",
|
||||
items: [
|
||||
{ text: "All guides", link: "/documentation" },
|
||||
{ text: "Authoring conventions", link: "/development/documentation" },
|
||||
{ text: "Feature specifications", link: "/specs/" },
|
||||
{ text: "Architecture decisions", link: "/adr/" },
|
||||
{ text: "Installation and setup", link: "/installation" },
|
||||
{ text: "Client configuration", link: "/client-configuration" },
|
||||
{ text: "CLI and Evidence", link: "/cli" },
|
||||
{ text: "Agent prompts", link: "/mcp-prompts" },
|
||||
{ text: "Roadmap", link: "/roadmap" },
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Investigation guides",
|
||||
items: [
|
||||
{ text: "Native binaries", link: "/native-investigation" },
|
||||
{ text: "Go build metadata", link: "/go-binaries" },
|
||||
{ text: "IL2CPP metadata", link: "/il2cpp-metadata" },
|
||||
{ text: "Model artifacts", link: "/model-artifacts" },
|
||||
{ text: "Rust dependencies", link: "/rust-binaries" },
|
||||
{ text: "PE certificates", link: "/pe-certificates" },
|
||||
{ text: "ELF notes", link: "/elf-notes" },
|
||||
{ text: "CPython pyc files", link: "/python-pyc" },
|
||||
{ text: "Erlang BEAM", link: "/erlang-beam" },
|
||||
{
|
||||
text: "JavaScript and Electron",
|
||||
link: "/javascript-artifact-reconstruction",
|
||||
},
|
||||
{
|
||||
text: "JavaScript workflows",
|
||||
link: "/javascript-application-workflows",
|
||||
},
|
||||
{ text: "Browser observation", link: "/browser-observation" },
|
||||
{
|
||||
text: "Web network captures",
|
||||
link: "/web-network-captures",
|
||||
},
|
||||
{ text: "Agent environments", link: "/agent-environments" },
|
||||
{ text: "Electron observation", link: "/electron-observation" },
|
||||
{ text: "Electron fuses", link: "/electron-fuses" },
|
||||
{ text: "Process capture", link: "/process-capture" },
|
||||
{ text: ".NET assemblies", link: "/managed-code-analysis" },
|
||||
{
|
||||
text: "Python packaged applications",
|
||||
link: "/python-packaged-applications",
|
||||
},
|
||||
{ text: "Android applications", link: "/android-analysis" },
|
||||
{ text: "Apple applications", link: "/apple-application-analysis" },
|
||||
{ text: "Desktop installations", link: "/desktop-applications" },
|
||||
{ text: "Firmware", link: "/firmware-analysis" },
|
||||
{ text: "SQLite databases", link: "/sqlite-databases" },
|
||||
{ text: "Packet captures", link: "/packet-captures" },
|
||||
{ text: "Signal recordings", link: "/signal-recordings" },
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Provider guides",
|
||||
collapsed: true,
|
||||
items: [
|
||||
{ text: "IDA", link: "/ida-provider" },
|
||||
{ text: "Kuna", link: "/kuna-provider" },
|
||||
{ text: "Ghidra on Windows", link: "/windows-ghidra-p0" },
|
||||
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
|
||||
{ text: "Ghidra raw binary", link: "/raw-binary" },
|
||||
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
|
||||
{ text: "Ghidra PowerPC", link: "/ghidra-ppc" },
|
||||
{ text: "Ghidra semantics", link: "/ghidra-provider" },
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Reference and development",
|
||||
collapsed: true,
|
||||
items: [
|
||||
{ text: "MCP contracts", link: "/mcp-contracts" },
|
||||
{ text: "Tool design", link: "/tool-design" },
|
||||
{
|
||||
text: "Admission limits and recovery",
|
||||
link: "/design/limits-and-recovery",
|
||||
},
|
||||
{ text: "Testing", link: "/testing" },
|
||||
{ text: "Client verification", link: "/client-verification" },
|
||||
{
|
||||
text: "Reconstruction readiness",
|
||||
link: "/reconstruction-readiness",
|
||||
},
|
||||
{ text: "Releasing", link: "/releasing" },
|
||||
],
|
||||
},
|
||||
...documentationSections.map((section) => ({
|
||||
text: section.text,
|
||||
collapsed: section.text !== "Getting started",
|
||||
items: section.items.map((item) => ({
|
||||
text: item.text,
|
||||
link: item.route,
|
||||
})),
|
||||
})),
|
||||
],
|
||||
search: { provider: "local" },
|
||||
socialLinks: [
|
||||
|
||||
@@ -1,684 +0,0 @@
|
||||
// Authored guide ownership: physical sources and stable public routes.
|
||||
export const documentationSections = [
|
||||
{
|
||||
text: "Getting started",
|
||||
items: [
|
||||
{
|
||||
text: "Installation and setup",
|
||||
source: "getting-started/installation.md",
|
||||
route: "/installation",
|
||||
},
|
||||
{
|
||||
text: "Client configuration and troubleshooting",
|
||||
source: "getting-started/client-configuration.md",
|
||||
route: "/client-configuration",
|
||||
},
|
||||
{
|
||||
text: "CLI and Evidence",
|
||||
source: "getting-started/cli.md",
|
||||
route: "/cli",
|
||||
},
|
||||
{
|
||||
text: "Guided MCP prompts and completion",
|
||||
source: "getting-started/mcp-prompts.md",
|
||||
route: "/mcp-prompts",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Native binaries",
|
||||
items: [
|
||||
{
|
||||
text: "Native binary investigation",
|
||||
source: "guides/native/native-investigation.md",
|
||||
route: "/native-investigation",
|
||||
},
|
||||
{
|
||||
text: "Go binary build metadata",
|
||||
source: "guides/native/go-binaries.md",
|
||||
route: "/go-binaries",
|
||||
},
|
||||
{
|
||||
text: "Offline binary diagnostics",
|
||||
source: "guides/native/binary-diagnostics.md",
|
||||
route: "/binary-diagnostics",
|
||||
},
|
||||
{
|
||||
text: "ELF and Mach-O initializer functions",
|
||||
source: "guides/native/native-initializers.md",
|
||||
route: "/native-initializers",
|
||||
},
|
||||
{
|
||||
text: "ELF unwind function ranges",
|
||||
source: "guides/native/elf-unwind-ranges.md",
|
||||
route: "/elf-unwind-ranges",
|
||||
},
|
||||
{
|
||||
text: "Linux kernel build config",
|
||||
source: "guides/native/kernel-build-config.md",
|
||||
route: "/kernel-build-config",
|
||||
},
|
||||
{
|
||||
text: "Recorded crash evidence",
|
||||
source: "guides/native/recorded-crashes.md",
|
||||
route: "/recorded-crashes",
|
||||
},
|
||||
{
|
||||
text: "Windows Shell Link shortcuts",
|
||||
source: "guides/native/windows-shortcut.md",
|
||||
route: "/windows-shortcut",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "JavaScript and Electron",
|
||||
items: [
|
||||
{
|
||||
text: "JavaScript Application Graph",
|
||||
source: "guides/javascript/javascript-application-graph.md",
|
||||
route: "/javascript-application-graph",
|
||||
},
|
||||
{
|
||||
text: "Cross-layer JavaScript application workflows",
|
||||
source: "guides/javascript/javascript-application-workflows.md",
|
||||
route: "/javascript-application-workflows",
|
||||
},
|
||||
{
|
||||
text: "JavaScript artifact reconstruction",
|
||||
source: "guides/javascript/javascript-artifact-reconstruction.md",
|
||||
route: "/javascript-artifact-reconstruction",
|
||||
},
|
||||
{
|
||||
text: "JavaScript source recovery",
|
||||
source: "guides/javascript/javascript-recovery.md",
|
||||
route: "/javascript-recovery",
|
||||
},
|
||||
{
|
||||
text: "Passive Node and Electron runtime observation",
|
||||
source: "guides/javascript/javascript-runtime-observation.md",
|
||||
route: "/javascript-runtime-observation",
|
||||
},
|
||||
{
|
||||
text: "JavaScript static/runtime reconciliation",
|
||||
source: "guides/javascript/javascript-runtime-reconciliation.md",
|
||||
route: "/javascript-runtime-reconciliation",
|
||||
},
|
||||
{
|
||||
text: "Electron file-page observation",
|
||||
source: "guides/javascript/electron-observation.md",
|
||||
route: "/electron-observation",
|
||||
},
|
||||
{
|
||||
text: "V8 code cache inspection",
|
||||
source: "guides/javascript/v8-code-cache.md",
|
||||
route: "/v8-code-cache",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Browser and captured websites",
|
||||
items: [
|
||||
{
|
||||
text: "Website observation with CDP",
|
||||
source: "guides/web/browser-observation.md",
|
||||
route: "/browser-observation",
|
||||
},
|
||||
{
|
||||
text: "Browser scenario contract",
|
||||
source: "guides/web/browser-scenario-contract.md",
|
||||
route: "/browser-scenario-contract",
|
||||
},
|
||||
{
|
||||
text: "Website runtime source attribution",
|
||||
source: "guides/web/web-runtime.md",
|
||||
route: "/web-runtime",
|
||||
},
|
||||
{
|
||||
text: "Historical web network captures",
|
||||
source: "guides/web/web-network-captures.md",
|
||||
route: "/web-network-captures",
|
||||
},
|
||||
{
|
||||
text: "HTTP/3 capture lab",
|
||||
source: "http3-capture-lab.md",
|
||||
route: "/http3-capture-lab",
|
||||
},
|
||||
{
|
||||
text: "Captured website source locations",
|
||||
source: "guides/web/web-source-location.md",
|
||||
route: "/web-source-location",
|
||||
},
|
||||
{
|
||||
text: "Captured website module relationships",
|
||||
source: "guides/web/website-module-trace.md",
|
||||
route: "/website-module-trace",
|
||||
},
|
||||
{
|
||||
text: "Captured website scripts and static JavaScript analysis",
|
||||
source: "guides/web/website-script-export.md",
|
||||
route: "/website-script-export",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Android applications",
|
||||
items: [
|
||||
{
|
||||
text: "Static Android APK analysis",
|
||||
source: "guides/android/android-analysis.md",
|
||||
route: "/android-analysis",
|
||||
},
|
||||
{
|
||||
text: "Apktool resource decoding",
|
||||
source: "guides/android/apktool-resource-analysis.md",
|
||||
route: "/apktool-resource-analysis",
|
||||
},
|
||||
{
|
||||
text: "ADB device inspection and acquisition",
|
||||
source: "guides/android/adb-device-analysis.md",
|
||||
route: "/adb-device-analysis",
|
||||
},
|
||||
{
|
||||
text: "Flutter build identification",
|
||||
source: "guides/android/flutter-analysis.md",
|
||||
route: "/flutter-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Apple applications",
|
||||
items: [
|
||||
{
|
||||
text: "Apple Application Inventory Projection",
|
||||
source: "guides/apple/apple-application-analysis.md",
|
||||
route: "/apple-application-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: ".NET assemblies",
|
||||
items: [
|
||||
{
|
||||
text: "Managed-code analysis",
|
||||
source: "guides/managed/managed-code-analysis.md",
|
||||
route: "/managed-code-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Firmware",
|
||||
items: [
|
||||
{
|
||||
text: "Firmware analysis",
|
||||
source: "guides/firmware/firmware-analysis.md",
|
||||
route: "/firmware-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "HarmonyOS packages",
|
||||
items: [
|
||||
{
|
||||
text: "Static HarmonyOS package inventory",
|
||||
source: "guides/harmony/harmony-analysis.md",
|
||||
route: "/harmony-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "WebAssembly",
|
||||
items: [
|
||||
{
|
||||
text: "Offline WebAssembly artifacts",
|
||||
source: "guides/wasm/wasm-artifacts.md",
|
||||
route: "/wasm-artifacts",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "EVM contracts",
|
||||
items: [
|
||||
{
|
||||
text: "Offline EVM bytecode",
|
||||
source: "guides/evm/evm-bytecode.md",
|
||||
route: "/evm-bytecode",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Databases and recorded packets",
|
||||
items: [
|
||||
{
|
||||
text: "SQLite database snapshots",
|
||||
source: "guides/data/sqlite-databases.md",
|
||||
route: "/sqlite-databases",
|
||||
},
|
||||
{
|
||||
text: "Recorded packet capture inspection",
|
||||
source: "guides/data/packet-captures.md",
|
||||
route: "/packet-captures",
|
||||
},
|
||||
{
|
||||
text: "Packet capture summaries",
|
||||
source: "guides/data/pcap-analysis.md",
|
||||
route: "/pcap-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Process capture and instrumentation",
|
||||
items: [
|
||||
{
|
||||
text: "Process capture",
|
||||
source: "guides/runtime/process-capture.md",
|
||||
route: "/process-capture",
|
||||
},
|
||||
{
|
||||
text: "Frida instrumentation",
|
||||
source: "guides/runtime/frida-instrumentation.md",
|
||||
route: "/frida-instrumentation",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Provider setup and formats",
|
||||
items: [
|
||||
{
|
||||
text: "Ghidra provider semantics",
|
||||
source: "providers/ghidra-provider.md",
|
||||
route: "/ghidra-provider",
|
||||
},
|
||||
{
|
||||
text: "DOS MZ and COM analysis with Ghidra",
|
||||
source: "providers/ghidra-dos.md",
|
||||
route: "/ghidra-dos",
|
||||
},
|
||||
{
|
||||
text: "Optional NativeAOT metadata recovery",
|
||||
source: "providers/ghidra-nativeaot.md",
|
||||
route: "/ghidra-nativeaot",
|
||||
},
|
||||
{
|
||||
text: "MIPS ELF analysis through Ghidra",
|
||||
source: "providers/ghidra-mips.md",
|
||||
route: "/ghidra-mips",
|
||||
},
|
||||
{
|
||||
text: "PSP static ELF analysis through Ghidra",
|
||||
source: "providers/ghidra-psp.md",
|
||||
route: "/ghidra-psp",
|
||||
},
|
||||
{
|
||||
text: "RISC-V ELF analysis through Ghidra",
|
||||
source: "providers/ghidra-riscv.md",
|
||||
route: "/ghidra-riscv",
|
||||
},
|
||||
{
|
||||
text: "Windows Ghidra P0",
|
||||
source: "providers/windows-ghidra-p0.md",
|
||||
route: "/windows-ghidra-p0",
|
||||
},
|
||||
{
|
||||
text: "Headerless raw-binary analysis with Ghidra",
|
||||
source: "providers/raw-binary.md",
|
||||
route: "/raw-binary",
|
||||
},
|
||||
{
|
||||
text: "IDA provider",
|
||||
source: "providers/ida-provider.md",
|
||||
route: "/ida-provider",
|
||||
},
|
||||
{
|
||||
text: "JEB instrumentation and analysis",
|
||||
source: "providers/jeb-analysis.md",
|
||||
route: "/jeb-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Contracts and evidence",
|
||||
items: [
|
||||
{
|
||||
text: "MCP runtime contracts",
|
||||
source: "reference/mcp-contracts.md",
|
||||
route: "/mcp-contracts",
|
||||
},
|
||||
{
|
||||
text: "Reconstruction obligation ledgers",
|
||||
source: "reference/reconstruction-obligation-ledgers.md",
|
||||
route: "/reconstruction-obligation-ledgers",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Development and verification",
|
||||
items: [
|
||||
{
|
||||
text: "Settings and recovery",
|
||||
source: "design/limits-and-recovery.md",
|
||||
route: "/design/limits-and-recovery",
|
||||
},
|
||||
{
|
||||
text: "MCP tool design",
|
||||
source: "development/tool-design.md",
|
||||
route: "/tool-design",
|
||||
},
|
||||
{
|
||||
text: "Testing REA",
|
||||
source: "development/testing.md",
|
||||
route: "/testing",
|
||||
},
|
||||
{
|
||||
text: "Client and agent verification",
|
||||
source: "development/client-verification.md",
|
||||
route: "/client-verification",
|
||||
},
|
||||
{
|
||||
text: "Reconstruction readiness conformance",
|
||||
source: "development/reconstruction-readiness.md",
|
||||
route: "/reconstruction-readiness",
|
||||
},
|
||||
{
|
||||
text: "Releasing REA",
|
||||
source: "development/releasing.md",
|
||||
route: "/releasing",
|
||||
},
|
||||
{
|
||||
text: "REA roadmap",
|
||||
source: "development/roadmap.md",
|
||||
route: "/roadmap",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Release migrations",
|
||||
items: [
|
||||
{
|
||||
text: "Upgrading from REA 6.1 to 6.2",
|
||||
source: "migrations/migration-6.2.md",
|
||||
route: "/migration-6.2",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Signal recordings",
|
||||
items: [
|
||||
{
|
||||
text: "Recorded signal inspection",
|
||||
source: "guides/signals/signal-recordings.md",
|
||||
route: "/signal-recordings",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Additional guides",
|
||||
items: [
|
||||
{
|
||||
text: "Offline API contract inventory",
|
||||
source: "api-contracts.md",
|
||||
route: "/api-contracts",
|
||||
},
|
||||
{
|
||||
text: "Bun standalone executables",
|
||||
source: "bun-standalone.md",
|
||||
route: "/bun-standalone",
|
||||
},
|
||||
{
|
||||
text: "Deno compile executables",
|
||||
source: "deno-standalone.md",
|
||||
route: "/deno-standalone",
|
||||
},
|
||||
{
|
||||
text: "Capability composition and ownership",
|
||||
source: "capability-composition.md",
|
||||
route: "/capability-composition",
|
||||
},
|
||||
{
|
||||
text: ".NET single-file bundles",
|
||||
source: "dotnet-bundles.md",
|
||||
route: "/dotnet-bundles",
|
||||
},
|
||||
{
|
||||
text: "ESP flash images",
|
||||
source: "esp-flash-image.md",
|
||||
route: "/esp-flash-image",
|
||||
},
|
||||
{
|
||||
text: "U-Boot FIT images",
|
||||
source: "fit-images.md",
|
||||
route: "/fit-images",
|
||||
},
|
||||
{
|
||||
text: "Measure MCP catalog costs in Codex",
|
||||
source: "mcp-catalog-evaluation.md",
|
||||
route: "/mcp-catalog-evaluation",
|
||||
},
|
||||
{
|
||||
text: "Offline Playwright traces",
|
||||
source: "playwright-traces.md",
|
||||
route: "/playwright-traces",
|
||||
},
|
||||
{
|
||||
text: "Testing lanes and provider verification",
|
||||
source: "testing-lanes.md",
|
||||
route: "/testing-lanes",
|
||||
},
|
||||
{
|
||||
text: "UF2 and DfuSe images",
|
||||
source: "uf2-dfuse.md",
|
||||
route: "/uf2-dfuse",
|
||||
},
|
||||
{
|
||||
text: "WeChat mini-program wxapkg archives",
|
||||
source: "wxapkg.md",
|
||||
route: "/wxapkg",
|
||||
},
|
||||
{
|
||||
text: "Binary hardening",
|
||||
source: "binary-hardening.md",
|
||||
route: "/binary-hardening",
|
||||
},
|
||||
{
|
||||
text: "Desktop application capture",
|
||||
source: "desktop-application-capture.md",
|
||||
route: "/desktop-application-capture",
|
||||
},
|
||||
{
|
||||
text: "Desktop application export",
|
||||
source: "desktop-application-export.md",
|
||||
route: "/desktop-application-export",
|
||||
},
|
||||
{
|
||||
text: "Desktop application state",
|
||||
source: "desktop-application-state.md",
|
||||
route: "/desktop-application-state",
|
||||
},
|
||||
{
|
||||
text: "Desktop application tasks",
|
||||
source: "desktop-application-tasks.md",
|
||||
route: "/desktop-application-tasks",
|
||||
},
|
||||
{
|
||||
text: "Desktop build computer",
|
||||
source: "desktop-build-computer.md",
|
||||
route: "/desktop-build-computer",
|
||||
},
|
||||
{
|
||||
text: "Desktop build witness",
|
||||
source: "desktop-build-witness.md",
|
||||
route: "/desktop-build-witness",
|
||||
},
|
||||
{
|
||||
text: "iNES containers",
|
||||
source: "ines-containers.md",
|
||||
route: "/ines-containers",
|
||||
},
|
||||
{
|
||||
text: "Build RL environments with REA",
|
||||
source: "agent-environments-quickstart.md",
|
||||
route: "/agent-environments-quickstart",
|
||||
},
|
||||
{
|
||||
text: "Agent environments from recorded captures",
|
||||
source: "agent-environments.md",
|
||||
route: "/agent-environments",
|
||||
},
|
||||
{
|
||||
text: "APK signature schemes",
|
||||
source: "apk-signature.md",
|
||||
route: "/apk-signature",
|
||||
},
|
||||
{
|
||||
text: "Apple IPS crash reports",
|
||||
source: "apple-ips.md",
|
||||
route: "/apple-ips",
|
||||
},
|
||||
{
|
||||
text: "Static analysis in GitHub Actions",
|
||||
source: "ci-analysis.md",
|
||||
route: "/ci-analysis",
|
||||
},
|
||||
{
|
||||
text: "Desktop installation directories",
|
||||
source: "desktop-applications.md",
|
||||
route: "/desktop-applications",
|
||||
},
|
||||
{
|
||||
text: "Flattened devicetree inspection",
|
||||
source: "devicetree.md",
|
||||
route: "/devicetree",
|
||||
},
|
||||
{
|
||||
text: "Electron fuse state",
|
||||
source: "electron-fuses.md",
|
||||
route: "/electron-fuses",
|
||||
},
|
||||
{ text: "ELF note tables", source: "elf-notes.md", route: "/elf-notes" },
|
||||
{
|
||||
text: "Erlang BEAM files",
|
||||
source: "erlang-beam.md",
|
||||
route: "/erlang-beam",
|
||||
},
|
||||
{ text: "ESP images", source: "esp-image.md", route: "/esp-image" },
|
||||
{
|
||||
text: "GameCube DOL analysis through Ghidra",
|
||||
source: "ghidra-dol.md",
|
||||
route: "/ghidra-dol",
|
||||
},
|
||||
{
|
||||
text: "PowerPC ELF analysis through Ghidra",
|
||||
source: "ghidra-ppc.md",
|
||||
route: "/ghidra-ppc",
|
||||
},
|
||||
{
|
||||
text: "Godot PCK archives",
|
||||
source: "godot-pck.md",
|
||||
route: "/godot-pck",
|
||||
},
|
||||
{
|
||||
text: "Static CUDA and HIP code objects",
|
||||
source: "gpu-code-objects.md",
|
||||
route: "/gpu-code-objects",
|
||||
},
|
||||
{
|
||||
text: "Intel HEX and S-record images",
|
||||
source: "hex-records.md",
|
||||
route: "/hex-records",
|
||||
},
|
||||
{
|
||||
text: "IL2CPP metadata",
|
||||
source: "il2cpp-metadata.md",
|
||||
route: "/il2cpp-metadata",
|
||||
},
|
||||
{
|
||||
text: "Static JavaScript route inventory",
|
||||
source: "javascript-route-inventory.md",
|
||||
route: "/javascript-route-inventory",
|
||||
},
|
||||
{
|
||||
text: "Linux kernel modules",
|
||||
source: "kernel-modules.md",
|
||||
route: "/kernel-modules",
|
||||
},
|
||||
{
|
||||
text: "Kuna provider",
|
||||
source: "kuna-provider.md",
|
||||
route: "/kuna-provider",
|
||||
},
|
||||
{
|
||||
text: "Model file declarations",
|
||||
source: "model-artifacts.md",
|
||||
route: "/model-artifacts",
|
||||
},
|
||||
{
|
||||
text: "Match native functions across builds",
|
||||
source: "native-function-matching.md",
|
||||
route: "/native-function-matching",
|
||||
},
|
||||
{
|
||||
text: "Node.js single executable applications",
|
||||
source: "node-sea.md",
|
||||
route: "/node-sea",
|
||||
},
|
||||
{
|
||||
text: "PE Authenticode certificate tables",
|
||||
source: "pe-certificates.md",
|
||||
route: "/pe-certificates",
|
||||
},
|
||||
{
|
||||
text: "PE import and delay-load tables",
|
||||
source: "pe-imports.md",
|
||||
route: "/pe-imports",
|
||||
},
|
||||
{
|
||||
text: "PE TLS callbacks and load configuration",
|
||||
source: "pe-load-config.md",
|
||||
route: "/pe-load-config",
|
||||
},
|
||||
{
|
||||
text: "Python packaged applications",
|
||||
source: "python-packaged-applications.md",
|
||||
route: "/python-packaged-applications",
|
||||
},
|
||||
{
|
||||
text: "CPython pyc inspection",
|
||||
source: "python-pyc.md",
|
||||
route: "/python-pyc",
|
||||
},
|
||||
{
|
||||
text: "Reconstruction coverage",
|
||||
source: "reconstruction-coverage.md",
|
||||
route: "/reconstruction-coverage",
|
||||
},
|
||||
{
|
||||
text: "Reverse-engineering tool integrations",
|
||||
source: "reverse-engineering-tools.md",
|
||||
route: "/reverse-engineering-tools",
|
||||
},
|
||||
{
|
||||
text: "Rust binary dependency metadata",
|
||||
source: "rust-binaries.md",
|
||||
route: "/rust-binaries",
|
||||
},
|
||||
{
|
||||
text: "U-Boot legacy uImage containers",
|
||||
source: "uimage.md",
|
||||
route: "/uimage",
|
||||
},
|
||||
{
|
||||
text: "UnityFS AssetBundle archives",
|
||||
source: "unityfs.md",
|
||||
route: "/unityfs",
|
||||
},
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
export const documentationRewrites = Object.fromEntries(
|
||||
documentationSections.flatMap((section) =>
|
||||
section.items.flatMap((item) => {
|
||||
const target = `${item.route.slice(1)}.md`;
|
||||
return item.source === target ? [] : [[item.source, target]];
|
||||
}),
|
||||
),
|
||||
);
|
||||
-205
@@ -1,205 +0,0 @@
|
||||
# Documentation directory
|
||||
|
||||
REA documentation is organized by reader task and document owner. This directory
|
||||
links every authored guide; the [site](https://rea.tools/) also serves separate
|
||||
product pages. The VitePress documentation source is this `docs/` tree.
|
||||
|
||||
## Getting started
|
||||
|
||||
- [Installation and setup](getting-started/installation.md)
|
||||
- [Client configuration and troubleshooting](getting-started/client-configuration.md)
|
||||
- [CLI and Evidence](getting-started/cli.md)
|
||||
- [Guided MCP prompts and completion](getting-started/mcp-prompts.md)
|
||||
|
||||
## Native binaries
|
||||
|
||||
- [Native binary investigation](guides/native/native-investigation.md)
|
||||
- [Go binary build metadata](guides/native/go-binaries.md)
|
||||
- [Offline binary diagnostics](guides/native/binary-diagnostics.md)
|
||||
- [ELF and Mach-O initializer functions](guides/native/native-initializers.md)
|
||||
- [ELF unwind function ranges](guides/native/elf-unwind-ranges.md)
|
||||
- [Linux kernel build config](guides/native/kernel-build-config.md)
|
||||
- [Recorded crash evidence](guides/native/recorded-crashes.md)
|
||||
- [Windows Shell Link shortcuts](guides/native/windows-shortcut.md)
|
||||
|
||||
## JavaScript and Electron
|
||||
|
||||
- [JavaScript Application Graph](guides/javascript/javascript-application-graph.md)
|
||||
- [Cross-layer JavaScript application workflows](guides/javascript/javascript-application-workflows.md)
|
||||
- [JavaScript artifact reconstruction](guides/javascript/javascript-artifact-reconstruction.md)
|
||||
- [JavaScript source recovery](guides/javascript/javascript-recovery.md)
|
||||
- [Passive Node and Electron runtime observation](guides/javascript/javascript-runtime-observation.md)
|
||||
- [JavaScript static/runtime reconciliation](guides/javascript/javascript-runtime-reconciliation.md)
|
||||
- [Electron file-page observation](guides/javascript/electron-observation.md)
|
||||
- [V8 code cache inspection](guides/javascript/v8-code-cache.md)
|
||||
|
||||
## Browser and captured websites
|
||||
|
||||
- [Website observation with CDP](guides/web/browser-observation.md)
|
||||
- [Browser scenario contract](guides/web/browser-scenario-contract.md)
|
||||
- [Website runtime source attribution](guides/web/web-runtime.md)
|
||||
- [Historical web network captures](guides/web/web-network-captures.md)
|
||||
- [HTTP/3 capture lab](http3-capture-lab.md)
|
||||
- [Captured website source locations](guides/web/web-source-location.md)
|
||||
- [Captured website module relationships](guides/web/website-module-trace.md)
|
||||
- [Captured website scripts and static JavaScript analysis](guides/web/website-script-export.md)
|
||||
|
||||
## Android applications
|
||||
|
||||
- [Static Android APK analysis](guides/android/android-analysis.md)
|
||||
- [Apktool resource decoding](guides/android/apktool-resource-analysis.md)
|
||||
- [ADB device inspection and acquisition](guides/android/adb-device-analysis.md)
|
||||
- [Flutter build identification](guides/android/flutter-analysis.md)
|
||||
|
||||
## Apple applications
|
||||
|
||||
- [Apple Application Inventory Projection](guides/apple/apple-application-analysis.md)
|
||||
|
||||
## .NET assemblies
|
||||
|
||||
- [Managed-code analysis](guides/managed/managed-code-analysis.md)
|
||||
|
||||
## Firmware
|
||||
|
||||
- [Firmware analysis](guides/firmware/firmware-analysis.md)
|
||||
|
||||
## HarmonyOS packages
|
||||
|
||||
- [Static HarmonyOS package inventory](guides/harmony/harmony-analysis.md)
|
||||
|
||||
## WebAssembly
|
||||
|
||||
- [Offline WebAssembly artifacts](guides/wasm/wasm-artifacts.md)
|
||||
|
||||
## EVM contracts
|
||||
|
||||
- [Offline EVM bytecode](guides/evm/evm-bytecode.md)
|
||||
|
||||
## Databases and recorded packets
|
||||
|
||||
- [SQLite database snapshots](guides/data/sqlite-databases.md)
|
||||
- [Recorded packet capture inspection](guides/data/packet-captures.md)
|
||||
- [Packet capture summaries](guides/data/pcap-analysis.md)
|
||||
|
||||
## Process capture and instrumentation
|
||||
|
||||
- [Process capture](guides/runtime/process-capture.md)
|
||||
- [Frida instrumentation](guides/runtime/frida-instrumentation.md)
|
||||
|
||||
## Signal recordings
|
||||
|
||||
- [Recorded signal inspection](guides/signals/signal-recordings.md)
|
||||
|
||||
## Provider setup and formats
|
||||
|
||||
- [Ghidra provider semantics](providers/ghidra-provider.md)
|
||||
- [DOS MZ and COM analysis with Ghidra](providers/ghidra-dos.md)
|
||||
- [Optional NativeAOT metadata recovery](providers/ghidra-nativeaot.md)
|
||||
- [MIPS ELF analysis through Ghidra](providers/ghidra-mips.md)
|
||||
- [PSP static ELF analysis through Ghidra](providers/ghidra-psp.md)
|
||||
- [RISC-V ELF analysis through Ghidra](providers/ghidra-riscv.md)
|
||||
- [Windows Ghidra P0](providers/windows-ghidra-p0.md)
|
||||
- [Headerless raw-binary analysis with Ghidra](providers/raw-binary.md)
|
||||
- [IDA provider](providers/ida-provider.md)
|
||||
- [JEB instrumentation and analysis](providers/jeb-analysis.md)
|
||||
|
||||
## Contracts and evidence
|
||||
|
||||
- [MCP runtime contracts](reference/mcp-contracts.md)
|
||||
- [Reconstruction obligation ledgers](reference/reconstruction-obligation-ledgers.md)
|
||||
|
||||
## Development and verification
|
||||
|
||||
- [Settings and recovery](design/limits-and-recovery.md)
|
||||
- [Flutter budget calibration](verification/flutter-budget-calibration.md)
|
||||
- [Source-map budget calibration](verification/source-map-budget-calibration.md)
|
||||
- [Source-location delivery qualification](verification/source-location-delivery.md)
|
||||
- [JavaScript precision calibration](verification/javascript-precision-calibration.md)
|
||||
- [MCP tool design](development/tool-design.md)
|
||||
- [Testing REA](development/testing.md)
|
||||
- [Client and agent verification](development/client-verification.md)
|
||||
- [Reconstruction readiness conformance](development/reconstruction-readiness.md)
|
||||
- [Releasing REA](development/releasing.md)
|
||||
- [REA roadmap](development/roadmap.md)
|
||||
|
||||
## Release migrations
|
||||
|
||||
- [Upgrading from REA 6.1 to 6.2](migrations/migration-6.2.md)
|
||||
|
||||
## Architecture and authoring
|
||||
|
||||
- [Architecture map](https://github.com/morluto/rea/blob/main/docs/architecture/architecture.mermaid)
|
||||
- [Documentation ownership and contribution model](development/documentation.md)
|
||||
- [Feature specifications](specs/README.md)
|
||||
- [Architectural decision records](adr/README.md)
|
||||
|
||||
## Machine-owned documentation
|
||||
|
||||
| Path | Owner and update command |
|
||||
| ----------------------------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| `public/product-catalog.json` | Ignored contract projection; `npm run docs:generate` |
|
||||
| `verification/managed-conformance-*.json` | Ignored managed conformance projections; `npm run evidence:generate` |
|
||||
| `error-contract.schema.json` | Tracked schema from contracts; `npm run docs:error-schema` |
|
||||
| `user-visible-errors.csv` | Authored error classification inventory; preserve its schema |
|
||||
| `releases/<version>.contributions.json` | Durable release contribution record; follow the [release guide](development/releasing.md) |
|
||||
| `assets/` | Shared authored images referenced by guides |
|
||||
|
||||
Do not edit generated catalogs or evidence to change product facts. Change their
|
||||
canonical contracts or producer and regenerate. Historical release records retain
|
||||
the exact commit messages and references from the release they describe.
|
||||
|
||||
## Additional guides
|
||||
|
||||
- [Offline API contract inventory](api-contracts.md)
|
||||
- [Bun standalone executables](bun-standalone.md)
|
||||
- [Deno compile executables](deno-standalone.md)
|
||||
- [Capability composition and ownership](capability-composition.md)
|
||||
- [.NET single-file bundles](dotnet-bundles.md)
|
||||
- [ESP flash images](esp-flash-image.md)
|
||||
- [U-Boot FIT images](fit-images.md)
|
||||
- [Measure MCP catalog costs in Codex](mcp-catalog-evaluation.md)
|
||||
- [Offline Playwright traces](playwright-traces.md)
|
||||
- [Testing lanes and provider verification](testing-lanes.md)
|
||||
- [UF2 and DfuSe images](uf2-dfuse.md)
|
||||
- [WeChat mini-program wxapkg archives](wxapkg.md)
|
||||
- [Binary hardening](binary-hardening.md)
|
||||
- [Desktop application capture](desktop-application-capture.md)
|
||||
- [Desktop application export](desktop-application-export.md)
|
||||
- [Desktop application state](desktop-application-state.md)
|
||||
- [Desktop application tasks](desktop-application-tasks.md)
|
||||
- [Desktop build computer](desktop-build-computer.md)
|
||||
- [Desktop build witness](desktop-build-witness.md)
|
||||
- [iNES containers](ines-containers.md)
|
||||
- [Build RL environments with REA](agent-environments-quickstart.md)
|
||||
- [Agent environments from recorded captures](agent-environments.md)
|
||||
- [APK signature schemes](apk-signature.md)
|
||||
- [Apple IPS crash reports](apple-ips.md)
|
||||
- [Static analysis in GitHub Actions](ci-analysis.md)
|
||||
- [Desktop installation directories](desktop-applications.md)
|
||||
- [Flattened devicetree inspection](devicetree.md)
|
||||
- [Electron fuse state](electron-fuses.md)
|
||||
- [ELF note tables](elf-notes.md)
|
||||
- [Erlang BEAM files](erlang-beam.md)
|
||||
- [ESP images](esp-image.md)
|
||||
- [GameCube DOL analysis through Ghidra](ghidra-dol.md)
|
||||
- [PowerPC ELF analysis through Ghidra](ghidra-ppc.md)
|
||||
- [Godot PCK archives](godot-pck.md)
|
||||
- [Static CUDA and HIP code objects](gpu-code-objects.md)
|
||||
- [Intel HEX and S-record images](hex-records.md)
|
||||
- [IL2CPP metadata](il2cpp-metadata.md)
|
||||
- [Static JavaScript route inventory](javascript-route-inventory.md)
|
||||
- [Linux kernel modules](kernel-modules.md)
|
||||
- [Kuna provider](kuna-provider.md)
|
||||
- [Model file declarations](model-artifacts.md)
|
||||
- [Match native functions across builds](native-function-matching.md)
|
||||
- [Node.js single executable applications](node-sea.md)
|
||||
- [PE Authenticode certificate tables](pe-certificates.md)
|
||||
- [PE import and delay-load tables](pe-imports.md)
|
||||
- [PE TLS callbacks and load configuration](pe-load-config.md)
|
||||
- [Python packaged applications](python-packaged-applications.md)
|
||||
- [CPython pyc inspection](python-pyc.md)
|
||||
- [Reconstruction coverage](reconstruction-coverage.md)
|
||||
- [Reverse-engineering tool integrations](reverse-engineering-tools.md)
|
||||
- [Rust binary dependency metadata](rust-binaries.md)
|
||||
- [U-Boot legacy uImage containers](uimage.md)
|
||||
- [UnityFS AssetBundle archives](unityfs.md)
|
||||
@@ -25,21 +25,7 @@ REA never installs the Android SDK, an emulator, or a system image.
|
||||
Readiness is `rea inspect-adb-client`: it runs `adb version` and reports the
|
||||
resolved binary, its version, and where it was selected from. A missing
|
||||
binary reports `capability_unavailable` with the `REA_ADB_PATH` remediation
|
||||
rather than attempting an installation. A path that cannot resolve to a file
|
||||
(`ENOENT`, `ENOTDIR`, `ELOOP` or `ENAMETOOLONG`) is reported as missing. A
|
||||
selected path the host refuses to execute, such as a directory or a file
|
||||
without execute permission (`EACCES`) or one the host does not permit
|
||||
(`EPERM`), and any other spawn error report `capability_unavailable` naming
|
||||
that path. Exhausted host capacity (`EAGAIN`, `EMFILE`, `ENFILE` or `ENOMEM`)
|
||||
is not a configuration failure. `inspect_adb_client`, and any device
|
||||
operation whose adb cannot start at its version probe or at a later adb
|
||||
command, report a retryable `capacity` resource constraint rather than a
|
||||
capability refusal or an adb protocol failure; the `binary_session` inventory
|
||||
reports the provider as `runtime_unavailable`. Both say the host could not
|
||||
start adb and ask for a retry after freeing processes or descriptors, without
|
||||
blaming `REA_ADB_PATH`. When adb never started, the refusal diagnostics record
|
||||
the spawn error as `filesystem_error` with `path_source`, never as an
|
||||
`exit_code`.
|
||||
rather than attempting an installation.
|
||||
The `binary_session` inventory uses the same bounded client-version probe; it
|
||||
does not query devices, start the adb server, or establish that a requested
|
||||
serial is reachable. Device state is checked by each device operation.
|
||||
@@ -113,14 +99,10 @@ existing analysis families directly, for example
|
||||
| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); checks for existing targets and refuses them without explicit overwrite (the check is not atomic); compares digests when device `sha256sum` is available |
|
||||
| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves its Success/Failure verdict, records `replace_requested`, and leaves actual replacement `null` because prior package state is not observed |
|
||||
| `uninstall_adb_package` | **mutates device** | Requests package removal for user 0 via `adb shell pm uninstall --user 0`, without data retention |
|
||||
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -W -n`) |
|
||||
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -W -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
|
||||
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
|
||||
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
|
||||
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
|
||||
|
||||
Screen capture validates the PNG signature, IHDR framing, dimensions, and IHDR
|
||||
checksum before publishing. It does not validate encoding fields or decode and
|
||||
validate image contents; that limitation is retained in Evidence.
|
||||
|
||||
When `push_adb_file` receives an existing directory, adb places the local file
|
||||
inside it under the local basename; Evidence reports that effective file path
|
||||
while retaining the caller's directory path in the input parameters. The
|
||||
@@ -154,29 +136,6 @@ and during shutdown.
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- **Package numeric declarations retain failures.** `inspect_adb_package`
|
||||
admits the first valid `userId` or `appId` as `user_id`: a nonnegative safe
|
||||
integer, including zero. A null value means no valid declaration was found.
|
||||
Malformed declarations before or after a valid value remain in
|
||||
`numeric_issues`: the first ten issues with command, field, reason and original
|
||||
`raw_line`, plus the exact `total`. Reasons are `invalid_format` or
|
||||
`unsafe_integer`. Any issue makes package coverage partial; existing section
|
||||
omissions can also make it partial. `version_code` remains exact source text.
|
||||
- **Display nulls preserve uncertainty.** `inspect_adb_display` admits positive
|
||||
safe integers. Absent or malformed optional values are null; `numeric_issues`
|
||||
distinguishes malformed declarations from absence. It keeps the first ten
|
||||
issues and exact total, with command, field, original line and reason
|
||||
(`invalid_format`, `unsafe_integer` or `nonpositive`). The last recognized
|
||||
labeled declaration wins, even when malformed. A physical-density label
|
||||
suppresses bare-number fallback even when malformed; without that label, the
|
||||
first unsigned bare number wins, including zero or an unsafe integer that
|
||||
leaves the value null. Display coverage is complete only with known physical
|
||||
density and no numeric issues. Missing or malformed required physical size
|
||||
produces a protocol failure with `details.partial_observation`: an
|
||||
`inspect_adb_display` observation from provider `adb`, reason
|
||||
`physical_size_absent` or `physical_size_malformed`, and a `result` retaining
|
||||
client, serial, optional display facts, numeric issues, null physical size
|
||||
and partial coverage.
|
||||
- **Directory names retain whitespace and newlines.** Symlink targets do too,
|
||||
including trailing newlines. Records containing invalid UTF-8 remain as
|
||||
labelled hex in `unparsed_lines` and make directory coverage partial.
|
||||
@@ -188,7 +147,7 @@ and during shutdown.
|
||||
property value remains available in `properties`.
|
||||
- **Device paths are not host paths.** `base_apk_device_path` values live on
|
||||
the device filesystem. Only pulls and captures produce host-local paths,
|
||||
with byte size and SHA-256 digest on success. A pulled file's host name is
|
||||
each paired with byte size and SHA-256 digest. A pulled file's host name is
|
||||
the device path's final POSIX segment. REA preserves the original device
|
||||
path and filename; a missing final segment or `.`/`..` is refused before
|
||||
output creation. Windows hosts follow [Windows filename rules](https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file),
|
||||
@@ -217,18 +176,12 @@ and during shutdown.
|
||||
`uninstall_adb_package` declare `mutatesTarget` and never run implicitly:
|
||||
pushes check for existing device files before transfer and refuse them
|
||||
unless `overwrite` is set. A failed or unexpected existence probe stops
|
||||
transfer even with `overwrite`. A completed `test -e` exit of 1 establishes
|
||||
absence only with empty stdout and either empty stderr or a recognized,
|
||||
completed adb-server startup notice sequence. Startup notices are host
|
||||
observations, not remote-path diagnostics; successful push receipts retain
|
||||
them in `raw_result.device_path_probes`. Incomplete startup, extra diagnostics,
|
||||
truncation and cancellation do not establish absence. The check is not atomic, so a different
|
||||
transfer even with `overwrite`; only a clean `test -e` exit of 1 with no
|
||||
output is interpreted as absence. The check is not atomic, so a different
|
||||
process can create the path before `adb push` writes it. When available, the device's
|
||||
`sha256sum` verifies the transfer. A missing command, output without a
|
||||
recognizable digest, or host capacity that prevents the digest check from
|
||||
starting reports `device_digest_source: "unavailable"` with an
|
||||
unverified-transfer limitation rather than a retryable error; installs
|
||||
preserve the
|
||||
`sha256sum` verifies the transfer. A missing command or output without a
|
||||
recognizable digest reports `device_digest_source: "unavailable"` with an
|
||||
unverified-transfer limitation; installs preserve the
|
||||
device's `Failure [...]` reason verbatim.
|
||||
- **Producer identity stays attached.** Every result embeds the producing
|
||||
adb client (`binary_path`, `path_source`, `version`, `installed_path`).
|
||||
@@ -249,10 +202,6 @@ and during shutdown.
|
||||
- Pull and capture outputs are additive: the destination must not already
|
||||
exist. Re-running a completed pull refuses the existing directory instead
|
||||
of overwriting evidence.
|
||||
- If a single-file pull fails and REA observes a regular file at the destination,
|
||||
the error includes its `local_path` with `transfer_status: "unconfirmed"`
|
||||
and `digest_status: "unknown"`. The file remains in place; the receipt does
|
||||
not establish that the transfer completed.
|
||||
- Cancellation during a package pull preserves caller-selected output. The
|
||||
error includes partial observations identifying completed artifacts and the
|
||||
interrupted local path; remaining APKs were not attempted.
|
||||
@@ -281,25 +230,15 @@ beyond already-connected devices (`adb connect`) are not exposed yet.
|
||||
|
||||
## Real verification
|
||||
|
||||
`npm run verify:adb` exercises observation and transfer operations against a live device. It
|
||||
`npm run verify:adb` exercises every operation against a live device. It
|
||||
requires `adb` (or `REA_ADB_PATH`) and any attached emulator or phone;
|
||||
`--serial` selects one when several are attached, `--pull PACKAGE` pulls a
|
||||
real package and verifies the returned digests against the files on disk.
|
||||
The optional `--install-apk` lifecycle is unavailable until the selected APK
|
||||
identity and device cleanup ownership can be established safely.
|
||||
real package and verifies the returned digests against the files on disk,
|
||||
and `--install-apk PATH` drives a full install → resolve → uninstall
|
||||
lifecycle with a lane-owned probe.
|
||||
|
||||
Record: verified against adb 34.0.5 on Linux against an Android 14
|
||||
(API 34) x86_64 emulator — all operations above, a 224-package inventory,
|
||||
287 services, 92 features, a real two-APK split set pulled with byte-exact
|
||||
digests, a verified push/pull roundtrip, a 1.3 MB screen capture, and the
|
||||
complete package lifecycle; see `docs/development/testing.md` for the lane summary.
|
||||
|
||||
Intent integer extras use signed decimal text. The ADB argument encoder removes
|
||||
redundant leading zeros for Android's radix-sensitive `--ei` parser while retained
|
||||
Evidence keeps the caller's original spelling. Package inspection accepts both
|
||||
`userId` and Android 16's `appId` spelling for the legacy `user_id` result field.
|
||||
|
||||
A completed push and device digest verification are separate observations. A
|
||||
verification timeout, cancellation, or output budget failure returns the typed
|
||||
failure with partial Evidence of the completed transfer. An unavailable checksum
|
||||
command or unrecognized checksum output leaves verification explicitly unknown.
|
||||
complete package lifecycle; see `docs/testing.md` for the lane summary.
|
||||
@@ -1,9 +0,0 @@
|
||||
# Architectural decision records
|
||||
|
||||
Store one actual decision per `docs/adr/<domain>/<decision-slug>.md`, without a
|
||||
global sequence number. Use meaningful names and the Michael Nygard sections:
|
||||
Status, Context, Decision and Consequences. Link each decision from its owning
|
||||
feature specification.
|
||||
|
||||
See [the authoring guide](../development/documentation.md#feature-lifecycle) for
|
||||
the distinction between specifications, roadmaps and isolated decisions.
|
||||
@@ -1,206 +0,0 @@
|
||||
# Build RL environments with REA
|
||||
|
||||
REA turns a recorded web app session into a runnable environment for training
|
||||
and testing AI agents. An agent can practice tasks such as finding an issue,
|
||||
creating a folder, or editing a record without running the original app.
|
||||
|
||||
REA builds the tools the agent calls, the local data those tools use, and a
|
||||
test that checks the environment starts and responds. It supports changes
|
||||
when the recording confirms their effects. You can then use the environment
|
||||
with your RL framework to set task goals, check rewards, and run training.
|
||||
|
||||
## Before you start
|
||||
|
||||
Install [REA and its agent integration](getting-started/installation.md). To record an app,
|
||||
you also need a running test instance and a Chrome-family browser. Choose an
|
||||
instance where the agent is allowed to create, edit, and delete data. REA
|
||||
records the app you provide; you start it and prepare its test data.
|
||||
|
||||
To run the exported environment, install the
|
||||
[AgentEnv framework](https://github.com/scaleapi/agentenv-framework#install)
|
||||
and start Docker. REA's [compatibility check](development/testing.md#exported-environment-bundles)
|
||||
uses AgentEnv version `0.9.1431` with Python 3.12.
|
||||
|
||||
There are two ways to build an environment: ask your agent to record the app,
|
||||
or start from a recording you already have.
|
||||
|
||||
## 1. Record and build with your agent
|
||||
|
||||
Give your agent the app URL, the operations you need, your browser's executable
|
||||
path, and an output directory. For an app with folders, adapt this prompt:
|
||||
|
||||
```text
|
||||
Use REA to build an AgentEnv environment from my test web app at
|
||||
http://127.0.0.1:3000. Cover listing, creating, renaming, and deleting folders.
|
||||
You may create and remove test folders. Leave the existing data unchanged.
|
||||
|
||||
Sign in as admin using the password in APP_PASSWORD. Use the browser at
|
||||
/absolute/path/to/chromium. Save the environment to
|
||||
/absolute/path/to/my-env, which does not exist yet.
|
||||
|
||||
Record the session, check which operations the environment supports, and
|
||||
export it. Report any unsupported operations and what is missing.
|
||||
```
|
||||
|
||||
Replace the URL, account, paths, and folder operations with your own. Set
|
||||
`APP_PASSWORD` in the environment that starts REA's MCP server so REA can use
|
||||
it without putting the password in the prompt.
|
||||
|
||||
Your agent inspects the app, records a session, checks the result, and exports
|
||||
a folder containing the runnable environment. It reports which operations
|
||||
work and which need more evidence. The app is used during recording; the
|
||||
exported environment runs from the saved data.
|
||||
|
||||
After your agent exports the folder, skip to
|
||||
[run the environment](#run-the-environment). The next section is an
|
||||
alternative for an existing recording.
|
||||
|
||||
## 2. Or build from a saved recording
|
||||
|
||||
A HAR file records a browser's network requests and responses. Export one
|
||||
with API response bodies from your browser's network panel, or use a
|
||||
recording you already have. To include create, edit, and delete operations,
|
||||
record those actions and the reads that show their results.
|
||||
|
||||
Build the environment into a new or empty directory:
|
||||
|
||||
```sh
|
||||
rea environment-build ./session.har ./my-env --name my-env --json
|
||||
```
|
||||
|
||||
You can also pass capture Evidence JSON saved by REA's
|
||||
`capture-browser-scenario` or `inspect-web-network-capture` commands. Here,
|
||||
Evidence means REA's saved recording with its source information. The build
|
||||
accepts HAR files and those JSON recordings. It reads local data and writes
|
||||
the environment folder without contacting the app.
|
||||
|
||||
Add `--dry-run` to review the build report without writing the folder. For an
|
||||
unreleased REA checkout, use `--package-tarball` with its packed package so
|
||||
the exported environment runs the same implementation.
|
||||
|
||||
The build report tells you what the environment supports:
|
||||
|
||||
| Report field | What to check |
|
||||
| ----------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `state` | Whether the environment can answer from local state or uses recorded-response replay. |
|
||||
| `coverage.reads` | Which read operations use state and which replay recorded answers. |
|
||||
| `coverage.writes` | Which create, edit, and delete operations are supported. Each withheld operation includes a reason and a recording hint. |
|
||||
| `warnings` | Recorded content or local paths to review before sharing the folder. |
|
||||
|
||||
A successful build means the folder was created. Check the report to confirm
|
||||
it supports the operations your tasks need. `coverage: null` means the build
|
||||
could not prepare state for this environment and it will use replay.
|
||||
|
||||
<a id="run-the-environment"></a>
|
||||
|
||||
## 3. Run the environment
|
||||
|
||||
Whether your agent exported the folder or you built it from a HAR, run it
|
||||
with AgentEnv:
|
||||
|
||||
```sh
|
||||
agent-env run ./my-env
|
||||
```
|
||||
|
||||
Use the folder path your agent reported if it differs. The first run builds
|
||||
the container image, so it needs access to the image and package sources.
|
||||
|
||||
REA includes a basic test, called the smoke task. It checks the expected tool
|
||||
count and, when a suitable read tool exists, calls it and checks its response
|
||||
status. Passing this test shows the environment starts and responds in the
|
||||
tested setup. It does not yet score an agent on an application task.
|
||||
|
||||
The folder contains the recording, a tool manifest describing the available
|
||||
operations, a Dockerfile, environment configuration, and the test. The
|
||||
[export reference](agent-environments.md#export-a-bundle) describes each file.
|
||||
|
||||
## 4. Add tasks and reward checks
|
||||
|
||||
Start with one task the exported tools support. Give it a clear goal and a
|
||||
checkable result. For example, if folder creation is supported, ask your
|
||||
agent:
|
||||
|
||||
```text
|
||||
Use this exported environment to add an AgentEnv task that asks an agent
|
||||
to create a folder named Practice. Check the saved environment state to
|
||||
confirm the folder exists with that name, and use that check to score the
|
||||
result. Reset the environment before each run so attempts start with the
|
||||
same data. Use the AgentEnv task and verifier formats.
|
||||
```
|
||||
|
||||
This asks your agent to help build the task and its grading check around the
|
||||
environment REA generated. A retrieval task might check an answer against a
|
||||
recorded value. A task that changes data should check the resulting state.
|
||||
When state is available, the environment supports reset, load, and read-back
|
||||
operations for repeatable runs.
|
||||
|
||||
Use the [AgentEnv documentation](https://www.agentenvframework.com/docs) for
|
||||
task configuration, agent connections, scoring, and execution. Connect the
|
||||
resulting tasks to your chosen RL training workflow.
|
||||
|
||||
## Improve an operation that is missing
|
||||
|
||||
For REA to support a change, the recording needs to show what happened before
|
||||
and after it. When recording against your test app:
|
||||
|
||||
1. Read the list and an existing item before making changes. Repeat the
|
||||
initial reads to show which values change independently.
|
||||
2. Create a test item, read it back, and read the list before the next change.
|
||||
Repeat with a second item and different text.
|
||||
3. Edit each test item, then read the item and list again. Leave more than a
|
||||
second between creation and the first edit to show time-dependent changes.
|
||||
4. Delete the test items, then read them back and read the list to confirm
|
||||
their removal.
|
||||
|
||||
Also record filters, searches, sorting, and paging if your tasks need them.
|
||||
The agent should check that every recording step completed before using it.
|
||||
The [recording workflow](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/environments.md)
|
||||
has the detailed browser instructions, including timeouts and handling IDs
|
||||
assigned by the app.
|
||||
|
||||
Use the report's hints to decide what to record next:
|
||||
|
||||
| Reason | Next step |
|
||||
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `unconfirmed` | Read the item and its list after the change. |
|
||||
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
|
||||
| `no-observation` | Read back a created item. For an edit or delete, also read it before changing it. |
|
||||
| `contradicted` or `side-effect` | Inspect the reported disagreement. The operation may need behavior REA cannot model yet. |
|
||||
|
||||
See the [full reason table](agent-environments.md#build-a-bundle-in-one-step)
|
||||
for the other cases. Rebuild from the new recording and check the report
|
||||
again. A new recording replaces the build input; combining separate recording
|
||||
sessions is not supported.
|
||||
|
||||
## Compare with the original app
|
||||
|
||||
For a stronger check, record a separate sequence of calls against the real
|
||||
app, starting from the same test data as the original recording. This is a
|
||||
probe. Restore that starting data before each probe sequence, then compare
|
||||
the saved probe recordings with the environment:
|
||||
|
||||
```sh
|
||||
rea environment-compare ./my-env/envs/my-env/manifest.json \
|
||||
--evidence ./my-env/envs/my-env/evidence.json \
|
||||
--probe ./probe-create-edit-delete.json --probe ./probe-reads.json --json
|
||||
```
|
||||
|
||||
Each probe file must be capture Evidence JSON from REA. The comparison runs
|
||||
offline and reports differences in responses, unsupported calls, and missing
|
||||
observations. It accounts for supported differences in IDs, times, and
|
||||
modeled generated values. Agreement applies to the calls those probes check.
|
||||
See the [comparison reference](agent-environments.md#measure-fidelity-against-the-application)
|
||||
for the exact rules.
|
||||
|
||||
## Before sharing the environment
|
||||
|
||||
The exported folder includes retained response bodies and local source paths.
|
||||
Standard authentication headers are excluded, but custom headers, query
|
||||
values, and body tokens can remain. Review the content you plan to share.
|
||||
Credential-name warnings point to values worth reviewing; they do not remove
|
||||
those values or establish that the recording contains no credentials.
|
||||
|
||||
For direct MCP serving, replay behavior, state controls, and resource limits,
|
||||
see the [environment reference](agent-environments.md). The
|
||||
[implementation index](specs/agent-environments/README.md) links the contributing
|
||||
PRs and their verification records.
|
||||
+60
-433
@@ -1,27 +1,16 @@
|
||||
# Agent environments from recorded captures
|
||||
|
||||
For a first environment, start with the
|
||||
[quickstart](agent-environments-quickstart.md): record an app or use a saved
|
||||
session, build and run the environment, and add tasks and reward checks.
|
||||
The [implementation index](specs/agent-environments/README.md) links
|
||||
the contributing PRs and their verification records. This page is the detailed
|
||||
command and behavior reference.
|
||||
|
||||
Use `rea environment-build` to turn one recorded capture into an agent
|
||||
environment bundle in one offline step: a folder `agent-env run` builds and
|
||||
runs, whose MCP server answers from what the capture observed. See
|
||||
[Build a bundle in one step](#build-a-bundle-in-one-step).
|
||||
|
||||
The steps it composes are separate commands too. `rea environment-serve` serves
|
||||
an environment manifest as an agent environment: an MCP server whose tools
|
||||
answer from what one retained web network capture observed, plus the
|
||||
environment card the agentenv framework reads to discover it. By default the
|
||||
server derives the [state the capture observed](#the-state-a-capture-observed)
|
||||
at startup and answers read tools from it, and the write tools whose effect the
|
||||
capture confirmed by changing it; `--replay-only` replays the capture's records
|
||||
Use `rea environment-serve` to serve an environment manifest as an agent
|
||||
environment: an MCP server whose tools answer from what one retained web
|
||||
network capture observed, plus the environment card the agentenv framework
|
||||
reads to discover it. By default the server derives the
|
||||
[state the capture observed](#the-state-a-capture-observed) at startup and
|
||||
answers read tools from it, and the write tools whose effect the capture
|
||||
confirmed by changing it; `--replay-only` replays the capture's records
|
||||
instead. Nothing is fetched from the recorded application; every answer comes
|
||||
from retained Evidence. `rea environment-export` writes the same pair as a
|
||||
bundle folder; see [Export a bundle](#export-a-bundle).
|
||||
bundle folder `agent-env run` builds and runs; see
|
||||
[Export a bundle](#export-a-bundle).
|
||||
|
||||
```sh
|
||||
rea inspect-web-network-capture /absolute/session.har har --format json > capture.json
|
||||
@@ -30,7 +19,7 @@ rea environment-serve manifest.json --evidence capture.json --port 8765
|
||||
```
|
||||
|
||||
The first command inspects the capture; the second projects the
|
||||
[environment manifest](guides/web/web-network-captures.md#environment-manifest) view from
|
||||
[environment manifest](web-network-captures.md#environment-manifest) view from
|
||||
it, where `view.json` wraps the capture Evidence as the inline source:
|
||||
|
||||
```json
|
||||
@@ -47,198 +36,11 @@ The third command serves that manifest from the state it derives from the
|
||||
capture; see [Serve from the state](#serve-from-the-state). Both served files
|
||||
are complete REA Evidence JSON as the commands write it with `--format json`;
|
||||
the `evidence` member of `view.json` is the whole of `capture.json`.
|
||||
A `view.json` can also name the capture file as
|
||||
`{"kind": "evidence-file", "path": "capture.json"}`, which projects it as
|
||||
retained Evidence instead of charging the whole capture as inline input.
|
||||
|
||||
## Build a bundle in one step
|
||||
|
||||
```sh
|
||||
rea environment-build grafana.har ./grafana --json
|
||||
agent-env run ./grafana
|
||||
```
|
||||
|
||||
`rea environment-build <capture> [directory]` takes a `.har` file, or capture
|
||||
Evidence JSON from `inspect-web-network-capture` (HAR) or
|
||||
`capture-browser-scenario`, and writes the bundle
|
||||
[Export a bundle](#export-a-bundle) describes. It inspects a HAR as
|
||||
`inspect-web-network-capture` does (Evidence input skips this), projects the
|
||||
environment manifest from that Evidence as retained Evidence, authenticates
|
||||
the pair, prepares the state the bundle's server derives at startup, and
|
||||
writes the bundle with the export. For the same capture path, every file it
|
||||
writes is byte-identical to what `inspect-web-network-capture`,
|
||||
`inspect-analysis-view` and `environment-export` write, Evidence ids included,
|
||||
when `view.json` lists the same rules in the views' order and the export is
|
||||
given the build's name. It reads one file and writes one folder: it fetches
|
||||
nothing and launches no browser (HAR inspection runs REA's own decoder
|
||||
process), so record a browser scenario with `capture-browser-scenario` first
|
||||
and pass its Evidence. Other capture formats are refused, as the manifest view
|
||||
refuses them, and a HAR document saved as `.json` is refused with a hint to
|
||||
name it `.har`.
|
||||
|
||||
- `--template-rule <rule>`, repeated, replaces the default opt-in rules
|
||||
`after-uid-literal`, `opaque-token`, `sibling-variance` and
|
||||
`prefix-variance`, the rules the published capture-built environments use;
|
||||
`--format-rules-only` applies none, the views' own default. Rules apply in
|
||||
the views' fixed order, the report lists them, and a repeated rule is
|
||||
refused as the views refuse it.
|
||||
- `--name` names the env folder and env id. Without it the name is the
|
||||
manifest's, unless the capture's origin is a loopback name or an IP literal:
|
||||
the capture file then names it (`grafana.har` and `grafana.capture.json`
|
||||
become `grafana`) and a `name-from-file` warning says so. A file name that
|
||||
yields no portable label (kebab-case, at most 63 characters for a DNS service
|
||||
label, not a Windows device name or a label interpreted as an IP address) is
|
||||
refused; pass `--name`.
|
||||
- `--package-tarball` works as for the export, and `--capture-timeout-ms`
|
||||
as for `inspect-web-network-capture`.
|
||||
- `--dry-run` runs every stage and prints the report with `status: "planned"`
|
||||
and the files a build would write, but writes nothing and does not check the
|
||||
directory, which may then be omitted. It reads and hashes a
|
||||
`--package-tarball` as a build does before copying it, so it refuses the
|
||||
tarballs a build refuses, including one whose contents cannot be read.
|
||||
|
||||
The directory must be absent or empty, as for the export; the build never
|
||||
replaces a folder. A refusal before the write leaves nothing behind. A capture
|
||||
Evidence file, an inspected HAR's Evidence and the projected manifest must each
|
||||
fit the live heap headroom for parsing and sealing, checked before the file is
|
||||
read or the Evidence serialized; one that does not is refused as
|
||||
`resource_constraint`. Each
|
||||
refusal names its stage: `environment-build` for the build's own checks,
|
||||
`inspect_web_network_capture` and `inspect_analysis_view` for inspection and
|
||||
projection, `environment-export` for the bundle. SIGINT cancels the build
|
||||
(`cancelled`); before the bundle stage nothing is written, and a cancellation
|
||||
after publication preserves the bundle receipt. Before publication, a failed
|
||||
write attempts to remove partial output and reports cleanup failures. See
|
||||
[bundle publication and recovery](getting-started/cli.md#export-an-environment-bundle).
|
||||
Stage progress is logged to stderr
|
||||
at info level; the report is the only output on stdout:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "built",
|
||||
"directory": "/work/grafana",
|
||||
"name": "grafana",
|
||||
"capture": {
|
||||
"evidence_id": "ev_…",
|
||||
"operation": "inspect_web_network_capture",
|
||||
"records": 105,
|
||||
"input": "har"
|
||||
},
|
||||
"manifest": {
|
||||
"evidence_id": "ev_…",
|
||||
"tools": 48,
|
||||
"excluded_groups": 0,
|
||||
"template_rules": [
|
||||
"after-uid-literal",
|
||||
"opaque-token",
|
||||
"sibling-variance",
|
||||
"prefix-variance"
|
||||
]
|
||||
},
|
||||
"state": { "served": true, "evidence_id": "ev_…" },
|
||||
"coverage": {
|
||||
"reads": {
|
||||
"state": 36,
|
||||
"recorded": 6,
|
||||
"recorded_reasons": { "query-varies": 3, "…": 1 }
|
||||
},
|
||||
"writes": {
|
||||
"served": [{ "tool": "delete_folders_by_uid", "effect": "delete" }],
|
||||
"withheld": [
|
||||
{
|
||||
"tool": "post_folders",
|
||||
"effect": "create",
|
||||
"reason": "unconfirmed",
|
||||
"hint": "No later read showed the effect. Read the entity and its list after the write.",
|
||||
"exemplar_ordinal": 94,
|
||||
"contradicted_by": null
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"tool_count": 37,
|
||||
"package_source": { "kind": "registry", "specifier": "rea-agents@…" },
|
||||
"files": [{ "path": "README.md", "bytes": 4210 }],
|
||||
"warnings": [{ "code": "name-from-file", "message": "…" }],
|
||||
"next": ["agent-env run /work/grafana"]
|
||||
}
|
||||
```
|
||||
|
||||
`coverage` is the state's read views and write plans summarized, and `null`
|
||||
when the state is unavailable and the bundle serves replay. The
|
||||
[environment coverage](guides/web/web-network-captures.md#environment-coverage) view
|
||||
reports the same summary for any capture without building, so an agent can
|
||||
check it after each recording. The view summarizes the state as derived; it
|
||||
does not apply the serve-time admission checks, so for an environment whose
|
||||
served catalog exceeds the discovery budget, or whose state exceeds what
|
||||
`data/add` loads, it still reports the state while the build reports `coverage:
|
||||
null` and the bundle serves replay. Each withheld write carries a hint about
|
||||
what to record next; hints never change what is served:
|
||||
|
||||
| Reason | Hint |
|
||||
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `no-state-target` | The write could not be linked to a modelled target. Check the path template and recorded views; record missing reads. Some route forms may not be supported. |
|
||||
| `no-call` | No recorded call of this tool was modelled, for example because every call failed. Record a successful call. |
|
||||
| `body-unusable` | The retained request or response cannot be modelled. Record a successful call with a complete supported request and reproducible response. |
|
||||
| `no-observation` | Nothing showed the entity or document this write changed. Read a created entity back by id right after the write; for an update or delete the state must already hold the entity, so read it before the write or record the create that made it. |
|
||||
| `unexplained-change` | The write changed fields it did not send, in a way no rule explains. A second, different call of the same tool can show the pattern. |
|
||||
| `contradicted` | A later read disagreed with the modelled effect. Check the read at contradicted_by. |
|
||||
| `side-effect` | The write changed something besides its own entity, which the environment does not model. |
|
||||
| `unconfirmed` | No later read showed the effect. Read the entity and its list after the write. |
|
||||
| `incomplete-entities` | The write is confirmed, but the views that show its entity cannot render it in full. Read the written entity through every view that lists it. |
|
||||
| `unknown-clock` | The learned rules generate timestamps, but the capture establishes no global served clock start. Record usable start times and completed durations for every request. |
|
||||
|
||||
The bundle ships `evidence.json`, the whole capture Evidence, response bodies
|
||||
included. Inspection excludes the values of the standard authentication headers
|
||||
(`Authorization`, `Cookie` and `Set-Cookie` among them) and URL userinfo; a
|
||||
custom key header such as `X-Api-Key`, a token in the query string and tokens
|
||||
an application returns in bodies remain. The environment views refuse captures
|
||||
inspected with `--sensitive-value`, so scrub a capture before building it. The
|
||||
report's `credential-named-members` warning counts recorded header names, query
|
||||
parameter names and JSON or form body members whose names look like
|
||||
credentials (whole name segments such as `token`, `secret`, `password`,
|
||||
`api_key`, `authorization` or `session`) and hold a value, and lists the first
|
||||
20 by record and pointer, never by value. A body is read as a form or as JSON by
|
||||
its recorded media type, or by its shape when it has none or one naming
|
||||
neither. A browser scenario is
|
||||
scanned by event (`event 4 response /token` names
|
||||
`normalized_result.events.items[4]`): every retained request and response
|
||||
content event, including a response whose request began before the browser was
|
||||
attached and a transaction no record represents, and each transaction URL's
|
||||
query once. Bodies larger than 4 MiB of UTF-8 text are not
|
||||
scanned; `credential-scan-incomplete` counts them and lists up to 20 record or
|
||||
event pointers, including base64 bodies whose decoded scan was cut. An empty name
|
||||
warning does not establish that these bodies were checked. A browser scenario's
|
||||
saved values (`save` and `save_response`) named like credentials but not marked
|
||||
`secret: true` are recorded unredacted; `credential-named-saved-values` lists up
|
||||
to 20 of them by parameter pointer. Names do not establish sensitivity,
|
||||
so the build never refuses on them.
|
||||
|
||||
Capture Evidence also records the capture file's absolute path (`subject.local_path`,
|
||||
`parameters.capture_path`, `locations`, `artifact.path`), and the bundle
|
||||
carries it: build from a neutral folder when that path should not ship.
|
||||
|
||||
Over MCP, `export_environment_bundle` builds the same bundle and returns the
|
||||
same report. Its `source` is capture Evidence the session retained
|
||||
(`{"kind": "retained-evidence", "evidence_id"}`), such as a
|
||||
`capture_browser_scenario` or `inspect_web_network_capture` result, or a
|
||||
capture Evidence file (`{"kind": "evidence-file", "path"}`); inspect a HAR with
|
||||
`inspect_web_network_capture` first. Retained Evidence is written into the
|
||||
bundle as `--format json` writes it, so the bundle is the one
|
||||
`environment-build` writes from that file. `output_directory` must be an
|
||||
absolute path that does not exist yet, whose parent does; the tool creates it
|
||||
and refuses an existing directory, empty or not. `name`, `template_rules`,
|
||||
`package_tarball` and `dry_run` work as `--name`, `--template-rule`,
|
||||
`--package-tarball` and `--dry-run` do, and retained Evidence has no file to
|
||||
name the environment after, so a loopback or IP origin needs `name`. Unlike
|
||||
`--dry-run`, `dry_run` still checks `output_directory`. Destination refusals
|
||||
name `export_environment_bundle`; the others name their stage as
|
||||
`environment-build`'s do.
|
||||
|
||||
## From a browser scenario
|
||||
|
||||
The capture can also come from REA itself: a
|
||||
[browser scenario](guides/web/browser-scenario-contract.md) records a scripted session
|
||||
[browser scenario](browser-scenario-contract.md) records a scripted session
|
||||
in a browser REA launches, and its Evidence is a manifest parent like a HAR
|
||||
inspection. Select the `network` event family and response bodies. A replayed
|
||||
reply carries up to 256 KiB of a recorded body, but the scenario keeps at most
|
||||
@@ -323,54 +125,6 @@ the replayed body is the bytes the browser exposes for each response, with any
|
||||
declared secret replaced by its redaction marker (a body that redaction had to
|
||||
empty replays no body).
|
||||
|
||||
To record writes, sign in through the UI and then send the application's API
|
||||
calls with [request steps](guides/web/browser-scenario-contract.md#request-steps): each
|
||||
sends one request to the start origin through the page's own `fetch`, with the
|
||||
session's cookies, and a header can carry the CSRF cookie or stored token the
|
||||
application's own requests send. A write is served only when the capture shows
|
||||
its effect, so read each collection before writing, read every written entity
|
||||
back by id and in its list right after the write, and leave more than a second
|
||||
between a create and its first edit. A request step's path is fixed when the
|
||||
scenario is written and cannot use a value an earlier response returned, so an
|
||||
id must be known before the run: one the caller chooses, such as a `uid` in the
|
||||
create body, or one an earlier scenario created. When the server assigns ids,
|
||||
read the list after each create (the create's response or the list shows the
|
||||
new entity). A second scenario can record updates and deletes using those
|
||||
known ids, but produces a separate environment: recordings are not combined.
|
||||
That scenario must include its own before/after reads and every action the
|
||||
environment should support; it does not inherit the first scenario's create
|
||||
evidence. To capture create and subsequent edits together, make them through
|
||||
the UI in one scenario. The environment coverage view names each
|
||||
write it withholds and what to record next. A scenario with write requests
|
||||
changes the application's data: record against an instance whose data may
|
||||
change.
|
||||
|
||||
```json
|
||||
{
|
||||
"step_id": "create-folder-1",
|
||||
"action": "request",
|
||||
"method": "POST",
|
||||
"path": "/api/folders",
|
||||
"headers": [
|
||||
{
|
||||
"name": "X-CSRFToken",
|
||||
"value": { "source": "cookie", "name": "csrftoken" }
|
||||
}
|
||||
],
|
||||
"json": { "title": "Scratch folder 1" },
|
||||
"expected_status": [200, 201]
|
||||
}
|
||||
```
|
||||
|
||||
Each record takes its start and duration from the browser's timing (the time
|
||||
the scenario observed the request where the browser reported no start), so the
|
||||
state orders and overlaps
|
||||
records as it does a HAR's and learns a write clock: an update the application
|
||||
stamps with a modification time can be served. A scenario captured before
|
||||
these times were recorded has none; its records keep request order and the
|
||||
state learns no clock, so such updates stay `unexplained-change` until it is
|
||||
recorded again.
|
||||
|
||||
## What a manifest is
|
||||
|
||||
An environment manifest is a projection of one application's recorded request
|
||||
@@ -412,8 +166,7 @@ returns that record's recorded response. Selection compares the supplied path
|
||||
parameters against the record's path segments at the parameter indexes, then
|
||||
the supplied query values against the record's query string, and keeps the
|
||||
record with the most matches; ties keep manifest order. A query argument accepts
|
||||
one string, a number or boolean sent as its JSON text, or an array of strings for
|
||||
repeated values; the array is matched
|
||||
one string or an array of strings for repeated values; the array is matched
|
||||
against every occurrence in captured order. The raw recorded query remains the
|
||||
replay evidence. The result reports
|
||||
where the answer came from:
|
||||
@@ -462,7 +215,11 @@ text with `truncated: true`. `body_recorded` is false when the capture kept the
|
||||
response's status and content type but no body; `body` is then `null`, which
|
||||
is not a recorded JSON `null`. `body_encoding` is `base64` when the capture
|
||||
retained response bytes that are not valid UTF-8; `body` is then exactly those
|
||||
bytes in base64, never a lossy decoding. A retained base64 body is decoded only as far as the 256 KiB a reply can carry, and a leading byte-order mark is kept. For a `base64` body the limit applies to the recorded bytes, not their longer base64 spelling, and a cut keeps whole base64 groups. Request bodies passed to `POST`, `PUT` and `PATCH`
|
||||
bytes in base64, never a lossy decoding. An unsupported declared HAR encoding
|
||||
returns `body: null`, `body_recorded: true` and `body_encoding: "unsupported"`;
|
||||
the original encoding and encoded text remain in the cited capture Evidence.
|
||||
Such bodies supply no decoded JSON facts to state extraction.
|
||||
A retained base64 body is decoded only as far as the 256 KiB a reply can carry, and a leading byte-order mark is kept. For a `base64` body the limit applies to the recorded bytes, not their longer base64 spelling, and a cut keeps whole base64 groups. Request bodies passed to `POST`, `PUT` and `PATCH`
|
||||
tools are accepted and ignored: nothing is written, and no call changes what a
|
||||
later call returns. Arguments the tool's input schema does not name are
|
||||
refused by the MCP server as invalid arguments, so a misspelled parameter never
|
||||
@@ -487,7 +244,7 @@ captured text without changing the tool names or schema fields clients use.
|
||||
|
||||
Replay answers each call from one recording, so the answers reflect whichever
|
||||
moment of the capture recorded them. The
|
||||
[environment state](guides/web/web-network-captures.md#environment-state) view extracts
|
||||
[environment state](web-network-captures.md#environment-state) view extracts
|
||||
instead the data the capture's reads observed before the first related write:
|
||||
entity types with an identity field, entities with their recorded values,
|
||||
documents for reads that are not entities, and a render plan per read tool,
|
||||
@@ -509,14 +266,14 @@ see [Serve from the state](#serve-from-the-state).
|
||||
|
||||
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the
|
||||
environment, then to `127.0.0.1` and an ephemeral port, exactly as
|
||||
[`rea mcp --http`](getting-started/cli.md#serve-mcp-over-streamable-http) resolves them. The
|
||||
[`rea mcp --http`](cli.md#serve-mcp-over-streamable-http) resolves them. The
|
||||
bound MCP URL is written to stderr as one JSON log line
|
||||
(`"msg":"MCP transport listening"` with `url`); `REA_LOG_LEVEL` overrides that
|
||||
log level. `ENVIRONMENT_NAME` renames the served environment; otherwise the
|
||||
manifest's `name` is used, and a set but blank value is refused as
|
||||
`configuration_invalid` before binding rather than silently falling back. The
|
||||
listener serves one MCP client at a time with the same session, `Host` header
|
||||
and shutdown contract as [transports](reference/mcp-contracts.md#transports) describes.
|
||||
and shutdown contract as [transports](mcp-contracts.md#transports) describes.
|
||||
It runs until SIGINT or SIGTERM, then closes the listener, prints one
|
||||
`{"status":"stopped", …}` document whose `state` member says whether it served
|
||||
from the state, and exits `130` or `143`.
|
||||
@@ -591,7 +348,7 @@ names the method and path template, then what the tool returns, one level deep:
|
||||
`returns a JSON object with fields …`, `returns a JSON array of objects with
|
||||
fields …` or `returns a JSON object whose <member> lists objects with fields …`,
|
||||
where a list is an array of objects found by the
|
||||
[environment state](guides/web/web-network-captures.md#environment-state)'s list rule (the
|
||||
[environment state](web-network-captures.md#environment-state)'s list rule (the
|
||||
root array, or the one array of objects an envelope carries). Field and member
|
||||
names are written as JSON strings, so a captured name cannot read as more of
|
||||
the description; a member name too long to show whole is a quoted prefix marked
|
||||
@@ -620,9 +377,8 @@ shape and carries no annotations.
|
||||
Path parameters are required strings whose
|
||||
description names the segment index, the templating rule and the observed
|
||||
values (sampled within the list budget above; the manifest keeps them all);
|
||||
query parameter names accept an optional string, number or boolean (a number or
|
||||
boolean is sent as its JSON text) or array of strings (for repeated values in
|
||||
captured order); write methods accept an
|
||||
query parameter names accept an optional string or array of strings (for
|
||||
repeated values in captured order); write methods accept an
|
||||
optional `body` object described by the recorded top-level key types. Each
|
||||
input keeps its own argument: a query name equal to a path parameter's name is
|
||||
advertised as `query_<name>`, a query name longer than 64 characters as
|
||||
@@ -647,7 +403,7 @@ rea environment-serve manifest.json --evidence capture.json
|
||||
```
|
||||
|
||||
Before it binds, the server derives the
|
||||
[environment state](guides/web/web-network-captures.md#environment-state) the capture
|
||||
[environment state](web-network-captures.md#environment-state) the capture
|
||||
observed: it runs the `environment-state` view on the capture in-process with
|
||||
the manifest's `template_rules` in the same order, exactly as
|
||||
`inspect-analysis-view` projects it from the same capture, so the state
|
||||
@@ -679,58 +435,33 @@ member reports which applied:
|
||||
|
||||
Each read tool answers according to its view in the state:
|
||||
|
||||
| View | Answer |
|
||||
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`, whose `per_response_members` (such as a request id), like a list's or a document's, keep their recorded values. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`, except that a call matching one of the view's `projected_query_keys`, or on a view with `projected_record_ordinals` asking for some fields only with a value other than `*` no checked line gave or one a replayed read gave, replays its recording. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error, unless a read showed the application leaves that field out; a body is never invented. |
|
||||
| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"` or a line `all_record_ordinals` names, or the recorded members still held. A call with no query argument answers from such a line of the collection its path arguments name that only pages its list, with `replay.exact_match` false. On a line the view's `query` answers, and for a call whose every query argument has a role there (an argument-less call aside), the answer is the query evaluated over the state: filtered, searched, sorted and paged, with counters at the number listed and `query.totals` at the number matched, and `state.membership: "query"`; such a call answers on a query or whole line of its collection with `replay.exact_match` false. A value an `inert` parameter never recorded, a parameter without a role, a call matching one of the view's `projected_query_keys`, or a field selection the checked lines do not show (as for `item`), replays its recording. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
|
||||
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
|
||||
| `recorded` | The recording, as replay answers it. |
|
||||
| View | Answer |
|
||||
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error, unless a read showed the application leaves that field out; a body is never invented. |
|
||||
| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"` or a line `all_record_ordinals` names, or the recorded members still held. A call with no query argument answers from such a line of the collection its path arguments name that only pages its list, with `replay.exact_match` false. On a line the view's `query` answers, and for a call whose every query argument has a role there (an argument-less call aside), the answer is the query evaluated over the state: filtered, searched, sorted and paged, with counters at the number listed and `query.totals` at the number matched, and `state.membership: "query"`; such a call answers on a query or whole line of its collection with `replay.exact_match` false. A value an `inert` parameter never recorded, or a parameter without a role, replays its recording. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
|
||||
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
|
||||
| `recorded` | The recording, as replay answers it. |
|
||||
|
||||
A read with repeated query values uses replay, because the state query model
|
||||
does not infer repeated-parameter semantics. A repeated-query write is refused
|
||||
before it can change state for the same reason.
|
||||
|
||||
List counters retain their recorded number or digit-string representation.
|
||||
Changing a digit-string counter preserves its observed zero-padding width;
|
||||
query-total inference remains limited to canonical decimal counts. Path matching
|
||||
uses the same segment parser as extraction, preserving empty internal segments.
|
||||
|
||||
An item or list view that always spells a clock field otherwise than the
|
||||
entity's stored value names that spelling in `clock_formats`, and a value it
|
||||
never recorded, such as a created entity's clock, is rendered in it: a list that
|
||||
writes `+00:00` where item reads write `Z` lists a new entity the same way.
|
||||
A view that names a field in `sorted_arrays` lists that string array sorted, as
|
||||
a read-back showed the application does, while the state keeps it in the order
|
||||
it was written, so a write's own answer echoes it. If a later counted read
|
||||
contradicts that ordering, the view stays recorded and its earlier reads no
|
||||
longer confirm served writes. A view that names a field in `nested_fields`
|
||||
always showed that field's objects with the same keys, in any order, and fewer
|
||||
than an entity stores, such
|
||||
as a list showing a parent without the parent's own nested type; a stored
|
||||
object it never recorded, such as a created entity's parent, is rendered with
|
||||
those keys. Members a view names in
|
||||
`per_response_members`, such as a request id each response mints, keep the
|
||||
value its recording carried. A list's `counters` and `query.totals` are
|
||||
top-level members or members one level down in a paging object
|
||||
(`/pageInfo/totalRows`); either is set to the number listed or matched, and
|
||||
members beside it such as `isLastPage` stay as recorded.
|
||||
|
||||
A list or document call that names no checked line, including a line recorded
|
||||
after a write or with other query values, answers from the recording. Every
|
||||
answer has the replay shape plus `state`: `{"source": "state", "view": …}`
|
||||
naming the entity, collection or document path, or `{"source": "recording",
|
||||
"reason": …}` with the view's reason, `no-view`, `unrecorded-request`, `page-exceeded` (a call naming a page of a whole list that the collection has outgrown), `projection` (a call to an item or list tool with replayed reads asking for a selection of fields the checked lines do not show, or leaving out a projection parameter every checked line names) or `variant` (a call matching another `projected_query_keys` key: a parameter, or a value of one, only lines showing the entities in another representation named, or leaving out a parameter every other line named). A `projection` or `variant` answer is a recording, so it does not show served writes. A
|
||||
"reason": …}` with the view's reason, `no-view`, `unrecorded-request`, `page-exceeded` (a call naming a page of a whole list that the collection has outgrown) or `projection` (a call to an item tool whose recordings asked for some fields only, naming such a parameter for an entity the state holds; the answer is a recording, so it does not show served writes). A
|
||||
state answer carries the base record's status, content type and `replay`
|
||||
provenance with `truncated: false`; one whose JSON would exceed 256 KiB is
|
||||
refused as a `resource_constraint` tool error rather than cut. Extraction does
|
||||
not state-back a document whose rendered JSON exceeds 256 KiB; that read stays
|
||||
recorded (`rendered-oversize`) and replays the captured body. Query `POST`s,
|
||||
and `POST` tools the capture shows only reading (a lookup answering as a `GET`
|
||||
did, an RPC tunnel naming read operations, a session renewal), stay served and
|
||||
answer from their recordings with the reason `query-body`; they are not write
|
||||
tools.
|
||||
`HEAD`, `OPTIONS` and `TRACE` tools also replay their captured responses; their
|
||||
state views report `method-replay` and coverage counts them as recorded reads.
|
||||
recorded (`rendered-oversize`) and replays the captured body. Query `POST`s
|
||||
stay served and answer from their recordings with the reason `query-body`.
|
||||
|
||||
A line establishes whole-collection membership only from the beginning:
|
||||
`offset`, `start`, `_start` and `page` must be zero when present. Positive or
|
||||
@@ -739,64 +470,35 @@ query model may describe supported paging; an empty later page cannot start
|
||||
returning earlier entities merely because it was empty in the capture.
|
||||
|
||||
`POST`, `PUT`, `PATCH` and `DELETE` tools are served only when the state's
|
||||
[write plan](guides/web/web-network-captures.md#environment-state) for them is served:
|
||||
[write plan](web-network-captures.md#environment-state) for them is served:
|
||||
the capture's later reads confirmed the effect the plan models. Other write
|
||||
tools are not served, so no call claims a change it does not make; each one's
|
||||
plan states why in its `reason`. A served write changes the live state, which
|
||||
every MCP session and `data/get` then see:
|
||||
|
||||
| Effect | Call |
|
||||
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
|
||||
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, nested clock, increment, slug, stamp, derived, composed, reference, unique and renewed changes the plan models. |
|
||||
| `update` (action) | A `POST`, `PUT` or `PATCH` to an item path plus an action segment (`/{type}/{id}/archive`) sets, on the entity before the segment, each scalar the recorded calls showed the action set (`set`), or flips each boolean they showed it flipping both ways (`toggle`), with any sent members; it is served only when a recorded call found what it sets already set, or flipped back what it toggles. |
|
||||
| `delete` | Removes the entity the path names. |
|
||||
| bulk `update` or `delete` | A `DELETE`, `PUT` or `PATCH` at a collection, or a `POST` to it plus `delete`, `remove` or `destroy`, applies its effect to every entity the body names, as recorded: an array of ids or of objects carrying the id field, at the root or as the body's one array member (`{"IDs": […], "Read": true}`). |
|
||||
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
|
||||
|
||||
An update's unsent changes follow the plan's rules. A `held-clock` is a clock
|
||||
the client sends back as it read it and the application stamps anew (Ghost's
|
||||
`updated_at`): the call must send the entity's value and the field takes the
|
||||
write's clock. A `nested-clock` sets the clocks inside an object the entity
|
||||
holds, in that object's recorded format. A `stamp` writes the acting user
|
||||
the entity already names under its `created…by` counterpart into a
|
||||
`modified…by` or `updated…by` field. A `derived` field takes a value showing
|
||||
the one sent member's text that the entity stores nowhere (a field Ghost
|
||||
derives from `html`), only with that member's recorded value. An action's
|
||||
`set` rule sets a scalar to the value the recorded calls left it at, and its
|
||||
`toggle` rule flips a boolean, refusing a call on an entity that holds none
|
||||
there (Miniflux's `PUT /v1/entries/{id}/bookmark` stars and unstars).
|
||||
| Effect | Call |
|
||||
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
|
||||
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, increment, slug, unique and renewed changes the plan models. |
|
||||
| `delete` | Removes the entity the path names. |
|
||||
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
|
||||
|
||||
Its answer has the status and content type of the plan's recorded response,
|
||||
with the written entity or its identity rendered in, no body, or the recorded
|
||||
body. A call whose body is not an object, sends a member the capture did not
|
||||
show stored, sends a fixed member with another value or a member of another
|
||||
JSON type, or lacks a required member is a tool error naming the accepted
|
||||
members; so is a call sending query parameters other than the exemplar's (they
|
||||
may change what the body does), a body not wrapped in the exemplar's envelope
|
||||
(Ghost's `{"posts": [{…}]}`), a bulk call naming an id the state does not hold,
|
||||
a `held-clock` member sent with another value than the entity holds, a call
|
||||
sending a `derived` rule's member to an entity that does not hold the value the
|
||||
recorded call found, a nested clock on an entity holding no such object, and a
|
||||
rendered answer missing a field the recorded response carried, and an update or delete of an entity the state does not hold answers
|
||||
members; so is a rendered answer missing a field the recorded response
|
||||
carried, and an update or delete of an entity the state does not hold answers
|
||||
the recorded 400, 404 or 410 of the tool or of an item view of the type, if
|
||||
any. A served delete of a type whose item view has a `deleted_flag` (the
|
||||
member, `deleted`, `isDeleted` or `is_deleted`, its reads set `true` on a
|
||||
record the application soft-deleted, as EspoCRM does) keeps the entity until
|
||||
`data/reset` or `data/add`, within the size `data/add` loads, so a read of it by
|
||||
id answers its last fields with that flag `true`. A refused call leaves the
|
||||
state unchanged. Values a write generates are
|
||||
any. A refused call leaves the state unchanged. Values a write generates are
|
||||
deterministic: the clock starts at `write_clock` and advances one second per
|
||||
served write, sequences take one past both the largest held value and the
|
||||
largest they generated, and UUIDs and tokens derive from the state Evidence
|
||||
id, tool, field and write count, skipping held values. If `write_clock` is null,
|
||||
plans needing `clock`, `held-clock`, or `nested-clock` retain their learned rules
|
||||
but are withheld with `unknown-clock`; other confirmed plans can still be served.
|
||||
`data/reset` and
|
||||
id, tool, field and write count, skipping held values. `data/reset` and
|
||||
`data/add` restart the clock, counter and sequences, so the same calls after a
|
||||
reset give the same state. Answers carry `state: {"source": "state", "view":
|
||||
"create" | "update" | "delete", …}` or `"document-write"`; a bulk answer adds
|
||||
`ids`, every entity it wrote in body order, with `id` naming the first.
|
||||
"create" | "update" | "delete", …}` or `"document-write"`.
|
||||
|
||||
The data plane serves three JSON-RPC methods:
|
||||
|
||||
@@ -896,9 +598,8 @@ names a `document` view once, at its initial document's `path`. The `version`,
|
||||
and the `state_evidence_id` and `types` that `data/get` reports, must match
|
||||
the served state, so a `data/get` result loads back unchanged. A number without
|
||||
an exact native value, or whose serialized spelling would not be read as exact
|
||||
(an integer from 2^53 below 1e21, such as `1e20`, in any spelling), is refused
|
||||
rather than altered. Caller data preserves all own JSON members, including
|
||||
`__proto__`; unknown members of the state document's structure are refused.
|
||||
(an integer from 2^53 below 1e21, such as `1e20`, in any spelling), or a
|
||||
`__proto__` member is refused rather than altered.
|
||||
An invalid document is `-32602` naming the first problem by JSON pointer, as in
|
||||
`/rea_state/entities/3/collection` (also in `error.data.path`); a problem
|
||||
inside caller data (`fields`, `value`, `types`) is located at that member, so
|
||||
@@ -952,25 +653,6 @@ after an update and referential rules on delete are not modelled. An exported
|
||||
bundle carries no state file: its server derives the state at startup, as the
|
||||
export does to predict what it serves.
|
||||
|
||||
Known limits of served writes:
|
||||
|
||||
- A recorded view answers from its recordings after a served write: Mailpit's
|
||||
`get_messages`, recorded as `list-differs`, still lists a message a served
|
||||
delete removed.
|
||||
- Served n8n workflow updates accept any `versionId`, while served Ghost post
|
||||
updates require `updated_at` to match the entity (a `held-clock`).
|
||||
- A slug derived from a title can repeat across served creates with the same
|
||||
title, where the application would make it unique.
|
||||
- A `composed` string copies sent text as sent: an application that escapes or
|
||||
renders it (HTML entities, Markdown) differs for text that needs it, as
|
||||
BookStack's `description_html` would for a description holding `&`.
|
||||
- A served create's sequence id is the state's next value, not the gaps an
|
||||
application's own sequence would leave.
|
||||
- Nested clocks are served in the spelling of the write's own response, which
|
||||
can differ from what a later read of the application shows.
|
||||
- A served write to an entity that a replayed (unserved) create made answers
|
||||
as for a missing entity, since the state never held it.
|
||||
|
||||
## Export a bundle
|
||||
|
||||
`rea environment-export` writes the same pair as a bundle folder the agentenv
|
||||
@@ -1007,8 +689,7 @@ serves replay. The complete tool catalog the bundle serves must also fit the
|
||||
server's advertised catalog budget before any files are written. Privately parsed JSON is sealed once so authentication
|
||||
and replay share immutable response trees rather than copying the capture repeatedly.
|
||||
`--name` replaces the manifest's portable kebab-case name as the env folder and env id
|
||||
(at most 63 characters for its DNS service label; Windows device names and
|
||||
labels interpreted as IP addresses are refused);
|
||||
(Windows device names such as `con` and `lpt1` are refused);
|
||||
`--package-tarball <path>` copies a packed `rea-agents` tarball into the env
|
||||
folder and makes the Dockerfile install it instead of the registry package and
|
||||
remove it in the same layer, which is how a checkout rather than a release is
|
||||
@@ -1026,12 +707,7 @@ Windows inherits ACLs and REA does not establish owner-only access there. Choose
|
||||
an already private Windows destination when captured content needs it.
|
||||
|
||||
The Dockerfile sets `MCP_HOST=0.0.0.0` and `MCP_PORT=18765`, the port the
|
||||
framework maps for an MCP server env, and `MCP_ALLOWED_HOSTS` naming the loopback
|
||||
addresses and the environment name, because the container binds outside loopback
|
||||
and the server refuses to serve on such a bind without an operator-declared
|
||||
allowlist: the framework's published port is reached over loopback and its
|
||||
gateway proxies to the env container by the compose service name (the
|
||||
environment name). The framework's own deploy supplies
|
||||
framework maps for an MCP server env; the framework's own deploy supplies
|
||||
`MCP_HOST`, `MCP_PORT` and `ENVIRONMENT_NAME`, so the served card carries the
|
||||
env's name. It installs Debian's `python3` because the framework's local
|
||||
sandbox probes a container's readiness with `python3` inside it, which the slim
|
||||
@@ -1055,9 +731,8 @@ unknown recorded status stays unknown; it is not borrowed from another
|
||||
record). A bundle with no such tool yields a verifier that checks the count
|
||||
only and says so. `tasks/smoke.json` deploys the env on the local
|
||||
sandbox and runs that verifier against the env's MCP endpoint; the first run
|
||||
also pulls the framework's published infrastructure images from `ghcr.io`,
|
||||
building one only if its pull fails, and the run tears everything down when it
|
||||
ends.
|
||||
also builds the framework's own infrastructure, and the run tears everything
|
||||
down when it ends.
|
||||
|
||||
The README written into the bundle records the capture artifact's SHA-256, the
|
||||
capture and manifest Evidence ids, the manifest view digest, the served tool
|
||||
@@ -1100,53 +775,10 @@ recording replayed in its place and never from a recording whose body the probe
|
||||
two collections can reuse a live id, and an equal pair stays in that collection. A live id in a later request's path, query or body is translated before the
|
||||
environment answers, a path parameter by the collection the live path names (a nested collection keeps its parent values, each decoded once, and a literal route segment stays), except a query value that pages a list, such as `limit`
|
||||
or `page`. Each call is checked against the served tool's input schema first, as the MCP server checks it. After mapping, two clock values compare by format only (ISO-8601 strings, and epoch numbers under
|
||||
`at`, `time`, `date`, `timestamp`, `created`, `updated` or `modified` field names, including snake-case and camel-case forms).
|
||||
Numeric arrays retain that containing field's clock context through nested
|
||||
arrays; an object's own fields establish their own context. Numeric values at
|
||||
the body root, in root arrays, or under measurement fields remain literal.
|
||||
|
||||
Generated values normalize only where this confirmed, served write's entity
|
||||
response rules identify generation. A UUID identity rule must still compare two
|
||||
UUIDs, and a `unique` rule two UUIDs or two generated integers (10 to 20
|
||||
digits) of one JSON kind and length; a `renewed` rule applies only at its
|
||||
recorded pointers,
|
||||
with `*` matching actual array items. Token/sequence strings retain the existing
|
||||
format check: equal length of at least eight, the same character classes, and
|
||||
a digit. Numeric sequences continue to compare literally unless an ID mapping
|
||||
applies. String shape and names such as `token` or `sku` alone do not establish
|
||||
generation. Read and replay responses without these facts stay literal except
|
||||
for supported clocks and learned IDs. A missing write target replays an absence
|
||||
response and does not inherit the successful entity response's generation rules.
|
||||
Admitted generated values must also preserve aliases and distinct identities
|
||||
within that response. UUID identity uses the generator's case-insensitive
|
||||
matching; token and sequence strings retain their literal identity. This
|
||||
correspondence ends with the answer and is not inferred across calls.
|
||||
A `composed` response field must spell the value its parts build from that
|
||||
response's own fields. Its parts then compare by their rendered text or their
|
||||
source fields' clock and ID rules. A coherent archive URL can therefore differ
|
||||
only in normalized clock digits without reporting a value difference; an
|
||||
incoherent URL still differs even when both sides repeat the same stale text.
|
||||
Normalized compositions count under `generated`; source fields are counted once.
|
||||
The report counts each kind of normalised
|
||||
value; the generation rules remain model-derived facts, rather than new
|
||||
observations of the application.
|
||||
`at`, `time`, `date`, `timestamp`, `created`, `updated` or `modified` field names, including snake-case and camel-case forms), and two different values each side generated on its own compare equal: both UUIDs, or one length of
|
||||
eight or more with the same character classes and a digit. The report counts each kind of normalised value.
|
||||
ISO clock offsets are accepted only within RFC 3339's `00`–`23` hour and `00`–`59` minute ranges.
|
||||
Probe ordering retains every recorded fractional-second digit in ISO timestamps
|
||||
with a four-digit year, `T`, and a `Z` or `±HH:MM` suffix. Equal instants use ordinal order. Other native-Date
|
||||
spellings keep their existing millisecond interpretation; an absent or unreadable
|
||||
start time makes the whole probe use record order. Original capture timestamps
|
||||
remain unchanged. Clock-format comparison and respelling recognize four-digit
|
||||
ISO years `0000`–`0099` as well as later years, with normal calendar validation.
|
||||
Respelling preserves the original value when the target zone would require a year
|
||||
outside `0000`–`9999`; generated writes refuse a clock outside that format.
|
||||
Capture-state start ordering uses the same exact recorded timestamp facts,
|
||||
including minute-only times and fractions longer than nine digits. Recorded
|
||||
spellings may use `T` or a space, `Z` or an extended/compact offset, or no zone
|
||||
(interpreted as UTC). Equal starts use ordinal order; a missing or unreadable
|
||||
start makes the capture use record order. Request interval and overlap checks
|
||||
still use millisecond coordinates, and generated-clock progression compares
|
||||
milliseconds. Derived ISO clock formats still require seconds and at most nine
|
||||
fractional digits.
|
||||
Timestamp ordering retains submillisecond precision when the source records it.
|
||||
|
||||
Comparison keeps response representations distinct: JSON values compare structurally, text compares as text, and a
|
||||
retained base64 body compares by bytes (including with UTF-8 text carrying those same bytes), never by its base64
|
||||
@@ -1156,12 +788,7 @@ global conflict involving an equal pair is not used to normalize either pair; th
|
||||
|
||||
A step agrees when the statuses and the normalised bodies are equal. The report gives agreement overall, per probe and
|
||||
per tool, `agreement_ignoring_order` for steps that differ only in item order, and each step's differences, at most 20,
|
||||
with a JSON pointer and both values cut short. Each step reports `differences_total` before that cap,
|
||||
`differences_omitted`, and `differences_truncated` (true when entries were omitted). Agreement and categories use the
|
||||
complete comparison. Omitted entries are not retained, so `--full-output` cannot recover them; inspect the original
|
||||
capture and probe Evidence to investigate beyond the returned pointers.
|
||||
|
||||
The difference categories are:
|
||||
with a JSON pointer and both values cut short:
|
||||
|
||||
| Category | Meaning |
|
||||
| --------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
@@ -1178,22 +805,22 @@ The difference categories are:
|
||||
| `unmapped-id` | The application answered with success and the environment did not, for a path id the comparison never mapped |
|
||||
|
||||
What a probe did not observe is not counted as the application's behaviour. A record without a usable request line,
|
||||
such as a redacted URL or a transaction the capture cannot attribute, is not replayed, nor is a request to an origin different
|
||||
from the application's (`other-origin`: the probe's origin with the most JSON-like records, as the manifest
|
||||
such as a redacted URL or a transaction the capture cannot attribute, is not replayed, nor is a request to another
|
||||
origin than the application's (`other-origin`: the probe's origin with the most JSON-like records, as the manifest
|
||||
chooses its own), and a request with no recorded
|
||||
answer is made, since it may have changed the application, but not compared: each probe lists them under `unobserved` with the reason, and the summary counts them. A live
|
||||
body the probe did not record compares by status only and is counted as `unobserved_bodies`, unless the answer never
|
||||
carries one (204, 205, 304, `HEAD`). A browser probe captured without `capture.network.response_body` is refused, as
|
||||
is one that makes POST, PUT or PATCH calls without `capture.network.request_body`.
|
||||
|
||||
Agreement covers only the calls the probes make, after the mapping the answers allow. Clocks compared by format
|
||||
and fields normalized under explicit generation rules can still differ in content, and an equal answer after a write shows the state matched for that
|
||||
Agreement covers only the calls the probes make, after the mapping the answers allow. Values compared by form, clocks
|
||||
and generated values, can still differ in content, and an equal answer after a write shows the state matched for that
|
||||
read, not that the write had no other effect.
|
||||
|
||||
## Compatibility verification
|
||||
|
||||
`npm run verify:agentenv:environment` is the hosted `real-agentenv` lane
|
||||
described in [testing](development/testing.md#exported-environment-bundles). It projects
|
||||
described in [testing](testing.md#exported-environment-bundles). It projects
|
||||
the Grafana-shaped source-owned capture through the built CLI, packs the
|
||||
checkout, exports a bundle with `--package-tarball`, runs it with the pinned
|
||||
`agentenv-framework` release on the runner's Docker and requires the smoke task
|
||||
|
||||
@@ -0,0 +1,410 @@
|
||||
# Static Android APK analysis
|
||||
|
||||
REA wraps an existing headless JADX engine through `src/android/`. Its source is
|
||||
kept as an unmodified, commit-pinned Git submodule; see
|
||||
[upstream provenance](https://github.com/morluto/rea/blob/main/third_party/README.md). CLI and MCP use the same
|
||||
application workflow and return artifact-bound Evidence inline.
|
||||
|
||||
This family is present on repository main and in npm 4.1.0. Check the
|
||||
[released package boundary](installation.md#released-package-and-main) and the
|
||||
connected server's tool list before selecting it.
|
||||
|
||||
On Linux, JADX launches an REA-owned process and requires a procps-compatible
|
||||
`ps` on REA's `PATH`; see the shared
|
||||
[Linux process prerequisite](installation.md#linux-owned-process-prerequisite).
|
||||
|
||||
## Optional package delivery
|
||||
|
||||
The MCP `inspect_android_package` request keeps complete output when `detail` is
|
||||
omitted or `"complete"`. Select `detail: "summary"` explicitly to receive all
|
||||
package, SDK and permission declarations, engine identity and class/resource
|
||||
counts without decoded manifest text or duplicate raw provider replies. A
|
||||
retaining session keeps the complete original Evidence and returns a distinct
|
||||
linked `inspect_analysis_view` summary. This changes delivery, not JADX analysis.
|
||||
|
||||
Use the returned `normalized_result.parent_evidence_id` with
|
||||
`inspect_analysis_view`, source kind `retained-evidence`, and
|
||||
`view: {kind: "android", facet: "manifest", offset: 0, limit: 4096}` to read the
|
||||
exact decoded manifest without reopening the APK. Offsets and limits count UTF-16
|
||||
code units; follow `coverage.next_offset`. The page preserves `source_status` and
|
||||
`reported_total_bytes`: exhausting retained text does not make a partial decoded
|
||||
manifest complete. Complete source Evidence remains exportable under the existing
|
||||
session lifetime. Servers without retention reject summary before starting JADX.
|
||||
One-shot CLI output remains complete. Older servers require the existing complete
|
||||
request after schema discovery. APK authenticity and runtime behavior remain
|
||||
unverified.
|
||||
|
||||
## Supply tools
|
||||
|
||||
Use a **full JDK 17 or newer** (including its compiler module) and
|
||||
[**jadx-headless-mcp 0.7.1**](https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar).
|
||||
Verify the downloaded JAR's SHA-256 against
|
||||
[REA's pinned release metadata](https://github.com/morluto/rea/blob/main/src/android/JadxRelease.ts)
|
||||
before using it. The following download and verification commands work for npm
|
||||
package consumers as well as repository checkouts; the repository-only
|
||||
`npm run fixtures:android` helper is not included in the npm package.
|
||||
|
||||
On macOS or Linux, choose a destination and verify the pinned digest:
|
||||
|
||||
```sh
|
||||
mkdir -p "$HOME/.local/share/rea/android" &&
|
||||
cd "$HOME/.local/share/rea/android" &&
|
||||
curl -fL -o jadx-headless-mcp-0.7.1-all.jar https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar &&
|
||||
if command -v sha256sum >/dev/null; then
|
||||
echo '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75 jadx-headless-mcp-0.7.1-all.jar' | sha256sum -c -
|
||||
else
|
||||
echo '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75 jadx-headless-mcp-0.7.1-all.jar' | shasum -a 256 -c -
|
||||
fi &&
|
||||
export REA_JADX_MCP_JAR="$PWD/jadx-headless-mcp-0.7.1-all.jar"
|
||||
```
|
||||
|
||||
On Windows, run the equivalent in PowerShell:
|
||||
|
||||
```powershell
|
||||
$jar = Join-Path $HOME '.local\share\rea\android\jadx-headless-mcp-0.7.1-all.jar'
|
||||
New-Item -ItemType Directory -Force (Split-Path $jar) | Out-Null
|
||||
Invoke-WebRequest -Uri 'https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar' -OutFile $jar
|
||||
if ((Get-FileHash -Algorithm SHA256 $jar).Hash.ToLowerInvariant() -ne '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75') { throw 'JADX JAR SHA-256 does not match REA pinned metadata' }
|
||||
$env:REA_JADX_MCP_JAR = $jar
|
||||
```
|
||||
|
||||
REA runs its packaged Java metadata bridge in source-file mode against the
|
||||
unmodified engine JAR. REA does not install Java, an Android SDK, JADX, a device
|
||||
service or an emulator. Set the path in the shell or MCP server environment:
|
||||
|
||||
```sh
|
||||
export REA_JADX_MCP_JAR=/absolute/path/jadx-headless-mcp-0.7.1-all.jar
|
||||
# Optional: select an existing JDK rather than java on PATH.
|
||||
export JAVA_HOME=/absolute/path/existing-jdk
|
||||
```
|
||||
|
||||
On Windows x64, use a REA package with its matching bundled Windows native
|
||||
controls **including owned protocol stdin support**. This Windows boundary is
|
||||
new on repository main; an older released package can still reject this host.
|
||||
PowerShell accepts paths containing spaces without additional escaping
|
||||
inside these environment values:
|
||||
|
||||
```powershell
|
||||
$env:REA_JADX_MCP_JAR = 'D:\tools\jadx-headless-mcp-0.7.1-all.jar'
|
||||
$env:JAVA_HOME = 'D:\tools\jdk-21'
|
||||
rea inspect-android-package 'D:\targets\Example.apk'
|
||||
```
|
||||
|
||||
REA selects `JAVA_HOME\bin\java.exe`, or resolves `java.exe` on the selected
|
||||
Windows `PATH`. A missing or mismatched native artifact reports the package
|
||||
prerequisite rather than attempting an unowned Java launch. Windows arm64 is
|
||||
outside this provider's current native boundary.
|
||||
|
||||
Readiness checks the selected JAR's ZIP directory for the classes consumed by
|
||||
REA's metadata bridge, as well as Java's compiler and version modules. It does
|
||||
not load an APK or execute engine code. Invalid archives and missing bridge
|
||||
classes report JAR-specific recovery; classpath inventory alone does not prove
|
||||
that every class or engine operation will load successfully. Discovery limits
|
||||
each ZIP metadata read to 8 MiB before allocation; a larger directory reports
|
||||
the exhausted read budget and the selected JAR path.
|
||||
|
||||
The filtered class-page bridge (protocol 2) is verified on Linux x86-64 with
|
||||
Temurin 21, public Appium ApiDemos 6.0.18 and NewPipe 0.29.1 APKs. Earlier
|
||||
metadata-bridge records covered macOS arm64/OpenJDK 21 and Windows x64/JDK 21;
|
||||
this protocol revision has not been rerun on those hosts or Linux/JDK 25.
|
||||
Windows x64 uses an owned Job Object with bidirectional stdio, atomic job
|
||||
assignment and kill-on-owner-close. Real Windows verification uses JDK 21 and
|
||||
the pinned JADX 0.7.1 JAR; see the real verification lane below.
|
||||
|
||||
## Answer an analyst question
|
||||
|
||||
```sh
|
||||
rea inspect-android-package /path/Example.apk
|
||||
rea search-android-classes /path/Example.apk MainActivity
|
||||
rea inspect-android-class /path/Example.apk example.MainActivity
|
||||
rea inspect-android-method /path/Example.apk example.MainActivity onCreate
|
||||
rea inspect-android-method /path/Example.apk example.MainActivity select --overload-index 1
|
||||
rea trace-android-references /path/Example.apk example.MainActivity
|
||||
rea trace-android-references /path/Example.apk example.MainActivity --method-name onCreate
|
||||
rea trace-android-references /path/Example.apk example.MainActivity --method-name select --overload-index 1
|
||||
```
|
||||
|
||||
Equivalent MCP methods are `inspect_android_package`, `search_android_classes`,
|
||||
`inspect_android_class`, `inspect_android_method` and `trace_android_references`.
|
||||
Every request includes `path`. Class/method operations use `class_name`,
|
||||
`method_name` and optional `overload_index` as applicable. For example:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_android_method",
|
||||
"arguments": {
|
||||
"path": "/path/Example.apk",
|
||||
"class_name": "example.MainActivity",
|
||||
"method_name": "select",
|
||||
"overload_index": 1
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Use `inspect_android_class` to choose an overload. REA rejects an ambiguous
|
||||
method request rather than silently selecting the first match. Method reference
|
||||
queries accept the same `overload_index`; the bridge resolves the selected
|
||||
method node and the result reports its index and displayed signature. These
|
||||
indices are tied to the current artifact and engine, not stable across builds.
|
||||
|
||||
Smali fallbacks select one complete block using the original method name and DEX
|
||||
descriptor, before applying the source byte limit. This preserves the selected
|
||||
overload even when Java code generation renames a method. If the selected Java
|
||||
body was omitted, including a method merged into a synthetic bridge, REA returns
|
||||
its original smali with the `java_body_unavailable` marker. The marker describes
|
||||
missing Java; it does not infer why the method was omitted or name a replacement.
|
||||
Unresolved or duplicate disassembly identities fail instead of attributing another
|
||||
method's instructions to the selection.
|
||||
|
||||
Native and abstract methods are identified from access flags
|
||||
captured before code generation. They report `body_status: "not_available"`, an
|
||||
empty source, and the `native_method` or `abstract_method` marker; their declarations
|
||||
are not passed through the engine's source slicer. Other missing decompiled bodies
|
||||
do not establish either access flag.
|
||||
|
||||
Static initializers (`<clinit>`) return smali with the
|
||||
`static_initializer_disassembly` marker. Java decompilation can move their
|
||||
instructions into field declarations, leaving an empty or incomplete method
|
||||
body. The selected method's disassembly preserves those instructions and retains
|
||||
the same source byte limit and explicit truncation reporting.
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- Package inspection retains decoded manifest XML, summary declarations,
|
||||
permissions and class/resource counts. REA's bridge selects the exact root
|
||||
`AndroidManifest.xml` resource by original archive name and manifest type,
|
||||
independently of asset order. One namespace-aware JDK XML parse supplies
|
||||
summary attributes and direct permission/SDK declarations; namespace prefixes
|
||||
and XML entity spelling do not change their values. Raw decoded XML is retained
|
||||
exactly and only its displayed text is subject to the existing source byte cap.
|
||||
Summaries use the complete selected XML before that cap. Document types are
|
||||
unsupported; parsing does not load external DTDs, schemas, or XInclude files.
|
||||
Signature verification is explicitly `not_performed`.
|
||||
- Class search filters the engine's cached top-level names before transferring
|
||||
pages, then follows contiguous inventory positions without rescanning earlier
|
||||
rows. It checks selection, scope, examined ranges and counts before reporting
|
||||
complete coverage. Complete producer replies stay in Evidence; an empty `query`
|
||||
inventories all class-name rows; class names are decompiler representations.
|
||||
Repeated names remain repeated in `matches` so the inventory row count and
|
||||
producer evidence stay visible. `duplicate_names` identifies matching names
|
||||
with multiple rows and their occurrence counts; a name alone does not
|
||||
distinguish which provider class node a later lookup selects.
|
||||
- Class inventories read parsed metadata without generating source and return
|
||||
full reported display types, fields and inner class names. Synthetic members
|
||||
that code generation hides can appear in the inventory; it stays consistent
|
||||
after method decompilation within the retained session.
|
||||
Exact DEX prototypes are reported from frozen JADX method metadata as
|
||||
`dex_descriptor`, for example `(I)V`. Display names can be JADX aliases;
|
||||
the descriptor alone is not a complete raw DEX method ID. Overload indices
|
||||
are tied to this artifact and engine, not stable identities across builds.
|
||||
- Method inspection reports Java or smali, fallback markers and complete/partial
|
||||
returned text. Decompiled source is a derived representation. A complete text
|
||||
result does not establish complete semantics or a byte-identical reconstruction.
|
||||
- Reference results are incoming static relationships between provider nodes.
|
||||
`reference_unit=using_node` means `total` counts containing JADX nodes, not
|
||||
individual invoke instructions; two calls from one method can appear as one
|
||||
entry. `coverage=complete` covers this node inventory only. For a method
|
||||
reference whose using node has raw code, `direct_invokes` lists matching DEX
|
||||
invoke instructions with their DEX entry, raw source method ID, code-unit
|
||||
offset, method index and opcode. An empty array means no direct invoke was
|
||||
matched in that node; `null` means raw code was unavailable or the using node
|
||||
was not a method. This list does not establish complete call-site coverage
|
||||
outside JADX's using-node inventory or through reflection and dynamic loading.
|
||||
A virtual or interface invoke also does not establish its runtime implementation.
|
||||
Source lines and the containing node's own `dex_offset` remain unknown.
|
||||
Method-reference results also carry the selected `dex_descriptor`;
|
||||
class-reference results report `null`.
|
||||
These relationships are not observed runtime calls.
|
||||
- Original requests, APK SHA-256, actual engine JAR SHA-256 and raw producer
|
||||
responses remain in Evidence. Source revision is populated only for bytes
|
||||
matching the audited release JAR.
|
||||
|
||||
If a later step fails, the error's `details.partial_observation` retains the
|
||||
selected and resolved APK paths, digest, observed provider identity and versions,
|
||||
and each upstream request and reply collected before the failure. Failed replies
|
||||
remain available even when their payload cannot be interpreted. Cancellation
|
||||
also stops target hashing before provider admission.
|
||||
|
||||
These operations do not execute the APK or provide split APK/AAB handling,
|
||||
signature validation, full resource-table semantics, native-library analysis or
|
||||
Android runtime capture. An `.aab` suffix is a ZIP archive, not an APK, so
|
||||
Android tools and the application graph do not treat it as a standalone APK.
|
||||
Existing artifact inventory/extraction tools can supply archive evidence.
|
||||
Inventory records an APK root only for ZIP bytes whose name ends in `.apk`.
|
||||
The projection adds a limitation when no root `AndroidManifest.xml` was
|
||||
observed, because such a ZIP may not be an APK. Nested members take a ZIP or
|
||||
ASAR family from their bytes; an unread member keeps an unknown format.
|
||||
Inventory keeps ZIP names that differ only in case and records that a
|
||||
case-insensitive destination cannot store both spellings. Extraction fails
|
||||
only when that destination directory cannot keep both names. Project that
|
||||
Evidence with the execution-free inventory tool:
|
||||
|
||||
```sh
|
||||
rea project-android-application-graph '{"inventory_evidence":[<inventory_artifact Evidence>]}'
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "project_android_application_graph",
|
||||
"arguments": {
|
||||
"inventory_evidence": ["<inventory_artifact Evidence>"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The projection reports exact component paths and hashes, runtime-family hints,
|
||||
and path-based bridge hypotheses. File and slice occurrences supply component
|
||||
endpoints; directory trees and symlinks retain their identities in the source
|
||||
inventory without being classified as code files. It does not decode DEX or
|
||||
claim observed runtime calls. DEX/JVM runtime hints summarize the classified
|
||||
bytecode component lists. Framework hints match complete native library names,
|
||||
such as
|
||||
`libreactnativejni.so`, `libreactnative.so`, `libhermes.so`,
|
||||
`libhermes_executor.so`, `libflutter.so`, and `libunity.so`; unrelated names
|
||||
such as `libcommunity.so` and `libreactivity.so` do not identify those families.
|
||||
The signing list retains the `META-INF/MANIFEST.MF`, `.SF`, `.RSA`, `.DSA`, and
|
||||
`.EC` paths selected by Android's v1 verifier, including nested paths. These are
|
||||
inventory metadata candidates, not proof that a v1 signature verifies or that
|
||||
the APK is signed; use `inspect-apk-signature` for signature analysis.
|
||||
The bridge basis is inferred from the native path and repeated for each managed
|
||||
component, so it does not identify a specific JNI binding. Bridge hypotheses form a DEX/JVM-class by native-library
|
||||
Cartesian product. The projection computes the product and exact serialized
|
||||
candidate-array size before creating candidate objects, then retains a
|
||||
deterministic prefix within a 2 MiB UTF-8 JSON budget. If candidates are omitted,
|
||||
`bridge_candidate_coverage` reports total, emitted, and omitted counts; overall
|
||||
coverage is `partial` while component arrays still include every component
|
||||
from the supplied inventory pages. A path pair does not establish that a
|
||||
managed declaration names, loads, or calls the native library.
|
||||
|
||||
## Resource and lifecycle limits
|
||||
|
||||
REA retains one serialized session with private, immutable APK, engine JAR and
|
||||
metadata-bridge copies. Matching requests reuse the loaded engine. APK/engine
|
||||
digests and JVM configuration are checked before reuse; changed inputs retire
|
||||
the previous session. Sessions are cleaned after 60 seconds of inactivity, on
|
||||
target/configuration changes, failure, timeout, cancellation, MCP disconnect or
|
||||
server shutdown. One-shot CLI commands always join cleanup before returning.
|
||||
REA never writes to the original APK or launches target code.
|
||||
|
||||
On Windows, stdin writes preserve stream backpressure and run outside the Node
|
||||
event loop. Cancellation terminates the retained job, including descendants,
|
||||
before workspace cleanup. On Windows, a detached REA helper owns the private
|
||||
workspace's native directory handle. An abrupt CLI/server owner termination
|
||||
closes the Java job and the helper's input pipe; the helper then removes the
|
||||
workspace by handle. The real Windows lane checks both effects. If the helper
|
||||
also dies or the host loses power before it finishes, private files can remain.
|
||||
|
||||
If Java cannot start the bridge, REA reports the selected executable and retained
|
||||
startup diagnostics. Select a full JDK through `JAVA_HOME` or `PATH`, and use
|
||||
`java --list-modules` to confirm `jdk.compiler` is present. A runnable JRE alone
|
||||
cannot compile the bridge. Missing or ambiguous class and method selections are
|
||||
input failures; discover names with `search_android_classes` and
|
||||
`inspect_android_class` before correcting the selected argument.
|
||||
|
||||
If process cleanup cannot be confirmed, REA retains the workspace and reports its
|
||||
location. That provider instance blocks subsequent engine launches until the
|
||||
caller resolves the reported resources and starts a fresh instance.
|
||||
|
||||
The provider uses one JADX analysis worker independently of JVM processor and
|
||||
garbage-collector settings. REA supplies no default heap or visible-CPU override;
|
||||
Java uses its normal ergonomics and inherits caller JVM settings. Optional
|
||||
provider settings add explicit JVM arguments:
|
||||
|
||||
```sh
|
||||
export REA_JADX_HEAP_MIB=8192
|
||||
# Optional, independent of the single JADX worker:
|
||||
export REA_JADX_ACTIVE_PROCESSOR_COUNT=4
|
||||
# Or select settings through Java's standard environment options:
|
||||
export JAVA_TOOL_OPTIONS=-Xmx8g
|
||||
```
|
||||
|
||||
Both REA settings must be positive safe integers; processor counts must also fit
|
||||
the JVM's signed 32-bit range. REA preserves
|
||||
`JAVA_TOOL_OPTIONS`, `JDK_JAVA_OPTIONS` and `_JAVA_OPTIONS` rather than replacing
|
||||
them. Java's standard precedence applies: explicit command-line heap settings
|
||||
override `JAVA_TOOL_OPTIONS`, while `_JAVA_OPTIONS` can override command-line
|
||||
settings. Evidence reports the worker's observed `Runtime.maxMemory()` rounded
|
||||
down to MiB and retains exact bytes, Java version and visible processors in its
|
||||
raw runtime response. The observed allocatable heap can differ from configured
|
||||
`-Xmx`, especially with Serial GC. Heap is not a total RSS cap: the JVM also
|
||||
uses native memory. A typed JVM memory allocation failure during APK loading
|
||||
reports `resource_constraint` with the observed maximum heap in bytes. Known
|
||||
heap failures name `REA_JADX_HEAP_MIB` in remediation; other memory regions
|
||||
remain unknown. A healthy doctor configuration check does not establish that
|
||||
any particular APK fits. Failures before the bridge can report a typed result
|
||||
and upstream errors from other operations retain their existing diagnostics;
|
||||
error text alone does not establish memory exhaustion.
|
||||
|
||||
Independent REA instances have independent queues. Whole
|
||||
operations have a **120-second** execution deadline by default, starting after
|
||||
they reach the queue's front. For a known slow cold load, such as a large
|
||||
multidex APK, select another finite deadline:
|
||||
|
||||
```sh
|
||||
export REA_JADX_OPERATION_TIMEOUT_MS=600000
|
||||
```
|
||||
|
||||
The value must be a decimal millisecond count from 1 to 2147483647; anything
|
||||
else is refused before Java starts. REA reads it from the environment captured
|
||||
at process start. A change applies on the next process start; an already
|
||||
running server keeps the deadline it started with. The same deadline bounds
|
||||
each upstream call. A timeout names `REA_JADX_OPERATION_TIMEOUT_MS`, the
|
||||
effective milliseconds, and that range. A passing JADX configuration check in
|
||||
`rea doctor` reports the effective `operation_timeout_ms`. Explicit
|
||||
cancellation still ends the operation immediately, and partial observations
|
||||
and owned-process cleanup are unchanged. Raising the deadline does not extend
|
||||
the bridge's 30-second startup handshake or the upstream **90-second**
|
||||
decompilation limit. MCP clients enforce their own tool-call timeouts, and
|
||||
setup's derived [client tool timeout](client-configuration.md) does not include
|
||||
this setting, so also raise the client timeout when selecting a longer
|
||||
deadline.
|
||||
|
||||
Decoded manifest and method text have a **1 MiB** upstream byte budget and report
|
||||
truncation. Class-name pages use indexed slices of up to 4,096 names without changing the
|
||||
complete, case-sensitive search contract. A protocol frame above **8 MiB**, or combined stdout/stderr above
|
||||
**32 MiB per operation**, fails with a diagnostic instead of returning an incomplete inventory
|
||||
as complete. These are memory-safety boundaries, not undocumented result caps.
|
||||
|
||||
## Real verification
|
||||
|
||||
Download the explicit public fixtures once; this installs no tools:
|
||||
|
||||
```sh
|
||||
npm run fixtures:android
|
||||
export REA_JADX_MCP_JAR="$PWD/_reference/apk-integration/jadx-headless-mcp-0.7.1-all.jar"
|
||||
export REA_ANDROID_TEST_APK="$PWD/_reference/apk-integration/ApiDemos-debug.apk"
|
||||
npm run verify:android
|
||||
```
|
||||
|
||||
The main lane and Windows Java-selection checks validate the selected JDK and
|
||||
engine archive through `validateVerifierJadxConfiguration`. This shared verifier
|
||||
boundary supplies the production validator's resource owner and waits for
|
||||
retained cleanup before proceeding. Failed closes retain their exact owners for
|
||||
retry; prerequisite failures must not be mistaken for completed bridge tests.
|
||||
Recovery checks match the typed Java or archive failure and its applicable
|
||||
remediation.
|
||||
|
||||
The lane also uses the selected full JDK's `jlink` to create a disposable JRE
|
||||
without the compiler module. It checks real Java startup failures, missing
|
||||
selectors, recovery after failures, and CLI cancellation with owned cleanup.
|
||||
The original APK and host Java configuration remain unchanged.
|
||||
|
||||
The same lane runs on Windows x64 with the bundled native controls and a full
|
||||
JDK. It checks CLI/MCP results, real MCP cancellation and disconnect, and Java
|
||||
exit and workspace removal after abrupt CLI owner termination. Windows forced
|
||||
termination is checked separately from the POSIX CLI SIGTERM handler. A
|
||||
synthetic two-owner test confirms that one owner's exit does not remove the
|
||||
other owner's workspace. The lane does not claim a structured cancellation
|
||||
result or power-loss recovery.
|
||||
|
||||
The download script verifies fixed SHA-256 values, reuses matching files and
|
||||
refuses to overwrite a different existing file. APK/JAR files live under ignored
|
||||
`_reference/` and are excluded from package contents. The lane exercises actual
|
||||
REA CLI and MCP results with the real engine, separately from synthetic protocol
|
||||
regressions. It needs no Gradle build, emulator, Android SDK or Ghidra.
|
||||
|
||||
Direct invoke sites retain exact DEX code-unit offsets separately from each
|
||||
containing node's unknown `dex_offset`. `direct_invokes_exhaustive: false`
|
||||
means instruction rows were bounded by the local bridge frame budget; `null`
|
||||
means the producer did not establish instruction coverage. Using-node totals
|
||||
and coverage do not imply complete instruction evidence.
|
||||
@@ -1,178 +0,0 @@
|
||||
# Offline API contract inventory
|
||||
|
||||
`inspect_api_contract` / `rea inspect-api-contract` supports local OpenAPI JSON
|
||||
and GraphQL SDL (`format: "graphql-sdl"`). Neither profile contacts endpoints.
|
||||
|
||||
## OpenAPI JSON
|
||||
|
||||
The default profile reads one local OpenAPI
|
||||
3.0, 3.1 or 3.2 JSON document. It inventories inline operation slots under
|
||||
`paths`; the facts describe declarations, not implemented, reachable or
|
||||
authorized runtime behavior. YAML, AsyncAPI, callbacks/webhooks, schema graphs,
|
||||
contract diffs and traffic correlation are outside this version.
|
||||
|
||||
The OpenAPI importer never resolves references, opens sibling documents, evaluates
|
||||
examples or contacts declared servers. OpenAPI 3.2 `query` and
|
||||
`additionalOperations` follow the [Path Item contract](https://spec.openapis.org/oas/v3.2.1.html#path-item-object),
|
||||
including case-sensitive additional HTTP method spelling.
|
||||
|
||||
## Select a useful answer
|
||||
|
||||
MCP defaults to a summary retaining the complete inventory. An optional view
|
||||
returns selected facts immediately, taking precedence over `detail`:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/artifacts/openapi.json",
|
||||
"view": {
|
||||
"kind": "api-contract",
|
||||
"collection": "operations",
|
||||
"methods": ["GET"],
|
||||
"limit": 20
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`operations` supports exact `operation_ids`, `operation_keys`, `methods`,
|
||||
`paths`, `declared_ids` and `states` filters. Filters combine by intersection;
|
||||
empty arrays select nothing. `fields` chooses columns; identity, source pointer,
|
||||
state and unknown reason remain. The other collections are
|
||||
`path-item-references` (optional `reference_ids`) and `diagnostics` (optional
|
||||
`codes`), also with fields and offset/limit. The default page contains 64 rows.
|
||||
|
||||
Query a retained result with `inspect_analysis_view`, using the same view and
|
||||
`source: {"kind":"retained-evidence","evidence_id":"..."}`. Copy the complete
|
||||
parent ID from `normalized_result.parent_evidence_id`, not the summary's own ID.
|
||||
These pages use immutable retained facts without rereading the specification.
|
||||
Complete Evidence remains available through `get_evidence_bundle`.
|
||||
|
||||
CLI defaults to complete Evidence. For a selected answer, save that complete
|
||||
Evidence in an absent bundle path:
|
||||
|
||||
```sh
|
||||
rea inspect-api-contract ./openapi.json --detail summary \
|
||||
--view '{"kind":"api-contract","collection":"operations","methods":["GET"],"limit":20}' \
|
||||
--evidence_output ./contract.evidence.json --json
|
||||
```
|
||||
|
||||
Later `rea inspect-analysis-view` can read that bundle through an exact
|
||||
`evidence-file` source selection, even after the original specification is
|
||||
unavailable. Publication never overwrites an existing file. CLI preserves its
|
||||
original path spelling in `caller_input_path`; analysis locations and MCP inputs
|
||||
use absolute filesystem paths.
|
||||
|
||||
## Identity, provenance and coverage
|
||||
|
||||
Each operation has an artifact-scoped `operation_id`, an exact method/path
|
||||
`operation_key`, an optional `declared_id` from `operationId`, and its original
|
||||
JSON Pointer. Keys can correlate exact declarations in a caller-selected pair
|
||||
of revisions; they do not establish equivalence across unrelated APIs, renamed
|
||||
path templates or runtime handlers. Declared IDs are not assumed unique.
|
||||
The artifact includes the verified byte length, SHA-256 and selected path.
|
||||
|
||||
Malformed siblings retain useful declarations plus diagnostics. A Path Item
|
||||
`$ref`, including a local fragment, remains unresolved; adjacent inline slots
|
||||
are unknown because referenced fields may overlap. Missing or malformed paths
|
||||
are explicit, and missing required OpenAPI 3.0 paths does not become a complete
|
||||
empty inventory. `source_coverage` is limited to inline paths and records that
|
||||
reference resolution and full schema validation were not performed. Page
|
||||
exhaustion describes the selected rows and never upgrades source coverage.
|
||||
|
||||
Structured transport userinfo credentials are excluded from reference URIs;
|
||||
`credential_redacted` reports that change, and an ambiguous credential span leaves
|
||||
the displayed reference unknown. Ordinary reference query/fragment text and
|
||||
names are retained. Original bytes remain in the caller's artifact; complete
|
||||
inventory Evidence does not embed the entire source JSON. Duplicate JSON members
|
||||
use the shared parser's last-value-wins behavior, recorded in source coverage.
|
||||
|
||||
If decoding or later validation fails after a verified read, the typed error
|
||||
retains a compact `file-snapshot-hash` observation with the exact path, digest
|
||||
and byte length. Syntax diagnostics do not quote source JSON. Cancellation
|
||||
does not replace a parser failure that has already completed.
|
||||
|
||||
## Resource bounds and verification
|
||||
|
||||
The file-byte allowance and conservative expanded-facts allowance are each
|
||||
32 MiB resource policies, not OpenAPI format limits. `max_file_bytes` can lower
|
||||
the input allowance. Native JSON parsing additionally checks available heap
|
||||
against the shared representation-expansion estimate before decoding. The
|
||||
shared JSON nesting limit is 1,000, including uninterpreted schema regions.
|
||||
Capacity refusals are typed; reply paging does not increase parser allowances.
|
||||
|
||||
The committed [Petstore fixture](https://github.com/morluto/rea/blob/b7c8caedd4166e6032c49024cdc6fb1154f1a2df/tests/fixtures/openapi/README.md) is an
|
||||
original document from the published application, replayed offline through
|
||||
public CLI/MCP tests. These check advertised schemas, complete/selected parity,
|
||||
retained paging, partial references, zero requests to a loopback reference
|
||||
server and unchanged source bytes. They do not establish runtime endpoint
|
||||
behavior or full OpenAPI conformance.
|
||||
|
||||
## GraphQL SDL and operation documents
|
||||
|
||||
Select `format: "graphql-sdl"` for SDL. Optionally supply `operations_path` to
|
||||
one local document containing operations and fragments. Both MCP paths are
|
||||
absolute; CLI accepts relative spellings:
|
||||
|
||||
```sh
|
||||
rea inspect-api-contract ./schema.graphql --contractFormat graphql-sdl \
|
||||
--operationsPath ./queries.graphql --detail summary \
|
||||
--view '{"kind":"graphql-contract","collection":"references","limit":20}' \
|
||||
--evidence_output ./graphql.evidence.json --json
|
||||
```
|
||||
|
||||
Collections are `types`, `fields`, `definitions`, `references`, `fragment-spreads`
|
||||
and `diagnostics`. Each supports `fields`, `offset` and `limit` (default 64).
|
||||
Exact batch filters include type/field/definition/reference IDs, type names,
|
||||
`field_keys` such as `Query.viewer`, owning type names, binding/spread states
|
||||
and diagnostic codes. Filters intersect; an empty array selects nothing.
|
||||
Identity, source and applicable uncertainty fields always remain. Retained pages
|
||||
and bundle-based queries use the same `graphql-contract` views without rereading
|
||||
SDL or rebuilding its schema model.
|
||||
|
||||
Types and fields retain individual declaration occurrences, including extensions,
|
||||
list/non-null wrappers, implemented interfaces and union member names. Operation
|
||||
and fragment definitions own field references with aliases and parent-reference
|
||||
IDs. Each reference links to a supplied SDL field occurrence, remains `model-only`
|
||||
for a library-provided field such as `__typename`, or records an exact unknown
|
||||
reason. Rejected schema construction keeps declarations but cannot supply trusted
|
||||
bindings. Missing or duplicate fragment names remain explicit. Fragments are
|
||||
visited once in their declared type scope and linked by spread edges; cycles
|
||||
are never recursively expanded.
|
||||
|
||||
The versioned `graphql-sdl-field-bindings-v1` profile records the GraphQL library
|
||||
version. It performs syntax parsing and schema construction for field resolution;
|
||||
it does not establish whole-schema/operation validity, spread applicability,
|
||||
reachability, authorization or endpoint behavior. Arguments, defaults,
|
||||
descriptions, directive behavior, enum value inventory, introspection JSON,
|
||||
federation composition and version comparison remain outside this profile.
|
||||
|
||||
Every fact has a document role, SHA-256, UTF-16 start/end offsets and parser token
|
||||
line/column. Offsets address decoded UTF-8 text, including an optional BOM, rather
|
||||
than byte positions. The two files are independently verified snapshots, not an
|
||||
atomic pair. Fact IDs identify source occurrences; `field_key` separately supports
|
||||
exact name-based joins and does not assert equivalence across revisions. CLI
|
||||
retains its auxiliary path spelling in `caller_input_paths.operations_path`,
|
||||
outside semantic identity, as well as its primary `caller_input_path`.
|
||||
|
||||
Malformed operation text retains the SDL inventory with partial coverage and a
|
||||
source-located diagnostic. A primary parser refusal or later resource/cancellation
|
||||
failure retains known source identities and available interpreted Evidence;
|
||||
syntax diagnostics do not quote source text. Complete Evidence contains facts,
|
||||
not original document bytes. An exhausted page never upgrades source coverage.
|
||||
|
||||
Each file has the same configurable maximum of 32 MiB. Conservatively charged
|
||||
facts share a 32 MiB allowance; text/AST allocation is admitted against available
|
||||
heap. The parser admits at most 262,144 lexical tokens and 65,536 comments per
|
||||
document; subsequent iterative AST admission limits traversal depth to 128.
|
||||
These are REA resource policies, not GraphQL language limits. Very deep input can
|
||||
be refused by the parser before AST admission. Selected replies reduce transport
|
||||
and retained queries reuse immutable facts; they do not avoid the initial parse.
|
||||
|
||||
The committed SWAPI SDL is an unchanged public application fixture with provenance
|
||||
in `tests/fixtures/graphql/README.md`. It supplies 53 types and 243 fields;
|
||||
verification does not execute its API.
|
||||
|
||||
Large complete inventories can exceed the default MCP response budget even when
|
||||
selected pages fit. Use `export_evidence_bundle` with a caller-selected local
|
||||
path for complete retained Evidence, or CLI complete JSON/bundle publication.
|
||||
A transport refusal does not erase a completed retained analysis. Narrow pages
|
||||
and source coverage remain available on the same continuing connection.
|
||||
+21
-55
@@ -102,11 +102,7 @@ share the application workflow, projection, retention and publication owners.
|
||||
`SignerInfo`'s certificate and digest/signature algorithm OIDs. The PKCS#7
|
||||
`SignedData` envelope is read as BER, including constructed signature OCTET
|
||||
STRINGs; each embedded certificate is still interpreted as DER by the X.509
|
||||
owner. Issuer matching preserves the encoded Name and also admits equivalent
|
||||
ASCII PrintableString/UTF8String values for CN, C, L, ST, O and OU: case and
|
||||
repeated/edge spaces are normalized, RDN sequence order is preserved, and
|
||||
attribute order inside one multivalued RDN is ignored. Other name forms
|
||||
require exact encoded identity; semantic equivalence remains unestablished.
|
||||
owner.
|
||||
`certificate` is `null` when the block has more than one `SignerInfo` (with
|
||||
nothing verified there is no single signer) or names a certificate it does
|
||||
not carry. A block that cannot be read (malformed ASN.1, an unsupported
|
||||
@@ -148,10 +144,7 @@ share the application workflow, projection, retention and publication owners.
|
||||
- **Per signer:** the certificate's **SHA-256 fingerprint over the raw embedded
|
||||
DER** (the same value `apksigner --print-certs` prints), plus subject, issuer,
|
||||
serial, ISO 8601 validity and public-key algorithm when the X.509 parser can
|
||||
read the exact carried DER (each is `null` otherwise). The parser's returned
|
||||
DER must equal those bytes before readable facts are combined with their
|
||||
fingerprint; embedded PEM text and trailing data cannot select a different
|
||||
certificate. Raw signature algorithm IDs, declared
|
||||
read them (each is `null` otherwise); raw signature algorithm IDs, declared
|
||||
digest hex and digest algorithms, signature lengths and public-key length; and for
|
||||
v3/v3.1 the signer's declared min/max SDK.
|
||||
- **Key rotation (v3/v3.1):** the declared proof-of-rotation lineage, oldest
|
||||
@@ -179,31 +172,26 @@ share the application workflow, projection, retention and publication owners.
|
||||
- `schemes_agree_on_signer` (observed): every present block scheme presents
|
||||
the same _set_ of signer certificates. An extra signer in one scheme is
|
||||
disagreement; a signer without a certificate makes it `null` (unknown).
|
||||
A single present block scheme with known signers agrees with itself;
|
||||
v1 is compared separately below.
|
||||
Build-tools 33+ rotate through v3.1 by default, so a rotated app usually
|
||||
reads `false` here and `true` below.
|
||||
- `signer_change_declared_by_rotation` (declared): the signers differ, but
|
||||
each appears in a lineage the APK declares. Read the two together: block
|
||||
schemes with different signers covered by a declared lineage show `false`
|
||||
/ `true`; differing signers without such a lineage show `false` / `false`.
|
||||
A lineage alone does not establish a change between block schemes. It is
|
||||
`null` when a signer's identity is unknown, or when the signers differ and
|
||||
a declared lineage could not be decoded.
|
||||
each appears in a lineage the APK declares. Read the two together: a
|
||||
rotated APK shows `false` / `true`; signers that differ with no declared
|
||||
lineage containing them show `false` / `false`. It is `null` when a
|
||||
signer's identity is unknown, or when the signers differ and a declared
|
||||
lineage could not be decoded.
|
||||
- `v1_signers_match_block_signers` (observed): the v1 signer certificates
|
||||
equal what the lowest block scheme that has signers (v2, else v3, else
|
||||
v3.1) expects v1 to carry. That is its signer set, except that a v3/v3.1
|
||||
signer declaring a rotation lineage expects the lineage's first
|
||||
(original) certificate, as apksig requires. `false` is notable: the JAR
|
||||
signature names a different key than the block signature. When v1 uses
|
||||
the lineage's original key and v3 its newer key, this comparison reads
|
||||
`true`. If v3 is the only present block scheme, agreement is `true` and
|
||||
declared inter-block signer change is `false` even though rotation is
|
||||
present. If v2 also uses the original key, agreement is `false` and the
|
||||
declared inter-block change is `true`. `null` when
|
||||
v1 or a populated block scheme is absent, any signer's identity is unknown (a
|
||||
signature names a different key than the block signature. A key rotation
|
||||
keeps v1 (and v2) on the original key, so it reads `true` while
|
||||
`schemes_agree_on_signer` reads `false`, with or without v2. `null` when
|
||||
v1 or a block scheme is absent, any signer's identity is unknown (a
|
||||
`decode_error`, a `null` certificate or a `subjectKeyIdentifier`
|
||||
`SignerInfo`), or the lineage's original certificate cannot be
|
||||
identified. When identities or a lineage origin are unknown, limitations
|
||||
explain why; absence is shown in the scheme records.
|
||||
identified; a limitation then says why.
|
||||
- `strongest_scheme`, `rotation_present`, `source_stamp_present`,
|
||||
`self_consistent`.
|
||||
|
||||
@@ -226,9 +214,9 @@ Not yet read: v4 `.apk.idsig` (a sibling file), verity (`0x0421`/`0x0423`/`0x042
|
||||
## Bounds and errors
|
||||
|
||||
The archive is read once, bounded at 512 MiB, without following symlinks and
|
||||
with change detection. ZIP tail offsets, the central-directory size and signing
|
||||
block spans used by the reader are checked against their containing sections,
|
||||
including the complete central-directory extent before ZIP end records. A process-wide queue permits
|
||||
with change detection. Every size and offset in the ZIP tail and signing block
|
||||
is checked against its containing section before use, including the complete
|
||||
central-directory extent before ZIP end records. A process-wide queue permits
|
||||
one active whole-file inspection and eight pending scans across provider
|
||||
instances; pending scans retain no APK bytes and can be cancelled with their
|
||||
original reason. Further requests receive a capacity error. The active slot
|
||||
@@ -244,41 +232,19 @@ serialized size of the report together with its limitations (which repeat
|
||||
archive names) is then checked against an 8 MiB output budget (the exact check
|
||||
is authoritative); an over-budget report is refused, not truncated.
|
||||
|
||||
Signing blocks admit at most 4096 ID/value pairs; at most 4096 signing-relevant
|
||||
META-INF entries are retained. These fixed counts, signer/lineage counts and the
|
||||
report byte budget return nonretryable `resource_constraint` errors naming the
|
||||
effective allowance. This release has no runtime override for them. Waiting for
|
||||
an active scan cannot raise a fixed count, and complete CLI JSON cannot raise the
|
||||
report budget. Occupied scan-queue capacity remains retryable after a scan finishes.
|
||||
|
||||
The whole-file reader allocates the selected snapshot and bounded read scratch;
|
||||
total process memory exceeds the selected file's size. It also computes the complete
|
||||
The whole-file reader retains read chunks and a concatenated buffer; total
|
||||
process memory exceeds the selected file's size. It also computes the complete
|
||||
file SHA-256, so inspection is not header-only. Content digests are recomputed
|
||||
at most once per SHA-256/SHA-512 algorithm during one inspection. Input,
|
||||
inflation, decoded-record and serialized-output bounds keep work finite;
|
||||
resource refusals are distinct from malformed format errors.
|
||||
|
||||
If the stable read completes but closing its descriptor fails or is cancelled,
|
||||
the error retains `details.partial_observation` with the selected path, complete
|
||||
file SHA-256 and byte count. These facts describe the acquired file; they do not
|
||||
establish any decoded signing facts. Failed cleanup remains visible until the
|
||||
provider releases the descriptor on a cleanup retry.
|
||||
|
||||
A file that is not a ZIP is refused as an unsupported target. A damaged ZIP
|
||||
(local headers but no end-of-central-directory), a central directory that
|
||||
cannot be walked or whose declared record count does not account for all of
|
||||
its bytes, or a malformed signing block or signer is an input error naming the
|
||||
offset or scheme and reason. Zip64 tail damage includes a missing required
|
||||
locator, an impossible locator disk/count pair, a fixed end record that does
|
||||
not fit before its locator, or an incorrect end-record signature. These are
|
||||
input errors, not evidence of an unsupported format. Coherent multi-disk
|
||||
locator metadata and 64-bit offsets/counts/sizes beyond the reader's safe
|
||||
integer range remain unsupported. The locator offset is disk-relative, so
|
||||
multi-disk metadata is checked before applying local file bounds. This follows
|
||||
the record fields in [PKWARE APPNOTE 6.3.10, sections 4.3.14–4.3.15](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT);
|
||||
it does not establish full validation of Zip64 versions or extensible data.
|
||||
Signer framing includes signature records, the public-key field and matching
|
||||
signed/outer v3 SDK ranges. Suffix bytes beyond known
|
||||
scheme and reason. Signer framing includes signature records, the public-key
|
||||
field and matching signed/outer v3 SDK ranges. Suffix bytes beyond known
|
||||
fields are counted as uninterpreted, as Android readers may ignore extensions;
|
||||
their presence does not establish malformed signing.
|
||||
When the central directory is not immediately followed by
|
||||
|
||||
+13
-51
@@ -66,8 +66,6 @@ rea decode-android-resources /targets/Example.apk --locales de --locales pt-BR
|
||||
rea decode-android-resources /targets/Example.apk --locales de --locales pt-BR --string-tables-only
|
||||
rea decode-android-resources /targets/Example.apk --no-strings
|
||||
rea decode-android-resources /targets/Example.apk --decode-timeout-ms 900000
|
||||
rea decode-android-resources /targets/Example.apk --string-offset 2000 --string-limit 100 --string-tables-only
|
||||
rea decode-android-resources /targets/Example.apk --max-string-table-bytes 16777216 --max-total-string-table-bytes 67108864
|
||||
```
|
||||
|
||||
Equivalent MCP methods are `inspect_apktool_client` and
|
||||
@@ -89,51 +87,21 @@ responses and charges the aggregate output budget for one copy per table.
|
||||
Omitting the option preserves existing responses. `include_strings: false`
|
||||
still suppresses all string projection, including in this mode.
|
||||
|
||||
Use `string_offset` and `string_limit` (CLI `--string-offset` and
|
||||
`--string-limit`) to select a page of valid parsed entries from each requested
|
||||
table, in source order. Defaults are offset 0 and limit 2000. Explicit page or
|
||||
budget controls include `string_tables` even for a single primary table; its
|
||||
`page` gives the requested offset and limit, returned count, and `next_offset`.
|
||||
Retry that offset with the same locale and confirm `target.sha256` matches.
|
||||
Each call decodes a fresh owned snapshot; it does not retain a workspace or
|
||||
promise identity across changed APKs. A null continuation means the page reaches
|
||||
the end, while table coverage remains partial if earlier entries were omitted.
|
||||
An offset at or beyond the end returns an empty, present table with known counts.
|
||||
If the continuation equals the requested offset, no output entry fit: increase
|
||||
`max_total_strings` or use `string_tables_only` before retrying.
|
||||
|
||||
`max_total_strings` (CLI `--max-total-strings`, default 10000) bounds returned
|
||||
string copies across all tables. Raise it, request fewer locales, or use
|
||||
`string_tables_only` when aggregate output is exhausted. The legacy
|
||||
multi-table shape charges both copies of primary strings. `max_string_table_bytes`
|
||||
(CLI `--max-string-table-bytes`, default 8388608) bounds one XML source read;
|
||||
`max_total_string_table_bytes` (CLI `--max-total-string-table-bytes`, default 33554432) bounds XML reads across distinct physical tables. Locale aliases
|
||||
share the physical read and parsed page. Raise the relevant read budget to
|
||||
retry an oversized table; a larger string limit or timeout does not raise it.
|
||||
These controls take nonnegative safe-integer offsets and positive safe-integer
|
||||
budgets, apply on the next decode, and require no REA restart. Process memory
|
||||
and response transport capacity still apply.
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- **An owned APK snapshot is digested and decoded.** `target.sha256` and
|
||||
`target.bytes` bind the observation to the exact APK that was decoded.
|
||||
The Evidence subject carries the same snapshot digest and caller-selected
|
||||
path, including when the original file changes during decoding.
|
||||
A change during snapshot acquisition reports retryable `artifact_changed`;
|
||||
wait for a stable input before retrying. Host read permission failures report
|
||||
`access_denied`, while a missing or non-regular selected path reports
|
||||
`invalid_request` with the rejected `path` field.
|
||||
- **Metadata comes from apktool.yml as apktool printed it.** Version codes
|
||||
keep their own spelling (`'7'` versus `7`); `package_name` is read from
|
||||
the decoded manifest, not inferred.
|
||||
The `version: 2.9.0`-or-later header that Apktool 2.9+ and 3 emit selects
|
||||
their single-line string dialect: literal hashes, single quotes, empty
|
||||
strings, and the text `null` stay part of the value; producer backslashes
|
||||
and Unicode escapes decode once. Older/tagged documents retain YAML scalar
|
||||
semantics; double-quoted values also retain YAML string semantics.
|
||||
Apktool 3's emitted `version: 3.x.y` header selects its single-line string
|
||||
dialect: literal hashes, single quotes, empty strings, and the text `null`
|
||||
stay part of the value; producer backslashes and Unicode escapes decode
|
||||
once. Older/tagged documents retain YAML scalar semantics; double-quoted
|
||||
values also retain YAML string semantics.
|
||||
This distinction follows the released
|
||||
[2.9.0 metadata writer](https://github.com/iBotPeaches/Apktool/blob/b64e257c9b689eb9630ad0241f2fe4f4a749955b/brut.apktool/apktool-lib/src/main/java/brut/androlib/apk/VersionInfo.java#L38-L42),
|
||||
[3.0.3 metadata writer](https://github.com/iBotPeaches/Apktool/blob/18b5e99cb56ff9451e8aa55b065dcf5bbd616975/brut.apktool/apktool-lib/src/main/java/brut/androlib/meta/VersionInfo.java#L49-L55)
|
||||
and [string escape owner](https://github.com/iBotPeaches/Apktool/blob/18b5e99cb56ff9451e8aa55b065dcf5bbd616975/brut.j.yaml/src/main/java/brut/yaml/YamlStringEscapeUtils.java#L55-L116).
|
||||
- **The manifest is the decoded text**, bounded at 512 KiB and reported as
|
||||
@@ -144,8 +112,8 @@ and response transport capacity still apply.
|
||||
Malformed XML and a wrong document root are whole-document projection
|
||||
failures: the result is partial, the failure reason is disclosed, and entry
|
||||
counts remain unknown. They are not counted as one unsupported string.
|
||||
The default page returns up to 2000 entries with partial coverage when more
|
||||
exist; its limitation names the next `string_offset`. `locales` lists observed regular `strings.xml` tables whose exact
|
||||
The default table is bounded to 2000 entries with partial coverage when more
|
||||
exist. `locales` lists observed regular `strings.xml` tables whose exact
|
||||
`values-<locale>` directory is selectable by the request's locale grammar.
|
||||
Locale suffixes allow up to 23 ASCII characters, derived from Apktool's
|
||||
language, script, region, and variant fields; for example,
|
||||
@@ -160,10 +128,9 @@ and response transport capacity still apply.
|
||||
When multiple tables are requested with `locales`, `string_tables` records
|
||||
each projected table with its locale, status (`present`, `missing`,
|
||||
`oversized`, `malformed`, or `skipped`), entry counts, and coverage,
|
||||
with default aggregate budgets of 10,000 strings and 32 MiB XML. When `locales`
|
||||
is omitted, `string_tables_only` is not true and no string page or budget
|
||||
control is selected, legacy single-table payloads are preserved without adding
|
||||
`string_tables`.
|
||||
bounded at an aggregate budget of 10,000 strings and 32 MiB XML. When `locales`
|
||||
is omitted and `string_tables_only` is not true, legacy single-table payloads are preserved
|
||||
without adding `string_tables`.
|
||||
- **Smali is skipped by design** (`apktool d --no-src`): Java-source
|
||||
recovery belongs to the JADX family. Resource smali edits and rebuild are
|
||||
a follow-up lane with mutation authority.
|
||||
@@ -184,8 +151,8 @@ and response transport capacity still apply.
|
||||
- The version probe waits 20000 ms and retains 65536 bytes. This release has
|
||||
no runtime override for that probe. `decode_timeout_ms` does not raise it.
|
||||
- Manifest reads are bounded to 512 KiB before decoding UTF-8. Strings XML
|
||||
reads default to 8 MiB per table; larger documents are skipped with partial
|
||||
coverage and an explicit read-budget recovery, rather than reported as missing.
|
||||
reads are bounded to 8 MiB; larger documents are skipped with partial
|
||||
coverage and an explicit limit, rather than reported as missing.
|
||||
- `decoded_file_count` is capped at 100,000 files; the cap is reported in
|
||||
the limitations when hit.
|
||||
- An ordinary nonzero decode exit surfaces as `unsupported_target` naming
|
||||
@@ -202,7 +169,7 @@ OpenJDK 25 against a signed aapt2-built probe APK carrying two resource
|
||||
locales — launcher version, on-disk digest agreement, apktool.yml metadata
|
||||
(1.2.3, SDK 24–34), manifest package agreement, default and `de` string
|
||||
tables projected. Parser goldens in the test suite come from the same
|
||||
decode. See `docs/development/testing.md` for the lane summary.
|
||||
decode. See `docs/testing.md` for the lane summary.
|
||||
|
||||
Multi-locale and table-only projection were also verified through real CLI and
|
||||
stdio MCP workflows on Linux with Apktool 3.0.3/JDK 21 against NewPipe 0.29.1,
|
||||
@@ -211,8 +178,3 @@ exact facts/Evidence parity and bounded task costs are recorded in the
|
||||
[shared-decode and table-only checkpoint](https://github.com/morluto/rea/blob/42e1a17d6e6c0487f17be0bae248553c4caba7f7/experiments/analysis-efficiency/apktool-string-tables.md).
|
||||
This qualifies selected resource facts, not application execution or independent
|
||||
binary resource-decoder completeness.
|
||||
|
||||
If the selected APK changes while its private snapshot is copied, acquisition
|
||||
returns `artifact_changed`; request-schema correction cannot recover a stable
|
||||
snapshot. Launcher spawn permission denial is distinct from a missing launcher,
|
||||
and configuration diagnostics retain the selected command and system error.
|
||||
+3
-15
@@ -101,8 +101,8 @@ limitation reports how many inventoried entries were left out. IPA projection
|
||||
still lists every archive component, including `SwiftSupport/`.
|
||||
|
||||
`platforms` lists `macos` for application roots with `Contents/`. It lists `ios`
|
||||
for shallow application roots that have an `Info.plist` at their root. An
|
||||
application root with neither, such as an empty inventory, has no platform.
|
||||
for shallow application roots that have an `Info.plist`. An application root
|
||||
with neither, such as an empty inventory, has no platform.
|
||||
|
||||
## macOS bundle anatomy
|
||||
|
||||
@@ -149,11 +149,6 @@ directories rather than `Versions/Current`. A framework's `info_plist_path` is
|
||||
because `Versions/Current` decides which one applies) or when its single
|
||||
version directory has no `Resources/Info.plist`.
|
||||
|
||||
A `shallow` bundle's `info_plist_path` is its root `Info.plist`. Without one,
|
||||
it is `Resources/Info.plist`: a flattened macOS framework keeps its binary,
|
||||
`_CodeSignature/`, and `Resources/` directly in the `.framework`, like a single
|
||||
`Versions/<version>/` directory without the `Versions/` wrapper.
|
||||
|
||||
AppleDouble sidecar files (`._*` and `__MACOSX/`) are inventory facts, but they
|
||||
describe neighbouring files. They are excluded from roots, bundle roles, and the
|
||||
anatomy components.
|
||||
@@ -161,10 +156,6 @@ anatomy components.
|
||||
The projection does not read plists. Use `inspect_plist` on a bundle's
|
||||
`info_plist_path` or a launchd plist to read `CFBundleExecutable`, identifiers,
|
||||
`SMPrivilegedExecutables`, `BundleProgram`, URL schemes, and usage descriptions.
|
||||
MCP calls require an existing `open_binary` or `open_bundle_resources` session,
|
||||
including when `inspect_plist.path` is explicit. For a bundle without a resolved
|
||||
executable, open its resource session first. The CLI `rea inspect-plist <path>`
|
||||
opens the selected artifact for its one invocation.
|
||||
Use `inspect_signature` on an executable candidate for code-signing claims.
|
||||
|
||||
## Dylib load resolution
|
||||
@@ -218,9 +209,6 @@ Resolution follows dyld:
|
||||
already loaded image reuses it.
|
||||
- Symlinks such as `Versions/Current` are followed inside the analyzed root.
|
||||
An escape through `..` or an absolute link is `escapes-target`.
|
||||
- Dyld path components are POSIX names. On Windows hosts, link text uses `\`
|
||||
as its separator, and a dyld component that contains `\` or `:` cannot name
|
||||
a file in the analyzed root, so its candidate is `absent`.
|
||||
|
||||
Each edge lists every candidate path tried, with its outcome: `resolved`,
|
||||
`absent`, `not-mach-o`, `malformed`, `unsupported` (recognized Mach-O outside
|
||||
@@ -254,7 +242,7 @@ A `conditional` image is still traced, because it is an in-root candidate
|
||||
that may load. Every edge below it has `loader_conditional: true`, and a
|
||||
request that reuses it by install name is `conditional` too. An in-root dylib
|
||||
with no platform load command remains unknown unless the root is an executable
|
||||
with observed macOS platform and the candidate's CPU type is x86_64, i386, or arm64. In that macOS context,
|
||||
with observed macOS platform and the candidate’s CPU type is x86_64, i386, or arm64. In that macOS context,
|
||||
[Apple's dyld](https://github.com/apple-oss-distributions/dyld/blob/fd8d0c4d52320ebf64db34f3cb280310d905c5ae/mach_o/UnsafeHeader.cpp#L1137-L1145) supplies an implicit macOS platform to legacy dylibs on those CPU
|
||||
types. The candidate records `platform_basis: "macos-dyld-implicit"`; the
|
||||
reported slice fields `platform: null` and `platforms: []` remain the parsed
|
||||
@@ -1,186 +0,0 @@
|
||||
# Apple IPS crash reports
|
||||
|
||||
`inspect_apple_ips` / `inspect-apple-ips` answers whether one selected local
|
||||
Apple `.ips` file is a crash report (`bug_type` string `"309"`), which process,
|
||||
exception, and termination facts that report records, which threads and
|
||||
`usedImages` rows it lists, and which documented crash classes its strings
|
||||
match. It reads the file without executing a target and does not require Xcode,
|
||||
atos, a disassembler, or pwntools.
|
||||
|
||||
```bash
|
||||
rea inspect-apple-ips /absolute/path/to/report.ips --json
|
||||
```
|
||||
|
||||
The CLI resolves relative paths against the caller's working directory. MCP
|
||||
requires an absolute filesystem path and preserves that exact selection:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_apple_ips",
|
||||
"arguments": { "path": "/artifacts/report.ips" }
|
||||
}
|
||||
```
|
||||
|
||||
## Observations
|
||||
|
||||
The inline Evidence result contains the artifact path, SHA-256, and byte length.
|
||||
`metadata` is the first-line JSON object. `bug_type` stays a string. The crash
|
||||
body is parsed only when that string is `"309"`. Other types, including `"288"`
|
||||
stackshots, return no crash result.
|
||||
|
||||
`process`, `exception`, `termination`, `faulting_thread`, `threads`, and
|
||||
`images` reproduce documented body fields when present, including each image's
|
||||
`name` and each frame's `symbolLocation` (as `symbol_location`, the byte offset
|
||||
from the start of `recorded_symbol`). `termination` keeps the terminating
|
||||
process as `by_proc` and `by_pid` and its `flags`; `process.parent_pid` keeps
|
||||
`parentPid` next to `parent`; and `metadata` keeps `build_version` and
|
||||
`platform`. Integers keep their
|
||||
decimal literals and a hexadecimal form, including arm64e registers above
|
||||
2^53. Frame roles follow the recorded frame index: row zero of the resolved
|
||||
faulting thread is `faulting_instruction`, row zero of other threads is
|
||||
`current_instruction`, and later rows are `return_address`. Each runtime
|
||||
address uses that row's own image base and offset. The recorded `inline` flag
|
||||
does not supply a missing offset or authorize borrowing another row's address.
|
||||
When `faultingThread` is missing or does not resolve to a recorded thread
|
||||
object, row-zero roles are `unknown`; the recorded lookup remains available.
|
||||
A translated process records Rosetta's registers in a nested
|
||||
`threadState.rosetta` object; each one is reported as `rosetta.<name>` and
|
||||
counts toward the register budget. A [public ReportCrash `.ips` log](https://github.com/Mortennn/Dozer/issues/189)
|
||||
shows `rosetta.tmp2`, `tmp1`, and `tmp0` beside ordinary x86 registers. The
|
||||
test fixture quotes that register excerpt inside an authored report body; it
|
||||
does not generate a new Rosetta crash during the test.
|
||||
Observed JSON strings retain `""`; `null` means the field was missing or JSON
|
||||
`null`. This includes image paths, frame symbols, process text, and the older
|
||||
string `osVersion` form. Unlisted JSON keys and generic register names also
|
||||
retain empty spellings. An empty image architecture remains recorded as `""`
|
||||
while `uuid_unknown` stays true, since that spelling does not establish an
|
||||
architecture. An empty UUID is malformed, so its exact spelling appears in
|
||||
`coverage.type_mismatches` and its canonical UUID remains unknown.
|
||||
`process.os_version` keeps the producer's dictionary (`train`, `build`,
|
||||
`release_type`, `is_embedded`) or the older string form. Image UUIDs are uppercase 8-4-4-4-12; an all-zero UUID
|
||||
or a missing or empty `arch` is `uuid_unknown`. Image `source` letters keep their
|
||||
documented meaning (`P` process, `S` shared cache, `C` shared cache library,
|
||||
`K` kernel, `U` kernel cache, `T` kernel text exec, `A` absolute). Keys the
|
||||
documentation does not list remain in `unlisted_keys` or `unlisted_body_keys`.
|
||||
|
||||
Each frame's `image_index` refers to its entry in `images`. Image architecture,
|
||||
UUID and `uuid_unknown` are retained once on that image. Resolve the reference
|
||||
when using those facts; an absent or out-of-range index leaves the image
|
||||
unknown. This avoids repeating image text for every stack frame.
|
||||
|
||||
A present value whose JSON type differs from the documented one is never read
|
||||
as absent: it is listed in `coverage.type_mismatches` with its path, the
|
||||
expected type, and the observed type, and `coverage.status` becomes
|
||||
`partial`. A crash body without `threads` or `usedImages` is also `partial`.
|
||||
Malformed image UUID strings likewise make coverage partial; their exact spelling
|
||||
is retained in the mismatch's `recorded_value`, while the canonical image UUID
|
||||
remains unknown. Both 32 hexadecimal digits and canonical hyphenated UUIDs are accepted.
|
||||
Unresolved thread/image references also make coverage partial. A rejected
|
||||
integer lookup keeps its exact decimal literal in `recorded_integer`, including
|
||||
negative values and values above uint32. Valid uint32 lookup values remain in
|
||||
`faulting_thread` or `image_index` even when the referenced row is unavailable.
|
||||
|
||||
`classification` matches only documented SIGKILL codes (compared by their
|
||||
canonical hexadecimal form), `EXC_GUARD` subtypes,
|
||||
pointer-authentication suffixes, and dyld missing-library strings. A report
|
||||
that matches none of those rules is `unclassified`.
|
||||
|
||||
## Joining frames to the crashed Mach-O or dSYM
|
||||
|
||||
Pass `join_path` (`--join-path` in the CLI) to select one local Mach-O
|
||||
executable or dSYM. UUID bytes in the report and the artifact's `LC_UUID` load
|
||||
commands are compared in hexadecimal, so an uppercase `dwarfdump --uuid` value
|
||||
and a lowercase report `uuid` match.
|
||||
|
||||
```bash
|
||||
rea inspect-apple-ips /absolute/path/to/report.ips \
|
||||
--join-path /absolute/path/to/App.app/Contents/MacOS/App --json
|
||||
```
|
||||
|
||||
Shared-cache image sources `S` and `C` remain unjoined: UUID agreement does not
|
||||
establish that a standalone image has the cached layout.
|
||||
|
||||
For a FAT container every slice is examined. An image joins only when exactly
|
||||
one slice has that UUID **and** an architecture that agrees with the report's
|
||||
`arch`; `arm64` and `arm64e` stay distinct, and an all-zero UUID never joins. A
|
||||
UUID match with a different architecture, more than one compatible slice, or a
|
||||
slice without a readable `__TEXT` segment is reported as a rejected image with
|
||||
its reason and stays unjoined rather than being guessed.
|
||||
An unknown UUID in another slice prevents a unique join unless its architecture
|
||||
rules out a match. Missing or unrecognized architecture and unsupported CPU
|
||||
subtypes remain unknown; display-name fallbacks do not establish agreement.
|
||||
The parsed slice facts remain available even when no match can be derived.
|
||||
|
||||
`join.status` is `selected` with the artifact path, SHA-256, byte length, every
|
||||
slice's offset, arch, CPU type and subtype, UUID, and `__TEXT` vmaddr, plus
|
||||
`image_matches`, per-frame results, and join-specific limitations. A joined
|
||||
frame reports `static_address`, the slice `__TEXT` vmaddr plus the frame's
|
||||
recorded `imageOffset`, in decimal and hexadecimal. An unjoined frame reports
|
||||
a reason instead; an unknown image UUID, an absent `imageIndex` row, and a
|
||||
frame without `imageOffset` all stay unjoined rather than treated as aligned.
|
||||
|
||||
An artifact that is not a Mach-O image or FAT container reports
|
||||
`join.status: "unreadable"` with a typed reason and leaves every report fact
|
||||
intact. Symbol names printed inside the report stay recorded claims. Function
|
||||
or symbol lookup at a joined static address, and matching dSYM symbol tables,
|
||||
are not performed by this step.
|
||||
|
||||
Symbol names printed inside the report are recorded claims and are not
|
||||
verified. Linux ELF cores use the separate
|
||||
[recorded crash](guides/native/recorded-crashes.md) workflow.
|
||||
|
||||
## Limits and failures
|
||||
|
||||
The selected file is limited to 1 MiB by default (16 MiB maximum). At most 256
|
||||
threads, 8192 frames, 1024 `usedImages` rows, and 16384 `threadState`
|
||||
register slots are examined by default (4096 / 65536 / 4096 / 262144
|
||||
maximum). Budgets are checked before each row is parsed, and body members the
|
||||
report does not interpret are syntax-checked without being kept. Fixed
|
||||
allowances bound JSON nesting (64 levels), unlisted key names per object
|
||||
(1024), and `termination.reasons` entries (256). Malformed metadata, a
|
||||
truncated crash body, invalid UTF-8, a NUL in the text, and exhausted budgets
|
||||
fail without partial success.
|
||||
|
||||
`max_integer_text_bytes` (`--max-integer-text-bytes` in the CLI) separately
|
||||
bounds retained decimal and hexadecimal integer text, including every derived
|
||||
runtime address and rejected lookup literal. Its default is 8 MiB of UTF-16 text storage and its maximum is
|
||||
64 MiB. Each appearance counts, even when an object could be shared in memory
|
||||
but would be serialized more than once. The parser reserves a conservative
|
||||
upper bound before numeric conversion or address expansion, then releases the
|
||||
unused reservation after each value. It reuses the parsed source integer when
|
||||
the same image base contributes to many frames.
|
||||
|
||||
Raise this per-call allowance on the next call when more room is needed; no
|
||||
restart is required. File-size and row budgets do not raise it. This allowance
|
||||
is separate from JSON wire size, MCP response limits and the client's receive
|
||||
buffer. An exhausted allowance reports a typed resource error without partial
|
||||
success or truncated integer facts.
|
||||
|
||||
```bash
|
||||
rea inspect-apple-ips /absolute/path/to/report.ips --max-integer-text-bytes 16777216 --json
|
||||
```
|
||||
|
||||
`max_join_bytes` (`--max-join-bytes`) independently bounds the join artifact
|
||||
read, defaulting to 64 MiB. This workflow does not execute a target, launch
|
||||
atos, or search for dSYMs.
|
||||
|
||||
## Join admission and failures
|
||||
|
||||
The join uses the shared Mach-O container, slice, command-table and segment
|
||||
readers. Slice extents and `sizeofcmds` bound every identity observation; duplicate
|
||||
UUIDs or malformed commands cannot select the first plausible value. Unreadable
|
||||
join results retain the successfully read artifact's path, SHA-256 and byte count.
|
||||
|
||||
The integer-text allowance spans the report and join, including repeated integer
|
||||
facts and derived static addresses. Reservation precedes derived address
|
||||
expansion. If a selected join read, interpretation or allowance fails, the error
|
||||
retains the completed IPS report. Join read limits and recovery use
|
||||
`max_join_bytes`, even when the report and join paths are identical. Cancellation
|
||||
retains the caller's reason independently of those completed facts.
|
||||
|
||||
When a failure follows a completed join-file read, `partial_observation` retains
|
||||
both the decoded report and a `join_artifact` receipt with the independently
|
||||
observed path, SHA-256 and byte count. Frame rejection reasons reference
|
||||
`join.limitations`; source-sized architecture diagnostics are retained once per
|
||||
image rather than copied into every frame. Universal-image admission includes
|
||||
the shared reader's supported hidden ARM64 row and rejects overlapping slices.
|
||||
@@ -1,6 +1,4 @@
|
||||
graph TD
|
||||
%% Selected layer owners and dependency/wiring paths, not a directory or import inventory.
|
||||
%% Current deep-provider candidates are wired in src/composition/binary.ts.
|
||||
subgraph "Entry Points"
|
||||
CLI["src/cli.ts + src/cli/<br/>One-shot CLI adapter<br/>setup, diagnostics, analysis, decompilation"]
|
||||
MAIN["src/main.ts + src/main/<br/>MCP server adapter: stdio default, streamable HTTP opt-in<br/>config, session, process lifetime"]
|
||||
@@ -8,7 +6,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Production Composition"
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>typed capability bindings: construction, availability and retryable lifetimes<br/>pure capability declarations feed provider catalogs"]
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>static WASM/EVM descriptors: named contracts, factories, retryable lifetimes"]
|
||||
end
|
||||
|
||||
subgraph "Adapters"
|
||||
@@ -21,15 +19,14 @@ graph TD
|
||||
subgraph "Application Layer"
|
||||
APP["src/application/<br/>Shared CLI/MCP workflows<br/>enhanced analysis and investigations"]
|
||||
SESSION["SessionProviderRouter + BinarySession<br/>one immutable deep binding per target<br/>disjoint auxiliary routing"]
|
||||
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback<br/>ProviderInspectionScope: prerequisite observation lifetime"]
|
||||
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback"]
|
||||
RECORDS["src/application/investigation/<br/>composed InvestigationRecords + narrow ports<br/>EvidenceLedger and Unknown ownership"]
|
||||
LIFETIMES["CapabilityScope<br/>host-owned service close capabilities<br/>released peers removed; failed cleanup retained for retry"]
|
||||
SNAPSHOT["BinarySessionRecords + AnalysisSnapshotCache<br/>binary target/profile snapshots and session file ownership<br/>record coordination and post-commit notifications"]
|
||||
SNAPSHOT["BinarySessionRecords + AnalysisSnapshotCache<br/>binary target/profile snapshots<br/>compatibility facade and post-commit notifications"]
|
||||
JS_RECON["src/application/javascript/ + src/artifacts/javascript/<br/>JavaScript artifact reconstruction<br/>directory/ASAR reads<br/>AST-only bundle projection"]
|
||||
end
|
||||
|
||||
subgraph "Provider Layers (selected adapters)"
|
||||
PROVIDER_OWNERS["src/providers/<br/>shared deferred client owners<br/>exact cleanup retry and attributed failures"]
|
||||
subgraph "Provider Layers"
|
||||
PROVIDER_OWNERS["src/providers/<br/>shared deferred client and inspection owners<br/>exact cleanup retry and attributed failures"]
|
||||
PWNTOOLS["src/native/pwntools/ + src/native/pwndbg/<br/>BYO offline ELF/core adapters<br/>owned snapshots/Python and optional core-only GDB<br/>unchanged upstream parsers, ABI structures and mapping API"]
|
||||
WASM["src/wasm/<br/>caller-supplied WABT; immutable artifact inspection<br/>owned process supervision and retryable workspace cleanup"]
|
||||
APKSIG["src/apkSignature/<br/>in-process APK Signing Block + v1 reader<br/>bounded single read, no device or execution"]
|
||||
@@ -37,13 +34,8 @@ graph TD
|
||||
HOPPER["src/hopper/<br/>Hopper launch & Unix-socket protocol<br/>BridgeLauncher, HopperClient, protocol"]
|
||||
GHIDRA["src/ghidra/<br/>Ghidra discovery, profile & function analysis<br/>snapshot, Unix/Windows transport, bounded lifecycle"]
|
||||
IDA["src/ida/<br/>Upstream MCP contract adaptation<br/>live GUI binding or owned headless database"]
|
||||
KUNA["src/kuna/<br/>BYO Kuna deep analysis candidate<br/>discovery reads metadata; version probe on target open"]
|
||||
IDA_MCP["External ida-pro-mcp<br/>GUI proxy or database supervisor<br/>upstream installation and engine lifecycle"]
|
||||
|
||||
KUNA["src/kuna/<br/>BYO Kuna decompiler, a deep analysis candidate<br/>discovery reads metadata only; version probe when a target opens"]
|
||||
AUX["src/artifacts/ + src/native/<br/>artifact inventory, bounded directory triage and extraction<br/>format readers and native capabilities"]
|
||||
|
||||
|
||||
BROWSER["src/browser/<br/>CDP / Playwright capture<br/>captured scripts and native module resolution<br/>execution/: coverage, listener and source attribution"]
|
||||
JS_CODEC["src/javascript/sourceMaps/<br/>pinned upstream map codec<br/>owned heap-limited Node process, complete point evidence"]
|
||||
INSPECTOR["src/inspector/<br/>passive V8 Inspector discovery and capture<br/>script and target locations"]
|
||||
@@ -55,19 +47,12 @@ graph TD
|
||||
FIRMWARE["src/firmware/<br/>Binwalk / Unblob adapters<br/>owned processes, report normalization, private extraction"]
|
||||
ANDROID["src/android/<br/>Static APK adapter<br/>owned SDK stdio, pinned headless JADX, normalization"]
|
||||
REFERENCE["src/reference/<br/>Filesystem reference-source reader<br/>bounded traversal, validation, errors"]
|
||||
|
||||
ANDROID_TOOLS["src/adb/ + src/apktool/ + src/jeb/<br/>BYO device, resource-decoding and JEB adapters<br/>owned adb and Apktool processes; JEB engine client over MCP"]
|
||||
CAPTURE_TOOLS["src/captures/ + src/pcap/ + src/signals/ + src/sqlite/<br/>BYO tshark, pcap, sigrok and SQLite child inspection<br/>selected captures, recordings and databases; owned processes"]
|
||||
NATIVE_READERS["src/go/ + src/rust/ + src/il2cpp/ + src/flutter/ + src/gpu/<br/>Bundled in-process binary metadata readers<br/>Go build info, Rust dependencies, IL2CPP, Dart build ID, GPU code objects<br/>src/compression/: shared codec-result helper for rust and gpu"]
|
||||
FORMAT_READERS["src/model/ + src/python/ + src/erlang/ + src/electron/<br/>Bundled in-process format readers<br/>GGUF and safetensors headers, Python .pyc, Erlang BEAM, Electron fuses"]
|
||||
RUNTIME["src/frida/ + src/cutter/<br/>Frida instrumentation sessions and live Cutter bridge client"]
|
||||
end
|
||||
|
||||
subgraph "Provider Process Foundation"
|
||||
PROCESS["src/process/<br/>owned process groups, private roots<br/>deadlines, bounded file reads and diagnostics, cleanup"]
|
||||
WINDOWS["src/windows/<br/>Windows host boundary adapters<br/>package loader, opaque handles, owned jobs and private runtimes"]
|
||||
WINDOWS_NATIVE["native/windows/src/<br/>Node-API implementation<br/>NTFS admission, protected DACLs and atomic Job Object assignment"]
|
||||
FILESYSTEM["src/filesystem/<br/>owned file and directory handles<br/>stable reads, staged output, caller paths"]
|
||||
end
|
||||
|
||||
subgraph "Contracts Layer"
|
||||
@@ -76,7 +61,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Domain Layer"
|
||||
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>format declarations, provider selection, Evidence, comparisons, application graphs"]
|
||||
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>evidence, comparisons, application graphs"]
|
||||
end
|
||||
|
||||
subgraph "Configuration"
|
||||
@@ -87,8 +72,8 @@ graph TD
|
||||
TESTS["tests/<br/>Vitest suite<br/>fixtures/ holds fake launcher, bridge, and CDP seams"]
|
||||
end
|
||||
|
||||
subgraph "Development Evaluation"
|
||||
EVALUATION["tests/evaluation/model-evals/<br/>Agent release evaluation<br/>session transcript scoring"]
|
||||
subgraph "Evaluation"
|
||||
EVALUATION["src/evaluation/<br/>Agent release evaluation<br/>session transcript scoring"]
|
||||
end
|
||||
|
||||
DISPATCH --> CLI
|
||||
@@ -100,7 +85,6 @@ graph TD
|
||||
MAIN --> COMPOSITION
|
||||
CLI --> COMPOSITION
|
||||
SERVER --> COMPOSITION
|
||||
COMPOSITION --> LIFETIMES
|
||||
COMPOSITION --> SESSION
|
||||
APP --> FIRMWARE
|
||||
FIRMWARE --> PROCESS
|
||||
@@ -128,7 +112,6 @@ graph TD
|
||||
COMPOSITION --> HOPPER
|
||||
COMPOSITION --> GHIDRA
|
||||
COMPOSITION --> IDA
|
||||
COMPOSITION --> KUNA
|
||||
COMPOSITION --> AUX
|
||||
IDA --> IDA_MCP
|
||||
IDA --> CONTRACTS
|
||||
@@ -151,14 +134,14 @@ graph TD
|
||||
OBJDUMP --> DOMAIN
|
||||
COMPOSITION --> RIZIN
|
||||
RIZIN --> DOMAIN
|
||||
COMPOSITION --> GDB
|
||||
CLI --> GDB
|
||||
SERVER --> GDB
|
||||
GDB --> PROCESS
|
||||
GDB --> DOMAIN
|
||||
HOPPER --> HOPPER_BRIDGE
|
||||
GHIDRA --> GHIDRA_BRIDGE
|
||||
HOPPER --> PROCESS
|
||||
GHIDRA --> PROCESS
|
||||
KUNA --> PROCESS
|
||||
GHIDRA --> PROVIDER_OWNERS
|
||||
APP --> PROVIDER_OWNERS
|
||||
PROVIDER_OWNERS --> PROCESS
|
||||
@@ -176,7 +159,6 @@ graph TD
|
||||
CONTRACTS --> DOMAIN
|
||||
HOPPER --> DOMAIN
|
||||
GHIDRA --> DOMAIN
|
||||
KUNA --> DOMAIN
|
||||
AUX --> DOMAIN
|
||||
BROWSER --> DOMAIN
|
||||
PROCESS --> DOMAIN
|
||||
@@ -193,22 +175,3 @@ graph TD
|
||||
APP -->|EVM interface port| EVM
|
||||
EVM --> PROCESS
|
||||
EVM --> DOMAIN
|
||||
COMPOSITION --> KUNA
|
||||
KUNA --> PROCESS
|
||||
KUNA --> FILESYSTEM
|
||||
KUNA --> DOMAIN
|
||||
COMPOSITION --> ANDROID_TOOLS
|
||||
ANDROID_TOOLS --> PROCESS
|
||||
ANDROID_TOOLS --> DOMAIN
|
||||
COMPOSITION --> CAPTURE_TOOLS
|
||||
CAPTURE_TOOLS --> PROCESS
|
||||
CAPTURE_TOOLS --> DOMAIN
|
||||
COMPOSITION --> NATIVE_READERS
|
||||
NATIVE_READERS --> DOMAIN
|
||||
COMPOSITION --> FORMAT_READERS
|
||||
FORMAT_READERS --> DOMAIN
|
||||
COMPOSITION --> RUNTIME
|
||||
RUNTIME --> DOMAIN
|
||||
PROCESS --> FILESYSTEM
|
||||
AUX --> FILESYSTEM
|
||||
APP --> FILESYSTEM
|
||||
@@ -1,118 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="960" height="263" viewBox="0 0 960 263" role="img" aria-labelledby="title desc">
|
||||
<title id="title">REA showcases</title>
|
||||
<desc id="desc">Three illustrations. A brick-breaker screen with a speaker on each side stands for the DX-Ball sound-pan showcase. A browser window joined by a wire to a clipboard stands for the Notion clipboard-bridge showcase. A ring of 24 dots with one orange dot around a dark centre stands for the TH04 bullet-ring showcase.</desc>
|
||||
<rect width="960" height="263" fill="#11111b"/>
|
||||
<rect x="43" y="16.5" width="262" height="235.5" fill="#181825"/>
|
||||
<rect x="354" y="14.5" width="258" height="237.5" fill="#181825"/>
|
||||
<rect x="664" y="16.5" width="260" height="235.5" fill="#181825"/>
|
||||
<g id="dx-ball">
|
||||
<rect x="108.5" y="54" width="140" height="160" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="106.85" y="52.1" width="136" height="157" fill="#181825" stroke="#cdd6f4" stroke-width="3.7"/>
|
||||
<rect x="110.9" y="55.9" width="128.2" height="149.6" fill="none" stroke="#6c7086" stroke-width="2.2"/>
|
||||
<rect x="112.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="145.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="178.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="211.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="112.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="145.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="178.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="211.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="111.8" y="57.900000000000006" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="144.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="177.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="210.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="111.8" y="75.0" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="144.79999999999998" y="75.0" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="177.79999999999998" y="75.0" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<rect x="210.79999999999998" y="75.0" width="26.2" height="11.9" fill="#45475a" stroke="#cdd6f4" stroke-width="2.4"/>
|
||||
<ellipse cx="173.4" cy="157.2" rx="8.4" ry="8.2" fill="#000000" fill-opacity="0.45"/>
|
||||
<ellipse cx="171.9" cy="154.5" rx="7.3" ry="7.2" fill="#585b70" stroke="#cdd6f4" stroke-width="2"/>
|
||||
<rect x="165.5" y="149.5" width="3.2" height="2.6" fill="#a6adc8"/>
|
||||
<rect x="145.5" y="195.2" width="57.6" height="12.4" fill="#585b70" stroke="#cdd6f4" stroke-width="2.2"/>
|
||||
<rect x="144.5" y="204" width="59.3" height="6.5" fill="#cdd6f4"/>
|
||||
<g transform="translate(0 0)">
|
||||
<path d="M61.5 117.5H69.6L78 108.5V141.2L69.6 132.8H61.5Z" fill="#585b70" stroke="#cdd6f4" stroke-width="2.3" stroke-linejoin="miter"/>
|
||||
<path d="M78.1 106V143" stroke="#cdd6f4" stroke-width="3.4" fill="none"/>
|
||||
<g fill="none" stroke="#89b4fa" stroke-width="2.2" stroke-linecap="butt" stroke-dasharray="2.6 1.4">
|
||||
<path d="M84.8 118.2Q87.6 125 84.8 131.8"/>
|
||||
<path d="M90.2 112.4Q94.5 125 90.2 137.6"/>
|
||||
<path d="M95.8 108.2Q101.5 125 95.8 141.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(190 0)">
|
||||
<path d="M61.5 117.5H69.6L78 108.5V141.2L69.6 132.8H61.5Z" fill="#585b70" stroke="#cdd6f4" stroke-width="2.3" stroke-linejoin="miter"/>
|
||||
<path d="M78.1 106V143" stroke="#cdd6f4" stroke-width="3.4" fill="none"/>
|
||||
<g fill="none" stroke="#89b4fa" stroke-width="2.2" stroke-linecap="butt" stroke-dasharray="2.6 1.4">
|
||||
<path d="M84.8 118.2Q87.6 125 84.8 131.8"/>
|
||||
<path d="M90.2 112.4Q94.5 125 90.2 137.6"/>
|
||||
<path d="M95.8 108.2Q101.5 125 95.8 141.8"/>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="clipboard-bridge">
|
||||
<rect x="390.3" y="83.6" width="100.6" height="82.6" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="387.7" y="81.2" width="97.8" height="79.7" fill="#181825" stroke="#cdd6f4" stroke-width="2.9"/>
|
||||
<rect x="389.2" y="82.7" width="94.9" height="13" fill="#45475a"/>
|
||||
<path d="M387.7 97H485.5" stroke="#cdd6f4" stroke-width="3.1"/>
|
||||
<rect x="394.7" y="86.6" width="4.6" height="5" fill="#fab387"/>
|
||||
<rect x="402.7" y="86.6" width="4.6" height="5" fill="#f9e2af"/>
|
||||
<rect x="410.8" y="86.6" width="4.6" height="5" fill="#a6adc8"/>
|
||||
<rect x="398" y="107.4" width="10.6" height="9.4" fill="#7f849c"/>
|
||||
<rect x="412.4" y="107.9" width="16.8" height="6" fill="#7f849c"/>
|
||||
<rect x="434" y="107.9" width="42.4" height="6.2" fill="#7f849c"/>
|
||||
<rect x="412.2" y="117.3" width="64" height="2.6" fill="#6c7086"/>
|
||||
<rect x="412.2" y="124.9" width="55.3" height="3.6" fill="#6c7086"/>
|
||||
<rect x="412.2" y="133.2" width="50.3" height="3.8" fill="#6c7086"/>
|
||||
<rect x="412.2" y="142" width="13.2" height="3.4" fill="#6c7086"/>
|
||||
<rect x="430" y="142" width="14.4" height="3.4" fill="#6c7086"/>
|
||||
<rect x="449" y="142" width="27.6" height="3.2" fill="#313244"/>
|
||||
<rect x="412.2" y="150" width="12.6" height="2.9" fill="#6c7086"/>
|
||||
<path d="M455.2 162V187H476M497 187H508.2V163.2H523" fill="none" stroke="#89b4fa" stroke-width="3" stroke-linejoin="miter"/>
|
||||
<rect x="477.5" y="178.3" width="23" height="21" fill="#000000" fill-opacity="0.45"/>
|
||||
<path d="M484.7 173.2V177" stroke="#cdd6f4" stroke-width="2.2"/>
|
||||
<rect x="476.4" y="177.5" width="20.2" height="18.1" fill="#585b70" stroke="#cdd6f4" stroke-width="2.2"/>
|
||||
<rect x="481.4" y="178.6" width="4.6" height="16" fill="#313244"/>
|
||||
<rect x="477.2" y="191" width="4.6" height="4.4" fill="#f9e2af"/>
|
||||
<rect x="481.6" y="183.8" width="3.4" height="3" fill="#fab387"/>
|
||||
<rect x="525.4" y="114.6" width="60.6" height="75" fill="#000000" fill-opacity="0.45"/>
|
||||
<rect x="524.7" y="112.5" width="56.4" height="69.4" fill="#45475a" stroke="#cdd6f4" stroke-width="3"/>
|
||||
<rect x="531.6" y="118.6" width="42.3" height="53.4" fill="#1e1e2e"/>
|
||||
<rect x="531.6" y="172" width="42.3" height="8.4" fill="#45475a"/>
|
||||
<path d="M531.6 175.8H575" stroke="#cdd6f4" stroke-width="1.8" stroke-opacity="0.8"/>
|
||||
<path d="M547.4 103.3H556.6V106.2H559.6V109.2H564.9V120.3H540.1V109.2H545V106.2H547.4Z" fill="#585b70" stroke="#cdd6f4" stroke-width="2" stroke-linejoin="miter"/>
|
||||
<rect x="549.4" y="108.4" width="2.8" height="1.9" fill="#181825"/>
|
||||
<rect x="536.6" y="131.8" width="32.3" height="3.4" fill="#7f849c"/>
|
||||
<rect x="536.6" y="138.8" width="32.3" height="3.4" fill="#7f849c"/>
|
||||
<rect x="536.6" y="145.8" width="32.3" height="3.4" fill="#7f849c"/>
|
||||
<rect x="536.6" y="152.8" width="26.8" height="3.4" fill="#7f849c"/>
|
||||
<rect x="536.6" y="159.8" width="14.6" height="3.4" fill="#7f849c"/>
|
||||
</g>
|
||||
<g id="bullet-ring">
|
||||
<circle cx="783" cy="74.5" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="798" cy="72" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="812.5" cy="74.5" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="769.5" cy="81" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="826.5" cy="81" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="756.5" cy="90" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="839" cy="90" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="746.5" cy="103" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="849" cy="103" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="740.5" cy="117" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="854.5" cy="117" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="738" cy="134" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="856.5" cy="133.5" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="741" cy="149" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="854" cy="149" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="747.5" cy="163" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="848" cy="163" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="757.5" cy="175" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="770" cy="184" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="826.5" cy="184" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="838.5" cy="175" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="784" cy="191" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="798" cy="192.5" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="812" cy="190.5" r="3.8" fill="#89b4fa"/>
|
||||
<circle cx="833.5" cy="109" r="4.1" fill="#fab387"/>
|
||||
<circle cx="797.4" cy="133" r="10.9" fill="#cdd6f4"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 8.7 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 48 KiB |
@@ -1,118 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="960" height="263" viewBox="0 0 960 263" role="img" aria-labelledby="title desc">
|
||||
<title id="title">REA showcases</title>
|
||||
<desc id="desc">Three illustrations. A brick-breaker screen with a speaker on each side stands for the DX-Ball sound-pan showcase. A browser window joined by a wire to a clipboard stands for the Notion clipboard-bridge showcase. A ring of 24 dots with one orange dot around a dark centre stands for the TH04 bullet-ring showcase.</desc>
|
||||
<rect width="960" height="263" fill="#fafbfb"/>
|
||||
<rect x="43" y="16.5" width="262" height="235.5" fill="#f4f6f5"/>
|
||||
<rect x="354" y="14.5" width="258" height="237.5" fill="#f4f6f5"/>
|
||||
<rect x="664" y="16.5" width="260" height="235.5" fill="#f4f6f5"/>
|
||||
<g id="dx-ball">
|
||||
<rect x="108.5" y="54" width="140" height="160" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="106.85" y="52.1" width="136" height="157" fill="#f4f6f5" stroke="#191a1d" stroke-width="3.7"/>
|
||||
<rect x="110.9" y="55.9" width="128.2" height="149.6" fill="none" stroke="#a5a6a9" stroke-width="2.2"/>
|
||||
<rect x="112.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="145.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="178.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="211.6" y="58.5" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="112.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="145.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="178.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="211.6" y="77.39999999999999" width="28.4" height="14.3" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="111.8" y="57.900000000000006" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="144.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="177.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="210.79999999999998" y="57.900000000000006" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="111.8" y="75.0" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="144.79999999999998" y="75.0" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="177.79999999999998" y="75.0" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<rect x="210.79999999999998" y="75.0" width="26.2" height="11.9" fill="#5a636d" stroke="#191a1d" stroke-width="2.4"/>
|
||||
<ellipse cx="173.4" cy="157.2" rx="8.4" ry="8.2" fill="#000000" fill-opacity="0.27"/>
|
||||
<ellipse cx="171.9" cy="154.5" rx="7.3" ry="7.2" fill="#4d5863" stroke="#191a1d" stroke-width="2"/>
|
||||
<rect x="165.5" y="149.5" width="3.2" height="2.6" fill="#aeb1b4"/>
|
||||
<rect x="145.5" y="195.2" width="57.6" height="12.4" fill="#525c65" stroke="#191a1d" stroke-width="2.2"/>
|
||||
<rect x="144.5" y="204" width="59.3" height="6.5" fill="#191a1d"/>
|
||||
<g transform="translate(0 0)">
|
||||
<path d="M61.5 117.5H69.6L78 108.5V141.2L69.6 132.8H61.5Z" fill="#56616b" stroke="#191a1d" stroke-width="2.3" stroke-linejoin="miter"/>
|
||||
<path d="M78.1 106V143" stroke="#191a1d" stroke-width="3.4" fill="none"/>
|
||||
<g fill="none" stroke="#274b77" stroke-width="2.2" stroke-linecap="butt" stroke-dasharray="2.6 1.4">
|
||||
<path d="M84.8 118.2Q87.6 125 84.8 131.8"/>
|
||||
<path d="M90.2 112.4Q94.5 125 90.2 137.6"/>
|
||||
<path d="M95.8 108.2Q101.5 125 95.8 141.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(190 0)">
|
||||
<path d="M61.5 117.5H69.6L78 108.5V141.2L69.6 132.8H61.5Z" fill="#56616b" stroke="#191a1d" stroke-width="2.3" stroke-linejoin="miter"/>
|
||||
<path d="M78.1 106V143" stroke="#191a1d" stroke-width="3.4" fill="none"/>
|
||||
<g fill="none" stroke="#274b77" stroke-width="2.2" stroke-linecap="butt" stroke-dasharray="2.6 1.4">
|
||||
<path d="M84.8 118.2Q87.6 125 84.8 131.8"/>
|
||||
<path d="M90.2 112.4Q94.5 125 90.2 137.6"/>
|
||||
<path d="M95.8 108.2Q101.5 125 95.8 141.8"/>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="clipboard-bridge">
|
||||
<rect x="390.3" y="83.6" width="100.6" height="82.6" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="387.7" y="81.2" width="97.8" height="79.7" fill="#f4f6f5" stroke="#191a1d" stroke-width="2.9"/>
|
||||
<rect x="389.2" y="82.7" width="94.9" height="13" fill="#5a636d"/>
|
||||
<path d="M387.7 97H485.5" stroke="#191a1d" stroke-width="3.1"/>
|
||||
<rect x="394.7" y="86.6" width="4.6" height="5" fill="#c3693b"/>
|
||||
<rect x="402.7" y="86.6" width="4.6" height="5" fill="#be9a53"/>
|
||||
<rect x="410.8" y="86.6" width="4.6" height="5" fill="#b3b3b4"/>
|
||||
<rect x="398" y="107.4" width="10.6" height="9.4" fill="#666671"/>
|
||||
<rect x="412.4" y="107.9" width="16.8" height="6" fill="#666c74"/>
|
||||
<rect x="434" y="107.9" width="42.4" height="6.2" fill="#666c74"/>
|
||||
<rect x="412.2" y="117.3" width="64" height="2.6" fill="#7a7c82"/>
|
||||
<rect x="412.2" y="124.9" width="55.3" height="3.6" fill="#7a7c82"/>
|
||||
<rect x="412.2" y="133.2" width="50.3" height="3.8" fill="#6d7279"/>
|
||||
<rect x="412.2" y="142" width="13.2" height="3.4" fill="#6d7279"/>
|
||||
<rect x="430" y="142" width="14.4" height="3.4" fill="#6d7279"/>
|
||||
<rect x="449" y="142" width="27.6" height="3.2" fill="#e8e8e8"/>
|
||||
<rect x="412.2" y="150" width="12.6" height="2.9" fill="#6d7279"/>
|
||||
<path d="M455.2 162V187H476M497 187H508.2V163.2H523" fill="none" stroke="#274b77" stroke-width="3" stroke-linejoin="miter"/>
|
||||
<rect x="477.5" y="178.3" width="23" height="21" fill="#000000" fill-opacity="0.27"/>
|
||||
<path d="M484.7 173.2V177" stroke="#191a1d" stroke-width="2.2"/>
|
||||
<rect x="476.4" y="177.5" width="20.2" height="18.1" fill="#525c65" stroke="#191a1d" stroke-width="2.2"/>
|
||||
<rect x="481.4" y="178.6" width="4.6" height="16" fill="#323435"/>
|
||||
<rect x="477.2" y="191" width="4.6" height="4.4" fill="#be9a53"/>
|
||||
<rect x="481.6" y="183.8" width="3.4" height="3" fill="#c3693b"/>
|
||||
<rect x="525.4" y="114.6" width="60.6" height="75" fill="#000000" fill-opacity="0.27"/>
|
||||
<rect x="524.7" y="112.5" width="56.4" height="69.4" fill="#545e69" stroke="#191a1d" stroke-width="3"/>
|
||||
<rect x="531.6" y="118.6" width="42.3" height="53.4" fill="#ededec"/>
|
||||
<rect x="531.6" y="172" width="42.3" height="8.4" fill="#545e69"/>
|
||||
<path d="M531.6 175.8H575" stroke="#191a1d" stroke-width="1.8" stroke-opacity="0.8"/>
|
||||
<path d="M547.4 103.3H556.6V106.2H559.6V109.2H564.9V120.3H540.1V109.2H545V106.2H547.4Z" fill="#545e69" stroke="#191a1d" stroke-width="2" stroke-linejoin="miter"/>
|
||||
<rect x="549.4" y="108.4" width="2.8" height="1.9" fill="#f4f6f5"/>
|
||||
<rect x="536.6" y="131.8" width="32.3" height="3.4" fill="#706f74"/>
|
||||
<rect x="536.6" y="138.8" width="32.3" height="3.4" fill="#706f74"/>
|
||||
<rect x="536.6" y="145.8" width="32.3" height="3.4" fill="#706f74"/>
|
||||
<rect x="536.6" y="152.8" width="26.8" height="3.4" fill="#706f74"/>
|
||||
<rect x="536.6" y="159.8" width="14.6" height="3.4" fill="#706f74"/>
|
||||
</g>
|
||||
<g id="bullet-ring">
|
||||
<circle cx="783" cy="74.5" r="3.8" fill="#274b77"/>
|
||||
<circle cx="798" cy="72" r="3.8" fill="#274b77"/>
|
||||
<circle cx="812.5" cy="74.5" r="3.8" fill="#274b77"/>
|
||||
<circle cx="769.5" cy="81" r="3.8" fill="#274b77"/>
|
||||
<circle cx="826.5" cy="81" r="3.8" fill="#274b77"/>
|
||||
<circle cx="756.5" cy="90" r="3.8" fill="#274b77"/>
|
||||
<circle cx="839" cy="90" r="3.8" fill="#274b77"/>
|
||||
<circle cx="746.5" cy="103" r="3.8" fill="#274b77"/>
|
||||
<circle cx="849" cy="103" r="3.8" fill="#274b77"/>
|
||||
<circle cx="740.5" cy="117" r="3.8" fill="#274b77"/>
|
||||
<circle cx="854.5" cy="117" r="3.8" fill="#274b77"/>
|
||||
<circle cx="738" cy="134" r="3.8" fill="#274b77"/>
|
||||
<circle cx="856.5" cy="133.5" r="3.8" fill="#274b77"/>
|
||||
<circle cx="741" cy="149" r="3.8" fill="#274b77"/>
|
||||
<circle cx="854" cy="149" r="3.8" fill="#274b77"/>
|
||||
<circle cx="747.5" cy="163" r="3.8" fill="#274b77"/>
|
||||
<circle cx="848" cy="163" r="3.8" fill="#274b77"/>
|
||||
<circle cx="757.5" cy="175" r="3.8" fill="#274b77"/>
|
||||
<circle cx="770" cy="184" r="3.8" fill="#274b77"/>
|
||||
<circle cx="826.5" cy="184" r="3.8" fill="#274b77"/>
|
||||
<circle cx="838.5" cy="175" r="3.8" fill="#274b77"/>
|
||||
<circle cx="784" cy="191" r="3.8" fill="#274b77"/>
|
||||
<circle cx="798" cy="192.5" r="3.8" fill="#274b77"/>
|
||||
<circle cx="812" cy="190.5" r="3.8" fill="#274b77"/>
|
||||
<circle cx="833.5" cy="109" r="4.1" fill="#c3693b"/>
|
||||
<circle cx="797.4" cy="133" r="10.9" fill="#1c222a"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 8.7 KiB |
@@ -8,14 +8,9 @@ ordinary binary inspection use the same modular capability.
|
||||
|
||||
## Bring your own engine
|
||||
|
||||
Real verification covers **Linux x64** hosts and **EXEC/DYN/REL** objects in
|
||||
these profiles: ELF64 x86-64, ELF64 little-endian AArch64, ELF32 i386, ELF32
|
||||
little-endian ARM and ELF32 little- or big-endian MIPS. Other machines, classes
|
||||
and byte orders, including big-endian ARM, return `unsupported_target`. Other
|
||||
hosts, including Linux arm64, return `capability_unavailable` naming the
|
||||
detected host before any target is read. Provide an absolute
|
||||
Python executable whose environment already contains unchanged pwntools 4.15.0,
|
||||
pyelftools 0.33 and Unicorn 2.1.2:
|
||||
Initial real verification covers **Linux x64, ELF64 x86-64 little-endian
|
||||
EXEC/DYN/REL**. Provide an absolute Python executable whose environment already
|
||||
contains unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2:
|
||||
|
||||
```sh
|
||||
REA_PWNTOOLS_PYTHON=/absolute/isolated-env/bin/python \
|
||||
@@ -34,16 +29,6 @@ the selected object as a host process, or requests runtime library resolution. T
|
||||
uses isolated mode and an owned cache. Exact upstream profiles are recorded in
|
||||
[upstream provenance](https://github.com/morluto/rea/blob/main/third_party/pwntools/README.md).
|
||||
|
||||
The adapter skips pwntools kernel configuration and version discovery: those
|
||||
memory-content scans supply none of this report's fields. Ordinary `IKCFG_ST`
|
||||
data is not interpreted as a kernel-config request, and virtual zero-filled
|
||||
LOAD tails are reported from their headers without materializing them for that
|
||||
scan. GOT/PLT inference and the advertised static mitigation indicators still
|
||||
use the pinned upstream implementations. Header-table file ranges are checked
|
||||
before dependency seeks. Structural inventory and the convenience-map engine
|
||||
still parse the snapshot separately; eliminating that remaining duplication
|
||||
requires preserving the upstream GOT/PLT and mitigation facts.
|
||||
|
||||
## Interpreting results
|
||||
|
||||
- Artifact path, SHA-256 and size identify original selected bytes. All parsing
|
||||
@@ -105,12 +90,6 @@ one section or symbol, or a stable page without re-running the decoder. CLI
|
||||
one-shot invocations pass portable inline Evidence JSON; an MCP session uses
|
||||
the retained `evidence_id`. Page `limit` is required and bounded by the
|
||||
measured MCP stdio budget documented on the tool contract.
|
||||
An empty retained section or symbol collection has no valid item index. Its page
|
||||
view can still return an empty page.
|
||||
Malformed authenticated layout input remains rejected with the owning schema's
|
||||
safe field diagnostics under `source.normalized_result`, including the field
|
||||
path and type or value reason. Evidence authentication and decoder validation
|
||||
remain separate boundaries.
|
||||
|
||||
Over MCP, `inspect_binary_layout` with `"detail": "summary"` retains the
|
||||
complete layout Evidence in the session and returns only its summary view.
|
||||
@@ -120,30 +99,23 @@ CLI output are unchanged.
|
||||
|
||||
Complete results have a 32 MiB input, 64 MiB reply, 1 MiB combined diagnostics
|
||||
and 30-second owned command deadline. `REA_PWNTOOLS_MAX_INPUT_BYTES` raises the
|
||||
launch input default alone, up to 1 GiB. Per-call `producer_limits` selects
|
||||
`input_bytes`, `output_bytes`, `diagnostic_bytes`, `address_space_bytes`,
|
||||
`file_bytes`, and `cpu_seconds`; positive safe integers apply on the next call
|
||||
without restart. `input_bytes` overrides the launch default. File admission
|
||||
follows selected `output_bytes` unless `file_bytes` is explicit. The separate
|
||||
`pwntools_timeout_ms` controls each command wall deadline. On CLI pass
|
||||
`--producer-limits '{"cpu_seconds":60,"output_bytes":134217728}'`. The
|
||||
input bound alone, up to 1 GiB; the reply and deadline bounds are unchanged. The
|
||||
reply budget applies to the decoder record; the CLI/MCP Evidence envelope adds
|
||||
copies and encoding overhead. The
|
||||
pinned MCP SDK defaults to a 10 MiB receive buffer. Large complete results need
|
||||
a caller-configured `StdioClientTransport({ maxBufferSize: ... })` and a matching
|
||||
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](../../reference/mcp-contracts.md#tool-results).
|
||||
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](mcp-contracts.md#tool-results).
|
||||
These settings affect delivery, not decoder coverage or limits. The large
|
||||
extended-index verification uses 256 MiB and an explicit five-minute request
|
||||
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to the selected policy: defaults are 3 GiB
|
||||
virtual address space, 30 CPU seconds and 64 MiB file output. Inherited tighter soft/hard limits are
|
||||
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to at most 3 GiB virtual address space,
|
||||
30 CPU seconds and 64 MiB file output. Inherited tighter soft/hard limits are
|
||||
retained, and the effective values are returned in the result limitations. Virtual address space is not RSS;
|
||||
Unicorn needs a large virtual map. Limits fail with no partial success, and
|
||||
owned process/root cleanup completes independently of cancellation.
|
||||
|
||||
For core files and optional pwndbg mapping context, use the
|
||||
[recorded-crash workflow](recorded-crashes.md). The binary-layout operations
|
||||
described here do not provide debugger session state or control, or establish
|
||||
support for other operating systems/architectures.
|
||||
Core files, GDB session/state/control, optional pwndbg enrichment and instruction
|
||||
inspection are separate increments tracked by #969. This increment makes no
|
||||
real support claim for those features or other operating systems/architectures.
|
||||
|
||||
Typed decoder failures retain bounded `captured_output` (stdout, stderr and an
|
||||
explicit truncation flag) alongside the original failure category and reason.
|
||||
@@ -198,31 +170,6 @@ subject to earlier cancellation or the startup deadline. `output_incomplete`
|
||||
marks a snapshot taken before stream closure; cleanup ownership remains with
|
||||
the session until release succeeds.
|
||||
|
||||
A kernel or container OOM killer ends Ghidra with `SIGKILL` and prints no
|
||||
`OutOfMemoryError`. When a process or protocol failure records
|
||||
`exit_signal: "SIGKILL"` and no `stop_requested`, REA reports the signal
|
||||
instead of generic `rea doctor` advice. The process supervisor records
|
||||
`stop_requested: true` once REA begins stopping Ghidra, before it sends any
|
||||
signal, so REA's own SIGKILL is never reported as an outside kill.
|
||||
|
||||
On Linux with cgroup v2, REA also reads its own cgroup's `memory.events` before
|
||||
launch and again after the kill, locating the group through the cgroup2 mount
|
||||
in `/proc/self/mountinfo`. `memory_cgroup` records the group path,
|
||||
`oom_kill_before`, `oom_kill_after` and `memory_max`. If the count rose, REA
|
||||
returns `resource_constraint` with `resource: "memory"` and `cgroup_memory_max`
|
||||
in `reported_limits`. The count covers every process in REA's cgroup and its
|
||||
child groups, which can include processes outside REA's process tree when the
|
||||
group is shared, so a rise shows that the kernel OOM killer ended a process in
|
||||
that cgroup or a child group, not which process it chose. `memory_max` is the
|
||||
limit in bytes, `"max"` when the group sets none, or `null` when it could not be
|
||||
read; a limit on a parent group is not read. The limit also covers Ghidra's
|
||||
native decompiler processes, whose number follows the available CPUs, so the
|
||||
advice does not recommend a larger Java heap. When the session had already
|
||||
completed an operation, the advice keeps the instruction to reopen the target
|
||||
with `open_binary`. Without a cgroup v2 memory group, or when the count did not
|
||||
rise, the failure stays `execution_failure` and names the signal as a possible
|
||||
out-of-memory kill.
|
||||
|
||||
## Portable PE resources
|
||||
|
||||
`inspect_pe_resources` / `inspect-pe-resources` inspects an explicit PE32 or
|
||||
@@ -246,14 +193,6 @@ reported. Directory headers, entries, data entries and payloads retain original
|
||||
file offsets and lengths. Each opaque payload has its own SHA-256; code page and
|
||||
reserved data-entry fields are preserved.
|
||||
|
||||
Every `directories[]` row retains its identity `path` and the matching incoming
|
||||
`entry_locations`: both are empty for the root, then contain the type and
|
||||
optionally the name. Empty branches keep these identities even when they have
|
||||
no payload leaves. Shared directory bytes produce a row for each incoming path,
|
||||
with the original directory range and each path's own entry locations. These
|
||||
rows are included in the metadata budget. A present empty root remains distinct
|
||||
from an absent resource data directory (`directory: null`, `directories: []`).
|
||||
|
||||
RT_GROUP_ICON records expose original image descriptors and RT_ICON candidates.
|
||||
A reference resolves only when an icon with the same numeric ID and language
|
||||
exists. Missing references, other-language candidates and declared-size mismatch
|
||||
@@ -279,12 +218,6 @@ regular files only and rejects symbolic links.
|
||||
|
||||
## Byte entropy triage
|
||||
|
||||
Entropy admission preserves the caller's spelling in `caller_input_path`. On
|
||||
Windows, the selected regular file's case-only filesystem spelling is used
|
||||
consistently in committed paths through both CLI and MCP. Symlink, junction and
|
||||
short-name aliases are not replaced with a different path. General path
|
||||
normalization remains lexical and does not probe source-discovered paths.
|
||||
|
||||
`inspect_byte_entropy` / `inspect-byte-entropy` profiles any local regular file
|
||||
without an analysis engine, execution, extraction or network access.
|
||||
|
||||
@@ -385,48 +318,7 @@ rea inspect-native-exports ./runtimes/linux-x64/native/libexample.so --json
|
||||
```
|
||||
|
||||
The complete Evidence identifies the stable artifact snapshot by path, byte
|
||||
count and SHA-256. Complete delivery remains the default. For a small initial
|
||||
MCP reply, select `detail: "summary"`; this retains the complete Evidence and
|
||||
returns artifact identity, table coverage and row counts. Use its
|
||||
`normalized_result.parent_evidence_id` with `inspect_analysis_view`:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_<parent digest>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "native-exports",
|
||||
"collection": "exports",
|
||||
"names": ["selected_name"],
|
||||
"fields": ["name"],
|
||||
"offset": 0,
|
||||
"limit": 32
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Names match exactly and preserve every same-name version or alias. Optional
|
||||
`indices` selects original row positions, intersecting a name filter when both
|
||||
are supplied; empty filters select nothing. `source_index` always remains and,
|
||||
with the parent ID, identifies the original export or issue row. Fields default
|
||||
to the complete row; an empty field list returns only its identity. ELF `name`
|
||||
selection also returns raw-byte and encoding qualifications. Fields from the
|
||||
other format are rejected. `collection: "issues"` pages the complete diagnostic
|
||||
records, with optional original indices. Offset and limit default to 0 and 64
|
||||
and apply to matching rows in original order.
|
||||
|
||||
View `coverage` describes the selected page; `summary.coverage` or
|
||||
`exports.coverage` preserves source-table coverage. An exhausted or empty page
|
||||
does not make a partial source complete. Retained views never reopen the file.
|
||||
The CLI uses the same projector and requires a durable bundle for a summary:
|
||||
|
||||
```sh
|
||||
rea inspect-native-exports ./libexample.so --detail summary --evidence_output ./exports.bundle.json --json
|
||||
# Pass the summary's parent ID and the same view as JSON to inspect-analysis-view,
|
||||
# together with --evidence-bundle ./exports.bundle.json.
|
||||
```
|
||||
count and SHA-256.
|
||||
|
||||
- PE rows reproduce the export directory: each used export-address-table slot
|
||||
with its biased `ordinal` (`OrdinalBase` plus the slot index), one row per
|
||||
@@ -439,10 +331,8 @@ rea inspect-native-exports ./libexample.so --detail summary --evidence_output ./
|
||||
without an export directory is a complete empty inventory.
|
||||
- ELF rows list defined `.dynsym` symbols with global, weak or GNU-unique
|
||||
binding and default or protected visibility. Each row keeps `value`
|
||||
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`,
|
||||
`binding` and `visibility` (`"default"` or `"protected"`; `"unknown"` in older
|
||||
historical Evidence where visibility was not recorded). ELF string tables do
|
||||
not declare a text encoding. Names that
|
||||
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`
|
||||
and `binding`. ELF string tables do not declare a text encoding. Names that
|
||||
contain non-ASCII bytes are decoded as UTF-8 when those bytes form valid
|
||||
UTF-8; `name_utf8_decodable` records whether the non-ASCII name bytes are
|
||||
valid UTF-8 and `name_bytes_base64` retains those exact bytes (`null` for
|
||||
@@ -470,13 +360,9 @@ Unresolved symbol versions keep their rows: the version name is null when no
|
||||
definition or requirement names the index, and `version` is
|
||||
`"unknown"` when the version table cannot be read; coverage is likewise
|
||||
`partial`. A dynamic symbol table without its null entry is malformed. An ELF
|
||||
without section headers is `unsupported`. With section headers, a missing
|
||||
`SHT_DYNSYM` section is also `unsupported` when a `PT_DYNAMIC` segment is
|
||||
observed, including a directory containing only the inactive null section.
|
||||
Enumerating dynamic symbols from that segment is not implemented. A present
|
||||
section directory with neither a dynamic symbol table nor a dynamic segment
|
||||
reports a complete empty inventory. Mach-O exports are reported by
|
||||
`inspect_macho`.
|
||||
without section headers is `unsupported`:
|
||||
enumerating dynamic symbols from the dynamic segment is not
|
||||
implemented. Mach-O exports are reported by `inspect_macho`.
|
||||
|
||||
`max_file_bytes` defaults to 256 MiB (maximum 512 MiB; CLI `--max-file-bytes`).
|
||||
Export directories in PE headers are read like those in sections. An export or
|
||||
@@ -1,62 +0,0 @@
|
||||
# Binary hardening
|
||||
|
||||
Use `inspect_binary_hardening` to read compile-time and link-time hardening from
|
||||
an explicit local ELF, PE, or Mach-O file. No disassembler, pwntools install, or
|
||||
target execution is required. Universal Mach-O files return one image per slice.
|
||||
|
||||
```sh
|
||||
rea inspect-binary-hardening /absolute/path/to/program --json
|
||||
rea inspect-binary-hardening /absolute/path/to/program --max-file-bytes 1048576 --json
|
||||
```
|
||||
|
||||
MCP accepts `{ "path": "/absolute/path/to/program" }`. CLI paths may be relative
|
||||
to the current directory; MCP paths must be absolute.
|
||||
|
||||
Each image has one row per property: `property`, `state`, `confidence`, `basis`,
|
||||
`location`, `note`, and `next_probe`. States are `enabled`, `disabled`,
|
||||
`partial`, `unknown`, and `not_applicable`. `confidence` is `observed` for a
|
||||
single header or import, and `derived` when the state combines several fields,
|
||||
such as ELF PIE or full RELRO. An image's `architecture` is its CPU family, or
|
||||
null when REA has no family for it. `machine` and `slice.architecture` keep the
|
||||
exact identity, such as `arm64e`.
|
||||
|
||||
ELF rows cover PIE (`ET_DYN` with `PT_INTERP` or `DF_1_PIE`), NX (`PT_GNU_STACK`;
|
||||
a missing header is `unknown` and the architecture default is only noted), RELRO
|
||||
(`PT_GNU_RELRO` plus `BIND_NOW`), stack protector and FORTIFY undefined dynamic
|
||||
imports, CET and AArch64 BTI/PAC GNU property notes, `DT_RPATH`, `DT_RUNPATH`, and
|
||||
whether a `SHT_SYMTAB` section is present. Only an undefined dynamic import
|
||||
counts. Imports come from the table `DT_SYMTAB` declares. A `SHT_DYNSYM` section
|
||||
only supplies the symbol count when `DT_HASH` and `DT_GNU_HASH` leave it open,
|
||||
and only if it names the same symbol and string tables. Without that count the
|
||||
import rows are `unknown` and give the reason. A missing `__stack_chk_fail` or fortify import is `unknown`. That
|
||||
absence does not establish that the protection was left out of a static or
|
||||
stripped build. A present import is a sign that some code uses the protection,
|
||||
not proof that all code does. NX, RELRO, `DT_RPATH`, and `DT_RUNPATH` are
|
||||
`not_applicable` when `e_type` is neither `ET_EXEC` nor `ET_DYN`, and `unknown`
|
||||
for a linked image with no readable program headers.
|
||||
|
||||
PE rows read `DllCharacteristics` for dynamic base, high-entropy VA, NX
|
||||
compatibility, force integrity, no SEH, and guard CF. `cfg_instrumented` is the
|
||||
load-config `GuardFlags` linker claim when `Size` covers that field. An
|
||||
unreadable load configuration, including a TLS failure in that parser, leaves
|
||||
only `cfg_instrumented` `unknown`. Authenticode and debug rows report certificate
|
||||
and debug directory presence, not signature validity or debug content. A
|
||||
directory whose address and size disagree is `unknown`.
|
||||
|
||||
Mach-O rows read `MH_PIE`, `MH_ALLOW_STACK_EXECUTION`, `MH_NO_HEAP_EXECUTION`,
|
||||
an `__RESTRICT` segment, `LC_CODE_SIGNATURE` presence, encryption from the
|
||||
existing encryption reader, the arm64e subtype, and a stack-protector import.
|
||||
`hardened_runtime` is `unknown` with `next_probe` `inspect_signature`.
|
||||
CodeDirectory flags stay with that tool.
|
||||
|
||||
The report states presence of build-time features. It does not state
|
||||
exploitability. `inspect_binary_layout` is unchanged.
|
||||
|
||||
The default read budget is 64 MiB (`max_file_bytes`, ceiling 512 MiB). A changed
|
||||
file is refused. A malformed header, a Mach-O segment or encryption command that fails admission, or a program, section,
|
||||
or segment payload declared outside the file returns no partial report, except that one bad slice in a universal Mach-O leaves the other slices. A symbol table or
|
||||
symbol name over its budget leaves the import rows `unknown` and keeps the
|
||||
header rows. A GNU property note over its budget, or a compressed note section
|
||||
that may hold one, leaves the CET and BTI/PAC rows `unknown` and keeps the header
|
||||
rows. A dynamic-table budget still fails the ELF report. The image is never
|
||||
executed.
|
||||
@@ -12,26 +12,19 @@ For request/response bodies and traffic debugging, see
|
||||
[Fiddler Everywhere captures and MCP](web-network-captures.md#reading-fiddler-captures-saz).
|
||||
Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate MCP server.
|
||||
|
||||
HTTP discovery and browser WebSocket inputs require an explicit port. An
|
||||
explicit default `:80` remains valid and is retained in the canonical endpoint;
|
||||
omitting the port remains invalid. HTTP `/json/list` discovery preserves usable
|
||||
siblings when optional metadata is malformed: attachment becomes unknown and
|
||||
row/field diagnostics appear in `limitations`. Target identity and socket
|
||||
authority are validated independently before selection and connection.
|
||||
|
||||
## Shipped surfaces
|
||||
|
||||
- `list_browser_targets` / `rea list-browser-targets` discovers all eligible page targets at the endpoint by default, with an optional exact-origin filter.
|
||||
- `inspect_web_page` / `rea inspect-web-page` captures DOM structure, accessibility nodes, scripts, resources, safe response/DOM metadata, attach-window network and console metadata, WebSocket frame sizes, workers, quota, and optionally selected storage key names, redacted storage-content fingerprints, or script sources.
|
||||
- `analyze_web_bundle` / `rea analyze-web-bundle` parses selected script artifacts without execution and derives chunk edges, route and endpoint candidates, vendor fingerprints, static WebMCP declarations, and optionally source-map/original-source evidence.
|
||||
- `observe_web_session` / `rea observe-web-session` arms an observation window of the requested duration for an external user action and records ordered reload, SPA navigation, redirect, failure, lifecycle, and target-termination metadata.
|
||||
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return observed in-scope registrations and their complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. An omitted optional `inputSchema` is a valid parameterless registration, represented by null schema and digest. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. Declared `readOnly`, `untrustedContent`, `consequential`, and `autosubmit` annotation hints are reported per registration: `true`/`false` are declared values, `null` means the declaration did not provide the hint in that capture, and a missing `consequential` key means the hint was not recorded, either in a historical capture or because the observed annotation was invalid. Consult discovery completeness and exclusions to distinguish these cases. Malformed annotation values or registration stacks retain the registration with incomplete coverage and an `invalid_protocol_value` exclusion. Other malformed protocol payloads can fail discovery explicitly. REA never exposes `WebMCP.invokeTool`.
|
||||
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. Declared `readOnly`, `untrustedContent`, `consequential`, and `autosubmit` annotation hints are reported per registration: `true`/`false` are declared values, `null` means the declaration did not provide the hint in that capture, and a missing `consequential` key means the hint was not recorded, either in a historical capture or because the observed annotation was invalid. Consult discovery completeness and exclusions to distinguish these cases. Malformed annotation values or registration stacks retain the registration with incomplete coverage and an `invalid_protocol_value` exclusion. Other malformed protocol payloads can fail discovery explicitly. REA never exposes `WebMCP.invokeTool`.
|
||||
|
||||
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
|
||||
|
||||
For a compact route query, select `detail: "summary"` and a
|
||||
`view: {kind: "javascript-routes", collection: "routes", limit: 20}`. The
|
||||
shared [route inventory](../../javascript-route-inventory.md) preserves source IDs,
|
||||
shared [route inventory](javascript-route-inventory.md) preserves source IDs,
|
||||
precise locations, partial coverage, and the complete parent Evidence for
|
||||
subsequent filtered pages. Static candidates never establish complete application
|
||||
coverage or reachability.
|
||||
@@ -52,11 +45,7 @@ observations may omit the coverage field; treat omission as `unknown` rather
|
||||
than inferring completeness from the array.
|
||||
|
||||
Passive DOM comparison checks complete node inventories and complete frame
|
||||
parent/child structure independently. Node value lengths are comparable only
|
||||
when both nonempty snapshots declare the same `node_value_length_unit`.
|
||||
Missing or different units leave length comparison unknown; independent node
|
||||
or frame structural differences can still prove change. Empty inventories have
|
||||
no value lengths to compare. Frame IDs and sibling enumeration order
|
||||
parent/child structure independently. Frame IDs and sibling enumeration order
|
||||
are capture-local and do not establish a change; duplicate IDs, missing parents,
|
||||
and cyclic ancestry remain unknown. A complete frame difference can still prove
|
||||
change when DOM nodes are incomplete, and vice versa. Partial DOM or metadata
|
||||
@@ -65,20 +54,6 @@ from a protocol omission. Rejected DOM facts also mark derived URL and agent-hin
|
||||
metadata incomplete, while valid facts from other nodes and documents remain
|
||||
available in the capture.
|
||||
|
||||
Page network and console coverage starts independently at the first observed
|
||||
event from that domain or its successful enable acknowledgment. The capture's
|
||||
lifecycle start is earlier and does not establish domain readiness. Events
|
||||
received before the enable reply remain part of the observation.
|
||||
|
||||
Worker output is a final target-inventory snapshot. `attached` describes any
|
||||
debugging client, not an REA-owned observing session. The selected page-session
|
||||
collector does not collect worker-session requests, console events, or scripts,
|
||||
and workers that disappeared before the snapshot may be missing; worker
|
||||
coverage is therefore attach-limited. Missing ownership metadata is reported as
|
||||
`unattributed_target` with unavailable worker attribution, separately
|
||||
from a proven foreign owner excluded as `out_of_target_scope`. A shared origin
|
||||
alone establishes neither relationship.
|
||||
|
||||
Passive comparison keeps lookup identity, cross-capture correspondence, content
|
||||
evidence, and inventory coverage separate:
|
||||
|
||||
@@ -98,29 +73,12 @@ content. Actual JSON properties with those names remain content. Reference-only
|
||||
objects, unsupported values, and opaque cookie partition identities remain
|
||||
incomplete. Previously exported fingerprints are not recomputed; recapture both
|
||||
sides when comparing captures produced across this decoder change.
|
||||
IndexedDB schema fingerprints with `projection_version: 2` include the observed
|
||||
database version and object-store definitions. A missing, malformed, or imprecise
|
||||
version leaves the schema fingerprint incomplete while retaining observed store
|
||||
facts. Historical fingerprints without this marker covered object stores only:
|
||||
matching historical hashes cannot establish unchanged database versions, and a
|
||||
historical hash is not compared with a version-bearing hash. Recapture both
|
||||
sides for a complete IndexedDB version comparison; other proven storage changes
|
||||
remain reportable.
|
||||
|
||||
Script comparison groups observations by reported URL and compares unordered
|
||||
content samples. Producer inventory keys include optional metadata, so a changed
|
||||
key cannot by itself prove script removal or addition. Missing module, length,
|
||||
language or source-hash facts remain unknown; shared known values and complete
|
||||
retained source digests can still prove a group change. Partial inventories use
|
||||
the same omission rule as repeated HTTP samples. WebMCP dropped-tool counts also
|
||||
prevent equality, and two incomplete owner inventories cannot prove tool
|
||||
correspondence merely from their different retained registrations.
|
||||
|
||||
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
|
||||
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
|
||||
- The CLI comparison commands accept each capture, screenshot artifact or normalization policy as inline JSON or a JSON file path. Use files for screenshot artifacts and scenario captures, whose embedded PNG bytes can exceed the host's command-line length limit.
|
||||
- Every surface has equivalent CLI and MCP contracts and returns Evidence provenance.
|
||||
- By default, MCP tools return complete results inline and include the Evidence ID. Selected summary views retain the complete parent Evidence and return a projection. Session evidence can be exported with `export_evidence_bundle`.
|
||||
- MCP tools return complete results inline and include the Evidence ID. Session evidence can be exported with `export_evidence_bundle`.
|
||||
|
||||
Target discovery returns the complete in-scope target array in one result.
|
||||
|
||||
@@ -147,7 +105,7 @@ Caller-owned JSON/base64 strings, input parsing and validation, transport
|
||||
buffers, garbage-collector behavior, and other runtime memory are outside this
|
||||
estimate; the budget does not promise a bound on total process memory.
|
||||
|
||||
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](../javascript/electron-observation.md).
|
||||
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](electron-observation.md).
|
||||
|
||||
## Provider authorities
|
||||
|
||||
@@ -159,11 +117,6 @@ preserve these authority and lifecycle differences. They share one exclusion
|
||||
vocabulary (`BrowserExclusionReason`): every denial reason maps onto it, even
|
||||
where a wire schema keeps a narrower historical bucket.
|
||||
|
||||
Passive page and Inspector capture enable the `Debugger` domain. A target's
|
||||
`debugger` statement can then pause execution; these observation paths do not
|
||||
suppress pauses with `Debugger.setSkipAllPauses` or resume the target. Their
|
||||
lack of explicit pause commands does not guarantee uninterrupted execution.
|
||||
|
||||
CDP attachment is not a guarantee that the browser preserves all state.
|
||||
Chromium can clear another client's `navigator.platform` emulation when REA's
|
||||
page session disconnects, even if that session sent no emulation commands.
|
||||
@@ -177,7 +130,7 @@ See [the reproduction and Chromium ownership limitation](https://github.com/morl
|
||||
|
||||
Existing static application Evidence and passive web/Electron captures can be
|
||||
combined later through
|
||||
[JavaScript static/runtime reconciliation](../javascript/javascript-runtime-reconciliation.md).
|
||||
[JavaScript static/runtime reconciliation](javascript-runtime-reconciliation.md).
|
||||
|
||||
## Start a browser
|
||||
|
||||
@@ -228,7 +181,7 @@ in an existing profile.
|
||||
For such a browser, pass the browser socket instead of the HTTP endpoint:
|
||||
|
||||
```bash
|
||||
rea list-browser-targets "ws://127.0.0.1:9222/devtools/browser/<id>"
|
||||
rea list-browser-targets ws://127.0.0.1:9222/devtools/browser/<id>
|
||||
```
|
||||
|
||||
Chrome writes the port and the `/devtools/browser/<id>` path to the
|
||||
@@ -239,21 +192,10 @@ file for you.
|
||||
`observe_web_session`, `discover_webmcp_tools`, and `capture_web_screenshot`
|
||||
accept a browser socket. Website runtime attribution, Electron inspection,
|
||||
and scenario attach mode still require HTTP discovery.
|
||||
- An admitted browser socket is retained by the active browser provider across
|
||||
calls to the same normalized browser WebSocket URL, and concurrent cold-start
|
||||
calls share one handshake so MCP sessions do not re-open the browser socket on
|
||||
every step. Scheme case, uncompressed IPv6 loopback spellings, and leading
|
||||
port zeros normalize to one reuse key, while `127.0.0.1`, `[::1]`, and
|
||||
distinct `/devtools/browser/<id>` paths remain separate sockets. Each target
|
||||
operation detaches its flattened target session when finished: unconfirmed
|
||||
target attach or detach cleanup closes the shared browser socket for session
|
||||
containment, whereas a caller's own discovery deadline or cancellation
|
||||
releases only its lease when transport state remains clean. Provider shutdown
|
||||
closes the retained browser socket. One-shot CLI invocations close their
|
||||
provider on exit, so Chrome may still ask you to approve each separate CLI
|
||||
connection. Discovery (`Browser.getVersion`, `Target.getTargets`) defaults to
|
||||
a 10-second deadline and can be adjusted per call with `discovery_timeout_ms`
|
||||
(`--discovery-timeout-ms` on the CLI).
|
||||
- Each call opens one WebSocket and uses it for discovery
|
||||
(`Browser.getVersion`, `Target.getTargets`) and for the selected page.
|
||||
Discovery must finish within the same 10-second deadline as HTTP discovery.
|
||||
Chrome may ask you to approve each connection.
|
||||
- The browser identifier controls access to the socket. Evidence records the
|
||||
endpoint as `ws://127.0.0.1:9222/devtools/browser/[REDACTED]`.
|
||||
- `browser.revision` is the socket's `Browser.getVersion` value, such as
|
||||
@@ -345,11 +287,7 @@ read as registration state only after domain availability is confirmed.
|
||||
Discovery is attach-only: the command needs a reachable loopback CDP endpoint
|
||||
and a target ID from `list_browser_targets`, and it never navigates, reloads,
|
||||
or evaluates the page. Observe the page in the state you want to measure;
|
||||
registered tools reflect what the protocol delivered. If an in-scope subframe
|
||||
already exists when discovery attaches, `webmcp_tools` remains attach-limited:
|
||||
Chromium replay may omit its older registrations. Later live registrations
|
||||
retain their frame ownership but do not establish recovery of those missing
|
||||
descriptors. REA does not reload, evaluate, or invoke tools to fill that gap.
|
||||
registered tools reflect the document as loaded.
|
||||
|
||||
Accessibility names/descriptions, console primitive text, JSON body shapes, WebSocket shapes, script content, and storage key names are selected independently:
|
||||
|
||||
@@ -391,20 +329,6 @@ are reported in the result.
|
||||
Call `list_browser_targets` first because target IDs are browser-instance-specific. REA rechecks the selected target's current type and origin immediately before attaching. Without an origin filter, discovery lists all eligible pages; an individual capture defaults to the selected page's current origin.
|
||||
|
||||
REA establishes a main-frame origin boundary before enabling Runtime, Debugger, or Network observation, rechecks it before document capture, and validates it again afterward. If the main frame navigates during final capture, REA discards the mixed result and returns `target_changed`.
|
||||
An observed navigation also takes precedence over failures from stale
|
||||
document reads, such as a script ID invalidated by reload; cancellation remains
|
||||
cancellation. Retry after navigation settles, or refresh target discovery if
|
||||
the selected page was replaced. Origin-scope refusals from page inspection,
|
||||
runtime observation, screenshots, and WebMCP discovery explain the permitted
|
||||
HTTP(S) origin scope and direct the caller to select a permitted document or
|
||||
explicitly revise `allowed_origins`. Page capture also preserves an observed
|
||||
main-frame scope violation when a later document read fails. An unresolved
|
||||
initial main frame does not establish an origin violation; its bounded startup
|
||||
retries retain a distinct refusal.
|
||||
|
||||
Screenshot capture requires a valid main-frame identity. Malformed identity
|
||||
produces a protocol failure; a readable child frame cannot substitute for the
|
||||
main frame.
|
||||
|
||||
The web-session timeline uses CDP monotonic seconds where the event producer
|
||||
provides a comparable timestamp; absent or invalid timestamps are `null` rather
|
||||
@@ -415,37 +339,15 @@ contextual evidence with `timestamp: null`, accompanied by a limitation stating
|
||||
that it does not establish activity in the selected window. Main-frame scope
|
||||
enforcement still applies to contextual navigation events. Network redirect
|
||||
events retain their paired prior-hop response context when the timestamped
|
||||
next-request event falls inside the window. A redirect row's optional
|
||||
`redirect_response` contains the prior response `url`, CDP-reported `status`,
|
||||
and `scope`. The URL has transport credentials removed; URL and status are
|
||||
withheld unless its origin is approved. Missing or malformed response URLs have
|
||||
null scope and add `invalid_protocol_value` timeline coverage; a valid URL with an
|
||||
unsupported protocol has `unsupported` scope. Outside-policy and unsupported
|
||||
responses mark the timeline as policy-filtered with `disallowed_origin` or
|
||||
`unsupported_url`. Missing status remains null rather than zero. A supplied
|
||||
non-numeric status is also null and records `invalid_protocol_value` coverage;
|
||||
the other valid response facts remain available. Historical rows may omit this
|
||||
field; omission does not establish that no prior response existed. Response
|
||||
headers and bodies are discarded. The row timestamp belongs to the next request,
|
||||
not an exact prior response time. The pre-arm timeline buffer is bounded
|
||||
next-request event falls inside the window. The pre-arm timeline buffer is bounded
|
||||
by original CDP message bytes. If clock sampling completes after that budget is
|
||||
exhausted, the operation returns `payload_limit` instead of a partial window.
|
||||
Cancellation and clock-request failures retain their own outcomes. Session
|
||||
`load_failed` rows require a previously observed main-frame `Document` request;
|
||||
subresources, child-frame requests and failures without admitted request ownership
|
||||
are excluded. Redirects reuse their request identity, and completed/failed requests
|
||||
release it. Active request identity metadata uses the existing 8 MiB retained-event
|
||||
allowance; exhaustion fails the observation explicitly instead of guessing scope.
|
||||
Cancellation and clock-request failures retain their own outcomes.
|
||||
|
||||
## Data minimization
|
||||
|
||||
Response metadata parses repeated CDP header field lines independently. The
|
||||
Link projection retains their declarations in order. `csp.directives` lists the
|
||||
directives of every CSP policy in order, across field lines and comma-separated
|
||||
policies, without recording policy boundaries, so a directive name can repeat
|
||||
across policies. As in CSP3, a repeated name within one policy is ignored.
|
||||
Single-valued policy fields combine their lines as one comma-separated value,
|
||||
as RFC 9110 specifies. Each link keeps its target in `href` and
|
||||
The Link projection parses repeated CDP header field lines independently and
|
||||
retains their declarations in order. Each link keeps its target in `href` and
|
||||
its relationship context in `context_url` with a separate `context_scope`;
|
||||
absent `anchor` uses the response URL, while a relative `anchor` resolves
|
||||
against that URL. Anchored links remain in the projection, but a `link_rel`
|
||||
@@ -462,25 +364,14 @@ complete Structured Field Item, including parameters such as `report-to`.
|
||||
These are declarations, not an assessment of the document's effective policy;
|
||||
absent, malformed or non-token items project as `null`.
|
||||
|
||||
Console entries, WebSocket frame summaries, and selected request/response JSON
|
||||
shape outputs share one 8 MiB retained-observation allowance per capture. Shape
|
||||
inference admits expanded path bytes before allocating those paths, including
|
||||
escaped and Unicode property names. Replacing or removing a request's shapes
|
||||
refunds their retained charges. A refused body shape preserves network metadata
|
||||
and any captured peer shape with `resource_budget_exhausted` coverage; a refused
|
||||
WebSocket frame is counted as dropped rather than reclassified as non-JSON text.
|
||||
The allowance has no runtime override. Protocol messages and JSON input parsing
|
||||
remain subject to their separate transport admission; this output allowance does
|
||||
not bound the whole process heap.
|
||||
|
||||
REA retains local observation data and removes only URL userinfo credentials, explicitly declared scenario secrets, and values from structured credential/storage fields:
|
||||
|
||||
- URL observations preserve the complete local URL, including query values, fragments, duplicate parameters, and their order. Only username/password userinfo is removed; malformed original URL strings remain available with `origin: null`.
|
||||
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values. `dom.node_value_length_unit` is `utf16-code-units` for web observations and applies to every node's `node_value_length`; it counts JavaScript string code units, not Unicode code points or UTF-8 bytes. Electron observations declare `utf8-bytes` instead. Historical captures without the unit leave it unknown; reading them does not infer a unit or convert their stored lengths.
|
||||
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values.
|
||||
- Supported DOM URL attributes are projected separately as resolved destinations. Empty `action` on HTML or XHTML forms and empty `formaction` on HTML or XHTML submit controls resolve to the document URL; other relative URL attributes resolve against the captured document base.
|
||||
- Accessibility structure and reported roles are retained by default, while names and descriptions require `include_accessibility_text: true`. The browser protocol may omit an individual role; that role remains unknown for comparison. Other observed accessibility differences can still establish a change.
|
||||
- Network observations retain method, status, MIME type, size, type, initiator stack location, and complete URLs with userinfo credentials removed. Redirects reported by CDP are attached as ordered `redirects` on the one request ID, with the prior request URL, CDP response URL, method/type, response status/MIME/encoded length, request timestamp, and the next `requestWillBeSent` event timestamp. CDP delivers `redirectResponse` with that next request event, so the latter is an event boundary rather than an exact response timestamp. Redirect response headers and bodies are discarded; hops outside the selected origins are discarded with the request chain. Headers are discarded after an allowlisted projection of length/encoding, structured CSP/Link/policy fields, and untrusted agent hints. Cookies and authorization headers are never retained.
|
||||
- Request/response bodies are not requested or parsed by default. When selected, only allowed-origin JSON media types are read, converted immediately to complete value-free typed paths, types and observation counts, and discarded. Each path segment is either `{ "kind": "property", "name": "raw property name" }` or `{ "kind": "array-element" }`; array-element types include primitives and nested arrays. Literal `*` properties remain distinct from array elements. `Network.getResponseBody` is never sent unless JSON body shape capture is selected. A JSON-typed request body that CDP reports through `hasPostData` but omits from `postData`, or that cannot be parsed as JSON, makes `json_body_shapes` unavailable. That request's `body_shapes.status` is `partial`, not `included`, when only its response shape was captured. Base64 response bytes must be valid UTF-8 before JSON interpretation; invalid bytes leave the response shape unavailable and preserve any observed request shape and network metadata. Producer-decoded strings and genuine U+FFFD characters retain their reported meaning. Comparison uses body shapes only when both captures selected them, and compares request/response shapes only where both observations provide those facts. Unavailable or truncated selected coverage makes equivalence unknown; observed differences still report changed.
|
||||
- Request/response bodies are not requested or parsed by default. When selected, only allowed-origin JSON media types are read, converted immediately to complete value-free typed paths, types and observation counts, and discarded. Each path segment is either `{ "kind": "property", "name": "raw property name" }` or `{ "kind": "array-element" }`; array-element types include primitives and nested arrays. Literal `*` properties remain distinct from array elements. `Network.getResponseBody` is never sent unless JSON body shape capture is selected. A JSON-typed request body that CDP reports through `hasPostData` but omits from `postData`, or that cannot be parsed as JSON, makes `json_body_shapes` unavailable. That request's `body_shapes.status` is `partial`, not `included`, when only its response shape was captured. Comparison uses body shapes only when both captures selected them, and compares request/response shapes only where both observations provide those facts. Unavailable or truncated selected coverage makes equivalence unknown; observed differences still report changed.
|
||||
For normalized passive inputs with complete request inventories on both sides,
|
||||
request additions and removals remain observable even when selected body shapes,
|
||||
response metadata, or WebSocket coverage are unavailable. HTTP request and
|
||||
@@ -511,7 +402,7 @@ REA retains local observation data and removes only URL userinfo credentials, ex
|
||||
connection/frame inventories; matching direction/opcode sequences alone do
|
||||
not prove alignment after drops. Redirect resource types use only shared known metadata:
|
||||
an omitted type leaves equality unknown rather than proving a modification.
|
||||
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. Source locations are CDP's zero-based lines and UTF-16 columns. A negative column, which Chrome reports on the first line of some inline scripts, is kept as reported and does not locate a column in the source URL; an invalid line or a non-integer column is `null` (WebMCP `registration_source` is `null`) and is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
|
||||
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
|
||||
- WebSocket observations retain direction, opcode, and payload byte length. A missing or malformed CDP opcode is returned as `null`, with opcode-dependent payload byte length and shape also `null`, and marks `websocket_frames` incomplete. Missing or wrong-typed payload data is not treated as empty; its byte length is `null`, and a text payload shape is unknown. Numeric opcode `0` and an observed empty payload remain distinct from missing data. Opcode is nullable in the public inspection schema, so clients must handle unknown protocol values. When shape capture is selected, text messages are classified as text or complete value-free JSON shape; binary bytes, hashes, prefixes, and raw payloads are never retained.
|
||||
- Storage observations always redact values. Key names, IndexedDB names, and cache names require `include_storage_keys`; stable content fingerprints require the additional `include_storage_fingerprints` selection. Cookie fingerprints cover cookies applicable to the current main-frame URL, including its path, rather than every cookie path on that origin. Other storage remains scoped to the origin. Cache fingerprints hash all decoded response bytes, including bodies above 64 KiB, subject to the [64 MiB per-message CDP limit](#request-boundary). Missing or undecodable cache bodies and partial IndexedDB remote objects make fingerprint coverage incomplete, so identical observations remain `unknown`.
|
||||
- Unavailable or malformed storage inventories mark `storage_keys` unavailable, even without fingerprints. Comparisons suppress inferred key additions and removals while preserving modifications proven by complete fingerprints on both sides. Previously exported captures are not retroactively corrected; recapture them to obtain these coverage markers.
|
||||
@@ -521,11 +412,6 @@ Frames, resources, scripts, events, and workers outside the selected origin scop
|
||||
Retained workers include validated opener-target and parent-frame IDs when CDP
|
||||
provides them; these relationships do not expand the exact-origin boundary or
|
||||
prove static module ownership.
|
||||
Workers without either relationship are withheld with an `unattributed_target`
|
||||
exclusion and `workers` in `unavailable_sections`: a same-origin URL does not
|
||||
establish which page owns a worker. A supplied relationship to another target or
|
||||
frame remains `out_of_target_scope`. Missing ownership also remains unknown when
|
||||
the worker URL is empty; withheld target details are not returned.
|
||||
|
||||
## Completeness and limits
|
||||
|
||||
@@ -609,13 +495,13 @@ an unknown size, comparison reports unknown rather than treating the observation
|
||||
as a resource addition or removal; complete inventory coverage still establishes
|
||||
when an entire group is absent.
|
||||
|
||||
CDP discovery defaults to a 10-second response deadline, configurable per call
|
||||
on passive browser tools with `discovery_timeout_ms` (default 10000). HTTP
|
||||
discovery documents use an 8 MiB budget. Browser-socket discovery
|
||||
(`Browser.getVersion`, `Target.getTargets`) uses the 64 MiB protocol message
|
||||
budget. Neither byte budget has a runtime override. `list_browser_targets`
|
||||
accepts `discovery_timeout_ms` and no `command_timeout_ms`, because it performs
|
||||
only discovery without a post-discovery target command. Other CDP tools use
|
||||
CDP discovery has a 10-second response deadline. HTTP discovery documents
|
||||
use an 8 MiB budget. Browser-socket discovery (`Browser.getVersion`,
|
||||
`Target.getTargets`) uses the 64 MiB protocol message budget. Neither budget
|
||||
nor the discovery deadline has a runtime override. `list_browser_targets` accepts no
|
||||
`command_timeout_ms`: HTTP discovery and browser-socket discovery
|
||||
(`Browser.getVersion`, `Target.getTargets`) stay on that fixed deadline, and
|
||||
the socket closes without a later command. Other CDP tools use
|
||||
`command_timeout_ms` (default 20000) for each WebSocket handshake and command
|
||||
response, separate from the caller-selected observation window, discovery, and
|
||||
the 5000 ms cleanup grace. A browser-level socket's handshake and the following
|
||||
@@ -635,46 +521,12 @@ rejects it as unknown; other command rejections retain their reported failure.
|
||||
|
||||
Network and console coverage starts only after REA attaches and enables the relevant CDP domains. `prior_activity_available` is always `false`; absence from these arrays is not evidence that an event never occurred. Scripts are different: scripts the page parsed before REA attached are still reported under `scripts`. A page that loaded before attach can therefore list its scripts while `network.requests` lacks the requests that fetched them. If CDP reuses a request ID without supplying redirect metadata, REA retains the predecessor observation and marks network coverage incomplete.
|
||||
|
||||
Source maps are not fetched by inspection. Bundle analysis fetches them only when `fetch_source_maps` is selected, with exact-origin redirect checks and no credentials, cookies, or referrer. Each fetch operation uses `source_map_fetch_timeout_ms` (default 30000) and a default 64 MiB aggregate streamed-response budget across redirects. Select `source_map_limits.fetch_bytes` to change byte admission for the next call. A fetch overrun is an item limitation, not a thrown analysis timeout. A deadline or byte-limit failure remains explicitly unavailable and makes bundle completeness partial. Source-map decoding defaults to 64 MiB for raw resolved identities and 32 MiB for aggregate inline evidence; `source_map_limits.fetch_bytes` and `source_map_limits.output_bytes` own those independent admissions. Failed maps preserve their failure context when it fits; if another context cannot fit, collection stops with the full requested count, the returned processed count, and an explicit unknown-results limitation. Retained mappings and original sources are returned inline. See [source-location limits](web-source-location.md) for the structural decoder bounds. Original source artifacts with `.ts`, `.tsx`, `.mts`, or `.cts` suffixes carry `text/typescript`; other source names retain `text/javascript`.
|
||||
Source maps are not fetched by inspection. Bundle analysis fetches them only when `fetch_source_maps` is selected, with exact-origin redirect checks and no credentials, cookies, or referrer. Each fetch operation uses `source_map_fetch_timeout_ms` (default 30000) and a 64 MiB aggregate streamed-response budget across redirects. The byte budget has no runtime override. A fetch overrun is an item limitation, not a thrown analysis timeout. A deadline or byte-limit failure remains explicitly unavailable and makes bundle completeness partial. Source-map decoding also bounds raw resolved identities to 64 MiB and aggregate inline evidence to 32 MiB. Failed maps preserve their failure context when it fits; if another context cannot fit, collection stops with the full requested count, the returned processed count, and an explicit unknown-results limitation. Retained mappings and original sources are returned inline. See [source-location limits](web-source-location.md) for the structural decoder bounds. Original source artifacts with `.ts`, `.tsx`, `.mts`, or `.cts` suffixes carry `text/typescript`; other source names retain `text/javascript`.
|
||||
|
||||
A fully read response that fails UTF-8, JSON, mapping, or structural validation
|
||||
retains `artifact: null`, empty original-source, module-edge, and mapping
|
||||
arrays, and an explicit failure limitation. Invalid content is `invalid`;
|
||||
decoder or structural capacity refusal is `fetch_failed`. Its optional
|
||||
`response_identity` records the complete body's SHA-256 digest, byte count,
|
||||
and `fetch-response-body-bytes` representation without retaining malformed
|
||||
text or charging that text to the 32 MiB output budget. These are bytes
|
||||
delivered by Fetch after HTTP content decoding and before UTF-8 decoding, not
|
||||
HTTP wire bytes. The identity is absent when the body was not fully read, and
|
||||
older observations may omit it. A response whose text cannot fit the existing
|
||||
output budget is likewise `fetch_failed` with text omitted and completed-body
|
||||
identity; JSON parsing and source-map decoding are not attempted. Successful and partial maps keep their
|
||||
full text artifacts. Their original text, including an initial BOM or XSSI
|
||||
line, remains distinct from the parsing view that removes that framing.
|
||||
Collection reads the body once and does not raise its response or output limits.
|
||||
|
||||
When fetching is selected, source-map `excluded` facts preserve the capture
|
||||
ledger's reasons and counts for declarations rejected before download. No
|
||||
disallowed URL details are retained or fetched. `requested` and `processed`
|
||||
count the admitted fetch requests; exclusions are counted separately. Excluded
|
||||
declarations make the collection unavailable, or partial when maps were retained.
|
||||
An empty collection with no declarations remains `not_requested`.
|
||||
|
||||
If the caller cancels while later source maps are being fetched, the failure's
|
||||
partial Evidence retains the captured page and every completed map item. Its
|
||||
`normalized_result.kind` is `web-bundle-acquisition` and
|
||||
`analysis_status` is `not_performed`: no script-analysis result is claimed.
|
||||
`source_maps.requested` remains the admitted request count,
|
||||
`source_maps.processed` counts completed items, and any unprocessed requests
|
||||
have an explicit unknown-results limitation. Capture exclusions remain in the
|
||||
same collection. The cancellation error keeps the caller's exact reason;
|
||||
an already observed response resource or cleanup failure keeps its own error
|
||||
category and cleanup owner.
|
||||
|
||||
Source-map items retain the script's `declared_url` separately from the observed
|
||||
`effective_url`. Successful maps use the latter as their source-resolution base.
|
||||
Failed fetches retain the last response URL observed within the approved origins;
|
||||
`effective_url` is `null` when no response was observed. Query parameters are
|
||||
source-map items retain the script's `declared_url` separately from the observed
|
||||
`effective_url`. successful maps use the latter as their source-resolution base.
|
||||
failed fetches retain the last response URL observed within the approved origins;
|
||||
`effective_url` is `null` when no response was observed. query parameters are
|
||||
preserved, and both URLs count toward the inline evidence budget.
|
||||
Current fetches emit a string or null; historical observations may omit
|
||||
`effective_url` without changing their stored content or Evidence identity.
|
||||
@@ -689,9 +541,6 @@ validation.
|
||||
## Non-goals and threat model
|
||||
|
||||
The browser provider deliberately does not expose generic CDP commands, `Runtime.evaluate`, WebMCP invocation, navigation, input, downloads, page closure, or browser closure. Screenshot capture is the sole pixel surface and returns an image only when requested. REA disables the domains it enabled, detaches the target session, and closes only its own WebSocket.
|
||||
Discovery and attachment errors remain primary if closing REA's socket also
|
||||
fails. The cleanup failure is reported alongside the original command failure;
|
||||
it cannot replace the operation diagnostic.
|
||||
|
||||
This feature is not a browser sandbox or network containment mechanism. The attached page and browser continue running with their existing privileges and may make external requests independently of REA. CDP gives deep access to the selected browser profile, so use a dedicated profile, include only origins you intend to inspect, and treat other same-user processes as outside this boundary.
|
||||
|
||||
@@ -9,43 +9,28 @@ liveness deadlines. `INPUT_JSON` may be inline JSON or a JSON file path.
|
||||
|
||||
The minimal request contains only browser launch/connect selection, `start_url`,
|
||||
and at least one `actions` entry. Launch `headless` defaults to `true` and can
|
||||
be set to `false` where the host supports a visible browser. Launched Chromium
|
||||
keeps its sandbox; on hosts that cannot provide one, such as root in a
|
||||
container, set `REA_BROWSER_NO_SANDBOX=true` for the REA process. Termux on
|
||||
Android always launches without it. Navigations follow
|
||||
the destinations named by the scenario and the page's ordinary browser requests,
|
||||
and [request steps](#request-steps) send only to the start URL's origin;
|
||||
there is no separate origin allowlist. Request steps use a temporary dispatch
|
||||
gate for cancellation; ordinary page requests retain their browser behavior. Environment
|
||||
be set to `false` where the host supports a visible browser. Navigations follow
|
||||
the destinations named by the scenario and the page's ordinary browser requests;
|
||||
there is no separate origin allowlist or request-interception layer. Environment
|
||||
settings (including service workers blocked by default), empty storage, and a
|
||||
final sanitized URL capture are supplied by default. Set
|
||||
`environment.service_workers` to `allow` when the application requires them.
|
||||
The connected service-worker policy owner retains its exact CDP transport
|
||||
through discovery, attachment and restoration. Failed admission preserves its
|
||||
primary diagnostic when socket cleanup also fails; later policy cleanup retries
|
||||
that retained transport and clears cleanup uncertainty only after close succeeds.
|
||||
For attached browsers, `block` rejects captures when the selected page or its
|
||||
frames are controlled by a service worker or already have a registration,
|
||||
including after navigation. Policy checks bracket each artifact capture, including
|
||||
the final step. If an external worker takes control while artifacts are being
|
||||
captured, REA rejects the capture and retains the completed artifacts in
|
||||
`details.diagnostics.collected_artifacts`; they do not establish policy-compliant
|
||||
completion. Existing caller registrations are never removed.
|
||||
including after navigation. Existing caller registrations are never removed.
|
||||
Navigation can run destination code before a newly discovered worker is rejected;
|
||||
that rejection does not roll back browser effects. REA removes its owned
|
||||
future-document scripts, restores live frame methods, then disconnects.
|
||||
Add secret declarations, storage seeds, and additional artifact or event
|
||||
capture only when needed. Missing environment-backed secrets fail as input errors before browser acquisition, naming the declaration and required variable. An explicitly set empty string is a resolved value. Action, secret, and storage counts are not capped.
|
||||
capture only when needed. Action, secret, and storage counts are not capped.
|
||||
Duration, action, and navigation timeouts remain separate from CDP command
|
||||
deadlines. `command_timeout_ms` (default 20000; omit it to keep that default
|
||||
and leave it out of Evidence) bounds launch startup, the connect-over-CDP
|
||||
handshake, target lookup, environment commands, and storage CDP commands,
|
||||
including replies from attached child targets. It also bounds the initial
|
||||
`start_url` navigation through the load event; a redirect or replacement
|
||||
navigation cannot leave startup waiting indefinitely. Set it in the scenario JSON.
|
||||
including replies from attached child targets. Set it in the scenario JSON.
|
||||
`capture-browser-scenario` has no `--command-timeout-ms` option.
|
||||
It does not change action `timeout_ms`, `wait_for_timeout.duration_ms`, or
|
||||
action navigation. Service-worker restoration keeps a fixed 5000 ms grace and does
|
||||
navigation. Service-worker restoration keeps a fixed 5000 ms grace and does
|
||||
not inherit a raised `command_timeout_ms`. CDP discovery stays on its own
|
||||
fixed deadline. When service workers are blocked, the registration check
|
||||
before discovery has its own fixed 10000 ms deadline; `command_timeout_ms`
|
||||
@@ -54,30 +39,6 @@ covers endpoint discovery only. The following WebSocket handshake and
|
||||
`Target.attachToTarget` each use `command_timeout_ms`. `capture.limits` controls retained event and network-content
|
||||
budgets; it does not change those liveness deadlines.
|
||||
|
||||
## Validate a scenario before running it
|
||||
|
||||
`validate-browser-scenario INPUT_JSON` checks one scenario against the same
|
||||
`browserScenarioSchema` refinement that `capture_browser_scenario` admits, and
|
||||
stops there. `INPUT_JSON` may be inline JSON, a JSON file path, or `-` for
|
||||
standard input.
|
||||
|
||||
```bash
|
||||
rea validate-browser-scenario ./browser-scenario.json --json
|
||||
```
|
||||
|
||||
A valid scenario reports `{"valid":true,"actions":<count>,"secrets":<count>}`
|
||||
and exits 0. An invalid one exits 1 and reports the same field paths capture
|
||||
would report, so a correction made here is the correction capture needs:
|
||||
duplicate or reserved step IDs, duplicate secret identities and
|
||||
environment-variable names, undeclared secret references, request-step
|
||||
violations, and unknown fields.
|
||||
|
||||
Validation performs no browser discovery, launch, attachment, or navigation and
|
||||
constructs no provider. It does not read secret environment variables, so a
|
||||
scenario whose declared variables are unset still validates; the missing-value
|
||||
failure belongs to capture. A schema-valid scenario does not prove that the
|
||||
executable or target is available on the host.
|
||||
|
||||
## Minimal launch example
|
||||
|
||||
Save this as `browser-scenario.json`, replacing the executable path with an
|
||||
@@ -200,12 +161,6 @@ distinction: it closes and deletes provider-owned profiles, but only disconnects
|
||||
from an external CDP browser. Real-browser verification checks both outcomes and
|
||||
confirms that an attached external browser remains alive.
|
||||
|
||||
CDP connections leave Playwright's default-context overrides disabled. The
|
||||
selected page receives the declared scenario environment and temporary focus
|
||||
emulation, so actions can complete while that page is in the background.
|
||||
Its environment owner clears these overrides before disconnecting; other pages
|
||||
keep their media settings. Connecting does not create a bootstrap page.
|
||||
|
||||
That cleanup requires REA to remain running. Abrupt termination, including
|
||||
Windows process-tree termination or POSIX `SIGKILL`, can leave the launch
|
||||
profile and Playwright artifacts behind. A profile can retain cookies, history,
|
||||
@@ -239,9 +194,7 @@ language environments.
|
||||
HTTP(S) URLs may include ordinary query values and fragments directly. Use
|
||||
ordered structured query entries when a value needs to reference a declared
|
||||
environment-backed secret. Form, storage, and cookie values are either literal
|
||||
strings or declared secret references; request step headers can also read a
|
||||
cookie or local storage value from the page, and request steps and `goto`
|
||||
destinations can send [saved values](#saved-values). Raw URL userinfo credentials are rejected.
|
||||
strings or declared secret references. Raw URL userinfo credentials are rejected.
|
||||
Captured credential-header
|
||||
values are not retained, and declared secret values are redacted automatically.
|
||||
A secret may be declared solely to
|
||||
@@ -258,9 +211,8 @@ validation.
|
||||
|
||||
Cookies are installed before the initial navigation. Local storage entries are
|
||||
seeded once per browser storage partition; session storage entries are seeded
|
||||
once per origin in each instrumented top-level browsing context. Seeding occurs
|
||||
before the first document scripts for the selected page and its instrumented
|
||||
frames. An origin first reached by a later action
|
||||
once per origin in each top-level browsing context. Seeding occurs before the
|
||||
first document scripts for that area. An origin first reached by a later action
|
||||
receives its seed on that first visit. Application updates, deletions, and
|
||||
empty values then persist across later navigations in the same area; a new
|
||||
scenario starts with its own seeds. Partitioned frames can receive separate
|
||||
@@ -268,17 +220,6 @@ local storage seeds even when they share an origin. The pre-script mechanism
|
||||
uses CDP debugger instrumentation. The observer resumes only pauses whose top
|
||||
frame belongs to its private seed script; page breakpoints and `debugger`
|
||||
statements remain under the attached debugger client's control.
|
||||
If Chromium leaves the private isolated world's origin empty, REA looks up
|
||||
that exact frame's reported security origin before selecting a seed. It does
|
||||
not infer the origin from a frame URL or serialized storage key. Unresolved
|
||||
frame identity or security origin fails initialization rather than silently
|
||||
running with an unknown initial state.
|
||||
Popup discovery does not establish that its first script was paused by the
|
||||
selected page's storage instrumentation. When non-cookie seeds are declared
|
||||
and a popup is discovered, the capture retains completed observations and
|
||||
marks `storage` coverage missing, with equality ineligible. This applies even
|
||||
if popup lifecycle events were not selected. Popup first-script storage
|
||||
initialization is not currently guaranteed.
|
||||
|
||||
The result is Evidence with an initial state followed by one record per
|
||||
declared action. Each step reports action status, elapsed time, sanitized URLs,
|
||||
@@ -352,263 +293,6 @@ external browser without closing it. The request itself defines the operation;
|
||||
there is no additional REA grant step. The browser and host still enforce their
|
||||
own access and process rules.
|
||||
|
||||
## Request steps
|
||||
|
||||
A `request` step sends one HTTP request from the page with the page's own
|
||||
`fetch`, so the browser attaches the session's cookies and the request is
|
||||
recorded in the step's network events like any request the application makes.
|
||||
Use it to record an application's API, including writes, when driving the UI
|
||||
for each call would be slower or less exact than calling the endpoint the UI
|
||||
calls. This action directly sends a caller-declared HTTP request without a user
|
||||
gesture, so run scenarios with write requests only against an instance whose
|
||||
data you may change. This scenario creates one item:
|
||||
|
||||
```json
|
||||
{
|
||||
"browser": {
|
||||
"mode": "launch",
|
||||
"executable_path": "/absolute/path/to/chromium"
|
||||
},
|
||||
"start_url": { "url": "http://127.0.0.1:3000" },
|
||||
"actions": [
|
||||
{
|
||||
"step_id": "create-item",
|
||||
"action": "request",
|
||||
"method": "POST",
|
||||
"path": "/api/items",
|
||||
"query": [
|
||||
{ "name": "notify", "value": { "source": "literal", "value": "false" } }
|
||||
],
|
||||
"headers": [
|
||||
{
|
||||
"name": "X-CSRFToken",
|
||||
"value": { "source": "cookie", "name": "csrftoken" }
|
||||
},
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": {
|
||||
"source": "local_storage",
|
||||
"key": "auth",
|
||||
"json_pointer": "/token"
|
||||
},
|
||||
"prefix": "Bearer "
|
||||
}
|
||||
],
|
||||
"json": { "title": "Scratch item 1" },
|
||||
"expected_status": [201],
|
||||
"timeout_ms": 10000
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- `method` is `GET`, `HEAD`, `POST`, `PUT`, `PATCH` or `DELETE`. `json` is the
|
||||
body, sent with `Content-Type: application/json` unless a header sets one; a
|
||||
`GET` or `HEAD` step cannot carry it.
|
||||
- `path` starts with one `/` and must resolve to the start URL's origin, which
|
||||
validation checks before the browser starts. When the step runs, the page must
|
||||
be on that origin, checked again in the page as the request is sent; add a
|
||||
`goto` step first if an earlier action left it. A query in `path` is sent as
|
||||
written (a fragment is not sent), and `query` entries are appended after it
|
||||
in order and may reference declared secrets.
|
||||
- Request steps follow no redirects: a redirect answer fails the step, and
|
||||
nothing is sent to the location it names, though a write the redirect
|
||||
answered may already have been applied. `expected_status` cannot name a 3xx
|
||||
status. Manual redirects can be opaque to the page and may not produce a
|
||||
response event; their status and `Location` can therefore be unavailable even
|
||||
when network events and header values were selected. Such captures report
|
||||
incomplete event coverage.
|
||||
- Header values are literals, declared secrets, a cookie the browser holds for
|
||||
the request URL (`{"source": "cookie", "name"}`, HttpOnly cookies included,
|
||||
whose value the page's scripts can then see while the request runs), or a
|
||||
local storage entry on the start origin (`{"source": "local_storage",
|
||||
"key"}`, with an optional `json_pointer` to a string or number in a stored
|
||||
JSON value). `prefix` is sent before the value. A value read from the page is
|
||||
redacted like a declared secret, as `[REDACTED:page:<step_id>.<header>]`,
|
||||
from records made after the step reads it; a response recorded earlier, such
|
||||
as the login response that returned a token, keeps the value. A value read
|
||||
again keeps its first marker, and a value shorter than 8 characters is not
|
||||
redacted, since redacting it would rewrite ids, ports and hosts throughout
|
||||
the capture. A value is redacted as stored and as `fetch` sends it, without
|
||||
surrounding spaces and tabs. A missing cookie, key or pointer fails the step.
|
||||
Reading a page value raises the estimated cost of redacting every later
|
||||
body, so the bodies kept within `max_network_body_allocation_bytes` (1 MiB
|
||||
by default) get smaller: after a 32-character `X-CSRFToken` value, a body over
|
||||
about 47 KiB is kept as `unavailable`. Raise `capture.limits` when the
|
||||
application's responses are larger.
|
||||
- A header can send Basic credentials, `{"source": "basic_credentials", "parts":
|
||||
[username, password]}`: the base64 of `username:password`, after a `prefix`
|
||||
such as `"Basic "` or with none, as some APIs take encoded credentials in
|
||||
their own header. Each part is a literal, a declared secret, a saved value, a
|
||||
cookie or a local storage entry, so a header a frontend builds from a login
|
||||
token can be sent too; the password is never a literal other than empty, since
|
||||
Evidence records literals as written, and a username with a colon fails the
|
||||
step, since Basic authentication ends the username at its first colon. The
|
||||
encoded value is redacted however short, as
|
||||
`[REDACTED:page:<step_id>.<header>]`, and parts read from the page are
|
||||
redacted as page values are.
|
||||
- Validation refuses a header name given twice in a step, header names a
|
||||
page's `fetch` cannot set, such as `Cookie`, `Origin`, `Host` or `Sec-*`, and
|
||||
`User-Agent`, which Chrome drops; it refuses line breaks and other ASCII
|
||||
control characters in literal values and prefixes, and the step fails before sending
|
||||
when a secret or page value holds them.
|
||||
- The step completes when the whole response has been read. With
|
||||
`expected_status`, any other status fails the step and the later actions are
|
||||
recorded as `cancelled`; without it, any response completes the step. A
|
||||
network error fails the step. `timeout_ms` covers reading the header values
|
||||
and the request: a step whose time runs out, or that is cancelled, before
|
||||
its request is sent sends nothing, but a request already sent can still reach
|
||||
the application after the step fails.
|
||||
An absolute deadline also covers browser evaluation queued behind page work.
|
||||
A temporary route checks cancellation again at network dispatch and is removed
|
||||
when the request evaluation settles. A page controlled by a service worker
|
||||
is refused because its fetch can bypass that dispatch gate; use the default
|
||||
blocked service-worker policy for request steps.
|
||||
|
||||
The request runs in the page's main world, so an application that wraps
|
||||
`fetch` sees it, and it goes through the same network observation as other
|
||||
page requests: select network content (bodies or header values) to record it.
|
||||
Captures with request steps state in `limitations` that the steps sent their
|
||||
declared requests through the page's fetch. A scenario without request steps
|
||||
is unchanged.
|
||||
|
||||
## Saved values
|
||||
|
||||
A step can keep a string or number from a JSON response, by name, and a later
|
||||
step can send it, so a scenario can read back, edit and delete an object whose
|
||||
id the application assigns:
|
||||
|
||||
```json
|
||||
{
|
||||
"browser": {
|
||||
"mode": "launch",
|
||||
"executable_path": "/absolute/path/to/chromium"
|
||||
},
|
||||
"start_url": { "url": "http://127.0.0.1:3000/items" },
|
||||
"actions": [
|
||||
{
|
||||
"step_id": "save-item-1",
|
||||
"action": "click",
|
||||
"timeout_ms": 10000,
|
||||
"locator": { "kind": "role", "role": "button", "name": "Save" },
|
||||
"save_response": {
|
||||
"method": "POST",
|
||||
"path": "/api/items",
|
||||
"save": [{ "name": "item_1", "json_pointer": "/id" }]
|
||||
}
|
||||
},
|
||||
{
|
||||
"step_id": "open-item-1",
|
||||
"action": "goto",
|
||||
"timeout_ms": 10000,
|
||||
"destination": { "url": "http://127.0.0.1:3000/items/{saved:item_1}" },
|
||||
"wait_until": "load"
|
||||
},
|
||||
{
|
||||
"step_id": "copy-item-1",
|
||||
"action": "request",
|
||||
"timeout_ms": 10000,
|
||||
"method": "POST",
|
||||
"path": "/api/items/{saved:item_1}/copies",
|
||||
"json": {
|
||||
"source_id": "{saved:item_1}",
|
||||
"title": "Copy of {saved:item_1}"
|
||||
},
|
||||
"expected_status": [201]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- A `request` step saves with `save: [{name, json_pointer, secret}]`, from its
|
||||
own response. A `click` or `press` step saves with `save_response: {method,
|
||||
path, save}`, from a newly observed matching response while the step waits.
|
||||
The recorder watches before the action and excludes requests already running,
|
||||
such as an earlier autosave. Method/path matching does not prove that the UI
|
||||
caused the request: use a path that distinguishes the intended operation from
|
||||
background polling. Multiple observed matches before binding fail as ambiguous;
|
||||
requests starting after this observation window cannot be ruled out. It matches the method and a path on the
|
||||
start origin as the browser spells it, so a path may name `My List` or
|
||||
`My%20List` (`{*}` matches one segment, and the query is ignored), and fails
|
||||
the step unless a 2xx JSON response arrives before its `timeout_ms`; one still
|
||||
being read when the step times out or is cancelled saves nothing. A pointer
|
||||
that names no string or number fails the step, so later steps never send a
|
||||
missing value, as does a response over 4,000,000 characters: a request step
|
||||
stops reading and aborts its fetch there. A UI step reads a body only when
|
||||
its decoded size is bounded at 12,000,000 bytes or less: its received size,
|
||||
or the declared Content-Length of one served from cache, times 1032 for each
|
||||
gzip or deflate coding, the most DEFLATE expands. Other codings and unknown
|
||||
or zero sizes are refused before materializing the body; decoded text over
|
||||
the character limit also fails. The UI action may already have applied.
|
||||
- `{saved:name}` sends a saved value in a request step's `path`, including its
|
||||
query, and in a `goto` destination's `url`, percent-encoded; a destination's
|
||||
structured `query` values are sent as declared. A saved value that makes a
|
||||
path segment `.` or `..`, alone or with the text beside it (including `%2e`),
|
||||
fails the step, since the URL would drop it. In a request step's `json` body,
|
||||
a string that is only `{saved:name}` becomes the saved value with the JSON
|
||||
type it was saved with, so a saved number id is sent as a number, and
|
||||
`{saved:name}` inside a longer string is filled as text. A header value or
|
||||
Basic credentials part sends one as `{"source": "saved", "name"}`. Other
|
||||
braces, such as `{id}`, are sent as written, and so is a percent-encoded
|
||||
`%7Bsaved:name%7D` anywhere but a `goto` URL, whose parsed form spells the
|
||||
braces that way.
|
||||
- Validation refuses a saved value that no earlier step saves. A later save of
|
||||
the same name replaces the value. An integer past 2^53, which JSON parsing
|
||||
would round to another number, and a number that is not finite, such as
|
||||
`1e400`, negative zero, or a value that underflows to zero, such as `1e-400`, fail the step
|
||||
that saves it.
|
||||
- A scenario keeps at most 4,000,000 characters of saved values at once,
|
||||
counting distinct retained redaction spellings, including trimmed variants,
|
||||
of values saved as secret. Replacing a binding releases its live value
|
||||
charge, while secret spellings remain protected for the scenario lifetime.
|
||||
A save past the budget fails before changing the binding.
|
||||
- A value saved with `secret: true` is redacted however short, as a declared
|
||||
secret is, as `[REDACTED:page:<step_id>.<name>]`. The selected source
|
||||
response is protected before retaining its body or header values, including
|
||||
when only headers were requested. If that body cannot be read safely, its
|
||||
body and header values remain unavailable. Before publishing the capture,
|
||||
the selected source transaction's earlier URL, request headers and request
|
||||
body are also protected using the admitted values. Unresolved source values
|
||||
withhold that transaction's affected metadata/content with an explicit
|
||||
limitation. Before publication, retained events are rechecked
|
||||
with the final marked-response values, including immediate follow-up requests
|
||||
and their response bodies, console/page errors, WebSocket text and event URL/name fields. Learned redaction facts are independent of source-identity readiness. Unknown source values withhold only the selected
|
||||
source transaction. Other saved values, such as ids, stay visible, since
|
||||
environments model them. The environment build warns about saved values
|
||||
named like credentials that are not marked secret
|
||||
(`credential-named-saved-values`).
|
||||
- Page-derived cookie/local-storage values and constructed Basic credentials
|
||||
remain redacted only from observations made after they were read/built.
|
||||
They do not retroactively rewrite earlier facts. The final event projection
|
||||
borrows only declared and response-marked patterns; promotion of an already
|
||||
held page value to an explicit response secret preserves its first marker.
|
||||
Projection state is session-local, references admitted pattern/byte owners,
|
||||
and is invalidated on admission or response classification changes.
|
||||
- Screenshots are withheld as `missing` from the start when the scenario plans
|
||||
any response save marked `secret: true`, page-derived or Basic authentication
|
||||
header values, or whenever nonempty sensitive values
|
||||
are already known. Raw pixels cannot be safely text-redacted, so the provider
|
||||
does not request pixels in those cases. This also protects earlier steps and
|
||||
partial failures without retaining an image to scrub later. Other requested
|
||||
artifacts remain available; affected screenshot/aggregate completeness is
|
||||
partial and is not eligible for equality.
|
||||
- Saved URLs and expanded request JSON are admitted before building their expanded strings or
|
||||
serializing. Its allowance is the smaller of Node's string limit and
|
||||
one sixteenth of current V8 available heap bytes, allowing for the expanded
|
||||
tree, serialization, Playwright transport and browser-side copies. Size
|
||||
accounting includes JSON escaping and repeated references; each distinct
|
||||
saved value's size is measured once per request. An oversized request fails
|
||||
before browser dispatch; the complete request envelope, including repeated
|
||||
header values, is checked as well. URL encoding uses a conservative
|
||||
nine-characters-per-UTF-16-unit allowance before encoding a saved value.
|
||||
Reduce the payload/repetitions, or raise the parent
|
||||
Node heap and restart; this is a conservative allocation estimate, not a
|
||||
measurement or guarantee of browser/process-family memory.
|
||||
- Evidence records a request step's empty `save` list as absent, so a scenario
|
||||
that saves nothing records the same parameters and `scenario_sha256` as
|
||||
before saved values existed, and REA versions without saved values read its
|
||||
captures.
|
||||
|
||||
## Network evidence
|
||||
|
||||
`capture.events: ["network"]` retains request, response, completion, failure,
|
||||
@@ -649,18 +333,6 @@ and unfinished-request metadata. Select content independently:
|
||||
All three options default to `false`. Any selected network content also enables
|
||||
network metadata observation; listing `network` in `events` is then optional.
|
||||
Each observed request object receives a capture-scoped `transaction_id`.
|
||||
Each `request` event carries `observed_at`, the wall-clock time REA observed the
|
||||
request start. Playwright reports the browser's own start only once a request
|
||||
ends, so each `request-finished` or `request-failed` event carries `started_at`,
|
||||
the browser's wall-clock start, and `elapsed_ms`, its time from that start to
|
||||
the end of the response, when the browser reports them. The environment views
|
||||
time scenario records from these, as HAR entries are timed: the browser's start
|
||||
where it reported one, else `observed_at`, which trails it by up to tens of
|
||||
milliseconds while REA reads other responses. A duration is retained only from a
|
||||
unique terminal event that reports it; missing or ambiguous durations stay
|
||||
omitted, while an explicitly reported zero remains zero. Captures made before these fields existed lack them and
|
||||
are read in record order. REA versions before them refuse captures that carry
|
||||
them. Scenario comparisons ignore all three (`ignore_network_request_timing`).
|
||||
Requests sharing a URL remain distinct. Redirects link to the preceding
|
||||
transaction only when it was observed; a null predecessor does not establish
|
||||
that no earlier hop existed. Request events require both transaction fields;
|
||||
@@ -694,16 +366,8 @@ Proxy-Authorization, Cookie, and Set-Cookie values are null with
|
||||
and their standard URI/form or JSON-escaped representations are replaced without
|
||||
decoding unrelated binary bytes. Other encodings are not interpreted. Digests describe the
|
||||
retained representation after redaction.
|
||||
The selected body-byte limit also bounds that retained representation after
|
||||
secret expansion, before base64 serialization. Credential header values are
|
||||
discarded before retained-header allocation accounting; their names and null
|
||||
projections still count. The transport/API has already materialized incoming
|
||||
bodies and headers, so these retained limits do not bound producer-side input
|
||||
allocation.
|
||||
|
||||
Only completed responses are read. The producer-observed terminal state is
|
||||
recorded independently of whether its event metadata fit the retention budget. An omitted completion event cannot turn a
|
||||
finished transaction into an unfinished one. When the final step ends, observation
|
||||
Only completed responses are read. At the end of the final step, observation
|
||||
stops; requests still in flight emit `request-unfinished`, and their observed
|
||||
responses have explicitly unavailable bodies. Already-admitted content reads
|
||||
drain before browser cleanup. At most 16 reads run at once; queued work is
|
||||
@@ -758,8 +422,8 @@ and compared as an unordered collection after caller rules apply, because
|
||||
concurrent responses arrive in any order and can land in a later step; a
|
||||
redirect retains the complete predecessor ancestry, including each hop's own
|
||||
request, response outcome, and selected network content. Each transaction's
|
||||
capture-local IDs, sequence/index fields, content source-sequence anchor and
|
||||
request times (`observed_at`, `started_at`, `elapsed_ms`) are removed. Caller rules normalize retained strings before hashing. The
|
||||
capture-local IDs, sequence/index fields, and content source-sequence anchor
|
||||
are removed. Caller rules normalize retained strings before hashing. The
|
||||
`preserve_redirect_transaction_ancestry` built-in rule fingerprints each hop
|
||||
as canonical SHA-256 of `{ events, redirected_from }`: `events` is the sorted
|
||||
list of its normalized event SHA-256 digests, and `redirected_from` is its
|
||||
@@ -780,10 +444,7 @@ scope/flags, value fingerprints, and duplicate entries even when redaction or
|
||||
literal rules make names collide. Captured storage arrays retain their original
|
||||
order. The `compare_storage_entries_as_unordered_collection` built-in rule
|
||||
participates in the normalization commitment; new storage comparison digests
|
||||
therefore differ from earlier order-sensitive comparisons. The
|
||||
`ignore_network_request_timing` built-in rule likewise joins the commitment, so
|
||||
normalization digests differ from comparisons made before it, though captures
|
||||
with and without request times compare alike.
|
||||
therefore differ from earlier order-sensitive comparisons.
|
||||
|
||||
Each aligned step reports changed, unchanged, or unknown. Changed and unknown
|
||||
action, screenshot, DOM, accessibility, URL, history, storage, and event
|
||||
@@ -797,67 +458,7 @@ selection. Captures with different selections report event evidence as unknown
|
||||
with a recovery instruction, even when no selected family emitted an event.
|
||||
Captures written before this field existed omit it, so their event evidence is
|
||||
also unknown.
|
||||
Retained-capture readers admit that omission without deriving a selection from
|
||||
observed events; other artifact families remain comparable. Current capture
|
||||
producers still require and publish the selected event families. Manifest,
|
||||
environment and script-export readers share the retained-capture admission
|
||||
rules and preserve the same missing fact.
|
||||
|
||||
Network normalization also commits the removal of capture-local transaction IDs,
|
||||
attribution to the requesting step, and unordered transaction comparison. These
|
||||
built-in rule identities participate in the normalization digest.
|
||||
|
||||
## Summary delivery and retained views
|
||||
|
||||
`capture_browser_scenario` accepts `detail: "summary"` to retain the complete
|
||||
capture in the MCP session and return its summary. The default remains
|
||||
`complete`. Delivery detail does not change scenario parameters or identity.
|
||||
|
||||
The summary reports each step's status, error, URL and captured/missing/truncated
|
||||
section names. It omits `not_requested` sections and repeated `before_url` values.
|
||||
The accessibility tree comes from the first unsuccessful step, or the final
|
||||
step on success; when that step captured no tree, its nearest earlier captured
|
||||
tree is identified explicitly. Event and network counts cover retained records,
|
||||
with capture omissions reported separately.
|
||||
|
||||
For the one-shot CLI, save the complete Evidence before selecting a summary:
|
||||
|
||||
```bash
|
||||
rea capture-browser-scenario scenario.json --detail summary --evidence_output /absolute/path/scenario-evidence.json --json
|
||||
```
|
||||
|
||||
Use the returned `normalized_result.parent_evidence_id` with
|
||||
`inspect_analysis_view` source `{kind: "retained-evidence", evidence_id: "..."}`.
|
||||
For a CLI bundle, use an `evidence-file` source with its absolute path and the
|
||||
parent Evidence ID. Views read the recorded capture; they never replay actions.
|
||||
|
||||
- `{kind: "summary"}` returns the same overview.
|
||||
- `{kind: "browser-scenario", collection: "steps", step_indices: [2], artifacts: ["accessibility"], offset: 0, limit: 1}` returns selected recorded steps and artifacts.
|
||||
- `{kind: "browser-scenario", collection: "events", event_kinds: ["response"], transaction_ids: ["request-2"], offset: 0, limit: 20}` pages selected events with original item indices.
|
||||
|
||||
Omitted filters select all; empty arrays select none. Pagination counts matching
|
||||
retained rows. Exhausting a page does not establish capture completeness or
|
||||
current application state.
|
||||
|
||||
Transaction pages use `{kind: "browser-scenario", collection: "transactions", limit: 20}`.
|
||||
Their stable identity is the parent Evidence ID plus `transaction_id`; each row
|
||||
keeps original event indices. Metadata events are returned by default. Select
|
||||
`event_kinds: ["network-content"]` for recorded bodies and header values, or
|
||||
`event_kinds: []` for identity and indices alone. A `step_indices` filter selects
|
||||
a whole transaction when any retained event belongs to a selected step; this
|
||||
is receipt attribution, not proof that the action caused the request.
|
||||
|
||||
For a large retained tree, request `{kind: "browser-scenario", collection: "text",
|
||||
step_index: 10, artifact: "accessibility", offset: 0, limit: 2048}`. `dom` is also
|
||||
supported. Offsets and limits count UTF-16 code units. The full artifact digest
|
||||
and byte size remain attached to each page. Missing, truncated and unrequested
|
||||
text stay explicit; source truncation counts are preserved without treating an
|
||||
exhausted page as a complete capture.
|
||||
|
||||
Network summary counts include every retained transaction identity, including
|
||||
those whose request event was dropped. Missing request events, conflicting
|
||||
metadata and observed request failures have separate counts. `transactions` is
|
||||
`null` when network observation was unrequested or its legacy selection is
|
||||
unknown and there are no retained transactions; an observed empty inventory is
|
||||
`0`. Full capture omissions remain available and each page carries source
|
||||
completeness, dropped-event counts and omitted-transaction accounting.
|
||||
@@ -1,77 +0,0 @@
|
||||
# Bun standalone executables
|
||||
|
||||
`inspect-artifact` inventories embedded Bun module graphs in Mach-O `__BUN,__bun`,
|
||||
PE `.bun`, ELF `.bun`, and appended trailers. `extract-artifact` writes their stored
|
||||
module contents through the existing safe output tree. Neither operation executes
|
||||
the executable or decompiles bytecode.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./hello --json
|
||||
rea extract-artifact ./hello --json
|
||||
```
|
||||
|
||||
In MCP, open the executable with `open_binary`, then call `inspect_artifact`.
|
||||
Use `detail: "summary"` for large graphs and `inspect_analysis_view` to page retained
|
||||
Evidence. The root metadata's `entry_point_logical_path` is the reader-owned
|
||||
location of its entry module. Use it as an inventory `logical_paths` filter,
|
||||
then read selected node fields with the returned content identity. This also
|
||||
works for names mapped to indexed extraction paths. The complete Evidence stays
|
||||
retained; no source-name normalization is needed in the caller.
|
||||
|
||||
The root reports the carrier, layout, module count, entry point, raw flags and
|
||||
compile exec-argv. Each module retains its source name, loader and encoding values,
|
||||
entry-point status, sizes and SHA-256. Sourcemaps, bytecode and module-info regions
|
||||
are separate children. Source names such as `/$bunfs/root/main.js` remain in
|
||||
metadata; extraction uses a relative path. Unsafe or non-UTF-8 names receive an
|
||||
indexed path and retain their source spelling or bytes. Virtual names deeper
|
||||
than the selected `path_parts` allowance (default 256) also use an indexed path with an explicit limitation,
|
||||
preventing unbounded path-tree work while retaining the source fact. Duplicate names remain
|
||||
separate occurrences; ordinary extraction collision rules apply.
|
||||
|
||||
The recognized record shapes are the 36-byte Bun 1.3.3 layout and the 52-byte
|
||||
Bun 1.3.9 layout. These labels identify structural layouts, not a detected compiler
|
||||
version. Bun does not store an explicit revision identifier: unknown table sizes
|
||||
and ambiguous known layouts are unsupported. A future layout that is byte-for-byte
|
||||
compatible cannot be distinguished. Invalid pointers and truncated claimed sections
|
||||
are malformed. An absent trailer without a named section does not establish a Bun
|
||||
graph. Unknown enum values remain raw, with a null name.
|
||||
|
||||
Sourcemaps are stored Bun regions, not necessarily standalone JSON source maps.
|
||||
Extracted contents retain their original encoding. Use JavaScript analysis only on
|
||||
compatible source files; bytecode is retained as evidence, not decoded.
|
||||
|
||||
Module parsing reads exact metadata ranges rather than loading stored asset
|
||||
bodies or whole pages around small pointers. A cache retains at most 256 KiB
|
||||
and 1,024 ranges for one reader preparation. Cumulative uncached metadata reads
|
||||
default to 256 MiB, checked before allocation or I/O. The module table,
|
||||
decoded strings and combined logical path tree have separate admission checks;
|
||||
by default at most 250,000 distinct path prefixes are admitted before inventory construction.
|
||||
Stored bodies are range-streamed for hashing and extraction. Header/suffix observations are reused
|
||||
only for the same admitted file state, with selected-path checks before publication.
|
||||
|
||||
Both artifact operations accept `artifact_limits.bun`. Its independent controls
|
||||
are `payload_bytes`, `module_count`, `representation_bytes`, `metadata_read_bytes`,
|
||||
`recognition_table_bytes`, `recognition_load_command_bytes`, `path_parts` and
|
||||
`path_nodes`. Failed recognition remains explicitly limited instead of proving
|
||||
absence. ELF section names are examined at their actual offsets, including names
|
||||
beyond a large section-name prefix. Select `artifact_limits.expanded_bytes` to
|
||||
change the aggregate referenced-plaintext work allowance.
|
||||
|
||||
For a reported metadata-read refusal, keep the same carrier and retry with
|
||||
`--artifact-limits '{"bun":{"metadata_read_bytes":536870912}}'` or the equivalent
|
||||
MCP `artifact_limits` object. Omitted fields keep defaults. The response names
|
||||
the effective limit and exact repair; accepted overrides are reported separately
|
||||
from caller selections. Changes apply on the next call without restarting REA.
|
||||
Summary and selected extraction do not change the producer's admission.
|
||||
|
||||
See [limits and recovery](design/limits-and-recovery.md) for admission ceilings.
|
||||
Synthetic tests cover the carriers and malformed declarations. Run
|
||||
`node scripts/verify-bun-standalone.mjs` after `npm run build:cached` for local Bun
|
||||
verification, and add `--docker` to compile a Linux fixture with `oven/bun:1.4.0`.
|
||||
Bun and Docker are fixture producers only and are not installed by REA.
|
||||
|
||||
Layout references: [Bun 1.3.3 serializer](https://github.com/oven-sh/bun/blob/bun-v1.3.3/src/StandaloneModuleGraph.zig)
|
||||
and [Bun 1.3.9 serializer](https://github.com/oven-sh/bun/blob/bun-v1.3.9/src/StandaloneModuleGraph.zig).
|
||||
|
||||
The current 52-byte record order can also be checked in the
|
||||
[Bun 1.4.4 Rust serializer](https://github.com/oven-sh/bun/blob/bun-v1.4.4/src/standalone_graph/StandaloneModuleGraph.rs).
|
||||
@@ -1,107 +0,0 @@
|
||||
# Capability composition and ownership
|
||||
|
||||
REA keeps domain contracts and shared workflows separate from provider adapters
|
||||
and CLI/MCP translation. A capability can have several provider identities; a
|
||||
provider can contribute operations to several capabilities. Those associations
|
||||
are source facts, while host availability is an observation.
|
||||
|
||||
## Source declarations and runtime bindings
|
||||
|
||||
`src/composition/capabilityDeclarations.ts` contains static provider identities,
|
||||
named contract-group references and precise operation subsets. Complete groups
|
||||
refer to `TOOL_CONTRACT_GROUPS` in the existing contract owner; family definitions
|
||||
and provider catalogs read the same arrays, so a new group operation is included
|
||||
without updating a copied provider list. Metadata imports only the group ID type
|
||||
and pure metadata leaves; it must not load factories, schemas, engines or
|
||||
generated catalogs. Catalog generation resolves groups and subsets against the
|
||||
canonical contracts and rejects unknown groups/operations and conflicting
|
||||
identities. New provider provenance must come from the implementation
|
||||
that actually produces the Evidence, including format-specific capture adapters
|
||||
and derived workflows.
|
||||
|
||||
`src/composition/capabilityServices.ts` binds typed service factories to those
|
||||
capability keys. Each runtime entry owns construction, cleanup and any available
|
||||
host/configuration facts or live probes. A service entry requires a close
|
||||
capability; entries adapting managers retain their existing `closeAll` operation.
|
||||
The host maps entries without erasing their individual service/port types. There
|
||||
is no generic JSON operation dispatcher.
|
||||
|
||||
These two declarations have different loading needs. Static facts can be read
|
||||
without loading a schema or provider graph. The full runtime composition is
|
||||
materialized only when the MCP runtime needs it. One-shot CLI adapters import the
|
||||
selected factory rather than constructing a complete capability host. CLI and MCP
|
||||
reuse the same provider/service factories and application workflows, while
|
||||
keeping their respective argument translation and lifetime.
|
||||
|
||||
## Resource lifetime
|
||||
|
||||
`CapabilityScope` registers a module's actual close capability when its instance
|
||||
is composed. There is no independent server provider-close list to update.
|
||||
Concurrent shutdown calls join the same cleanup attempt. A successful peer is
|
||||
removed from pending cleanup; failed peers remain owned and are retried on the
|
||||
next close. Synchronous throws and rejections without a reason remain failures,
|
||||
and cannot prevent other peers from attempting cleanup. Shutdown seals further
|
||||
ownership registration.
|
||||
|
||||
The host distinguishes these lifetimes:
|
||||
|
||||
- Host-owned service instances are released by that host.
|
||||
- Caller-owned optional observations remain borrowed by SDK server instances;
|
||||
their process-level owner closes them, including discarded discovery probes.
|
||||
- CLI invocations use their existing finalizers to join cleanup and preserve the
|
||||
original observation, cancellation and cleanup errors.
|
||||
- Binary engines, bundle readers and investigation records retain their existing
|
||||
owners. The `BinarySession` transition/drain barrier still coordinates target
|
||||
changes, reset and disposal. A module refactor must not create independent
|
||||
barriers that allow late work to repopulate reset records.
|
||||
|
||||
An application workflow may borrow a shared resource scope, such as byte entropy
|
||||
using the host's artifact scope. Its wrapper does not acquire a second ownership
|
||||
claim over the borrowed scope.
|
||||
|
||||
## Availability and registration
|
||||
|
||||
Module bindings provide host support, selected configuration and precisely typed
|
||||
live availability probes through `CapabilityAvailabilityFacts`. Constructors do
|
||||
not run these probes or launch engines. Session discovery invokes the existing
|
||||
provider-owned probes only for advertised families; live failures remain fresh
|
||||
observations. Caller policy, selected environments and injected providers retain
|
||||
their existing precedence.
|
||||
|
||||
MCP registration remains in the MCP adapter. It consumes composed services,
|
||||
record ports, admission and logging; it must not add fallback constructors that
|
||||
can escape the host owner. Named contracts bind handlers and schemas. The complete
|
||||
tool-family registrar map is checked by TypeScript, and catalog order never
|
||||
selects a handler or schema.
|
||||
|
||||
## Extending a capability
|
||||
|
||||
1. Add or reuse the domain contract and application workflow, with explicit
|
||||
Evidence provenance, coverage, unknowns and caller choices.
|
||||
2. Put concrete construction in `src/composition/` and keep its default factory
|
||||
lazy. Reference adapter-owned pure identity metadata from both the provider
|
||||
and its static capability declaration.
|
||||
3. Add the runtime binding if a host-owned service is needed. Bind cleanup and
|
||||
availability/probes there, preserving borrowed versus owned resources.
|
||||
4. Add the CLI/MCP argument adapters and their named registrations. A one-shot
|
||||
command uses the shared selected factory and existing finalization helpers.
|
||||
5. Run source-boundary, discovery, SDK-schema and relevant public workflow checks.
|
||||
Exercise partial cleanup, retry, cancellation and caller-owned peers at the
|
||||
applicable production seam; use actual applications/providers for routing and
|
||||
complete Evidence verification. Recording ports do not prove real providers.
|
||||
|
||||
The constructor boundary guard covers the centralized Frida, Cutter, GDB and
|
||||
Rizin constructors without rejecting legitimate type-only injected interfaces.
|
||||
Other settled pure/provider boundaries continue checking type imports too. The
|
||||
shared parser records module-edge erasure once. Whole-declaration type imports
|
||||
are erased; inline type bindings retain their runtime module edge under
|
||||
`verbatimModuleSyntax`. Static dynamic imports remain runtime edges too.
|
||||
Tests belong with their semantic owner and must be
|
||||
found exactly once; provider roots need the corresponding adapter discovery lane.
|
||||
|
||||
This is an internal composition contract. External plugin distribution, automatic
|
||||
provider installation and a universal provider interface are separate concerns.
|
||||
|
||||
The [initial verification report](verification/capability-composition-2026-11-01.md)
|
||||
records the executed public workflows, cleanup boundaries, resource measurements
|
||||
and remaining provider qualification gaps for this composition increment.
|
||||
+2
-2
@@ -65,14 +65,14 @@ cancellation can still prevent upload.
|
||||
that analysis was complete. A successful operation can retain partial analysis,
|
||||
unknowns, and limitations. Exit zero is not a security verdict, proof of complete
|
||||
coverage, or a policy gate for those findings. Inspect the saved Evidence and
|
||||
coverage before making such a decision. See [output and exit status](getting-started/cli.md#output-and-exit-status).
|
||||
coverage before making such a decision. See [output and exit status](cli.md#output-and-exit-status).
|
||||
|
||||
A nonzero exit can represent invalid input, unavailable tooling, an unsupported
|
||||
host/capability, or another operation failure. Keep the original JSON error (when
|
||||
emitted), stderr, and exit code, and use their category/message to diagnose it.
|
||||
An early startup failure may leave `analysis.json` empty; do not parse that as
|
||||
successful analysis. Select a supported operation or explicitly provision its
|
||||
[documented prerequisites](getting-started/installation.md); do not silently switch providers,
|
||||
[documented prerequisites](installation.md); do not silently switch providers,
|
||||
execute the artifact, or ignore the failure to obtain a green job.
|
||||
|
||||
To exercise the failure path, temporarily set `ARTIFACT_PATH` to
|
||||
|
||||
Loaded 100 of 6563 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user