Compare commits

..
Author SHA1 Message Date
790cdfdefa test(experiment): freeze client wait guidance diagnostic
Keep identical analysis contracts, question, engine and complete catalogs. Test common opt-in wait guidance while retaining polling and native latency in whole-task costs.

Co-authored-by: gensokyo-dev <[email protected]>

Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 16:02:44 +00:00
c795081e78 docs(experiment): record real-agent selected-case results
Report both correct answers and complete task costs. Preserve the native latency and client polling confounds; no general Code Mode advantage is established.

Co-authored-by: gensokyo-dev <[email protected]>

Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 15:58:53 +00:00
f44dca9c37 feat(experiment): expose native case-to-edge facts
Share exact jump-table metadata and paged case relationships across CLI,
MCP and Code Mode. Keep labels, row identities, recovered edges and machine
defaults separate; reclaim independent facts under output capacity.

Verify all 182 SQLite table targets against ELF bytes through both MCP
adapters and retain cases/calls when a real 2 MiB native budget omits IR.
Freeze the next analyst question and oracle before real-model trials.

[AUTOMATED] Experimental native patch; no upstream integration or PR.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 15:32:12 +00:00
29e0472812 test(experiment): measure queued and repeated native reads
Compare sequential and overlapping requests through both real MCP adapters
with the same preserved engine, selected instruction windows and full graph.
Retain the unexplained model-run anomaly without claiming a speed fix.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 14:25:22 +00:00
4882f6b5dc docs(experiment): retain guided model comparisons and limits
Verify both repeated answers and exact discovery calls against unchanged
source facts. Record complete task costs, overlapping requests, unexplained
capture-time variation and the client executor available to both lanes.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 14:14:27 +00:00
7a7c4a6d7a docs(experiment): record real-agent control-flow evidence
Review both complete answers against the precommitted oracle and independent
ELF instruction bytes. Retain all costs, validation retries, discovery noise
and the limits of this single observational pair.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 14:01:07 +00:00
e2205313a5 feat(experiment): expose bounded full recovered graphs
Preserve complete graph facts with a bounded lossless native wire codec and
shared paged Evidence views. Verify both MCP adapters and retained legacy
records, then freeze a control/data-flow question and oracle for real agents.

[AUTOMATED] Exploratory native patch; no upstream integration or PR.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 13:47:33 +00:00
7d0da075e7 perf(experiment): measure native analysis phase costs
[AUTOMATED] Add a gated diagnostic overlay and production-owner profiling runner. Preserve actual phase/resource results, disabled-output control and lossless full-graph encoding feasibility without claiming a public codec or model speedup.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 12:41:35 +00:00
4986b8644f docs(experiment): record indexed discovery model costs
Retain both correctness-equal wide-PHI pairs, complete discovery costs and the remaining whole-function native computation boundary.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 12:22:02 +00:00
a440049026 feat(experiment): expose a schema-free method index
Discover declared methods through the shared reference owner before requesting named contracts; preserve the existing default and runtime capability semantics.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 12:15:10 +00:00
e678be3f9d docs(experiment): record selected operand model results
Both adapters satisfy the fixed wide-PHI oracle with one capture and selected slots. Preserve mixed timing/token costs and the observed discovery overhead rather than inferring a Code Mode advantage.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 12:12:09 +00:00
e38a4c4394 feat(experiment): page native SSA operand relationships
Keep operation and slice defaults compact; preserve exact input slots and phi edges, distinguish unretained seeds, and align advertised relation schemas with runtime validation. Pin a seeded wide-PHI model task after real public MCP verification.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 12:05:15 +00:00
a225d7c552 fix(experiment): retain direct analysis failures in trial ledgers
Record the second PHI model pair, including the native timeout and successful retry; keep adapter conclusions limited to observed whole-task costs.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 11:57:25 +00:00
14bc2bca21 fix(experiment): align named reference lookup with input contracts
Share displayed method names with direct registration, support explicit multi-name selection and unmatched terms, and cache immutable schema conversions by schema identity. Export input-side schemas so defaulted fields stay optional; verify both actual MCP catalogs and named contracts before repeating the unchanged model task.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 11:41:48 +00:00
71be07de42 docs(experiment): record real-model phi projection results
Both answers satisfy the fixed four-alternative oracle with one VM capture and scoped facts. Preserve whole-task costs and source checks, including the direct reference-name lookup miss that needs a shared discovery fix.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 11:30:18 +00:00
8114bd675e feat(experiment): expose bounded call-rooted definition facts
Share native membership and operand ownership across full and projected facts. Preserve exact phi bindings and definition frontiers when the full graph is unavailable, keep CFG/use coverage explicit, and expose selected reads through the same CLI/SDK/MCP contracts. Verify real SQLite/zlib fidelity, encoded fallback, bounded metadata, and cache reuse; pin the next real-model question and oracle.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 11:16:26 +00:00
4d7fccbce6 docs(experiment): measure bounded native definition projections
Preserve a diagnostic-only patch and reproducible lossless encoding/projection probe. The real VM graph exceeds several public capacities; selected SSA neighborhoods are much smaller, with explicit frontiers. Record actual resource costs and keep public engine/API limits unchanged.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 10:30:19 +00:00
af53e0fbe5 fix(experiment): retain ownership after tagged cleanup failures
Check the native cleanup Result before clearing the lease and preserve cleanup resources through the MCP error adapter. Reproduce the old false success, then verify real SDK close/replacement ownership and both MCP retry paths with explicit fault injection.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 10:25:17 +00:00
f08c191ed3 docs(experiment): compare compact call counts in real model tasks
Retain the successful answer and source checks from both adapters and all measured task costs. Both avoid call-set paging; direct beats Code Mode in the new sample and before/after effects remain mixed.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 10:07:36 +00:00
da6f5470be perf(experiment): summarize retained calls before paging
Expose a compact call distribution in snapshot metadata through the shared validated owner. Preserve coverage, unknown targets, dead rows, immutable selected reads, and empty versus unavailable semantics. Retain the same boundary question and oracle for paired real-model comparison.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 10:00:04 +00:00
fb9adbf384 docs(experiment): record real-model call boundary results
Both lanes satisfy the pre-pinned SQLite task with one retained VM analysis. Record full answer/source review, actual model and resource counters, and mixed context costs without a general efficiency claim.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 09:53:53 +00:00
f9f9dc209d feat(experiment): retain call facts beyond full graph capacity
Separate native call and boundary-value ownership from the bounded recovered graph. Validate facets independently and preserve shared identities, source evidence, and explicit partial coverage. Pin the SQLite boundary task and oracle before paired real-model trials.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 09:46:42 +00:00
04c081fc61 experiment(re): verify model reference aggregation against machine calls
Record both completed model workflows, whole-task costs and source/identity checks. Compare every VM direct-call pair with independent disassembly, preserve indirect-call limits, verify unpatched stock through the shared CLI/API, and retain failed/partial research costs.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 09:01:54 +00:00
3aa685db39 feat(experiment): compose shallow outgoing references with explicit scope
Share one native parser for both directions, preserve occurrence identity and unknown function applicability, push kind filters to Kuna and reuse sources across fields/pages/scopes. Verify real SQLite machine references independently, including calls omitted from optimized IR and VM queries beyond graph capacity. Pin the aggregate model task before its paired runs.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 08:48:24 +00:00
56182b2217 experiment(re): verify cross-function model answers and mixed adapter costs
Record the pre-pinned explicit-return-path pair with source and identity validation. Both models complete the context-flow task; Code Mode has fewer MCP requests without a demonstrated overall efficiency advantage. Preserve large-function and shallow-reference coverage limits.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 08:20:24 +00:00
e9348df0c3 experiment(re): align cross-function question with pinned return-path oracle
Retain both initial real-model answers and whole-task costs. Both prove the core context connection but omit the oracle-required return-code distinctions; clarify that requirement before the next pair without changing the oracle or supplying addresses.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 08:10:41 +00:00
01e5c59551 fix(experiment): validate native engine pins before model trials
Preserve the rejected cross-function launch caused by a non-embedded Cargo test build replacing the mutable CLI. Check explicit engine identity before profile creation and after the task; use an independent executable copy for measured runs.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 08:03:30 +00:00
33c4282b8a feat(experiment): expose structured native call and argument relations
Keep recovered callee identities separate from fspec annotations and SSA argument values separate from passing storage. Add selected calls and function-input reads through the shared fact owner, with real SQLite MCP coverage and a pre-pinned cross-function model task.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 07:58:01 +00:00
f34425dc24 fix(experiment): enforce Code Mode payload budgets on guest errors
Reject oversized error delivery with explicit omission metadata, account for JSON escaping, and retain source observations and their lookup after failure. Reproduce and verify the real MCP path without claiming model-token savings from byte counts.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 07:33:09 +00:00
dad0228df8 experiment(re): verify model composition of class and dispatch tables
Record matching six-byte table oracles, real byte-tool use, model response counts and whole-task costs without generalizing a single paired observation.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 07:23:40 +00:00
73db4fad54 feat(experiment): compose bounded native byte windows with RE facts
Add shared SDK/CLI/MCP byte reads over the existing Kuna data protocol, preserve partial batches and exact u64 ranges, and retain complete source Evidence. Reuse named read lifecycles, bound aggregate hex expansion, and verify stock/native/direct MCP paths plus a shipped-file byte oracle before pinning a table-dispatch model task.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 07:17:18 +00:00
c53c2a9b90 experiment(re): record equal-coverage model use of retained Pragma code
Keep actual response/MCP counts, whole-task token usage and resource receipts separate. Both adapters complete the local task with explicit missing IR; no general speedup is inferred.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 06:58:46 +00:00
9e1b39358a experiment(re): pin a model task over recovered Pragma code
Exercise the restored code facet with a question fixed before trial and independent instruction anchors, using identical envelope guidance in both atomic calling modes.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 06:52:03 +00:00
1346e97dca fix(experiment): preserve code when encoded native graphs exceed capacity
Bound compact native JSON with the existing writer, keep a valid code facet when the graph cannot fit, and pass the supervised byte budget explicitly. Verify real SQLite Parser/Pragma captures, exact preserved C/mappings/graph facts and public MCP reads. Keep string inventory parsing/projection under its own owner.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 06:50:20 +00:00
dc022bcd0f experiment(re): separate model rounds from MCP requests in fair trials
Validate per-response usage totals, retain the negative large-VM comparison and qualify the client executor, cache and investigation-depth differences. Add optional common envelope/stopping guidance without relabeling historical runs.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 06:29:26 +00:00
5aa78c873e experiment(re): pin coverage criteria for a large SQLite VM task
Record independent machine-code anchors before model trials, separate cache regimes and incomplete answers, and exercise real recovered-IR capacity failure without hiding retained code.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 06:09:46 +00:00
15d75ea1ea fix(experiment): preserve cached fact ownership for SDK callers
Freeze parsed inventories once at their owning boundary and share the rule with native IR and function views. Verify the real SDK mutation regression and retain corrected equal-depth SQLite model costs, including persistent-cache limitations.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 06:07:46 +00:00
e9b9a8dd0e experiment(re): describe direct atomic trials without a Code Mode tool
Keep the direct adapter benchmark instructions consistent with its actual registered tools; retain historical trial profiles unchanged.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 05:48:57 +00:00
f3f209d03f experiment(re)!: preserve partial captures and resolve function entries
Keep validated code available when native graph extraction is unavailable, bind output to the actual input/entry, preserve cancellation observations, and share batch source/error semantics. Add filtered function/alias reads and slice field selection across adapters.

Verified 38 focused tests, targeted lint, real stock/native MCP workflows, large SQLite graph-capacity fallback, and real cold/warm MCP reuse. Retain actual model costs and unequal-depth/selection limitations instead of claiming general superiority.

BREAKING CHANGE: Experimental native snapshots may succeed with readable code and an unavailable recovered_ir facet. Inspect facets before graph reads; unavailable graph counts are null/unknown.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 05:48:03 +00:00
4f13bd6733 experiment(re)!: separate IR origins from machine instruction facts
A real model conflated an optimized CFG edge with the bytes at its instruction origin. Make provenance explicit and add bounded, batched instruction windows from the existing Kuna CLI, preserving exact bytes, per-window failures and Evidence. Record the task-cost comparison without treating the incomplete machine-path answer as an equal-quality win.

Verified 33 focused tests, targeted lint, real stock/patched MCP decoding with mixed success, and a real 4096-row capacity run under the resource observer.

BREAKING CHANGE: Experimental operation.address/addresses and block.start become origin_address/origin_addresses and origin_start. Printed CFG target_address becomes target_origin; historical source captures are unchanged.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 04:51:55 +00:00
f2f029c5dd experiment(re): retain native model trace and task-cost limits
The real model uses recovered controllers and indexed phi/value relations, but the first native-IR trial still does not beat the traditional local-question workflow. Record actual whole-task usage and prepare value-origin and small-function counterexamples.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 04:33:55 +00:00
a7e9f8843b experiment(re): expose native CFG and value identities for composition
[AUTOMATED] Preserve the opt-in pinned Kuna exporter as a reproducible source patch. Share typed graph/value reads and derived control dependence through the SDK, CLI and MCP adapters, retaining unknown runtime and memory semantics.

Validated 29 focused tests, lint, actual diagnostic/native MCP workflows and a real zlib CLI composition. C and line mappings match the prior stock capture. Full upstream corpus/workspace gates and revised real-model costs remain unverified.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 04:28:22 +00:00
fcfc87b2b4 experiment(re): record actual three-way model task costs
Retain failed attempts, cached input accounting and payload-exit process-family counters. The first comparison favors the traditional workflow; keep the negative finding explicit.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 04:00:19 +00:00
c5cefd622b experiment(re): refine composition contracts from a real model trace
Record genuine Code Mode use and full model usage rather than response-only estimates. Describe return shapes, permit byte-budgeted internal C reads, separate guest CPU from analysis deadlines, and move subsequent model profiles outside the source tree. Verify lease, admission, cleanup-retry and cancellation semantics.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 03:28:30 +00:00
5aa1c5c64f experiment(re): expose shared atomic facts through isolated Code Mode
Add a compact rea-codemode MCP server, bounded QuickJS execution and shared SDK/direct adapters. Reuse native target and process owners; retain evidence and lease-scoped snapshots. Extend the real-model trial runner with an isolated experimental MCP registration. Capability and evaluation work remains in progress.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 03:17:05 +00:00
2e11fc23fd experiment(re): validate guard slices and record token tradeoffs
Verify three repaired zlib mappings, a SQLite call guard and all ordinary 16-bit header values against retained native evidence. Record producer omissions, failed-run costs and the small-function context counterexample. Preserve existing capture outputs with exclusive writes.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 02:59:46 +00:00
fd79a273f3 experiment(re): compose native SSA uses and branch dependencies
Retain bounded native snapshots and public MCP evidence, then query explicit SSA relations without interpreting reconstructed C. Preserve unknown flow, cloned block identity and dependency frontiers.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 02:59:46 +00:00
d711c55aed experiment(re): distinguish point facts from object selectors
Apply the interior-reference counterexample to the facade: label exact-address scope and emit location endpoints while preserving caller selectors separately. Carry canonical target-content provenance through experimental MCP replies and verify it on the real client. Keep original measurements pinned to the v1 response shape. Nine experiment checks and real Kuna lifecycle/context checks pass.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 02:59:46 +00:00
72b24433ea experiment(re): record native sharing and composition counterexamples
Measure twelve equivalent MCP navigation workflows, native reference plans on SQLite/zlib, a compiled interior-reference oracle and a failed exact code-mapping composition. Preserve failed attempts and measurement limits, including unavailable live-agent effectiveness evidence. Add bounded native probes and deterministic mapping tests. All 191 Kuna cases pass serially; nine experiment tests and one-checker typecheck pass.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 02:59:46 +00:00
2e19ece025 experiment(kuna): preserve typed reference facts at their owner
Expose source-owned reference kind, endpoints, function identities and instruction text for experiments. Keep the existing address-only public result and retained-cache shape; materialize richer facts only when explicitly requested. Record prior art, composition rules and the distinction between fact, transport and compute units. Verified 18 producer boundary cases and an independently checked native interior-reference example.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 02:59:46 +00:00
27f981352d experiment(re): compare fact granularity through real MCP
Add a non-shipping MCP facade over the existing admitted analysis port, real-provider comparison runner and offline token accounting. Keep the full catalog, source Evidence and per-input failures. Separate verification runs from equivalent-work timings. Kuna zlib comparisons pass; broader comparisons remain in progress.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 02:59:45 +00:00
0d99e8c5c5 experiment(re): define composable facts and context boundaries
Start a non-integrating experiment around fact identity, per-input outcomes, paging and shared primitive composition. Include deterministic boundary checks; real-provider comparisons remain in progress.

Co-authored-by: gensokyo-dev <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
2026-10-16 02:59:45 +00:00
7458 changed files with 237861 additions and 1507614 deletions

No files matched your search

-21
View File
@@ -1,21 +0,0 @@
{
"name": "rea",
"interface": {
"displayName": "REA"
},
"plugins": [
{
"name": "rea",
"source": {
"source": "npm",
"package": "rea-agents",
"version": "6.8.0"
},
"policy": {
"installation": "AVAILABLE",
"authentication": "ON_INSTALL"
},
"category": "Developer Tools"
}
]
}
@@ -1,118 +0,0 @@
---
name: rea-analysis-optimization
description: Optimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.
---
# REA analysis optimization
Find a concrete source of unnecessary work, change it at its owner, and verify
equivalent completed investigations. A useful outcome can also be a measured
decision to leave an implementation alone.
Follow the checkout's [performance priorities](../../../AGENTS.md#performance),
[tool design](../../../docs/development/tool-design.md), and
[testing guide](../../../docs/development/testing.md). This skill supplies an optimization
workflow, not additional permission to execute targets, publish, or merge.
Use the current contracts, provider prerequisites, and user authorization.
Reserve performance measurements for changes that target performance or have a
concrete regression concern. Routine cleanup uses correctness checks and does
not require this optimization workflow or synthetic benchmarks merely because
helper calls, imports, or representations change.
Read [the Kuna case study](references/kuna-case-study.md) when selecting a
reuse strategy or interpreting a benchmark. Its pinned observations illustrate
tradeoffs; they are not current-provider guarantees or universal speedups.
## Pin the question and the correctness oracle
Describe the analyst's completed task, including the facts and relationships
needed to answer it, requested detail, source access, and meaningful unknowns.
Choose a representative real artifact and a small or empty case. Record the
baseline revision, target digest, engine identity, options, host, and entrypoint.
Use a separate checkout for changes without disturbing other work.
Establish an oracle independent of the changed implementation: source-built
fixture expectations, independently mapped file bytes, or a native producer
paired with checks of its interpretation. Comparing two wrappers around the
same faulty decoder does not establish correctness. Preserve address widths,
same-address alternatives, unresolved endpoints, coverage, and source linkage.
An empty result proves absence only within the actually examined scope.
## Locate the earliest unnecessary work
Trace one public CLI or MCP task through shared application code to its producer.
Measure relevant phases: admission and identity checks, native analysis,
decoding, lookup, projection, serialization, Evidence retention, and cleanup.
Include catalog discovery and extra calls when they affect the completed task.
Use the evidence to choose the smallest structural correction:
| Observed cost | Candidate correction | Condition to establish |
| --------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| Repeated scans of validated facts | Reuse existing order or lookup structure | The owner establishes ordering and identity; every matching alternative survives |
| Repeated equivalent engine work | Share a validated result or in-flight producer | Inputs, freshness, lifetime, and per-caller outcomes remain explicit |
| Full reports for selected facts | Push selection or projection to the earliest capable owner | Required facts and evidence remain reachable; examined coverage is reported |
| Large temporary transport buffers | Decode incrementally or use bounded source reads | Validate the whole admitted document; retained facts and copies still consume memory |
| Many repeated homogeneous reads | Share admission, decoding, or batch work | Preserve item identity, order where required, and individual failures |
Do not introduce an index, generic cache, batch endpoint, or orchestration layer
without identifying the work it removes and its maintenance/resource cost.
Projection after materializing a full report may reduce output while leaving
analysis time and peak memory unchanged. Fix missing or incorrect facts before
benchmarking an alternative composition against a better-covered workflow.
## Keep one interpretation and a defined reuse lifetime
Let the producer boundary validate and normalize facts once. Consumers query
that representation through the shared CLI/MCP implementation; adapters should
not rebuild relationships or independently reinterpret raw reports. Keep lookup
identity separate from display formatting. Reuse contracts when their intent
matches, honoring requested detail and complete Evidence access.
For every cache or shared producer, state its key, owner, invalidation, retained
size, and lifetime. Account for result-affecting target bytes, engine contents,
analysis options, architecture/load mapping, and parser/schema versions where
they affect reuse. Validate live state before calling a cached observation
current. Cache a refusal only when it is deterministic for that validated key;
malformed, partial, transient, or cancelled attempts need explicit retry policy.
One cancelled waiter must not invalidate a successful peer. Preserve producer
failure context and each caller's cancellation reason. Retain ownership and
capacity charges until shared work and cleanup settle; preserve completed
observations when cleanup fails. Exercise invalidation, concurrent consumers,
last-waiter cancellation, close, and cleanup retry when those paths change.
Bound the process family and retained state, not just a parent heap or worker
count. Account for decoding and serialization expansion before allocation.
Choose CPU, memory, concurrency, and disk limits from the host and workload;
stop before OOM and verify owned cleanup. A guard-stopped task is incomplete.
Do not replace a transport buffer cap with an unbounded retained inventory or
hide partial results behind a success status.
## Compare completed tasks and limit the claim
Run the affected real public workflow before and after with equivalent inputs,
results, coverage, and Evidence semantics. Include small inputs so setup costs
and regressions are visible. Distinguish fresh sessions, warm runtime caches,
and OS file caches. Alternate or repeat runs when variance affects the decision.
Choose meaningful boundary regressions for cases the actual engine cannot
reliably reproduce; a fake engine does not qualify a real provider.
Report task outcome, relevant phase timings, total elapsed and CPU time, peak
process-family memory and its accounting method, resource stops, and cleanup.
Check the payload exit status as well as the profiler's. Keep failed attempts
and identify unmeasured costs; a passing retry does not explain a failure.
For token claims, count schemas/discovery, arguments, replies, retries, and
continuations needed to finish the task. Label tokenizer estimates separately
from actual model usage and record the client/catalog behavior. Smaller JSON
alone establishes neither fewer model tokens nor better investigation quality.
Do not claim a complex-software advantage without completed representative
investigations and comparable correctness/coverage.
Finish with the bottleneck, owning change, preserved semantics, executed
checks, before/after measurements, and remaining limits. Keep any integration
record current with its tested revision and distinguish merged production work
from experiment-only code. Stop when the demonstrated bottleneck is resolved
or further benefit remains speculative.
@@ -1,4 +0,0 @@
interface:
display_name: "REA analysis optimization"
short_description: "Optimize analysis with preserved Evidence and measured costs"
default_prompt: "Use $rea-analysis-optimization to find and verify a concrete analysis bottleneck while preserving correctness and Evidence."
@@ -1,112 +0,0 @@
# Kuna: reuse before adding machinery
Read this reference for concrete examples of the skill's decisions. The records
below concern static Linux analysis with caller-supplied Kuna. They do not
establish parity with another provider or a general advantage for Code Mode,
atomic tools, or any particular orchestration design.
## Exact-address lookup: a measured, narrow improvement
[PR #4695](https://github.com/morluto/rea/pull/4695) reused the validated string
inventory's numeric address order. Previously, each `list_strings(address)`
filtered all cached facts. The owner now performs a binary lower-bound lookup
and returns every encoding at that address: O(log n + matches), without a
second retained lookup index.
The owning implementation is
[`KunaStringInventory.ts`](../../../../src/kuna/KunaStringInventory.ts), consumed
by [`KunaStringQueries.ts`](../../../../src/kuna/KunaStringQueries.ts). Inspect
their current versions before adapting the example. Numeric u64 addresses,
same-address encodings, duplicate rejection, source digests, caller selection,
and the existing session lifetime were part of the contract. Full listings and
literal/regex searches kept their semantics. An attempted removal of the
duplicate-detection set was dropped: no stable memory benefit was established,
and it delayed malformed-input rejection.
### Recorded workload and results
The benchmark used unmodified Kuna 1.833 and LLVM 16 on Linux x86-64, with
602,544 string facts. Baseline was `be0b2d138`; the final feature was
`5f564962da0e1b17f00a68a74a312539f0209a01`, subsequently integrated in
`c6b62754b095a134d6d67877508fa34311a2eda2`. These are historical observations,
not a benchmark of every subsequent main revision.
The same production stdio MCP task discovered the catalog, opened the binary,
made one cold selected read and 28 warm exact reads, checked retained Evidence,
and closed the session. Five selected addresses and two absent selectors were
read four times. A native filtered scan selected the strings; an independent
ELF load-header mapping checked the source bytes and terminators. Returned
facts, source linkage, target digests, and cleanup were checked.
Three alternating fresh sessions per implementation used one CPU, a 4 GiB
process-family hard limit, a 3 GiB early stop, and no swap. OS file caches were
warm. Whole-harness costs include the independent native oracle, hashing,
catalog/client work, and correctness checks.
| Median | Before | After |
| ---------------------------------------- | ---------: | ---------: |
| 28 warm exact MCP reads | 495.2 ms | 90.8 ms |
| Cold selected MCP read | 11.30 s | 11.01 s |
| Whole harness elapsed | 28.24 s | 27.43 s |
| Whole process-family CPU | 27.71 s | 26.89 s |
| Kernel family peak, including file cache | 2237.6 MiB | 2240.6 MiB |
Warm totals ranged from 491.7–544.3 ms before to 84.8–99.5 ms after: about
5.5 times faster at this phase. Cold/whole-task timing and memory overlapped;
no stable improvement in those costs was established. One small SQLite pair
(7,985 facts) measured 20 warm reads at 88.6 → 80.5 ms and the whole harness at
13.68 → 13.94 s. The large-inventory lookup benefit does not generalize to a
whole-task speedup on small inputs.
Catalog hashes, call counts, and normalized facts were unchanged. A
`cl100k_base` estimate over one pass of raw catalog JSON, arguments, and replies
was about 846,000 tokens in both lanes, with about 802,562 from the catalog.
This was transport serialization, not rendered model context or actual usage;
client discovery, prompts, replay, and protocol framing were unmeasured.
**No token reduction or improved agent investigation outcome was established.**
Pinned target SHA-256:
`f62d254b7f2bf42df8c8b07d46ee3bb4c2cafeca436b2e6bc6ccbe4581f58f40`.
Pinned engine SHA-256:
`d858918b0766435cd96b999a77f2d5a31a0cc4f07256421c883b5b9dc4dd58e4`.
All six normalized-result sequence digests were
`4d2f90269da8568984090b20274163c4a7c2a0538aab63530bd2fe44840a0bba`.
Selected byte agreement does not prove native discovery completeness; missing
strings and relationships can remain unknown.
## Other transferable changes and their limits
- [Cached string inventory and search, #4095](https://github.com/morluto/rea/pull/4095):
share validated facts across queries in a defined session. Audit key inputs
and lifetime before extending reuse across targets or engine revisions.
- [Shared single-function work, #4045](https://github.com/morluto/rea/pull/4045)
and [repeat-request reuse, #4169](https://github.com/morluto/rea/pull/4169):
avoid duplicate native work while retaining per-caller cancellation and
cleanup ownership. A query count alone cannot reveal duplicate subprocesses.
- [Streamed strings, #4318](https://github.com/morluto/rea/pull/4318)
and [streamed call graph, #4329](https://github.com/morluto/rea/pull/4329):
remove fixed aggregate transport-buffer failures through incremental parsing.
Streaming transport does not bound every retained object or final reply.
These links establish changes; no numerical speedup is asserted here.
Typed reference facts, native string-query projection, and shared native byte
reads remained experimental at the 2026-10-17 checkpoint
`65f2f9a22bc892f61d2b00f9a4611124f57d4bff` on
`experiment/atomic-codemode-20261016`. Some experiments used a patched native
engine. Check current integration and bring-your-own engine capabilities before
proposing them; a working experiment is not a stock-provider guarantee.
## Applying the reasoning elsewhere
For another provider or format, inspect its actual representation and native
capabilities before borrowing the implementation. Existing numeric ordering
may support range reads; an on-disk native query may avoid materializing an
inventory; an immutable parse may support several views. Measure the relevant
alternative rather than requiring every provider to use Kuna's cached array.
For live observations, immutable artifact caching rules are insufficient:
validate the process/session identity and freshness required by the caller.
For compact replies, preserve access to source Evidence and report examined
coverage; projection must not silently turn an unresolved edge into no edge.
For batching, test mixed successes and failures and measure the entire task,
including discovery, extra calls, native work, and serialization.
+4 -5
View File
@@ -6,7 +6,7 @@ description: Prepare or rewrite REA release changelogs and GitHub release notes
# REA changelog update
Use this for release-note preparation and contributor attribution. Read
[docs/development/releasing.md](../../../docs/development/releasing.md) from the repository root for
[docs/releasing.md](../../../docs/releasing.md) from the repository root for
publication authority and the existing checkpoint checks. This skill prepares
notes; it does not authorize merging, tagging, or publishing.
@@ -68,10 +68,9 @@ closing-issue reporters. Grouping preserves all those credits in one line.
When Highlights, detailed entries or migrations reference the same work,
put its thanks in one of those entries, not all of them. Include issue links.
Direct-commit bullets carry the commit link and known human credit.
Exclude GitHub bots and known agent accounts. REA's observed Anthropic, Cursor,
Codex, and Copilot co-author trailers resolve to `@claude`, `@cursoragent`,
`@codex`, and `@Copilot` with GitHub type `User`; the credit policy excludes
them while retaining their metadata.
Exclude GitHub bots and known agent accounts. REA's observed Anthropic, Cursor
and Codex co-author trailers resolve to `@claude`, `@cursoragent` and `@codex`
with GitHub type `User`; the credit policy excludes them while retaining their metadata.
At the REA maintainer's request, omit `@morluto` self-thanks while preserving
authorship in the inventory. Other human contributors remain eligible.
Use explicit GitHub links for cross-repository references to avoid ambiguous
+1 -1
View File
@@ -9,7 +9,7 @@ Establish which checks still catch real regressions and which costs can be
removed without losing that protection. The outcome may be a verified fix,
an evidence-backed recommendation, or an explicit decision to stop.
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/development/testing.md),
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/testing.md),
and [contribution rules](../../../CONTRIBUTING.md). Use their current commands
and pinned toolchain, rather than commands remembered from an earlier audit.
Preserve the requested scope: investigation alone does not authorize edits;
+3 -26
View File
@@ -1,43 +1,20 @@
---
name: rea-tool-design
description: Audit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
description: Design or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
---
# REA tool design
Use the [tool-design guide](../../../docs/development/tool-design.md) for tool boundaries,
Use the [tool-design guide](../../../docs/tool-design.md) for tool boundaries,
contract semantics, provider ownership, and discoverability. Inspect the current
contract and nearest existing tool relevant to the analyst question; source,
documentation, and live catalogs may describe different versions.
For provider-backed work, exercise the analyst's task through REA and compare
relevant native provider and independent artifact observations where available.
Check applicable partial results, recovery and session boundaries; report what
was not exercised. For failure guidance, follow [design recovery around the
failure](../../../docs/development/tool-design.md#design-recovery-around-the-failure).
Use [workflow verification](../../../docs/development/testing.md#provider-backed-workflow-verification)
to select evidence for the proposed contract instead of seeking provider API
parity.
For an audit, follow [audit existing tools](../../../docs/development/tool-design.md#audit-existing-tools):
scope representative tasks, exercise the live workflow, and distinguish
reproduced bugs, supported limitations, design opportunities and unverified
claims. Check relevant rendered MCP prompts, guided workflows and packaged skill
advice against the live contracts and exercise their suggested handoffs. Report
working alternatives and evidence limits as well as findings.
For wording audits and agent trials, use
[focused tool-description trials](../../../docs/development/client-verification.md#focused-tool-description-trials).
Start with representative workflow boundaries and repeat failures or ambiguous
outcomes; keep actual arguments and supported final claims separate. Reuse the
repository's evaluation tooling and report actual runtime/model versions and
synthetic versus production coverage rather than copying a harness into a skill.
For a design request, return the proposed boundary, input/result shape, Evidence
semantics, nearest alternative, and verification approach. Implement when
requested, preserving the user's scope and existing authorization.
For implementation or verification, consult [testing](../../../docs/development/testing.md)
For implementation or verification, consult [testing](../../../docs/testing.md)
for the affected consumer and provider lanes, and
[contributing](../../../CONTRIBUTING.md) for generated-file ownership. Read
provider/workflow guides only as the task requires.
+70 -318
View File
@@ -1,14 +1,14 @@
---
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "62"
version: "42"
---
# REA
Use REA when a claim depends on a shipped binary or package, decompilation,
requested runtime evidence, or comparison with
passive application runtime evidence, or comparison with
behavior not established by available source. For ordinary analysis of a
complete source repository, use normal repository tools and do not run REA
readiness or provider commands.
@@ -17,9 +17,6 @@ readiness or provider commands.
Installing this skill supplies instructions; it does not register the REA MCP
server, install analysis engines, or add tools to an already-running agent.
The exception is a Claude Code or Codex plugin install: the plugin bundles
this skill and registers its own `rea` server, outside the state that
`rea setup` and `rea doctor` manage.
If REA tools are available and their registration is not known to be stale,
proceed directly to the target. Do not run diagnostics or setup before every
@@ -33,30 +30,39 @@ unavailable, read [connection and recovery](references/connection-and-recovery.m
Diagnose the observed blocker, continue useful supported work, and resume the
original task after repair. Client approval, shell PATH, registration and engine
availability have different remedies; avoid repeating setup for all of them.
If the host times out without a REA result, do not repeat the unchanged call or
assume work stopped. Read the recovery guide's deadline distinction before retrying.
If the MCP connection drops, retry that call once only when it is safe to
repeat or known not to have started. If it may already have changed the
target, or it fails again, tell the user and ask them to restart the MCP
client. If REA could not start its connection, retry once, then tell the user
and ask them to restart the MCP client. Do not retry a startup failure that
returns the same unchanged result. For plugin installs, follow
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin).
For setup-owned registrations, run `rea doctor` when REA could not start
or tools stay missing after that restart. For `cleanup_incomplete`, a retryable
capacity limit, or `configuration_invalid`, follow the recovery guide's ladder
before asking for a restart. Read that guide before a second repair attempt.
Choose the installation owner before repairing registration. Plugin installs use
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin);
only setup-owned registrations use the guide's
[registration repair plan](references/connection-and-recovery.md#repair-registration-when-needed).
A plugin-only doctor result of `missing` does not call for `rea setup`.
1. Diagnose without changing REA or client configuration. `npx` may download
the package and write to npm's cache. For Codex, run
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
`commandcode`, `vscode`, `grok_build`, `omp`, or `grok_bot`. `grok_bot` has no local
file registration: doctor reports the Grok Bot chat step, and setup does not
write `mcp.json`. If the client is unknown, use `doctor --json` and
inspect its registration results before choosing a setup scope.
2. Distinguish the reason. Missing, malformed, or stale registration needs a
scoped configuration repair. An aligned registration with no tools in the
active session needs a restart/reconnection; doctor cannot prove that the
current agent has connected. A missing provider affects only tasks requiring
that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
and Android inspection has separate bring-your-own JADX/Java prerequisites.
3. For a configuration repair, prepare the read-only plan:
`npx -y rea-agents@latest setup --client codex --dry-run --json`.
Use the current client's ID, show its exact proposed paths, backups, and
changes, and obtain approval before setup writes configuration or installs
Hopper. Setup normally installs the matching bundled skill too; include that
replacement in the reviewed plan. After approval, apply the same scope with
`npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
only if that separate installation was needed and explicitly approved.
4. Restart/reconnect the affected agent. Verify that REA tools actually appear
in the session, then resume the original investigation. If they remain
absent, inspect the client's MCP launch error rather than repeating setup.
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
JADX, Kuna, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
software to repair MCP registration. For an unsupported client, use manual stdio
registration with an existing installation’s absolute runtime and entry script, or continue through the CLI.
software to repair
MCP registration. For an unsupported client, use manual stdio registration
with a version-pinned `rea-agents` package or continue through the CLI.
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
@@ -69,13 +75,11 @@ graph, limitations, and unknowns with the same care as an MCP result; the CLI
returns the Evidence record directly. This fallback does not establish that
MCP is configured. Native CLI tasks still require their selected engine.
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
is within the authorized scope, use `npx -y rea-agents@latest` in its place from a
trusted directory outside the target project, with absolute target paths. Managed
setup binds the current installation directly; retain its runtime and package
cache entry, or restore that installation before repairing its registration.
For setup-owned registration, configuration repair starts with a read-only setup plan; show paths, backups and
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
Configuration repair starts with a read-only setup plan; show paths, backups and
skill replacement before applying authorized changes. Preserve existing approval
and target-execution boundaries.
and target-execution boundaries. REA setup never installs or upgrades unrelated
software, including Node.js, npm, Homebrew, Java, Ghidra, IDA, JADX, Kuna, Binwalk or Unblob.
## Route the target first
@@ -85,28 +89,9 @@ explicit path or endpoint and do not need it.
- ASAR or extracted JavaScript/Electron tree:
`analyze_javascript_application`.
- Archive/package member inventory (ZIP/APK/IPA/MSIX/AppX, DMG, or a PyInstaller
executable): `open_binary` with the supplied local path; use `inspect_artifact`
when its graph and findings help answer the question. A PyInstaller CArchive
is the application layer; the bootloader is only the native prefix. Start with
`detail: "summary"` for large inventories, then use retained inventory views
for selected member fields/pages. A complete small inventory can cost less
than extra calls and schemas. Runtime options are metadata, not files.
Compact delivery still inventories and hashes members. See the
[Python packaged applications guide](https://github.com/morluto/rea/blob/main/docs/python-packaged-applications.md).
- Bun `--compile` executable: `open_binary`, then `inspect_artifact`; the Bun
module graph is the application layer. Extract stored modules and analyze the
JavaScript. See [Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
- Deno `deno compile` executable: `open_binary`, then `inspect_artifact`; the
embedded virtual filesystem is the application layer. Extract stored files and
analyze compatible JavaScript. Environment-file values and CA certificate bytes
are retained in local Evidence with their counts and byte lengths. The npm
snapshot length is reported and not decoded, and the executable is not run. See
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
`open_bundle_resources`, then the selected resource tool. This does not
require `CFBundleExecutable` or a deep provider. Resource sessions have no
executable digest or binary snapshot; use `open_binary` for native code.
- Archive/package member inventory (ZIP/APK/IPA/MSIX/AppX or DMG):
`open_binary` with the supplied local path; use `inspect_artifact` when its
graph and findings help answer the question.
- Android APK code, classes, methods, or incoming references:
`inspect_android_package`, then focused Android tools when advertised.
Archive member inventory still uses the archive route above.
@@ -114,53 +99,23 @@ explicit path or endpoint and do not need it.
- Firmware image: `inspect_firmware_regions` when advertised. Use
`extract_firmware` when extraction is requested, with a caller-selected new
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md).
- ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact`
without a provider; see the ESP section of `references/native-and-artifacts.md`.
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
optional metadata recovery and its supported host/layout boundary.
- Native program behavior, live calls, instrumentation, or command-run effects:
choose the matching [native runtime workflow](references/runtime-observation.md#native-runtime-questions).
`open_binary` and `trace_native_values` establish static facts; they do not
observe execution. Runtime tools have distinct launch, attach, mutation and
cleanup effects, and do not require a preliminary static session unless their
own contract says so.
- User-owned browser page already open: `list_browser_targets`.
- Passive traffic capture of a SPA: the default scope is the page's own origin,
and data APIs usually live on another origin. Discover API origins from
absolute URLs in loaded bundles or script text via JavaScript analysis tools
(another passive capture without those origins still scopes to the page
origin, and `completeness.excluded` counts lack URLs). Pass both the page
origin and those API origins in `allowed_origins` — the list replaces the
default scope — and read `completeness.excluded` before reporting that an
API is absent. For long captures with body shapes, use
`export_evidence_bundle` instead of expecting an inline result.
- Browser action the agent must perform (click, type, navigate):
`capture_browser_scenario` with declared actions. See
[runtime observation](references/runtime-observation.md) for connect/launch
and the focused follow-up browser tools.
- Agent environment (AgentEnv bundle) of a web application: read
[environments](references/environments.md) before recording; it records with
`capture_browser_scenario` and writes the bundle with `export_environment_bundle`.
- Task for a desktop application's exported environment, from a
`capture-desktop-application` folder: read
[desktop application tasks](references/environments.md#tasks-for-a-desktop-application-environment).
- Environment of a Linux desktop app with a task, from a request such as
"build me an Inkscape environment where the agent changes a poster's
headline": read [desktop builds](references/desktop-build.md). You drive the
app on a build computer from `start_build_computer`, then export and witness.
- Fiddler Everywhere traffic or a native `.saz` capture: read the
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
connect Everywhere's separate MCP server for live traffic.
- Retained HAR/SAZ/native mitmproxy/aioquic qlog capture: `inspect_web_network_capture`. Select
original record ordinals when useful; inspect producer fields and byte/number
sidecars without fetching recorded URLs or claiming live attribution. HAR/SAZ/qlog
old-generation heap is `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` (default 192 MiB,
floor 8, maximum 16384), set where REA starts. Host memory and `NODE_OPTIONS` do not
raise it. See
[historical captures](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md) for the exact upstream
sidecars without fetching recorded URLs or claiming live attribution. See
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
or establish network loss. Prefer detail summary, then kind qlog trace/stream
@@ -172,10 +127,8 @@ explicit path or endpoint and do not need it.
and field names/types before choosing fields for `inspect_packet_capture`.
Query/page the relevant records; registrations do not establish observed traffic.
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
display filters and decode-as rules. Requires Linux, util-linux prlimit and
TShark 4.x on an absolute `PATH` entry or via the optional absolute
`REA_TSHARK_COMMAND` override; see
[capture guide](https://github.com/morluto/rea/blob/main/docs/guides/data/packet-captures.md).
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
Packet limits count input packets before filtering. Output defaults to 50 matching
frames; use `match_offset` / `match_limit` for later pages, or explicit null
limit for complete matching output. Preserve the source digest across pages.
@@ -183,7 +136,7 @@ explicit path or endpoint and do not need it.
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
and capinfos, with Linux prlimit or Windows Job Objects; see
[packet summaries](https://github.com/morluto/rea/blob/main/docs/guides/data/pcap-analysis.md).
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
Select an exact DNS question for address pages; page exhaustion does not prove
network completeness. This reads recordings and does not acquire traffic.
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
@@ -191,7 +144,7 @@ explicit path or endpoint and do not need it.
relevant details before choosing settings; this does not scan hardware.
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/guides/signals/signal-recordings.md).
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
Output defaults to 50 literal lines; select annotation classes and page with
`line_offset` / `line_limit`, or explicit null for all remaining output.
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
@@ -200,179 +153,30 @@ explicit path or endpoint and do not need it.
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
Select JavaScript glue and candidate paths explicitly and preserve URL/path
ambiguity. No implicit fetch or target execution; WAT is not original source.
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/guides/wasm/wasm-artifacts.md).
- Explicit local Android APK (or ZIP/AAB) signing: `inspect_apk_signature`.
Prefer its summary or selected signer/digest/lineage page; retain complete
Evidence and use `inspect_analysis_view` kind `apk-signature` for follow-ups.
Reports which schemes are present (v1 JAR, v2, v3, v3.1), each signer's
certificate SHA-256 fingerprint (v1 included, from its PKCS#7 block), v3 key-rotation lineage, source stamp, and
whether each signed digest still matches the bytes (self-consistency). This
is identity and integrity, not a trust or authenticity verdict.
See the [APK signature guide](https://github.com/morluto/rea/blob/main/docs/apk-signature.md).
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/wasm-artifacts.md).
- Explicit local EVM bytecode carrier: `inspect_evm_interface` with caller-selected
`raw` or `hex` encoding. The owned worker deadline is `timeout_ms` (default
30000 ms, range 1..2147483647, next call). It does not raise the fixed byte,
address-space, or CPU allowances. Preserve carrier/decoded digests and treat selectors,
`raw` or `hex` encoding. Preserve carrier/decoded digests and treat selectors,
argument strings and mutability as inferred candidates; this performs no chain
lookup or target execution. For the complete inferred interface without hex or
duplicate stdout, discover support and select `detail: "summary"`; complete
Evidence stays retained under its original identity. Use the returned
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
Candidate coverage does not establish interface completeness. See the
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/guides/evm/evm-bytecode.md).
- Embedded Go compiler, module, build-setting metadata, or static `(ptr, len)`
string literals: `inspect_go_binary` with the explicit local file (pass
`include_string_literals: true` and optional `max_string_literals` for bounded
inferred static string-header recovery, and page retained rows with
`inspect_analysis_view` kind `go` collection `string_literals`). This built-in
target-free inspection requires no Go installation or disassembler. Preserve
raw module bytes and source ranges; absent metadata does not prove a binary is
not Go, static string recovery excludes compiler-inlined stack or immediate
strings, and unparsed text records remain explicit. See the
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/go-binaries.md).
- Embedded CUDA/HIP code objects: `inspect_gpu_code_objects`; inventory targets
first, then decode selected payloads and query retained GPU Evidence.
- Bundled GGUF or safetensors model declarations: `inspect_model_artifact`
with the explicit local file. Default header hashing leaves full-file identity
unknown; `hash: "full"` additionally reads all weights for SHA-256. MCP returns
retained summaries; optional `view` selects one page in the initial call. Use
`inspect_analysis_view` kind `model-artifact` for
batched metadata keys, tensor names or type totals, selected fields and pages.
Preserve header-scoped indices and array preview omissions; do not infer model
behavior from declared architecture or tensor types. See the
[model file guide](https://github.com/morluto/rea/blob/main/docs/model-artifacts.md).
- Unity IL2CPP names, member owners and string literals from `global-metadata.dat`:
`inspect_il2cpp_metadata` with the explicit local file; MCP defaults to compact
counts. Query sections, images, types, methods, fields, properties, events,
parameters or string_literals through `inspect_analysis_view`
with the returned `parent_evidence_id`, using exact filters, batch indices and
selected fields. Parameters accept `method_indices`, avoiding a scan of all
parameter pages. Missing collections in historical Evidence remain unknown.
Versions 24, 27, 29, 31 and 39; no Unity, IL2CPP
binary or disassembler is run. Unrecognized magic (possibly encrypted) and
other versions are unsupported. Optional matching `binary_path` enables static native addresses and type references; query `method_addresses` with indices or addresses/states, preserving every shared-body method and both artifact identities. Without a linked binary native counts remain unknown. Byte-pattern fallback is inferred; no runtime behavior or same-build pairing is proved. See the
[IL2CPP metadata guide](https://github.com/morluto/rea/blob/main/docs/il2cpp-metadata.md).
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
with the explicit local file. This built-in target-free inspection requires no
Go installation or disassembler. Preserve raw module bytes and source ranges;
absent metadata does not prove a binary is not Go, and unparsed text records
remain explicit. See the
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
- Embedded cargo-auditable Rust dependency metadata: `inspect_rust_binary`
with the explicit local ELF, PE, or thin Mach-O file. This built-in
target-free inspection requires no Rust installation or disassembler. Preserve
package order, dependency indices, and the section file range. A missing
`.dep-v0` section is null metadata, not proof the binary has no dependencies.
A rustc commit hash is an inference from a read-only `/rustc/<40-hex>/` path.
For selected crates, use `view: {collection: "packages", names: [...],
fields: ["name", "version", "source"]}` on the first call; explicitly include
`dependencies` for direct index references. Use `detail: "summary"` for counts
and ranges, or retained `inspect_analysis_view` Rust package/edge pages for
subsequent filters. Keep the complete parent Evidence accessible; CLI compact
delivery requires `--evidence_output`.
See the
[Rust binary guide](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
- PE Authenticode certificate table: `inspect_pe_certificates` with the explicit
local PE32/PE32+ file. This built-in target-free inspection requires no
Windows SDK. Preserve WIN_CERTIFICATE ranges, types, payload SHA-256, and
PKCS#7 signer/X.509 identity fields, including `subject_key_identifier_hex`
when the signer uses that form. A missing security directory is observed
absence of that table, not proof the image is unsigned in every catalog sense.
Signatures are not verified. See the
[PE certificate guide](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
- PE import and delay-load tables: `inspect_pe_imports` with the explicit
local PE32/PE32+ file. This built-in target-free inspection requires no
Windows SDK. Preserve DLL names, named and ordinal functions, ILT/IAT RVAs,
delay-load descriptors, and the pefile import hash of the standard import
table. A missing import directory is observed absence of that table, not
proof the image never imported APIs through another table. Bound snapshots
and loaded IAT contents remain unobserved. See the
[PE import guide](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
- Windows Shell Link shortcuts: `inspect_windows_shortcut` with the explicit
local `.lnk` file. This built-in target-free inspection requires no Windows
SDK or COM. Preserve header flags and FILETIMEs, IDList items, LinkInfo
volume and network names, StringData arguments and icon location, ExtraData
signatures, and Tracker droid facts labelled derived. Unknown IDList items
and ExtraData signatures stay raw with SHA-256. The target path is not
resolved or executed. See the
[Windows shortcut guide](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
- PE TLS callbacks and load configuration: `inspect_pe_load_config` with the
explicit local PE32/PE32+ file. This built-in target-free inspection requires
no Windows SDK. Preserve callback VAs as strings with their RVA, file offset
and section, and the size-covered security cookie, SafeSEH, CFG,
EH-continuation and XFG fields. Guard-flag mitigation claims are inferred
linker declarations, not proof of enforcement. A missing directory is observed
absence of that declaration. See the
[PE load configuration guide](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
- ELF, PE, and Mach-O hardening: `inspect_binary_hardening` with the explicit
local file. This built-in target-free inspection requires no pwntools or other
host tool. One image per Mach-O slice. Each property is `enabled`, `disabled`,
`partial`, `unknown`, or `not_applicable`, with the header field or import
behind it. A missing stack-protector or fortify import is `unknown`, not
proof the protection was omitted. Authenticode and Mach-O code signatures
report presence, not validity. Hardened runtime stays unknown and names
`inspect_signature`. The image is not executed. See the
[binary hardening guide](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
- ELF note tables: `inspect_elf_notes` with the explicit local ELF32/ELF64
file. This built-in target-free inspection requires no disassembler. Preserve
PT_NOTE and SHT_NOTE ranges, owners, types, descriptor SHA-256, and GNU
build-id or ABI-tag fields. A missing table is observed absence of those
declarations, not proof the image never carried notes. See the
[ELF notes guide](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
- Code that runs before `main` or at exit in ELF or Mach-O:
`inspect_native_initializers` with the explicit local file (and
`architecture` for a universal Mach-O). This built-in inspection requires no
disassembler. Preserve slot and target addresses, the `resolution` of each
entry, and its limitations. An `unresolved` slot is not a zero pointer, and a
missing table is not proof that no constructor exists. See the
[initializer guide](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
- ELF unwind function ranges: `inspect_elf_unwind_ranges` with the explicit
local ELF32/ELF64 file. This built-in target-free inspection requires no
disassembler. Preserve FDE start, end, size, containing section, exact
symbol name, CIE index, LSDA, and personality. Page with `offset` and
`limit`. A missing `.eh_frame` is observed absence of that declaration,
not proof the image has no functions. `.debug_frame` is reported present
or absent and is not decoded. CFA instructions are counted and not
evaluated. See the
[ELF unwind ranges guide](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
- Apple `.ips` crash reports: `inspect_apple_ips` with the explicit local
file. This built-in target-free inspection requires no Xcode, atos, or
pwntools. Preserve first-line metadata, `bug_type` as a string, process and
exception facts, lossless integer registers, usedImages UUIDs, and
documented classification matches. Only `bug_type` `"309"` yields a crash
body; other types, including `"288"` stackshots, return no crash result.
Opened Mach-O and dSYM UUID joins remain unobserved. See the
[Apple IPS guide](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
- Linux kernel module metadata: `inspect_kernel_module` with the explicit local
ELF32/ELF64 relocatable file. This built-in target-free inspection requires
no Linux host or `modinfo`. Preserve `.modinfo` key order and duplicate keys,
`__versions` CRC hex, section file ranges, and the appended signature
trailer's PKCS#7 SHA-256. A missing section is observed absence when section
names are known, not proof the module never carried that metadata.
`struct module` is not decoded. The signature is not validated. gzip, xz, and
zstd containers are unsupported; decompress into a separate file with an
external codec utility, then inspect the decompressed `ET_REL` image. See the
[kernel module guide](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
- Linux kernel version and build config (`CONFIG_*` options such as
`CONFIG_SECURITY_SELINUX`): `inspect_kernel_build_config` with the explicit
local `vmlinux`, raw `Image`, x86 `bzImage`, or `configs.ko`. This built-in
target-free inspection requires no Linux host or `extract-ikconfig`. Preserve
the banner version, config SHA-256, option count, and value kinds; use `name`
for one option. An `IKCFG_ST` string is a config only with a valid gzip member
and `IKCFG_ED`. An absent config may mean the kernel was built without
`CONFIG_IKCONFIG`. Non-gzip payloads are unsupported; decompress externally,
then inspect the result. See the
[kernel build config guide](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
- CPython 3.7–3.13 `.pyc` header, nested marshal code objects and bytecode
identity: `inspect_python_pyc` with the explicit local file. This built-in
target-free inspection requires no Python installation. Preserve magic,
PEP 552 hash or mtime/size, nested names, and bytecode SHA-256. Bytecode is
not decompiled, and a decoded header does not prove the matching `.py` is
present. See the
[CPython pyc guide](https://github.com/morluto/rea/blob/main/docs/python-pyc.md).
- Erlang BEAM IFF container: `inspect_erlang_beam` with the explicit local
`.beam` file. This built-in target-free inspection requires no Erlang
installation. Preserve chunk names, atom order, export/import/lambda tables,
and the Code header. For selected facts, prefer `detail: "summary"` plus a
`beam` view (collection, names/indices, fields and paging), retaining the full
Evidence. CLI summary requires `--evidence_output`. Missing table counts stay
null; decoded empty tables count zero. Bytecode is not executed. See the
[Erlang BEAM guide](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
- Explicit local SQLite database snapshot: `inspect_sqlite_database`. Inspect
with Node.js 24.x >=24.15 or 26+ providing native SQLite limits. Read schema
@@ -381,22 +185,20 @@ explicit path or endpoint and do not need it.
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
report truncation and have unspecified order. Use a quiescent snapshot rather
than assuming a live database/WAL pair was copied atomically. See the
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
- Explicit Linux ELF file for offline layout, symbols, relocations or static
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
caller-supplied pwntools without opening a disassembler database. Preserve its
raw locations and inference/coverage limits; see the
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
- Supplied Linux x86-64 ELF core: `inspect_recorded_crash`. Read every recorded
thread, raw note and source-bound register without opening a live target.
Historical PIDs do not select live processes. Request `include_debugger_context`
only when core-only mapping candidates help; see the
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md).
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md).
- PE resource type/name/language trees and static icon group references:
`inspect_pe_resources` with an explicit path. This portable operation requires
no active target or engine; opaque payloads retain file ranges and SHA-256.
- PE export directories and ELF dynamic-symbol exports: `inspect_native_exports`
with an explicit path; portable, no active target or engine.
- Native executable, library, or analysis database: `open_binary`, then
use focused analysis tools directly; call `binary_overview` when metadata or
inventory context is useful and available from the selected provider.
@@ -407,12 +209,7 @@ The legacy attached profile binds the already-open GUI input by SHA-256 and
leaves its database open. The modern headless profile creates and closes an
owned private database without saving. Installation and registration reuse
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md).
For migration to the separate native IDA MCP plugin, follow the guide's
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md#migrate-a-legacy-attached-registration).
Replace the upstream registration explicitly; modern mrexodia tools are not the
native profile. Native attachment preserves the database but does not provide
direct callers; check session availability and the guide's verification limits.
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
Proceed directly to `analyze_function`, pseudocode, or function/string searches
after opening; IDA does not supply `binary_overview`. Consult session availability
when composing broader workflows. Modern direct callers and unsupported dossier
@@ -443,28 +240,12 @@ native target whose analysis tool operates on the active target. Do not call an
unadvertised tool; inspect `binary_session` with `{}` and its
`result.tool_availability` for availability reasons and remediation. Expand each
group's `names` into entries, inheriting the inventory-level client requirements
unless the group overrides them. The grouped name union is the complete tool
inventory; use `tools/list` for canonical tool schemas. The default summary keeps
each provider candidate's availability and target-support status, code and
reason; `omitted_fields` lists what it leaves out. Request `{"detail": "complete"}`
only when you need an unavailable candidate's remediation, provider
diagnostics, candidate capability sets, or tool surface, effect and annotation
flags; in those closed flag vocabularies, listed
flags are true and omitted known flags are false. Provider candidate facts
describe possible providers, while top-level session capabilities describe the
current route and `analysis_provider_binding` names the selected provider. Do
not treat those scopes as interchangeable.
For editable Ghidra analysis that must survive a later agent process, select
the [owned Program checkpoint workflow](references/native-and-artifacts.md#continue-editable-ghidra-analysis-across-processes)
before making metadata edits. It is separate from Evidence/Unknown persistence.
For a Ghidra startup timeout on a large native target, follow
[deadline recovery](references/native-and-artifacts.md#ghidra-startup-deadline-recovery)
before abandoning deep analysis or repeating the same cold import. The default
330 seconds can be changed for new startup attempts through
`update_analysis_settings` using `ghidra.startup_timeout_ms`. Changing the
server's launch environment instead requires a new REA process.
unless the group overrides them. In its closed effect and annotation vocabularies,
listed flags are true and omitted known flags are false. The grouped name union
is the complete tool inventory; use `tools/list` for canonical tool schemas.
Provider candidate facts describe possible providers, while top-level session
capabilities describe the current route and `analysis_provider_binding` names the
selected provider. Do not treat those scopes as interchangeable.
## Work summary-first
@@ -472,22 +253,7 @@ Start with the default result and use its inline Evidence and graph context.
For a JavaScript application or ELF layout whose size is unknown or large,
call `analyze_javascript_application` or `inspect_binary_layout` with
`"detail": "summary"`, then inspect the retained `parent_evidence_id` with
`inspect_analysis_view`. Over MCP, `analyze_function` is summary-first without
a flag: its default reply is the summary, and the complete dossier stays
retained under `parent_evidence_id` for facet paging. Pass that ID as a
`retained-evidence` reference to `compare_functions`, or inside
`match_native_functions` `left_observations`/`right_observations` or
`build_call_path.functions`; request `"detail": "complete"` only when the
dossier must travel inline.
Use the same exact-reference form in other advertised logical Evidence inputs,
including optional sources, arrays and nested sources across domains. Each
source can independently be inline or retained; preserve its source identity,
coverage, partial facts and unknowns. A summary or inspection page has its own
Evidence ID: use `normalized_result.parent_evidence_id` when the consumer needs
the complete source. For portable CLI follow-ups, commands advertising
`--evidence-bundle` resolve references from that selected exported bundle.
See [Evidence composition](references/evidence-workflows.md) for scope and
recovery; file selectors are supported only where explicitly advertised.
`inspect_analysis_view`.
Page module identities and select the returned `node_id` before requesting an
item; a single file path can represent several graph roles. Do not repeat an
identical analysis or rejected call without a changed input or resolved blocker.
@@ -506,10 +272,7 @@ output also streams without a single MCP frame.
When the retained parent ID is unknown, use `get_evidence_bundle` with
`detail: "summary"` and exact filters for operation, target SHA-256,
analysis-profile digest or canonical procedure address. Select its returned
Evidence ID for the next view instead of requesting the complete bundle. When
that record has no supported view (for example an annotation), read it with
`detail: "records"` and an `evidence_id` filter; `missing_evidence_links`
names linked sources this session does not retain.
Evidence ID for the next view instead of requesting the complete bundle.
`retention: "complete-record"` means the whole record is stored; native
value-flow truncation and unknown metadata remain separate. For an exported
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
@@ -578,26 +341,15 @@ as complete while required questions remain open.
[references/javascript-applications.md](references/javascript-applications.md)
- Android APK declarations, classes, methods, and static references:
[references/android-applications.md](references/android-applications.md)
- Native instrumentation, debugger and process capture; browser/Electron
observation; static/runtime comparison:
- Browser/Electron observation (passive and scenario) and static/runtime
reconciliation:
[references/runtime-observation.md](references/runtime-observation.md)
- Evidence paging, comparisons, residual unknowns, and verification:
[references/evidence-workflows.md](references/evidence-workflows.md)
- Agent environments (AgentEnv bundles) of a web application, recorded from
a live instance or a HAR:
[references/environments.md](references/environments.md)
- Tasks for a desktop application's exported environment:
[references/environments.md](references/environments.md#tasks-for-a-desktop-application-environment)
- A desktop app environment from a request, built and witnessed on build
computers: [references/desktop-build.md](references/desktop-build.md)
## Finish the task
Explain findings in plain language and tie them to returned evidence. When the
user asks to build something, use normal coding tools and separate observed
behavior from design choices. Close an opened native session with
`close_binary` when its engine is no longer needed. Evidence references and Unknown
history remain in memory on the same MCP connection. End the investigation with
`reset_investigation`; supply `save_path` to checkpoint the complete bundle before
clearing. Import the saved bundle to restore original references and Unknown
history in a new process. Retention is bounded and is not automatic persistence.
`close_binary` when the investigation is complete.
@@ -13,47 +13,18 @@ own APK path and explicit selectors; do not open a native session first.
Choose an `overload_index` from the class inventory when methods are ambiguous
instead of guessing.
For JNI implementations, discover `inspect_android_jni_bindings`. Select raw
DEX class/method identities and optional descriptors, ABI or logical library
paths. Prefer `detail: "summary"`; complete Evidence remains retained and
`android-jni` views page declarations, libraries, candidates or binding rows
with exact fields. Global native-pair ambiguity survives class selection.
Preserve slot/relocation provenance and source coverage: matched means observed
candidate presence; partial scans do not establish uniqueness or absence.
This reads DEX/ELF bytes without loading the APK into the decompiler or executing
target code. Runtime registration and loading remain unknown. See the
[JNI guide](https://github.com/morluto/rea/blob/main/docs/guides/android/android-analysis.md#link-native-declarations-to-jni-candidates).
For an R8/ProGuard release build with its `mapping.txt`, pass the absolute
`mapping_path` to `inspect_android_class` or `inspect_android_method`. Prefer
`detail: "summary"` to receive normalized symbols or the selected body without
duplicate raw class metadata. Complete Evidence stays retained under the linked
parent ID and can be exported; CLI summary delivery requires `--evidence_output`.
Omitted or complete detail preserves the full inline response. Keep
selecting by observed names; the separate `mapping` block reports each symbol
as `mapped`, `ambiguous` or `unmapped` after checking DEX types and prototypes.
REA does not verify that the mapping belongs to the APK build.
The provider requires a separately supplied headless JADX MCP JAR 0.7.1 and
Java 17 or newer. `REA_JADX_MCP_JAR` selects the absolute JAR path; `JAVA_HOME`
can select Java. Use `rea doctor --provider jadx --json` to check the JADX
configuration and obtain remediation for missing prerequisites. Java and the
headless JADX MCP JAR remain caller-supplied; REA setup does not install or
configure them. Configuration readiness does not establish successful analysis
of a particular APK.
can select Java. REA setup does not install or configure that JAR, and doctor
provider scopes cover Hopper/Ghidra rather than JADX. Use existing tools and
report missing prerequisites for the Android task specifically.
Linux and macOS use the POSIX owned-process boundary. Windows x64 requires the
matching bundled native controls and uses an owned Job Object with bidirectional
stdio. A full JDK, including `jdk.compiler`, is required on every host; Windows
`JAVA_HOME` selects `bin\java.exe`. Consult the Android guide for actual host
verification coverage. Calls use one worker and provider deadlines; optional
`REA_JADX_HEAP_MIB` selects the JVM heap and `REA_JADX_OPERATION_TIMEOUT_MS`
the per-operation deadline (integer milliseconds from 1 to 2147483647, default
120000). To change that deadline on a running MCP server, set
`jadx.operation_timeout_ms` through `update_analysis_settings`, then retry after
work and cleanup settle. Newly accepted operations use the override; accepted
work keeps its budget. Changing the launch environment requires a new process.
The MCP client's tool timeout must also allow the selected deadline. The APK is never launched,
`REA_JADX_HEAP_MIB` selects the JVM heap. The APK is never launched,
and no emulator is required.
Manifest entries and decompiled text are static observations, not execution or
@@ -7,23 +7,18 @@ operation and the next useful step; after repair, continue the investigation.
Commands below use an installed `rea`. If none is available, use the package
invocation in `SKILL.md` when acquiring REA is authorized. Follow the connected
server's schemas and the installed CLI's help rather than assuming repository
main and an older release expose the same arguments. Tool-name prefixes belong
to the client: find the canonical REA operation in its actual tool listing and
server association before concluding that it is absent. A shared prefix alone
does not establish which server an approval wildcard covers. See
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/getting-started/client-configuration.md#client-tool-names-and-discovery).
main and an older release expose the same arguments.
## Choose the remedy from the observed failure
| Observation | Meaning and next step |
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| No REA tools in the active session | If REA was installed as a client plugin, follow the plugin section below. Otherwise inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
| MCP connection dropped or REA could not start | If the connection drops, retry that call once only when it is safe to repeat or known not to have started. If it may already have changed the target, or the same failure returns, tell the user REA's connection failed and ask them to restart the MCP client. If REA could not start its connection, retry once, then tell the user and ask them to restart the MCP client. Do not retry a startup failure that returns the same unchanged result. Run `rea doctor` when REA could not start or tools stay missing after restart. Doctor cannot restore a live connection by itself. |
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. Request `{"detail": "complete"}` only for provider diagnostics. |
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
| Observation | Meaning and next step |
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. |
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
If the client cannot launch its shell/sandbox helper or read the installed skill,
inspect its launch diagnostic. For example, Codex can refuse helper aliases when
@@ -39,109 +34,6 @@ answers part of the question, and name the unavailable evidence separately.
Do not substitute another target. Label observations from ordinary shell tools
as such; only actual REA output supplies REA Evidence.
## Distinguish client and server deadlines
A host message such as `timed out awaiting tools/call` with no REA result means
the client's deadline expired before a result arrived. It does not establish
that REA finished, canceled successfully, or released all resources. Do not
repeat the unchanged analysis. Inspect available session/provider state and
retained Evidence only when useful; an idle `binary_session` is not proof that
every target-free operation stopped. Never invent a missing Evidence reference.
Allow the host tool deadline to cover execution, cancellation/cleanup, and
response delivery. In Codex, the setting is
`mcp_servers.<server-id>.tool_timeout_sec` in **seconds**, not
`startup_timeout_sec`. Select a value for the actual operation budget and
headroom, not a universal timeout. See the
[official MCP configuration reference](https://learn.chatgpt.com/docs/extend/mcp?surface=cli).
Apply configuration changes only within the authorized scope, preserving
unrelated entries and backups; reconnect/restart as required by the host.
When configuration is not editable, report the setting needed or use an
authorized CLI fallback. Do not silently broaden permissions or switch targets.
A returned REA `provider_timeout` instead means the **server/provider** reached
its deadline. Follow its `remediation.action`, reuse accessible partial Evidence,
and resolve reported incomplete cleanup before retrying. Change a request
field or a `REA_*` setting only when the returned remediation explicitly
recommends that change. Restart REA when that text says the value is read at
startup. A mention that a control does not raise the deadline is not a
recommendation to change it. Do not invent a setting the remediation does not
name. Raising only the client deadline does not change
this server budget. For static JavaScript,
`analysis_timeout_ms` (CLI `rea analyze-javascript-application <path>
--analysis-timeout-ms <milliseconds>`) controls each source's
startup, analysis and transfer deadline in **milliseconds**. Increase that
budget or reduce semantic work / select exact inventoried `source_paths` when
the task permits. Omit `source_paths` for all sources; `[]` is invalid.
`detail: "summary"` reduces delivery, not analysis work. A per-source budget
does not bound the elapsed time for an entire multi-source application.
Partial-Evidence inspection can avoid recomputation; an unavailable reader is
not itself a reason to block a corrected static-analysis retry.
The shorthand `rea analyze <path>` does not expose this timeout flag; use the
explicit JavaScript command when changing that budget.
For invalid arguments, use the connected tool's current input schema, or the
installed CLI's command help. Refresh the catalog if stale; do not guess fields
from an older skill or return the entire catalog to repair one field. Correct
all reported fields before retrying.
## Recovery ladder
Follow the returned remediation. Change a request field or a `REA_*` setting
only when that text explicitly recommends the change. Restart REA when it says
the value is read at startup. A mention that a control does not raise a limit
is not a recommendation to change it. Do not invent a setting. Host memory and
`NODE_OPTIONS` do not raise a REA admission unless that remediation says so.
- MCP connection dropped: retry that call once only when it is safe to repeat
or known not to have started. If it may already have changed the target, or
it fails again, tell the user REA's connection failed and ask them to restart
the MCP client. Do not run `rea doctor` for the drop itself.
- REA could not start its connection: retry once, then tell the user and ask
them to restart the MCP client. Run `rea doctor` when REA could not start or
tools stay missing after that restart. Do not retry a startup failure that
returns the same unchanged result.
- `cleanup_incomplete`, or retained cleanup owners in the error details: retry
that exact cleanup owner. Do not start new work that needs those resources
until that owner confirms release. Repeating the original call is not that
retry.
- Capacity, when the result is `resource_constraint` and `retryable` is true:
wait for the active admission to finish, then retry within the reported
limits. Do not restart REA to free the slot.
- `configuration_invalid`: correct the named setting in the environment that
starts REA (the shell or the MCP registration), then restart REA. Run
`rea doctor` to see the `runtime-configuration:` check. Re-run `rea setup`
for that client when the registration must carry the new setting, then
reconnect.
- `invalid_request`: correct the reported fields from the current schema. Do
not repeat the unchanged call.
## REA installed as a client plugin
A Claude Code or Codex plugin owns its `rea` server and this skill. Doctor's
`--client` and `--skill` checks and `rea setup` read and write only
setup-owned state, so for a plugin-only install they report `missing`, and
applying setup would add a second `rea` server. If this skill was loaded from
a plugin, or the client lists a `rea` plugin, do not prepare a setup plan for
missing tools. Instead:
- Check that the plugin is installed and enabled with the client's own
listing: `claude plugin list` and `/mcp` in Claude Code, `codex plugin list`
in Codex.
- Restart or reconnect the client, then read its launch error for the `rea`
server if tools are still absent. The plugin starts its bundled runtime; a missing build or version mismatch
requires a reinstall, or a fresh build when developing from source.
- Check analysis engines through the connected server's
`binary_session.result.tool_availability`. `rea doctor --provider <id> --json`
from a shell reports that shell's environment, which can differ from the
environment the client gave the plugin server. If you still run it through
the package runner, use the version pinned in the plugin manifest's server
entry (`rea-agents@<version>`), not `@latest`, so it matches the server the
plugin starts.
Use the setup repair below only when the user wants a setup-owned registration
instead of the plugin, and remove or disable the plugin first.
## Repair registration when needed
For Codex:
@@ -154,12 +46,9 @@ rea setup --client codex --dry-run --json
Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
`cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`,
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `trae`, `grok_build`, `omp`, `pi`,
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, `freebuff`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
For unsupported clients, use manual stdio registration with the selected
installation’s absolute runtime and entry script, or continue through the CLI.
Acquire packages from a trusted directory outside the target project. Keep a
package-runner installation and its runtime available while registrations use it.
For unsupported clients, use version-pinned manual stdio registration or CLI.
Show the plan's exact paths, backups, changes and bundled-skill replacement.
Obtain approval for writes when that concrete scope is not already authorized,
@@ -204,17 +93,6 @@ writing. Managed policy can constrain client settings. Consult the
for the installed client's supported settings. This is an optional recovery
choice, not a prerequisite for every REA task.
## Repair an editable REA limit
If `details.limit` names an editable setting and its bounds, call
`update_analysis_settings` directly with a `changes` entry containing `setting`,
`action: "set"` and `value`. `inspect_analysis_settings` is optional when those
facts are already present. Retry the same input after work and cleanup settle.
Runtime defaults affect new work only; explicit per-call arguments take
precedence and must be corrected directly when selected. Reset restores launch
values. Updates survive target close/open but disappear on REA restart. They do
not edit host deadlines, response buffers, registrations or fixed limits.
## Continue through an installed CLI
Check the command environment only if the launcher fails:
@@ -1,860 +0,0 @@
# Desktop app environments from a request
Use this when someone asks for an environment of a Linux desktop app with a
task, such as "Build me an Inkscape environment where the agent changes a
poster's headline and saves it". You are the building agent: you drive the
app on a build computer through REA's MCP tools, write the task, export the
bundle and witness it in a fresh computer. REA runs no model. The tools are
MCP-only and need a Linux host with Docker, agentenv and REA's packed
runtime. When a start reports one missing, tell the person to run
`rea doctor --provider desktop_build` and apply its fixes, or
`rea setup --client <client> --desktop-build` and restart the client, and
stop.
For the whole build:
- Drive the app only with `act_on_build_computer`. Never script it, run
commands in the computer or write its files another way. The one command
you write for the computer is a task's reader, which REA runs to check a
file; it never makes the seed or the edit.
- Ask the upfront questions once, then work on your own. Pause only when
blocked.
- The seed's content is synthetic. Never put real personal data in it.
- Close every computer you start, on success and on failure.
- Note the time you start; the report gives the build time.
## Ask once, upfront
In one message, ask only what the request leaves open, and keep the list for
the report. Skip each question the request answers; when it answers them all,
start without asking:
1. The app and the task, if either is unclear: what changes, in which
document.
2. Whether they have a starting file of their own; otherwise you make one.
3. Whether they want the tasks measured after the witness: with which
model, within which budget, and whether to harden a task until it lands
in the band.
4. If so, whether they want to review the tasks before those solves.
When you cannot ask, as in a headless `claude -p` run, do not stop: take
these defaults for whatever the request leaves open, say which you took in
your first message and in the report, and go on. Install from the Ubuntu
archive, make a synthetic seed, skip the review, and measure each task
within $4 in all, with the model the request names, without hardening.
## When you are blocked
Stop, say what you need and wait only for:
- a licence the app or its installer asks someone to accept: never accept it
on your own; name it and where to read it, and go on only once the person
says they accept it. Then record their words with `record_build_consent`,
kind `licence`, and accept it on their behalf;
- credentials: ask the person to write each secret, and nothing else, in a
file named after a handle you choose, such as `keepassxc-master`, in
`~/.config/rea/build-secrets/` on this host (under `$XDG_CONFIG_HOME` when
it is set), readable only by them (`chmod 600`), and wait until they say it
is there; then type it with `type_secret` (see Drive the app). Never ask
for a password or key in chat, and never read, list or print that folder;
- a signed-in state in the seed: by default the export withholds it (see
Sign-in and licence state). Seed it only when the person says the bundle
may hold it: record their words with `record_build_consent`, kind
`seed_sign_in`, and export with `seed_sign_in: true`;
- an install this flow cannot do, below.
Decide everything else yourself and note it in the report.
## Sign-in and licence state
What an app writes when someone signs in or activates a licence is withheld
from the seed: the bundle's receiver supplies their own. Mark
`pre-sign-in` with `mark_build_checkpoint` just before the sign-in or
activation and `post-sign-in` just after, both before `seed`: every file
under the owned paths that changed between them is withheld, and the draft
leaves it out. A file the app writes at first launch, such as a trial's
licence file, changes before you can mark anything: name it in
`export_build_environment`'s `withhold`. The export lists each withheld file
in `withheld_files` with the environment variable a deploy reads it from and
the handle of the secret REA keeps its bytes in; a check may not name one.
The export refuses a seed file holding a withheld file's exact bytes, such as
a `.bak` copy, but a backup that differs from it is not caught: trash backups
before `seed`.
Explicitly withhold any other files the person identifies as sensitive;
REA does not infer secrecy from a value's shape or a file name.
## Choose the install
The computer is Ubuntu 24.04 (noble) on this host's architecture only. Use
the first source that has the app:
| Source | `install` |
| ---------------------------- | ------------------------------------------------------------------------------------------- |
| Ubuntu archive | `{"kind": "archive", "package": "inkscape"}` |
| PPA or vendor apt repository | `{"kind": "apt_repository", "package", "uri", "suite", "components", "key_file"}` |
| `.deb` by URL | `{"kind": "package_file", "url", "sha256", "architecture", "dependencies"}` |
| AppImage by URL | `{"kind": "appimage", "url", "sha256", "architecture", "extra_packages"}` |
| Tarball by URL | `{"kind": "tarball", "url", "sha256", "architecture", "program", "name", "extra_packages"}` |
- A PPA's `uri` is `https://ppa.launchpadcontent.net/OWNER/NAME/ubuntu`,
`suite` `noble`, `components` `["main"]`. `key_file` is the repository's
signing key saved on this host; for a PPA, fetch it from
keyserver.ubuntu.com by the fingerprint its Launchpad page shows.
- When the app exits at launch for want of a runtime piece its package does
not install, start a new build computer with that package in
`extra_packages`, up to 16 archive names; the export installs them too.
MyPaint needs
`{"kind": "archive", "package": "mypaint", "extra_packages": ["librsvg2-common"]}`
for its SVG icons, and Freeplane `["openjdk-17-jre"]`, since its launcher
refuses noble's Java 21. Name one up front, too, when a package the app
only recommends holds its content, such as QElectroTech's element library
`qelectrotech-data`, without which no component can be placed.
- Take the `.deb` path only when the app is not in the archive at all, not
to add a package. Ask the requester for its URL and SHA-256; do not fetch
a vendor's download page yourself. List its `Depends` as `dependencies`,
with any runtime piece it lacks: it installs with the network off.
- Take an AppImage or a tarball only when no package has the app. Pin it by
the release file's URL for this host's architecture and the SHA-256 its
project publishes, such as a GitHub release asset's digest or the
release's `SHA256SUMS`; ask the requester when the project publishes none.
The computer downloads it when the bundle is put, refuses other bytes, and
unpacks it under `/opt/rea-apps/<desktop_entry>` with the network off; the
export downloads the same file.
- An AppImage installs its own desktop entry, so `desktop_entry` is the
`.desktop` file at the top of the image, such as FreeCAD's
`org.freecad.FreeCAD`. A wrong name fails the start with the entries the
image has: start again with one of them. The image brings its libraries,
so `extra_packages` is rarely needed; FreeCAD 1.1.4's needs none.
- A tarball has no entry of its own. `program` is the program's path inside
the archive, such as `firefox/firefox`; REA writes an entry named
`desktop_entry` that runs it with the files it opens, and `name` is what
the Applications menu shows. A tarball uses the computer's libraries: name
what its program lacks in `extra_packages`. gzip, bzip2 and xz archives
are read, zstd ones are not.
- Flatpak and snaps are not supported: say so and stop. Flatpak's sandbox
needs user namespaces the computer's container does not grant, so a
Flathub app installs but never opens; ask for the project's AppImage,
tarball or `.deb` when it ships one. An archive package that only installs
a snap, such as noble's `firefox`, counts as a snap. A package with no
build for this host's architecture (some PPAs are amd64 only) cannot be
installed either.
- `desktop_entry` is the package's `.desktop` file name under
`/usr/share/applications` without `.desktop`, such as
`org.inkscape.Inkscape`. Add `window_class` when you know it; Inkscape's
is `Inkscape`. AbiWord's entry is `abiword` and its class `Abiword`.
- To find the entry, read the package's file list: its page on
packages.ubuntu.com, or `apt-get download PKG && dpkg -c PKG_*.deb` where
apt runs. The entry may ship in a companion `-data` or `-common` package,
as Planner's in `planner-data`: read that package's list too. The entry's `StartupWMClass` may not be the window's real
class, so do not take `window_class` from it: MuseScore 3's says
`mscore3`, its window's is `MuseScore3`. The export uses the class the
capture saw.
## Drive the app
- Pass `computer_id` and the action's fields at the top level: click
`{x, y, count}`, scroll `{x, y, direction, amount}`, a screenshot region
`[x0, y0, x1, y1]`. The rest are in
[the build-computer guide](https://github.com/morluto/rea/blob/main/docs/desktop-build-computer.md).
- Take a `screenshot` before pointing: points are pixels of the latest
frame. If a response is too large, ask for a `region`, or `jpeg` with a
lower `quality`. A screenshot with `region` does not capture the screen
again; it crops the latest frame, so take a full screenshot, or a `wait`
with `screenshot: true`, first. The crop then becomes the latest frame:
take a full screenshot before pointing again.
- A drawn caret does not prove keyboard focus. After using a toolbar
control, click into the document, then type a few characters and look
before a long text.
- Confirm a dialog with `key` Return, not a trailing `\n` in `type`.
- An action's own screenshot is taken at once. After opening a menu, a
dialog or a window, look with
`{"action": "wait", "ms": 500, "screenshot": true}`.
- On a computer started from a request the app is not open at first, and
`start_build_capture` opens it: never open it yourself before that. Do not
close it while capturing; the capture ends when the app exits. When the
app is closed at any other time, open it from Applications → Other → the
environment's name.
- Type a secret with `{"action": "type_secret", "secret": "<handle>"}`,
only into a field that masks what is typed: check in a screenshot that it
is a password field, which shows dots. A screenshot shows any field that
shows its text, such as many licence-key boxes, so never type a secret into
one: say the app needs a secret the build cannot type unseen, and stop.
`type_secret` takes no `screenshot`; look with a `wait` after it. A refusal
names the secret's file and what is wrong with it: tell the person, never
the value.
- Prefer shortcuts and typing to long pointer paths: `key` takes `keys`, an
array holding one chord's keys, such as
`{"action": "key", "keys": ["Control", "s"]}`, and `type` turns `\n` into
Enter.
- Before `type`, check in a screenshot that a text cursor or a focused
field is there. In apps with single-key shortcuts, such as Inkscape, text
typed without one runs as commands.
- Save As differs by app. Check the dialog's current folder first: most
open in the home, so click Documents in the sidebar, but Scribus opens in
`~/Documents` and its sidebar has no Documents entry. Type only the file
name. A dialog taller than the 1280×800 screen hides its buttons:
maximize it from its title bar, or confirm it with `key` Return, as when
HomeBank's Save button is off-screen. When a GTK Save dialog ignores
clicks or runs off the screen, type `~/Documents/<name>` in its Name field
and press Return.
- Answer the app's save questions, such as keeping the format, then check
the saved path in the title or status bar, or the dialog's folder when
the app shows only the name, and that it shows no unsaved mark, before
you mark a checkpoint or check. Where the title never shows one, as in
HomeBank, a greyed Save or Revert, or the file reopened from recent
files, confirms the save.
- Qt spin boxes change their value on scroll: move the pointer off them
before you scroll.
- gLabels edits an object's text in Object properties → Text, not on the
canvas.
- In Grisbi's transaction form Return moves to the next field; only Return
in the last field, Notes, records the transaction.
- ghostwriter autosaves a new document as `~/Documents/untitled-1.md`: name
it with File → Rename, not Save As; Ctrl+Shift+S does nothing, and Enter
after a checklist item starts another `- [ ]` item.
- KMyMoney's first launch opens the New File Setup wizard; Finish asks for a
storage type (XML), then a Save As where typing `Documents/<name>.kmy`
works. Its date field is a sectioned Qt editor: double-click the month and
type each part, since Ctrl+A garbles it. A new transaction's payee comes
prefilled and selected. Amounts are saved as reduced fractions, such as
`-1749/20`, so compare them exactly. Untick Help → Tip of the Day → "Show
tips on startup" before `seed`, or the tip appears at every boot.
- QElectroTech: an element dropped from Collections follows the pointer until
a click on the folio places it. Its title shows only the app, so it needs
`state_paths`; its settings are in `~/.qet`.
- Rosegarden: after a dialog its Transport window can take keyboard focus,
so save with File → Save.
- Pencil2D: the Tools dock is too narrow for its buttons, so use shortcuts
(B for the brush); double-click a layer's name to rename it.
- Umbrello: the title shows no unsaved mark, so confirm a save with a probe
checkpoint. An Untitled document autosaves `~/autosave.xmi`: delete it
before `seed`. Add attributes from the tree: right-click the class → New →
Attribute.
- Hydrogen: the computer has no audio device, so Hydrogen falls back to a
null driver and its BPM field snaps back to 120 within about 100 ms.
Before `seed`, set Preferences → Audio System → ALSA with device `null`
and click "Apply and restart output", which `~/.hydrogen/hydrogen.conf`
keeps for the seed; also set autosaves per hour to 0, and dismiss the
development-version dialog with "Don't show again". `pulseaudio` as an
extra package does not help.
- Poedit cannot add entries by hand. Use File → New, pick the language and
save to `Documents` first (Sources Paths stays locked until then), then
Properties → Sources Paths → + → Add Files with a source such as
`/usr/lib/python3.12/argparse.py`. It writes `fr.mo` beside the `.po`, and
the language box's autocomplete popup takes the first OK click.
- CherryTree: its first-run autosave "Choose Storage Type" dialog swallows
keys, so save once early as `.ctd` and untick autosave.
- VSCodium (its tarball, with `extra_packages: ["libnss3",
"libasound2t64"]`) runs on outside REA's reach, so `stop_build_capture`
fails to clean it up: after `end`, quit it with Ctrl+Q, then stop. Enter
accepts its word suggestions, so press Escape after typing a line.
- Glade: a new widget's id is empty until you set it, and a second save
writes a `.glade~` backup, which shows in the draft as an unrelated
change. A backup present at `seed` is seeded with the document, so trash
it before `seed`. The draft names a button's `<child>` by the id
its widget carries, `[name=child][>object@id=submit_button]`, which moving
the button keeps; give the widget an id before `seed` so it can.
- VYM: the title never names the map, so pass `Documents/<map>.vym` in
`state_paths`. Its settings live in `.config/InSilmaril` and are written
only at a clean exit. It writes `<map>.vym.lock` while a map is open, so
close the map before `seed`. Press A only after clicking a branch on the
canvas: keys typed in the tree editor run other shortcuts.
- KmPlot: Create → Cartesian Plot, then edit the formula in the Function
tab and press Return. It stores minus as U+2212 (`−`). Its file:// title
still names the file.
- RedNotebook keeps each month in `~/.rednotebook/data/YYYY-MM.txt`, YAML
read as text, folded at about column 80: pick check phrases the fold keeps
on one line.
- Qt5 apps saving with QDom, such as KmPlot, write attributes in a random
order in each process, so check their XML by element, never by whole line.
- Gnumeric's desktop entry is `org.gnumeric.gnumeric`. Its Save As dialog
runs off the 1280×800 screen: type `~/Documents/<name>` in Name and press
Return. Rename a sheet with right-click on its tab, then Rename….
- Parley's title shows the collection's title, so name the file in
`state_paths`. File → New opens the new collection's Properties dialog:
confirm it with OK, since Cancel keeps the old collection and its autosave
lock open. A new collection starts with 15 blank entries: delete them.
Pick languages from the dropdown; a typed name keeps the previous locale.
While a document is open KDE keeps an autosave and its `.lock` in
`~/.local/share/stalefiles/parley/`, and there is no File → Close: before
`seed`, use File → New, then reopen the collection from Open Recent.
- Sigil: pass `extra_packages: ["libqt5svg5"]`, or its SVG toolbar icons are
blank. Its title names the book, so it needs no `state_paths`. It unpacks
the open book into `.local/share/sigil-ebook/sigil/workspace/Sigil-XXXXXX/`
and deletes it on exit, so never name the whole `sigil-ebook` folder: that
seeds the stale copy.
- What the app saved under the home before `seed`, such as a document in
`~/Documents` or its settings, is part of the seed. A file you move to the
Trash is left out, and the export lists it in `limitations`. Trash a stray
file rather than starting over. JabRef 3.8 writes `<file>.bak` and
`<file>.sav` beside a `.bib` it saves; in the owned folder they are seeded
and show in the draft as noise, so trash them before `seed` if they should
not ship.
- Dismiss first-run dialogs, such as a welcome screen, before `seed`. A
choice the app saves only later, such as Inkscape's "show this every
time", may not reach the seed, so expect the dialog in the witness too.
- Some apps write their settings late: Freeplane writes
`~/.config/freeplane/1.7.x/auto.properties` only once Preferences is
confirmed, and a seed without it reopens its documentation map. Before
`seed`, open the app's preferences and confirm them, then read a probe
checkpoint's `changed_files` to see the settings file written.
- Before `seed`, click into the document to clear any selection the app
would save: Gnote saves its cursor and selection with the note.
- An app whose state folder is named after it, such as
`~/.local/share/gnote`, or after it and its version, such as TreeLine's
`~/.treeline-3.1`, is found without `state_paths`.
- TreeLine: File → New opens a template window. Book List's CATEGORY makes
AUTHOR children by default; change it under Data → Configure Data Types →
Default Child Type. A new child opens an inline rename showing "", so
press Escape and fill its fields in the Data Edit tab. Ctrl+A adds a child
node even while a field has focus. Clicking a node already selected starts
an inline rename, so Ctrl+X then cuts its text: cut a node from its
context menu. It saves nodes sorted by their
generated uid1, so their order changes from run to run. The draft names a
new node by its format and a field of its `data`,
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, and checks its other
fields through that, so give each new node a title no other node has.
- After an app crashes, capture again on a fresh computer, so the seed
carries no crash-recovery state, such as MyPaint's "Recover interrupted
work?".
- Give what the task changes a stable name where the app has one, such as an
id in Inkscape's Object Properties, so a check can select it.
- Expect apps to split text into runs: Writer's revision ids split an
edited paragraph into spans and may give it an automatic style such as
`P1`, and AbiWord wraps each insertion in `<c>`. Check such text whole
with `text_equals` (see Write the task), and find the paragraph by
something the edit keeps, such as AbiWord's `xid`.
- Freeplane's first launch opens a documentation map: close it with Ctrl+W.
Its Ctrl+N asks for a template.
- MyPaint 2.0.1 crashes on brush strokes on the build computer: build its
seed with flood fill and layer operations.
- GnuCash's Ctrl+S on a new book opens a Save dialog with no format choice:
use File > Save As to pick XML. Close the book (File > New File) before
`seed`, so its `.LCK` lock is not seeded.
## Build the seed and the edit
1. `start_build_computer` with
`{"source": {"kind": "request", "desktop_entry": ..., "install": ...}}`.
The first start of an app builds its image, and starts at the same time
on this host wait for each other; either can take minutes. A failed
start returns the build's error: fix the request, or stop when the
install cannot work, and close any `computer_id` the error names.
2. `start_build_capture`. It returns once the app's window is open. A
`package_lists_error` may stop the capture finding the package's
sources; the export says when that matters. When the app exits before
its window opens, the error gives its exit code and the end of its
stderr, which may name what it lacks, such as an icon loader or a Java
runtime: close the computer and start a new one with that package in
`extra_packages` (see Choose the install), or stop as for an install
this flow cannot do.
- REA counts a file as the app's document only when the app's window
title shows its file name or `~/` path. When the title shows something
else, such as MuseScore's score title, pass the path you will save the
document at in `state_paths`, relative to the home:
`{"computer_id": ..., "state_paths": ["Documents/MuseScore3/Scores/Morning_Song.mscz"]}`.
It need not exist yet. Add the app's settings folder when its name
differs from the app's, such as `.config/MuseScore`, so the seed keeps
the dismissed first-run dialogs. Do not pass the document as a launch
argument to make REA see it: `state_paths` is the way. gLabels' title
leaves out the file's extension and Grisbi's shows the entity's name,
so both need it.
- An app whose document is a folder of pages, such as a Zim notebook
(`Notebooks/Notes`, a `.txt` file per page), never shows a file in its
title: pass the folder in `state_paths`. Every page in it is then a
document, so the bundle opens none at launch and the app starts where
it keeps its place; check the page the task changes.
3. Make the seed through the app and save it from the app under
`~/Documents`: build the document in the app, or write its source where
the app edits its format as text, then save it again from the app. Make
it look real but synthetic: the thing the task changes, plus a few things
the change must leave alone. Keep it to one document: the bundle's app
opens the seed's document at launch only when there is exactly one, and
otherwise starts on its own screen. For a starting file of the requester's,
read it here and rebuild it in the app with synthetic data in place of
personal data; REA cannot copy a file into the computer.
- An app that keeps its own data and saves after every change, such as
Osmo's contacts in `~/.local/share/osmo/contacts_records.xml`, has no
document: that state is the seed. Build it in the app and mark `seed`
once the app has written. To see whether it has, mark a throwaway
checkpoint, such as `probe-1`, and read its `changed_files`: up to ten
files that changed since the checkpoint before, those a window title
named first, then those under `state_paths` and those the app held
open, caches last, with `more_changed_files` counting the rest.
- The app's own compressed format works when it is gzip, such as
Gnumeric's `.gnumeric`, Dia's `.dia` or Inkscape's `.svgz`: REA reads
the document inside, and checks point into it as into the same
document uncompressed. Compression may be off by default: Dia's
"Compress diagram files" is unticked, which REA reads as plain XML. A format REA reads as binary, such as one
compressed with xz, has no view: save in a format the app offers that
REA reads, then reopen the file once and check that a plain Ctrl+S
keeps that format. When the task is about the app's own binary
format, such as GIMP's `.xcf` or Blender's `.blend`, keep it and check
it through a reader (see Read a binary file with a reader).
4. Only once the seed is saved, `mark_build_checkpoint` with name `seed`.
5. Make the change the task asks for, as a user would, and save it.
6. Only once the change is saved, `mark_build_checkpoint` with name `end`.
For more tasks in this environment, make each next task's change on top
of the last as a user would, save it, and mark `end-<task name>`, such as
`end-add-bread`. Each task is checked at its own end, so its checks must
not read what a later task changes, and a solving agent starts each from
the seed, so a task must not need an earlier one done first.
Then, for each task you can, make a near miss of its change on top, a
plausible wrong edit such as a typo or a wrong number, save it and mark
`near-<task name>`: the export refuses a task whose near miss passes every
check. Take the near misses after the last end, since each later draft
starts from the end before it.
7. `stop_build_capture`. Read `draft.files` (each file that changed, with
its net change and its `values`), `draft.task.checks` (suggested
checks), `draft.guards`, `draft.notes` and `limitations`; each
`later_drafts` entry is the same for one later task, from the end
checkpoint before its own, so it shows only that task's change. When
`draft` is null, read `draft_refusal` first: a missing `seed` or `end`
checkpoint means capturing again on a new build computer, while a failed
draft command names its own cause to fix. If a declared secret was used,
automatic drafting is withheld: write the task manually and export it.
When `draft.files` is empty,
`draft.notes` says why:
- `no_change`: nothing was saved between `seed` and `end`. Capture again
on a new build computer, and save before marking `end`. When the app
keeps the task's state only on screen, such as a calculator's result,
judge the screen instead (see Judge the screen).
- `unowned_changes` with `no_document`: the app saved your document
where REA did not count it as the app's. Its `files` list what changed,
with their roles; the document is among them. Close the computer and
capture again on a new one with that path in `state_paths`; the export
refuses a check on a file outside the owned paths.
- `uncheckable_changes`: the app saved the change in dconf, as gedit does
its preferences, and no check reads dconf. Saving again does not help:
choose a change the app saves in a file, and capture again on a new
build computer, or judge the screen (see Judge the screen).
- `unreadable_changes`: a dconf database changed but REA could not read
it; `error` says why. This is not evidence the app saved there: report
the error rather than changing the task.
A note with a `file` says why that file's suggestions are thin:
- `by_position`: only position names what changed; its `elements` lack
a naming attribute. Name a record that keeps its fields as child
elements by one of them, as Osmo's contact by
`[name=contact][>first_name=Delphine]`, a JSON record by a field
inside it, `[>data/Title=…]` (`~1` for the `/` inside a pointer), or
an element by an id its child carries, `[>object@id=…]`; count records
with `member_count`, `matching` `[name=contact]`; otherwise check an
ancestor's text, or give the element an id in the app and capture again.
- `view_state_only`: every change is view state, a renumbered id or a
time the app stamps, and its `pointers` are the checks kept. The edit
was likely not saved: capture again and save it before `end`.
- `long_text`: the element's text is too long to check whole; check a
phrase in it with `text_contains`.
- `cut_view` or `unreadable_view`: REA's view holds only part of the
file, or none a check can read, such as a binary format (step 3). For
a binary file, read it with a reader (see Read a binary file with a
reader).
## Write the task
Write `{"name", "prompt", "checks"}` from the draft and what you did, one
per task, each with its own name.
- `name`: lowercase letters, digits and hyphens, not `smoke`.
- `prompt`: what the requester wants, in their words and the app's terms,
starting with the app's name, naming the document as a user would and
ending with saving it: "In Inkscape, change the poster's headline in
Documents/poster.svg to Summer Bake Sale and save it." Name the app: the
solving agent's own shell cannot see the computer's files, and the app's
name sends it to the screen. Leave saving out for an app that saves on its own, after every change
or a few seconds after the last one, such as Osmo or Gnote: grading waits
until the app's files have held for 3 s, so the solving agent need not
wait for the save. A save more than 3 s after the agent stops is still
missed; name such an app in the report. Never say how the file stores it,
and never give the checks.
- Checks are file-based: content checks that prove the edit, never only that
a file exists or changed. Start from the suggestions; keep those that say
what the user did and drop what follows from how the app saves.
- Compare exact values, so a typo or a wrong number fails.
- Read each file's `values` before writing checks: every value the change
added, changed and removed, with `before` and `after` (an uncut scalar, or
a preview cut at 600 characters) at the pointer a check names it by. Copy
the pointer. Copy `after` into `value` only when it is an uncut scalar: a
string without a trailing `…`, a number, or a boolean. Compact XML and a
JSON array or object are preview strings, so a `value_equals` check needs
the structured value from the file's view, and element text uses
`text_equals` or `text_contains`.
Only XML files, archives, JSON and key files have values: a text file's
or SQLite database's `change` already lists its lines and rows.
`refers_to` says what a number or id stands for, such as `category="44"`
naming `<cat key="44" name="Groceries"/>`, or a new KMyMoney
transaction's splits naming their account and payee; use it for the
prompt's words. `alternatives` gives a changed value's pointer by the
element's other names, such as a label beside an id, to pick another key
without another export.
Skip entries whose `note` is `by_position` (no check can use that
pointer) or noise (`view_state`, `renumbered`, `volatile`,
`timestamp`, `upkeep`, `layout`). `values_cut` means a limit left some out.
- Select elements by what they hold, never by position: SVG text by id,
`/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0`,
or an ODF paragraph by style,
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Title]/children/0`.
A selector must match exactly one element, so give the target a unique id
or style in the seed. The draft's pointers already do this: a name-like
attribute such as Scribus's `ANNAME` before a coordinate, and two
attributes together where one is not enough, as a spreadsheet cell's
`[@Col=3][@Row=2]`. Only an element's text is read by position, as
`children/0`.
- When an app may split an element's text into runs, check it whole:
`text_equals` (or `text_contains` for a value inside a sentence) on the
element compares all its text, however it is split. Add
`normalize_whitespace: true` when the app may reflow spaces. Example, an
AbiWord paragraph by its stable id:
`{"kind": "text_contains", "file": "Documents/memo.abw", "pointer": "/root/children/[name=section]/children/[name=p][@xid=6]", "value": "at 3:30 PM"}`.
Point a text check at the paragraph or another element the edit keeps,
never at a run such as AbiWord's `[name=c]`: a reopened document splits
the text into several runs. The draft lists only what changed; an
unchanged element is in `draft.guards` when it is a like sibling of a
changed one, and otherwise use the app's usual numbering, such as another
paragraph's `xid`, and let the export confirm it.
- In an ODS spreadsheet no attribute names a row: the draft names it by its
label cell, `[name=table:table-row][>table:table-cell=February]`. Where no
cell names it alone, name the sheet in the seed, and check its whole
text: `text_equals` on
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:spreadsheet]/children/[name=table:table][@table:name=Q1 Sales]`,
or a stretch of it with `text_contains`. Cell texts join with nothing
between, so a row reads `February23100`.
- For a line added to free text, such as an item on a note's list, the
draft compares the whole text with `text_equals`. This rejects longer or
negated items, but can reject a correct edit with different ordering or
spacing. Report that limitation when the task permits such variations.
- For a note placed by its own `x` and `y`, such as Xournal++'s `<text>`
on a layer, the draft checks its text on the layer with `text_contains`,
which passes wherever the user puts it, and guards each note it kept.
- Some apps keep text in an attribute, one element per run, such as
Scribus's `<ITEXT CH=...>`: add `"text_attribute": "CH"` to `text_equals`
or `text_contains` on the frame,
`/root/children/[name=DOCUMENT]/children/[name=PAGEOBJECT][@ANNAME=Headline]`,
to read its runs' values joined.
- When the draft shows no values or no checks, a scratch export can expose
`envs/<env>/seed/` only after the task has at least one valid check: it must
pass on the captured end state and fail on the seed. Before the first
successful export, use `act_on_build_computer` to open the saved file in an
available GUI text editor or viewer and inspect its actual representation
without changing it. Use the visible saved content and the task's edit to
write a check; an export refusal can also name the end state's value, such
as `is "44", not "1"`. If the file cannot be inspected or expressed by a
supported check, report the limitation instead of guessing a check.
- Where easy, add one check that the rest is untouched, such as another
text's value the edit must leave alone. `draft.guards` offers them at
their seed values, which the end holds too: a field on the nearest like
siblings, one the user sets, such as Planner's `work`, before a default
they all share; the text an addition kept; a text file's lines beside its
new one. Copy one or two into `checks`; they pass on the seed, which is
fine, since the export needs only one check to fail there.
- The draft suggests a decimal the app writes as text, such as `146.83`,
as a number with `tolerance: 0.001`; keep the tolerance for any number
you check that way.
- When the draft shows an XML document's changes as `lines`, REA read it
as text, for example because its DOCTYPE declares entities. Selectors are
impossible there, and `line_count` with `contains` only approximates a
value: `3:30` also matches `13:30`. Bound it with neighbouring text and
name the check as approximate in the report's limitations.
- Some apps save a document reloaded from disk differently from one typed
in the same session: AbiWord wraps each new insertion in its own markup.
Your build's end state hides this, which is why whole-text checks matter;
expect the witness to show the real format.
The kinds and pointer rules are in
[the desktop tasks section](environments.md#pick-the-checks).
### Read a binary file with a reader
When the draft notes `unreadable_view` on a binary document, add a reader to
the task: the app's own headless command, which REA runs in the computer on a
private copy of the file, and whose output a check reads instead of the file.
Write it in the app's scripting mode, print each fact on a line starting
with `REA ` (with the space), and name the copy `{file}`:
- GIMP `.xcf`, layer names, as `lines`:
```json
{
"id": "layers",
"file": "Documents/poster.xcf",
"argv": ["gimp-console-2.10", "-i", "-d", "-f", "-b", "-"],
"stdin": "(let ((image (car (gimp-file-load RUN-NONINTERACTIVE \"{file}\" \"{file}\")))) (for-each (lambda (layer) (display \"\\nREA layer \") (display (car (gimp-item-get-name layer)))) (vector->list (cadr (gimp-image-get-layers image)))) (newline))\n(gimp-quit 0)\n",
"output": "lines"
}
```
checked with `{"kind": "lines_present", "file": "Documents/poster.xcf", "reader": "layers", "lines": ["layer Summer Sale"]}`.
Script-Fu's prompt `ts> ` starts a line, so print a newline before each
marked line, as `\nREA` does. A text layer's text is
`(car (gimp-text-layer-get-text layer))`.
- Blender `.blend`, objects with their type and location, as `json`:
```json
{
"id": "objects",
"file": "Documents/scene.blend",
"argv": [
"blender",
"-b",
"--factory-startup",
"{file}",
"--python-expr",
"import bpy, json; print('REA ' + json.dumps({o.name: {'type': o.type, 'location': [round(v, 4) for v in o.location]} for o in bpy.data.objects}, sort_keys=True))"
],
"output": "json"
}
```
checked with `{"kind": "value_equals", "file": "Documents/scene.blend", "reader": "objects", "pointer": "/Crate/location/0", "value": 2, "tolerance": 0.001}`.
Print every value a check compares as a number or string a check can
name, rounded where the app stores floats.
Blender 4 shows its Quick Setup splash at every launch until its
preferences are saved: with `.config/blender` in `state_paths`, confirm
Quick Setup with Continue, save the preferences (Edit → Preferences, then
Save Preferences in the menu at its bottom left), and read a probe
checkpoint's `changed_files` for `.config/blender/4.0/config/userpref.blend`
before `seed`, so solves start on the scene. Blender often takes the first click on a
menu or a dialog's button only to focus it: click again.
- A format LibreOffice opens but REA cannot read, such as a legacy `.xls`,
converted with `--convert-to`:
```json
{
"id": "cells",
"file": "Documents/budget.xls",
"argv": [
"sh",
"-c",
"soffice --headless --convert-to csv --outdir out \"$1\" >/dev/null && sed 's/^/REA /' out/*.csv",
"sh",
"{file}"
],
"output": "lines",
"timeout_s": 120
}
```
Pass `{file}` to `sh -c` as an argument, as here, rather than inside the
script.
The reader runs without the display, in a fresh home with the app's default
settings, so use the app's headless or batch mode; set `display: true` only
for an app that has none. It must exit 0 within `timeout_s` (60 s by
default, at most 300) and print at most 1 MiB. Pick facts the change sets and
the near miss gets wrong, such as a name or a number, not the app's whole
output.
Give the task the reader in `readers`, and each check through it `reader`
and the reader's `file`. `json` readers serve `value_equals`, `value_absent`
and `member_count`; `lines` readers `lines_present`, `lines_absent` and
`line_count`. The export runs the reader in the build computer on the seed
and the end: when a check through it fails on the end, the refusal shows
what the reader printed, and a reader that fails shows its error and the end
of its stderr. Fix the reader or the check from that and export again.
### Judge the screen
Only when no file shows the change, as when the draft notes `no_change`
because the app keeps its state on screen, write the task's checks as
screen judges: `{"id": "shown", "kind": "screen_judge", "question": "Does the
calculator's display show 8.568?"}`. A model, the judge, answers the question
on a screenshot three times, and the check passes when most answers are yes.
- REA keeps a screenshot at each checkpoint. Mark `seed` with the screen
showing the state before the task, where the answer is no, and `end` with
the change showing, where it is yes: the bundle ships both as the judge's
examples.
- Ask about what the change makes visible, with exact values, so the near
miss is a no: a result, a label, a selected item. Avoid what a window
moved or a cursor blinking can change.
- The prompt asks to leave the result showing, since the judge sees only
the screen when the agent stops.
- A task with screen judges holds only screen judges and no readers. The
export refuses one when a file shows the change, and checks none of its
questions: the witness does, running the bundle's own judge with the
runner's model endpoint (`LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
`[model]` in `.agentenv/config.toml`) on a fresh screenshot at each stage.
Without an endpoint the witness's check fails: say what to set and stop.
- The judge's model is `[model.roles] judge`, else `[model] default`, else
`anthropic/claude-sonnet-5-5`. `agent-env run --judge-model` does not
reach it. Each grade costs about $0.02.
## Export
`export_build_environment` with `tasks` and `directory`, the absolute path of
a new or empty folder here, such as `<home>/rea-builds/poster/bundle`. Each
entry of `tasks` is `{"task": ..., "end_checkpoint": ...}`: leave
`end_checkpoint` out for the task checked at `end`, and give
`end-<task name>` for each later one. The
export seeds what was saved at `seed`, replaces the computer's host name,
`user@host` and home path (a bare user name, such as AbiWord's author
`user`, stays), and checks each task against `seed`, its own end and its
`near-<task name>` when you took one. It refuses a task
unless at least one check fails on the seed, every check passes at its
end and one fails at its near miss, and says which task and why, such as a
pointer that names nothing at the end. A
check that the rest is untouched passes on the seed too, which is fine. On
a refusal, fix the task and export again, into another folder if that one
is no longer empty. The bundle's eval lists every task, so a run of the
bundle runs them all. A refusal naming a secret's handle means a file of the
bundle holds that secret, most often an app's sign-in or licence state in
the seed: do not work around it, but report it and stop. Keep
`seed_files`, `scrubbed_files`, `unscrubbed_files` and `limitations` for the
report.
The limitations name the folders the app wrote that the bundle neither owns
nor seeds, such as `~/.config/MuseScore`, so the app starts without them:
expect its first-run wizard, a tour or a "restore session?" question at every
boot. If the witness shows one, capture again on a new build computer with
those folders in `state_paths`, and dismiss the dialogs before `seed`. Lock
files the app holds while it runs, such as LibreOffice's `.lock` and
`.~lock.<name>#`, GnuCash's `<book>.LCK` or Grisbi's `.<file>.lock`, are
left out of the seed and listed after "Left out of the seed". An app can
still start fine yet refuse every save when a lock was seeded, so close the
document before `seed` where the app lets you (apps other than LibreOffice,
such as Rosegarden, also write `.~lock.<name>#` and autosaves beside a
document, and closing it covers them too), and treat "file is locked"
on a save in the witness as a seeded lock: delete it from the bundle's seed
and witness again in a new computer. In LibreOffice, own the profile
(`state_paths: [".config/libreoffice"]`) for a clean boot, and close the
document before `seed`, then reopen it from Recent Documents to make the
change: the export also leaves out the recovery list LibreOffice writes while
a document is open, which would start the bundle into document recovery. A
limitation naming `~/.config/dconf/user` as unread means the app's dconf
settings, such as a first-run tip you turned off, are not seeded: capture
again on a new build computer. The limitations also name seeded WebKit,
Chromium or QtWebEngine storage and caches; when the witness's boot is clean,
list them in the report.
Keep the build computer open until the witness holds: a task fix exports
again from it.
## Witness
Witness each task in its own fresh computer of the bundle, from the prompt,
as a solving agent would; a computer is witnessed once, so the steps below
repeat for each task, each with its own record folder. Do not replay the build's actions or coordinates, and do not
check between stages to steer the edit: a solving agent never sees the
checks. Give every `check_build_task` the task exactly as exported.
1. `start_build_computer` with
`{"source": {"kind": "bundle", "path": "<bundle>"}}`, adding
`"settings": {"<variable>": "<handle>"}` for each of the export's
`withheld_files`, or the deploy is refused for want of them, and agentenv
keeps that bundle's build failed, so start from a fresh copy of it. The app starts
with the seed's files in place and opens the seed's document when the
seed holds exactly one and the app takes a file. When the export's
`limitations` say it opens no document, open it from the prompt's path,
as a solving agent would. Take a `wait` screenshot first, since the
desktop may not have drawn yet. When the start fails because the
computer never got ready, and its error names lock files in the bundle's
seed, delete them from the bundle and start again.
2. `check_build_task` with `stage: "seed"` and `directory`, the absolute
path of a new or empty folder for the record. A check must fail.
Recording starts. Later stages take no `directory`.
3. Do the task from the prompt with `act_on_build_computer` and save.
`check_build_task` with `stage: "after_edit"`: every check must pass.
Each check waits until the app's files have held for 3 s, as the
bundle's verifier does, so an app that saves on its own needs no wait of
yours. A result with `settled: false` means the files were still
changing at the 15 s bound, and `read_cut: true` that a slow read was
still running there.
4. Make a near miss on top of it: a plausible wrong edit, such as a typo or
a wrong number, and save. `check_build_task` with `stage: "near_miss"`:
a check must fail.
5. `act_on_build_computer` with `action: "reset"`, then `check_build_task`
with `stage: "after_reset"`: a check must fail again.
6. The last stage ends the witness: `witness.record` has the verdict, the
path of `witness.json` and the recording's segments in `directory`.
`close_build_computer` on the witness computer.
A stage that does not hold ends the witness as `failed`. Read each check's
result and decide:
- `after_edit` failed though the saved file is right: the check is brittle.
Fix it, export again and witness in a new computer.
- `after_edit` failed because your edit was wrong: witness again in a new
computer; a computer is witnessed once.
- `near_miss` passed: the checks are too loose. Tighten them, export again
and witness again.
- `after_reset` passed: the reset did not restore the seed. Report it as a
limitation.
Stop after three witnesses of a task and report what still fails.
## Measure
Only if the requester asked for it, or by the headless default above. If
they wanted to review the tasks first, show them each prompt and checks and
wait for their approval. Then `measure_build_task` each task with `bundle`,
`task`, `model` and `budget_usd`, its share of the budget. It runs the task
in fresh computers, 3 times, then 2 more unless they passed none or all, and
keeps a task that passes 1 to 3 of 5 (`band: "in_band"`). It stops before a
solve the largest cost so far would take past the budget, and writes the
record into the bundle's `difficulty.json` and README. The solver needs a
model endpoint where REA starts: `LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
`base_url` and `api_key` under `[model]` in `.agentenv/config.toml`; when the
measure says a solve did not run for want of one, tell the person what to
set and stop. Read the result's `failures_per_check` and each solve's
`failed_checks` before calling a task hard: a check that fails in most failed
solves while the app shows the right change is brittle, so fix the check,
export and witness again, rather than count it as difficulty.
## Harder tasks
When the requester wants tasks worth evaluating or training on, harden a
task that came out `too_easy` and measure it again, within the budget. The
solver sees and fixes its slips on screen, so one edit with the value in the
prompt is solved almost every time; so far, on TreeLine with Sonnet 5.5, so
were one more dialog, a conditional move of seven records, and ten
corrections read from a second document and applied across the list. Try,
one at a time, measuring each:
- A longer chain of exact edits than that, each a value the checks compare
whole, so one slip in the chain fails the task.
- Facts read in one view and used in another: values the solver must read
from a second document the app opens, or from another record, rather than
from the prompt, such as "the same year as The Quiet Glacier's".
- Distractors beside each target: records with near-identical names or
values, and a guard check on each that it stays as it was.
- Mistakes that do not show once saved: a value the app displays rounded or
formatted unlike what it stores, or an edit the app keeps only when its
field loses focus.
Stop when the budget left cannot cover 3 solves of the hardened task, and
report the bands measured.
Each hardened task stays sound: its checks still fail on the seed and pass on
its end, its near miss fails, and another valid way of doing it passes, so
check values, never the order or method the solver used. Put the second
document in the seed. Count records with `member_count` and check each
value by a selector that names its record, since no check joins two records.
## Close and report
`close_build_computer` on every computer still open, also after a failure.
If a close fails, call it again; if it still fails, tell the requester what
is left.
Report:
- the bundle path and the environment name;
- each task's name, prompt and end checkpoint;
- each check, and why it proves the edit and fails the near miss, and each
reader's command;
- each task's witness stages with their results, the verdict, and the paths of
`witness.json` and the recording;
- the seed's contents: `seed_files` and what each holds;
- `scrubbed_files` and `unscrubbed_files`; an unscrubbed file still names
the host or user, so it needs review before sharing. `recompressed_files`
are gzip seeds whose headers were cleared, their content unchanged: give
their count;
- limitations: the capture's, the export's and anything you could not do;
- each measured task's band, passes of finished solves, turns, cost and
failures per check, from `difficulty.json`;
- the build time and the questions you asked.
Never publish or share the bundle yourself.
@@ -1,399 +0,0 @@
# Agent environments from a live application
Use this when the user asks for an agent environment (an AgentEnv bundle) of a
web application: a folder `agent-env` deploys, whose MCP tools answer from what
one recording of the application observed and serve the writes that recording
confirmed. The workflow records the application once with
`capture_browser_scenario`, checks what the environment would serve, records
again if a hint names a gap, then writes the bundle with
`export_environment_bundle`. For a HAR the user already has, skip to
[export](#export-and-hand-off) through `inspect_web_network_capture`, or use
`rea environment-build <capture.har> <dir>`. For a task in a desktop
application's exported environment, read
[its section](#tasks-for-a-desktop-application-environment).
## Before recording
- Record only against an instance the user says is disposable or seeded for
this purpose: every write step creates, edits or deletes its data. If the
user has not said so, ask before any scenario that writes. REA does not
start, seed or reset applications.
- Credentials come from environment variables the user names. Declare each as a
scenario secret, `{secret_id, environment_variable}`; the variable must be set
in the environment REA's MCP server started with. Never ask for a password in
chat. Declared secrets are redacted as the capture records.
- Map the application first with `inspect_web_page` (and `capture_web_screenshot`
when layout matters): the login form's locators, and the API calls behind the
pages that list and edit the entities the user asked about.
- `inspect_web_page` cannot sign in. To see pages behind sign-in, run a short
`capture_browser_scenario` that signs in and opens them, with
`"capture": {"after_each_step": ["accessibility"]}` (`"dom"` for markup).
Read a step's tree with `inspect_analysis_view`, `view: {kind:
"browser-scenario", collection: "text", step_index, artifact:
"accessibility", offset: 0, limit: 2048}`, paging with `offset`. Capture
options are only `after_each_step`, `at_end`, `events`, `network` and
`limits`.
## Record a session an environment can learn writes from
One `capture_browser_scenario` records the whole session. Start from this
shape, with the application's own locators, paths and headers:
```json
{
"browser": { "mode": "launch", "executable_path": "/usr/bin/chromium" },
"start_url": { "url": "http://127.0.0.1:3000/login" },
"secrets": [
{ "secret_id": "admin_password", "environment_variable": "APP_PASSWORD" }
],
"capture": {
"network": {
"request_body": true,
"response_body": true,
"header_values": true
}
},
"actions": [
{
"step_id": "user",
"action": "fill",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=username]" },
"value": { "source": "literal", "value": "admin" }
},
{
"step_id": "password",
"action": "fill",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=password]" },
"value": { "source": "secret", "secret_id": "admin_password" }
},
{
"step_id": "sign-in",
"action": "press",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=password]" },
"key": "Enter"
},
{
"step_id": "signed-in",
"action": "wait_for",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "nav a[href='/settings']" },
"state": "visible"
},
{
"step_id": "folders-1",
"action": "request",
"timeout_ms": 15000,
"method": "GET",
"path": "/api/folders",
"expected_status": [200]
},
{
"step_id": "new-folder-1",
"action": "click",
"timeout_ms": 15000,
"locator": { "kind": "role", "role": "button", "name": "New folder" }
},
{
"step_id": "folder-title-1",
"action": "fill",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=title]" },
"value": { "source": "literal", "value": "Scratch folder 1" }
},
{
"step_id": "create-folder-1",
"action": "click",
"timeout_ms": 15000,
"locator": { "kind": "role", "role": "button", "name": "Create" },
"save_response": {
"method": "POST",
"path": "/api/folders",
"save": [{ "name": "folder_1", "json_pointer": "/id" }]
}
},
{
"step_id": "read-folder-1",
"action": "request",
"timeout_ms": 15000,
"method": "GET",
"path": "/api/folders/{saved:folder_1}",
"expected_status": [200]
}
]
}
```
Give each step that accepts `timeout_ms` a deadline. Without one, a step waits
until it completes or the scenario is cancelled, so a locator that never
matches hangs the capture and the client with it. The `wait_for_timeout` pause
instead uses `duration_ms` and does not accept `timeout_ms`. Match `wait_for`
locators to exactly one element.
1. Sign in through the UI: `fill` the user and password fields (the password
as a secret reference), submit, then `wait_for` an element only a signed-in
user sees.
2. Read with `request` steps: each collection the environment needs, twice
before any write, so fields that change on their own show up; then each
object a list shows by id, up to ten per collection. An object known only
from a list that leaves out fields its own read shows cannot be read by id
or updated in the environment.
3. Write the way the application's UI does: fill its forms and click its
buttons, so each request carries the fields the UI sends, including ones
the server would otherwise derive. Give the click or key press that submits
a create `save_response` to keep the new object's ID, then `goto` its page
(`/items/{saved:item_1}`) or read it with a `request` step. Never guess an
ID, and never create a probe object to learn the next one. Write with
`request` steps instead only when the UI cannot reach the operation (an
API-only resource, or a UI that does not call the JSON API), or when coverage
withholds the UI-shaped write as `unexplained-change` or `contradicted`;
that counts as one of the two re-recordings. A `request` step write that
coverage withholds that way usually lacks fields the UI sends: record it
through the UI instead. If a journey cannot be scripted, leave the write
unserved and report the limitation.
Record two numbered scratch objects per write type,
with varied free text. After each write, and before the next one, read the
object back by id and then its list: create 1, read 1, list, create 2,
read 2, list; never several writes before their reads.
4. Leave more than a second between two creates and between a create and its
first edit (`wait_for_timeout` with `duration_ms: 1100`), so lists sorted
by creation time show them in order.
5. Update both objects and read each back, changing the fields a person
would change (a name, an email address, a status, a due date) rather than
only a description: a served update accepts only the members a recorded
update stored. Run an action that turns something on or off (star,
bookmark, archive) twice in a row on one object, reading it back after
each: the second call shows whether it sets or flips the field, and the
environment serves the action only once a call has shown it. Delete both
and read each back (expect 404, or the object marked deleted) and the list.
6. Vary what the two creates send, however you write them: the first every
field the application takes, the second only what a person must give (its
title or name), with the other fields left out or empty. A served create
requires each member both creates filled in and lets a caller leave out one
a create left out or sent empty, so an agent that names only a title is not
refused for fields it had no reason to set.
About `request` steps:
- `path` is on the start URL's origin; the page must be on that origin.
- Header values are literals, declared secrets, a cookie the browser holds
(`{source: "cookie", name}`), a local storage entry
(`{source: "local_storage", key, json_pointer}`), a saved value
(`{source: "saved", name}`), or Basic credentials
(`{source: "basic_credentials", parts: [username, password]}`, sent as the
base64 of `username:password` and redacted), with an optional `prefix` such as
`"Bearer "` or `"Basic "`. Each credential part is a literal, secret, saved
value, cookie or local storage entry, so a header the frontend builds from a
login token (the username and the token) can be copied too; the password is
a secret or saved value, never a literal. Copy the
headers the application's own requests send (a CSRF header such as
`X-CSRFToken` from the `csrftoken` cookie, a bearer token). Values read from the page are redacted from records made after they
are read, not from the login response that returned them, and values
shorter than 8 characters are not redacted at all.
- `save: [{name, json_pointer, secret}]` keeps a string or number from the
step's JSON response, such as the created object's `/id`, for later steps;
`secret: true` redacts it, as for a token the API returns, so mark every
saved token, key or session value secret. This also protects the selected
source response before its body or headers are retained. Compressed or
unknown-size UI responses cannot supply saved values. On a `click` or `press`,
`save_response: {method, path, save}` keeps values from the response to the
new request observed while the step waits (`{*}` in `path` matches one segment).
Matching does not prove UI causality: choose a path distinct from polling;
multiple observed matches fail as ambiguous. The step
fails unless a 2xx JSON response arrives before its `timeout_ms`. Later
steps send a saved value as `{saved:name}` in a request `path` (and its
query) or a `goto` URL; in the `json` body, where a string that is only
`{saved:name}` keeps the saved JSON type; or as `{source: "saved", name}` in
a header or credential part.
- Reading a page value makes later bodies costlier to keep: after a CSRF
token, bodies over about 47 KiB are kept as `unavailable`. When the API
returns larger responses, raise `capture.limits`
(`max_network_body_allocation_bytes`, `max_total_event_allocation_bytes`).
- Request steps follow no redirects: a redirect fails the step, and
`expected_status` cannot name a 3xx status. Use the path the redirect names,
such as the one with the trailing slash.
- Set `expected_status` on every step, so a refused call fails its step and
the later steps report not having run, instead of recording read-backs of
nothing.
Keep it to about 60 to 160 requests. A scenario with no `request` step is
recorded exactly as before.
The capture returns its whole Evidence, every recorded body included, which is
often more than a client shows inline. Before checking coverage, confirm that
every entry of `normalized_result.steps` has `status: "completed"`; if the
client saved the result to a file, search it for `"status":"failed"` and read
that step's `error`. A failed sign-in or write cancels every step after it.
Use the result's `evidence_id` for coverage and export; do not paste the
Evidence back into another tool.
## Check coverage
`inspect_analysis_view` with `source: {kind: "retained-evidence",
evidence_id}` (the capture's id) and `view: {kind: "environment-coverage",
template_rules: ["after-uid-literal", "opaque-token", "sibling-variance",
"prefix-variance"]}`. It reports read tools
answered from the state or replayed (`reads.recorded_reasons`), the served write
tools, and each withheld write with its `reason` and `hint`. It does not apply
the serve-time catalog and state size checks; the export reports those.
Re-record a withheld write at most twice, and only for a recording gap. Stop at
once when the reason is application behaviour, and report the write as
withheld with its hint:
| Reason | What to do |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
| `no-observation` | Re-record: read the entity back by id after the write; read it first for an update or delete. |
| `unconfirmed` | Re-record: read the entity and its list after the write. |
| `incomplete-entities` | Re-record: read the entity through every view that lists it. |
| `unexplained-change` | Re-record once with a second, different call of the same write. |
| `no-call` | Re-record once with a write that succeeds. |
| `side-effect` | Stop: the application changes other entities. |
| `contradicted` | Stop: a later read disagrees with the modelled effect. |
| `body-unusable` | Re-record a UI write once with `request` steps (its body was not exposed); else stop. |
Use new scratch names on each attempt, and delete what an earlier attempt
created before recording again (through the UI, or a `request` step with the
id that capture shows): the environment starts from what the recording first
reads, so a leftover object becomes seeded data that write tasks then find.
New browser scenario captures include request times, so a fully timed capture
can learn a write clock and serve updates that stamp modification times. Older
captures or ones missing a request start keep record order and learn no clock.
When coverage reports that limitation, record again with the current version
before concluding that the timestamp change cannot be served.
## Export and hand off
`export_environment_bundle` with `source: {kind: "retained-evidence",
evidence_id}` (or `{kind: "evidence-file", path}`), an absolute
`output_directory` that does not exist yet and whose parent does, and a `name`
(required when the application runs on `localhost` or an IP address). It returns
the coverage again, `tool_count`, the written files and warnings; `dry_run:
true` reports without writing. The bundle installs `rea-agents` of this REA's
version from the registry; when the user runs an unreleased REA build, pass the
`package_tarball` they give you (a packed `rea-agents` tarball) instead.
Tell the user how many tools answer from the state, which writes are served and
which are withheld and why, and the run command (`agent-env run <folder>`). Say
that the bundle ships the recorded responses and the capture's local paths
(for a browser scenario, the browser executable's path), and
list any `credential-named-members` warning pointers for review. Never publish
or share the bundle yourself.
See
[agent environments](https://github.com/morluto/rea/blob/main/docs/agent-environments.md)
for what the state, replay and the bundle establish, and
[evidence workflows](evidence-workflows.md) for the manifest and state views.
## Tasks for a desktop application environment
Use this when the user has a folder from `rea capture-desktop-application` and
wants a task for the environment `rea export-desktop-application` makes from
it: a prompt for a computer-use agent and checks that grade what the agent
left in the app's files. You write the prompt and the checks; REA validates
the task against the capture and exports it. These commands are CLI only. To
build the environment and its task from a request instead, follow
[desktop builds](desktop-build.md).
### Describe the change
Run `rea describe-desktop-application-task CAPTURE --format json`. Read:
- `owned_paths`: the only paths whose files the app serves, so the only files
a check can read.
- `files`: each file there whose state differs between the capture's first
and last version, with its `role`, the `baseline` and `final` versions and
the net `change`. Text changes list each added and removed line once, and
`line_counts` gives lines found at both ends whose count changed.
- `values` in each of those files: what the change added, changed and
removed, `before` and `after`, at the pointer a check names it by, with
`refers_to` naming the element a key such as `category="44"` stands for.
Skip entries noted `by_position` or as noise. Only XML files, archives,
JSON and key files have values: a text file's or database's change
already lists its lines and rows.
- `task`: a draft whose `checks` are suggestions from the net change of the
documents the user worked on, or of the app's own state when no document
changed.
- `guards`: optional checks that the rest kept its seed values, such as a
like sibling's field or a text file's neighbouring lines; add one or two
to show the rest is untouched.
- `notes`: why `files` is empty, or why a file's suggestions are thin, such
as `by_position` when no attribute names the changed element.
### Write the prompt
Say what the user wants done, in their words and the app's terms, naming the
app: "In Mousepad, add buy milk to my to-do list and save it". A solving
agent's own shell cannot see the computer's files; the app's name sends it to
the screen. Never say how the file stores it: no
paths the user would not name, line formats, XML, keys or SQL, and never the
checks. Replace the draft's placeholder; the export refuses it.
### Pick the checks
Checks must express only the observed change:
- Keep a suggestion only if it says what the user did. Drop those that follow
from how the app saves: a format rewrite, renumbered lines, counters. A
LibreCAD drawing gaining four holes was rewritten from DXF R12 to 2007, and
the draft suggests counts of `SECTION`, `ENDSEC` and `CIRCLE`; only
`CIRCLE` equals 9 is the user's change.
- Never check unchanged content, whole files, or values the app stamps:
times, ids it assigns, window sizes. REA refuses ISO 8601 and `asctime`
date-times (`Thu Nov 5 09:20:31 2026`); the
rest is your judgment.
- Prefer a few robust checks: whole lines over positions, rows by meaningful
columns over generated ids, `at_least` when the user may add more.
Kinds: `file_present` and `file_absent`; `lines_present` and `lines_absent`
(`lines`); `line_count` (`exact` or `contains`, and `equals`, `at_least` or
`at_most`); `value_equals` and `value_absent` (a JSON `pointer` into the view,
such as `/groups/GROUP/KEY` for a key file); `text_equals` and
`text_contains` (an XML element's whole text at a `pointer`, optional
`normalize_whitespace`); `row_present` and `row_absent` (`table`, `where` by
column) and `row_count`. Each names its `file` as a path under the home. A
document saved as a zip archive, such as FreeCAD's `.FCStd`, reads as its
entries: point into an entry's view, and name array members by selector
rather than position, as
`/entries/[name=Document.xml]/view/root/children/[name=Object][@name=Hole]`,
so the check survives an app version that writes them in another order. A
gzip-compressed document, such as Gnumeric's `.gnumeric`, reads as the
document inside: point into it as into the same document uncompressed. Give
`value_equals` a `tolerance` to compare a number the app writes as text, such
as `"5.0000000000000000"`, with a number. Check what the user wrote in a
document with `text_equals` on its paragraph element, not `value_equals` on a
text node: apps such as AbiWord and Writer split a paragraph into runs as they
save, so its first text node holds only part of it; use `text_contains` for a
value within a sentence, such as a time. A binary document, such as GIMP's
`.xcf`, has no view: give the task a `readers` entry, the app's own headless
command printing `REA ` lines, and a check naming it in `reader`, as
[desktop builds](desktop-build.md#read-a-binary-file-with-a-reader) shows. The
[desktop application tasks](https://github.com/morluto/rea/blob/main/docs/desktop-application-tasks.md)
guide has the exact rules.
### Export the task
Write the task as JSON with a `name`, the `prompt` and the `checks`, then run
`rea export-desktop-application CAPTURE DIRECTORY --task TASK.json`. REA keeps
the task only when its checks fail on the seed and all pass on the captured
end state. A refusal names the check: one that fails on the end state does not
match what the user left; "every check already passes on the seed" means no
check captures the change. For several tasks from one capture, repeat
`--task` and name each later task's end with `--task-end NAME=CHECKPOINT`.
When the export refuses an application that no apt repository installs, ask
the user for the `.deb` file they installed it from and capture with
`--package-file`, then ask where the computer may download that file and pass
it as `--package-url`. Never fetch a vendor's download page yourself: such
sites often refuse automated downloads, and the copy and its licence are the
user's to choose.
Tell the user the task's name, prompt and checks, and how to measure a solve
rate: run `agent-env run DIRECTORY --task NAME --agent claude-code` with
`--model MODEL` three times and count the runs reported `passed`. The runs
need a model endpoint: `LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
`base_url` and `api_key` under `[model]` in `.agentenv/config.toml`. Do not run
agents unless asked. The bundle's seed is the user's real data: keep it local.
@@ -11,44 +11,6 @@ returned inline. When complete `inspect_binary_layout` or
`inspect_analysis_view` for a summary, one section/symbol/module, or a stable
page instead of repeating the producer.
For any logical Evidence source that advertises retained references, pass
`{kind: "retained-evidence", evidence_id}` instead of copying the complete
record back into a call. The same contract covers individual, optional, array
and nested sources across native, process, JavaScript, managed, mobile and WASM
workflows. Sources can independently mix inline records and exact references.
Resolution preserves the source's identity, provenance, coverage, partial facts
and unknowns; it neither reruns analysis nor makes incomplete facts complete.
Keep domain role and identity requirements: a native dossier cannot substitute
for a managed member observation, for example.
When starting from a summary or inspection page, use its
`normalized_result.parent_evidence_id` to reference the complete source.
The view's own `evidence_id` names
only that view. `compare_artifacts` explicitly supports artifact summary inputs
and resolves their parents; do not assume every consumer does. When a workflow
follows source links, keep those linked records available too.
MCP references belong to the current connection's ledger. If the ID is known,
pass it directly; no discovery call or provider setup is needed. If it is missing,
import the exported bundle or supply complete inline Evidence. Do not retry an
unchanged reference or rerun a producer merely to move an existing record.
Export before closing the session when the source must survive it. For CLI JSON
workflows that advertise `--evidence-bundle`, the same reference JSON resolves
against one explicitly selected authenticated bundle:
```bash
rea trace-application-feature request.json --evidence-bundle /absolute/path/evidence.json --json
```
This includes JavaScript traces/comparisons and runtime reconciliation, managed
reconstruction/native verification/graph projection, Android/Apple/Harmony graph
projection, WASM interface comparison and retained analysis views. The managed
member comparison CLI still takes two assembly paths; process comparison takes
two capture Evidence JSON paths. Check command help for the installed version.
An input bundle grants no implicit file lookup: `evidence-file` and historical
graph file selectors are accepted only by contracts advertising them. See the
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/getting-started/cli.md#import-export-and-compare-evidence).
Completed `inspect_artifact` inventory also supports a summary and
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
Omit `logical_paths` to page all available occurrences; `[]` selects none.
@@ -62,25 +24,6 @@ retain its complete source and receive that view immediately. Without that
advertised choice, keep the existing complete call. CLI inspection returns
portable complete Evidence; save it before projecting a selected inline view.
For native procedure/name/string inventories, select `detail: "summary"` when
the connected tool advertises it and the task needs counts or selected rows.
Use its parent Evidence ID with
`view: {kind: "native-inventory", offset: 0, limit: 32, fields: ["value"]}`.
Search tools retain the existing pattern/mode/case/document selection. Omit
`fields` for complete records; `record.address` and `source_index` remain on
selected rows. Selecting `value` keeps its string/decoding qualifications.
Parent Evidence ID plus source index preserves same-address
alternatives. Follow `coverage.next_offset` only as far as the question needs;
exhausting retained rows does not establish complete program discovery.
`batch_decompile` returns every body unless you select `detail: "summary"` when
the connected tool advertises it. Use its parent Evidence ID with
`view: {kind: "batch-decompile", collection: "items"}` for body-free rows and
exact `where.indices`, `addresses`, `procedure_addresses` or `statuses`. Add
`include_pseudocode: true` for several chosen bodies, or use
`collection: "pseudocode"` with one row's `index` to page a large body. Batch
limitations aggregate every item, so do not attribute them to one successful row.
Use `record_unknown` to track unresolved questions in the user's investigation
and name the authority or environment still required. Supply supporting and
contradicting evidence IDs. It updates the session registry directly.
@@ -92,22 +35,16 @@ Use comparisons with complete, compatible page sets when claiming equivalence
or absence:
- `compare_artifacts` compares inventory pages by occurrence path, content,
metadata, and graph relations. Name each side as
`{kind: "retained-evidence", evidence_id}` (an `inspect_artifact` result,
its nested inventory or its summary) or
`{kind: "evidence-file", path, evidence_id?}` instead of resending Evidence.
metadata, and graph relations.
- `compare_functions` compares explicit function Evidence, not fuzzy
whole-binary matches. It rejects an `analyze_function` summary reply; pass
the summary's `parent_evidence_id` as a `retained-evidence` reference or
the complete dossier.
whole-binary matches.
- `compare_bundles` compares canonical bundle membership and unknown history.
- `find_changed_behavior` combines existing comparisons; static differences
remain candidates, not causal proof. Its `comparisons` accept exact references.
remain candidates, not causal proof.
- `build_call_path` needs explicit function dossiers from one artifact/provider;
pass complete parent IDs in `functions`. An incomplete frontier makes absence
unknown.
an incomplete frontier makes absence unknown.
- `correlate_static_and_runtime` uses explicit mappings; matching patterns are
hypotheses, not causality. Both comparison arrays accept exact references.
hypotheses, not causality.
- `verify_reconstruction` evaluates a finite typed specification. A pass covers
only declared comparable claims, not global equivalence.
@@ -132,45 +69,3 @@ When the connected schema advertises `include_query_parameter_names`, select
it in the view for distinct observed names with original record ordinals. Query
values are omitted; the names establish neither required parameters nor a
complete server schema. Keep the original parent Evidence for exact URLs.
Use `inspect_analysis_view` with `view: {kind: "environment-manifest"}` on the
same retained HAR Evidence to project the primary origin's request lines into
named tools with path templates, parameters, observed query names, value-free
body shapes and record pointers for an agent environment. Opt into
`template_rules` only when recorded identifiers need them; every template,
parameter and merged sibling is an inference, no header, cookie or body value is
copied, and the three stated unknowns stay unresolved.
Use `view: {kind: "environment-state"}` with the manifest's `template_rules` to
extract the data those reads observed instead: entity types and identity
fields, entities with recorded values, documents, and per-tool render plans
checked against the recordings. The state comes only from reads before the
first related write; types, identity and list membership are inferences. The
view also gives each state-changing tool a write plan, served only when a later
recorded read confirms its effect and none contradicts it.
`rea environment-serve manifest.json --evidence capture.json` derives that
state at startup, answers read tools from it, serves the write tools whose plan
is served, each applying its effect to that state, and serves `data/reset`,
`data/add` and `data/get` on `/agentenv`; other write tools are not served, and
each plan states why in its `reason`. It replays the capture instead with
`--replay-only` or when the state is unavailable (not derivable, or a catalog
over the discovery budget), and its stopped document's `state` says which
applied.
Use `view: {kind: "environment-coverage"}` with the same rules to see what an
environment built from a capture would serve: read tools answered from the
state or replayed with their reason, and each write tool served or withheld
with a hint about what to record next; it does not apply the serve-time
catalog and state size checks. `rea environment-build <capture.har> <dir>`
inspects, projects and exports in one step and prints the same coverage, or
`null` when the bundle serves replay.
For a PE/ELF export question, `inspect_native_exports` accepts `detail: "summary"`
and retains the complete table. Query `inspect_analysis_view` with
`view: {kind: "native-exports", collection: "exports", names: ["selected_name"],
fields: ["name"], offset: 0, limit: 32}` and the reported parent Evidence ID.
All same-name versions and aliases survive; use `source_index` with the parent ID
for stable identity. `collection: "issues"` pages interpretation failures. Page
coverage describes selected retained rows; `summary.coverage` or `exports.coverage`
still describes the source table, so an empty partial page does not prove absence.
CLI `inspect-native-exports --detail summary --evidence_output PATH` saves the
complete bundle for later `inspect-analysis-view --evidence-bundle PATH` requests.
@@ -6,7 +6,7 @@ For captured or bundled JavaScript that needs readable source recovery, use
`additional_paths` when advertised, then analyze the returned `analysis_input`
directory together. Each input retains its own snapshot, digest and byte-range
provenance; inputs and recovered code are never executed. Recovery requires a
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/guides/javascript/javascript-recovery.md).
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
extracted tree. When the question concerns known source paths and the connected
@@ -17,21 +17,10 @@ the complete artifact inventory and identity but reads source text and analyzes
only selected files. The result and summary record `source_selection`; files
outside scope and their relationships remain unknown, so partial application
coverage does not mean the selected source failed. Use repeated `--source-path`
on either static application CLI route, or `--source-paths-json` with a JSON
file for selections too long for one command line. Omit the selection when the
question requires complete application analysis; `source_paths: []` is invalid,
not an unfiltered request. `detail: "summary"` reduces delivery
on either static application CLI route. Omit the selection when the question
requires complete application analysis. `detail: "summary"` reduces delivery
and does not by itself reduce analysis work.
ASAR header/index admission defaults to 16 MiB / 16777216 bytes;
an admission error names `asar.max_header_bytes` and its bounds (at most
256 MiB). Call `update_analysis_settings` with that setting, `action: "set"`
and enough bytes, then retry the same full-archive request. No inspection or
REA restart is required. `REA_ASAR_MAX_HEADER_BYTES` seeds future REA processes.
`source_paths`, `detail`, and `max_heap_mb` do not bypass or raise header admission.
Runtime JavaScript timeout/heap defaults are also editable; explicit per-call
`analysis_timeout_ms` and `max_heap_mb` take precedence.
Exact `#alias` specifiers resolve through the importer's
enclosing package `imports` map to package-local files; nested `node_modules`
packages use their own map. Wildcard, recursive, and external-package import
@@ -46,9 +35,7 @@ the complete application Evidence inline when it fits.
When the connected schema advertises `max_heap_mb` and `analysis_timeout_ms`,
these select the isolated source analyzer's old-space heap and per-source
deadline. Defaults are 1024 MiB and 300000 ms. They do not change the MCP
process heap or the client's tool deadline. For a host timeout without REA
output, use the [deadline recovery guide](connection-and-recovery.md#distinguish-client-and-server-deadlines)
instead of raising the server budget. A Bun-hosted REA does not enforce the heap choice; on Windows it
process heap. A Bun-hosted REA does not enforce the heap choice; on Windows it
instead limits the analyzer's committed memory to that choice plus a 512 MiB
runtime reserve.
Resource-limited analyses preserve completed static facts and
@@ -64,29 +51,6 @@ in the same session. A returned `error.details.partial_observation` reference
can be supplied directly to `inspect_analysis_view`. If the SDK cancels locally, retrieve its recorded ID from
the Evidence bundle before inspecting the partial analysis.
## Inventory route candidates without returning the application graph
When the advertised schema supports `view`, use `detail: "summary"` with
`view: {kind: "javascript-routes", collection: "routes", fields: ["path", "confidence"], limit: 20}`.
The same view works with `analyze_web_bundle` on already selected captured sources.
The complete source Evidence is retained; use its `parent_evidence_id` with
`inspect_analysis_view` for `summary`, `routes`, `occurrences`, `sources`, or
`unknowns`. Filter and batch IDs before requesting more fields, and follow
`coverage.next_offset`. CLI selections require `--evidence_output` to keep the
complete record. Static patterns, inferred deployment bases, heuristic system
classification, and unavailable sources never establish route reachability or
the complete application route denominator. See the
[route inventory guide](https://github.com/morluto/rea/blob/main/docs/javascript-route-inventory.md).
For captured endpoint observations and inference facts, `analyze_web_bundle`
also accepts `view: {kind: "web-bundle", collection: "endpoints", limit: 32}`
with `detail: "summary"`. Read later `endpoints`, `inferences`, or `summary`
pages from the retained parent. Filter before requesting detail; retain
`source_index` together with parent ID and collection, and request `location`
or `basis` when tracing sources. Page exhaustion does not mean producer
completeness: check `completeness`, structured `source_unknowns`, and limitations.
Endpoint candidates remain inferences rather than confirmed network behavior.
## Read a module without repeating analysis
Use the actual returned parent ID, on the same connection, as `source.evidence_id`.
@@ -169,10 +133,7 @@ leaving it, and `both` when the question needs both. Trace traversal includes
containment relationships, so even a narrow seed can expand many nodes; a
bidirectional trace can reach unrelated sibling modules. A module item or page
is preferable when its observations already answer the question. Do not treat
all reached nodes as participants in the feature. For a broad trace, pass
`view: {"kind": "feature-trace", "collection": "paths"}` (or `summary`) to
retain the complete trace and read terminal paths first; page `nodes` or `edges`
with exact `where.node_ids` through `inspect_analysis_view` as needed.
all reached nodes as participants in the feature.
Reuse retained parent Evidence when the server advertises it, or supply complete
Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest
@@ -234,33 +195,11 @@ accept complete inline Evidence or
`{"kind":"retained-evidence","evidence_id":"ev_<64 lowercase hex characters>"}`
for their application input (`application`, or `left`/`right`). `inspect_analysis_view`
uses the same retained-reference form in `source`, or portable inline Evidence,
to project a summary, module page, or one module with its recorded observations.
This notation is
to project a summary, module page, or one module with its recorded observations. This notation is
a template: replace it with the actual returned ID. Versions before 4.1.0 accept only full
inline Evidence. Use the exact ID
returned by the producer on the same MCP connection. Resolution does not run
analysis or select a provider; findings remain inline. For summary or inspection
replies, select the complete `normalized_result.parent_evidence_id`. Native
observation arrays and runtime reconciliation's `static_layers[].analysis` and
`runtime_observations[]` also accept exact references where advertised. Each
source can independently be inline or retained, preserving identity, coverage
and unknowns. CLI workflows advertising `--evidence-bundle` resolve these inputs
from the explicitly selected bundle. `close_binary` releases the engine while
retaining references and complete Unknown history on the same MCP connection.
`reset_investigation` explicitly clears them; optional `save_path` checkpoints the
complete bundle before clearing. Import that bundle in another connection to
restore its original references. Retention is bounded, in-memory and never silently
evicted; `binary_session.investigation` reports the budget. A missing reference
includes its ID and recovery guidance.
For exact file/native metadata or processing gaps, use
`view: {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"], limit: 32}`.
Rows retain canonical node/observation identity and original Evidence. Paths
match Evidence artifact-path/source-range locations; missing properties remain
omitted and source coverage stays separate from page exhaustion. Batch paths and
select only needed property keys; complete Evidence remains accessible by export.
If names are unknown, opt into `include_available_property_keys: true`; it lists
the original keys per returned observation without unselected values. Keep each
row's node/observation identity paired when composing facts.
These rows select application-graph observations; semantic relationships and
semantic unknown records remain in the complete parent Evidence and trace tools.
analysis or select a provider; findings remain inline. Native observation
arrays still take full Evidence. `close_binary` clears retained references;
export a bundle before closing, or supply portable inline Evidence in another
connection. A missing reference includes its ID and recovery guidance.
@@ -1,118 +1,19 @@
# Native, managed, and packaged artifacts
For release comparisons on one MCP connection, retain `list_procedures` Evidence
from each target and reuse its exact IDs after target switches or `close_binary`.
For a handoff or new process, export the complete bundle and import it there,
including its source closure. `reset_investigation` explicitly clears retained
records; optional `save_path` checkpoints them before clearing.
`match_native_functions` pairs unique exact names and, when supplied,
complete observed body bytes. Reuse optional same-context `list_names`,
`procedure_info`, `read_bytes` and existing `analyze_function` dossiers as
`left_observations`/`right_observations`; an MCP `analyze_function` summary
reply contributes its `parent_evidence_id` as a `retained-evidence` reference,
not the summary itself. Avoid decompiling every function merely to match it. Its summary retains a parent for
`inspect_analysis_view` `function-matches` collections `matches` and `functions`.
Select columns, batch IDs and page individual facts; aliases remain unresolved.
Unpaired observations do not prove additions/removals. Existing matched dossier
pairs include inputs for `compare_functions`; otherwise analyze just chosen pairs.
Aggregate completed comparisons with `find_changed_behavior`, selecting each
comparison's exact `retained-evidence` ID instead of resending full records.
Inline comparison Evidence remains accepted. References belong to the current
server: after handoff, import the original bundle including its source closure.
Do not substitute dossier IDs or summary views. Static differences remain
behavior candidates, with runtime behavior unknown unless separately observed.
For a matching representation refusal, inspect `details.limit` and raise
`native.matching_representation_bytes` with `update_analysis_settings` before
retrying the unchanged MCP request. CLI launch seed:
`REA_NATIVE_MATCHING_REPRESENTATION_BYTES`. The default is 96 MiB; the policy
range is 1 byte to 1 GiB and does not bound process RSS. Keep the complete
inventories: splitting them changes global name/body uniqueness.
## Apple app resources
Use `open_bundle_resources` with the selected app path, then `inspect_plist`,
`decode_interface_builder`, `inspect_asset_catalog`, or `inspect_keyed_archive`.
The resource session reads the app without resolving `CFBundleExecutable` or
starting a deep provider. Preserve `resource_root` provenance, per-resource
hashes where reported, and null executable identity; a directory path is not
an executable digest. Export retained Evidence before closing or switching
roots. Binary analysis snapshots do not apply. Use `open_binary` to switch to
executable analysis, and `close_binary` to release either session kind.
The four matching CLI commands accept the app path directly. For an archive
inside the app, use `inspect-keyed-archive APP_PATH RELATIVE_ARCHIVE_PATH`.
Host requirements for plist and compiled asset/Interface Builder tooling still
apply; a resource-only session does not establish provider availability.
## Desktop installation directories
When an installation directory is supplied and its runtime is unknown, use
`inspect_desktop_application` with its absolute path. Default MCP summary
delivery retains complete Evidence. Read `files`, `omissions`, `candidates`, `components` or
`next_probes` with `inspect_analysis_view` kind `desktop-application`, selecting
exact paths, batched file/omission IDs or families and only needed fields and pages.
Summary `probe_issues` retains counts and bounded previews. Select files with
`has_issues: true` and fields `path`, `read_status`, `issues` to retrieve exact
failures directly; preview text is display data, while `file_id` selects full facts. Preserve source
coverage, read-range identity and unknowns; directory snapshot identity is not
a digest of every file's contents. Component path markers do not establish
runtime presence. For compact CLI output, use
`inspect-desktop-application PATH --detail summary --evidence_output BUNDLE`.
See [desktop installations](https://github.com/morluto/rea/blob/main/docs/desktop-applications.md).
## Go build metadata
Use `inspect_go_binary` for an explicit local ELF, PE or Mach-O (universal
files need `architecture`) when the
Use `inspect_go_binary` for an explicit local ELF, PE or thin Mach-O when the
question concerns the embedded compiler version, module/dependency versions,
local replacements, build settings, or static Go string literals packed without
NUL terminators. Pass `include_string_literals: true`
(`--include-string-literals`, optional `max_string_literals` /
`--max-string-literals`) to recover bounded inferred word-aligned `(ptr, len)`
UTF-8 string slices from read-only data and page them through `go`
`string_literals` views. No `open_binary`, Go installation or
local replacements or build settings. No `open_binary`, Go installation or
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
Preserve artifact SHA-256, original file ranges, framed module bytes and every
unparsed module-text line. A missing record is an unknown, not proof of a
non-Go binary; observed metadata does not authenticate its claims, and static
`(ptr, len)` string recovery excludes compiler-inlined stack/immediate strings
without proving runtime reachability. Function,
non-Go binary; observed metadata does not authenticate its claims. Function,
source and type recovery remain a separate native-analysis question.
## Model files
Use `inspect_model_artifact` (`inspect-model-artifact PATH`) for local GGUF v2/v3
and safetensors declarations without a model runtime. Header-only mode does not
read weights or establish full-file identity. `hash: "full"` explicitly streams
all bytes. Retained `model-artifact` views page `metadata`, `tensors` and derived
`types`, with exact batch filters and selected fields. Arrays retain typed counts
and eight-element previews; omitted values, unknown types and runtime behavior
stay explicit. See [model files](https://github.com/morluto/rea/blob/main/docs/model-artifacts.md).
## IL2CPP metadata
Use `inspect_il2cpp_metadata` for an explicit local Unity `global-metadata.dat`
(versions 24, 27, 29, 31 and 39) when the question concerns assembly images,
names, member owners, string literals, namespaces or index ranges. No `open_binary`, Unity installation or target
execution is required. The CLI equivalent is `inspect-il2cpp-metadata PATH`.
MCP defaults to counts with complete retained Evidence. Query `sections`,
`images`, `types`, `methods`, `fields`, `properties`, `events`, `parameters` or
`string_literals` through `inspect_analysis_view`; use exact names,
batch indices and selected fields. Row indices are scoped by artifact SHA-256
and collection; page offsets count filtered matches. Parameters support batch
`method_indices` to read one method's parameters directly. Historical reports
without member collections retain unknown counts rather than empty inventories. Preserve
artifact SHA-256 and index widths. Unrecognized magic may mean encryption and
is unsupported, not decrypted. Linking names to `GameAssembly` or
`UnityFramework` code addresses remains a separate native-analysis question.
See [IL2CPP metadata](https://github.com/morluto/rea/blob/main/docs/il2cpp-metadata.md).
## Rust dependency metadata
Use `inspect_rust_binary` for an explicit local ELF, PE, or Mach-O (universal
files need `architecture`) when
Use `inspect_rust_binary` for an explicit local ELF, PE, or thin Mach-O when
the question concerns the cargo-auditable crate list embedded in `.dep-v0`
(Mach-O: `__DATA,.dep-v0`). No `open_binary`, Rust installation, or target
execution is required. The CLI equivalent is `inspect-rust-binary PATH`.
@@ -122,189 +23,6 @@ dependencies. An omitted package kind stays null. A rustc commit hash is
inferred from a read-only `/rustc/<40-hex>/` path and does not by itself
identify the compiler. See [Rust binaries](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
## PE Authenticode certificate tables
Use `inspect_pe_certificates` for an explicit local PE32/PE32+ image when the
question concerns the Authenticode certificate table, WIN_CERTIFICATE types, or
embedded PKCS#7 signer and X.509 identity fields. No `open_binary`, Windows SDK,
or target execution is required. The CLI equivalent is
`inspect-pe-certificates PATH`. Preserve artifact SHA-256, table file ranges,
payload hashes, original validity-time spellings, and
`subject_key_identifier_hex` when the signer uses that form. A missing
`IMAGE_DIRECTORY_ENTRY_SECURITY` directory is observed absence of that table,
not proof the image is unsigned in every catalog or overlay sense. Signatures,
chains, timestamps, and Authenticode digest coverage remain unobserved. See
[PE Authenticode certificate tables](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
## PE import and delay-load tables
Use `inspect_pe_imports` for an explicit local PE32/PE32+ image when the
question concerns imported DLL and function names, ordinals, delay-load
descriptors, or the pefile import hash. No `open_binary`, Windows SDK, or
target execution is required. The CLI equivalent is `inspect-pe-imports PATH`.
Preserve artifact SHA-256, directory file ranges, original DLL spellings, and
ILT/IAT RVAs. A missing `IMAGE_DIRECTORY_ENTRY_IMPORT` directory is observed
absence of that table, not proof the image never imported APIs through another
table or overlay. Bound snapshots, loaded IAT contents, and export-forward
resolution remain unobserved. See
[PE import and delay-load tables](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
## Windows Shell Link shortcuts
Use `inspect_windows_shortcut` for an explicit local `.lnk` file when the
question concerns the recorded target path, arguments, working directory, icon
location, volume serial, network share names, ExtraData blocks, or Tracker
droid identifiers. No `open_binary`, Windows SDK, COM, or target execution is
required. The CLI equivalent is `inspect-windows-shortcut PATH`. Preserve
artifact SHA-256, FILETIME raw decimals, IDList type bytes, and ExtraData
signatures. Unknown IDList items and ExtraData signatures remain raw with
SHA-256; they are not guessed. Version-1 UUID MAC and timestamp values are
labelled derived. A missing optional section is observed absence of that
structure. See
[Windows Shell Link shortcuts](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
## PE TLS callbacks and load configuration
Use `inspect_pe_load_config` for an explicit local PE32/PE32+ image when the
question concerns code that runs before the entry point through TLS callbacks,
or the security cookie, SafeSEH, CFG, EH-continuation and XFG declarations in
the load configuration. No `open_binary`, Windows SDK, or target execution is
required. The CLI equivalent is `inspect-pe-load-config PATH`. Preserve artifact
SHA-256, directory and callback-array file ranges, and callback VAs as
hexadecimal strings. Fields outside the declared `Size` are `null`. Guard-flag
mitigations carry `evidence_kind: "inferred"`; enforcement remains unobserved.
See
[PE TLS callbacks and load configuration](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
## Binary hardening
Use `inspect_binary_hardening` for an explicit local ELF, PE, or Mach-O file
when the question concerns PIE or ASLR, NX, RELRO, stack-protector or FORTIFY
signs, control-flow flags, or stripped state. No `open_binary`, pwntools, or
target execution is required. The CLI equivalent is
`inspect-binary-hardening PATH`. Universal Mach-O files return one image per
slice. A missing import is `unknown`, not proof the protection was omitted.
Authenticode and Mach-O code-signature rows report presence, not validity.
`hardened_runtime` stays unknown and names `inspect_signature`. See
[binary hardening](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
## ELF note tables
Use `inspect_elf_notes` for an explicit local ELF32/ELF64 image when the
question concerns PT_NOTE/SHT_NOTE owners, GNU build-id, or GNU ABI-tag
fields. No `open_binary` or target execution is required. The CLI equivalent
is `inspect-elf-notes PATH`. Preserve artifact SHA-256, table file ranges,
descriptor hashes, and original owner spellings. A missing table is observed
absence of those declarations, not proof the image never carried notes in
another container. See
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
## ELF and Mach-O initializer functions
Use `inspect_native_initializers` for an explicit local ELF or Mach-O image
when the question is which functions run before `main` or at exit: ELF
`DT_PREINIT_ARRAY`, `DT_INIT`, `DT_INIT_ARRAY`, `DT_FINI`, `DT_FINI_ARRAY` and
legacy `.ctors`/`.dtors`, or Mach-O `__mod_init_func`, `__init_offsets` and
`__mod_term_func`. Pass `architecture` for a universal Mach-O. The CLI
equivalent is `inspect-native-initializers PATH`. PIE slots are resolved
through RELATIVE, RELR and section relocations, and Mach-O slots through
chained fixups. Keep each entry's `resolution`: `unresolved` means the value was
not established, and `unresolved_import` names an external symbol. Page with
`offset` and `limit`. See
[ELF and Mach-O initializer functions](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
## ELF unwind function ranges
Use `inspect_elf_unwind_ranges` for an explicit local ELF32/ELF64 image when
the question concerns function starts and lengths that survive `strip`, which
functions have an LSDA, or which ranges have no exact symbol. No
`open_binary` or target execution is required. The CLI equivalent is
`inspect-elf-unwind-ranges PATH`. Preserve artifact SHA-256, FDE start and
end, containing section, exact symbol name, CIE index, LSDA, and personality.
Page later ranges with `offset` and `limit`. A missing `.eh_frame` is
observed absence of that declaration and of `PT_GNU_EH_FRAME`, not proof the
image has no functions. Header disagreement is a limitation and does not drop
decoded ranges. `.debug_frame` is reported present or absent and is not
decoded. CFA instructions are counted and not evaluated. Indirect EH pointers
are not dereferenced. See
[ELF unwind function ranges](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
## Apple IPS crash reports
Use `inspect_apple_ips` for an explicit local Apple `.ips` file when the
question concerns crash metadata, exception and termination facts, threads,
usedImages, lossless registers, or documented crash classes. No `open_binary`,
Xcode, atos, or target execution is required. The CLI equivalent is
`inspect-apple-ips PATH`. Preserve artifact SHA-256, `bug_type` as a string,
uppercase image UUIDs, and integer literals that exceed IEEE-754 range. Only
`bug_type` `"309"` yields a crash body; other DiagnosticReports types return
no crash result. Opened Mach-O and dSYM UUID joins remain unobserved. See
[Apple IPS crash reports](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
## Linux kernel modules
Use `inspect_kernel_module` for an explicit local ELF32/ELF64 relocatable
object when the question concerns `.modinfo` license, vermagic, alias,
depends, or parm strings, the `__versions` symbol CRCs, or an appended module
signature trailer. No `open_binary`, Linux host, or `modinfo` program is
required. The CLI equivalent is `inspect-kernel-module PATH`. Preserve
artifact SHA-256, key order, duplicate keys, CRC hex, and section file ranges.
A missing section is observed absence when section names are known, not proof
the module never carried that metadata. `.gnu.linkonce.this_module` is reported
as present or absent; `struct module` stays unknown because its layout changes
with the kernel version. `__ksymtab` records are not decoded. The PKCS#7 range
and its SHA-256 are observed; certificate, signer, and digest coverage remain
unvalidated. An absent trailer is not proof the module is unsigned in another
container. gzip, xz, and zstd whole-file containers are unsupported and name
`external_decompression` as the next probe: use an external codec utility to
decompress into a separate file, then call `inspect_kernel_module` on that file. See
[Linux kernel modules](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
## Linux kernel build config
Use `inspect_kernel_build_config` for an explicit local kernel image (`vmlinux`
ELF, raw arm64 `Image`, x86 `bzImage`/`zImage`, or `configs.ko`) when the
question is which kernel version it is or how it was configured. No
`open_binary`, Linux host, or `extract-ikconfig` is required. The CLI
equivalent is `inspect-kernel-build-config PATH`; pass `name` for one exact
`CONFIG_*` option and `offset`/`limit` to page the rest. Preserve the banner
version, builder, and compiler, the config SHA-256 and option count, and each
entry's value kind (`y`, `m`, `string`, `number`, `not_set`). A config is
accepted only when `IKCFG_ST` is immediately followed by a valid gzip member
and `IKCFG_ED`, so an ordinary `IKCFG_ST` string is not a config. Offsets with
`space: "inflated_payload"` are positions in an inflated bzImage payload, not
file offsets. An absent config describes only the searched bytes; the kernel
may have been built without `CONFIG_IKCONFIG` or with it as a module. Truncated
payloads are incomplete, not absent. A framed blob that does not decode, has a
bad CRC, or holds non-config text is `malformed`, not absent. xz, lz4, lzma, zstd, bzip2, and lzo
payloads are unsupported and name `external_decompression`: decompress with an
external utility, then call `inspect_kernel_build_config` on the result. See
[Linux kernel build config](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
## CPython pyc files
Use `inspect_python_pyc` for an explicit local CPython 3.7–3.13 `.pyc` when the
question concerns the header magic, PEP 552 source hash or timestamp, nested
code-object names, or bytecode identity. No `open_binary`, Python installation,
or target execution is required. The CLI equivalent is `inspect-python-pyc PATH`.
Preserve artifact SHA-256, the 16-byte header fields, nested code names, and
bytecode SHA-256. Bytecode is not decompiled. A decoded header does not prove
the matching `.py` is present. See [CPython pyc inspection](https://github.com/morluto/rea/blob/main/docs/python-pyc.md).
## Erlang BEAM files
Use `inspect_erlang_beam` for an explicit local `FOR1`/`BEAM` file when the
question concerns IFF chunks, atom names, exports, imports, lambdas, or the
Code header. No `open_binary`, Erlang installation, or bytecode execution is
required. The CLI equivalent is `inspect-erlang-beam PATH`. Preserve artifact
SHA-256, chunk file ranges, 1-based atom indices, and original atom spellings.
Missing optional tables stay null. For a few facts, use `detail: "summary"` with
a `beam` view selecting a collection, exact names/indices and fields; read later
pages from `parent_evidence_id` or the complete Evidence bundle. Same-name
exports retain every arity and original atom index. CLI summary delivery requires
`--evidence_output`. Null table counts remain distinct from decoded empty tables. See
[Erlang BEAM files](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
## SQLite snapshots
Use `inspect_sqlite_database` with an explicit local database snapshot for its
@@ -318,7 +36,7 @@ Choose `row_limit` when a finite sample is useful and keep truncation and
unspecified row order explicit. Virtual tables, shadow tables and views are
schema facts, not row-read targets. REA inspects private copies and leaves
the source files untouched; obtain a coherent quiescent database/WAL snapshot
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
## Recorded Linux crashes
@@ -333,7 +51,7 @@ The optional `include_debugger_context` facet adds mapping candidates through
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
establish current file identity; zero reported flags leave permissions unknown.
Requested unavailable context fails with setup guidance. See
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md)
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md)
for exact upstream profiles, bounds and verification coverage.
## Offline ELF layout
@@ -359,30 +77,6 @@ and tool profile. Text offsets count UTF-16 code units; follow `next_offset`.
The complete source remains retained after an oversized MCP reply. No target
execution or network acquisition is implied, and WAT is decoded representation.
## Offline APK signatures
Use `inspect_apk_signature` for a selected local APK/ZIP/AAB. Default MCP summaries
retain complete Evidence. Select `view.collection: signers/digests/lineage`,
with scheme or ID batches, fields and paging; use `inspect_analysis_view` kind
`apk-signature` for retained reads after deletion. IDs retain artifact/scheme
scope and signer links; `fields: []` keeps identity links. Empty pages preserve
partial decoding reasons and unknown authenticity. CLI compact delivery needs
`--evidence_output` to preserve the complete parent.
It reads the APK
Signing Block (v2/v3/v3.1) and v1 `META-INF` signing without executing anything,
and reports each signer's certificate facts (for v1, the certificate the PKCS#7
block's SignerInfo names), the declared v3 rotation lineage,
the declared source-stamp certificate and, for SHA-256/SHA-512 digests, whether
the declared content digest matches the current bytes. That match is not a
signature verification: anyone who changes the bytes can recompute the digest.
`posture` summarises v1-only signing (which cannot tell a v1-only APK from one
whose v2/v3 block was removed), observed signer agreement across schemes,
whether the v1 signer equals the block-scheme signer (`false` is notable),
whether every differing signer appears in a declared (unverified) rotation
lineage, and the strongest scheme.
It never claims authenticity: no signature or lineage is cryptographically
verified.
## Native targets
After `open_binary`, use focused search, procedure, or function tools directly.
@@ -392,24 +86,9 @@ cross-references. Addresses and recovered pseudocode are analysis observations,
not original source. Provider unavailability and unsupported metadata remain
unknown rather than false.
For large pseudocode, request `procedure_pseudo_code` with `detail: "summary"`
on the first MCP call. It retains the complete observation and returns the
original selection, text availability and length without the body. Use
`normalized_result.parent_evidence_id` with `inspect_analysis_view` and
`view: {kind: "native", facet: "pseudocode", offset: 0, limit: 20000}`;
follow `coverage.next_offset` for later pages. Choose a smaller limit if one
page exceeds the client or transport budget. Offsets and limits count UTF-16
code units, not lines or tokens. Pages reuse the same observation without
repeating decompilation. Null and empty text remain distinct. Omitted or
`complete` detail preserves full inline delivery. Summary delivery requires a
retaining session and does not reduce the work or memory of decompilation.
For a completed `analyze_function` dossier, use `inspect_analysis_view` with
`view: {kind: "native", facet: "value_flow_operations", offset: 0, limit: 32}`
and the exact retained Evidence ID to read selected facts without reanalysis.
The default MCP reply is already a summary of that dossier: its
`normalized_result.parent_evidence_id` is the retained ID to page, and its
null counts mark facets the provider did not record or declared unavailable.
Native facets also include procedure, pseudocode, assembly, basic blocks,
comments, callers/callees, references, unresolved calls, referenced strings and
names, the native API record, and value-flow summaries. Pseudocode offsets and limits count UTF-16
@@ -429,164 +108,22 @@ run, use `observe_native_calls` with explicit breakpoints and a bounded
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
runs and may change files or use the network. Hardened-runtime targets need the
`get-task-allow` entitlement.
Function breakpoints use exact LLDB runtime names: pass an `inspect_macho` export
or import `runtime_name` (`main`), not its symbol-table `name` (`_main`). REA does
not translate breakpoint names. Set `module` to the image
file name when selecting that image only; otherwise loaded system images also
match. Inspect retained locations and module paths before attributing a call.
Use `binary_session` with no arguments to check the open target, selected
provider, and alignment. Its default summary `result.tool_availability` carries
the complete tool inventory grouped by current status. Expand each group's
`names` and inherit the inventory-level client requirements unless a group
overrides them; availability, reasons, remediation, and modes remain inline.
Provider candidates keep availability and target-support status, code, and
reason. `omitted_fields` names what the summary leaves out; `{"detail":
"complete"}` adds each unavailable candidate's `remediation`, provider
diagnostics, candidate capability sets, and tool surface, `effects`, and
`annotations`. Those arrays name true flags from fixed
vocabularies, so an omitted known flag is false. `tools/list` remains the
canonical source for full tool schemas when target or provider state changes.
provider, and alignment. Its default `result.tool_availability` carries a
complete inventory grouped by shared facts. Expand each group's `names` and
inherit the inventory-level client requirements unless a group overrides them;
availability, reasons, remediation, modes, and flags remain inline. The
`effects` and `annotations` arrays name true flags from fixed vocabularies, so an
omitted known flag is false. `tools/list` remains the canonical source for full
tool schemas when target or provider state changes.
Provider capability groups carry provider identity and ordered limitations.
Prepend each set's `shared_limitations` to a group's `limitations` to recover the
operation's full ordered list. Candidate capabilities, returned with complete
detail, describe possible providers. Top-level session capabilities describe the current route, and
operation's full ordered list. Candidate capabilities describe possible
providers. Top-level session capabilities describe the current route, and
`analysis_provider_binding` identifies the selected provider.
## Continue editable Ghidra analysis across processes
For a task spanning agent sessions, check `binary_session` after selecting
Ghidra: the current provider's `capabilities[].shared_limitations` states whether
Program checkpoints are enabled or the session is ephemeral. Successful native
queries distinguish initial analysis from an actually restored packed Program;
do not infer that restoration happened merely because checkpoint mode is enabled.
In ephemeral mode, choose a private absolute directory and set
`REA_GHIDRA_CHECKPOINT_DIR` in the environment starting REA. Restart the MCP server
after changing that launch configuration; no current tool call or runtime
analysis-setting update enables it. Stock ELF/PE/Mach-O imports on Linux/macOS
are supported; see the [checkpoint setup and limitations](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program).
With the setting enabled, normal opens restore compatible saved Programs
automatically. Initial completed analysis is saved, and `close_binary` saves
later metadata edits; no separate save/restore tool is needed. Repair a reported
save failure and retry close. Cancellation or process failure can lose edits
after the last successful save; unfinished initial analysis cannot resume.
Keep the same checkpoint configuration for later processes. Evidence/Unknown
history and immutable REA query snapshots have separate persistence workflows.
## Reuse caller-selected Ghidra analysis
When the caller supplies an expanded project, configure `REA_GHIDRA_PROJECT_PATH`
(the `.gpr` marker) and optionally `REA_GHIDRA_PROGRAM_NAME` (one root program),
then restart/reconnect before `open_binary`. CLI uses the same configuration.
REA opens only a fingerprint-bound owned copy with `-process -noanalysis -readOnly`;
original project files are not opened by the engine or saved. Database edits,
import overrides and repair scripts are unavailable in this mode. Treat original
analysis completeness/settings as unknown and prior edits as caller-project state.
The program's recorded import digest must match the target; changed project content
requires reopening for a fresh profile. See the
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#reuse-an-existing-ghidra-project).
## Ghidra startup deadline recovery
`provider_timeout` during the first Ghidra query can cover import, auto-analysis,
bridge startup, and readiness. The default 330,000 ms is configurable through
`REA_GHIDRA_STARTUP_TIMEOUT_MS`; a smaller query still needs the same cold import.
When the error names `ghidra.startup_timeout_ms`, call
`update_analysis_settings` with `action: "set"` and a larger value within the
reported bounds. Resolve retained cleanup before the next startup attempt.
No REA restart is needed; an already accepted startup keeps its original
budget. Follow the error's session-specific restart guidance, and preserve useful Evidence before closing
and reopening the target when remediation requires that lifecycle step.
For a per-function timeout, update `ghidra.decompile_timeout_seconds` and retry
the same function on the existing Program. Explicit `null` removes that
per-function deadline. Inspection is optional when the error supplies the
setting and bounds. Runtime updates disappear when REA exits; environment
changes only seed future REA processes.
The client's tool-call deadline must allow startup plus the requested query;
server settings and progress notifications cannot override it. CLI analysis
avoids the MCP client deadline, but retains the provider deadline. For example:
```sh
REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000 rea analyze /absolute/path/to/app.exe --provider ghidra --json
```
Thirty minutes is an example budget, not a completion guarantee. Keep host CPU
and memory controls in place. Disable specific expensive analyzers only when
their missing coverage is acceptable and retain the resulting limitations.
Exported C and embedded resources remain useful independent evidence, but a
timeout does not establish complete analysis or absence of code. See the
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
## Bridge request ceiling
Hopper and Cutter share `REA_BRIDGE_MAX_REQUEST_BYTES`. Unset, both sides use
16777216 bytes. The payload budget excludes the terminating LF. A decimal
integer from 7464 to 536870887 raises or lowers that ceiling. The floor fits
the largest bounded symbol-name command. A tight Cutter status still returns
its session id and up to 64 completion receipts. Its path is included when it
fits, and is unknown when it does not, so an older descriptor path is not
paired with the new generation. The maximum is the encoded payload, one byte under Node's string ceiling so
the framing LF still fits. Cutter measures, signs, and writes a fitting
response in bounded chunks. The bridge searches only the newly read bytes
for the framing LF. The client unescapes each socket chunk into one response
string before parsing it, so this ceiling does not keep the paged wire frame
and another concatenated copy. Invalid values are configuration errors and do not fall back to 16777216. Set the variable in the environment that starts REA, restart REA,
and start Hopper or Cutter from that same environment. An out-of-band bridge
without the setting stays at 16777216 bytes. The client reports that mismatch
during discovery only after the hello proves a live authenticated bridge, and
does not send a status request or the command. A stale descriptor is ignored.
Discovery fails if any such live ceiling differs, even when another live
session matches. A command response discarded for that ceiling stays an
unknown execution with `output_truncated`. A hello
that never sends its authenticated frame does not hold the accept loop.
At most 8 connection threads run; a further accept is closed, and a failure
to start a thread does not stop the listener. `completed_connection` is the
longest prefix of connections that have terminated. Dense scalar replies are
encoded in bounded batches.
The 25000 ms Cutter request deadline and the 256-candidate discovery budget
still have no runtime override.
## Selected native facts and call graphs
For a large `get_call_graph`, or repeated `procedure_address`,
`procedure_callees`, `procedure_callers` and `resolve_containing_procedure`
queries, select `detail: "summary"` when the first 64 recorded observations
suffice. Read further `native-observation` pages from the returned parent Evidence
ID. Graph message IDs index that page's exact-text dictionary; keep level,
source index, failed nodes, and partial/unknown relationship coverage.
Read referenced source context through `native-context` facets `overview`,
`analysis-profile`, `limitations` and `result-limitations`. Reuse inspected
context/profile values only for the same digest; until caveats are read,
interpretation beyond the returned facts remains unknown. Complete Evidence
remains retained and exportable. Small complete replies can cost fewer tokens
than additional context/page calls; count the completed task and discovery.
## Static memory and original file bytes
For Ghidra, `read_bytes` reads initialized analysis memory, including loader
fixups; it does not read a running process. Check `returned_bytes` and `complete`
before decoding a table. An incomplete read is missing coverage, not a zero-filled
tail. Use `list_segments` to inspect the selected block's initialization and address-space
facts, and retain unavailable facts as unknown.
Use `address_to_file_offset` to anchor one observed address to original file
bytes. A mapping for the first address does not establish a contiguous mapping
for an entire table, and loader fixups can make analysis bytes differ from source
bytes. Keep source ranges and analysis-memory observations separate.
A pointer table can refer to writable slots populated by a constructor rather
than directly to immutable resources. An uninitialized block or an observed zero
does not establish the slot's later runtime value. Follow `xrefs` and the relevant
writer's assembly or pseudocode; distinguish recorded assignments from inferred
execution order. Do not replace unreadable bytes with zeros or conclude that a
resource is absent without the necessary initialization evidence.
## DOS MZ with Ghidra
For an admitted DOS MZ target, use `inspect_native_load_image` to check measured
@@ -594,18 +131,12 @@ loaded bytes, file mappings, relocations and the entry against the immutable
snapshot. Keep `mismatch` and `unsupported` explicit. Verified import covers the
reported static image, not runtime DOS or PC-98 hardware behavior.
For a large load-image report, select `detail: "summary"` and query its
`parent_evidence_id` with `inspect_analysis_view` kind `native-load-image`.
Choose a collection such as `relocations` or `mappings`, select only needed
fields, and page with offset/limit. Original indices preserve same-address
alternatives. Null collections remain unrecorded or inapplicable; paging does
not upgrade unsupported verification. CLI requires `--evidence_output` for
summary delivery so complete source Evidence remains accessible.
For Ghidra decompilation, `procedure_pseudo_code` also accepts `procedure: "file-offset:0x..."`; CLI `rea decompile <path> --file-offset=0x... --provider ghidra` resolves the source byte offset internally. An unresolved 0x-prefixed numeric procedure selector also falls back to a unique file mapping. Existing symbols and analysis addresses win; multiple mappings fail as ambiguous. Preserve the returned file-offset and address locations and raw selection observations. ELF virtual addresses are distinct from file offsets; never assume an arbitrary offset plus image base is an analysis address.
Complete function body ranges use inclusive ends and can contain gaps. Do not
treat the enclosing span as code.
Use `read_bytes` for initialized analysis memory and `address_to_file_offset` to
anchor one observed address to original file bytes. Loader fixups can change a
word; a source offset does not imply byte equality. Complete function body ranges
use inclusive ends and can contain gaps. Do not treat the enclosing span as code.
For packed targets, retain the original and separately derived artifact identities;
decompiling an unpacking stub does not recover the unpacked program.
@@ -645,11 +176,6 @@ names), coverage and per-collection counts. Page rows with
`view: {kind: "managed-members", collection: "call_edges", offset: 0, limit: 64}`
over `types`, `exported_types`, `fields`, `methods`, `member_refs`, `call_edges`,
`field_accesses` or `user_strings`.
Use exact `names` and `declaring_types` filters before paging supported
collections. For a compact method lookup, request `fields: ["name",
"declaring_type", "signature"]`; omit `body` to avoid anchors. Identity tokens,
original `source_index`, module/MVID and source coverage remain. Check
`unresolved_filter_rows` before interpreting an empty match as absence.
### .NET NativeAOT
@@ -672,7 +198,7 @@ broader metadata recovery on Linux; it requires an explicitly configured
NativeAOT JAR. These are separate routes. The built-in parser also runs on
Windows Ghidra P0; read frozen literals from its report, because
`search_strings` does not include them. See the
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-nativeaot.md)
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/ghidra-nativeaot.md)
for setup, supported layouts, and host restrictions. Preserve binary identity
and distinguish derived metadata from Ghidra database observations. Recovered
types and native pseudocode are not original C# source. Windows Ghidra P0 has
@@ -700,77 +226,37 @@ every executable in the bundle. It also lists missing or weak loads and earlier
`architecture`. System paths stay undetermined because the dyld shared cache
provides them.
Use `extract_artifact` when materialized files from a supported format are needed.
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md).
Use `extract_artifact` when materialized files are needed. It extracts the active
target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md).
It materializes all regular files into a fresh temporary directory chosen by
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
The result reports the materialized directory; callers do not choose the
destination. No separate permission grant is needed.
For classic iNES cartridges, `open_binary` recognizes the byte signature without
an executable architecture. `inspect_artifact` retains header facts and each
PRG/CHR file unit's offset, length and SHA-256; its nested inventory's
`ines.reconstruction` verifies the actual ordered region bytes against the
whole-file digest. Extraction includes the original header, optional trainer
and uninterpreted trailing bytes. These file units are not mapper/CPU bank
maps. NES 2.0, UNIF, dirty legacy headers and truncated payloads are explicitly
refused. No emulator or 6502 analysis is performed. See the
[iNES guide](https://github.com/morluto/rea/blob/main/docs/ines-containers.md).
Native DMG traversal automatically uses a read-only mount on supported macOS
hosts, with an owned temporary mount directory and cleanup. It needs no approval
flag. Unsupported hosts or mount failures remain explicit limitations; do not
claim child inventory when only root identity is available. DMG extraction is
unavailable through `extract_artifact`. To access DMG files on macOS, create a
read-only mount with your own tools, select files from that mount for analysis,
and detach the mount you created when finished. REA's inspection mount is
temporary, not a retained export of the image's contents.
claim child inventory when only root identity is available. Extraction is a
separate filesystem-writing operation.
For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
`--target-format dos-com`). Inspect the load image before trusting function
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-dos.md).
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
For other headerless images (embedded firmware without ELF/Mach-O/PE/MZ
headers), open with `format: "raw-binary"`, `architecture` (`x86`, `x86_64`,
`arm`, `arm64`, `6502`, or `m68k`), and `entry_point` (`0x` hex). File offset 0 is mapped at
`arm`, `arm64`, or `6502`), and `entry_point` (`0x` hex). File offset 0 is mapped at
that address. Hopper and IDA refuse this interpretation; Ghidra uses
`BinaryLoader` with a generic language: big-endian for `m68k`, little-endian
otherwise. Big-endian ARM, Thumb-only, and Cortex-M variants are unsupported.
`6502` selects NMOS 6502; the entry and complete file mapping must fit within
`0x0000..0xffff`. 65C02, 65816, cartridge containers, bank switching, and mapper
semantics are unsupported. `m68k` requires an even entry and decodes the 68040
superset of the 68000 family; Amiga/Atari containers and vector-table entry
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/providers/raw-binary.md).
Intel HEX, S-record, UF2 and DfuSe files are containers, not raw images:
`inspect_artifact` lists address regions (UF2 per familyID, DfuSe per element
with its alternate setting), and `extract_artifact` writes each region. Open an
extracted region as `raw-binary` with its start address as `entry_point`. UF2
numbering gaps leave coverage unknown; a DfuSe suffix CRC mismatch follows
`integrity_policy`. See [UF2 and DfuSe](https://github.com/morluto/rea/blob/main/docs/uf2-dfuse.md).
`BinaryLoader` with a little-endian generic language. Big-endian, Thumb-only,
and Cortex-M variants are unsupported. `6502` selects NMOS 6502; the entry and
complete file mapping must fit within `0x0000..0xffff`. 65C02, 65816, cartridge
containers, bank switching, and mapper semantics are unsupported. Windows
Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
### Function annotations in Ghidra
Fresh-import sessions admit annotations; existing-project reuse is read-only.
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits. Normal close saves them when owned Program checkpoint mode is enabled; ephemeral mode discards them. CLI `annotate-native-function` follows the same setting when closing after its updated result. Select the [checkpoint workflow](#continue-editable-ghidra-analysis-across-processes) before edits that must survive a later process. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier. Ghidra rejects batch addresses that request the same name or a name another selected address holds; `raw_result.failures` gives each failure's reason.
A provider switch or different target can retire the active Ghidra database and discard
its edits, even if the new provider fails on a later call. Same-target,
same-profile reopen keeps the active database. Retained annotation Evidence
records what happened; it does not restore the edits.
For `compare_functions`, obtain `analyze_function` Evidence before and after the
edit while the MCP session is open; annotation Evidence is not a comparison
input. Export the needed Evidence before close if comparison will happen later.
Authenticated same-artifact coordinates preserve renamed entries as
`matched / same-entry`, with the canonical `entry_address` inline; name and
exact-text changes remain separate dimensions. This requires compatible
architecture, provider-owned mapping and selected-image commitments. Equal
numeric addresses alone cannot establish correspondence; inspect retained
source Evidence and coverage when the match remains ambiguous.
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier.
## Offline PE resource inventory
@@ -782,150 +268,10 @@ code pages, payload hashes and static icon-group references. Same-language icon
resolution is explicit; missing references and size mismatch remain visible.
Other payloads are opaque, and this operation does not decode images or simulate
Windows resource loading. Consult the budgets and supported tree profile in
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
## Offline PE and ELF export tables
Use `inspect_native_exports` with an absolute `path`, or
`rea inspect-native-exports ./libexample.so --json`, to read a PE export
directory or ELF dynamic symbol table on any host without opening a native
target. Ordinal-only, forwarded and hidden-version exports stay distinct;
partial or unsupported coverage means the inventory is incomplete, not that
exports are absent. Use `inspect_macho` for Mach-O. See
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
For exact core-module import/export differences, use `compare_wasm_interfaces`
with two complete inspection Evidence records (or current-session retained
references) and explicit scope. Preserve unknown types/duplicate names; static
interface equality is not runtime compatibility. CLI accepts complete inline
Evidence in `rea compare-wasm-interfaces input.json --json`, or exact references
with `--evidence-bundle /absolute/path/evidence.json`. Use the complete parent
ID when the inspection returned a summary.
## Godot PCK
For a standalone PCK, `open_binary` then `inspect_artifact` with
`detail: "summary"` retains the complete static inventory. Use the returned
`parent_evidence_id` with `inspect_analysis_view` inventory occurrences filtered
by exact `logical_paths` or `occurrence_ids`, and join `artifact_id` to nodes
filtered by `artifact_ids`. Select only needed fields; identity links remain.
Prefer `inspect-artifact --detail summary --evidence_output PATH` for CLI tasks;
the same CLI discovers embedded packs via their EOF trailer without opening a
native analysis provider. Normal native opening remains unchanged.
Distinguish declared MD5 from observed SHA-256 and graph/page completeness from
`source_coverage`. Encrypted directories and sparse bundles have unknown entry
coverage; encrypted members have unknown content identity. An empty filtered
page cannot prove absence in unknown coverage. Retained views are historical,
not fresh filesystem checks. No decryption, Godot resource decoding, GDScript
decompilation or target execution is provided. See
[Godot PCK](https://github.com/morluto/rea/blob/main/docs/godot-pck.md).
UnityFS AssetBundle containers (`.bundle`, `.ab`, YooAsset packages and other
byte-identified files) list `CAB-*` SerializedFiles and `.resS` members with
header and node facts; none/LZ4/LZ4HC member blocks stream on demand. Bundles
declaring asset-bundle encryption keep header facts with unknown directory
coverage; LZMA member bytes retain named occurrences with unavailable hashes.
Virtual member offsets are not physical carrier locations. Format 6–8 readers
cache one decoded block and support retained filtered/paged views. No
decryption or SerializedFile object decoding is provided. See
[UnityFS](https://github.com/morluto/rea/blob/main/docs/unityfs.md).
WeChat mini-program packages (`.wxapkg` files with `0xBE`/`0xED` marks) list
members with declared names and absolute carrier offsets; member bytes are
stored verbatim. `V1MMWX` PC-client packages report `encrypted` directory
status with unknown coverage and are refused on extraction; no decryption is
attempted. See
[wxapkg](https://github.com/morluto/rea/blob/main/docs/wxapkg.md).
U-Boot legacy uImages use the same inspection/extraction workflows. Prefer a
retained summary and selected inventory views for header facts or component
IDs. Header addresses/CPU/OS are declarations; payloads remain as stored.
Multi-file components retain their aligned physical ranges; the full payload
is also materialized. Unknown name text retains its original 32 bytes. See
[U-Boot uImage](https://github.com/morluto/rea/blob/main/docs/uimage.md).
## ESP images
For an ESP32-family app or bootloader image, `open_binary` then
`inspect_artifact` with `detail: "summary"`. `root_format_metadata` holds the
observed `header` and `descriptor` (its `kind` is the app or bootloader role)
and the derived `chip`, `checksum`, `appended_sha256` and
`undescribed_trailing` ranges. Segment rows (`segment-NN.bin`) carry
`load_addr`, `classification` (`mapped`, `loaded`, `not-loaded` or `unknown`,
per ESP-IDF `should_map`/`should_load`; RAM ranges are not checked) and
`source_location`; select them with `inspect_analysis_view` and
`logical_paths`. Signature blocks are `unverified`; `public_key_sha256`
matches eFuse key digests.
Unknown `source_coverage` means truncation or undescribed trailing bytes, such
as the flash segments of a `--ram-only-header` image; do not infer absence
from it. Erased flash is described. Integrity mismatches are facts, not
refusals. ESP8266 images are refused, and flash-encrypted images are not
recognized. Segments are not disassembled, and RISC-V or Xtensa segments
cannot be opened as raw-binary. See
[ESP images](https://github.com/morluto/rea/blob/main/docs/esp-image.md).
## ESP flash images
A whole-flash dump opens as `esp-flash-image`. `root_format_metadata` holds
`partition_table` (its `method`: `default`, `caller` or a derived
`scan`), `bootloader` and `apps` ESP image facts, the `otadata` records,
`partitions_outside_dump` and `boot_selection`. Every table has a
`boot_selection`: its `basis` names the bootloader's branch (`no-otadata`
without otadata, `none` when the bootloader rejects the table, `unknown`
when a record is missing), and it lists its `assumptions`;
`running_partition` is always `unknown`. Each image names its entry
`path`. With `detail: "summary"` the root kind is `esp-flash-image-summary`
and each image is a compact `summary` (chip, entry point, segment count,
classified load ranges, signature block count, descriptor identity,
integrity); complete detail keeps every segment. Partition rows
(`partitions/<label>.bin`) carry the declared `type`, `subtype`, names,
`flash_address`, `size` and flags; `encrypted` is a storage flag, not
proof of ciphertext. `written_offset`/`written_end` bound the bytes that
are not erased: an app slot whose facts say `erased` can still hold an older
image, and a limitation says so. For a dump read from another flash address, call
`open_binary` with `format: "esp-flash-image"` and `flash_offset`; an
`esp-image` whose `flash_dump` is set names that offset in a next probe; a
relocated table can be named with `partition_table_offset`. No table found
means unknown, often flash encryption. Extraction writes `nvs_keys` as
ordinary data. See
[ESP flash images](https://github.com/morluto/rea/blob/main/docs/esp-flash-image.md).
## Bun standalone executables
A `bun build --compile` executable stores its module graph in Mach-O
`__BUN,__bun`, a PE or ELF `.bun` section, or an appended trailer.
`open_binary` then `inspect_artifact` reports root format `bun-standalone` with
the carrier, structural layout, module count, entry point, raw flags and
compile exec-argv; each module keeps its `/$bunfs/...` source name, loader,
encoding, sizes and SHA-256, with sourcemap, bytecode and module-info regions as
separate children. Use summary root metadata `entry_point_logical_path` as the
exact inventory `logical_paths` selector, then read the selected content node by
its artifact ID; the reader owns virtual-name normalization and indexed fallback. `extract-artifact` writes stored contents through the safe
output tree; route extracted JavaScript to `analyze_javascript_application`.
Layout labels describe record shapes, not a detected compiler version; unknown
or ambiguous layouts are unsupported. Bytecode is retained, not decoded, and the
executable is never run. See
[Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
## Deno compile executables
A `deno compile` executable stores its application in a Mach-O `__SUI,d3n0l4nd`
section, an Intel Mach-O `<~sui-data~>` trailer, an ELF `SUI` note of type
`0x53554901`, or a PE `RT_RCDATA` resource named `D3N0L4ND`.
`open_binary` then `inspect_artifact` reports root format `deno-standalone`
with the carrier, entry point, recorded options, and one occurrence per virtual
file, including size and SHA-256. Transpiled output, CJS export analysis, and
source maps are separate occurrences with the file's `index` and their own
`role`. They normally use a `.deno-*` suffix on the file path and move to an
indexed path on a collision, so use the reported `logical_path`. Symlink targets
stay recorded path parts. A virtual filesystem that cannot be listed leaves
coverage unknown.
Environment-file values and CA certificate bytes are retained in local Evidence
with their counts and byte lengths. The npm snapshot length is reported and not
decoded. `extract-artifact` writes
stored files through the safe output tree; route extracted JavaScript to
`analyze_javascript_application`. Other layouts are unsupported, and the
executable is never run. See
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
Evidence in `rea compare-wasm-interfaces input.json --json`.
@@ -1,58 +1,4 @@
# Runtime observation
## Native runtime questions
Start from the requested observation. State the selected target, input, expected
result, failure signal and stopping condition before a run. Reuse static Evidence
to choose a function or hypothesis; `trace_native_values` follows recovered
static value dependencies and does not measure runtime values. Prefer static
inspection or existing capture comparison when it already answers the question.
| Question | Existing workflow | Host and effects |
| -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Which selected native entry is reached, with which integer argument registers? | `observe_native_calls` on the active Mach-O, with exact function/Objective-C breakpoints and a duration/event bound | macOS with LLDB and required debugger access. Launches a new owned process, stops at entries, records and continues; target execution may write files, show UI or use the network. It never attaches to an existing process. Inspect termination and partial coverage after the run. |
| What does caller-authored instrumentation observe? | `list_frida_devices` / `list_frida_processes` when discovery is needed; `start_frida_session`, `load_frida_script`, then status/unload/close | Requires compatible installed Frida bindings and selected device access; the real REA verifier covers Linux. Attach selects an existing PID; spawn selects a program and starts it paused until `resume_frida_session`. Scripts execute in the target and can change memory, files, network activity and behavior. This is not passive inspection. |
| Which debugger state or controlled execution explains the behavior? | `start_gdb_session` and `gdb_console`, or `start_rizin_debug_session` and `rizin_debug_command` | Persistent debugger workflows are Linux-only. GDB starts a debugger; console commands choose/load/attach/run targets. Rizin debug start launches the selected target and requires a durable `inferior_tty`. Both consoles are unrestricted and can change target memory/registers, run shell commands, write files and use the network. |
| How do terminal output, process lifetime and selected files differ between runs? | `capture_process_scenario`, then `compare_process_captures` with compatible recorded inputs | New captures require Linux or macOS and a working native PTY backend; macOS also needs `xcrun`/Swift for ownership inspection. Capture launches an owned command with ordinary user permissions. Observation paths select snapshots, not a filesystem sandbox. Comparing retained captures does not execute targets and is also available on Windows. |
| What does an existing desktop window show, or what changes after a declared UI action? | `observe_native_ui`; use `capture_native_ui_scenario` only for requested actions | macOS 14+ with required Screen Recording/Accessibility access. Select an existing PID/window. Observation does not launch or own the app; scenario actions can change its data or trigger network activity, and leave its state as-is. |
Close each persistent session with its matching tool. `close_frida_session`
unloads scripts and detaches; it does not kill the target. A still-paused spawn
is resumed only after instrumentation cleanup is confirmed. GDB and Rizin close
can detach a live inferior and let it continue only after verifying its durable
output terminal. When disposition or cleanup is unresolved, preserve the owned
session and reported recovery; do not assume that closing the debugger terminated
the target. Process capture and LLDB call observation instead own their launched
processes and report their termination/cleanup outcome. Cancellation is not proof
that cleanup completed.
Unrestricted GDB/Rizin commands and relevant start/close operations retire an
active binary-analysis session in the same MCP server. Close an active Ghidra
session explicitly first; after target effects settle, reopen the artifact before
new static analysis. A Cutter bridge command is likewise unrestricted and may
affect its GUI/target; it is not a passive native observation substitute.
To test a static hypothesis across two builds, retain compatible static
comparison Evidence and `compare_process_captures` Evidence. Supply explicit
finding/dimension mappings to `correlate_static_and_runtime`. It does not ingest
arbitrary Frida, LLDB or debugger traces, auto-match names, or prove causality
from cochange. Preserve those traces as their own Evidence and leave unobserved
behavior unknown. Record truncation, unmatched breakpoints, changed target
identity and cleanup uncertainty as limitations, rather than treating no retained
event as proof that the behavior never occurs.
For same-session MCP comparisons, pass each capture inline or reuse it as
`{"kind":"retained-evidence","evidence_id":"ev_..."}`. Default comparison
keeps capture-local timing as provenance without treating scheduler or observer
latency as a behavior change; use an exact `trace_spec` event when timing is part
of the analyst question.
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/guides/runtime/frida-instrumentation.md),
[GDB/Rizin/Cutter](https://github.com/morluto/rea/blob/main/docs/reverse-engineering-tools.md),
[process capture](https://github.com/morluto/rea/blob/main/docs/guides/runtime/process-capture.md),
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/guides/native/native-investigation.md#native-desktop-observation).
## Browser and Electron
# Browser and Electron runtime observation
Supply the existing browser or Electron instance's literal-port loopback CDP
endpoint in the request. No separate permission configuration or per-call
@@ -92,8 +38,6 @@ Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
browser and profile. Check `browser.cleanup` for the outcome.
Request only the DOM, accessibility, or network details needed for the question.
The default final capture contains the URL; step outcomes are reported separately.
For multi-step flows with trees or events, select `detail: "summary"` and page
exact steps, transactions, text slices or events with `inspect_analysis_view` kind `browser-scenario`.
Add a wait action for the expected page state before capturing it. Report that
REA performed the declared actions.
@@ -125,16 +69,10 @@ with `inspect_analysis_view` and `view: {kind: "capture", detail: "preview", off
Select complete record pages only for needed headers, metadata or payloads; views
read the historical parent without repeating capture decoding.
For selected protobuf/gRPC bodies, keep raw bytes as the source and use
`inspect_analysis_view` kind `protobuf`, collection `bodies`, `frames` or
`fields`. Batch exact record ordinals and field numbers/IDs, and select
`schema_value` or `text` only when needed. Supply a local descriptor set and
message type for schema interpretation; compressed payloads remain unknown.
Inspect native `.saz` snapshots with `inspect_web_network_capture` and
`format: "saz"`; encrypted archives are unsupported. For live
traffic analysis, connect Everywhere's separate MCP server and use its
advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or
`AgentCalls`). Preserve existing sessions and filters when inspecting a
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md)
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md)
for the workflow and Telerik's public setup and traffic-debugging skills.
-21
View File
@@ -1,21 +0,0 @@
{
"$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
"name": "rea",
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"owner": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"plugins": [
{
"name": "rea",
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"source": {
"source": "npm",
"package": "rea-agents",
"version": "6.8.0"
},
"category": "development"
}
]
}
-29
View File
@@ -1,29 +0,0 @@
{
"name": "rea",
"version": "6.8.0",
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"author": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"homepage": "https://rea.tools/",
"repository": "https://github.com/morluto/rea",
"license": "MIT",
"keywords": [
"reverse-engineering",
"binary-analysis",
"decompiler",
"mcp-server"
],
"skills": "./skills",
"mcpServers": {
"rea": {
"command": "node",
"args": [
"${CLAUDE_PLUGIN_ROOT}/scripts/plugin-mcp-launcher.mjs",
"6.8.0"
]
}
},
"icon": "./assets/rea-plugin.png"
}
-13
View File
@@ -1,13 +0,0 @@
{
"mcpServers": {
"rea": {
"command": "node",
"args": [
"./scripts/plugin-mcp-launcher.mjs",
"6.8.0"
],
"cwd": ".",
"startup_timeout_sec": 30
}
}
}
-40
View File
@@ -1,40 +0,0 @@
{
"name": "rea",
"version": "6.8.0",
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"author": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"homepage": "https://rea.tools/",
"repository": "https://github.com/morluto/rea",
"license": "MIT",
"keywords": [
"reverse-engineering",
"binary-analysis",
"decompiler",
"mcp-server"
],
"skills": "./skills",
"mcpServers": "./.codex-plugin/mcp.json",
"interface": {
"displayName": "REA",
"shortDescription": "Reverse engineer anything",
"longDescription": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"developerName": "morluto",
"category": "Developer Tools",
"capabilities": [
"Interactive",
"Read",
"Write"
],
"websiteURL": "https://rea.tools/",
"defaultPrompt": [
"Use REA to explain how this binary implements a feature.",
"Use REA to compare two versions of this application.",
"Use REA to recover the sources from this JavaScript bundle."
],
"composerIcon": "./assets/rea-plugin.png",
"logo": "./assets/rea-plugin.png"
}
}
-12
View File
@@ -1,12 +0,0 @@
# Review changes to security-sensitive automation and dependency inputs.
/.github/CODEOWNERS @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/.github/dependabot.yml @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/.github/workflows/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/.github/actions/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/scripts/verify-tools/skills/package.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/scripts/verify-tools/skills/package-lock.json @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/scripts/fixtures/http3-lab-requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/website/requirements.txt @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
/third_party/ @morluto @kaoru0822-kitauji @N0zoM1z0 @gensokyo-dev
+1 -1
View File
@@ -27,7 +27,7 @@ body:
attributes:
label: Documentation location
description: Link the page, section, command, API, skill, or inline comment.
placeholder: README.md#quick-start or docs/reference/mcp-contracts.md
placeholder: README.md#quick-start or docs/mcp-contracts.md
validations:
required: true
+3 -13
View File
@@ -1,10 +1,5 @@
name: Cache npm package downloads
description: Restore npm's download cache. Trusted default-branch jobs that opt in may save it.
inputs:
save:
description: Save the shared download cache when this event is allowed to write it.
required: false
default: "false"
description: Restore npm's download cache and save it only from trusted default-branch events.
outputs:
can-save:
description: Whether this run may update the shared npm download cache.
@@ -28,25 +23,20 @@ runs:
can_save=true
fi
cache_key="npm-download-${RUNNER_OS}-${runtime_arch}-${LOCKFILES_HASH}"
# Same platform only. A lockfile change can reuse tarballs from the previous key.
restore_key="npm-download-${RUNNER_OS}-${runtime_arch}-"
{
printf 'can-save=%s\n' "${can_save}"
printf 'path=%s\n' "${cache_path}"
printf 'key=%s\n' "${cache_key}"
printf 'restore-key=%s\n' "${restore_key}"
} >> "${GITHUB_OUTPUT}"
- name: Restore and save npm package downloads
if: steps.context.outputs.can-save == 'true' && inputs.save == 'true'
if: steps.context.outputs.can-save == 'true'
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ${{ steps.context.outputs.path }}
key: ${{ steps.context.outputs.key }}
restore-keys: ${{ steps.context.outputs.restore-key }}
- name: Restore npm package downloads without saving
if: steps.context.outputs.can-save != 'true' || inputs.save != 'true'
if: steps.context.outputs.can-save != 'true'
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ${{ steps.context.outputs.path }}
key: ${{ steps.context.outputs.key }}
restore-keys: ${{ steps.context.outputs.restore-key }}
@@ -24,7 +24,19 @@ runs:
BUILD_CACHE_PREFIX: rea-portable-build-v2-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.npmrc', '.nvmrc', 'package-lock.json') }}
run: |
npm run --silent test:prepare -- --dry=json |
node scripts/ci/portable-build-cache.mjs key >> "$GITHUB_OUTPUT"
node --input-type=module -e '
import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";
const { tasks } = JSON.parse(readFileSync(0, "utf8"));
const hashes = tasks.map(({ taskId, hash }) => {
if (typeof hash !== "string")
throw new Error(`Build task ${taskId} must have an eager Turbo hash`);
return `${taskId}:${hash}`;
}).sort();
const hash = createHash("sha256").update(hashes.join("\n")).digest("hex");
console.log(`key=${process.env.BUILD_CACHE_PREFIX}-${hash}`);
console.log(`prefix=${process.env.BUILD_CACHE_PREFIX}-`);
' >> "$GITHUB_OUTPUT"
- name: Restore portable build outputs
if: inputs.build_cache_artifact == '' && inputs.save != 'true'
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
-21
View File
@@ -1,21 +0,0 @@
name: REA production code
# Match the existing default-setup scan's local-input threat model.
threat-models: [local]
# Retain root traversal, including future production roots. Hidden directories
# need their own root; use a directory so Python can share this configuration.
paths:
- .
- docs/.vitepress
# Keep the default query suite and all production roots, including bridge/,
# install.sh, runtime scripts, release controllers, and GitHub Actions.
# Exclude test inputs at extraction time so they cannot taint production callers.
paths-ignore:
- tests
- scripts/fixtures
- "**/*.test.*"
- "**/*.fixture.*"
- "**/test_*.py"
- "**/*_test.py"
-33
View File
@@ -1,33 +0,0 @@
version: 2
updates:
- package-ecosystem: npm
directories:
- /
- /scripts/verify-tools/skills
schedule:
interval: weekly
cooldown:
default-days: 7
open-pull-requests-limit: 5
groups:
npm-minor-and-patch:
update-types:
- minor
- patch
- package-ecosystem: pip
directory: /website
schedule:
interval: weekly
open-pull-requests-limit: 5
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
actions-minor-and-patch:
update-types:
- minor
- patch
+3 -14
View File
@@ -17,10 +17,7 @@ describe the use case and observable outcome. -->
## Change and scope
<!-- Explain the chosen approach, why it fits REA's architecture, and what is
intentionally not included. If adding complexity for a speculative or vanishingly
unlikely race or security edge case, record the concrete failure mode, likelihood,
complexity cost, and the user's buy-in obtained before implementation. Do not
silently expand scope for theoretical robustness. -->
intentionally not included. -->
Follow the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
Unresolved scope, authority, side effects, or required verification block
@@ -36,9 +33,6 @@ unchecked. Maintainers can help resolve open questions.
Use "none" or "not applicable" only when established. -->
- Semantic owner and earliest changed stage:
<!-- Name the producer of parsed facts or resources and its affected consumers.
Summarize superseded paths removed and any distinct policy, lifecycle,
lazy-loading, or compatibility responsibility retained. -->
- Contribution scope fit and linked maintainer decision:
- CLI and MCP/tool-catalog contract:
- Provider, bridge, target-format, or platform compatibility:
@@ -71,10 +65,6 @@ source, link, or rendered-document checks. -->
- User-visible CLI/MCP output (if applicable):
- Remaining proof gaps:
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#boundary-regressions)
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#lifecycle-regressions)
covered by the evidence above, and explain relevant gaps.
For evidence-bearing changes:
- [ ] Observed, derived, and inferred claims remain distinguishable.
@@ -99,15 +89,14 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
its new behavior and migration, and why compatibility/deprecation cannot
preserve it. Every release increments minor, including breaking changes;
keep breaking markers and migration notes so callers can assess compatibility.
See docs/development/releasing.md. -->
See docs/releasing.md. -->
- Real Hopper/Ghidra, browser, or OS coverage:
- Package or release metadata impact:
- Security, privacy, process, or containment review:
## Author readiness
<!-- The human owner completes these declarations before requesting review,
unless the PR author is exempt as listed in CONTRIBUTING.md.
<!-- The human owner completes these declarations before requesting review.
The PR author is the owner by default. A different owner must explicitly accept
ownership on this PR.
AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA.
-64
View File
@@ -1,64 +0,0 @@
name: Cache retention
on:
workflow_run:
workflows: ["CI"]
types: [completed]
# Qualification calls ci.yml with actions: read. A nested workflow cannot raise
# that token, so cache deletion stays in this workflow.
permissions:
contents: read
actions: write
concurrency:
group: cache-retention-${{ github.event.workflow_run.head_sha }}
cancel-in-progress: false
jobs:
retain:
name: Retain Turbo snapshots and npm download caches
# Restrict the triggering-source checkout to same-repository default-branch
# CI events. The helper runs from this trusted workflow revision; triggering
# source is used only for cache identity and is never executed. Static and
# build save snapshots before later lanes finish, so failed, timed-out, or
# cancelled runs still retain those snapshots.
if: >-
(github.event.workflow_run.conclusion == 'success' ||
github.event.workflow_run.conclusion == 'failure' ||
github.event.workflow_run.conclusion == 'timed_out' ||
github.event.workflow_run.conclusion == 'cancelled') &&
github.event.workflow_run.head_repository.full_name == github.repository &&
github.event.workflow_run.head_branch == github.event.repository.default_branch &&
(github.event.workflow_run.event == 'push' ||
github.event.workflow_run.event == 'schedule' ||
github.event.workflow_run.event == 'workflow_dispatch')
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
ref: ${{ github.workflow_sha }}
path: trusted-retention
persist-credentials: false
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
ref: ${{ github.event.workflow_run.head_sha }}
path: triggering-source
persist-credentials: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: trusted-retention/.nvmrc
- name: Retain current Turbo snapshots and npm download caches
env:
GH_TOKEN: ${{ github.token }}
CI_REPOSITORY: ${{ github.repository }}
CACHE_PREFIX: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-
CACHE_KEEP_KEY: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('triggering-source/.nvmrc', 'triggering-source/.npmrc', 'triggering-source/package-lock.json') }}-${{ github.event.workflow_run.head_sha }}
CACHE_DEEP_PREFIX: rea-static-deep-v1-${{ runner.os }}-${{ runner.arch }}-
CACHE_DEEP_KEEP_KEY: rea-static-deep-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('triggering-source/.nvmrc', 'triggering-source/.npmrc', 'triggering-source/package-lock.json') }}-${{ github.event.workflow_run.head_sha }}
CACHE_KNIP_PREFIX: knip-meta-v1-${{ runner.os }}-
CACHE_KNIP_KEEP_KEY: knip-meta-v1-${{ runner.os }}-${{ hashFiles('triggering-source/knip.json', 'triggering-source/tsconfig.json', 'triggering-source/package.json', 'triggering-source/package-lock.json') }}-${{ github.event.workflow_run.head_sha }}
CACHE_PORTABLE_PREFIX: rea-portable-build-v2-${{ runner.os }}-${{ runner.arch }}-
DEDUPE_NPM_DOWNLOADS: "true"
run: node trusted-retention/scripts/ci/cache-retention.mjs
+39 -296
View File
@@ -15,15 +15,6 @@ on:
description: Exact qualification source; must match the caller revision.
required: true
type: string
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted release qualification passes it.
required: false
permissions:
contents: read
@@ -38,15 +29,6 @@ concurrency:
env:
HUSKY: "0"
# Set Actions secret TURBO_TOKEN and variable TURBO_TEAM. OIDC can mint
# TURBO_TOKEN via vercel/setup-turborepo-remote-cache-action; do not commit a token.
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Release qualification sets remote_cache_write and passes the
# token. Open pull requests get an empty token and the local cache only.
# Remote reads need a credential, and pull requests do not receive one.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
jobs:
changes:
@@ -61,9 +43,6 @@ jobs:
qualification_run_id: ${{ steps.qualification.outputs.run_id }}
code: ${{ steps.plan.outputs.code }}
reverse_tools: ${{ steps.plan.outputs.reverse_tools }}
python: ${{ steps.plan.outputs.python }}
packet_capture: ${{ steps.plan.outputs.packet_capture }}
signal_recording: ${{ steps.plan.outputs.signal_recording }}
sqlite: ${{ steps.plan.outputs.sqlite }}
runtime: ${{ steps.plan.outputs.runtime }}
package: ${{ steps.plan.outputs.package }}
@@ -75,13 +54,11 @@ jobs:
browser: ${{ steps.plan.outputs.browser }}
binary_layout: ${{ steps.plan.outputs.binary_layout }}
evm: ${{ steps.plan.outputs.evm }}
apk_signature: ${{ steps.plan.outputs.apk_signature }}
javascript_recovery: ${{ steps.plan.outputs.javascript_recovery }}
recorded_crash: ${{ steps.plan.outputs.recorded_crash }}
web_network: ${{ steps.plan.outputs.web_network }}
web_runtime: ${{ steps.plan.outputs.web_runtime }}
web_source_map: ${{ steps.plan.outputs.web_source_map }}
agentenv: ${{ steps.plan.outputs.agentenv }}
nativeaot: ${{ steps.plan.outputs.nativeaot }}
ghidra_bridge: ${{ steps.plan.outputs.ghidra_bridge }}
website: ${{ steps.plan.outputs.website }}
@@ -147,15 +124,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Validate source architecture before dependency installation
if: needs.changes.outputs.code == 'true'
run: node scripts/verify/architecture-guards.mjs
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
with:
# Linux x64 writer. Parallel Linux jobs restore this key.
save: "true"
- run: npm ci
- name: Restore static task cache
if: needs.changes.outputs.code == 'true'
@@ -179,10 +150,14 @@ jobs:
if: needs.changes.outputs.code == 'true'
env:
CI_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
CI_STATIC_CONTEXT_SCRIPT: check:ci:fast:tasks
run: npm run ci:static-context
- if: needs.changes.outputs.code == 'true'
run: npm run check:ci:fast
run: npm run check:ci
- if: needs.changes.outputs.code == 'true'
run: npm run verify:test-discovery
- name: Check LLDB producer retention and launch contracts
if: needs.changes.outputs.code == 'true'
run: npm run test:python
- if: needs.changes.outputs.code != 'true'
run: npm run format:check
- name: Save validated static task cache
@@ -204,93 +179,6 @@ jobs:
if-no-files-found: ignore
retention-days: 7
static-deep:
name: Deep static checks
needs: changes
# Full qualification only. Provider lanes wait on fast static, not this job.
if: needs.changes.outputs.tier == 'full' && needs.changes.outputs.code == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- name: Restore deep static task cache
id: static-cache
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: .turbo/cache/
key: rea-static-deep-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-${{ github.sha }}
restore-keys: rea-static-deep-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-
- name: Restore Knip analysis cache
id: knip-cache
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: node_modules/.cache/knip
# Each trusted commit publishes a new entry. Restore still matches the
# configuration prefix from the previous successful full run.
key: knip-meta-v1-${{ runner.os }}-${{ hashFiles('knip.json', 'tsconfig.json', 'package.json', 'package-lock.json') }}-${{ github.sha }}
restore-keys: knip-meta-v1-${{ runner.os }}-${{ hashFiles('knip.json', 'tsconfig.json', 'package.json', 'package-lock.json') }}-
- name: Record deep static task source and execution identity
continue-on-error: true
env:
CI_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
CI_STATIC_CONTEXT_SCRIPT: check:ci:deep:tasks
run: npm run ci:static-context
- name: Run deep static checks
run: npm run check:ci:deep
- name: Save validated deep static task cache
if: >-
success() && steps.npm-cache.outputs.can-save == 'true' &&
steps.static-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: .turbo/cache/
key: ${{ steps.static-cache.outputs.cache-primary-key }}
- name: Save Knip analysis cache
if: >-
success() && steps.npm-cache.outputs.can-save == 'true' &&
steps.knip-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: node_modules/.cache/knip
key: ${{ steps.knip-cache.outputs.cache-primary-key }}
- name: Upload complete deep static diagnostics
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: static-deep-check-reports
path: .cache/static-checks/
include-hidden-files: true
if-no-files-found: ignore
retention-days: 7
python:
name: Python producer tests
needs: changes
if: needs.changes.outputs.python == 'true' && needs.changes.outputs.tier == 'fast'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
# Analysis-efficiency checks import compiled process modules.
- name: Reuse portable build outputs across runs
uses: ./.github/actions/portable-build-cache
- run: npm run test:prepare
- run: npm run test:python
test-source:
name: Source tests
needs: changes
@@ -330,11 +218,9 @@ jobs:
# every shard's diagnostics and coverage. Passing PRs run the full suite.
fail-fast: ${{ github.event_name == 'pull_request' && needs.changes.outputs.tier == 'fast' }}
matrix:
# Fast feedback uses one source group and eight built groups so the
# slowest built shard job stays inside a ten-minute execution budget.
# Classification, build, and report merge sit outside that job.
# Full qualification retains six complete shards with coverage.
shard: ${{ fromJSON(needs.changes.outputs.tier == 'fast' && '["1/8","2/8","3/8","4/8","5/8","6/8","7/8","8/8"]' || '["1/6","2/6","3/6","4/6","5/6","6/6"]') }}
# Fast feedback spends the same six test jobs on one source group and
# five built groups; full qualification retains six complete shards.
shard: ${{ fromJSON(needs.changes.outputs.tier == 'fast' && '["1/5","2/5","3/5","4/5","5/5"]' || '["1/6","2/6","3/6","4/6","5/6","6/6"]') }}
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
@@ -412,15 +298,11 @@ jobs:
needs.changes.outputs.browser == 'true' ||
needs.changes.outputs.binary_layout == 'true' ||
needs.changes.outputs.evm == 'true' ||
needs.changes.outputs.apk_signature == 'true' ||
needs.changes.outputs.javascript_recovery == 'true' ||
needs.changes.outputs.recorded_crash == 'true' ||
needs.changes.outputs.web_network == 'true' ||
needs.changes.outputs.web_runtime == 'true' ||
needs.changes.outputs.web_source_map == 'true' ||
needs.changes.outputs.agentenv == 'true' ||
needs.changes.outputs.packet_capture == 'true' ||
needs.changes.outputs.signal_recording == 'true'
needs.changes.outputs.web_source_map == 'true'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
@@ -440,9 +322,7 @@ jobs:
with:
save: ${{ github.event_name != 'pull_request' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}
- run: npm run test:prepare
# Fast and full shards measure different file costs. Download only the
# matching history so one tier cannot pack the other with its weights.
- name: Find available trusted timing history
- name: Find available trusted full-suite timing history
id: history
continue-on-error: true
env:
@@ -450,14 +330,13 @@ jobs:
CI_REPOSITORY: ${{ github.repository }}
CI_REPOSITORY_ID: ${{ github.repository_id }}
CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
CI_TIMING_TIER: ${{ needs.changes.outputs.tier }}
run: node scripts/ci/test-history.mjs
- name: Download previous timing report
if: steps.history.outputs.run-id != ''
continue-on-error: true
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: ${{ steps.history.outputs.artifact-name }}
name: test-timings
run-id: ${{ steps.history.outputs.run-id }}
github-token: ${{ github.token }}
path: .cache/ci-test-history
@@ -478,23 +357,17 @@ jobs:
# test:prepare caches only portable JS, skill, and catalog outputs.
# Native package jobs still run their build and prepack checks; Turbo
# restores matching outputs or rebuilds when their inputs differ.
- name: Stage current portable build task cache
shell: bash
run: |
# A new Turbo dry run can evict the entries this build just used.
node scripts/ci/portable-build-cache.mjs snapshot < .cache/ci-build-plan.json
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: rea-build-cache
path: .cache/ci-build-cache/
path: .turbo/cache/
include-hidden-files: true
if-no-files-found: error
retention-days: 7
sqlite-runtime:
# Admit expensive host/provider work after fast static validation
# (typecheck, lint, format). Deep static does not gate these lanes.
# Portable tests still provide early feedback.
# Preserve all selected coverage, but admit expensive host/provider work
# only after static validation. Portable tests still provide early feedback.
needs: [changes, static, build]
if: needs.changes.outputs.sqlite == 'true'
uses: ./.github/workflows/sqlite-runtime.yml
@@ -544,7 +417,7 @@ jobs:
--coverage.reporter=text
--coverage.reporter=json-summary
--coverage.reporter=lcov
--reporter=dot
--reporter=default
--reporter=junit
--reporter=json
--reporter=./scripts/ci/test-timing-reporter.mjs
@@ -559,7 +432,7 @@ jobs:
needs.test-source.result == 'success' || needs.test-source.result == 'failure')
run: >-
npm exec -- vitest --merge-reports
--reporter=dot
--reporter=default
--reporter=junit
--reporter=json
--reporter=./scripts/ci/test-timing-reporter.mjs
@@ -589,22 +462,11 @@ jobs:
include-hidden-files: true
if-no-files-found: error
retention-days: 30
# Default-branch runs only. Pull-request artifacts share this name and
# the lookup reads one page, so publishing them would hide main.
- name: Publish successful timing history
if: >-
success() &&
needs.changes.outputs.code == 'true' &&
needs.test-shard.result == 'success' &&
(
needs.changes.outputs.tier == 'full' ||
(needs.changes.outputs.tier == 'fast' && needs.test-source.result == 'success')
) &&
github.ref == format('refs/heads/{0}', github.event.repository.default_branch) &&
(github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')
if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'full' && success() && needs.test-shard.result == 'success'
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: test-timings-${{ needs.changes.outputs.tier }}
name: test-timings
path: .cache/ci-test-history/test-results.json
include-hidden-files: true
if-no-files-found: error
@@ -708,9 +570,6 @@ jobs:
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
with:
# ubuntu-latest is the Linux x64 writer above. Other hosts save their own key.
save: ${{ matrix.os != 'ubuntu-latest' }}
- name: Verify native runtime identity
env:
EXPECTED_CI_PLATFORM: ${{ matrix.platform }}
@@ -779,23 +638,9 @@ jobs:
npm run verify:legacy-simulator
npm run verify:apple-dispatch
npm run verify:interface-builder
- name: Verify portable iNES CLI, MCP and source boundaries
run: >-
npm run test:focused --
tests/acceptance/inesContainer.test.ts
tests/boundary/filesystem/inesArtifact.test.ts
tests/boundary/filesystem/inesCancellation.test.ts
- name: Run native Apple and process boundary regressions
run: >-
npm run test:focused --
tests/boundary/providers/hopper/hopperMacLaunchArguments.test.ts
tests/acceptance/providers/hopperStartupRecovery.test.ts
tests/boundary/providers/hopper/hopperClient.helperFailure.test.ts
tests/boundary/providers/hopper/bridgeBootstrapCleanup.test.ts
tests/boundary/providers/hopper/bridgeTargetLease.integration.test.ts
tests/acceptance/artifactDmgAttachmentRecoveryCli.test.ts
tests/boundary/providers/native/nativeDmgArtifactReader.test.ts
tests/process-global/dmgCancellationReason.test.ts
tests/boundary/filesystem/installScript.test.ts
tests/boundary/filesystem/nibNativeHierarchy.test.ts
tests/boundary/filesystem/interfaceBuilderBundle.test.ts
@@ -803,15 +648,9 @@ jobs:
tests/boundary/cli/nativePlistPrototypeKeyCli.test.ts
tests/boundary/cli/nativePlistSelectionFailureCli.test.ts
tests/boundary/cli/nativeDemangleCli.test.ts
tests/boundary/cli/nativeDispatchPropertyAttributes.test.ts
tests/boundary/mcp/nativeDemangleMcp.test.ts
tests/boundary/mcp/nativeMachoSummaryMcp.test.ts
tests/boundary/providers/native/nativeSignatureTargetBinding.test.ts
tests/boundary/providers/native/nativeUiOffscreenWindow.test.ts
tests/boundary/process/nativeUiVerifierLane.test.ts
tests/boundary/cli/nativeSignatureCli.test.ts
tests/boundary/cli/nativeMachoArm64_32Cli.test.ts
tests/boundary/cli/nativeMachoEncryptionCli.test.ts
tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts
src/process/capture/FilesystemSnapshot.test.ts
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
@@ -836,7 +675,7 @@ jobs:
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- name: Verify static CLI/MCP journeys and synthetic report conformance
- name: Run the public CLI/MCP reconstruction journey
env:
REA_READINESS_REPORT_PATH: readiness-report.json
run: |
@@ -866,24 +705,17 @@ jobs:
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
with:
save: "true"
- run: npm ci
- name: Typecheck with the official Windows compiler
env:
GOMAXPROCS: "2"
GOMEMLIMIT: "3GiB"
run: npm run typecheck -- --checkers 2
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: rea-build-cache
path: .turbo/cache/
- run: npm run build:cached
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-native-controls
path: native/windows/build/
- run: npm run verify:windows-native:artifact
- run: npm run build:cached
- name: Record named Windows host capabilities
run: node scripts/windows-capabilities.mjs | Out-File -Encoding utf8NoBOM windows-capabilities.json
- name: Validate Windows host capability report
@@ -899,54 +731,24 @@ jobs:
# The matching native artifact establishes real Job Object and private
# runtime boundaries for these fixtures. Ghidra and Java are not needed;
# real engine claims remain in the separate controlled-provider workflow.
# Run native proofs before the broad curated suite so an unrelated
# fixture failure cannot hide the real control and recovery checks.
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-process-boundary-fixture
path: .cache/windows-process-boundary/
# Passing the fixture adds the breakaway, caller/child token, environment
# semantics and ancestor reparse observations, which the lane skips
# entirely when it is absent.
- run: >-
npm run verify:windows-native -- .
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-native-inspection-fixture
path: .cache/windows-native-inspection-fixture/
- name: Verify native inspection failure ownership and recovery
run: node scripts/verify-windows-native-inspection.mjs
- name: Run curated Windows capability tests
run: >-
npx vitest run --coverage.enabled=false
src/process/capture/FilesystemSnapshot.test.ts
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
tests/boundary/filesystem/typescriptCompiler.test.ts
tests/boundary/filesystem/releaseProposal.test.ts
tests/acceptance/byteEntropyWindowsIdentity.test.ts
tests/boundary/filesystem/binaryTarget.test.ts
src/contracts/sessionPathInputs.test.ts
tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts
tests/boundary/mcp/rawBinaryMcp.test.ts
tests/boundary/filesystem/safeOutputTree.test.ts
tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts
tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts
tests/boundary/filesystem/setupUnavailableProvider.test.ts
tests/boundary/filesystem/windowsNpxRegistration.test.ts
tests/boundary/filesystem/claudeDesktopSkillUninstallWindows.test.ts
tests/boundary/filesystem/clientConfigurationEnvironment.test.ts
tests/boundary/filesystem/clientConfigurationLayeredEnvironment.test.ts
tests/acceptance/setup/registrationRuntimeEnvironmentCli.test.ts
tests/boundary/android/workspaceGuardianRetry.test.ts
tests/acceptance/setup/homeDirectoryFallback.test.ts
tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts
tests/boundary/filesystem/webScriptManifestPaths.test.ts
tests/boundary/process/ProcessEvidence.host.test.ts
tests/boundary/process/packageNpmInvocation.test.ts
tests/boundary/process/packedBridgeVerification.test.ts
tests/boundary/process/windowsOutputCapture.test.ts
tests/boundary/process/windowsExecutableSelection.test.ts
tests/acceptance/webModuleCli.test.ts
tests/acceptance/webSourceLocationCli.test.ts
tests/boundary/cli/referenceSourceImportStatus.test.ts
@@ -968,9 +770,23 @@ jobs:
src/process/ProcessOwnership.validation.test.ts
src/application/Update.test.ts
tests/process-global/updateMaintenance.test.ts
tests/process-global/runnerCancellation.test.ts
tests/process-global/windowsOwnedCommand.test.ts
tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-process-boundary-fixture
path: .cache/windows-process-boundary/
# Passing the fixture adds the breakaway, caller/child token, environment
# semantics and ancestor reparse observations, which the lane skips
# entirely when it is absent.
- run: >-
npm run verify:windows-native -- .
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-native-inspection-fixture
path: .cache/windows-native-inspection-fixture/
- name: Verify native inspection failure ownership and recovery
run: node scripts/verify-windows-native-inspection.mjs
- name: Verify real Node Inspector discovery and observation
timeout-minutes: 10
env:
@@ -986,7 +802,7 @@ jobs:
tests/boundary/filesystem/javascriptArtifactContentIdentity.test.ts
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: 22.23.4
node-version: 22.19.0
- name: Verify native path boundaries on Node 22
run: >-
npx vitest run --coverage.enabled=false
@@ -1002,7 +818,7 @@ jobs:
retention-days: 30
docs:
name: Documentation
name: Docs Generation
needs: changes
if: needs.changes.outputs.docs == 'true'
runs-on: ubuntu-latest
@@ -1017,8 +833,6 @@ jobs:
uses: ./.github/actions/npm-cache
- run: npm ci
- run: npm run docs:check
- name: Verify published routes, source links and local anchors
run: npm run verify:docs:routes
- name: Generation leaves authored files unchanged
run: git diff --exit-code
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
@@ -1030,126 +844,61 @@ jobs:
skills/reverse-engineer-anything
if-no-files-found: error
# Provider lanes wait for fast static and the portable build, not deep static.
browser:
needs: [changes, static, build]
if: needs.changes.outputs.browser == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-browser.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
binary_layout:
needs: [changes, static, build]
if: needs.changes.outputs.binary_layout == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-binary-layout.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
evm:
needs: [changes, static, build]
if: needs.changes.outputs.evm == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-evm-interface.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
apk_signature:
needs: [changes, static, build]
if: needs.changes.outputs.apk_signature == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-apk-signature.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
javascript_recovery:
needs: [changes, static, build]
if: needs.changes.outputs.javascript_recovery == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-javascript-recovery.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
recorded_crash:
needs: [changes, static, build]
if: needs.changes.outputs.recorded_crash == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-recorded-crash.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
web_network:
needs: [changes, static, build]
if: needs.changes.outputs.web_network == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-web-network-captures.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
web_runtime:
needs: [changes, static, build]
if: needs.changes.outputs.web_runtime == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-web-runtime.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
web_source_map:
needs: [changes, static, build]
if: needs.changes.outputs.web_source_map == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-web-source-map.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
agentenv:
needs: [changes, static, build]
if: needs.changes.outputs.agentenv == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-agentenv.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
packet_capture:
needs: [changes, static, build]
if: needs.changes.outputs.packet_capture == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-packet-capture.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
signal_recording:
needs: [changes, static, build]
if: needs.changes.outputs.signal_recording == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-signal-recording.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
nativeaot:
needs: [changes, static]
@@ -1174,12 +923,8 @@ jobs:
[
changes,
static,
static-deep,
build,
reverse-tools-quality,
python,
packet_capture,
signal_recording,
test-shard,
test-source,
sqlite-runtime,
@@ -1194,13 +939,11 @@ jobs:
browser,
binary_layout,
evm,
apk_signature,
javascript_recovery,
recorded_crash,
web_network,
web_runtime,
web_source_map,
agentenv,
nativeaot,
ghidra_bridge,
website,
-61
View File
@@ -1,61 +0,0 @@
name: CodeQL production
on:
push:
branches: [main]
pull_request:
branches: [main]
schedule:
- cron: "37 5 * * 1"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: codeql-production-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
analyze:
# A repository administrator must disable default setup before enabling
# this workflow; default setup rejects advanced CodeQL result uploads.
if: vars.REA_CODEQL_ADVANCED == 'true'
name: Analyze production (${{ matrix.language }})
runs-on: ubuntu-latest
timeout-minutes: 45
permissions:
contents: read
security-events: write
strategy:
fail-fast: false
max-parallel: 1
matrix:
language: [javascript-typescript, python, actions]
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
with:
persist-credentials: false
- name: Initialize production analysis
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
with:
languages: ${{ matrix.language }}
build-mode: none
config-file: .github/codeql/production.yml
threads: 2
ram: 4096
- name: Analyze production code
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
with:
category: /language:${{ matrix.language }}
output: ${{ runner.temp }}/codeql-production-results
threads: 2
ram: 4096
- name: Retain complete analysis evidence
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: codeql-production-${{ matrix.language }}
path: ${{ runner.temp }}/codeql-production-results
if-no-files-found: error
retention-days: 7
+4 -81
View File
@@ -25,16 +25,16 @@ jobs:
node-version-file: .nvmrc
- name: Validate the SDK pin against the provider's verified release
run: node scripts/verify-ghidra-bridge-sources.mjs --check-sdk-pin >> "$GITHUB_ENV"
# The bridge probe imports only source-owned ARM/Thumb bytes and never
# executes target code. All SDK/project state stays in runner storage.
- name: Acquire checksum-verified Ghidra SDK in isolated runner storage
# Only the jars and application.properties are extracted: the check
# compiles against them and never launches Ghidra or imports a target.
- name: Acquire checksum-verified Ghidra jars in isolated runner storage
shell: bash
run: |
ghidra_root="${RUNNER_TEMP}/rea-ghidra"
mkdir -p "${ghidra_root}"
curl -fsSL --retry 3 --write-out 'ghidra_download_bytes=%{size_download} ghidra_download_seconds=%{time_total}\n' -o "${ghidra_root}/ghidra.zip" "${GHIDRA_RELEASE_URL}"
echo "${GHIDRA_RELEASE_SHA256} ${ghidra_root}/ghidra.zip" | sha256sum -c -
unzip -q "${ghidra_root}/ghidra.zip" -d "${ghidra_root}"
unzip -q "${ghidra_root}/ghidra.zip" '*/Ghidra/*.jar' '*/Ghidra/application.properties' -d "${ghidra_root}"
rm "${ghidra_root}/ghidra.zip"
properties="$(find "${ghidra_root}" -path '*/Ghidra/application.properties' -print -quit)"
test -n "${properties}"
@@ -46,80 +46,3 @@ jobs:
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
test -n "${!java_home_variable}"
JAVA_HOME="${!java_home_variable}" npm run verify:ghidra:bridge-sources -- --verified-sdk
- name: Inspect real ARM and Thumb instruction context through the bridge
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
probe_root="${RUNNER_TEMP}/rea-arm-context"
mkdir -p "${probe_root}/project"
printf '\036\377\057\341\160\107\000\000' > "${probe_root}/arm-thumb.bin"
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/project" ArmContext \
-import "${probe_root}/arm-thumb.bin" -loader BinaryLoader -processor ARM:LE:32:v7 \
-noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
-postScript ReaArmContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
cat "${probe_root}/output.log"
grep -F 'REA_ARM_CONTEXT_OK arm=0 thumb=1 unavailable=null (injected provider value)' "${probe_root}/output.log"
# Each probe imports source-owned bytes at 0x1000, away from the Cortex-M
# vector symbols that ARMCortex.pspec defines at 0, and decodes them from
# the language's default context, so M-profile Thumb state must hold at
# the first byte. The bytes are never executed.
- name: Decode M-profile ARM and Xtensa bytes in their default context
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
probe_root="${RUNNER_TEMP}/rea-decode"
mkdir -p "${probe_root}"
# mrs r0, msp; bx lr
printf '\357\363\010\200\160\107' > "${probe_root}/cortex.bin"
# mrs r0, msplim (ARMv8-M only); bx lr
printf '\357\363\012\200\160\107' > "${probe_root}/v8m.bin"
# entry a1, 32; retw.n
printf '\066\101\000\035\360' > "${probe_root}/xtensa.bin"
decode() {
local name="$1" language="$2"
shift 2
mkdir -p "${probe_root}/${name}-project"
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/${name}-project" Decode \
-import "${probe_root}/${name}.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
-processor "${language}" -noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra" \
-postScript ReaDecodeProbe.java "${language}" "$@" \
> "${probe_root}/${name}.log" 2>&1 || { cat "${probe_root}/${name}.log"; exit 1; }
cat "${probe_root}/${name}.log"
grep -F "REA_DECODE_OK ${language} $*" "${probe_root}/${name}.log"
}
decode cortex ARM:LE:32:Cortex mrs/4/msp bx/2/lr
decode v8m ARM:LE:32:v8-m mrs/4/msplim bx/2/lr
decode xtensa Xtensa:LE:32:default entry/3/a1 retw.n/2
# A call through a writable slot keeps Ghidra's recorded target as an
# observed callee while callee coverage stays partial; a direct call is
# the complete control. The source-owned x86-64 bytes are never executed.
- name: Keep computed-call targets non-exhaustive in callee coverage
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
probe_root="${RUNNER_TEMP}/rea-computed-call"
mkdir -p "${probe_root}/project"
{
printf '\270\001\000\000\000\303\314\314' # 0x1000: mov eax, 1; ret
printf '\270\002\000\000\000\303\314\314' # 0x1008: mov eax, 2; ret
printf '\377\025\052\000\000\000\303\314' # 0x1010: call [rip + 0x2a]; ret
printf '\377\320\303\314\314\314\314\314' # 0x1018: call rax; ret
printf '\350\333\377\377\377\303\314\314' # 0x1020: call 0x1000; ret
printf '\314%.0s' $(seq 24) # 0x1028: int3 padding
printf '\000\020\000\000\000\000\000\000' # 0x1040: slot = 0x1000
} > "${probe_root}/computed-call.bin"
test "$(wc -c < "${probe_root}/computed-call.bin")" -eq 72
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/project" ComputedCall \
-import "${probe_root}/computed-call.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
-processor x86:LE:64:default -noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
-postScript ReaComputedCallCoverageProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
cat "${probe_root}/output.log"
grep -F 'REA_COMPUTED_CALL_COVERAGE_OK slot=observed-complete-discovery-unknown register=partial direct=complete callers=discovery-unknown' "${probe_root}/output.log"
-5
View File
@@ -14,11 +14,6 @@ concurrency:
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
jobs:
mutation:
+1 -1
View File
@@ -27,4 +27,4 @@ jobs:
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- run: npm run verify:docs:routes
- run: npm run docs:build
-99
View File
@@ -1,99 +0,0 @@
name: Real agentenv environment bundle
on:
workflow_dispatch:
workflow_call:
inputs:
build_cache_artifact:
description: Portable build cache artifact from the calling CI run.
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
permissions:
contents: read
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
NODE_OPTIONS: --max-old-space-size=768
PIP_DISABLE_PIP_VERSION_CHECK: "1"
jobs:
real-agentenv:
name: Export and run an environment bundle with the agentenv runner
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
# The pinned framework builds its gateway and service-db images only when pulling them from
# ghcr.io fails, from public ECR base images; public ECR caps anonymous pulls by data per source
# IP and hosted runners share IPs, so such a build can fail here for hours. mirror.gcr.io serves
# the same Docker official images under the same digests, so each base image those Dockerfiles
# name is pulled there and named as its Dockerfile names it, as the framework's own CI does.
# Digest-pinned names are set in containerd, whose images Docker reads only on the containerd
# image store; switching stores restarts dockerd, so it happens before anything uses Docker.
- name: Use the containerd image store
run: |
[ -s /etc/docker/daemon.json ] || echo '{}' | sudo tee /etc/docker/daemon.json >/dev/null
jq '.features["containerd-snapshotter"] = true' /etc/docker/daemon.json > "$RUNNER_TEMP/daemon.json"
sudo mv "$RUNNER_TEMP/daemon.json" /etc/docker/daemon.json
sudo systemctl restart docker
docker info --format '{{json .DriverStatus}}' | grep -q io.containerd.snapshotter
- name: Acquire the pinned agentenv runner in isolated runner storage and check host Docker
shell: bash
run: |
runner_root="${RUNNER_TEMP}/rea-agentenv-runner"
python -m venv "${runner_root}"
"${runner_root}/bin/python" -m pip install --require-virtualenv --no-cache-dir 'agentenv-framework==0.9.1431'
"${runner_root}/bin/agent-env" --version
docker version --format '{{.Server.Version}}'
echo "REA_AGENT_ENV_COMMAND=${runner_root}/bin/agent-env" >> "${GITHUB_ENV}"
echo "REA_DOCKER_COMMAND=$(command -v docker)" >> "${GITHUB_ENV}"
- name: Pull the gateway and service-db public ECR base images from mirror.gcr.io
shell: bash
run: |
package="$("${REA_AGENT_ENV_COMMAND%/agent-env}/python" -c 'import agent_env, pathlib; print(pathlib.Path(agent_env.__file__).parent)')"
grep -rhoE '^FROM +public\.ecr\.aws/docker/library/[^ ]+' --include='Dockerfile*' "${package}/env/gateway" "${package}/env/envs/service_db" | awk '{print $2}' | sort -u |
while read -r image; do
name=${image#public.ecr.aws/docker/library/}
source=mirror.gcr.io/library/$name
case $name in *@*) source=mirror.gcr.io/library/${name%%[:@]*}@${name#*@} ;; esac
for attempt in 1 2 3; do
docker pull --quiet "$source" && break
if [ "$attempt" = 3 ]; then echo "::warning::couldn't pull $source; $image's build will pull it"; continue 2; fi
sleep $((attempt * 10))
done
case ${name%@*} in *:*) docker tag "$source" "${image%@*}" ;; esac
case $image in *@*) sudo ctr --namespace moby images tag "$source" "$image" >/dev/null ;; esac
done
- name: Export a bundle from the source-owned capture and run it with the agentenv runner
run: npm run verify:agentenv:environment
@@ -1,63 +0,0 @@
name: Real Android metadata bridge
on:
push:
branches: ["**"]
paths:
- .github/workflows/real-android-metadata.yml
- bridge/android/**
- src/android/JadxRelease.ts
- scripts/fixtures/android/**
- tests/conformance/android/**
- tests/fixtures/android/static-initializers/**
- tests/fixtures/android/direct-invokes/**
- tests/fixtures/android/external-invokes/**
- .nvmrc
workflow_dispatch:
workflow_call:
permissions:
contents: read
concurrency:
group: android-metadata-${{ github.ref }}
cancel-in-progress: true
jobs:
android-metadata:
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Fetch checksum-pinned engine and public APK fixture
run: node scripts/fixtures/android/fetch.mjs "$RUNNER_TEMP/rea-android"
- name: Compile and exercise the real metadata bridge
shell: bash
run: |
export JAVA_HOME="$JAVA_HOME_21_X64"
android_root="$RUNNER_TEMP/rea-android"
classes="$android_root/classes"
jar="$android_root/jadx-headless-mcp-0.7.1-all.jar"
mkdir -p "$classes"
"$JAVA_HOME/bin/javac" -J-Xmx512m -J-XX:ActiveProcessorCount=2 \
-classpath "$jar" -d "$classes" \
bridge/android/ReaJadxBridge.java \
tests/conformance/android/ReaJadxMetadataRegression.java \
tests/conformance/android/ReaJadxExternalReferencesRegression.java
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
-classpath "$classes:$jar" ReaJadxMetadataRegression \
"$android_root/ApiDemos-debug.apk" \
"$GITHUB_WORKSPACE/tests/fixtures/android/static-initializers" \
"$classes" "$GITHUB_WORKSPACE/tests/fixtures/android/direct-invokes" \
| tee "$android_root/metadata.log"
grep -F 'RAW_CLASS_SELECTOR_REGRESSION_PASS nested=true metadata=true references=true' \
"$android_root/metadata.log"
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
-classpath "$classes:$jar" ReaJadxExternalReferencesRegression \
"$GITHUB_WORKSPACE/tests/fixtures/android/external-invokes" "$classes" \
| tee "$android_root/external-references.log"
grep -F 'EXTERNAL_REFERENCE_REGRESSION_PASS prototypes=2 using_nodes=3 sites=4' \
"$android_root/external-references.log"
-97
View File
@@ -1,97 +0,0 @@
name: Real APK signature inspection
on:
workflow_call:
inputs:
build_cache_artifact:
description: Portable build cache artifact from the calling CI run.
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
permissions:
contents: read
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
NODE_OPTIONS: "--max-old-space-size=768"
RAYON_NUM_THREADS: "1"
GOMAXPROCS: "2"
UV_THREADPOOL_SIZE: "2"
JAVA_TOOL_OPTIONS: "-Xmx512m -XX:ActiveProcessorCount=2"
jobs:
real-apk-signature:
runs-on: ubuntu-latest
timeout-minutes: 12
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- name: Prepare pinned SDK oracle without installing host software
shell: bash
run: |
python3 - <<'PYTHON'
import hashlib, os, urllib.request, zipfile
from pathlib import Path
root = Path(os.environ["RUNNER_TEMP"]) / "rea-apk-signature-sdk"
packages = [
("build-tools_r36.1_linux.zip", "a7b5889e4a79fcf3b0976bef40d401f4240fb1eed891d9d91169da1111e11d78", 64195204, "build-tools/36.1.0", {"apksigner", "aapt2", "lib/apksigner.jar", "source.properties"}),
("platform-36_r02.zip", "37607369a28c5b640b3a7998868d45898ebcb777565a0e85f9acf36f29631d2e", 65878410, "platforms/android-36", {"android.jar", "source.properties"}),
]
for name, expected, expected_size, directory, selected in packages:
archive = root / name
archive.parent.mkdir(parents=True, exist_ok=True)
digest = hashlib.sha256()
total = 0
with urllib.request.urlopen("https://dl.google.com/android/repository/" + name, timeout=60) as response, archive.open("wb") as output:
while chunk := response.read(1024 * 1024):
total += len(chunk)
if total > expected_size:
raise RuntimeError("SDK archive exceeds the pinned package size budget")
digest.update(chunk)
output.write(chunk)
if total != expected_size or digest.hexdigest() != expected:
raise RuntimeError("SDK oracle archive digest mismatch")
extracted = set()
with zipfile.ZipFile(archive) as source:
for entry in source.infolist():
relative = entry.filename.partition("/")[2]
if relative not in selected:
continue
target = root / directory / relative
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(source.read(entry))
target.chmod((entry.external_attr >> 16) & 0o777 or 0o644)
extracted.add(relative)
if extracted != selected:
raise RuntimeError("Pinned SDK package is missing required oracle files")
with Path(os.environ["GITHUB_ENV"]).open("a") as environment:
environment.write("ANDROID_HOME=" + str(root) + "\n")
PYTHON
- run: npm run verify:apk-signature -- --require-tools
-22
View File
@@ -14,15 +14,6 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
permissions:
contents: read
@@ -30,13 +21,6 @@ permissions:
env:
NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
REA_VERIFY_LARGE_ELF_MCP: ${{ inputs.large_mcp && '1' || '0' }}
jobs:
@@ -55,12 +39,6 @@ jobs:
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- name: Install AArch64 and ELF32 cross toolchains for source-owned fixtures
run: |
sudo apt-get update
sudo apt-get install --yes --no-install-recommends \
gcc-aarch64-linux-gnu gcc-i686-linux-gnu gcc-arm-linux-gnueabi gcc-mips-linux-gnu gcc-mipsel-linux-gnu \
libc6-dev-arm64-cross libc6-dev-i386-cross libc6-dev-armel-cross libc6-dev-mips-cross libc6-dev-mipsel-cross
- name: Acquire exact upstream libraries in an isolated runner environment
shell: bash
run: |
+1 -23
View File
@@ -8,26 +8,10 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
permissions:
contents: read
@@ -48,13 +32,7 @@ jobs:
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
# Own a fresh bus so Chromium never depends on an inherited session address.
- run: dbus-run-session -- npm run verify:browser
env:
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
REA_BROWSER_NO_SANDBOX: "true"
# The preceding verifier builds the runtime used by this request journey.
- run: dbus-run-session -- node scripts/verify-browser-request.mjs
- run: npm run verify:browser
env:
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
REA_BROWSER_NO_SANDBOX: "true"
-8
View File
@@ -8,14 +8,6 @@ on:
permissions:
contents: read
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
jobs:
windows-native-artifact:
name: Build Windows bridge security boundary
-16
View File
@@ -9,15 +9,6 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
permissions:
contents: read
@@ -25,13 +16,6 @@ permissions:
env:
NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
jobs:
real-evm-interface:
@@ -6,11 +6,6 @@ on:
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
permissions:
contents: read
+12 -5
View File
@@ -13,8 +13,20 @@ on:
- "src/composition/go.ts"
- "src/cli/goCommands.ts"
- "src/server/registerGoTools.ts"
- "src/server/createServer.ts"
- "src/cli.ts"
- "src/cliCommandNames.ts"
- "src/contracts/toolContracts.ts"
- "src/contracts/toolEffects.ts"
- "src/artifacts/readStableArtifact*.ts"
- "src/artifacts/ArtifactReader.ts"
- "src/process/PrivateRuntimeRoot.ts"
- "scripts/verify-go-binary.mjs"
- "scripts/lib/mcp-verifier-results.mjs"
- "scripts/rea.mjs"
- "package.json"
- "package-lock.json"
- ".nvmrc"
permissions:
contents: read
@@ -26,11 +38,6 @@ concurrency:
env:
NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
jobs:
real-go-binary:
-5
View File
@@ -5,11 +5,6 @@ on:
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
permissions:
contents: read
-5
View File
@@ -5,11 +5,6 @@ on:
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
permissions:
contents: read
@@ -8,15 +8,6 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
permissions:
@@ -24,13 +15,6 @@ permissions:
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
NODE_OPTIONS: "--max-old-space-size=768"
RAYON_NUM_THREADS: "1"
+36 -38
View File
@@ -1,35 +1,43 @@
name: Real packet capture
on:
workflow_call:
inputs:
build_cache_artifact:
description: Portable build cache artifact from the calling CI run.
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
pull_request:
paths:
- "tests/boundary/captures/**"
- "tests/boundary/cli/captureCleanup.test.ts"
- "tests/boundary/mcp/packetCaptureMcp.test.ts"
- "src/captures/**"
- "src/pcap/**"
- "src/domain/pcap/**"
- "src/domain/analysisView/pcapCaptureView.ts"
- "src/application/pcap/**"
- "src/contracts/pcap/**"
- "src/server/registerPcapTools.ts"
- "tests/boundary/pcap/**"
- "src/domain/captures/**"
- "src/application/captures/**"
- "src/contracts/captures/**"
- "src/composition/captures.ts"
- "src/cli/captureCommands.ts"
- "src/server/registerCaptureTools.ts"
- "src/server/createServer.ts"
- "src/server/sessionAvailabilityPolicy.ts"
- "src/application/CapabilityInventory.ts"
- "src/contracts/toolContracts.ts"
- "src/contracts/toolEffects.ts"
- "src/cli.ts"
- "src/cliCommandNames.ts"
- "src/process/**"
- "src/artifacts/readStableArtifact.ts"
- "scripts/verify-packet-capture.mjs"
- ".github/workflows/real-packet-capture.yml"
- "package*.json"
- ".nvmrc"
permissions:
contents: read
concurrency:
group: real-packet-capture-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
tshark:
runs-on: ubuntu-24.04
@@ -44,18 +52,8 @@ jobs:
uses: ./.github/actions/npm-cache
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- run: npm run build:cached
- name: Run focused tests and require Linux resource and cleanup cases
run: |
report=".cache/rea-packet-required-tests.json"
mkdir -p .cache
rm -f "$report"
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts tests/acceptance/pcapVerifier.test.ts
node scripts/ci/require-provider-tests.mjs packet "$report"
- run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
- run: node scripts/verify-packet-capture.mjs
env:
REA_TSHARK_COMMAND: /usr/bin/tshark
+5 -28
View File
@@ -9,15 +9,6 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
permissions:
contents: read
@@ -25,13 +16,6 @@ permissions:
env:
NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
UV_CONCURRENT_BUILDS: "1"
UV_CONCURRENT_INSTALLS: "1"
UV_PYTHON_DOWNLOADS: never
@@ -40,19 +24,7 @@ jobs:
real-recorded-crash:
runs-on: ubuntu-24.04
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
image: ["ubuntu:24.04", "debian:trixie-slim"]
container: ${{ matrix.image }}
steps:
- name: Install fixture and debugger prerequisites in the disposable container
shell: bash
run: |
apt-get update -qq
apt-get install --yes --no-install-recommends \
ca-certificates git python3-venv gcc libc6-dev strace gdb
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
@@ -64,6 +36,11 @@ jobs:
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- name: Install GDB on the disposable CI runner
shell: bash
run: |
sudo apt-get update -qq
sudo apt-get install --yes --no-install-recommends gdb
- name: Acquire exact unchanged upstreams in isolated runner storage
shell: bash
run: |
+29 -38
View File
@@ -1,35 +1,36 @@
name: Real signal recording
on:
workflow_call:
inputs:
build_cache_artifact:
description: Portable build cache artifact from the calling CI run.
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
pull_request:
paths:
- "tests/boundary/signals/**"
- "tests/boundary/cli/signalCleanup.test.ts"
- "tests/boundary/mcp/signalRecordingMcp.test.ts"
- "src/signals/**"
- "src/domain/signals/**"
- "src/application/signals/**"
- "src/contracts/signals/**"
- "src/composition/signals.ts"
- "src/cli/signalCommands.ts"
- "src/server/registerSignalTools.ts"
- "src/server/createServer.ts"
- "src/server/sessionAvailabilityPolicy.ts"
- "src/application/CapabilityInventory.ts"
- "src/contracts/toolContracts.ts"
- "src/contracts/toolEffects.ts"
- "src/cli.ts"
- "src/cliCommandNames.ts"
- "src/process/**"
- "src/artifacts/readStableArtifact.ts"
- "scripts/verify-signal-recording.mjs"
- ".github/workflows/real-signal-recording.yml"
- "package*.json"
- ".nvmrc"
permissions:
contents: read
concurrency:
group: real-signal-recording-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
sigrok:
runs-on: ubuntu-24.04
@@ -44,18 +45,8 @@ jobs:
uses: ./.github/actions/npm-cache
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- run: npm run build:cached
- name: Run focused tests and require Linux resource and cleanup cases
run: |
report=".cache/rea-signal-required-tests.json"
mkdir -p .cache
rm -f "$report"
node scripts/run-exact-tests.mjs run --reporter=default --reporter=json --outputFile.json="$report" \
tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
node scripts/ci/require-provider-tests.mjs signal "$report"
- run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
- run: node scripts/verify-signal-recording.mjs
env:
REA_SIGROK_COMMAND: /usr/bin/sigrok-cli
-5
View File
@@ -12,11 +12,6 @@ permissions:
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
jobs:
real-termux:
@@ -9,28 +9,12 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
permissions:
contents: read
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
NODE_OPTIONS: --max-old-space-size=768
jobs:
+1 -18
View File
@@ -7,27 +7,11 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
permissions:
contents: read
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
jobs:
real-web-runtime:
runs-on: ubuntu-latest
@@ -44,8 +28,7 @@ jobs:
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
# Own a fresh bus so Chromium never depends on an inherited session address.
- run: dbus-run-session -- npm run verify:browser:runtime
- run: env -u DBUS_SESSION_BUS_ADDRESS npm run verify:browser:runtime
env:
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
REA_BROWSER_NO_SANDBOX: "true"
+1 -21
View File
@@ -7,27 +7,11 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
permissions:
contents: read
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
jobs:
real-web-source-map:
runs-on: ubuntu-latest
@@ -46,12 +30,8 @@ jobs:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- name: Install the pinned compiler into an isolated fixture directory
run: npm install --prefix "$RUNNER_TEMP/rea-web-source-map-compiler" --no-audit --no-fund --package-lock=false [email protected]
# Own a fresh bus so Chromium never depends on an inherited session address.
- run: dbus-run-session -- npm run verify:browser:source-maps
- run: npm run verify:browser:source-maps
env:
# Retain browser launch argv/PID/stderr in the verifier's captured
# CLI failure diagnostics without changing its deadline or retrying.
DEBUG: pw:browser
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
REA_BROWSER_NO_SANDBOX: "true"
REA_WEB_SOURCE_MAP_COMPILER: ${{ runner.temp }}/rea-web-source-map-compiler/node_modules/esbuild/lib/main.js
+40 -127
View File
@@ -21,12 +21,6 @@ on:
env:
HUSKY: "0"
# Remote cache stays local when either value is empty.
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
concurrency:
group: release-${{ inputs.release_branch || github.event.pull_request.merge_commit_sha || github.ref_name }}
@@ -47,63 +41,19 @@ jobs:
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
GH_REPO: ${{ github.repository }}
run: |
# Filter at the repository connection, without a search-index dependency.
pages=$(gh api graphql --paginate --slurp \
-f owner="${GH_REPO%/*}" -f name="${GH_REPO#*/}" \
-f query='query($owner:String!,$name:String!,$endCursor:String) {
repository(owner:$owner,name:$name) {
nameWithOwner
pullRequests(first:100,after:$endCursor,states:OPEN,baseRefName:"main",labels:["rea:release-notes-finalized"]) {
nodes { number headRefName headRepository { nameWithOwner } }
pageInfo { hasNextPage endCursor }
}
}
}')
finalized=$(jq -er '
if type != "array" or length == 0 then error("Missing GitHub pages") else . end |
if .[-1].data.repository.pullRequests.pageInfo.hasNextPage != false
then error("Incomplete GitHub pagination") else . end |
map(
if (.errors // [] | length) != 0 then error("GitHub query failed") else . end |
.data.repository |
if (.nameWithOwner | type) != "string" or (.pullRequests.nodes | type) != "array" or
(.pullRequests.pageInfo.hasNextPage | type) != "boolean"
then error("Invalid GitHub repository connection") else . end |
.nameWithOwner as $repo | .pullRequests.nodes[] |
if (.number | type) != "number" or (.headRefName | type) != "string" or (has("headRepository") | not) or
(.headRepository != null and (.headRepository.nameWithOwner | type) != "string")
then error("Invalid GitHub pull request") else . end |
select(.headRepository.nameWithOwner == $repo) |
select(.headRefName | startswith("release-please--branches--main--")) | .number
) | map(tostring) | join(" ")' <<< "$pages")
finalized=$(gh api --paginate "repos/${GH_REPO}/pulls?state=open&base=main&per_page=100" \
--jq '.[] | select(.head.repo.full_name == .base.repo.full_name) | select(.head.ref | startswith("release-please--branches--main--")) | select(any(.labels[]; .name == "rea:release-notes-finalized")) | .number')
if [[ -n "${finalized}" ]]; then
echo "Finalized release notes on PR ${finalized}; automatic proposal refresh is paused." >> "$GITHUB_STEP_SUMMARY"
echo "frozen=true" >> "$GITHUB_OUTPUT"
fi
- name: Check out proposal controller and complete Git history
if: steps.editorial.outputs.frozen != 'true'
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
with:
fetch-depth: 0
persist-credentials: false
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
if: steps.editorial.outputs.frozen != 'true'
with:
node-version-file: .nvmrc
package-manager-cache: false
- name: Install proposal controller dependencies
if: steps.editorial.outputs.frozen != 'true'
run: npm ci
- name: Update release proposal
if: steps.editorial.outputs.frozen != 'true'
env:
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
SOURCE_SHA: ${{ github.sha }}
run: node scripts/release-proposal.mjs --target-branch main --source-sha "$SOURCE_SHA"
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
target-branch: main
skip-github-release: true
merged-release:
name: Release reviewed main merge
@@ -276,25 +226,15 @@ jobs:
node scripts/release-notes.mjs publication --repository "$GH_REPO" \
--source "$SOURCE_SHA"
- name: Prepare checkpoint release proposal
if: inputs.phase == 'prepare'
id: proposal
env:
# A dedicated credential starts PR CI automatically; the built-in
# token creates approval-required PR runs for maintainers to approve.
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
RELEASE_BRANCH: ${{ inputs.release_branch }}
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
run: node scripts/release-proposal.mjs --target-branch "$RELEASE_BRANCH" --source-sha "$SOURCE_SHA"
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
if: inputs.phase == 'publish'
id: release
with:
# A dedicated credential starts PR CI automatically; the built-in
# token creates approval-required PR runs for maintainers to approve.
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
target-branch: ${{ inputs.release_branch }}
skip-github-release: false
skip-github-pull-request: true
skip-github-release: ${{ inputs.phase == 'prepare' }}
skip-github-pull-request: ${{ inputs.phase == 'publish' }}
- name: Require a merged release PR for publication
if: inputs.phase == 'publish' && steps.release.outputs.release_created != 'true'
@@ -314,26 +254,26 @@ jobs:
fi
- name: Check out release pull request
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
with:
ref: ${{ fromJSON(steps.proposal.outputs.pr).headBranchName }}
ref: ${{ fromJSON(steps.release.outputs.pr).headBranchName }}
fetch-depth: 0
persist-credentials: false
- name: Set up Node.js for generated documentation
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
with:
node-version-file: .nvmrc
package-manager-cache: false
- name: Install dependencies
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
run: npm ci
- name: Validate prepared release version and migration notes
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
env:
RELEASE_BRANCH: ${{ inputs.release_branch }}
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
@@ -343,11 +283,11 @@ jobs:
--source-sha "$SOURCE_SHA"
- name: Regenerate release documentation
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
run: npm run docs:generate
- name: Validate generated release documentation
if: inputs.phase == 'prepare' && steps.proposal.outputs.prs_created == 'true'
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
run: npm run docs:check && git diff --exit-code
qualification:
@@ -361,17 +301,11 @@ jobs:
contents: read
actions: read
uses: ./.github/workflows/ci.yml
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
with:
source_sha: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
remote_cache_write: true
rehearsal:
name: Rehearse qualified release on a clean runner
env:
TURBO_TOKEN: ""
TURBO_CACHE: local:rw
needs: [merged-release, release-please, qualification]
if: >-
always() && !cancelled() && needs.qualification.result == 'success'
@@ -405,9 +339,6 @@ jobs:
always() && !cancelled() &&
needs.qualification.result == 'success' && needs.rehearsal.result == 'success'
runs-on: ubuntu-latest
env:
TURBO_TOKEN: ""
TURBO_CACHE: local:rw
permissions:
contents: read
id-token: write
@@ -416,56 +347,41 @@ jobs:
with:
ref: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
persist-credentials: false
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
with:
node-version-file: .nvmrc
package-manager-cache: false
registry-url: https://registry.npmjs.org
- name: Download the original qualified package
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: verified-release-package
path: .cache/verified-package/
- name: Bind publication to qualified intent
id: package
run: node scripts/ci/prepare-npm-publication.mjs
- name: Publish
if: steps.package.outputs.published == 'false'
env:
NPM_RELEASE_TAG: ${{ steps.package.outputs.tag }}
run: >-
npm publish .cache/verified-package/package.tgz --ignore-scripts
--registry=https://registry.npmjs.org --access=public --tag="$NPM_RELEASE_TAG"
verify-npm:
name: Verify published npm package
needs: [merged-release, release-please, publish]
if: >-
always() && !cancelled() && needs.publish.result == 'success'
runs-on: ubuntu-latest
env:
TURBO_TOKEN: ""
TURBO_CACHE: local:rw
permissions:
contents: read
actions: read
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
with:
ref: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
persist-credentials: false
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
with:
node-version-file: .nvmrc
package-manager-cache: false
- run: npm ci
- uses: ./.github/actions/prepare-release-package
id: package
- name: Inspect existing npm version
id: registry
env:
VERIFIED_PACKAGE_TARBALL: ${{ steps.package.outputs.tarball }}
VERIFIED_PACKAGE_VERSION: ${{ steps.package.outputs.version }}
run: node scripts/ci/npm-release.mjs inspect "$VERIFIED_PACKAGE_TARBALL" "$VERIFIED_PACKAGE_VERSION"
- name: Publish
if: steps.registry.outputs.published == 'false'
shell: bash
env:
VERIFIED_PACKAGE_TARBALL: ${{ steps.package.outputs.tarball }}
VERIFIED_PACKAGE_VERSION: ${{ steps.package.outputs.version }}
run: |
tag="$(node scripts/release-npm-tag.mjs "$VERIFIED_PACKAGE_VERSION")"
npm publish "${VERIFIED_PACKAGE_TARBALL}" --access public --tag "${tag}"
- name: Verify published npm bytes and dist-tag
env:
VERIFIED_PACKAGE_TARBALL: ${{ steps.package.outputs.tarball }}
VERIFIED_PACKAGE_VERSION: ${{ steps.package.outputs.version }}
run: node scripts/ci/npm-release.mjs verify "$VERIFIED_PACKAGE_TARBALL" "$VERIFIED_PACKAGE_VERSION"
- name: Verify published CLI from npm
env:
VERIFIED_PACKAGE_VERSION: ${{ steps.package.outputs.version }}
@@ -473,12 +389,9 @@ jobs:
publish-mcp:
name: Publish to MCP Registry
env:
TURBO_TOKEN: ""
TURBO_CACHE: local:rw
needs: [merged-release, release-please, verify-npm]
needs: [merged-release, release-please, publish]
if: >-
always() && !cancelled() && needs.verify-npm.result == 'success' &&
always() && !cancelled() && needs.publish.result == 'success' &&
((needs.merged-release.result == 'success' && needs.merged-release.outputs.release_created == 'true') ||
(needs.release-please.result == 'success' && inputs.phase == 'publish' && needs.release-please.outputs.release_created == 'true'))
runs-on: ubuntu-latest
+3 -3
View File
@@ -17,10 +17,10 @@ jobs:
strategy:
fail-fast: false
matrix:
# Exercise unavailable-runtime diagnostics and rolling supported releases.
# Security floors prevent fallback to an older hosted-toolcache patch.
# Exercise unavailable-runtime diagnostics, the first supported release,
# and new patches on both supported lines without hand-editing versions.
# The repository-pinned Node 24 version also runs the complete test suite.
node: ["^22.23.4", "24.22.1", "^24.22.1", "^26.12.1"]
node: ["22.x", "24.15.0", "24.x", "26.x"]
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
-4
View File
@@ -21,10 +21,6 @@ jobs:
python-version: "3.12"
cache: pip
cache-dependency-path: website/requirements.txt
- name: Set up Node.js for the website Worker checks
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Install sharing-image dependencies
run: |
sudo apt-get update
+8 -50
View File
@@ -54,10 +54,6 @@ jobs:
python-version: "3.12"
cache: pip
cache-dependency-path: website/requirements.txt
- name: Set up Node.js for the website Worker checks
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Install sharing-image dependencies
run: |
sudo apt-get update
@@ -79,21 +75,10 @@ jobs:
PY
- name: Configure Pages
uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
# upload-pages-artifact excludes dot directories, including .well-known.
- name: Package the website including agent discovery
run: |
tar --dereference --hard-dereference --directory website/_site \
--exclude=.git --exclude=.github -cf "$RUNNER_TEMP/artifact.tar" .
tar -tf "$RUNNER_TEMP/artifact.tar" \
./.well-known/ard.json ./.well-known/ai-catalog.json \
./.well-known/agent-skills/index.json > /dev/null
- name: Upload the website
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0
with:
name: github-pages
path: ${{ runner.temp }}/artifact.tar
retention-days: 1
if-no-files-found: error
path: website/_site
deploy:
needs: prepare
@@ -145,24 +130,14 @@ jobs:
PUBLISH_TARGET: ${{ inputs.target }}
run: |
python3 -u - <<'PY'
import hashlib
import json
import os
import sys
import time
from urllib.error import URLError
from urllib.parse import urljoin, urlsplit
from urllib.parse import urljoin
from urllib.request import Request, urlopen
def open_public(url, accept=None):
headers = {
"Cache-Control": "no-cache",
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
}
if accept:
headers["Accept"] = accept
return urlopen(Request(url, headers=headers), timeout=10)
expected = os.environ["GITHUB_SHA"]
pages_url = os.environ["PAGES_URL"]
if not pages_url:
@@ -178,7 +153,11 @@ jobs:
for name, base in list(pending.items()):
url = urljoin(base, "deployment.json") + "?commit=" + expected
try:
with open_public(url) as response:
request = Request(url, headers={
"Cache-Control": "no-cache",
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
})
with urlopen(request, timeout=10) as response:
actual = json.load(response).get("commit")
if actual == expected:
print(f"{name} serves {expected}: {base}")
@@ -193,27 +172,6 @@ jobs:
time.sleep(5)
if pending:
sys.exit("Could not verify published revision on: " + ", ".join(pending))
for name, base in hosts.items():
with open_public(urljoin(base, ".well-known/agent-skills/index.json")) as response:
skill, = json.load(response)["skills"]
# Fetch from each selected host, including the /rea/ mirror.
artifact_url = urljoin(base, urlsplit(skill["url"]).path.lstrip("/"))
with open_public(artifact_url) as response:
digest = "sha256:" + hashlib.sha256(response.read()).hexdigest()
if digest != skill["digest"]:
sys.exit(name + " serves a skill archive with the wrong digest")
for manifest in ("ard.json", "ai-catalog.json"):
with open_public(urljoin(base, ".well-known/" + manifest)) as response:
if not json.load(response).get("entries"):
sys.exit(name + " serves an empty discovery manifest")
if name == "Cloudflare":
with open_public(base, accept="text/markdown") as response:
vary = {value.strip().lower() for value in response.headers.get("Vary", "").split(",")}
if response.headers.get_content_type() != "text/markdown" or "accept" not in vary:
sys.exit("Cloudflare did not negotiate Markdown with Vary: Accept")
if 'rel="ard"' not in response.headers.get("Link", ""):
sys.exit("Cloudflare is missing its discovery Link header")
print(name + " serves the discovery manifests and verified skill archive")
with open(os.environ["GITHUB_STEP_SUMMARY"], "a", encoding="utf-8") as summary:
links = " and ".join(f"[{name}]({url})" for name, url in hosts.items())
summary.write(f"Published commit `{expected}` to {links}.\n")
+1 -1
View File
@@ -50,7 +50,7 @@ Thumbs.db
# Local reference sources and analysis artifacts
/_reference/
/experiments/analysis-efficiency/
/experiments/analysis-efficiency/results/
# Generated website download
/website/public/examples/notes-example.zip
+1 -1
View File
@@ -1,5 +1,5 @@
{
"threshold": 4,
"threshold": 3.5,
"reporters": ["consoleFull"],
"ignore": [
"node_modules/**",
+1 -1
View File
@@ -1 +1 @@
24.22.1
24.18.0
+1 -4
View File
@@ -10,10 +10,7 @@
"node_modules/**",
"CHANGELOG.md",
"package-lock.json",
".claude-plugin/plugin.json",
".codex-plugin/plugin.json",
"_reference/**",
"third_party/**",
".codex-plugin/mcp.json"
"third_party/**"
]
}
+3 -95
View File
@@ -1,13 +1,7 @@
{
"$schema": "./node_modules/oxlint/configuration_schema.json",
"plugins": ["typescript", "import", "unicorn", "oxc", "vitest"],
"plugins": ["typescript", "import", "unicorn", "oxc"],
"rules": {
"rea/no-buffer-deep-equality": "error",
"rea/cancellation-cause": "error",
"no-async-promise-executor": "error",
"no-unreachable": "error",
"no-unsafe-optional-chaining": "error",
"no-useless-escape": ["warn", { "allowRegexCharacters": ["-"] }],
"complexity": ["warn", 20],
"max-depth": ["warn", 4],
"max-lines": [
@@ -29,38 +23,7 @@
"max-params": ["warn", 4],
"no-var": "warn",
"no-var-requires": "error",
"typescript/no-import-type-side-effects": "error",
"vitest/no-standalone-expect": "off",
"vitest/no-conditional-expect": "off",
"vitest/expect-expect": "off",
"vitest/no-disabled-tests": "off",
"vitest/require-mock-type-parameters": "off",
"vitest/require-to-throw-message": "off",
"import/no-duplicates": [
"warn",
{
"preferInline": false
}
],
"vitest/valid-expect": [
"warn",
{
"maxArgs": 2
}
],
"rea/no-evidence-json-preadmission": "error",
"rea/no-positional-tool-contract": "error",
"rea/no-fixture-root-cleanup": "error",
"typescript/no-floating-promises": "error",
"typescript/no-misused-promises": "warn",
"typescript/return-await": ["error", "error-handling-correctness-only"],
"typescript/switch-exhaustiveness-check": [
"error",
{
"considerDefaultExhaustiveForUnions": true
}
],
"rea/no-shared-module-mock": "error"
"typescript/no-import-type-side-effects": "error"
},
"overrides": [
{
@@ -73,12 +36,6 @@
"no-var-requires": "off"
}
},
{
"files": ["src/**/*.ts"],
"rules": {
"typescript/no-misused-promises": "error"
}
},
{
"files": ["tests/**/*.ts", "src/**/*.test.ts", "src/**/*.fixture.ts"],
"rules": {
@@ -90,8 +47,7 @@
"skipBlankLines": true,
"skipComments": true
}
],
"typescript/no-misused-promises": "warn"
]
}
},
{
@@ -135,45 +91,6 @@
}
]
}
},
{
"files": ["tests/**/*.test.ts", "src/**/*.test.ts"],
"rules": {
"vitest/valid-expect": [
"warn",
{
"maxArgs": 2
}
],
"vitest/valid-expect-in-promise": "error",
"vitest/no-focused-tests": "error"
}
},
{
"files": [
"scripts/lib/http3-capture-assertions.test.mjs",
"scripts/lib/http3-verifier-lifecycle.test.mjs"
],
"rules": {
"typescript/no-floating-promises": [
"error",
{
"allowForKnownSafeCalls": [
{
"from": "package",
"package": "@types/node",
"name": "test"
}
]
}
]
}
},
{
"files": ["scripts/fixtures/javascript-recovery/code-split/entry.mjs"],
"rules": {
"typescript/no-floating-promises": "off"
}
}
],
"env": {
@@ -192,14 +109,5 @@
".devin",
".factory",
"third_party"
],
"options": {
"reportUnusedDisableDirectives": "error"
},
"jsPlugins": [
{
"name": "rea",
"specifier": "./scripts/lint/index.mjs"
}
]
}
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "6.8.0"
".": "6.5.0"
}
+13 -40
View File
@@ -14,24 +14,18 @@ Preserve caller-selected inputs and local evidence, including output, URLs, path
## Architecture and Cleanup
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture/architecture.mermaid](docs/architecture/architecture.mermaid) when changing composition.
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture.mermaid](docs/architecture.mermaid) when changing composition.
Give each interpretation and resource a clear owner. Consumers must use that owner's parsed facts rather than reinterpret raw source. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
For lifecycle changes, follow [ownership and completion invariants](docs/development/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/development/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
Before adding complexity for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode, likelihood, and complexity cost to the user and get their buy-in. Do not silently expand scope for theoretical robustness.
For lifecycle changes, follow [ownership and completion invariants](docs/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
Assess brittleness in touched code during implementation, debugging, review, and maintenance. Fix the producing representation or ownership rule before adding downstream repairs. Normalize once at the owning boundary using format-aware parsers. Remove superseded representations, aliases, wrappers, and redundant tests after verifying affected consumers. Introduce abstractions only when they remove observed complexity.
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
After confirming a bug, search sibling adapters, consumers and similar workflows
for the same failure. Verify suspected matches, record working comparisons, and
group findings by root cause instead of filing duplicate reports.
When addressing a review thread, inspect the producing boundary and affected consumers for brittleness. Complete directly related refactoring in the same change. Before resolving the thread, explain the root cause, structural fix, validation, and any remaining cleanup.
Parse unknown boundary values and model expected failures with the tagged error algebra and `Result`. Preserve partial facts and meaningful failure reasons. Keep identity and lookup values separate from display formatting; preserve source values and explicit unknowns when normalization loses information. Absolute paths, file URLs, and HTTP paths have different semantics.
@@ -57,24 +51,15 @@ Optimize in this order:
3. CPU and memory. Avoid unnecessary work and copies; bound concurrency,
retained state and serialization growth.
Reserve performance measurements for changes that target performance or have a
concrete regression concern. Routine cleanup uses correctness checks; changes
to helper calls, imports, or representations alone do not require synthetic
benchmarks. When measurement is warranted, measure representative completed
tasks and compare equivalent workflows before and after where available. Report
token usage, elapsed time, CPU time and peak process-family memory. Separate cold
For changes affecting these costs, measure representative completed tasks and
compare equivalent workflows before and after where available. Report token
usage, elapsed time, CPU time and peak process-family memory. Separate cold
and warm runs, label estimates, and state unmeasured costs. Fewer bytes alone
do not establish fewer tokens or faster analysis.
## Development
Documentation ownership and authoring conventions live in
[docs/development/documentation.md](docs/development/documentation.md). A feature
uses `docs/specs/<domain>/<feature-slug>/spec.md`, one `plan.md` and `tasks.md`;
individual architectural decisions use `docs/adr/<domain>/<decision-slug>.md`.
Consult the [documentation directory](docs/README.md) before adding a guide.
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`. Prefix `PATH="/usr/local/bin:$PATH"` before `node` and `npm`.
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`.
- `npm ci`: install locked dependencies.
- `npm run build:cached`: build the runtime and packaged skill.
@@ -85,24 +70,13 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
When changing installers or setup recovery, follow
[Checking installers and publisher downloads](docs/development/testing.md#checking-installers-and-publisher-downloads).
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/development/testing.md#developer-commands).
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
## Tool and Test Changes
For tool design, follow [docs/development/tool-design.md](docs/development/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
For user- or agent-facing tool problems, prioritize fixing the owning behavior,
contract, and actionable result or recovery at the point of use. Document the
result in product docs when explanation is needed. Add skill guidance only for
workflow judgment or context that the tool cannot reasonably convey; do not use
a skill as the primary repair for a discoverability or contract defect.
For test changes or provider claims, use [docs/development/testing.md](docs/development/testing.md). Select applicable [boundary regressions](docs/development/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
@@ -130,12 +104,11 @@ distinguish source-derived conclusions, proposed checks, and unavailable coverag
Address actionable automated review findings and explain disagreements.
Leave the template's **Author readiness** declarations unchecked for the human
owner to complete. Never attest to human review on their behalf. Draft status
is optional; preserve the PR's current draft status unless the user asks to change
it. Later pushes do not invalidate the readiness declaration or require another
human sign-off. Keep the PR body and affected validation current as work changes.
The readiness check validates the filled declaration or the author exemption
listed in CONTRIBUTING.md; CI and automated review results do not establish that
owner to complete. Never attest to human review on their behalf. Keep the PR in
draft until the human declares the PR initially ready. Later pushes do not
invalidate that declaration or require another human sign-off. Keep the PR body
and affected validation current as work changes. The readiness check validates
the filled declaration; CI and automated review results do not establish that
human review occurred, maintainer scope approval, or merge permission.
<!-- BEGIN:turborepo-agent-rules -->
+7 -197
View File
@@ -1,198 +1,8 @@
# Changelog
## [6.8.0](https://github.com/morluto/rea/compare/rea-agents-6.7.0...rea-agents-6.8.0) (2026-10-25)
### Highlights
- Build local agent environments from network captures, browser scenarios, or a prompt, then serve derived reads and evidence-gated writes and compare them with fresh application captures.
- Inspect more shipped formats, including GPU code objects, BEAM, Python bytecode, Unity IL2CPP metadata, Godot PCK, PyInstaller, Node.js SEA, devicetree, and additional PE/ELF tables.
- Keep large investigations focused with retained summaries, selected fact queries, configurable recovery limits, and row-oriented CLI exports. Complete Evidence remains available.
### ⚠ BREAKING CHANGES
- **Function analysis:** `analyze_function` now defaults to a retained summary. Set `detail: "complete"` for the previous inline dossier; the CLI retains its complete default and requires `--evidence_output` when selecting a summary. [#5660](https://github.com/morluto/rea/pull/5660). Thanks @CryptoJones, @gensokyo-dev, @RishTheFish21.
- **Accessibility:** Accept JSON numbers and booleans in node `value`, alongside strings and null. Update consumers that previously assumed text-only values. [cc90771](https://github.com/morluto/rea/commit/cc90771af14f045538651615147d8bf86b576549), [#6759](https://github.com/morluto/rea/pull/6759), [#6467](https://github.com/morluto/rea/issues/6467). Thanks @coygeek, @gensokyo-dev, @mvanhorn.
- **JSON Lines:** `--format jsonl` now emits object-collection rows followed by a `$rea: "jsonl/v1"`, `type: "metadata"` record; scalar selections and unsupported reports are refused. Parse control records separately, select a single supported collection when necessary, and use `--format json` for the previous whole-document output. See [JSON Lines framing](https://github.com/morluto/rea/blob/rea-agents-6.8.0/docs/cli.md#json-lines-record-output). [#5629](https://github.com/morluto/rea/pull/5629), [#5487](https://github.com/morluto/rea/issues/5487). Thanks @gensokyo-dev, @jayykernel, @vijaykiran06.
- **Interface Builder:** Keyed-archive graph identities use decimal table UIDs instead of authored `objectID`/`object-id`. Treat authored IDs as attributes and update persisted graph lookups; ibtool and native NIBArchive identities keep their source semantics. [6fbe67d](https://github.com/morluto/rea/commit/6fbe67d66e3d716db69929503a3a232a7ada0860).
- **Frida:** `instrument-with-frida` JSON returns the canonical Evidence directly. Read `normalized_result.cleanup_error` instead of the former top-level wrapper field. [48f700e](https://github.com/morluto/rea/commit/48f700ee35e0988ed973e5486dc9db33f2a5d944).
- **Reconstruction proof:** ID-array proof records no longer establish verified or ready ledgers. Supply implementation/revision bindings and individual claim outcomes; handle inapplicable, unavailable, and failed proof diagnostics. [f8b65e7](https://github.com/morluto/rea/commit/f8b65e78723b1e2169f94dd1226198ea3139707e).
- **Mach-O imports:** Handle a null import `name` when identity is ambiguous; use `raw_name` and `name_status` to distinguish observed expressions from exact symbols. [5d0d243](https://github.com/morluto/rea/commit/5d0d243a45c9132e1aff75d454ab53d86a74cc31), [169512a](https://github.com/morluto/rea/commit/169512a35bb015ff209168fb8bc1c9b31a2c678b).
- **Node.js:** Use a runtime satisfying `^22.23.4 || ^24.22.1 || >=26.12.1`; Node 23 and 25 are unsupported. Upgrade Node before installing or updating REA; the installer does not upgrade it. [#6315](https://github.com/morluto/rea/pull/6315), [#6283](https://github.com/morluto/rea/issues/6283). Thanks @GamerDubz, @gensokyo-dev, @ShikamaruNaaraa.
- **Browser storage:** Historical hashed storage remains readable, but comparisons stay unknown until both captures are recaptured in the new representation. [#6341](https://github.com/morluto/rea/pull/6341), [#6289](https://github.com/morluto/rea/issues/6289). Thanks @coygeek, @gongfpp.
- **Environment serving:** `environment-serve` serves derived state by default, including write tools whose effects are confirmed by recorded evidence. Use `--replay-only` for stateless replay of manifest tools. Read stop-summary `state` instead of `state_evidence_id`. [#5351](https://github.com/morluto/rea/pull/5351), [#6158](https://github.com/morluto/rea/pull/6158). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
- **MCP schemas:** Input advertisements default to compact. Set `REA_MCP_INPUT_SCHEMA_PROFILE=full` for complete advertised guidance and nested constraints. [#5975](https://github.com/morluto/rea/pull/5975). Thanks @gensokyo-dev.
- **Browser console arguments:** Accept null `argument_types` and `text_capture.status: "unavailable"` when the protocol argument inventory is omitted or malformed; preserve that uncertainty. [#6016](https://github.com/morluto/rea/pull/6016), [#5934](https://github.com/morluto/rea/issues/5934). Thanks @coygeek, @kaoru0822-kitauji.
- **Reconstruction summaries:** `summary.by_evidence_authority` is keyed by Evidence authority; read the former source-state counts from required `summary.by_source_state`. Regenerate ledgers whose closure digests depended on the old projection. [#5875](https://github.com/morluto/rea/pull/5875), [#5838](https://github.com/morluto/rea/issues/5838). Thanks @coygeek, @kaoru0822-kitauji.
- **Swift bindings:** Conformance `type_name` and `protocol_name` may be null; consume typed `type_binding`/`protocol_binding` evidence. Replace the former Objective-C-specific import-binding type/schema with `MachoImportBinding`. [d0067ba](https://github.com/morluto/rea/commit/d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f).
- **WebSocket frames:** Accept null opcodes and preserve unknown frame facts; numeric opcodes are restricted to the four-bit protocol range. [0bc2b42](https://github.com/morluto/rea/commit/0bc2b42001f684beffcf47b70e507443b1a5d320).
- **Resource-only inspection:** Evidence subjects and executable digests may be null. Use `resource_root` and each resource's reported provenance instead of requiring an executable target. [6a6adf7](https://github.com/morluto/rea/commit/6a6adf703fa44100643947d20fbc670f86260b1b).
- **Browser timestamps:** Console and session-timeline timestamps may be null when the protocol supplies no finite/comparable time. Keep unknown time explicit in sorting and comparisons. [690fc51](https://github.com/morluto/rea/commit/690fc518540bae982038a74f47ef7781201785aa), [2a0410f](https://github.com/morluto/rea/commit/2a0410f1717b3126e8a95063d77dea66a597010a).
- **Reference sources:** Read failed root enumeration from `rootFailure` rather than a synthetic failed child at `"."`; preserve the top-level source-graph limitation. [6296fdf](https://github.com/morluto/rea/commit/6296fdf560f1ac4aa55985558edaa98300aacdbc).
- **Filesystem capture:** Accept `entry_unavailable` and `identity_changed` enumeration omission reasons in addition to the previous reason literals. [c85b44e](https://github.com/morluto/rea/commit/c85b44e1b14851c54e65327a4807f79ad5cd5a61). Thanks @rudrakshagarwal4200-ai.
- **APK locales:** Replace numeric legacy selectors such as `es-r419` with `es-419` or `b+es+419`. Two-letter regions remain supported. [29910d1](https://github.com/morluto/rea/commit/29910d16b98fe4b607a7aa15928789a73614b0b9). Thanks @mertkayacs.
- **XML plist scalars:** Recognize `$plist_type: "unknown"` and read `element`, `literal`, and `reason`; unresolved XML scalars use this wrapper instead of the former real/null unknown marker or malformed scalar prefixes. Supported extended UTC years preserve their timestamp. [3537e58](https://github.com/morluto/rea/commit/3537e58c1ab29309f06eabf50bc52adba9817a0c).
### Changes
- **Binary layouts:** Inspect ELF32 i386, little-endian ARM, and little- or big-endian MIPS `EXEC`/`DYN`/`REL` layouts on Linux x64 using a caller-supplied pwntools environment. [#7052](https://github.com/morluto/rea/pull/7052), [#6934](https://github.com/morluto/rea/issues/6934). Thanks @roblennon, @salmanalmutairi1.
- **Agent environments:** Project network captures and Evidence files into manifests, preserve template path pairs and positions, export AgentEnv bundles, and serve manifests as replay environments. [#4954](https://github.com/morluto/rea/pull/4954), [#5293](https://github.com/morluto/rea/pull/5293), [#5664](https://github.com/morluto/rea/pull/5664), [#4999](https://github.com/morluto/rea/pull/4999), [#4975](https://github.com/morluto/rea/pull/4975). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
- **Capture to environment:** Build environments from browser scenarios or a prompt, extract the initial observed state, and compare the environment with application probe captures. [#5182](https://github.com/morluto/rea/pull/5182), [#6946](https://github.com/morluto/rea/pull/6946), [#5319](https://github.com/morluto/rea/pull/5319), [#5699](https://github.com/morluto/rea/pull/5699). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
- **State-backed tools:** Render read filters, search, sorting and paging from state; apply confirmed write effects with broader state targets and recording-based write admission. [#5352](https://github.com/morluto/rea/pull/5352), [293832b](https://github.com/morluto/rea/commit/293832bafc9a054914ec514fe906dd4936c7a372), [#5800](https://github.com/morluto/rea/pull/5800), [#5787](https://github.com/morluto/rea/pull/5787), [#5600](https://github.com/morluto/rea/pull/5600), [5375ff0](https://github.com/morluto/rea/commit/5375ff0f9aadb4fd0d46333815aa0431e3df1010), [#6169](https://github.com/morluto/rea/pull/6169), [#6385](https://github.com/morluto/rea/pull/6385), [#6897](https://github.com/morluto/rea/pull/6897). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
- **State learning:** Model application-assigned values, preserve view-specific clock spellings, learn write clocks from scenario timing, derive fields, and learn creates across sorted arrays and referenced entities. [7ec9b6e](https://github.com/morluto/rea/commit/7ec9b6e56e1b3762285dca881db6d900a389ab9c), [#6136](https://github.com/morluto/rea/pull/6136), [957bcfa](https://github.com/morluto/rea/commit/957bcfa149c1a0593bb0aa2466cc56d1d4f62e7d), [#6138](https://github.com/morluto/rea/pull/6138), [#6967](https://github.com/morluto/rea/pull/6967), [#7025](https://github.com/morluto/rea/pull/7025), [#7034](https://github.com/morluto/rea/pull/7034). Thanks @earakely-scale.
- **Recorded writes:** Serve creates whose compatible read-backs add omitted fields or express timestamps at a different precision, and whole-object updates that regenerate observed slugs or modification times. Conflicting observations still withhold writes. State views and bundles carrying the new read-back data need REA 6.8.0 or later to serve. [#7080](https://github.com/morluto/rea/pull/7080). Thanks @earakely-scale.
- **Native investigations:** Page native inventory summaries, match functions across builds, export pseudocode to resumable directories, and select objdump disassembly by symbol. [#5378](https://github.com/morluto/rea/pull/5378), [#5390](https://github.com/morluto/rea/pull/5390), [#6170](https://github.com/morluto/rea/pull/6170), [#5171](https://github.com/morluto/rea/pull/5171), [#4249](https://github.com/morluto/rea/issues/4249), [#5343](https://github.com/morluto/rea/issues/5343), [#5159](https://github.com/morluto/rea/issues/5159). Thanks @coygeek, @CryptoJones, @gensokyo-dev, @kaoru0822-kitauji.
- **Engine workflows:** Bound a single Ghidra decompilation without closing its session, analyze GameCube/Wii DOL executables, and inspect static CUDA/HIP code objects. [#5662](https://github.com/morluto/rea/pull/5662), [#6240](https://github.com/morluto/rea/pull/6240), [#5813](https://github.com/morluto/rea/pull/5813), [#5340](https://github.com/morluto/rea/issues/5340), [#5345](https://github.com/morluto/rea/issues/5345). Thanks @arielfikru, @coygeek, @gensokyo-dev, @RishTheFish21.
- **PE/ELF:** Inspect export tables without an engine, PE imports/delay-loads, Authenticode certificates, TLS callbacks/load configuration, ELF notes, Linux kernel module metadata, and Zstandard section-name tables. [#6278](https://github.com/morluto/rea/pull/6278), [#6509](https://github.com/morluto/rea/pull/6509), [#4830](https://github.com/morluto/rea/pull/4830), [#7008](https://github.com/morluto/rea/pull/7008), [#4837](https://github.com/morluto/rea/pull/4837), [#7010](https://github.com/morluto/rea/pull/7010), [#6987](https://github.com/morluto/rea/pull/6987), [#4057](https://github.com/morluto/rea/issues/4057). Thanks @gensokyo-dev, @kaoru0822-kitauji, @mvanhorn, @Rinkia, @ShuaixinHuang.
- **Apple/native metadata:** Inspect IPS crash reports, decode Swift nominal declarations, and select universal Mach-O slices in Go/Rust metadata readers. [#6511](https://github.com/morluto/rea/pull/6511), [#6257](https://github.com/morluto/rea/pull/6257), [#6219](https://github.com/morluto/rea/pull/6219), [#5341](https://github.com/morluto/rea/issues/5341), [#4102](https://github.com/morluto/rea/issues/4102). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @mvanhorn, @Rinkia.
- **Packaged applications:** Triage desktop installation directories and inspect/extract Godot PCK, PyInstaller archives, and Node.js single-executable application blobs. [#5207](https://github.com/morluto/rea/pull/5207), [#3634](https://github.com/morluto/rea/pull/3634), [afa5da7](https://github.com/morluto/rea/commit/afa5da745e0b139e68029a96b776ae30e48ea2b2), [#2433](https://github.com/morluto/rea/pull/2433), [#7000](https://github.com/morluto/rea/pull/7000), [#4399](https://github.com/morluto/rea/issues/4399). Thanks @0xthearchitect, @coygeek, @gensokyo-dev, @mvanhorn.
- **Unity and managed code:** Inventory IL2CPP images/types/methods and metadata versions 24/27; read member names and literals, link native methods and generic instances, and export Ghidra seeds. Project managed call/field-access edges and page managed member summaries. [#3807](https://github.com/morluto/rea/pull/3807), [#5326](https://github.com/morluto/rea/pull/5326), [52d1a72](https://github.com/morluto/rea/commit/52d1a72af047a73b787b8935b032786a34966036), [#5327](https://github.com/morluto/rea/pull/5327), [#5785](https://github.com/morluto/rea/pull/5785), [#6260](https://github.com/morluto/rea/pull/6260), [#5167](https://github.com/morluto/rea/pull/5167), [#3784](https://github.com/morluto/rea/issues/3784), [#5112](https://github.com/morluto/rea/issues/5112), [#4056](https://github.com/morluto/rea/issues/4056), [#5148](https://github.com/morluto/rea/issues/5148). Thanks @8rulerstar, @alexlanghart, @elricfd, @gensokyo-dev, @jaelliot, @kaoru0822-kitauji, @Rinkia.
- **Android:** Inspect APK signing declarations and structured manifest components; report static-final DEX field values. [e93e3b1](https://github.com/morluto/rea/commit/e93e3b1e7d6e7e4f49c2de24b92452ec29e853f8), [#4158](https://github.com/morluto/rea/pull/4158), [#5025](https://github.com/morluto/rea/pull/5025), [#6981](https://github.com/morluto/rea/pull/6981), [#4537](https://github.com/morluto/rea/issues/4537), [#6397](https://github.com/morluto/rea/issues/6397). Thanks @Abelo9996, @BaaRai, @FerroLx, @gensokyo-dev, @sserrgo12.
- **Bytecode and firmware:** Inspect and query CPython pyc and BEAM facts, inspect flattened devicetrees, and attribute MMIO peripheral regions in native disassembly. [#4826](https://github.com/morluto/rea/pull/4826), [#6928](https://github.com/morluto/rea/pull/6928), [#4833](https://github.com/morluto/rea/pull/4833), [#6943](https://github.com/morluto/rea/pull/6943), [#7009](https://github.com/morluto/rea/pull/7009), [#5348](https://github.com/morluto/rea/pull/5348), [#6922](https://github.com/morluto/rea/issues/6922), [#6938](https://github.com/morluto/rea/issues/6938). Thanks @gensokyo-dev, @liziye627-design, @mvanhorn.
- **JavaScript:** Retain queryable static route inventories, link assignment values to writes, and accept source selections as a JSON array or file. [#6019](https://github.com/morluto/rea/pull/6019), [#6206](https://github.com/morluto/rea/pull/6206), [#5029](https://github.com/morluto/rea/pull/5029), [#4772](https://github.com/morluto/rea/issues/4772), [#6193](https://github.com/morluto/rea/issues/6193). Thanks @gensokyo-dev, @IvenKooLab, @jacobmalibiran, @JoshOsullivan-au, @kaoru0822-kitauji, @KJLavender.
- **Data inspection:** Decode selected protobuf/gRPC bodies, inspect GGUF/safetensors declarations, and produce byte-entropy profiles. [#5031](https://github.com/morluto/rea/pull/5031), [#5603](https://github.com/morluto/rea/pull/5603), [#2331](https://github.com/morluto/rea/pull/2331), [#4986](https://github.com/morluto/rea/issues/4986), [#3323](https://github.com/morluto/rea/issues/3323). Thanks @AayushJB03, @gensokyo-dev, @kaoru0822-kitauji, @KJLavender, @michaelkillgta, @mvanhorn.
- **CLI input/output:** Accept structured request documents on stdin and expose version/runtime output formats. Row-oriented JSONL exports use the migration above. [#7040](https://github.com/morluto/rea/pull/7040), [#5486](https://github.com/morluto/rea/issues/5486), [#6530](https://github.com/morluto/rea/pull/6530), [169512a](https://github.com/morluto/rea/commit/169512a35bb015ff209168fb8bc1c9b31a2c678b), [#5483](https://github.com/morluto/rea/issues/5483). Thanks @Abelo9996, @ahmed-husssain, @CassieFox23, @DeryFerd, @eco-null, @ezrabrilliant, @GamerDubz, @gensokyo-dev, @gongfpp, @hifzabuildsai, @jayykernel, @mhamzasakhi, @pablodiazjorge, @Plastic-time, @raidnouredden-ux, @SandstromPL, @ShuaixinHuang, @Sriram-PR, @XiaoBinGan.
- **MCP and setup:** Add opt-in Streamable HTTP and FreeBuff setup registration. FreeBuff support first appears in 6.8.0; installed native FreeBuff loading and skill discovery remain unverified. [#4955](https://github.com/morluto/rea/pull/4955), [#2736](https://github.com/morluto/rea/pull/2736), [#3243](https://github.com/morluto/rea/issues/3243). Thanks @0xLLVEL, @codebuff-team, @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
- **Recovery controls:** Adjust per-call WASM/SQLite/firmware deadlines, MCP startup budgets, regex matching deadlines/worker heap, capture decoder heap, ASAR header admission, Apple dispatch input, and the bridge request ceiling. MCP runtime limit recovery reports actionable settings. [#5798](https://github.com/morluto/rea/pull/5798), [#5818](https://github.com/morluto/rea/pull/5818), [aa53c06](https://github.com/morluto/rea/commit/aa53c065387c58b4189d82c47f1f3445fc505e30), [#5863](https://github.com/morluto/rea/pull/5863), [#6304](https://github.com/morluto/rea/pull/6304), [#6267](https://github.com/morluto/rea/pull/6267), [#6301](https://github.com/morluto/rea/pull/6301), [#6299](https://github.com/morluto/rea/pull/6299), [#6209](https://github.com/morluto/rea/pull/6209), [#6233](https://github.com/morluto/rea/pull/6233), [#6376](https://github.com/morluto/rea/pull/6376), [#7049](https://github.com/morluto/rea/pull/7049), [#5797](https://github.com/morluto/rea/issues/5797), [#5812](https://github.com/morluto/rea/issues/5812), [#5851](https://github.com/morluto/rea/issues/5851), [#6176](https://github.com/morluto/rea/issues/6176), [#3917](https://github.com/morluto/rea/issues/3917). Thanks @gensokyo-dev, @jaychou813, @kaoru0822-kitauji.
- **APK output:** Set `string_tables_only` or CLI `--string-tables-only` to omit duplicate primary strings and avoid their output-capacity charge. [7c458f0](https://github.com/morluto/rea/commit/7c458f04b9155707c8ba406f8b455b10016f1b99), [#4990](https://github.com/morluto/rea/pull/4990), [#4982](https://github.com/morluto/rea/issues/4982). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
- **Retained analysis:** Reuse successful validation of authenticated immutable native/imported-JavaScript parents and captures. Query retained Go build identity, Rust dependencies, JavaScript source-to-bundle mappings, and native export facts without repeating full output. [8f7d12e](https://github.com/morluto/rea/commit/8f7d12e30d63f3d3b4cfe2a90336362841438f56), [#4962](https://github.com/morluto/rea/pull/4962), [#5349](https://github.com/morluto/rea/pull/5349), [#5524](https://github.com/morluto/rea/pull/5524), [#5815](https://github.com/morluto/rea/pull/5815), [#6128](https://github.com/morluto/rea/pull/6128), [#6907](https://github.com/morluto/rea/pull/6907), [#4810](https://github.com/morluto/rea/issues/4810), [#5499](https://github.com/morluto/rea/issues/5499), [#5802](https://github.com/morluto/rea/issues/5802), [#6112](https://github.com/morluto/rea/issues/6112), [#6900](https://github.com/morluto/rea/issues/6900). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
- **Compact results:** Use retained Frida/GDB/objdump summaries, omit redundant Ghidra raw results, and defer the analysis runtime during MCP catalog discovery. [#5661](https://github.com/morluto/rea/pull/5661), [#5614](https://github.com/morluto/rea/pull/5614), [#5355](https://github.com/morluto/rea/pull/5355), [#4424](https://github.com/morluto/rea/pull/4424), [#5640](https://github.com/morluto/rea/issues/5640), [#5584](https://github.com/morluto/rea/issues/5584), [#5337](https://github.com/morluto/rea/issues/5337). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @leeguooooo.
- **Analysis work:** Reuse Hopper literal-string records, JavaScript workers and captured prefix selections, Interface Builder validation and plist scalar schemas; bound stable reads to one snapshot buffer and avoid decoding omitted GGUF strings. [#5017](https://github.com/morluto/rea/pull/5017), [#5819](https://github.com/morluto/rea/pull/5819), [31de177](https://github.com/morluto/rea/commit/31de177404461d122761d8243abe4fd24564f4da), [#5885](https://github.com/morluto/rea/pull/5885), [85e38ac](https://github.com/morluto/rea/commit/85e38ac199dff15e8c43343d9170db7e0e46d284), [#6966](https://github.com/morluto/rea/pull/6966), [#5781](https://github.com/morluto/rea/pull/5781), [#5010](https://github.com/morluto/rea/issues/5010), [#5855](https://github.com/morluto/rea/issues/5855), [#6954](https://github.com/morluto/rea/issues/6954), [#5774](https://github.com/morluto/rea/issues/5774). Thanks @gensokyo-dev, @kaoru0822-kitauji.
- **Native scheduling:** Batch modern IDA callee lookups, schedule decompilation reads on demand, index NativeAOT type closure, and retain call-path ancestry once per visited node. [0c6fed9](https://github.com/morluto/rea/commit/0c6fed90b74c75a9fd23d1ec1b510a7274f85887), [06fe7ee](https://github.com/morluto/rea/commit/06fe7eecbc8c387414b4c1aa0d88ad6314f54f0c), [ce61680](https://github.com/morluto/rea/commit/ce61680c04c95a9cdfad5335632a99542af26d37), [5e4ec0f](https://github.com/morluto/rea/commit/5e4ec0fa1b678aa3bf20704ec9110224ba8c1676).
- **Selected processing:** Prepare CLI positional schemas only for the selected command, separate entropy reads from report resolution, prune typed-array reference escapes, index state-write cutoffs, decode only selected replay query values, and skip payload text during WAT framing. [315b234](https://github.com/morluto/rea/commit/315b23403f0819967878cee418c3f9cb4b1a69d9), [#6950](https://github.com/morluto/rea/pull/6950), [#5139](https://github.com/morluto/rea/pull/5139), [#5350](https://github.com/morluto/rea/pull/5350), [96ea32b](https://github.com/morluto/rea/commit/96ea32b7e4e28d3c9cd79a1de4b8e1ba3b4244e9), [70835d3](https://github.com/morluto/rea/commit/70835d3d5f030deadf606e1c79c70a60883e50c9), [#5766](https://github.com/morluto/rea/pull/5766), [#6947](https://github.com/morluto/rea/issues/6947), [#5760](https://github.com/morluto/rea/issues/5760). Thanks @galleonli, @gensokyo-dev, @kaoru0822-kitauji.
### Fixes
- **IPS report fields:** Preserve the recorded terminating process and PID, termination flags, parent PID, and build/platform metadata. [#7079](https://github.com/morluto/rea/pull/7079), [#7078](https://github.com/morluto/rea/issues/7078). Thanks @rodricksz4h5.
- **Ghidra language selection:** Accept stock language IDs containing `LEBE` or slash-separated processor names, and report conflicting DOL language overrides as configuration errors naming `REA_GHIDRA_LANGUAGE_ID`. [#7070](https://github.com/morluto/rea/pull/7070), [#7072](https://github.com/morluto/rea/issues/7072), [#7081](https://github.com/morluto/rea/pull/7081), [#7082](https://github.com/morluto/rea/issues/7082). Thanks @Adolanium.
- **Setup and Hopper:** Preserve the Linux embedding runtime, supported x86_64 host restriction, readiness-based provider choices, orderly demo-helper shutdown, and installed apt dependency versions. Keep cleanup and skill publication ownership through retries. [#5041](https://github.com/morluto/rea/pull/5041), [#3422](https://github.com/morluto/rea/pull/3422), [f5d1012](https://github.com/morluto/rea/commit/f5d1012fabfe99af82523c3078ce9d93eb3780c1), [#4966](https://github.com/morluto/rea/pull/4966), [#6974](https://github.com/morluto/rea/pull/6974), [211e353](https://github.com/morluto/rea/commit/211e3533b34f81ae0910ad4f53f2a3d3c8587d3e), [#6920](https://github.com/morluto/rea/pull/6920), [#5015](https://github.com/morluto/rea/issues/5015), [#3419](https://github.com/morluto/rea/issues/3419), [#4921](https://github.com/morluto/rea/issues/4921), [#6975](https://github.com/morluto/rea/issues/6975). Thanks @Abel-I-Akhigbe, @Adolanium, @DavidFB-creator, @DeryFerd, @gensokyo-dev, @kaoru0822-kitauji, @sx4im.
- **Client setup:** Treat empty APPDATA as unset, distinguish MSIX Claude Desktop process views, preserve aliased Hermes server tables, reuse client dialect/alias bindings, and materialize the pinned package-runner cache before the first start. [#5126](https://github.com/morluto/rea/pull/5126), [#4903](https://github.com/morluto/rea/pull/4903), [#3785](https://github.com/morluto/rea/pull/3785), [b7eeded](https://github.com/morluto/rea/commit/b7eededf5af5061a834f93f57f2c888acf870344), [#7032](https://github.com/morluto/rea/pull/7032), [#5104](https://github.com/morluto/rea/issues/5104), [#4278](https://github.com/morluto/rea/issues/4278), [#3320](https://github.com/morluto/rea/issues/3320), [#7027](https://github.com/morluto/rea/issues/7027). Thanks @CmsChase, @coygeek, @myso-kr, @pablodiazjorge, @vaidehi-snk, @wxj006007.
- **Target admission:** Preserve thin arm64_32 Mach-O support, distinguish JVM identity before FAT admission, validate FAT format/CPU identity, preserve original DOS header padding, and reject NULs in local objdump/rizin paths. [#5788](https://github.com/morluto/rea/pull/5788), [6843dbe](https://github.com/morluto/rea/commit/6843dbea9bce4709ee2e8bf46c38f0bbf640716b), [497037e](https://github.com/morluto/rea/commit/497037e6e3975cbb37cfd89ea0ad300faa2a4769), [#7013](https://github.com/morluto/rea/pull/7013), [#3600](https://github.com/morluto/rea/pull/3600), [#5713](https://github.com/morluto/rea/issues/5713), [#6510](https://github.com/morluto/rea/issues/6510). Thanks @coygeek, @dhruvac29, @kaoru0822-kitauji, @Lew1sWong, @qorvr.
- **Mach-O and dylibs:** Preserve container framing, facts when dyld rejects a missing UUID, verified-signature snapshot identity, unsigned diagnostics, absolute paths inside the selected root, legacy platform rules, lazy-load payloads, and reasons for rejecting present dylibs. [cf3288d](https://github.com/morluto/rea/commit/cf3288d8cf7a09685acf89103b36c32359be56fa), [2b263a7](https://github.com/morluto/rea/commit/2b263a7fe9e9d4c185c7d6af6b4e67b21a73af30), [eb7acc7](https://github.com/morluto/rea/commit/eb7acc70bf06f119e8639e30a78081d5ba82b0bf), [1502cfb](https://github.com/morluto/rea/commit/1502cfbc8ca0d1d1d86eebf334018dfb418bbac0), [90062c0](https://github.com/morluto/rea/commit/90062c092228dea9d33fadc0d7e1b968105b8b2c), [#5650](https://github.com/morluto/rea/pull/5650), [3b67b36](https://github.com/morluto/rea/commit/3b67b362aa2cd79eac5af57d0965e81a7d0b8c2b), [#5551](https://github.com/morluto/rea/issues/5551). Thanks @coygeek.
- **Native facts:** Admit direct dispatch-metadata providers; preserve record-local Objective-C omissions, snapshot profiles, class-import binding identity, stripped class labels, version-like names, and qualified Swift protocol identities. [50b6975](https://github.com/morluto/rea/commit/50b6975d861069e2c6cfa202780621449f863802), [9eec57b](https://github.com/morluto/rea/commit/9eec57b469087fe9bff3d29dcc02f47074159e91), [0ba4d6f](https://github.com/morluto/rea/commit/0ba4d6f43ad610dacc861a1548be1a5c4712cd61), [4d7de00](https://github.com/morluto/rea/commit/4d7de00d4cc9a95ec2b7a3123b04e01db2daafa0), [22c84ce](https://github.com/morluto/rea/commit/22c84ce64aa6c7601aa81af86d4117b5a4c2feac), [4852c83](https://github.com/morluto/rea/commit/4852c830af1e3b9df437235b7a8fa8970a6d1c59), [aa9043f](https://github.com/morluto/rea/commit/aa9043f6b61c453e4735795cfe6a559bf6b6cc16).
- **Apple resources:** Preserve independent plist scalars, processing instructions, nonstandard table widths, unsupported NIB connector coverage, and strict IPS JSON surrogate validation; classify Interface Builder objects by their declared superclass chains. [fd728f2](https://github.com/morluto/rea/commit/fd728f271e7de7bcfa63890b573715720d5f948c), [#5806](https://github.com/morluto/rea/pull/5806), [4526862](https://github.com/morluto/rea/commit/4526862728af01f1135714079c0d8a8c754eb364), [14cd78c](https://github.com/morluto/rea/commit/14cd78c3224edcf79fca152915077112f547b9ed), [#6990](https://github.com/morluto/rea/pull/6990), [#5805](https://github.com/morluto/rea/issues/5805), [#5228](https://github.com/morluto/rea/pull/5228), [#5218](https://github.com/morluto/rea/issues/5218). Thanks @coygeek, @DeryFerd, @kaoru0822-kitauji, @Mangolycheematcha, @mhamzasakhi, @Shreyash021104.
- **Archives and files:** Complete positional ZIP reads, preserve bounded ASAR signatures, report stable symlink loops as resolution failures, retain target applicability, and report unknown traversal coverage. Retain reported integrity contradictions for members present in only one compared artifact. [#4414](https://github.com/morluto/rea/pull/4414), [e8da7e1](https://github.com/morluto/rea/commit/e8da7e1edd075feec0e6beedec93d5645fed8125), [#5571](https://github.com/morluto/rea/pull/5571), [#5763](https://github.com/morluto/rea/pull/5763), [#5867](https://github.com/morluto/rea/pull/5867), [#5521](https://github.com/morluto/rea/issues/5521), [#5731](https://github.com/morluto/rea/issues/5731), [#5837](https://github.com/morluto/rea/issues/5837), [#7076](https://github.com/morluto/rea/pull/7076). Thanks @Afloat16, @coygeek, @jdotc1, @kaoru0822-kitauji, @WenyuChiou.
- **Extraction and file ownership:** Restrict destination-case recovery to owned collisions, retain physical-read failures through late cancellation/cleanup, preserve cleanup capabilities for retry, and retain independent firmware extraction failures. [932fbb7](https://github.com/morluto/rea/commit/932fbb7d84a9a995b9aee3c1be7da05a8c4d7bbb), [9499667](https://github.com/morluto/rea/commit/949966712e68d8857a50a165ecddf04c2d7e6d3b), [56679dd](https://github.com/morluto/rea/commit/56679dd2ae31dda58e181f8bf99775014f7bb82b), [b3c912a](https://github.com/morluto/rea/commit/b3c912ab79ad147a37b6cfbe1552a1a5eecec629), [7298bd0](https://github.com/morluto/rea/commit/7298bd08d134860b84b4073f6703021916a5a4c5).
- **Processes and cancellation:** Retain partial samples and shutdown gaps, unresolved descendants and caller abort reasons; bind cleanup errors to their owners, propagate comparison contradictions, and report no normal exit code for signalled targets. [ba866a2](https://github.com/morluto/rea/commit/ba866a26ecbe179a717e746cbc0d1605ae616ad5), [#5270](https://github.com/morluto/rea/pull/5270), [#6144](https://github.com/morluto/rea/pull/6144), [a721c42](https://github.com/morluto/rea/commit/a721c42c07dc674c32ab939627355b8a1bee97a5), [#6298](https://github.com/morluto/rea/pull/6298), [#6486](https://github.com/morluto/rea/pull/6486), [#5935](https://github.com/morluto/rea/issues/5935), [#5944](https://github.com/morluto/rea/issues/5944), [#6100](https://github.com/morluto/rea/issues/6100), [#6101](https://github.com/morluto/rea/issues/6101), [#6110](https://github.com/morluto/rea/issues/6110), [#6113](https://github.com/morluto/rea/issues/6113), [#6121](https://github.com/morluto/rea/issues/6121), [#6131](https://github.com/morluto/rea/issues/6131), [#6132](https://github.com/morluto/rea/issues/6132), [#6218](https://github.com/morluto/rea/issues/6218), [#6445](https://github.com/morluto/rea/issues/6445). Thanks @agorevski, @coygeek, @gongfpp, @kwakhyun, @mhamzasakhi.
- **Provider cleanup:** Preserve Windows native handles until close is confirmed, native/Inspector/Electron cleanup ownership, Kuna probe identity and cleanup, IDA registration configuration, and Hopper preparation ownership. [#5772](https://github.com/morluto/rea/pull/5772), [87ab054](https://github.com/morluto/rea/commit/87ab054ea384b3fcbae2998b1f94685edcd5f624), [ff78165](https://github.com/morluto/rea/commit/ff78165978de9dbcf9fa16c2acf74bf85a69dc44), [04ae07a](https://github.com/morluto/rea/commit/04ae07ab8c9ae1a6697b6ad15489bbde515da8b5), [dd0015a](https://github.com/morluto/rea/commit/dd0015aaf156c5701ef9598bcc6c885e6b8da803), [b231c28](https://github.com/morluto/rea/commit/b231c28cf64c2f799ada99343f8f21e0e6acb15e), [#4505](https://github.com/morluto/rea/issues/4505), [#5616](https://github.com/morluto/rea/issues/5616).
- **Provider diagnostics:** Give Ghidra restart recovery, stable Homebrew aliases, unsupported-host results for Linux-only integrations, correct IDA regex/readiness handling, and clear diagnostics when Windows refuses a verified native addon. [#5356](https://github.com/morluto/rea/pull/5356), [7c6142e](https://github.com/morluto/rea/commit/7c6142efdc19a42c148d0e8c96b69d18e63c7f87), [#5793](https://github.com/morluto/rea/pull/5793), [#4862](https://github.com/morluto/rea/pull/4862), [99ccfb6](https://github.com/morluto/rea/commit/99ccfb6e709ac4bc0afe176315280abde9ebcb9f), [#6832](https://github.com/morluto/rea/pull/6832), [#5730](https://github.com/morluto/rea/issues/5730), [#4860](https://github.com/morluto/rea/issues/4860), [#5723](https://github.com/morluto/rea/issues/5723). Thanks @coygeek, @dark13b, @gensokyo-dev, @jgt87, @jpmorris41488, @Lew1sWong.
- **MCP results:** Preserve protocol envelopes on oversized errors, enforce and advertise absolute Evidence paths, retain typed output-filter errors and selected JSON paths, and give client/provider timeout recovery. Compact advertisements now normalize reported enum and union shapes for strict gateway translation. [562dab0](https://github.com/morluto/rea/commit/562dab00c5cbf76a9b2695bad93792eccdc9a49d), [3cf6206](https://github.com/morluto/rea/commit/3cf6206614e65b5ebb6284a5709090dfb8f9aaee), [0e83d92](https://github.com/morluto/rea/commit/0e83d92c583ae36afe00b2c23dd2a0f1d025cbca), [4714c7e](https://github.com/morluto/rea/commit/4714c7eaa3d1d8b6a5df0aff586e844b9d9ab06d), [1bfa60a](https://github.com/morluto/rea/commit/1bfa60ac20e1a344bcf3132c08d15b6860b05c61), [#5608](https://github.com/morluto/rea/pull/5608), [#3806](https://github.com/morluto/rea/pull/3806), [#5598](https://github.com/morluto/rea/issues/5598), [#3800](https://github.com/morluto/rea/issues/3800). Thanks @gensokyo-dev, @Hahaa909, @wang1xiang.
- **Browser lifecycles:** Retain historical outcomes through cleanup, stop late navigation before session release, stop expired work, observe aborted fetch settlement, and mark later scenario actions skipped after failure or cancellation. [2e5171e](https://github.com/morluto/rea/commit/2e5171e391b83f8a467d0861fd452c1f78621fd8), [599fdac](https://github.com/morluto/rea/commit/599fdac029912f96c511f859660483405d2de26f), [b2fad36](https://github.com/morluto/rea/commit/b2fad362f2b37f81dc9a6f784f4521e4ca24d41e), [13a8640](https://github.com/morluto/rea/commit/13a864020414577006dce58d7fdaa877ddd0cb52), [#6023](https://github.com/morluto/rea/pull/6023), [#5943](https://github.com/morluto/rea/issues/5943). Thanks @coygeek, @kaoru0822-kitauji.
- **Browser comparisons:** Preserve frame identity, default origin scope, unknown body/shape coverage, correspondence and uncertainty, and independent retained-observation budgets. [#4040](https://github.com/morluto/rea/pull/4040), [#4413](https://github.com/morluto/rea/pull/4413), [#4476](https://github.com/morluto/rea/pull/4476), [a0c22ff](https://github.com/morluto/rea/commit/a0c22ff0125b8af7612db54d3708413e2bfecb25), [a0fcc60](https://github.com/morluto/rea/commit/a0fcc60d89eb0cc09b08181eb9f961acfd07d8e9), [#3878](https://github.com/morluto/rea/issues/3878), [#3910](https://github.com/morluto/rea/issues/3910), [#3941](https://github.com/morluto/rea/issues/3941), [#4385](https://github.com/morluto/rea/issues/4385), [#4477](https://github.com/morluto/rea/issues/4477). Thanks @Adolanium, @coygeek, @eco-null, @gensokyo-dev, @kaoru0822-kitauji, @XY-code1.
- **Electron observations:** Preserve listener callback ownership and count registrations by source callsite; block trash calls in the active hook, retain window handles through cancelled selection, and preserve cancellation during finalization. [dfe8378](https://github.com/morluto/rea/commit/dfe8378cf5a727e3d93d47a5986233175c6ef898), [5064007](https://github.com/morluto/rea/commit/506400748eb1d6c674f0cf51330073929c45f939), [816a9dc](https://github.com/morluto/rea/commit/816a9dc98886e6d824cd8ae88b68731b161d5dda), [1707d17](https://github.com/morluto/rea/commit/1707d171d750955bf6387e028e378183b360ffe3), [4e2a566](https://github.com/morluto/rea/commit/4e2a566d6c7e053ecbbee1b684311ed31ea163bf).
- **JavaScript source maps:** Keep unmapped and missing mappings unknown, preserve partial indexed-map facts and section paths, propagate preparation capability errors, and retain observed worker failures through cancellation. [a4dbb36](https://github.com/morluto/rea/commit/a4dbb36d8ea83e9f94dd289ac1a2d220d2c72c9b), [2b2e362](https://github.com/morluto/rea/commit/2b2e362a00fc5ba2f93fa6c62e2d8780072d1b4b), [498a667](https://github.com/morluto/rea/commit/498a667234b0140c1cfa43c819b35454ab11d14e), [#6149](https://github.com/morluto/rea/pull/6149), [#6214](https://github.com/morluto/rea/pull/6214), [f01dcca](https://github.com/morluto/rea/commit/f01dccab77f712fb4ac936d0a6e92a70d12f0789), [#6082](https://github.com/morluto/rea/issues/6082), [#6061](https://github.com/morluto/rea/issues/6061). Thanks @coygeek, @diyasharma05, @kaoru0822-kitauji.
- **JavaScript semantics:** Preserve overflowing timer uncertainty, unavailable-scope comparisons, directory-form URL/ESM path intent, and artifact confinement. Replaced globals lose built-in effects; erased imports no longer become runtime bindings. Treat a known truthy `fork()` `silent` option as piped stdio unless a string/array `stdio` override takes precedence; unresolved options stay dynamic. [#3456](https://github.com/morluto/rea/pull/3456), [8d198d6](https://github.com/morluto/rea/commit/8d198d6ec707402d75276f516e0090d71767b7e4), [#4646](https://github.com/morluto/rea/pull/4646), [#2700](https://github.com/morluto/rea/pull/2700), [#4565](https://github.com/morluto/rea/pull/4565), [#6994](https://github.com/morluto/rea/pull/6994), [#3400](https://github.com/morluto/rea/issues/3400), [#3882](https://github.com/morluto/rea/issues/3882), [#3367](https://github.com/morluto/rea/issues/3367), [#7083](https://github.com/morluto/rea/pull/7083), [#7084](https://github.com/morluto/rea/issues/7084). Thanks @Adolanium, @amyanger, @coygeek, @gensokyo-dev, @HawkForge, @rohitg00, @viniciusdsandrade.
- **JavaScript routes and imports:** Treat path data as route candidates, preserve route flag and lexical RPC identities, package-main trailing-slash selection, catch aliases, and factory parameter ownership. [#6360](https://github.com/morluto/rea/pull/6360), [38ecc02](https://github.com/morluto/rea/commit/38ecc0248e1993d25a2f7dd350024484cce797bb), [#4219](https://github.com/morluto/rea/pull/4219), [21097b1](https://github.com/morluto/rea/commit/21097b192b1ab94647e3446b1a2a4dd00e924d5e), [f774dc4](https://github.com/morluto/rea/commit/f774dc420b3befe2b8e68971b4116da2d62fc800), [#6349](https://github.com/morluto/rea/issues/6349), [#4189](https://github.com/morluto/rea/issues/4189). Thanks @coygeek, @hemv-857, @kaoru0822-kitauji, @Nikhi00718.
- **Managed/IL2CPP:** Preserve extension-only scope uncertainty and type discovery after member limits, nested names, ReadyToRun/declaration facts, selected type scope, valid shared-suffix strings, and truncated IL2CPP metadata prefixes. [#5674](https://github.com/morluto/rea/pull/5674), [#5312](https://github.com/morluto/rea/pull/5312), [87fa1b6](https://github.com/morluto/rea/commit/87fa1b613d2b6bda1722855a00f1390f27731b64), [bc2d494](https://github.com/morluto/rea/commit/bc2d494030efd1ccabaeaa0bd49226aa87c378a2), [1f0eb22](https://github.com/morluto/rea/commit/1f0eb228e50908ee2a2aaa72d329178ae1a2cdbd), [4c5fae5](https://github.com/morluto/rea/commit/4c5fae5179dd54c4a0d427cd2a115b9c887e3841), [b3aadaf](https://github.com/morluto/rea/commit/b3aadaf474e59b33e5ff465236008a67eccefb87), [#5467](https://github.com/morluto/rea/issues/5467), [#5146](https://github.com/morluto/rea/issues/5146). Thanks @coygeek, @eco-null, @elricfd, @umi008.
- **Format boundaries:** Preserve independent facts across malformed inputs in model, GPU, WASM, Rust, Android, and SQLite inspection; report optional metadata warnings without discarding core results. [eb41f35](https://github.com/morluto/rea/commit/eb41f358cf83d5626f09de68332f9718f9aa6482), [#5789](https://github.com/morluto/rea/pull/5789), [00a9e95](https://github.com/morluto/rea/commit/00a9e95de91a4bd4d2a41f8aee704a53fa575750), [fc63fe8](https://github.com/morluto/rea/commit/fc63fe8682654086261540dcf3c59afc36c7d58f), [39a91d4](https://github.com/morluto/rea/commit/39a91d4db96ca13224c25936330ea84fe78970e5), [6c6e2ad](https://github.com/morluto/rea/commit/6c6e2ad62b20f810b54f5a567e66fc55241514b2), [929a0a7](https://github.com/morluto/rea/commit/929a0a76c870d9991e015dde58308fd9df0280b8), [d137a4c](https://github.com/morluto/rea/commit/d137a4cdde0049de967f1d5cd429ed9e1f7d0a10), [ef475fc](https://github.com/morluto/rea/commit/ef475fcf8b0601c61052b7091568ba574546417d), [#5749](https://github.com/morluto/rea/issues/5749). Thanks @coygeek, @gensokyo-dev.
- **Capture decoding:** Preserve malformed protobuf trailer facts and wire/schema identity, HTTP/2 URI and UTF-8 failure facts, WebMCP property coverage, and offline capture availability without a browser provider. Report invalid record ordinals as selection errors with actual bounds where available, and trim surrounding SP/HTAB from gRPC-web trailer values while preserving internal whitespace. [9cf7a7c](https://github.com/morluto/rea/commit/9cf7a7cb0d3c72a1121fde280cb11bf911af8055), [b3c3b21](https://github.com/morluto/rea/commit/b3c3b21e43cab4e71b460d08829d8c6e4e9b9c21), [4cb1f1d](https://github.com/morluto/rea/commit/4cb1f1d820df4498ee464bcf72a5f47d40cf8dab), [51e5938](https://github.com/morluto/rea/commit/51e593877642ab42fb5bf25812f4a4110ee7ac26), [d23be5c](https://github.com/morluto/rea/commit/d23be5c0898c39c9f0c54e2e0b43ee746d92d316), [#6725](https://github.com/morluto/rea/pull/6725), [#3947](https://github.com/morluto/rea/issues/3947), [#7067](https://github.com/morluto/rea/pull/7067), [#7068](https://github.com/morluto/rea/issues/7068), [#7069](https://github.com/morluto/rea/pull/7069), [#7071](https://github.com/morluto/rea/issues/7071). Thanks @Adolanium, @coygeek, @Sriram-PR.
- **Evidence and errors:** Authenticate ledger Evidence before committing; preserve parent-schema diagnostics, canonical correction details and owned cleanup outcomes; identify the rejected function-Evidence side/kind. Fixed allowances now name their actual limits. [0407bbf](https://github.com/morluto/rea/commit/0407bbfbe65e8fa475e8bb0c78c30170e2c2f55e), [#6361](https://github.com/morluto/rea/pull/6361), [1629266](https://github.com/morluto/rea/commit/1629266e3c08c90fed6bba5383528af7986a12b7), [#6969](https://github.com/morluto/rea/pull/6969), [#5860](https://github.com/morluto/rea/pull/5860), [#6034](https://github.com/morluto/rea/issues/6034), [#5932](https://github.com/morluto/rea/issues/5932), [#5847](https://github.com/morluto/rea/issues/5847). Thanks @AyoubElleuch, @coygeek, @dhruvac29, @kaoru0822-kitauji.
- **ADB and firmware recovery:** Report an unparsed push checksum as unverified, normalize absent installer-package sentinels, describe extraction-relative Unblob links, and avoid recommending a budget increase at its fixed ceiling. [#6010](https://github.com/morluto/rea/pull/6010), [#4073](https://github.com/morluto/rea/pull/4073), [#4573](https://github.com/morluto/rea/pull/4573), [#6971](https://github.com/morluto/rea/pull/6971), [#5924](https://github.com/morluto/rea/issues/5924), [#4175](https://github.com/morluto/rea/issues/4175). Thanks @coygeek, @Ganesh2325, @kaoru0822-kitauji, @Rinkia, @XiaoBinGan, @yankikucuk.
- **CLI formatting:** Share Markdown report formatting, exclude inherited row values, and preserve arrays of empty objects as JSON. Render Markdown table headers and cells literally while keeping intraword underscores readable. [b784912](https://github.com/morluto/rea/commit/b784912811fd10ac9dc425696dc0e9af632afa34), [#6984](https://github.com/morluto/rea/pull/6984), [#7058](https://github.com/morluto/rea/pull/7058), [#7057](https://github.com/morluto/rea/issues/7057), [#7073](https://github.com/morluto/rea/pull/7073). Thanks @Plastic-time, @ShuaixinHuang, @viniciusdsandrade.
- **Format identity:** Preserve Python marshal scalar/decode limits, empty optional BEAM chunks, PE resource branch identities and certificate alignment, unsupported compressed ELF notes, and codec-framed GPU bundle extents. [0603223](https://github.com/morluto/rea/commit/06032237c6755e3dd5f9a8db732c89958b238cc7), [c46064c](https://github.com/morluto/rea/commit/c46064c3f6eaba121c5de7b17390bfdecdd132ef), [66671e8](https://github.com/morluto/rea/commit/66671e874888e98f7e2b27e68a7c6fb320bbb827), [03077d5](https://github.com/morluto/rea/commit/03077d54afc49613db4e38f70c506bc60f5223aa), [#6721](https://github.com/morluto/rea/pull/6721), [9fb8c11](https://github.com/morluto/rea/commit/9fb8c114a03ad60e57c11586967c6df845a3f97e), [f6a1e2c](https://github.com/morluto/rea/commit/f6a1e2c892dc23c7583c3d03af9edb01632ff5a5), [#6694](https://github.com/morluto/rea/issues/6694). Thanks @coygeek, @Deepak-Kambala.
- **WASM and APK metadata:** Preserve ambiguous WASM identity and partial facts, glue URL query/fragment text across line terminators, APK signer identity, and raw issuer identity when comparison rules are unknown. [af8f0ba](https://github.com/morluto/rea/commit/af8f0badf64fd3546392b6aa4625777904b2f051), [#6802](https://github.com/morluto/rea/pull/6802), [f783f0f](https://github.com/morluto/rea/commit/f783f0f1e82472cdddd4de2156d974f729e332a6), [115a78f](https://github.com/morluto/rea/commit/115a78fe241276c34917b496621954c5f29bfc15), [#6639](https://github.com/morluto/rea/issues/6639). Thanks @coygeek, @gensokyo-dev, @gongfpp.
- **Source identity:** Preserve Next route spelling, module grammar and loader request identity, directory package-main behavior, accessor effects, exact source targets, and resolver-discovery versus module-loading distinctions. [ddd7ac7](https://github.com/morluto/rea/commit/ddd7ac7b846b48191ead1f7dcd0e3bd735468fa8), [9974512](https://github.com/morluto/rea/commit/99745126009b15060d71c38809418858a07d5afa), [f2e5fd1](https://github.com/morluto/rea/commit/f2e5fd148a85fa737302dd194a33172c4561093e), [baf1678](https://github.com/morluto/rea/commit/baf16784af3592534687052431c145daef55cfd1), [6e324b2](https://github.com/morluto/rea/commit/6e324b224405f2befd953234f6dc210cf72ad50a), [75daf0c](https://github.com/morluto/rea/commit/75daf0c7df108f04ab41f34aec1a31d4a7ceb322), [f91b7f8](https://github.com/morluto/rea/commit/f91b7f8009c9af27a5e0148e3b07d20bfe3b7a2a).
- **Graphs and selected facts:** Preserve JavaScript graph omissions/version facts, Electron native-loader facts, web JSON identity/coverage, module-path selection, feature-trace qualifications, selected native paths, endpoint context, and mixed Swift demangler outcomes. [30db499](https://github.com/morluto/rea/commit/30db499db310f073d9bf3d3acfd77985e1cb7e2e), [5a7bddb](https://github.com/morluto/rea/commit/5a7bddb3aeb0345854e76b79d529aed1c40e2054), [79805f2](https://github.com/morluto/rea/commit/79805f244a1ec0850f3109929168a9c8ebb1b908), [f719dc4](https://github.com/morluto/rea/commit/f719dc46c51a55b3926eec9659d06e7dfb95132b), [7784fdb](https://github.com/morluto/rea/commit/7784fdbad78f8b384d513f0e7dae8e983cd48ef8), [76500f1](https://github.com/morluto/rea/commit/76500f12b54e07b1ea700a403b2593890d63b42a), [676d29f](https://github.com/morluto/rea/commit/676d29f9eef794cea668f5f09f0ff510c848a513), [edf7b3d](https://github.com/morluto/rea/commit/edf7b3d32c3248f221e27f7ce5a3d34e8590285b), [93de7bc](https://github.com/morluto/rea/commit/93de7bc7e324a761162a31625b838ba0d4e3824d).
- **Managed boundaries:** Honor native platform symbol lookup, consume resolved IL2CPP ELF declarations, and attribute malformed comparison inputs to the supplied argument. [64fa1db](https://github.com/morluto/rea/commit/64fa1db1309218283df19f3ee4dba61f5fbd2a67), [87745af](https://github.com/morluto/rea/commit/87745af52ab0ccd3a172c2534c516d41b0095035), [96ad94f](https://github.com/morluto/rea/commit/96ad94f69eda5d68c7335ff775b6927b1376fd2b).
- **Provider and worker recovery:** Preserve output loss after successful command exit, physical refusal diagnostics, bounded worker control framing, JavaScript worker-phase/budget attribution, admitted ambiguity choices, and selected EVM limiter availability. Scope objdump debug-link limitations to DWARF requests and keep process-capture preparation nonexecuting. [e639edf](https://github.com/morluto/rea/commit/e639edf6204c05285d96840478926592daf95876), [171b33b](https://github.com/morluto/rea/commit/171b33b2c4defc6cbea49545bedf4272b72d1ea2), [34563fc](https://github.com/morluto/rea/commit/34563fcc9bd92af30668820dcac5635e27181094), [3f9357b](https://github.com/morluto/rea/commit/3f9357b5990d166a38440f56f439452266650093), [cec8f1e](https://github.com/morluto/rea/commit/cec8f1e8f305d6332272457a6ee419d3c5fb6359), [a74d859](https://github.com/morluto/rea/commit/a74d859a19122fb8f1282b3632d320997b1caae0), [c284168](https://github.com/morluto/rea/commit/c284168c4b1cd214ce7f504d43da28528d82d603), [4fb936f](https://github.com/morluto/rea/commit/4fb936fe86bb7a07f94f819ae8510301f2ef69c9).
- **Setup and limits:** Retain unknown environment-budget facts, selected-skill/maintenance uncertainty, Hopper URL admission and bookmark verification, and listener failure recovery that matches the system cause. [a718037](https://github.com/morluto/rea/commit/a7180370332350bc0f998fbd643888221759570e), [d1fbebb](https://github.com/morluto/rea/commit/d1fbebb578538dc94fd5e376e6e1959218efa835), [60accad](https://github.com/morluto/rea/commit/60accad2de8f2862fe91901502fe26cc5390fb36), [fd7da77](https://github.com/morluto/rea/commit/fd7da77fc02dcfaf7d1bc9f2fb240d71cdd27c59), [0c440bf](https://github.com/morluto/rea/commit/0c440bfc07f3cb343e2e836d27cccaed539cfe9c).
- **State and Evidence:** Retain export tarball ownership and SQLite primary outcomes through cleanup retry, count emitted SQLite schema names across encodings, scope display refresh to matching snapshot bindings, and report retained-result validation failures consistently. Reconstruction uses one effective completion scope and kind-based labels when names are blank. [ab58edc](https://github.com/morluto/rea/commit/ab58edcdf50b4bd687900ee2dc362ecc5c3dce15), [1006714](https://github.com/morluto/rea/commit/100671427105791a5c5913a918b63a394f6bc83e), [a47788a](https://github.com/morluto/rea/commit/a47788a6a4abb067273f954275a9d0a269adf1e7), [8aa40e9](https://github.com/morluto/rea/commit/8aa40e9a61155cff8b6f3c138c601bf805b20653), [b8c1b45](https://github.com/morluto/rea/commit/b8c1b45316882beeab4a9de49381d94e131fab0f), [a35b933](https://github.com/morluto/rea/commit/a35b9335a0124674ee5e3abe8c075f1fff6b4fa6), [#6137](https://github.com/morluto/rea/pull/6137), [#6126](https://github.com/morluto/rea/issues/6126). Thanks @carlogiovannineri-rgb, @mohithhhh.
- **Capture coverage:** Name browser bundle budgets and deadline recovery, separate domain readiness from worker attribution, attribute rendered-frame omissions to their producing stage, share offline-capture host admission with discovery, preserve selected signal-file semantics, and retain short Flutter GNU build IDs. [#6036](https://github.com/morluto/rea/pull/6036), [ba78133](https://github.com/morluto/rea/commit/ba781330d6d28b22d15498e63c144fb40e086156), [5934387](https://github.com/morluto/rea/commit/5934387f4185129efe9dc8b7338d8bc3fa5b25bd), [ad629b2](https://github.com/morluto/rea/commit/ad629b2e31c3a9495b27179360b508fa021fe86a), [928ba76](https://github.com/morluto/rea/commit/928ba76882bfd216dde8ed79118789b2c3aad4f9), [678c4cf](https://github.com/morluto/rea/commit/678c4cfa68e2fee12f620075e12dc87d9b1195b0).
- **Additional boundary facts:** Retain parser format/query occurrence identity and independent malformed-input facts, unresolved nested Swift identifiers, and completed signing-capture coverage; describe crash documents without assuming a provider. [5f702dd](https://github.com/morluto/rea/commit/5f702dd6ccd9e8e505e224854b91678439c3b41f), [99d2d7a](https://github.com/morluto/rea/commit/99d2d7ad22bcbd780919760008aba6a88e7d1077), [90e433d](https://github.com/morluto/rea/commit/90e433df06b3f83417176b8db1b0a905f9df1d3a), [#5548](https://github.com/morluto/rea/pull/5548), [ee12d51](https://github.com/morluto/rea/commit/ee12d517a25d34853d40e31fc039c7d1576c60e1), [#5538](https://github.com/morluto/rea/issues/5538). Thanks @coygeek, @ShuaixinHuang.
- **Documentation:** Add the Romanian README. [#7026](https://github.com/morluto/rea/pull/7026). Thanks @An3treiu.
Full contribution record: [1740 commits and 497 merged PRs](https://github.com/morluto/rea/blob/rea-agents-6.8.0/docs/releases/6.8.0.contributions.json).
## [6.7.0](https://github.com/morluto/rea/compare/rea-agents-6.6.0...rea-agents-6.7.0) (2026-10-18)
### Highlights
- Client configuration edits follow the duplicate-member semantics used by MCP clients.
### Changes
- Includes the [6.6.0 changes and migration notes](https://github.com/morluto/rea/releases/tag/rea-agents-6.6.0). That version was tagged but was not published to npm.
### Fixes
- **Client configuration:** Setup edits the last repeated JSON/JSONC member, matching the value clients actually read. Setup and uninstall refuse a removal that would reactivate a shadowed duplicate, preserving the original configuration. ([#4957](https://github.com/morluto/rea/pull/4957), [morluto/rea#4952](https://github.com/morluto/rea/issues/4952)). Thanks @kaoru0822-kitauji, @xu91102.
Full contribution record: [4 commits and 1 merged PR](https://github.com/morluto/rea/blob/rea-agents-6.7.0/docs/releases/6.7.0.contributions.json).
## [6.6.0](https://github.com/morluto/rea/compare/rea-agents-6.5.0...rea-agents-6.6.0) (2026-10-18)
### Highlights
- Add Kuna as a caller-supplied static-analysis provider.
- Query selected JavaScript facts and multiple APK locale tables without repeating the underlying analysis.
- Extend Ghidra analysis to Windows ARM PE targets and explicit raw m68k images.
- Preserve completed Evidence, cancellation reasons and retryable cleanup across provider and CLI finalization.
### Changes
- **Kuna analysis:** Use a caller-installed Kuna for procedure listings, pseudocode, assembly, call relationships, exact-address xrefs and cached string searches. Project exports retain source spans and digests; repeated requests reuse completed work. Recorded real-provider coverage uses Kuna 1.833 on Linux; Windows, PE and universal Mach-O are unsupported. See [the provider guide](docs/kuna-provider.md). ([#3832](https://github.com/morluto/rea/pull/3832), [#3854](https://github.com/morluto/rea/pull/3854), [#4071](https://github.com/morluto/rea/pull/4071), [#4095](https://github.com/morluto/rea/pull/4095)). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
- **Retained JavaScript facts:** Query exact application paths, selected properties and filtered module references through `inspect_analysis_view`, with paging and complete row Evidence. These views reuse completed analysis and preserve source selection and unknown coverage. ([#4265](https://github.com/morluto/rea/pull/4265), [morluto/rea#4190](https://github.com/morluto/rea/issues/4190), [#4097](https://github.com/morluto/rea/pull/4097), [morluto/rea#4092](https://github.com/morluto/rea/issues/4092)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **APK locale tables:** Request several resource locales in one decode with repeatable CLI `--locales` or MCP `locales`. Each table reports its status and coverage; existing single-locale fields remain available. ([#4804](https://github.com/morluto/rea/pull/4804), [morluto/rea#4777](https://github.com/morluto/rea/issues/4777)). Thanks @abhay-2108, @N0zoM1z0.
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/providers/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/providers/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
- **JADX deadlines:** Set `REA_JADX_OPERATION_TIMEOUT_MS` to extend APK loading and upstream operation deadlines together. Doctor reports the effective value and setup forwards an explicit selection. ([#4466](https://github.com/morluto/rea/pull/4466), [morluto/rea#3918](https://github.com/morluto/rea/issues/3918)). Thanks @jaychou813, @SandstromPL.
- **Electron window options:** Resolve stable local constant `BrowserWindow` options while retaining source provenance. Relevant writes and escapes keep the result unknown. ([#4400](https://github.com/morluto/rea/pull/4400), [morluto/rea#4366](https://github.com/morluto/rea/issues/4366)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **Language editions:** Add the Greek website edition and Malayalam documentation, with Korean typography and translation corrections across the site. ([#3520](https://github.com/morluto/rea/pull/3520), [#4792](https://github.com/morluto/rea/pull/4792), [#3821](https://github.com/morluto/rea/pull/3821), [morluto/rea#3820](https://github.com/morluto/rea/issues/3820), [#3799](https://github.com/morluto/rea/pull/3799), [morluto/rea#3798](https://github.com/morluto/rea/issues/3798), [#4344](https://github.com/morluto/rea/pull/4344), [#3856](https://github.com/morluto/rea/pull/3856), [#4166](https://github.com/morluto/rea/pull/4166)). Thanks @gensokyo-dev, @hi2242, @kalayciburak, @kaoru0822-kitauji, @kraken-503, @kukinghan, @N0zoM1z0, @yeseong0412.
### Fixes
- **Cancellation and cleanup:** Keep completed observations, exact cancellation reasons and the original physical failure through finalization. Failed cleanup retains its process or file owner for retry; recovered resources disappear from pending-cleanup reports without replacing the original error. ([#4238](https://github.com/morluto/rea/pull/4238), [morluto/rea#4232](https://github.com/morluto/rea/issues/4232), [#4274](https://github.com/morluto/rea/pull/4274), [morluto/rea#3923](https://github.com/morluto/rea/issues/3923), [3e82366](https://github.com/morluto/rea/commit/3e8236639abffa39deb5d5e939ec8baf6aaf3ca3), [16924e2](https://github.com/morluto/rea/commit/16924e275e939d0543ba01493324959a9e1d731c), [bbfe6ed](https://github.com/morluto/rea/commit/bbfe6edd60290f9bfbec68c8a0bcf605d8bbf5b3), [1f23e97](https://github.com/morluto/rea/commit/1f23e977b50301e91f02a4a17c9f8d48bb7b60f3), [08fbeb8](https://github.com/morluto/rea/commit/08fbeb8bdab27118b6c57039aef7c0fa8b6463bc)). Thanks @coygeek.
- **Provider startup:** Join prerequisite probes and in-flight readiness work before shutdown, preserve typed timeout and JVM-memory failures, and retain startup diagnostics through cleanup. Ghidra recognizes keg-only Homebrew JDKs and names the setting responsible for a startup timeout. ([376eeca](https://github.com/morluto/rea/commit/376eeca770820b31f7f8d4c5fcc397315621643e), [ceeff5a](https://github.com/morluto/rea/commit/ceeff5a41788e47e5aee96a4baf191a6ce7fde4b), [3ad1032](https://github.com/morluto/rea/commit/3ad1032b11edea2dcb054e78f951cc43159ae83d), [964decd](https://github.com/morluto/rea/commit/964decd989acfa84c4bfc3295362c4c46877fc20), [3fa972a](https://github.com/morluto/rea/commit/3fa972af7c7cb6cf014ffc426bdf7130dc6f2fd2), [#4557](https://github.com/morluto/rea/pull/4557), [morluto/rea#4556](https://github.com/morluto/rea/issues/4556), [#4590](https://github.com/morluto/rea/pull/4590), [morluto/rea#4241](https://github.com/morluto/rea/issues/4241)). Thanks @DanielCvz1, @J0UH, @Rinkia, @RishTheFish21, @vadlachary2002.
- **Client configuration:** Preserve the selected launcher, destination file, caller timeout choices and unresolved configuration evidence. Setup/uninstall recovery stays bound to the exact published file, keeps current verified backups, and transfers failed cleanup ownership for retry. ([d0253c9](https://github.com/morluto/rea/commit/d0253c99d65afbbd12732a2fdaf711640574e17f), [7e4eed9](https://github.com/morluto/rea/commit/7e4eed9dd48485b6a7100300aa98401811c2deb9), [2fb5e20](https://github.com/morluto/rea/commit/2fb5e204a35723f6f843d56ca47b0db94d8fcc44), [8235210](https://github.com/morluto/rea/commit/823521085a28f825e7fee1929d852d420042d1f7), [d818c3d](https://github.com/morluto/rea/commit/d818c3d2639d3115edb8a5269a754101aea6c3bd), [19329e4](https://github.com/morluto/rea/commit/19329e43bb38ca9d64512d801b25abcb188d2512), [dadf909](https://github.com/morluto/rea/commit/dadf909dce2e8b3d055541de69b4118d21276316), [333e0ab](https://github.com/morluto/rea/commit/333e0ab0dfc126a9915907d5079ed6195190e38e), [c15c206](https://github.com/morluto/rea/commit/c15c2062ad815cc9a3222ccfaeb7ef53cacd8b72)). Thanks @Adolanium.
- **Configuration formats:** Read signed 64-bit TOML integers without rounding, handle large TOML documents without argument-spread failures, preserve JSONC trivia and Hermes YAML scalar text, and reject NUL bytes in registration environment values. ([#4937](https://github.com/morluto/rea/pull/4937), [morluto/rea#4917](https://github.com/morluto/rea/issues/4917), [c43bc0b](https://github.com/morluto/rea/commit/c43bc0bf66a4e245a0cafa40def0d0207bcbd73c), [4fd28fc](https://github.com/morluto/rea/commit/4fd28fc9e0c4f8358147af38c72ab9e814f52d61), [#3568](https://github.com/morluto/rea/pull/3568), [#4335](https://github.com/morluto/rea/pull/4335)). Thanks @alfredo1491, @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0, @ShuaixinHuang, @xu91102.
- **Claude Code configuration:** Setup, doctor and uninstall now accept only the strict JSON format Claude Code reads, including a leading UTF-8 BOM. Comments and trailing commas are refused before backup or writing; clients that support JSONC retain their existing behavior. ([#4944](https://github.com/morluto/rea/pull/4944), [morluto/rea#4940](https://github.com/morluto/rea/issues/4940)). Thanks @kaoru0822-kitauji, @xu91102.
- **CLI output and dispatch:** Accept `--flag=value` and spaced global output values, keep array positions in filtered JSON, validate integer tokens before coercion, and exit quietly when the output reader closes. Cancellation remains effective through command finalization; app bundles and static native inspections select the intended workflow. ([#3811](https://github.com/morluto/rea/pull/3811), [#4823](https://github.com/morluto/rea/pull/4823), [2ae4e16](https://github.com/morluto/rea/commit/2ae4e1608f290c95ff7cff28c4d16a79b301d03d), [8d82579](https://github.com/morluto/rea/commit/8d8257928bbe8d43fedffc4db5fca26c8108820f), [#3781](https://github.com/morluto/rea/pull/3781), [279c590](https://github.com/morluto/rea/commit/279c590ef6bb737d4a48ea696a0ae5eb30010b3d), [#3511](https://github.com/morluto/rea/pull/3511), [morluto/rea#3321](https://github.com/morluto/rea/issues/3321), [f31b231](https://github.com/morluto/rea/commit/f31b231a92a61be6bd1800b16be0b3210ae82944)). Thanks @arjunprabhulal, @coygeek, @kaoru0822-kitauji, @ShuaixinHuang, @Sriram-PR, @Tapraj-Pathak.
- **Updates:** Route `--update` through the verified updater and bound retained package-manager diagnostics. ([4f7b5fb](https://github.com/morluto/rea/commit/4f7b5fb6023ce2ec449561782d09e3c7e5ac7c7b), [01cdbbd](https://github.com/morluto/rea/commit/01cdbbdc556262c47d04a5b3391dd9c17946d3fc)).
- **MCP contracts:** Advertise self-contained tool inputs while preserving tuple, array, Boolean and nested-JSON constraints across schema profiles. Schema-reference handling no longer rewrites literal metadata; catalog identity includes the output-schema profile. ([5e30969](https://github.com/morluto/rea/commit/5e30969cc5f1255d44ec7a1f1ca8dbac0af5e7c4), [10aa74d](https://github.com/morluto/rea/commit/10aa74dfedc3ae776f50d1fa1eb83b64ae82d3d6), [204b9f3](https://github.com/morluto/rea/commit/204b9f3ac140a993c3e458bb0e117de31818ab29), [b3389a5](https://github.com/morluto/rea/commit/b3389a5b0834265fe7a40391be315d810a53a97e), [a2e3016](https://github.com/morluto/rea/commit/a2e30168a046f3b79319eb7fbaf807b802104199), [ba98130](https://github.com/morluto/rea/commit/ba98130049fa40ade66b03611ff70c7ddc011a1d), [c5fb86d](https://github.com/morluto/rea/commit/c5fb86d1948af0a287a581b29b2b1d554743c607)). Thanks @akushonkamen, @HarsH-1877, @kaoru0822-kitauji, @xingzihai.
- **Session ordering:** Keep snapshot admission ahead of later calls on the same connection, retain shutdown signals until cleanup succeeds, and preserve inspection authority through investigation publication. ([#3661](https://github.com/morluto/rea/pull/3661), [morluto/rea#3334](https://github.com/morluto/rea/issues/3334), [8873b39](https://github.com/morluto/rea/commit/8873b39770e89eae22b7693bc492d7e423b22bea), [663d07f](https://github.com/morluto/rea/commit/663d07f02cd62c99d81c59391d2c6d745c875335)). Thanks @coygeek, @rudycelekli, @Swatantra-66.
- **JavaScript exports and resolution:** Retain CommonJS exports created with `Object.defineProperty` and track export receiver lifetimes. Resolve scoped packages, package `main`, JSON requires and `.mts`/`.cts` imports using the owning loader; overwritten `require` bindings and erased type-only imports no longer create false runtime links. ([#4725](https://github.com/morluto/rea/pull/4725), [morluto/rea#4724](https://github.com/morluto/rea/issues/4724), [#3567](https://github.com/morluto/rea/pull/3567), [morluto/rea#3541](https://github.com/morluto/rea/issues/3541), [#4838](https://github.com/morluto/rea/pull/4838), [morluto/rea#4632](https://github.com/morluto/rea/issues/4632), [#3844](https://github.com/morluto/rea/pull/3844), [morluto/rea#3383](https://github.com/morluto/rea/issues/3383), [#4164](https://github.com/morluto/rea/pull/4164), [morluto/rea#3915](https://github.com/morluto/rea/issues/3915), [#4223](https://github.com/morluto/rea/pull/4223), [#3801](https://github.com/morluto/rea/pull/3801), [morluto/rea#3393](https://github.com/morluto/rea/issues/3393), [#4165](https://github.com/morluto/rea/pull/4165), [morluto/rea#3932](https://github.com/morluto/rea/issues/3932)). Thanks @Abishek0070, @coygeek, @JNTMTMTM, @kaoru0822-kitauji, @prasad2512, @rodricksz4h5, @xu91102.
- **JavaScript value flow:** Preserve values captured before later assignments, guarded factory fallthrough and nested destructuring. Track exact property keys and setter writes; direct `eval`, compound assignments and incoming callbacks preserve unresolved effects instead of inventing certainty. ([#4196](https://github.com/morluto/rea/pull/4196), [morluto/rea#3881](https://github.com/morluto/rea/issues/3881), [#4341](https://github.com/morluto/rea/pull/4341), [morluto/rea#3963](https://github.com/morluto/rea/issues/3963), [bfb7ed0](https://github.com/morluto/rea/commit/bfb7ed0bd43edc605e00fc52c6e21e735b285904), [#3480](https://github.com/morluto/rea/pull/3480), [#3831](https://github.com/morluto/rea/pull/3831), [morluto/rea#3312](https://github.com/morluto/rea/issues/3312), [9ac76d3](https://github.com/morluto/rea/commit/9ac76d3d9486856d25395b413d7ac5ffa744cad0), [#4342](https://github.com/morluto/rea/pull/4342), [morluto/rea#4206](https://github.com/morluto/rea/issues/4206), [#4243](https://github.com/morluto/rea/pull/4243), [morluto/rea#3880](https://github.com/morluto/rea/issues/3880)). Thanks @coygeek, @kaoru0822-kitauji, @maxazure, @N0zoM1z0, @ShuaixinHuang.
- **JavaScript relationships:** Distinguish `Map` reads from network endpoints, associate response endpoints only with writes that reach the read, and preserve child-process receiver ownership and effective stdio choices. Version comparisons keep relocation and unknown relationships distinct from application changes. ([fbfdb6a](https://github.com/morluto/rea/commit/fbfdb6abe9d663be0be146a15a489e33ccba0f68), [#4759](https://github.com/morluto/rea/pull/4759), [morluto/rea#4606](https://github.com/morluto/rea/issues/4606), [#3819](https://github.com/morluto/rea/pull/3819), [morluto/rea#3362](https://github.com/morluto/rea/issues/3362), [#4336](https://github.com/morluto/rea/pull/4336), [morluto/rea#3355](https://github.com/morluto/rea/issues/3355), [556504e](https://github.com/morluto/rea/commit/556504edcbb0f8cd9eaf81a91b18ff891b8f5457), [4d980d0](https://github.com/morluto/rea/commit/4d980d017622d3175d86aaf3b0dcfbdf4ca3d091)). Thanks @Abdooo2235, @coygeek, @kaoru0822-kitauji, @Xinglan233.
- **JavaScript source admission:** Parse JavaScript with its own grammar, refuse FIFO recovery inputs without blocking, and report oversized or resource-skipped files with their actual capacity and coverage state. Malformed runtime events stay excluded rather than becoming policy-filtered facts. ([#3782](https://github.com/morluto/rea/pull/3782), [morluto/rea#3780](https://github.com/morluto/rea/issues/3780), [#3788](https://github.com/morluto/rea/pull/3788), [morluto/rea#3332](https://github.com/morluto/rea/issues/3332), [#4757](https://github.com/morluto/rea/pull/4757), [morluto/rea#4629](https://github.com/morluto/rea/issues/4629), [#3852](https://github.com/morluto/rea/pull/3852), [morluto/rea#3670](https://github.com/morluto/rea/issues/3670), [#4755](https://github.com/morluto/rea/pull/4755), [morluto/rea#4685](https://github.com/morluto/rea/issues/4685)). Thanks @andresaragon, @coygeek, @kaoru0822-kitauji, @sx4im.
- **Browser capture:** Bound CDP handshakes and command responses, preserve malformed inventory uncertainty and partial cleanup diagnostics, and report cancellation after content reads and final cleanup. Accessibility comparisons require observed coverage; attached emulation teardown limits remain explicit. ([#3555](https://github.com/morluto/rea/pull/3555), [morluto/rea#3553](https://github.com/morluto/rea/issues/3553), [#3901](https://github.com/morluto/rea/pull/3901), [morluto/rea#3379](https://github.com/morluto/rea/issues/3379), [4cc044b](https://github.com/morluto/rea/commit/4cc044be68f813e96907e6cedcf422d0a3170de3), [#4406](https://github.com/morluto/rea/pull/4406), [morluto/rea#4403](https://github.com/morluto/rea/issues/4403), [#2893](https://github.com/morluto/rea/pull/2893), [0123d59](https://github.com/morluto/rea/commit/0123d5976828bfe621bd51daa4cf39fee48a1349), [2ff0867](https://github.com/morluto/rea/commit/2ff086796ad7b8f60397956750584e170dbe2910)). Thanks @coygeek, @dakjdakd, @kaoru0822-kitauji, @maxazure, @N0zoM1z0, @YunzhouLi-hub.
- **Browser page facts:** Preserve URL path bytes when removing credentials, repeated document `Link` headers and complete COOP/COEP structured items. WebMCP discovery reads the actual schema properties and exact receiver path. ([#4731](https://github.com/morluto/rea/pull/4731), [df2fc25](https://github.com/morluto/rea/commit/df2fc251e318a1255309dba232f9f73c700777b6), [c360cfc](https://github.com/morluto/rea/commit/c360cfcd6d7901514b16dd6113eba2e0b0fdf422), [ebd5a82](https://github.com/morluto/rea/commit/ebd5a82c439f6deb3a06062225ce009eb03257cf), [38f92d1](https://github.com/morluto/rea/commit/38f92d108e3148cdf47e4ffbfe3de8a76e795d51)). Thanks @Abdooo2235, @geekwolf, @ShuaixinHuang.
- **Electron runtime capture:** Keep native IPC listener and one-shot dispatch behavior, block synchronous shell effects during capture, refuse active scenarios when the inspect fuse is disabled, and preserve fuse/transport cleanup ownership. ([#4389](https://github.com/morluto/rea/pull/4389), [morluto/rea#4390](https://github.com/morluto/rea/issues/4390), [#4605](https://github.com/morluto/rea/pull/4605), [morluto/rea#4593](https://github.com/morluto/rea/issues/4593), [#4562](https://github.com/morluto/rea/pull/4562), [morluto/rea#4393](https://github.com/morluto/rea/issues/4393), [#3605](https://github.com/morluto/rea/pull/3605), [morluto/rea#3453](https://github.com/morluto/rea/issues/3453), [f9de413](https://github.com/morluto/rea/commit/f9de41371f7536e2e22ce6e43c9314fa2f238a30), [e7df00e](https://github.com/morluto/rea/commit/e7df00e80777757b329bbe8aba6d421d8892b9d5)). Thanks @Adolanium, @gensokyo-dev, @gkobayashi0, @HarsH-1877, @kaoru0822-kitauji, @keeponlight, @N0zoM1z0, @Nitin13-git, @xu91102.
- **APK resource accuracy:** Retain completed tables and decode facts after interrupted or failed reads. Derive locale limits from resource configuration fields, disclose unselectable qualified tables, distinguish document failures from omitted entries, and bind Evidence to the decoded snapshot. ([34a88d6](https://github.com/morluto/rea/commit/34a88d6c1674710953e2c94bd0a17332cd69ca6f), [f5fcf52](https://github.com/morluto/rea/commit/f5fcf5216ffb3a266b8c00f6d9a0943051084ab5), [6385ef4](https://github.com/morluto/rea/commit/6385ef4355d7a794b3154ee38529cbddee583336), [98886ad](https://github.com/morluto/rea/commit/98886adddfcbd7b4f4236b3a9825f91878c3cb11), [dc2e811](https://github.com/morluto/rea/commit/dc2e8117b60c4481372ed89b077e6c1cf81ee653), [#4771](https://github.com/morluto/rea/pull/4771), [morluto/rea#4768](https://github.com/morluto/rea/issues/4768)). Thanks @abhay-2108, @gensokyo-dev, @Iyanuoluwa007, @kaoru0822-kitauji, @N0zoM1z0.
- **Apktool guidance:** Retain the observed version and disclose the matching platform/release advisory windows without treating a version string as proof of decoder safety. Missing prerequisites and decode refusals identify the selected APK or launcher. ([d38f099](https://github.com/morluto/rea/commit/d38f099692394ec90c582f9f0a14b036b421b160), [63696d2](https://github.com/morluto/rea/commit/63696d277328f556c971d1fc9f81ae83128d468b), [cc038da](https://github.com/morluto/rea/commit/cc038da218cada68c87700477aba3db4739791ff)). Thanks @GovinduSasanka, @peter-research.
- **Android method bodies:** Recover original bytecode bodies when Java source is omitted while preserving genuine native/abstract body absence and primary failures during retryable cleanup. ([63c65a2](https://github.com/morluto/rea/commit/63c65a22430a4bbc0c1807a0298edb867775001f), [742b6f5](https://github.com/morluto/rea/commit/742b6f587f3c4ef1b8212e7376d6f3a305b194dc), [9fa6d2b](https://github.com/morluto/rea/commit/9fa6d2b254cc390665974efb52463386a029e9fc)). Thanks @balkan16, @Chessing234.
- **ADB observations and transfers:** Preserve display-scoped focus, distinguish missing declarations from absent focus, retain numeric physical density and mark unknown permission coverage partial. Transfers use consumed snapshot identity and refuse unknown destination existence; uninstall requests explicitly select user zero. ([7086768](https://github.com/morluto/rea/commit/7086768a27242a15daa93486aefbaacc3c891e78), [3d028db](https://github.com/morluto/rea/commit/3d028dbc74a465f6f8df1813a931fbb5362e7dc3), [#4074](https://github.com/morluto/rea/pull/4074), [morluto/rea#3942](https://github.com/morluto/rea/issues/3942), [#4663](https://github.com/morluto/rea/pull/4663), [#4665](https://github.com/morluto/rea/pull/4665), [1342127](https://github.com/morluto/rea/commit/1342127e91d4db8b4113a7a4e85be909bc96e1db), [61e6880](https://github.com/morluto/rea/commit/61e688070f48016b6d178cd04c02fcc2f33e19ad), [be0b2d1](https://github.com/morluto/rea/commit/be0b2d138c46bb951fc3c8ee57ef7b0d57fed312)). Thanks @Copilot, @coygeek, @gongfpp, @Tapraj-Pathak, @yankikucuk.
- **Apple metadata:** Preserve exact plist/keyed-archive integers, negative zero, non-finite real identity and dictionary keys. Reject malformed XML before value decoding, retain undecodable dates as unknown and report unsupported OpenStep entries explicitly. ([#4324](https://github.com/morluto/rea/pull/4324), [morluto/rea#4292](https://github.com/morluto/rea/issues/4292), [2cf9c28](https://github.com/morluto/rea/commit/2cf9c28f27261b7e64af2ba7ecb4e23ea7714aee), [#4839](https://github.com/morluto/rea/pull/4839), [morluto/rea#4834](https://github.com/morluto/rea/issues/4834), [f2b7bbb](https://github.com/morluto/rea/commit/f2b7bbb31279a820f4cf32d439570f8e5381d010), [6c98708](https://github.com/morluto/rea/commit/6c98708793c23615de8053b2cd14d9f164b960c5), [78cf610](https://github.com/morluto/rea/commit/78cf6100f33ad2412c0a3d60b83f8a44a43fbbb5), [c7e9e09](https://github.com/morluto/rea/commit/c7e9e096d3e2444e4c251123b8e914e9c0941f4e), [#3793](https://github.com/morluto/rea/pull/3793), [#4105](https://github.com/morluto/rea/pull/4105), [morluto/rea#3892](https://github.com/morluto/rea/issues/3892)). Thanks @coygeek, @kaoru0822-kitauji, @mertkayacs, @pablodiazjorge, @pollychen-lab, @xu91102.
- **Apple application structure:** Derive bundle components and launchd domains from their owning directories, retain unavailable components, and select only exact `Assets.car` names as asset catalogs. Interface Builder keeps unresolved classes and counts hierarchy records beyond the returned object page. ([149a0fc](https://github.com/morluto/rea/commit/149a0fc06c3c67ee45624e6da14dfb0754473758), [5d6029e](https://github.com/morluto/rea/commit/5d6029ea62a65ec34fd739ee40223c77c6fdb63b), [c192c58](https://github.com/morluto/rea/commit/c192c58815ddeaf5cf30159e7082c742c0bc36b7), [#4094](https://github.com/morluto/rea/pull/4094), [morluto/rea#3914](https://github.com/morluto/rea/issues/3914), [#4118](https://github.com/morluto/rea/pull/4118), [morluto/rea#3958](https://github.com/morluto/rea/issues/3958), [#4162](https://github.com/morluto/rea/pull/4162), [morluto/rea#3950](https://github.com/morluto/rea/issues/3950)). Thanks @coygeek, @kaoru0822-kitauji.
- **Native names and layout:** Keep valid Objective-C records beside malformed neighbors, preserve leading U+FEFF and repeated load-command fields, and select exact arm64e.x1 slices. PE resource ownership and NativeAOT reads stay within declared image/header ranges. ([#3833](https://github.com/morluto/rea/pull/3833), [morluto/rea#3336](https://github.com/morluto/rea/issues/3336), [#4549](https://github.com/morluto/rea/pull/4549), [morluto/rea#4540](https://github.com/morluto/rea/issues/4540), [morluto/rea#4541](https://github.com/morluto/rea/issues/4541), [#4730](https://github.com/morluto/rea/pull/4730), [#4048](https://github.com/morluto/rea/pull/4048), [morluto/rea#4008](https://github.com/morluto/rea/issues/4008), [#3500](https://github.com/morluto/rea/pull/3500), [morluto/rea#3389](https://github.com/morluto/rea/issues/3389), [#4463](https://github.com/morluto/rea/pull/4463), [morluto/rea#4373](https://github.com/morluto/rea/issues/4373), [morluto/rea#4455](https://github.com/morluto/rea/issues/4455), [3fc0075](https://github.com/morluto/rea/commit/3fc007534bb2104d662b43cc712281f9d1d6a7d3), [4a076f3](https://github.com/morluto/rea/commit/4a076f368c62bdfb19415accad342e03a38d1005)). Thanks @coygeek, @Iyanuoluwa007, @jayinvers, @kaoru0822-kitauji, @kevinjosh10, @maxazure, @ShuaixinHuang.
- **Native analysis providers:** Keep Ghidra byte matches within examined coverage, retain PE PDB/function-boundary limitations and preserve configured MIPS choices. Attached IDA refuses universal Mach-O rather than silently choosing a slice; request timeout identity and captured output survive adapter handling. ([#4531](https://github.com/morluto/rea/pull/4531), [#2933](https://github.com/morluto/rea/pull/2933), [morluto/rea#2259](https://github.com/morluto/rea/issues/2259), [01a1b3f](https://github.com/morluto/rea/commit/01a1b3fda6a7dacd24a0b10619d41d12153080d4), [50dd9fc](https://github.com/morluto/rea/commit/50dd9fcad1efb47cb404192c30bdf190dcdaca58), [#4765](https://github.com/morluto/rea/pull/4765), [morluto/rea#3871](https://github.com/morluto/rea/issues/3871), [c0cfccb](https://github.com/morluto/rea/commit/c0cfccb45fe43e7eeff18e56c64caa5b121988f9)). Thanks @BeLit-Development-2, @coygeek, @kaoru0822-kitauji, @N0zoM1z0, @ShuaixinHuang.
- **Recorded crashes:** Prevent GDB from discovering an executable named by an untrusted core, reject core mapping paths that escape the isolated sysroot, and preserve pwntools worker ownership across launcher exec. Completed decode facts and cancellation reasons remain available through failed final cleanup. ([5bf4fff](https://github.com/morluto/rea/commit/5bf4fffc1b835130ddb16f96db9ce13b639591db), [b33c0df](https://github.com/morluto/rea/commit/b33c0df02e93fabca00a44f2de625b1a63637ee9), [#4435](https://github.com/morluto/rea/pull/4435), [morluto/rea#4011](https://github.com/morluto/rea/issues/4011)). Thanks @coygeek, @N0zoM1z0.
- **Cutter and Rizin:** Read Cutter’s active document from core configuration and attribute its host version correctly. Keep interrupted Rizin frames and output whitespace before completion markers, with exact command and cleanup failure attribution. ([#4507](https://github.com/morluto/rea/pull/4507), [morluto/rea#4497](https://github.com/morluto/rea/issues/4497), [#4514](https://github.com/morluto/rea/pull/4514), [morluto/rea#4498](https://github.com/morluto/rea/issues/4498), [#4459](https://github.com/morluto/rea/pull/4459), [morluto/rea#4015](https://github.com/morluto/rea/issues/4015), [d5fc7e6](https://github.com/morluto/rea/commit/d5fc7e6ebf9af050081fc4c6e18b9079cfaa5cdc), [199396f](https://github.com/morluto/rea/commit/199396f61c8f61da326d059606a992304a760147)). Thanks @coygeek, @N0zoM1z0, @Xander-AJ.
- **Windows cleanup:** Retain unpublished runtimes after failed removal, handle read-only attributes and sharing conflicts, preserve primary snapshot failures and honor caller environment overrides. ([f78d137](https://github.com/morluto/rea/commit/f78d137b42741a0ed5fb8ac161286f9a50220c4d), [d69acc3](https://github.com/morluto/rea/commit/d69acc3455c33f90a2fc4381db43d38b6af50622), [0679a79](https://github.com/morluto/rea/commit/0679a7972b4f2b02eee6a86a44f50f4c2967d3cb), [5f9ac2c](https://github.com/morluto/rea/commit/5f9ac2c98a6fe11988c41afb737df9b6d5c3a1b3), [1bbf4bd](https://github.com/morluto/rea/commit/1bbf4bd16ca134cdb2187eaf29291cae90d14a57)). Thanks @DeryFerd.
- **Managed metadata:** Compare nominal signatures and declaring identities across builds, preserve empty names without display substitutions, verify ABI-prefixed Mach-O exports exactly, and charge cached metadata once. Metadata errors report artifact file offsets; mixed-mode assemblies retain their actual boundary classification. ([#3059](https://github.com/morluto/rea/pull/3059), [#3079](https://github.com/morluto/rea/pull/3079), [#3853](https://github.com/morluto/rea/pull/3853), [morluto/rea#3352](https://github.com/morluto/rea/issues/3352), [#3974](https://github.com/morluto/rea/pull/3974), [morluto/rea#3961](https://github.com/morluto/rea/issues/3961), [#3491](https://github.com/morluto/rea/pull/3491), [morluto/rea#3449](https://github.com/morluto/rea/issues/3449), [#4418](https://github.com/morluto/rea/pull/4418), [morluto/rea#4372](https://github.com/morluto/rea/issues/4372), [#3516](https://github.com/morluto/rea/pull/3516), [morluto/rea#3366](https://github.com/morluto/rea/issues/3366)). Thanks @8rulerstar, @coygeek, @duckhunt113, @Hahaa909, @JayOfTheKeyboard, @kaoru0822-kitauji, @keeponlight, @N0zoM1z0, @tomyrms.
- **Archive and filesystem identity:** Reject colliding extraction paths before writing, classify ZIP offsets beyond EOF as malformed, normalize ZIP current-directory prefixes once, and preserve symlink and unreadable-path provenance. A directory named `*.map` is no longer mistaken for a source-map file. ([862a0b2](https://github.com/morluto/rea/commit/862a0b2631785492f8dc20c8acd396646c545fd1), [c209dcb](https://github.com/morluto/rea/commit/c209dcb209118069f123e789949eeba1ef78853e), [6b1c36e](https://github.com/morluto/rea/commit/6b1c36e336eeacbb7159b211e687b4b84f99b9f5), [25c2222](https://github.com/morluto/rea/commit/25c22222abe7a681491bdb4d7316250fdcf32973), [d6e4fe4](https://github.com/morluto/rea/commit/d6e4fe4a4dd8914ec3db50d2a7fc3d0802649262), [#4131](https://github.com/morluto/rea/pull/4131), [morluto/rea#3909](https://github.com/morluto/rea/issues/3909)). Thanks @Adkid-Zephyr, @coygeek, @jackie-cqz, @kaoru0822-kitauji, @maxazure, @XY-code1.
- **Evidence and reconstruction:** Authenticate all retained inventory substeps, preserve source-attributed caveats in composed workflows, and require comparable cited Evidence before declaring reconstruction coverage or readiness. Missing Evidence and residual replay uncertainty stay explicit. ([9ac91e0](https://github.com/morluto/rea/commit/9ac91e0c1e13e91cd21341635ad5be89b0e8642d), [c4608a2](https://github.com/morluto/rea/commit/c4608a29620ea3e7bb8259b47357b942fe4e4709), [#4075](https://github.com/morluto/rea/pull/4075), [morluto/rea#3357](https://github.com/morluto/rea/issues/3357), [#3526](https://github.com/morluto/rea/pull/3526), [morluto/rea#3363](https://github.com/morluto/rea/issues/3363), [153017d](https://github.com/morluto/rea/commit/153017d750322188c1a112b2284c1b8065c67320)). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0, @rudycelekli.
- **Offline data:** Preserve exact UTC packet-capture instants, SQLite column identity and snapshot admission failures, and cooked WASM directive literals. Cleanup cancellation retains completed EVM/WASM observations rather than losing them. ([796faa0](https://github.com/morluto/rea/commit/796faa0e4e6bf745bcdc33acef65af8ccfc3622d), [748ac7f](https://github.com/morluto/rea/commit/748ac7f0df340f0d0569eca2e803a59cd51ffdbd), [bc51d0e](https://github.com/morluto/rea/commit/bc51d0ea5df7dcee1fcc42dfae3ab0c62c9cca5e), [4f87f02](https://github.com/morluto/rea/commit/4f87f02bb19bf77349a683a5bcb2da2d1da427d0), [d694aef](https://github.com/morluto/rea/commit/d694aef227e90164e6654ea9e5ce6a854d099404), [321cacb](https://github.com/morluto/rea/commit/321cacb84a020f447e1110df5c58fa71d99b7bf1)). Thanks @himamaheswarreddy, @mertkayacs, @rollroyces, @viniciusdsandrade.
- **Kuna recovery:** Share concurrent single-function requests and cache repeats, validate warm exports once, stream string inventories beyond the former fixed buffer ceiling, and keep unproven processes in their run slots until close confirms release. Failure messages distinguish final buffer refusals from configurable deadlines. ([#4045](https://github.com/morluto/rea/pull/4045), [#4169](https://github.com/morluto/rea/pull/4169), [#4049](https://github.com/morluto/rea/pull/4049), [#4318](https://github.com/morluto/rea/pull/4318), [#4052](https://github.com/morluto/rea/pull/4052), [#4098](https://github.com/morluto/rea/pull/4098), [#4343](https://github.com/morluto/rea/pull/4343)). Thanks @kaoru0822-kitauji.
- **Flutter snapshots:** Check symbol-table ranges against loadable segments and retain completed inspection facts when cancellation occurs during snapshot cleanup. ([#3859](https://github.com/morluto/rea/pull/3859), [morluto/rea#3346](https://github.com/morluto/rea/issues/3346), [#4766](https://github.com/morluto/rea/pull/4766), [morluto/rea#4717](https://github.com/morluto/rea/issues/4717), [#4762](https://github.com/morluto/rea/pull/4762), [morluto/rea#4640](https://github.com/morluto/rea/issues/4640)). Thanks @coygeek, @kaoru0822-kitauji.
- **Native metadata recovery:** Keep unknown Objective-C category classes, empty protocol names, undecoded Swift aliases and unfinished property types explicit instead of discarding neighboring valid records. Preserve unnamed segment identity and exclude a dylib’s own install name from dependencies. ([#4093](https://github.com/morluto/rea/pull/4093), [morluto/rea#4012](https://github.com/morluto/rea/issues/4012), [#4078](https://github.com/morluto/rea/pull/4078), [morluto/rea#3873](https://github.com/morluto/rea/issues/3873), [#4085](https://github.com/morluto/rea/pull/4085), [morluto/rea#3934](https://github.com/morluto/rea/issues/3934), [#3825](https://github.com/morluto/rea/pull/3825), [morluto/rea#3345](https://github.com/morluto/rea/issues/3345), [#4051](https://github.com/morluto/rea/pull/4051), [morluto/rea#3875](https://github.com/morluto/rea/issues/3875), [#4050](https://github.com/morluto/rea/pull/4050), [morluto/rea#3872](https://github.com/morluto/rea/issues/3872)). Thanks @coygeek, @kaoru0822-kitauji.
- **Document selection:** Preserve explicitly selected plist paths and bundle defaults, and only use document fallback after confirming its applicability. Finalization keeps completed selection facts and cancellation outcomes. ([e6c4877](https://github.com/morluto/rea/commit/e6c487799bb64f559e66b69be1863e6ec5187548), [#3774](https://github.com/morluto/rea/pull/3774), [9df559b](https://github.com/morluto/rea/commit/9df559b7e712298a761ff26e9caae8d659b7e54b)). Thanks @kaoru0822-kitauji.
- **Capture and comparison accuracy:** Admit schema-valid HAR numeric values while retaining exact-publication limits, preserve empty URL path segments, and keep unrelated process traces and unnamed reference endpoints distinct or unknown. ([#3159](https://github.com/morluto/rea/pull/3159), [morluto/rea#3145](https://github.com/morluto/rea/issues/3145), [#3792](https://github.com/morluto/rea/pull/3792), [morluto/rea#3307](https://github.com/morluto/rea/issues/3307), [#4178](https://github.com/morluto/rea/pull/4178), [morluto/rea#3931](https://github.com/morluto/rea/issues/3931), [#3809](https://github.com/morluto/rea/pull/3809), [morluto/rea#3388](https://github.com/morluto/rea/issues/3388)). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0, @pablodiazjorge, @xu91102.
- **Evidence publication:** Recheck staged output identity after its final write, classify missing review Evidence and duplicate ledger entries as caller input, and preserve the selected freshness requirement in comparison parameters. ([#4764](https://github.com/morluto/rea/pull/4764), [morluto/rea#4613](https://github.com/morluto/rea/issues/4613), [#4452](https://github.com/morluto/rea/pull/4452), [morluto/rea#4444](https://github.com/morluto/rea/issues/4444), [#4065](https://github.com/morluto/rea/pull/4065), [morluto/rea#4000](https://github.com/morluto/rea/issues/4000)). Thanks @amyanger, @coygeek, @kaoru0822-kitauji, @Rinkia, @Swatantra-66.
### ⚠ BREAKING CHANGES
- **Saved Evidence:** Evidence and analysis snapshots now require explicit nullable `relationship_coverage` in their identity commitments. Earlier records whose digest omitted null coverage are incompatible even when the serialized field is explicitly null. Artifact inventories require scoped coverage and explicit `integrity_contradictions`; obsolete `coverage_status` and prose-based coverage inference are removed. Regenerate old Evidence/snapshots and inventory observations with this release: saved files are not migrated and omitted facts are not filled with guessed defaults. ([db8128d](https://github.com/morluto/rea/commit/db8128dc0d1c4f731f3d19877fa00d25cf17f657), [0405c9d](https://github.com/morluto/rea/commit/0405c9d361de6d904dfa16cc0b6d8350407f2c9b), [a9588be](https://github.com/morluto/rea/commit/a9588be4ebdd20eeda975cb63ffcdb1cd868b340)).
- **Browser capture results:** Consumers must retain resolved capture limits, event families and omission facts. Set desired budgets through the supported capture options; omitted caller inputs still select defaults, but stored observations must report the resolved values. Admitted content reads queue behind bounded concurrency. ([57a23f0](https://github.com/morluto/rea/commit/57a23f0a2da0fdc3835e203718ded8acea5a34e6)).
- **Android version codes:** `inspect_adb_package.version_code` is a decimal string rather than a JSON number. Read it as a string, or use `BigInt` for numeric comparison. ([#3485](https://github.com/morluto/rea/pull/3485), [morluto/rea#3368](https://github.com/morluto/rea/issues/3368), [7666a48](https://github.com/morluto/rea/commit/7666a482b69dc7cabdbac3bbaf89a43e32300192)). Thanks @coygeek, @kaoru0822-kitauji.
- **Native UI failures:** Failed or cancelled native UI scenarios now return typed errors with the collected report in `partial_observation`. Handle the error path and read that report instead of expecting a successful result containing failed steps. ([d909919](https://github.com/morluto/rea/commit/d9099190ab3a819bd3a9fd62a8714cef398016d5)).
- **Provider status:** `binary_session` candidate availability can be unknown during owner finalization. Preserve that state rather than treating it as unavailable; internal consumers must await the asynchronous session status API. ([150e996](https://github.com/morluto/rea/commit/150e996a86f2bec4982ba87f37bf502a8d5e8214)).
- **Swift symbol locations:** `swift_symbols` entries may have a null `address` when the provider supplies no location. Handle unknown addresses; optional `evidence.reported_address` retains the original provider value. ([8116dc0](https://github.com/morluto/rea/commit/8116dc06512b4609d8db63cebc9f2ff8cb964151)).
- **Linux Hopper admission:** Older builds previously admitted by a fixed allowlist need retained publisher-signed package evidence or an approved update to the current public demo. Update explicit `HOPPER_LAUNCHER_PATH` selections when migrating; the live feed offers only its current release. The official RPM metadata mismatch remains unresolved and is refused. ([#3590](https://github.com/morluto/rea/pull/3590), [morluto/rea#3577](https://github.com/morluto/rea/issues/3577), [70cd4cb](https://github.com/morluto/rea/commit/70cd4cbfb849b61f05d97d2690372a9782831426)). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
Full contribution record: [571 commits and 236 merged PRs](https://github.com/morluto/rea/blob/rea-agents-6.6.0/docs/releases/6.6.0.contributions.json).
## [6.5.0](https://github.com/morluto/rea/compare/rea-agents-6.4.0...rea-agents-6.5.0) (2026-10-13)
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/development/testing.md).
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/testing.md).
### Highlights
@@ -220,7 +30,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
- **HarmonyOS packages:** inventory HAP/HSP/App Pack structures without executing code. The recorded real workflow covers one pinned HAP through repository-built CLI/MCP; installed-package and real HSP/App Pack coverage remain unverified. ([#1447](https://github.com/morluto/rea/pull/1447)). Thanks @BaaRai.
- **Ghidra raw inputs and references:** compatible Linux/macOS Ghidra profiles accept caller-selected raw architecture/entry information and expose typed incoming references at exact addresses. Windows Ghidra P0 admits the raw NMOS 6502 profile described below; other raw targets and the incoming-reference facet remain outside Windows P0. ([#1868](https://github.com/morluto/rea/pull/1868), [issue morluto/rea#1836](https://github.com/morluto/rea/issues/1836), [#1715](https://github.com/morluto/rea/pull/1715), [issue morluto/rea#1245](https://github.com/morluto/rea/issues/1245)). Thanks @bugstan, @casesolved-co-uk, @mvanhorn, @N0zoM1z0.
- **Ghidra MIPS:** Add bounded MIPS32r2/o32 relocatable objects, soft-float integer-register storage, admitted MIPS16 shared ELF, and a narrowly admitted recordless legacy MIPS32 executable profile. The latter pins an endian-matched 32-bit language and records inferred o32 as an analysis assumption. Admission follows each documented ABI/flag/relocation boundary. ([#1949](https://github.com/morluto/rea/pull/1949), [issue morluto/rea#1937](https://github.com/morluto/rea/issues/1937), [#1950](https://github.com/morluto/rea/pull/1950), [issue morluto/rea#1938](https://github.com/morluto/rea/issues/1938), [#1991](https://github.com/morluto/rea/pull/1991), [issue morluto/rea#1851](https://github.com/morluto/rea/issues/1851), [#1898](https://github.com/morluto/rea/pull/1898), [issue morluto/rea#1620](https://github.com/morluto/rea/issues/1620)). Thanks @AurelioB, @BaaRai, @bqw371, @kaoru0822-kitauji, @N0zoM1z0.
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/providers/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
- **PSP static ELF profile:** the Ghidra profile is Linux x64, bounded little-endian ELF32 `ET_EXEC` with the documented PSP/ABI declarations and a compatible caller-installed Allegrex extension. PRX, shared/relocatable images, PBP/ISO/CSO and runtime emulation are outside scope. Historical extension experiments are separate from public-provider verification; real public Ghidra PSP coverage remains unverified. ([#1599](https://github.com/morluto/rea/pull/1599)). Thanks @jaelliot.
- **Ghidra analysis settings:** apply validated analyzer overrides before analysis while retaining the selected analysis profile. ([#1913](https://github.com/morluto/rea/pull/1913)). Thanks @akushonkamen, @kaoru0822-kitauji, @N0zoM1z0.
- **Attached native IDA MCP:** add an optional native bridge alongside existing legacy/headless profiles. Recorded native runs use Linux IDA Free 9.4 with a navigation correction and Windows IDA Pro 9.3 with a loopback-listener change; stock native switching, native macOS and Windows IDA Free remain unverified. Attached databases remain read-only and are never saved or closed. ([#1544](https://github.com/morluto/rea/pull/1544)). Thanks @imissapixel, @N0zoM1z0.
@@ -292,7 +102,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
- **Source maps:** original-source and mapping rows preserve a null source name when unavailable. Handle null explicitly instead of expecting an invented unknown-source string; new index fields remain optional when reading older Evidence. ([ee50ed1](https://github.com/morluto/rea/commit/ee50ed121e38cc6e1a2ff0dc8b789f15f71c5402)).
- **Streamed filtered JSON:** `--filter-output` now emits JSON `null` instead of `{}` when the selected value is null; consumers must distinguish a present null from a missing selection. Use `--full-output` when an object envelope is required. ([#2333](https://github.com/morluto/rea/pull/2333)). Thanks @Jackmeson1.
- **Call graphs:** `get_call_graph.normalized_result` is now `{ graph, limitations }`; read traversal-depth groups under `graph` and retain relationship-specific caveats. Contract/fixture validation does not establish new Ghidra or IDA engine coverage. ([5dfb9d0](https://github.com/morluto/rea/commit/5dfb9d050c2cc44a61b183dae765bef4d22a53b7)).
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/reference/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **Provider admission:** MIPS targets with unresolved GNU ABI attributes are refused. Supply supported ABI declarations. Saved Hopper databases on macOS are unavailable because the launcher cannot run the analysis bridge when opening a database; select an executable target instead. Database support requires a script-capable database launch route. ([d4bd65c](https://github.com/morluto/rea/commit/d4bd65c53c0254d0195cc7ab8da163dcadfd3b9b)).
- **Segment extents:** `binary_overview` segment length may be null when Hopper coordinates cannot establish a finite extent in one address space. Preserve unknown rather than converting it to zero or a saturated value. ([36db19c](https://github.com/morluto/rea/commit/36db19c59)).
@@ -372,7 +182,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
### ⚠ BREAKING CHANGES
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/guides/runtime/process-capture.md) and [browser scenario contract](docs/guides/web/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/process-capture.md) and [browser scenario contract](docs/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
* **browser:** JSON shape paths now contain typed property and array-element segments. WebMCP input_schema_shape is replaced by input_schema and input_schema_sha256, retaining declared examples and defaults.
* **process:** Filesystem digest omission entries now require system_code; budget and changed-file omissions use null, and unavailable-file omissions preserve the operating-system error code.
* **contracts:** Provider analysis outputs must explicitly supply metadata previously repaired with defaults; unsupported nullable facets must be reported as null.
@@ -540,7 +350,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
### ⚠ BREAKING CHANGES
Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
* **MCP / Evidence / snapshots / artifacts:** Success results are direct canonical Evidence; read `normalized_result` rather than the old `result`/`evidence` wrapper. Evidence requires nullable `analysis_profile`, session provider aliases are removed, snapshots require complete profile-bound query records, and artifact roles/permissions belong to occurrences. Recreate incompatible saved data; adding fields without recomputing their semantic identity is invalid. ([63bf914](https://github.com/morluto/rea/commit/63bf91405cf97b114a84f337fb1272d98a267999))
* **Process captures:** Legacy v3 conversion is removed. Explicit executable identity and `event_journal` are required; retain old captures and recapture when current comparisons are needed instead of inventing launch identity or event order. ([2a77469](https://github.com/morluto/rea/commit/2a7746998d47446c4db6d4f524e0f183d6803f3f))
@@ -2683,7 +2493,7 @@ Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before u
* enforce conventional pull request titles ([3a21f16](https://github.com/morluto/rea/commit/3a21f16c1832430765083636511a8c12dbccf2dd))
* verify installed package with Linux Hopper ([4f97f69](https://github.com/morluto/rea/commit/4f97f699e4d5471557d1ce0ff91e144065d06a61))
## [0.3.0](https://github.com/morluto/rea/compare/rea-0.2.1...rea-agents-0.3.0) (2026-07-12)
## [0.3.0](https://github.com/morluto/rea/compare/rea-agents-0.2.1...rea-agents-0.3.0) (2026-07-12)
### Features
@@ -2736,7 +2546,7 @@ Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before u
* redesign and localize README ([051b6db](https://github.com/morluto/rea/commit/051b6db65687fd14826b6462cd623feb2504f123))
* redesign and localize README ([910eda6](https://github.com/morluto/rea/commit/910eda6f323ba1151b0771fe5de6c60a91e3d46d))
## [0.2.0](https://github.com/morluto/rea/releases/tag/rea-0.2.0) (2026-07-12)
## [0.2.0](https://github.com/morluto/rea/compare/rea-0.1.0...rea-0.2.0) (2026-07-12)
### Features
+18 -124
View File
@@ -10,45 +10,19 @@ and maintenance cost. Keep cosmetic polish with useful fixes; confusing or
incorrect user instructions warrant standalone corrections. Collect other
cosmetic-only suggestions in an issue rather than separate PRs.
For selected architectural owners and dependency paths, use the
[architecture map](docs/architecture/architecture.mermaid). It is not a complete directory or
import inventory. For current provider wiring, start with
[`src/composition/binary.ts`](src/composition/binary.ts) for deep candidates and
the other `src/composition/` factories for tool services, including
[`gpu.ts`](src/composition/gpu.ts) and [`model.ts`](src/composition/model.ts).
Live Frida, Cutter, GDB and Rizin construction also uses shared composition
factories. For service ownership, availability and catalog declarations, follow
the [capability composition guide](docs/capability-composition.md).
Production factories construct fresh providers without starting engines or
acquiring targets; optional adapter failures must preserve successful peers. Run
For capability organization and provider composition, use the
[architecture map](docs/architecture.mermaid). Production factories in
`src/composition/` construct fresh providers without starting engines or acquiring
targets; optional adapter failures must preserve successful peers. Run
`npm run verify:test-discovery` after adding or moving tests. Resolved import
boundaries are checked by `npm run verify:module-boundaries`, included in lint.
Before retiring an alias or facade, trace runtime registration, dynamic imports,
JavaScript verifiers, subprocess scripts, and packaged entrypoints as well as
TypeScript references. Tests may be the last consumers of a superseded internal
path; migrate their distinct assertions to the current production path before
removing it. Preserve validation, lazy loading, caller policy, and resource
acquisition and cleanup supplied by the old boundary. Published APIs, persisted
formats, and independently versioned bridges require an explicit compatibility
decision; an internal representation alone does not establish that obligation.
See [interpreting boundary facts once](docs/development/tool-design.md#interpret-boundary-facts-once)
for admission ownership and the distinction between repeated validation and
independent checks.
For analysis workflows and agent interfaces, follow the
[performance priorities](AGENTS.md#performance). Explain the selected output
scope, reuse/cache decisions, and relevant before/after measurements when these
costs change.
Use the [analysis optimization skill](.agents/skills/rea-analysis-optimization/SKILL.md)
to locate provider and shared-workflow bottlenecks, preserve Evidence, and
compare equivalent real tasks before claiming an improvement.
When adding or changing an MCP tool, follow the [tool design guide](docs/development/tool-design.md) and preserve the canonical contracts and generated catalog.
When adding or changing an admission ceiling, budget, or deadline, follow
[analysis settings, limits, and recovery](docs/design/limits-and-recovery.md)
for ownership, operation snapshots, provider integration, and the steps for
adding a runtime setting.
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
## Reporting issues
@@ -93,8 +67,7 @@ Before requesting maintainer review, the human owner must:
passing. Follow the [verification requirements](#development-feedback-and-pr-verification)
for required checks and real-provider coverage.
- Complete the **Author readiness** declaration in the PR template, explicitly
stating that the PR is ready for initial review, unless the PR author is
exempt from the declaration as listed below.
stating that the PR is ready for initial review.
Keep the PR description concise and current. Explain the problem, observable
outcome, approach, and useful evidence; link long logs or put them in optional
@@ -103,11 +76,7 @@ Human ownership and review remain required when CI or an automated reviewer
reports success. Readiness requests maintainer review; it does not grant scope
approval or permission to merge.
Draft status is optional. Draft PRs may request early feedback with open questions
and incomplete checks. Agents preserve the current draft status unless asked to
change it; missing readiness declarations or later updates do not require a PR
to become a draft.
Draft PRs may request early feedback with open questions and incomplete checks.
Leave the readiness declaration unchecked until the human owner has completed
it. The declaration records initial readiness; later pushes do not invalidate
it or require another human sign-off. Keep the PR description and affected
@@ -115,12 +84,9 @@ validation current as work changes. AI agents must leave human declarations to
the human owner and must not attest to a human review on their behalf.
The `PR readiness` check reports pending for drafts and requires a named human
owner and all three template declarations for non-drafts. PRs authored by
@morluto, @kaoru0822-kitauji, @N0zoM1z0, or @gensokyo-dev are exempt from
the declaration; their human ownership, review, and evidence responsibilities
above still apply. The check identifies authors by GitHub account ID. It checks
that the declaration is filled, not whether the code or evidence is correct.
Existing PRs can copy the **Author readiness** section from the current template.
owner and all three template declarations for non-drafts. It checks that the
declaration is filled, not whether the code or evidence is correct. Existing PRs
can copy the **Author readiness** section from the current template.
Open PRs must have distinct head commits because GitHub shares commit statuses.
Maintainers enable merge enforcement after the workflow is merged: run
@@ -130,19 +96,6 @@ status checks alongside its existing checks. Deploy the workflow to each
protected target branch before requiring it there. The workflow reads PR
metadata using trusted default-branch code and writes only a commit status.
When an open PR may overlap another open change or a recently landed fix, note
the related PR, issue, or commit in the description or a comment as soon as the
overlap is known. Prefer one clear pointer over repeated status updates.
If an overlapping change merges while a PR awaits review, link that commit on
the open PR and state what remains useful. Rebase or otherwise align with the
current base when the remaining work is still wanted, narrow the PR to the
uncovered portion, or close it when the change is fully superseded. Keep the
description and validation current after that update. Maintainers may also note
the expected disposition (rebase, narrow, or close) when they land or recognize
an overlapping fix. Readiness still does not grant merge permission or a review
date.
Maintainers may return a PR to the author or close it when the problem is
unsupported, validation is missing, or the owner cannot explain the change.
Contributors are responsible for completing this work before requesting review.
@@ -247,9 +200,7 @@ guarantee.
Use the development toolchain pinned in `.nvmrc` and
`package.json#packageManager`. Run `nvm use` before installing dependencies;
installed-package runtime support is documented in
[installation](docs/getting-started/installation.md#start-setup).
Use Node.js and npm for source builds and verifier scripts; the installed
package's Bun runtime support does not extend to those development scripts.
[installation](docs/installation.md#start-setup).
```bash
npm ci
@@ -264,15 +215,6 @@ catalogs, documentation projections, or managed verification evidence. Turbo
caches deterministic builds and static checks across Git worktrees. After a package, lockfile, or managed-skill version change, run
`npm run metadata:generate` before building.
Dependabot proposes updates for the npm lockfiles, website Python requirements,
and GitHub Actions. Review dependency changes and run the checks relevant to
their consumers. For dependencies maintained outside Dependabot, including
`third_party/` and the HTTP/3 lab's pinned requirements, the maintainer should
review upstream security advisories when updating or using them. Preserve source
and version provenance, update license notices when needed, and report which
verification was actually repeated. The lab requirements describe a recorded
environment; a dependency update does not verify the revised environment.
Repository invariants live in [AGENTS.md](AGENTS.md). Keep provider-specific
protocols in adapters and shared CLI/MCP workflows in the application layer.
@@ -283,10 +225,7 @@ manifest. A source checkout does not: `native/windows/build/` is ignored by
Git, and neither `npm ci` nor `npm run build:cached` (or `npm run build`)
produces `rea-windows-x64.node` or `manifest.json` in that directory.
Windows workflows and tests that require native controls need these artifacts;
this includes `run-exclusive` commands, which use the native Job Object to keep
the artifact lock until every command descendant has exited. Without the
artifacts, those controls are unavailable even after a successful runtime
build.
without them, those controls are unavailable even after a successful runtime build.
Follow the [Windows native build guide](native/windows/README.md#build) before
running those workflows from source. The artifact build requires Linux,
@@ -294,28 +233,8 @@ MinGW-w64, and `tar`; the build script does not run on Windows. See that guide
for the build and artifact-verification commands and the separate real Windows
verification prerequisites. Keep the generated addon and manifest uncommitted.
The general source suite (`npm run test:fast`, `npm run test:changed`, and
`npm run test:local`) runs on Linux in CI, and Windows CI runs a curated lane.
On a Windows checkout, some source tests fail because fixtures use POSIX paths,
`#!/bin/sh` tool stubs, or POSIX rename semantics. Compare failures against the
base revision, or run the suite on Linux or WSL, before attributing them to a
change. The embedded-artifact parent-swap tests exercise the selected-path
replacement when the rename succeeds; if Windows refuses it while the admitted
file is open, they instead verify the refusal and cleanup. See
[testing](docs/development/testing.md#developer-commands) for command scopes
and lane selection.
`git submodule update --init --recursive` fails at the pinned Unblob submodule
on Windows and can leave later submodules unchecked out. Apply the
[Unblob checkout workaround](docs/guides/firmware/firmware-analysis.md#windows-submodule-checkout),
then rerun the command.
## Documentation website
Use the [documentation directory](docs/README.md) and
[authoring guide](docs/development/documentation.md) for document ownership,
domain-based feature specifications and architectural decision records.
The VitePress site uses the Markdown files in `docs/`. Run `npm run docs:dev`
for live editing, `npm run docs:build` to check the production build and
links, and `npm run docs:preview` to preview that build at `/rea/`.
@@ -336,23 +255,6 @@ The separate GitHub Pages host uses only the manual
**Settings → Pages → Build and deployment → Source** must be set to
**GitHub Actions** before deploying there.
### Translation corrections
README editions and website editions have separate translation sources:
- Find the language's `readme` path in the
[language inventory](website/locales/languages.json) and edit that file.
Locale codes and README suffixes can differ. Use the
[English README](README.md) to check commands, requirements, and analysis claims.
- Edit `msgstr` values in `website/locales/<language>/messages.po` for website
text. Follow the [website translation guide](website/locales/README.md) for
the Python environment, catalog validation, and local preview. Preserve source
strings, contexts, placeholders, commands, paths, and recorded evidence.
Before submitting a website translation, run the catalog and website checks in
that guide. Translated HTML in `website/_site/` is generated; edit the PO catalog
rather than the generated pages.
## Development feedback and PR verification
For routine merges, wait for the required checks to pass for the current PR
@@ -369,7 +271,7 @@ npm run check:fast
`test:local` runs source tests without building. `check:changed` combines cached
static checks with source tests affected since the merge base with `origin/main`.
Import-graph selection can miss runtime registration, bridges, and generated data;
select relevant boundary tests explicitly. See [testing](docs/development/testing.md#developer-commands)
select relevant boundary tests explicitly. See [testing](docs/testing.md#developer-commands)
for commands, test placement, pruning, and real-provider prerequisites.
Before requesting review, run relevant tests and `npm run check`; add
@@ -420,17 +322,9 @@ Real-provider execution is uncached.
### Temporary files and diagnostics
Tests use `createTestTempDirectory` from `tests/fixtures/temporaryDirectory.ts`
for exact-path, awaited cleanup, including failures and timeouts. It registers
deletion with Vitest's `onTestFinished`; do not also add its result to a roots
array or an `afterEach` removal. `createTestWorkspace` from
`tests/support/workspace/workspaceFixture.ts` leaves cleanup with its caller,
which must await `removeTestWorkspace`. Keep explicit resource teardown before
directory deletion, using the case's `finally` or `afterEach`; Vitest runs
`afterEach` before `onTestFinished` and awaits finish hooks in reverse
registration order.
for exact-path, awaited cleanup, including failures and timeouts.
`npm run verify:test-temp-hygiene` runs the suite under a fresh `TMPDIR` and
rejects remaining REA-owned paths. It cannot detect duplicate deletion owners or
prove resource-close ordering. Never glob-clean shared `/tmp/rea-*` content.
rejects remaining REA-owned paths. Never glob-clean shared `/tmp/rea-*` content.
`REA_LOG_LEVEL` selects `trace`, `debug`, `info`, `warn`, `error`, `fatal`, or
`silent`. Logs go to stderr: MCP defaults to `info`, and one-shot CLI logging
@@ -444,8 +338,8 @@ Run `npm run verify:hopper` on a supported macOS host or
`npm run verify:hopper:linux` on the supported Linux demo runner. These build
native fixtures and verify the installed engine through CLI/MCP; package or
simulated-provider checks do not establish engine behavior. See
[real-toolchain verification](docs/development/testing.md#end-to-end-integration-and-golden-evidence)
for prerequisites and scope, and [Hopper setup](docs/getting-started/installation.md#hopper)
[real-toolchain verification](docs/testing.md#end-to-end-integration-and-golden-evidence)
for prerequisites and scope, and [Hopper setup](docs/installation.md#hopper)
for supported hosts and unattended startup.
Use the default `build/conformance/manifest.json` unless explicitly verifying
@@ -454,7 +348,7 @@ Generated fixtures and manifests remain ignored.
## Maintainer release checklist
Use the [release guide](docs/development/releasing.md). Main pushes automatically refresh a
Use the [release guide](docs/releasing.md). Main pushes automatically refresh a
Release Please PR with the next version and changelog. Review its migration
notes, wait for the current CI checks to pass, and merge it into main to publish.
+60 -156
View File
@@ -1,7 +1,7 @@
<div align="center">
**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md)
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md) · [Română](docs/readme/README_ro.md) · [Български](docs/readme/README_bg.md)
· [Ελληνικά](docs/readme/README_el.md)
# REA: Reverse Engineer Anything
@@ -10,7 +10,7 @@
**See a feature you like. Understand how it works, down to the binary level.**
[![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/reference/mcp-contracts.md#generated-catalog)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog)
[![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#what-you-can-analyze)
[![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything)
[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE)
@@ -20,9 +20,7 @@
**[Website](https://rea.tools/) · [Guides](https://rea.tools/guides/) · [Showcases](https://rea.tools/showcase/)**
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Join the REA community on Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [Build RL environments](#build-rl-environments) · [FAQ](#faq) · [Documentation](#documentation)
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [FAQ](#faq) · [Documentation](#documentation)
<code>npx rea-agents setup</code>
@@ -64,7 +62,7 @@ See a feature in an app that you want in your own product? Ask your agent to inv
REA connects your agent to tools for inspecting native binaries, JavaScript and Electron apps, .NET assemblies, and websites. You can also use the same tools from your terminal. Analysis runs locally, and results include the evidence and limitations behind each conclusion.
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, IDA or Kuna installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, or IDA installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
> **[Visit the REA website](https://rea.tools/)** for setup instructions, illustrated guides, and real case studies.
@@ -72,7 +70,7 @@ Setup registers REA with your agent and installs matching workflow instructions.
On Linux, Ghidra, JADX, firmware tool launches and static JavaScript/Electron
analysis require a procps-compatible `ps` on REA's `PATH` for process ownership
checks. Recovery steps are in `docs/getting-started/installation.md` under
checks. Recovery steps are in `docs/installation.md` under
"Linux owned-process prerequisite".
### Set up your agent
@@ -85,27 +83,12 @@ npx rea-agents setup
Choose your agents, review the proposed changes, and approve them. Setup adds
REA's MCP server and matching workflow instructions, with backups of existing
configuration. Restart your agent afterward. Setup records the selected runtime and REA
entry script directly. If acquired through a package runner, keep its cache entry
available. Run package acquisition from a trusted directory outside the target
project. Installation and recovery details are in the setup guide below.
configuration. Restart your agent afterward.
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
[other agents](docs/getting-started/installation.md#supported-agents). See
[installation and setup](docs/getting-started/installation.md) for provider configuration and
manual MCP registration and Claude Code/Codex plugin installation.
MCP uses compact input schemas by default to reduce context overhead. Set
`REA_MCP_INPUT_SCHEMA_PROFILE=full` in your MCP registration when you need the
complete advertised field guidance and nested validation details. Both profiles
keep the full tool set and canonical server validation. See
"Compact input schema profile" in the MCP contracts guide.
Complete output schemas are advertised by default. If your client caps the
complete `tools/list` frame, set `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact` to
bound each advertised output schema, or `off` to omit output schemas entirely.
Both settings keep canonical server-side validation and complete structured
results. See "Output schemas" in the MCP contracts guide.
[other agents](docs/installation.md#supported-agents). See
[installation and setup](docs/installation.md) for provider configuration and
manual MCP registration.
### Ask your agent
@@ -135,7 +118,7 @@ rea --help
```
For native analysis, configure a provider first. See the
[CLI and Evidence guide](docs/getting-started/cli.md) for native commands, provider selection,
[CLI and Evidence guide](docs/cli.md) for native commands, provider selection,
snapshots and scripting.
### Update REA
@@ -149,14 +132,6 @@ For a CLI installed with `npm install --global` or `bun add --global`:
rea update
```
`rea --update` is an alias for this verified workflow and accepts the same global
output flags. Use `rea update` when combining it with other command controls.
Updater commands retain at most 4 MiB of stdout and stderr combined. Oversized
metadata replies are rejected after the command exits; verbose installs continue
with bounded diagnostics. Human progress output respects terminal backpressure
and waits for submitted writes to finish.
Bun global updates require Bun 1.2.17 or later. With an older Bun release,
upgrade Bun manually before running `rea update`.
@@ -188,50 +163,34 @@ CLI commands use the same workflows.
## What you can analyze
REA requires Node.js 22.x (>=22.23.4), 24.x (>=24.22.1), or 26.12.1+, plus npm.
REA requires Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+, plus npm.
Additional tools and host support depend on the target:
| Target | What REA returns | Requirements and guide |
| ----------------------- | ------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/guides/native/native-investigation.md) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/guides/native/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
| Windows shortcuts | Shell Link target, arguments, working dir, ExtraData and Tracker droid facts | Built-in static LNK inspection; [Windows shortcuts](docs/guides/native/windows-shortcut.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
| Binary hardening | PIE, NX, RELRO, CFG and related header flags for ELF, PE and Mach-O | Built-in static header inspection; [binary hardening](docs/binary-hardening.md) |
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/guides/evm/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/guides/native/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/guides/javascript/javascript-application-workflows.md) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/guides/web/browser-observation.md) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/guides/web/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/guides/managed/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/guides/android/android-analysis.md) |
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/guides/android/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/providers/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/guides/android/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/guides/android/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/guides/firmware/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/guides/data/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/guides/data/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/guides/signals/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/guides/javascript/javascript-artifact-reconstruction.md), [Apple applications](docs/guides/apple/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/guides/runtime/process-capture.md) |
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
| Deno compile binaries | Virtual files, entry point, recorded options, npm snapshot length | None (static only); [Deno compile executables](docs/deno-standalone.md) |
| Target | What REA returns | Requirements and guide |
| ---------------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
Static JavaScript and .NET inspection read the supplied files without running
the application. Runtime capture runs or interacts with the selected target
@@ -245,25 +204,20 @@ or update either provider.
<a id="choosing-a-deep-analysis-provider"></a>
Native formats and host support vary by provider. See
[Hopper and Ghidra setup](docs/getting-started/installation.md#hopper), the
[IDA guide](docs/providers/ida-provider.md) (including an optional native MCP bridge), the
[Hopper and Ghidra setup](docs/installation.md#hopper), the
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), the
[Kuna decompiler guide](docs/kuna-provider.md), and
[experimental Windows Ghidra support](docs/providers/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/providers/ghidra-dos.md).
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
For large binaries, raise the provider's startup deadline with
`REA_GHIDRA_STARTUP_TIMEOUT_MS` or `REA_HOPPER_STARTUP_TIMEOUT_MS`.
For provider selection, see the [CLI guide](docs/getting-started/cli.md#choose-a-provider).
Check [release availability](docs/getting-started/installation.md#released-package-and-main)
For provider selection, see the [CLI guide](docs/cli.md#choose-a-provider).
Check [release availability](docs/installation.md#released-package-and-main)
for features added since the latest npm release.
## Showcases
<a href="https://rea.tools/showcase/">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/rea-showcases-dark.svg" />
<img alt="Illustrations of the DX-Ball sound-pan, Notion clipboard-bridge, and TH04 bullet-ring showcases" src="docs/assets/rea-showcases.svg" />
</picture>
</a>
[![Illustrations of the DX-Ball sound-pan, Notion clipboard-bridge, and TH04 bullet-ring showcases](docs/assets/rea-showcases.png)](https://rea.tools/showcase/)
### DX-Ball: reconstruct a sound-pan calculation
@@ -290,78 +244,28 @@ historical compiler output.
[Read the case study](https://rea.tools/showcase/th04/) ·
[Reconstruction repository](https://github.com/N0zoM1z0/th04)
### Build RL environments with REA
Record a web app session, turn its API into tools, and export an environment an AI agent can use. Run it with AgentEnv, then add tasks and reward checks for RL training or evaluation.
[Read the guide](https://rea.tools/guides/rl-environments/) ·
[Watch the video](https://x.com/reatools/status/2112574822584918075?s=20)
If you've used REA on something interesting, we'd love to see it. Share your
case in an [issue](https://github.com/morluto/rea/issues) or a
[pull request](https://github.com/morluto/rea/pulls), including the target,
your question, how REA helped, and what you found.
## Build RL environments
Give an AI agent a local environment to practice tasks such as finding an issue or creating a folder, without running the original app. Use it for reinforcement learning (RL) or to test how well an agent completes tasks.
REA turns recorded API requests and responses into tools the agent can call. The environment uses the recorded data and supports changes only when the recording confirms their effects.
With REA set up in your agent, replace the example URL and folder tasks below with your own:
```text
Use REA to build an RL environment from my test web app at
http://127.0.0.1:3000.
Cover listing, creating, renaming, and deleting folders. You may create and
remove test folders. Leave existing data unchanged.
Save it in a folder named my-env. Check which operations work and tell me
how to run it.
```
<details>
<summary>Use the command line instead</summary>
Use these commands if you already have a file containing the app’s recorded browser requests and responses. Running the result also requires AgentEnv and Docker.
```sh
rea environment-build ./session.har ./my-env --name my-env --json
agent-env run ./my-env
```
</details>
REA exports a runnable environment with a basic test to check that it starts and responds. Use it with your RL framework to define task goals, add reward checks, and run training.
[Build an environment](https://rea.tools/guides/rl-environments/) ·
[Environment reference](docs/agent-environments.md) ·
[Watch the demo on X](https://x.com/reatools/status/2112574822584918075?s=20)
### Desktop app environments
REA can also build an environment of a Linux desktop app, which an agent uses through screenshots, the mouse and the keyboard. From a one-line request, such as "Build me a Gnote environment where the agent adds olive oil to the shopping list note", your agent installs the app on a local build computer, writes the task, exports the environment and checks the task in a fresh computer. The machine that builds it needs Linux on amd64 or arm64, Docker and the agentenv framework version the guide pins.
[Set up desktop app environments](docs/desktop-build-computer.md)
## FAQ
<details>
<summary><strong>Which agents can use REA?</strong></summary>
Any agent that supports local MCP servers. Setup configures the
[supported agents](docs/getting-started/installation.md#supported-agents); other clients can use
[manual MCP registration](docs/getting-started/installation.md#mcp-registry).
[supported agents](docs/installation.md#supported-agents); other clients can use
[manual MCP registration](docs/installation.md#mcp-registry).
</details>
<details>
<summary><strong>Do I need a native analysis engine?</strong></summary>
<summary><strong>Do I need Hopper, Ghidra or IDA?</strong></summary>
Deep native analysis uses Hopper, Ghidra, IDA or Kuna, depending on the target
and operation. Static JavaScript and .NET inspection work without a native
analysis engine. Setup can install Hopper after approval; Ghidra, IDA and Kuna
use your existing installations. See [provider setup](docs/getting-started/installation.md#hopper)
for the options and their supported targets.
Deep native analysis uses one of them. Static JavaScript and .NET inspection
work without a native analysis engine. Setup can install Hopper after approval;
Ghidra and IDA use your existing installations. See [provider setup](docs/installation.md#hopper).
</details>
@@ -372,7 +276,7 @@ REA starts Hopper when an operation needs it. On macOS, a first-run dialog may
ask you to choose demo mode or activate your license. Complete that prompt in the
same user session before your first CLI/MCP operation or unattended job. Open
Hopper manually if needed. See
[Hopper startup and troubleshooting](docs/getting-started/installation.md#launcher-paths-and-troubleshooting).
[Hopper startup and troubleshooting](docs/installation.md#launcher-paths-and-troubleshooting).
</details>
@@ -381,7 +285,7 @@ Hopper manually if needed. See
The skill supplies investigation instructions for your agent. Use `npx rea-agents setup`
to register REA's MCP server and install the matching instructions, then restart
your agent. See [skill-only installation](docs/getting-started/installation.md#skill-only-installation).
your agent. See [skill-only installation](docs/installation.md#skill-only-installation).
</details>
@@ -430,18 +334,18 @@ with your REA version, target type, steps to reproduce and error output.
Start with the website's [worked guides](https://rea.tools/guides/).
For exact options, prerequisites and result contracts:
- [Installation and setup](docs/getting-started/installation.md): agent registration, provider configuration, updates and uninstall.
- [Readiness and troubleshooting](docs/getting-started/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/getting-started/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/reference/mcp-contracts.md) and [agent prompts](docs/getting-started/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/reference/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/development/roadmap.md): planned work and capability trackers.
- [Installation and setup](docs/installation.md): agent registration, provider configuration, updates and uninstall.
- [Readiness and troubleshooting](docs/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/mcp-contracts.md) and [agent prompts](docs/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/roadmap.md): planned work and capability trackers.
Report vulnerabilities through [SECURITY.md](SECURITY.md).
## Star history
🎉 **400,000 GitHub stars — thank you!**
🎉 **250,000 GitHub stars — thank you!**
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
@@ -470,8 +374,8 @@ report a bug or suggest a feature, or [send a pull request](https://github.com/m
to improve the code or docs.
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
[testing](docs/development/testing.md) for verification lanes, and the
[architecture map](docs/architecture/architecture.mermaid) for selected layers and dependency paths.
[testing](docs/testing.md) for verification lanes, and the
[architecture map](docs/architecture.mermaid) for the project structure.
## License
+28 -112
View File
@@ -4,124 +4,13 @@
Security fixes are provided for the latest published version of REA. Upgrade to the latest release before reporting a problem that may already have been resolved.
## Security boundary
REA is a local reverse-engineering and automation tool. CLI and MCP use the
same application workflows and run with the host process's permissions.
REA is not a general-purpose sandbox or a multi-tenant authorization service.
### Threat model
- **Trusted authority:** the operating-system account running REA, the caller
or agent host that selects operations, and configured analysis providers,
their executables and extensions, and service endpoints. An MCP client can
invoke the operations exposed to it; its host must enforce the user's tool
permissions and approvals.
- **Untrusted data:** target binaries, archives, source files, project contents,
captures, web pages and returned target content can be malicious. Selecting
an input for static inspection does not authorize commands embedded in that
input. Agents must treat target text and Evidence as data rather
than instructions granting additional authority.
- **Operation-specific effects:** static inspection parses target data and may
launch an analysis engine. Runtime, debugger, instrumentation, browser,
desktop and environment workflows can deliberately execute code, attach to
processes, make network requests, write files or interact with applications
and devices. Review the selected operation's effects and limitations; static
inspection does not authorize launching or interacting with the target
application. See
[runtime and network responsibilities](DISCLAIMER.md#runtime-network-and-device-operations).
Standard client registrations use a dedicated stdio process. Opt-in
[HTTP MCP](docs/reference/mcp-contracts.md#transports) uses plain HTTP without
TLS or client authentication and shares provider/active-target state across
protocol sessions. Its session ID is not a user-authentication or isolation
boundary. Keep this endpoint within a trusted local or isolated environment.
### Intended security properties
Static JavaScript and .NET inspection do not run the target application.
Unexpected execution of input code through a data-only analysis path is a
security concern. Analysis engines themselves are executable software trusted
with the host account's permissions; parsing an input in an external engine
does not isolate that engine from the host.
REA authenticates its managed Hopper and Ghidra bridges with per-session random
capability tokens transferred through private session files, rather than
process arguments or environment variables. POSIX uses current-user Unix
sockets. Windows Ghidra uses authenticated loopback TCP and a private NTFS
runtime whose DACL admits only the current user and SYSTEM. These controls
protect the managed session boundary, not every external provider or debugger
endpoint. See [Windows Ghidra transport and lifecycle](docs/providers/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
Provider ownership and mutation rules remain specific to each workflow:
- Ghidra's working projects are temporary and owned by REA. Opt-in
[checkpoints](docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program)
retain REA-owned Program state. A
[caller-selected existing project](docs/getting-started/installation.md#reuse-an-existing-ghidra-project)
is fingerprinted and copied into an owned runtime for read-only processing;
Ghidra does not open or save the original database through that workflow.
- [IDA analysis](docs/providers/ida-provider.md) is read-only. An attached GUI
database is never saved or closed by REA; owned headless sessions have a
separate lifecycle.
- Cleanup must stay within resources REA owns. Failed or unverified cleanup
can retain a workspace or session for retry; closing a session
does not imply that every investigation artifact has been erased.
`rea setup` shows planned configuration and installation changes and requires
approval before applying them. Ghidra, Java and IDA are user-managed. Approved
Linux Hopper installation authenticates the package's detached PGP signature
against the pinned publisher identity and checks the signed launcher against
the selected release's digest. The release feed's display version is separate
from that authenticated identity. See [installation and publisher verification](docs/getting-started/installation.md#hopper).
### Limits and caller responsibilities
- **Host isolation:** private runtimes and capability tokens do not protect
against malicious software already running as the same operating-system
user, or a compromised trusted client or provider. Use an appropriately
isolated account, VM or restricted environment when handling hostile targets.
- **Resource bounds:** timeouts, cancellation, streaming and size limits are
operation-specific. They are not a universal CPU, memory, disk or wall-time
limit on every target and third-party tool. Enforce hard host resource limits
externally when needed. Failures of a documented boundary or cancellation
control still warrant investigation.
- **Data and network exposure:** Evidence, caches, checkpoints, exports and
captures can retain private target data, paths, URLs and memory contents.
Local execution does not make these outputs safe to publish. Requested
network operations, agent hosts and provider features such as
[IDA Free's cloud decompiler](docs/providers/ida-provider.md) can communicate
outside the host. Review their data handling and secure any exposed endpoint.
- **Interpretation:** analysis, reconstruction and generated environments can
be incomplete or incorrect. Identity checks, signatures and successful
analysis do not establish that a target, provider or generated program is
safe to execute. Review the Evidence and proposed actions before relying on
them.
### Vulnerability scope
Please report suspected violations of these boundaries privately. Examples
include unintended code execution during data-only inspection, escaping a
documented extraction/output boundary, bypassing bridge authentication or
target/session identity checks, unauthorized effects on an unrelated resource,
and disclosure of transport credentials or explicitly marked sensitive values.
An attacker-triggered bypass of a documented resource or cancellation control
can also be security-relevant without code execution.
Incorrect analysis, unsupported formats, crashes, hangs and high resource use
without a security-boundary impact are generally ordinary bug reports. Effects
explicitly requested through an execution or interaction workflow are not by
themselves a vulnerability. A dependency advisory needs its actual REA exposure
assessed; a complete exploit or diagnosis is not required to report a concern.
If you are unsure whether a boundary is affected, use the private channel below.
## Reporting a vulnerability
Do not disclose vulnerabilities, proof-of-concept exploits, credentials,
private binaries, Hopper documents, or private Ghidra projects in a public
issue.
Use **Report a vulnerability** in the repository's **Security** tab as the preferred way to submit the report privately. If GitHub private reporting is unavailable to you, email [[email protected]](mailto:[email protected]). Do not open a public issue to request a private contact channel.
Use **Report a vulnerability** in the repository's **Security** tab to submit the report privately. Do not open a public issue to request a private contact channel.
Include the affected REA version, operating system/distribution, selected
provider and version, impact, reproduction conditions, and any suggested
@@ -132,3 +21,30 @@ unrelated local paths from logs or examples.
The maintainer will acknowledge the report, assess its scope, and coordinate
remediation and disclosure timing with the reporter. Please allow time for
provider-dependent behavior to be reproduced safely.
## Security boundary
REA authenticates its Hopper and Ghidra bridges with per-session random
capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses
authenticated loopback TCP with a private NTFS runtime whose DACL admits only
the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
Tokens are transferred through private session descriptors rather than process
arguments or environment variables.
This is not a sandbox and does not protect against malicious processes already
running as the same operating-system user. Opening an untrusted binary
delegates parsing and analysis to the selected local provider with that user's
permissions.
Ghidra sessions use a packaged Java `HeadlessScript`, an isolated temporary
project, and private home, cache, and runtime directories. REA imports only the
requested target, bounds startup and protocol messages, requests bounded CPU
and heap settings, and deletes the temporary project when the session closes.
These controls limit accidental persistence and resource use; they do not make
Ghidra a sandbox. REA does not open or modify user-owned Ghidra projects.
`GHIDRA_INSTALL_DIR` and optional `JAVA_HOME` identify user-managed
installations. Setup can add those values to detected client configuration only
after showing its plan and receiving approval. REA does not download, install,
upgrade, or modify Ghidra or Java.
On Linux, REA accepts release metadata only from Hopper's HTTPS endpoint, restricts package URLs to Hopper's public download origin, bounds the package size, and compares the downloaded bytes with Hopper's published checksum before requesting installation. The published SHA-1 value is a corruption check, not a modern package signature; HTTPS origin validation remains part of the trust boundary. Dependency installation is delegated without shell evaluation to `apt-get`, `dnf`, or `pacman`, directly as root or through `pkexec`. REA never invokes `sudo`.
+21 -23
View File
@@ -16,8 +16,6 @@ notices supplied with the software you actually obtain or redistribute.
| Material | Use and licensing records |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| npm dependencies | [Complete inventory](licenses/npm-dependencies.txt) of every entry in `package-lock.json`, including transitive, optional platform, development, and website dependencies. [Runtime notices](licenses/npm-runtime-notices.txt) preserve the copyright, license, and NOTICE documents from integrity-verified release archives, plus the MIT notice for Rolldown's generated runtime helpers. Development dependencies are inventoried but are not all distributed in REA's npm package. |
| pefile ordinal tables | Import-hash ordinal names are derived from [pefile 2024.8.26](https://github.com/erocarrera/pefile/tree/4b3b1e2e568a88d4f1897d694d684f23d9e270c4), copyright 2004–2024 Ero Carrera. Preserve the [MIT license](licenses/pefile-2024.8.26-LICENSE.txt). The Python package is not a runtime dependency. |
| UF2 family names | UF2 family IDs and short names are copied from [microsoft/uf2 `utils/uf2families.json`](https://github.com/microsoft/uf2/blob/cf3bd7f9c274b949528f8cd6a19eef2502a687d2/utils/uf2families.json), copyright Microsoft Corporation. Preserve the [MIT license](licenses/microsoft-uf2-MIT.txt). |
| NativeAOT parser | REA's derived parser uses the MIT-licensed [ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot/tree/effeb734fc570c32650f88b159608979dc7b423e) work of Washi and the NativeAOT layout definitions in [dotnet/runtime](https://github.com/dotnet/runtime/tree/a3b2d40328be0be3f8dd43f0e5cc925b8827b044). Preserve [Washi's MIT notice](licenses/nativeaot-parser-MIT.txt) and the [.NET Foundation MIT notice](licenses/dotnet-runtime-MIT.txt). |
| EVM analysis | `[email protected]`, MIT, copyright 2023 Maxim Andreev. The published npm archive omits its license document; REA retains the [original upstream license](third_party/evmole/LICENSE) and [provenance](third_party/evmole/README.md). It is also included in the runtime notices. |
| Windows native addon | The release addon is compiled with MinGW-w64 and statically links GCC runtime libraries. Preserve the [MinGW-w64 runtime notices](licenses/MinGW-w64-runtime.txt), [GPLv3 text](licenses/GPL-3.0.txt), and [GCC Runtime Library Exception 3.1](licenses/GCC-Runtime-Library-Exception-3.1.txt). The exception governs eligible compiled combinations; it does not relicense the runtime itself under MIT. The Node-API build headers are from Node 22.19.0; its [original license and component notices](licenses/Node-22.19.0-LICENSE.txt) are retained. Node itself is supplied by the user. Snapshot provenance and release-toolchain review requirements are in [licenses/README.md](licenses/README.md). |
@@ -54,27 +52,27 @@ REA's adapters do not grant, replace, or waive these licenses. The upstream
executables and their dependency graphs are supplied separately. Git submodules
pin inspection sources; they are not included in the npm package.
| Project | License and source | REA relationship |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/getting-started/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
| Project | License and source | REA relationship |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra 12.1.4. Copies of its [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.833/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.833/NOTICE) | Bring your own `kuna`; REA verified 1.833 and never downloads, installs or bundles it. REA invokes it as a separate process. |
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
Ghidra's original notice credits software developed at the National Security
Agency. It explains that US-government-created portions are not subject to US
-10
View File
@@ -1,10 +0,0 @@
# Plugin icon
`rea-plugin.png` is a square adaptation of the left panel of REA's GitHub
social preview, retrieved on 2026-10-28:
https://repository-images.githubusercontent.com/1209966933/1a3fef9a-be4b-41f4-8830-a94c0e8288cc
The image was adapted with imagegen, preserving the cyan REA lettering,
pixel typography, orange rule, and taglines. Both plugin manifests reference
this shared asset; only the PNG is included in the npm package.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

-516
View File
@@ -1,516 +0,0 @@
"""Run REA inside an agentenv build computer for REA on the host: capture the app, draft its task and export it.
REA sends this file's source through the computer's ``computer-exec/v1`` route as
``python3 -I -c <source> <command> <JSON arguments>``. JSON that would exceed the bytes Linux
passes as one string travels as several arguments, which this file joins before parsing. Each command prints
one JSON line and exits. ``start``
leaves the capture running on its own: its stdin is a FIFO it holds open for reading and writing, so it never
sees the end of its input, and ``mark`` and ``stop`` write their requests there and wait for the capture's own
acknowledgement on its progress log. A small shell around the capture writes its exit status to a file, the
mark a later command reads. REA runs as the session user in the home, with no ``AGENTENV_*`` variable, so
neither REA nor the app it launches holds the computer's control token, and with ``APT_CONFIG`` naming the
package lists ``start`` downloads as that user, so the capture finds the package's sources.
"""
from __future__ import annotations
import hashlib
import json
import os
import re
import shutil
import subprocess
import sys
import time
POLL_S = 0.2
TAIL_BYTES = 4096
def _say(value: dict) -> int:
print(json.dumps(value), flush=True)
return 0
def _tail(path: str) -> str:
try:
with open(path, "rb") as handle:
handle.seek(max(0, os.path.getsize(path) - TAIL_BYTES))
return handle.read().decode("utf-8", "replace")
except OSError:
return ""
def _environment(work: str) -> dict:
environment = {name: value for name, value in os.environ.items() if not name.startswith("AGENTENV_")}
environment["APT_CONFIG"] = os.path.join(work, "apt", "apt.conf")
return environment
def _apt(work: str, timeout: float) -> dict:
"""Download the package lists into the work folder as the session user, until one update succeeds."""
root = os.path.join(work, "apt")
lists = os.path.join(root, "lists")
updated = os.path.join(root, "updated")
os.makedirs(os.path.join(lists, "partial"), exist_ok=True)
os.makedirs(os.path.join(root, "cache", "archives", "partial"), exist_ok=True)
with open(os.path.join(root, "apt.conf"), "w", encoding="utf-8") as conf:
conf.write(
f'Dir::State::Lists "{lists}";\n'
f'Dir::Cache "{root}/cache";\n'
'Debug::NoLocking "true";\n'
'APT::Update::Error-Mode "any";\n'
)
# apt keeps lists compressed as it fetched them, such as *_Packages.lz4 on Ubuntu.
count = lambda: len([name for name in os.listdir(lists) if re.search(r"_Packages(\.\w+)?$", name)]) # noqa: E731
# A marker left by an update that fetched nothing is not a success; download again.
if os.path.exists(updated) and count() > 0:
return {"lists": count(), "updated": False}
log = os.path.join(root, "update.log")
try:
with open(log, "wb") as output:
code = subprocess.run(["apt-get", "update"], env=_environment(work), stdin=subprocess.DEVNULL,
stdout=output, stderr=subprocess.STDOUT, timeout=timeout).returncode
except (OSError, subprocess.TimeoutExpired) as error:
return {"lists": count(), "updated": True, "error": f"{type(error).__name__}: {error}"}
lists_fetched = count()
if code == 0 and lists_fetched > 0:
# Only a whole update that fetched lists is kept, so a failed or empty one runs again.
open(updated, "w").close()
return {"lists": lists_fetched, "updated": True, "exit_code": code}
if code == 0:
return {"lists": lists_fetched, "updated": True, "exit_code": code,
"error": "apt-get update exited 0 without a package list"}
return {"lists": lists_fetched, "updated": True, "exit_code": code,
"error": f"apt-get update exited {code}: {_tail(log)[-1500:].strip()}"}
def _exit_code(capture: str):
try:
with open(os.path.join(capture, "exit"), encoding="utf-8") as handle:
return int(handle.read().strip() or "-1")
except (OSError, ValueError):
return None
def _ended(capture: str, code: int) -> dict:
"""A capture that has exited: its status, the end of its progress log and its output's size, which REA reads
whole for the Evidence or failure it ended with, whichever step was waiting on it."""
return {"exited": code, "progress": _tail(os.path.join(capture, "progress.log")),
"output_bytes": os.path.getsize(os.path.join(capture, "capture.out"))}
def _event_line(path: str, offset: int, kinds: set, name):
"""The first progress line after ``offset`` whose event is of one of ``kinds``, for ``name`` when given."""
try:
with open(path, "rb") as handle:
handle.seek(offset)
lines = handle.read().decode("utf-8", "replace").splitlines()
except OSError:
return None
for line in lines:
try:
event = json.loads(line)["rea_progress"]["event"]
except (ValueError, KeyError, TypeError):
continue
if isinstance(event, dict) and event.get("kind") in kinds and (name is None or event.get("name") == name):
return line
return None
def _wait(capture: str, offset: int, kinds: set, name, timeout: float) -> dict:
"""Wait for the capture's event, or for it to exit, or for time to run out."""
log = os.path.join(capture, "progress.log")
deadline = time.monotonic() + timeout
while True:
line = _event_line(log, offset, kinds, name)
if line is not None:
return {"line": line}
code = _exit_code(capture)
if code is not None:
return _ended(capture, code)
if time.monotonic() > deadline:
return {"timed_out": True, "progress": _tail(log)}
time.sleep(POLL_S)
def _within(path: str, folder: str) -> bool:
return path == folder or path.startswith(folder + os.sep)
def _state_paths(values: list, work: str):
"""Each state path as an absolute path strictly under the home and outside REA's work folder, both as named
and with its links resolved, or the refusal of the first that is not. The capture resolves the links again
and retains what they lead to, so a link out of the home is refused here. Each passes on as named, so the
capture keeps what its spelling means, such as a hidden settings folder."""
home = os.path.normpath(os.path.expanduser("~"))
real_home, real_work = os.path.realpath(home), os.path.realpath(work)
spellings = {home, real_home}
paths = []
for value in values:
relative = value[2:] if value.startswith("~/") else value
path = os.path.normpath(os.path.join(home, relative))
if ".." in relative.split("/") or not any(path.startswith(spelling + os.sep) for spelling in spellings):
return None, f"The state path {value} is not under the computer's home, {home}."
# Every link on the way is followed, a dangling one too; a path that does not exist yet resolves
# through its nearest existing folder.
resolved = os.path.realpath(path)
if not resolved.startswith(real_home + os.sep):
return None, f"The state path {value} leads through a link to {resolved}, which is not under the computer's home, {home}."
if _within(path, work) or _within(resolved, real_work):
return None, f"The state path {value} is in REA's work folder {work}, which every capture leaves out."
paths.append(path)
return paths, None
def start(arguments: dict) -> int:
work, capture = os.path.abspath(arguments["work"]), os.path.abspath(arguments["capture"])
rea = shutil.which("rea")
if rea is None:
return _say({"error": "REA is not installed in this computer; start the computer from a request, which installs it."})
state_paths, refusal = _state_paths(arguments.get("state_paths", []), work)
if refusal is not None:
return _say({"refused": refusal})
if os.path.exists(capture):
return _say({"error": f"the capture folder {capture} is already used"})
apt = _apt(work, arguments.get("apt_timeout_s", 600))
try:
os.makedirs(capture)
except FileExistsError:
return _say({"error": f"a stop claimed {capture} before the capture launched", "apt": apt})
control = os.path.join(capture, "control")
os.mkfifo(control, 0o600)
argv = [rea, "capture-desktop-application", "--desktop-entry", arguments["desktop_entry"],
"--output", os.path.join(capture, "capture"), "--exclude", work, "--json"]
for value in arguments.get("arguments", []):
argv += ["--argument", value]
for path in state_paths:
argv += ["--state-path", path]
if arguments.get("package_file"):
argv += ["--package-file", arguments["package_file"]]
# Reading and writing, so the capture never reads the end of its input between requests.
reader = os.open(control, os.O_RDWR)
process = None
try:
try:
with open(os.path.join(capture, "capture.out"), "wb") as out, \
open(os.path.join(capture, "progress.log"), "wb") as progress:
process = subprocess.Popen(
["sh", "-c", 'done=$1; shift; "$@"; code=$?; printf "%s" "$code" > "$done.tmp" && mv "$done.tmp" "$done"',
"sh", os.path.join(capture, "exit"), *argv],
cwd=os.path.expanduser("~"), env=_environment(work), stdin=reader, stdout=out, stderr=progress,
start_new_session=True)
except Exception as cause:
# Nothing will write exit. Record one and drop the FIFO, so stop does not wait on it.
try:
with open(os.path.join(capture, "exit"), "w", encoding="utf-8") as handle:
handle.write("1")
except OSError:
pass
try:
os.unlink(control)
except OSError:
pass
return _say({"error": f"the capture did not start: {cause}"})
finally:
os.close(reader)
if process is None:
return _say({"error": "the capture did not start"})
waited = _wait(capture, 0, {"window_open"}, None, arguments.get("timeout_s", 120))
return _say({"pid": process.pid, "apt": apt, **waited})
def _request(capture: str, line: str) -> bool:
"""Write one request to the running capture; False when nothing reads its FIFO any more."""
try:
fd = os.open(os.path.join(capture, "control"), os.O_WRONLY | os.O_NONBLOCK)
except OSError:
return False
try:
os.write(fd, (line + "\n").encode("utf-8"))
finally:
os.close(fd)
return True
def mark(arguments: dict) -> int:
"""Request a checkpoint and answer with the capture's acknowledgement line as it wrote it: the checkpoint's
counts and up to ten of the files that changed, home-relative, with a count of the rest."""
capture, name = os.path.abspath(arguments["capture"]), arguments["name"]
code = _exit_code(capture)
log = os.path.join(capture, "progress.log")
if code is not None:
return _say(_ended(capture, code))
# A checkpoint already taken, whose answer was lost, is answered from the capture's own log.
taken = _event_line(log, 0, {"checkpoint"}, name)
if taken is not None:
return _say({"line": taken})
offset = os.path.getsize(log)
if not _request(capture, f"checkpoint {name}"):
return _say({"error": "the capture is not reading its requests"})
return _say(_wait(capture, offset, {"checkpoint", "checkpoint_refused"}, name, arguments.get("timeout_s", 600)))
def stop(arguments: dict) -> int:
capture = os.path.abspath(arguments["capture"])
if _exit_code(capture) is None and not os.path.exists(os.path.join(capture, "control")):
# Never launched: claim the folder, so a start still on its way finds it used and launches nothing.
os.makedirs(capture, exist_ok=True)
return _say({"absent": True})
if _exit_code(capture) is None:
_request(capture, "stop")
deadline = time.monotonic() + arguments.get("timeout_s", 600)
while (code := _exit_code(capture)) is None and time.monotonic() < deadline:
time.sleep(POLL_S)
log = os.path.join(capture, "progress.log")
if code is None:
return _say({"timed_out": True, "progress": _tail(log)})
output = os.path.join(capture, "capture.out")
return _say({"exit_code": code, "progress": _tail(log), "output_bytes": os.path.getsize(output)})
def run(arguments: dict) -> int:
"""Run one REA command with its output in a file, after writing the files it reads."""
rea = shutil.which("rea")
if rea is None:
return _say({"error": "REA is not installed in this computer"})
os.makedirs(os.path.dirname(arguments["stdout"]), exist_ok=True)
for path, text in arguments.get("files", {}).items():
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as handle:
handle.write(text)
with open(arguments["stdout"], "wb") as out:
try:
done = subprocess.run([rea, *arguments["argv"]], cwd=os.path.expanduser("~"),
env=_environment(os.path.abspath(arguments["work"])), stdin=subprocess.DEVNULL, stdout=out,
stderr=subprocess.PIPE, timeout=arguments.get("timeout_s", 1800))
except subprocess.TimeoutExpired:
return _say({"timed_out": True})
return _say({"exit_code": done.returncode, "stdout_bytes": os.path.getsize(arguments["stdout"]),
"stderr": done.stderr[-TAIL_BYTES:].decode("utf-8", "replace")})
# The controller is sent with python -c, so a detached supervisor needs its
# own source. It alone holds the child process handle and signals its group.
LONG_RUN = r'''
import json, os, signal, subprocess, sys, threading, time
job = sys.argv[1]
result = {}
child = None
drainer = None
stderr_tail, stderr_errors = bytearray(), []
def drain(stream):
try:
while chunk := stream.read1(65536):
stderr_tail.extend(chunk)
del stderr_tail[:-4096]
except OSError as error:
stderr_errors.append(str(error))
finally:
try:
stream.close()
except OSError as error:
stderr_errors.append(str(error))
def timed_out_marker():
try:
open(os.path.join(job, "timed_out"), "x").close()
except FileExistsError:
pass
try:
request = json.loads("".join(sys.argv[2:]))
if os.path.exists(os.path.join(job, "cancel")):
result = {"absent": True}
else:
with open(request["stdout"], "wb") as out:
child = subprocess.Popen(request["argv"], cwd=request["home"], stdin=subprocess.DEVNULL,
stdout=out, stderr=subprocess.PIPE, start_new_session=True)
drainer = threading.Thread(target=drain, args=(child.stderr,), daemon=True)
drainer.start()
deadline = time.monotonic() + request["timeout_s"]
interrupted = False
timed_out = False
while child.poll() is None:
timed_out = timed_out or time.monotonic() >= deadline
if not interrupted and (timed_out or os.path.exists(os.path.join(job, "cancel"))):
interrupted = True
if timed_out:
timed_out_marker()
try:
os.killpg(child.pid, signal.SIGINT)
except OSError:
pass
time.sleep(0.2)
# A descendant can still hold stderr after the leader exits. Do
# not publish completion until that stream has actually closed.
while drainer.is_alive():
drainer.join(0.2)
if time.monotonic() >= deadline:
timed_out = True
timed_out_marker()
result = {"exit_code": child.returncode, "stdout_bytes": os.path.getsize(request["stdout"]),
"timed_out": timed_out, "cancelled": interrupted and not timed_out,
"stderr": bytes(stderr_tail).decode("utf-8", "replace")}
if stderr_errors:
result["error"] = f"The REA command's stderr could not be read: {stderr_errors[0]}"
except Exception as error:
result["error"] = f"The REA command supervisor failed: {error}"
# Keep the child handle until it exits, including when cancellation arrives
# after an unexpected supervisor error.
while child is not None and child.poll() is None:
if os.path.exists(os.path.join(job, "cancel")):
try:
os.killpg(child.pid, signal.SIGINT)
except OSError:
pass
time.sleep(0.2)
if drainer is not None and drainer.ident is not None:
drainer.join()
if child is not None:
result.setdefault("exit_code", child.returncode)
result.setdefault("stderr", bytes(stderr_tail).decode("utf-8", "replace"))
try:
result.setdefault("stdout_bytes", os.path.getsize(request["stdout"]))
except OSError:
pass
if child is None or child.poll() is not None:
temporary = os.path.join(job, "result.tmp")
with open(temporary, "w", encoding="utf-8") as out:
json.dump(result, out)
os.replace(temporary, os.path.join(job, "result.json"))
'''
def run_begin(arguments: dict) -> int:
"""Start one long REA command in its own capture-owned job folder."""
rea = shutil.which("rea")
if rea is None:
return _say({"error": "REA is not installed in this computer"})
job = os.path.abspath(arguments["job"])
os.makedirs(os.path.dirname(job), exist_ok=True)
try:
os.mkdir(job, 0o700)
except FileExistsError:
return _say({"refused": "the command job folder is already used"})
process = None
try:
with open(os.path.join(job, "owner"), "x", encoding="ascii") as out:
out.write(arguments["token"])
for path, text in arguments.get("files", {}).items():
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as handle:
handle.write(text)
os.makedirs(os.path.dirname(arguments["stdout"]), exist_ok=True)
request = json.dumps({"argv": [rea, *arguments["argv"]], "stdout": arguments["stdout"],
"home": os.path.expanduser("~"), "timeout_s": arguments["timeout_s"]})
chunks = [request[i:i + 60000] for i in range(0, len(request), 60000)]
with open(os.path.join(job, "supervisor.log"), "wb") as log:
process = subprocess.Popen([sys.executable, "-I", "-c", LONG_RUN, job, *chunks], cwd=os.path.expanduser("~"),
env=_environment(os.path.abspath(arguments["work"])), stdin=subprocess.DEVNULL,
stdout=log, stderr=log, start_new_session=True)
except (OSError, ValueError) as error:
stage = "did not start" if process is None else "started but its setup did not finish"
result = {"error": f"The REA command supervisor {stage}: {error}"}
if process is None:
temporary = os.path.join(job, "result.tmp")
with open(temporary, "w", encoding="utf-8") as out:
json.dump(result, out)
os.replace(temporary, os.path.join(job, "result.json"))
return _say(result)
return _say({"started": True})
def _run_result(job: str):
try:
with open(os.path.join(job, "result.json"), encoding="utf-8") as source:
return json.load(source)
except FileNotFoundError:
return None
def _owns_run(job: str, token: str) -> bool:
try:
with open(os.path.join(job, "owner"), encoding="ascii") as owner:
return owner.read() == token
except OSError:
return False
def run_poll(arguments: dict) -> int:
job = os.path.abspath(arguments["job"])
if not _owns_run(job, arguments["token"]):
return _say({"error": "the command job ownership is not confirmed"})
deadline = time.monotonic() + arguments.get("timeout_s", 30)
while (result := _run_result(job)) is None and time.monotonic() < deadline:
time.sleep(POLL_S)
if result is None:
pending = {"pending": True, "timed_out": os.path.exists(os.path.join(job, "timed_out"))}
progress = _tail(os.path.join(job, "supervisor.log"))
return _say({**pending, **({"progress": progress} if progress else {})})
return _say({"finished": True, **result})
def run_cancel(arguments: dict) -> int:
job = os.path.abspath(arguments["job"])
os.makedirs(os.path.dirname(job), exist_ok=True)
try:
os.mkdir(job, 0o700)
except FileExistsError:
pass
else:
# A late begin cannot launch into a folder cancellation claimed.
with open(os.path.join(job, "owner"), "x", encoding="ascii") as owner:
owner.write(arguments["token"])
return _say({"absent": True})
if not _owns_run(job, arguments["token"]):
return _say({"error": "the command job ownership is not confirmed"})
with open(os.path.join(job, "cancel"), "a", encoding="utf-8"):
pass
return run_poll(arguments)
def listing(arguments: dict) -> int:
"""Every regular file under a folder, by its path relative to the home, with its size and SHA-256.
The list goes to the file ``output`` names, outside the folder, and the answer gives its size: a large
bundle's list outgrows the 1 MiB of stdout a command answers with, so REA reads it through ``data/get``.
"""
home = os.path.expanduser("~")
files, skipped = [], []
def fail_walk(error: OSError) -> None:
raise error
for directory, folders, names in os.walk(os.path.abspath(arguments["root"]), onerror=fail_walk):
folders.sort()
# os.walk does not follow directory symlinks; disclose them to the consumer.
for name in folders[:]:
path = os.path.join(directory, name)
if os.path.islink(path):
skipped.append(os.path.relpath(path, home))
folders.remove(name)
for name in sorted(names):
path = os.path.join(directory, name)
relative = os.path.relpath(path, home)
if os.path.islink(path) or not os.path.isfile(path):
skipped.append(relative)
continue
digest = hashlib.sha256()
with open(path, "rb") as handle:
for chunk in iter(lambda: handle.read(1 << 20), b""):
digest.update(chunk)
files.append({"path": relative, "size": os.path.getsize(path), "sha256": digest.hexdigest()})
output = os.path.abspath(arguments["output"])
os.makedirs(os.path.dirname(output), exist_ok=True)
with open(output, "w", encoding="utf-8") as handle:
json.dump({"files": files, "skipped": skipped}, handle)
return _say({"listing_bytes": os.path.getsize(output)})
COMMANDS = {"start": start, "mark": mark, "stop": stop, "run": run, "list": listing,
"run-begin": run_begin, "run-poll": run_poll, "run-cancel": run_cancel}
if __name__ == "__main__":
sys.exit(COMMANDS[sys.argv[1]](json.loads("".join(sys.argv[2:]))))
-83
View File
@@ -1,83 +0,0 @@
"""Keep one agentenv computer up for REA until stdin closes or SIGTERM arrives.
REA runs ``computer.py <bundle folder> [task]`` with the agentenv runner's own interpreter, so ``agent_env``
imports. It deploys the bundle's task, ``smoke`` by default, on this machine with
``run_bundle(keep=True, emulate=False)``, prints one JSON line naming the computer's address, MCP URL, control
token and sandbox id, or ``{"error": ...}``, and tears the computer down once its stdin closes or it gets SIGTERM
or SIGINT. Everything else it or agentenv prints, a build's output included, goes to stderr. It exits 0 once
the teardown left nothing up, 1 when it did, and 2 when no computer came up.
"""
from __future__ import annotations
import json
import os
import signal
import sys
import threading
def _protocol():
"""A private copy of stdout for the JSON line; fd 1 then points at stderr, as do children that inherit it."""
protocol = os.fdopen(os.dup(1), "w", encoding="utf-8")
os.dup2(2, 1)
sys.stdout = sys.stderr
return protocol
def _say(protocol, value: dict) -> None:
protocol.write(json.dumps(value) + "\n")
protocol.flush()
def _computer(result):
"""The run's deployed computer, or why there is none."""
run = result.runs[0] if result.runs else None
if run is None:
return None, "the bundle ran no task"
if run.error is not None or run.failed_steps:
return None, f"the task failed: {run.error or run.failed_steps}"
for env in run.context.deployed_envs:
if env.environment_url and env.mcp_url and getattr(env, "sandbox_id", None):
return env, None
return None, "the task deployed no computer"
def main(argv: list[str]) -> int:
protocol = _protocol()
bundle, task = argv[1], argv[2] if len(argv) > 2 else "smoke"
try:
from agent_env.bundle import run_bundle
except ImportError as error:
_say(protocol, {"error": f"agent_env does not import with {sys.executable}: {error}"})
return 2
try:
result = run_bundle(bundle, tasks=[task], keep=True, emulate=False, sandbox="local",
on_progress=lambda line: print(line, file=sys.stderr, flush=True))
except BaseException as error: # noqa: BLE001 - a signal's RunInterrupted too; run_bundle tore its runs down
_say(protocol, {"error": f"{type(error).__name__}: {error}"})
return 2
stop = threading.Event()
for signum in (signal.SIGTERM, signal.SIGINT):
signal.signal(signum, lambda *_: stop.set())
env, problem = _computer(result)
try:
if env is None:
_say(protocol, {"error": problem})
else:
_say(protocol, {"base_url": env.environment_url, "mcp_url": env.mcp_url,
"control_token": env.control_token, "sandbox_id": env.sandbox_id})
threading.Thread(target=lambda: (sys.stdin.buffer.read(), stop.set()), daemon=True).start()
while not stop.wait(0.5):
pass
finally:
torn_down = result.teardown()
still_up = [sandbox for run in torn_down.runs if run.torn_down is not None for sandbox in run.torn_down.still_up]
if still_up:
print(f"still up after the teardown: {still_up}", file=sys.stderr, flush=True)
return 1
return 0 if env is not None else 2
if __name__ == "__main__":
sys.exit(main(sys.argv))
-121
View File
@@ -1,121 +0,0 @@
"""Run one solve of a bundle's task in a fresh agentenv computer for REA and print how it ended.
REA runs ``solve.py <bundle folder> <task> <model> <agent>`` with the agentenv runner's own interpreter, so
``agent_env`` imports. It runs the task once on this machine with ``run_bundle``, the solver deployed as the agent
and asked on the model, the endpoint taken from the environment or agentenv's config as a run takes it, and the
computer torn down as the run ends. It prints one JSON line: the run's outcome, instance id, seconds, each check's
result, the solver's turns and cost from its trajectory when that is a local file, and agentenv's version; or
``{"error": ...}`` when the task could not run. Everything else, agentenv's progress included, goes to stderr. It exits
0 after a run and 2 when none ran.
"""
from __future__ import annotations
import json
import os
import sys
from importlib import metadata
from pathlib import Path
#: The most characters of a check's description or a run's error REA keeps.
MAX_TEXT = 400
def _protocol():
"""A private copy of stdout for the JSON line; fd 1 then points at stderr, as do children that inherit it."""
protocol = os.fdopen(os.dup(1), "w", encoding="utf-8")
os.dup2(2, 1)
sys.stdout = sys.stderr
return protocol
def _say(protocol, value: dict) -> None:
protocol.write(json.dumps(value) + "\n")
protocol.flush()
def _field(value, name: str):
"""A field of agentenv's dataclasses, or of the dicts a stored context holds."""
return value.get(name) if isinstance(value, dict) else getattr(value, name, None)
def _trajectory_end(context) -> tuple:
"""The solver's turns and cost from the last record of its local trajectory, or None for each."""
for response in reversed(_field(context, "prompt_responses") or []):
url = _field(response, "agent_trajectory_object_url") or ""
if not url.startswith("file://"):
continue
try:
records = json.loads(Path(url[len("file://"):]).read_text(encoding="utf-8"))
except (OSError, ValueError):
return None, None
last = records[-1] if isinstance(records, list) and records and isinstance(records[-1], dict) else {}
turns, usd = last.get("num_turns"), last.get("total_cost_usd")
return (turns if isinstance(turns, int) else None,
float(usd) if isinstance(usd, (int, float)) and not isinstance(usd, bool) else None)
return None, None
def _checks(context, task: str) -> list:
"""Each check's result as the task's verifier recorded it under the instance's verifications."""
verifications = (_field(context, "metadata") or {}).get("verifications") or {}
found = verifications.get(task) or next(iter(verifications.values()), {})
results = found.get("results") if isinstance(found, dict) else None
return [{"id": str(result.get("id")), "passed": result.get("result") is True,
"description": str(result.get("description", ""))[:MAX_TEXT]}
for result in results or [] if isinstance(result, dict)]
def _version() -> str | None:
try:
return metadata.version("agentenv-framework")
except metadata.PackageNotFoundError:
return None
def _cleanup(result) -> dict:
"""Only a returned teardown report can confirm what agentenv released."""
if result is None:
return {"unknown": True, "sandboxes": []}
unknown, sandboxes = False, []
for run in result.runs:
report = getattr(run, "torn_down", None)
if report is None:
unknown = True
else:
sandboxes.extend({"id": item.sandbox_id, "type": item.sandbox_type} for item in report.still_up)
return {"unknown": unknown, "sandboxes": sandboxes}
def main(argv: list[str]) -> int:
protocol = _protocol()
bundle, task, model, agent = argv[1:5]
try:
from agent_env.bundle import run_bundle
except ImportError as error:
_say(protocol, {"error": f"agent_env does not import with {sys.executable}: {error}",
"cleanup": {"unknown": False, "sandboxes": []}})
return 2
try:
result = run_bundle(bundle, tasks=[task], model=model, agent=agent, sandbox="local", emulate=False,
on_progress=lambda line: print(line, file=sys.stderr, flush=True))
except BaseException as error: # noqa: BLE001 - a signal's RunInterrupted too; run_bundle tore its run down
_say(protocol, {"error": f"{type(error).__name__}: {error}"[:MAX_TEXT * 4],
"cleanup": _cleanup(getattr(error, "result", None))})
return 2
run = result.runs[0] if result.runs else None
if run is None:
_say(protocol, {"error": "the bundle ran no task", "cleanup": _cleanup(result)})
return 2
turns, usd = _trajectory_end(run.context)
error = getattr(run, "error", None)
_say(protocol, {"outcome": run.outcome.value, "instance_id": run.instance_id,
"seconds": round(float(run.duration), 1), "checks": _checks(run.context, task),
"turns": turns, "usd": usd, "agentenv": _version(),
"error": None if error is None else f"{type(error).__name__}: {error}"[:MAX_TEXT],
"cleanup": _cleanup(result)})
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
-63
View File
@@ -1,63 +0,0 @@
"""Recover local sandboxes under one stopped solve's private agentenv root."""
from __future__ import annotations
import asyncio
import contextlib
import json
import os
import subprocess
import sys
from pathlib import Path
def _docker(*args: str) -> str:
command = os.environ.get("REA_DOCKER_COMMAND") or "docker"
result = subprocess.run([command, *args], capture_output=True, text=True, check=False)
if result.returncode:
raise RuntimeError(f"Docker {' '.join(args[:2])} failed: {result.stderr[-1000:]}")
return result.stdout.strip()
async def cleanup(root: Path) -> None:
# A helper has already stopped. agentenv lists the sandboxes under the root
# (AGENT_ENV_LOCAL_SANDBOX_DIR); anything else there remains owned for inspection.
if not any(root.iterdir()):
return
from agent_env.providers.sandbox_providers.local_docker.provider import (
LocalSandboxProvider, local_sandbox_ids, remove_local_work_dir,
)
for sandbox_id in local_sandbox_ids():
await LocalSandboxProvider().terminate_sandbox(sandbox_id)
# The pinned provider reports incomplete teardown and retains the folder.
# Keep REA's independent Docker/Linux checks before releasing that owner.
if _docker("ps", "-aq", "--filter", f"name=^agent-{sandbox_id}$"):
raise RuntimeError(f"Sandbox {sandbox_id} still has its agent container")
if _docker("ps", "-aq", "--filter", f"label=agentenv.sandbox={sandbox_id}"):
raise RuntimeError(f"Sandbox {sandbox_id} still has labeled containers")
if _docker("network", "ls", "-q", "--filter", f"label=agentenv.sandbox={sandbox_id}"):
raise RuntimeError(f"Sandbox {sandbox_id} still has labeled networks")
if sys.platform != "linux":
raise RuntimeError(f"Cannot verify host processes for sandbox {sandbox_id} on {sys.platform}")
from agent_env.providers.sandbox_providers.local_docker.provider import _marked_pids
if _marked_pids(sandbox_id):
raise RuntimeError(f"Sandbox {sandbox_id} still has marked host processes")
if remove_local_work_dir(sandbox_id) is None:
raise RuntimeError(f"Could not confirm removal of sandbox {sandbox_id}'s work folder")
leftover = sorted(entry.name for entry in root.iterdir())
if leftover:
raise RuntimeError(f"Unexpected sandbox root entries: {', '.join(leftover)}")
def main() -> int:
try:
with contextlib.redirect_stdout(sys.stderr):
asyncio.run(cleanup(Path(os.environ["AGENT_ENV_LOCAL_SANDBOX_DIR"])))
print(json.dumps({"ok": True}))
return 0
except Exception as error:
print(json.dumps({"ok": False, "error": f"{type(error).__name__}: {error}"}))
return 1
if __name__ == "__main__":
sys.exit(main())
-232
View File
@@ -1,232 +0,0 @@
"""Exercise the capture controller's filesystem inventory boundary."""
import json
import io
from contextlib import redirect_stdout
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
from bridge.agentenv import capture
CONTROLLER = Path(__file__).with_name("capture.py")
class CaptureListingTest(unittest.TestCase):
def run_listing(self, root, output):
return subprocess.run(
[
sys.executable,
str(CONTROLLER),
"list",
json.dumps({"root": str(root), "output": str(output)}),
],
text=True,
capture_output=True,
check=False,
)
def test_directory_symlink_is_disclosed_without_following_it(self):
with tempfile.TemporaryDirectory() as parent:
parent = Path(parent)
root = parent / "root"
target = parent / "target"
output = parent / "listing.json"
root.mkdir()
target.mkdir()
(root / "present.txt").write_text("present")
(target / "outside.txt").write_text("outside")
try:
(root / "linked").symlink_to(target, target_is_directory=True)
except OSError as error:
self.skipTest(f"directory symlinks unavailable: {error}")
result = self.run_listing(root, output)
self.assertEqual(result.returncode, 0, result.stderr)
listing = json.loads(output.read_text())
self.assertEqual(
[item["path"] for item in listing["files"]],
[os.path.relpath(root / "present.txt", Path.home())],
)
self.assertEqual(
listing["skipped"], [os.path.relpath(root / "linked", Path.home())]
)
def test_missing_root_does_not_produce_an_empty_successful_inventory(self):
with tempfile.TemporaryDirectory() as parent:
parent = Path(parent)
output = parent / "listing.json"
result = self.run_listing(parent / "missing", output)
self.assertNotEqual(result.returncode, 0)
self.assertFalse(output.exists())
@unittest.skipIf(sys.platform == "win32", "desktop build computers use Linux")
class LongRunTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
self.root = Path(self.tmp.name)
self.bin = self.root / "bin"
self.bin.mkdir()
rea = self.bin / "rea"
rea.write_text(
f"#!{sys.executable}\n"
"import os, signal, subprocess, sys, time\n"
"signal.signal(signal.SIGINT, lambda *_: sys.exit(130))\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'wait':\n"
" open(sys.argv[2], 'w').close()\n"
" time.sleep(30)\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'ignore':\n"
" signal.signal(signal.SIGINT, signal.SIG_IGN)\n"
" open(sys.argv[2], 'w').close()\n"
" while not os.path.exists(sys.argv[3]): time.sleep(0.02)\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'stderr':\n"
" sys.stderr.write('x' * 100000 + 'TAIL')\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'hold':\n"
" subprocess.Popen([sys.executable, '-c', 'import os,sys,time\\nwhile not os.path.exists(sys.argv[1]): time.sleep(0.02)', sys.argv[2]], stdout=subprocess.DEVNULL, stderr=sys.stderr, start_new_session=True)\n"
" sys.exit(0)\n"
"print(os.environ.get('AGENTENV_CONTROL_TOKEN', 'no token'))\n"
)
rea.chmod(0o755)
self.environment = {**os.environ, "HOME": str(self.root), "PATH": f"{self.bin}{os.pathsep}{os.environ.get('PATH', '')}",
"AGENTENV_CONTROL_TOKEN": "not-for-the-child"}
self.job = self.root / "job"
self.output = self.root / "out.json"
def command(self, name, **values):
result = subprocess.run([sys.executable, str(CONTROLLER), name, json.dumps(values)],
env=self.environment, text=True, capture_output=True, timeout=10, check=True)
return json.loads(result.stdout)
def begin(self, argv, timeout=60):
return self.command("run-begin", job=str(self.job), work=str(self.root), stdout=str(self.output),
token="owner-token", argv=argv, files={}, timeout_s=timeout)
def test_runs_and_reports_its_output(self):
self.assertEqual(self.begin([]), {"started": True})
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertEqual(result["exit_code"], 0)
self.assertEqual(self.output.read_text().strip(), "no token")
def test_passes_large_command_without_persisting_its_arguments(self):
self.assertEqual(self.begin(["done", *("x" * 4000 for _ in range(20))]), {"started": True})
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertEqual(result["exit_code"], 0)
self.assertFalse((self.job / "request.json").exists())
def test_keeps_only_a_bounded_stderr_tail(self):
self.assertEqual(self.begin(["stderr"]), {"started": True})
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertEqual(len(result["stderr"]), 4096)
self.assertTrue(result["stderr"].endswith("TAIL"))
def test_cancels_a_running_child_and_confirms_exit(self):
ready = self.root / "ready"
self.assertEqual(self.begin(["wait", str(ready)]), {"started": True})
import time
deadline = time.monotonic() + 5
while not ready.exists() and time.monotonic() < deadline:
time.sleep(0.02)
self.assertTrue(ready.exists())
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=5)
self.assertTrue(result["cancelled"])
self.assertEqual(result["exit_code"], 130)
def test_cancel_before_late_start_claims_the_job(self):
self.assertEqual(self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=1), {"absent": True})
self.assertIn("already used", self.begin([])["refused"])
self.assertFalse(self.output.exists())
def test_cancel_refuses_a_folder_without_the_matching_owner(self):
self.job.mkdir()
(self.job / "owner").write_text("someone-else")
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=1)
self.assertIn("ownership is not confirmed", result["error"])
self.assertFalse((self.job / "cancel").exists())
def test_reports_supervisor_output_if_completion_is_unconfirmed(self):
self.job.mkdir()
(self.job / "owner").write_text("owner-token")
(self.job / "supervisor.log").write_text("supervisor failed")
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=0)
self.assertEqual(result["progress"], "supervisor failed")
self.assertTrue(result["pending"])
def test_reports_a_timed_out_child_that_has_not_stopped(self):
ready, release = self.root / "ready", self.root / "release"
self.assertEqual(self.begin(["ignore", str(ready), str(release)], timeout=3), {"started": True})
try:
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=4)
self.assertTrue(result["pending"])
self.assertTrue(result["timed_out"])
self.assertTrue(ready.exists())
finally:
release.touch()
self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
def test_waits_for_an_inherited_stderr_pipe_after_the_child_exits(self):
release = self.root / "release"
self.assertEqual(self.begin(["hold", str(release)], timeout=3), {"started": True})
try:
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=4)
self.assertTrue(result["pending"])
self.assertTrue(result["timed_out"])
finally:
release.touch()
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertTrue(result["finished"])
def test_failure_closing_supervisor_log_after_launch_does_not_claim_exit(self):
ready = self.root / "ready"
original_open = open
original_popen = subprocess.Popen
supervisors = []
class FailingClose:
def __init__(self, file):
self.file = file
def __enter__(self):
return self.file
def __exit__(self, *args):
self.file.close()
raise OSError("log close failed")
def opening(path, *args, **kwargs):
file = original_open(path, *args, **kwargs)
return FailingClose(file) if str(path).endswith("supervisor.log") else file
def spawning(*args, **kwargs):
process = original_popen(*args, **kwargs)
supervisors.append(process)
def cleanup():
if process.poll() is None:
(self.job / "cancel").touch()
process.wait(timeout=5)
self.addCleanup(cleanup)
return process
arguments = {"job": str(self.job), "work": str(self.root), "stdout": str(self.output),
"token": "owner-token", "argv": ["wait", str(ready)], "files": {}, "timeout_s": 60}
with patch.dict(os.environ, self.environment, clear=True), patch("builtins.open", side_effect=opening), patch.object(capture.subprocess, "Popen", side_effect=spawning), redirect_stdout(io.StringIO()) as output:
capture.run_begin(arguments)
self.assertIn("started but its setup did not finish", json.loads(output.getvalue())["error"])
import time
deadline = time.monotonic() + 5
while not ready.exists() and time.monotonic() < deadline:
time.sleep(0.02)
self.assertTrue(ready.exists())
self.assertFalse((self.job / "result.json").exists())
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=5)
self.assertTrue(result["cancelled"])
supervisors[0].wait(timeout=5)
if __name__ == "__main__":
unittest.main()
-108
View File
@@ -1,108 +0,0 @@
"""Exercise cleanup decisions without a live agentenv provider."""
import asyncio
import tempfile
import types
import unittest
from pathlib import Path
from unittest.mock import patch
from bridge.agentenv import solve_cleanup
class SolveCleanupTest(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.folder = self.root / "agent-env-local-a1b2c3d4-abcdef12"
self.folder.mkdir()
self.terminated = []
self.module = types.ModuleType("agent_env.providers.sandbox_providers.local_docker.provider")
test = self
class Provider:
async def terminate_sandbox(self, sandbox_id):
test.terminated.append(sandbox_id)
self.module.LocalSandboxProvider = Provider
self.module.local_sandbox_ids = self._listed
self.module.remove_local_work_dir = lambda sandbox_id: self._remove(sandbox_id)
self.module._marked_pids = lambda _sandbox_id: []
self.patch_module = patch.dict("sys.modules", {self.module.__name__: self.module})
self.patch_module.start()
self.addCleanup(self.patch_module.stop)
def _listed(self):
# As agentenv lists them: `agent-env-<id>-<suffix>` folders, whatever the id's shape.
found = set()
for entry in self.root.iterdir():
sandbox_id, _, suffix = entry.name.removeprefix("agent-env-").rpartition("-")
if entry.name.startswith("agent-env-") and sandbox_id and suffix and entry.is_dir():
found.add(sandbox_id)
return sorted(found)
def _remove(self, sandbox_id):
folder = next(self.root.glob(f"agent-env-{sandbox_id}-*"))
folder.rmdir()
return folder
def run_cleanup(self):
asyncio.run(solve_cleanup.cleanup(self.root))
def test_terminates_owned_sandbox_before_removing_its_folder(self):
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
self.run_cleanup()
self.assertEqual(self.terminated, ["local-a1b2c3d4"])
self.assertFalse(self.folder.exists())
def test_retains_folder_when_docker_reports_container_then_retries(self):
with patch.object(solve_cleanup, "_docker", return_value="still-running"), patch.object(solve_cleanup.sys, "platform", "linux"):
with self.assertRaisesRegex(RuntimeError, "still has its agent container"):
self.run_cleanup()
self.assertTrue(self.folder.exists())
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
self.run_cleanup()
self.assertFalse(self.folder.exists())
self.assertEqual(len(self.terminated), 2)
def test_partial_cleanup_and_late_folder_are_retried(self):
later = self.root / "agent-env-local-f1223344-abcdef12"
later.mkdir()
def docker(*args):
return "still-running" if "local-f1223344" in " ".join(args) else ""
with patch.object(solve_cleanup, "_docker", side_effect=docker), patch.object(solve_cleanup.sys, "platform", "linux"):
with self.assertRaisesRegex(RuntimeError, "still has its agent container"):
self.run_cleanup()
self.assertTrue(later.exists())
self.assertFalse(self.folder.exists())
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
self.run_cleanup()
self.assertEqual(list(self.root.iterdir()), [])
def test_macos_keeps_folder_when_host_process_cleanup_cannot_be_verified(self):
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "darwin"):
with self.assertRaisesRegex(RuntimeError, "Cannot verify host processes"):
self.run_cleanup()
self.assertTrue(self.folder.exists())
def test_cleans_up_each_sandbox_agentenv_lists_whatever_its_id_looks_like(self):
self.folder.rmdir()
listed = self.root / "agent-env-local-a1b2c3d4e5-abcdef12"
listed.mkdir()
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
self.run_cleanup()
self.assertEqual(self.terminated, ["local-a1b2c3d4e5"])
self.assertFalse(listed.exists())
def test_unexpected_entry_remains_owned(self):
self.folder.rmdir()
(self.root / "unexpected").mkdir()
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
with self.assertRaisesRegex(RuntimeError, "Unexpected sandbox root entries: unexpected"):
self.run_cleanup()
self.assertEqual(self.terminated, [])
self.assertTrue((self.root / "unexpected").exists())
if __name__ == "__main__":
unittest.main()
File diff suppressed because it is too large. Load diff
+105 -619
View File
@@ -11,7 +11,6 @@ import json
import hashlib
import hmac
import os
import re
import secrets
import socket
import stat
@@ -29,156 +28,14 @@ if os.name == "nt":
import cutter
try:
from PySide6.QtCore import QCoreApplication, QObject, Signal
from PySide6.QtCore import QObject, Signal
except ImportError:
from PySide2.QtCore import QCoreApplication, QObject, Signal
# Request and response frame budget in bytes, excluding the trailing LF.
# REA_BRIDGE_MAX_REQUEST_BYTES selects it. Unset stays 16777216.
# 7464 is the authenticated frame of the largest bounded symbol name.
# The 64-receipt recovery status is smaller with current_file null. A path
# that fits is kept. A path that does not fit is null, not the descriptor path.
# 536870887 is Node's 536870888-character string ceiling minus the framing LF.
# The ceiling is the encoded payload; JSON expansion is measured before it.
# A fitting response is signed from bounded JSON pieces and written in 64 KiB
# chunks, so this maximum does not retain several copies of the frame.
_BRIDGE_MAX_REQUEST_BYTES_SETTING = "REA_BRIDGE_MAX_REQUEST_BYTES"
DEFAULT_BRIDGE_MAX_REQUEST_BYTES = 16 * 1024 * 1024
MINIMUM_BRIDGE_MAX_REQUEST_BYTES = 7464
MAXIMUM_BRIDGE_MAX_REQUEST_BYTES = 536870887
# The outer authenticated frame around the escaped JSON payload, excluding LF.
_AUTHENTICATED_PREFIX = b'{"response":"'
_AUTHENTICATED_PROOF_MARK = b'","proof":"'
_PROOF_HEX_LENGTH = 64
_AUTHENTICATED_FRAME_OVERHEAD = (
len(_AUTHENTICATED_PREFIX)
+ len(_AUTHENTICATED_PROOF_MARK)
+ _PROOF_HEX_LENGTH
+ len(b'"}')
)
_AUTHENTICATED_WRITE_BUFFER_BYTES = 64 * 1024
# Raw JSON is produced in pieces no larger than this many characters.
# One character expands to at most six JSON bytes (\u00XX) before the outer
# string escape, so a piece stays far below the frame ceiling.
_JSON_CHUNK_CHARS = 64 * 1024
# Fixed admission cap. It does not follow REA_BRIDGE_MAX_REQUEST_BYTES.
# A further accept is closed. Starting a worker must not stop the listener.
MAX_CONNECTION_WORKERS = 8
def _close_socket(connection: socket.socket) -> None:
try:
connection.close()
except OSError:
pass
def _bridge_max_request_bytes_constraint():
return (
"REA_BRIDGE_MAX_REQUEST_BYTES must be a positive decimal integer "
"from %d to %d bytes; the default is %d bytes (16 MiB). "
"A value below %d bytes cannot carry the largest admitted "
"symbol-name command, and a larger value is refused so the encoded "
"payload plus its framing LF stays one string under Node's "
"single-string ceiling."
% (
MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
DEFAULT_BRIDGE_MAX_REQUEST_BYTES,
MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
)
)
def _parse_bridge_max_request_bytes(raw):
"""Refuse a non-decimal or out-of-range ceiling. Never fall back."""
constraint = _bridge_max_request_bytes_constraint()
if isinstance(raw, bool):
raise RuntimeError(constraint)
if isinstance(raw, int):
value = raw
elif isinstance(raw, str):
# ASCII digits only. str.isdigit() accepts numerals such as U+0668
# that the TypeScript setting rejects.
if re.fullmatch(r"[1-9][0-9]*", raw) is None:
raise RuntimeError(constraint)
value = int(raw)
else:
raise RuntimeError(constraint)
if (
value < MINIMUM_BRIDGE_MAX_REQUEST_BYTES
or value > MAXIMUM_BRIDGE_MAX_REQUEST_BYTES
):
raise RuntimeError(constraint)
return value
def _resolve_bridge_max_request_bytes():
"""Prefer an injected global, then the process environment, then 16 MiB."""
injected = globals().get(_BRIDGE_MAX_REQUEST_BYTES_SETTING)
if injected is not None:
return _parse_bridge_max_request_bytes(injected), "setting"
raw = os.environ.get(_BRIDGE_MAX_REQUEST_BYTES_SETTING)
if raw is None:
return DEFAULT_BRIDGE_MAX_REQUEST_BYTES, "default"
return _parse_bridge_max_request_bytes(raw), "setting"
MAX_FRAME_BYTES, BRIDGE_MAX_REQUEST_BYTES_SOURCE = (
_resolve_bridge_max_request_bytes()
)
def _bridge_max_request_bytes_recovery():
return (
"The effective REA_BRIDGE_MAX_REQUEST_BYTES value is %d bytes "
"(source %s; default %d, floor %d, maximum %d). "
"Set REA_BRIDGE_MAX_REQUEST_BYTES in the environment that starts REA "
"to a decimal integer from %d to %d, then restart REA. "
"Restart Hopper or Cutter from that same environment so the bridge "
"uses this ceiling."
% (
MAX_FRAME_BYTES,
BRIDGE_MAX_REQUEST_BYTES_SOURCE,
DEFAULT_BRIDGE_MAX_REQUEST_BYTES,
MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
)
)
def _frame_limit_exceeded():
return ValueError(
"frame-limit-exceeded: Bridge request exceeds the %d-byte payload limit. %s"
% (MAX_FRAME_BYTES, _bridge_max_request_bytes_recovery())
)
from PySide2.QtCore import QObject, Signal
MAX_FRAME_BYTES = 16 * 1024 * 1024
SOCKET_TIMEOUT_SECONDS = 20
AUTH_FRAME_TIMEOUT_SECONDS = 1
# The client request deadline. The authenticated frame read may use up to
# this long; it does not grow past it when the byte ceiling is raised.
REQUEST_DEADLINE_SECONDS = 25
def _authenticated_frame_timeout_seconds():
"""Time to hash and transfer one maximum-sized authenticated frame.
The hello frame stays at one second. The following command frame is
built only after that challenge, so its read budget follows the
configured payload size, at one second per mebibyte, and never exceeds
the 25 second client deadline.
"""
mebibytes = (MAX_FRAME_BYTES + (1024 * 1024) - 1) // (1024 * 1024)
return min(
REQUEST_DEADLINE_SECONDS,
max(AUTH_FRAME_TIMEOUT_SECONDS, mebibytes),
)
MAX_DESCRIPTOR_BYTES = 64 * 1024
MAX_COMPLETION_RECEIPTS = 64
@@ -469,16 +326,15 @@ class _RequestDispatcher(QObject):
self._receipt_lock = threading.Lock()
self.current_file = self._read_current_file()
try:
value = QCoreApplication.applicationVersion()
self.cutter_version = value if isinstance(value, str) and value else None
self.cutter_version = str(cutter.core().getVersionInformation())
except Exception:
self.cutter_version = None
@staticmethod
def _read_current_file() -> str | None:
try:
value = cutter.core().getConfig("file.path")
return value if isinstance(value, str) and value else None
value = cutter.core().currentlyOpenFile()
return str(value) if value else None
except Exception:
return None
@@ -623,17 +479,6 @@ class _BridgeServer(threading.Thread):
self.token = secrets.token_urlsafe(32)
self.poisoned = False
self.connection_ordinal = 0
# The client proof is inside the authenticated frame, so that read
# cannot start after proof. It runs off the accept loop. One
# authenticated command still runs at a time. completed_connection
# is the longest prefix of these ordinals whose handlers have
# terminated, not "this status ordinal minus one".
self._command_lock = threading.Lock()
self._connections_lock = threading.Lock()
self._connections: set[socket.socket] = set()
self._workers: list[threading.Thread] = []
self._finished_through = 0
self._finished_ordinals: set[int] = set()
self.directory = _bridge_directory()
self.descriptor = self.directory / f"cutter-{os.getpid()}-{self.session_id}.json"
@@ -654,7 +499,6 @@ class _BridgeServer(threading.Thread):
"current_file": self.dispatcher.current_file,
"cutter_version": self.dispatcher.cutter_version,
"identity_status": "partial",
"max_request_bytes": MAX_FRAME_BYTES,
}
if os.name == "nt":
_write_windows_descriptor(self.directory, self.descriptor, json.dumps(payload).encode("utf-8"))
@@ -696,46 +540,8 @@ class _BridgeServer(threading.Thread):
continue
except OSError:
break
with self._connections_lock:
self._workers = [
worker for worker in self._workers if worker.is_alive()
]
at_capacity = len(self._workers) >= MAX_CONNECTION_WORKERS
if at_capacity:
_close_socket(connection)
continue
self.connection_ordinal += 1
connection_ordinal = self.connection_ordinal
worker = threading.Thread(
target=self._serve_connection,
args=(connection, connection_ordinal),
name="rea-cutter-bridge-connection",
daemon=True,
)
with self._connections_lock:
self._connections.add(connection)
self._workers.append(worker)
try:
worker.start()
except RuntimeError:
with self._connections_lock:
self._connections.discard(connection)
if worker in self._workers:
self._workers.remove(worker)
self._note_connection_finished(connection_ordinal)
_close_socket(connection)
with self._connections_lock:
workers = list(self._workers)
for worker in workers:
worker.join(timeout=1)
self.server.close()
try:
self.descriptor.unlink(missing_ok=True)
except OSError:
pass
def _serve_connection(self, connection: socket.socket, connection_ordinal: int) -> None:
try:
with connection:
connection.settimeout(AUTH_FRAME_TIMEOUT_SECONDS)
authenticated = False
@@ -743,73 +549,60 @@ class _BridgeServer(threading.Thread):
hello = self._receive_frame(connection)
if not isinstance(hello, dict) or hello.get("kind") != "rea-auth-v2" or not isinstance(hello.get("nonce"), str) or len(hello["nonce"]) != 64:
self._send(connection, {"ok": False, "error": "authentication-failed"})
return
continue
client_nonce = hello["nonce"]
server_nonce = f"{connection_ordinal:016x}{secrets.token_hex(24)}"
server_proof = hmac.new(self.token.encode(), f"rea-cutter-server-v2:{self.session_id}:{client_nonce}:{server_nonce}".encode(), hashlib.sha256).hexdigest()
self._send(connection, {"session_id": self.session_id, "nonce": server_nonce, "proof": server_proof})
authenticated_frame = self._receive_frame(
connection, _authenticated_frame_timeout_seconds()
)
authenticated_frame = self._receive_frame(connection)
request_bytes = authenticated_frame.get("request") if isinstance(authenticated_frame, dict) else None
presented_proof = authenticated_frame.get("proof") if isinstance(authenticated_frame, dict) else None
if not isinstance(request_bytes, str) or not isinstance(presented_proof, str):
expected_proof = hmac.new(self.token.encode(), f"rea-cutter-client-v2:{self.session_id}:{client_nonce}:{server_nonce}:{request_bytes}".encode(), hashlib.sha256).hexdigest()
if not isinstance(request_bytes, str) or not isinstance(authenticated_frame.get("proof"), str) or not hmac.compare_digest(authenticated_frame["proof"], expected_proof):
self._send(connection, {"ok": False, "error": "authentication-failed"})
return
# Update the prefix and the request separately. One joined
# preimage would retain another copy of a maximum-sized command.
expected_proof = hmac.new(self.token.encode(), digestmod=hashlib.sha256)
expected_proof.update(
f"rea-cutter-client-v2:{self.session_id}:{client_nonce}:{server_nonce}:".encode()
)
expected_proof.update(request_bytes.encode())
if not hmac.compare_digest(presented_proof, expected_proof.hexdigest()):
self._send(connection, {"ok": False, "error": "authentication-failed"})
return
continue
request = json.loads(request_bytes)
if not isinstance(request, dict) or request.get("session_id") != self.session_id:
self._send(connection, {"ok": False, "error": "authentication-failed"})
return
continue
authenticated = True
with self._command_lock:
connection.settimeout(SOCKET_TIMEOUT_SECONDS)
if self.poisoned and request.get("kind") not in ("status", "ack"):
self._send_authenticated(connection, {
"ok": False,
"error": "session-poisoned-after-ui-timeout",
"execution_state": "not_started",
}, client_nonce, server_nonce)
return
if (
request.get("kind") == "command"
and not self.dispatcher.can_accept_command()
):
self._send_authenticated(connection, {
"ok": False,
"error": "completion-receipt-capacity-exceeded",
"execution_state": "not_started",
}, client_nonce, server_nonce)
return
request["session_id"] = self.session_id
event = threading.Event()
cancelled = threading.Event()
reply: dict[str, Any] = {}
request["_event"] = event
request["_cancelled"] = cancelled
self.dispatcher.dispatch.emit(request, reply)
if not event.wait(SOCKET_TIMEOUT_SECONDS):
cancelled.set()
self.poisoned = True
self._send_authenticated(connection, {
"ok": False,
"error": "ui-thread-request-timeout",
"execution_state": "unknown",
"message": "The UI operation may still complete after this timeout; restart Cutter before issuing further commands.",
}, client_nonce, server_nonce)
else:
if request.get("kind") == "status" and reply.get("ok") is True:
reply["completed_connection"] = self._completion_watermark()
self._send_authenticated(connection, reply, client_nonce, server_nonce)
connection.settimeout(SOCKET_TIMEOUT_SECONDS)
if self.poisoned and request.get("kind") not in ("status", "ack"):
self._send_authenticated(connection, {
"ok": False,
"error": "session-poisoned-after-ui-timeout",
"execution_state": "not_started",
}, client_nonce, server_nonce)
continue
if (
request.get("kind") == "command"
and not self.dispatcher.can_accept_command()
):
self._send_authenticated(connection, {
"ok": False,
"error": "completion-receipt-capacity-exceeded",
"execution_state": "not_started",
}, client_nonce, server_nonce)
continue
request["session_id"] = self.session_id
event = threading.Event()
cancelled = threading.Event()
reply: dict[str, Any] = {}
request["_event"] = event
request["_cancelled"] = cancelled
self.dispatcher.dispatch.emit(request, reply)
if not event.wait(SOCKET_TIMEOUT_SECONDS):
cancelled.set()
self.poisoned = True
self._send_authenticated(connection, {
"ok": False,
"error": "ui-thread-request-timeout",
"execution_state": "unknown",
"message": "The UI operation may still complete after this timeout; restart Cutter before issuing further commands.",
}, client_nonce, server_nonce)
else:
if request.get("kind") == "status" and reply.get("ok") is True:
reply["completed_connection"] = f"{connection_ordinal - 1:016x}"
self._send_authenticated(connection, reply, client_nonce, server_nonce)
except socket.timeout:
if authenticated:
self._send_authenticated(connection, {
@@ -820,217 +613,77 @@ class _BridgeServer(threading.Thread):
}, client_nonce, server_nonce)
except Exception as error:
self._send(connection, {"ok": False, "error": str(error)})
finally:
self._note_connection_finished(connection_ordinal)
with self._connections_lock:
self._connections.discard(connection)
def _note_connection_finished(self, ordinal: int) -> None:
"""Record one accepted connection as terminated.
The watermark is the longest prefix 1..W that has terminated. A later
status cannot report an earlier connection that is still reading or
still waiting to run.
"""
with self._connections_lock:
if ordinal <= self._finished_through or ordinal in self._finished_ordinals:
return
self._finished_ordinals.add(ordinal)
while self._finished_through + 1 in self._finished_ordinals:
self._finished_through += 1
self._finished_ordinals.discard(self._finished_through)
def _completion_watermark(self) -> str:
with self._connections_lock:
return f"{self._finished_through:016x}"
self.server.close()
try:
self.descriptor.unlink(missing_ok=True)
except OSError:
pass
@staticmethod
def _receive_frame(
connection: socket.socket,
timeout_seconds: float = AUTH_FRAME_TIMEOUT_SECONDS,
) -> dict[str, Any]:
def _receive_frame(connection: socket.socket) -> dict[str, Any]:
data = bytearray()
deadline = time.monotonic() + timeout_seconds
newline = -1
while newline < 0:
if len(data) > MAX_FRAME_BYTES:
raise _frame_limit_exceeded()
deadline = time.monotonic() + AUTH_FRAME_TIMEOUT_SECONDS
while b"\n" not in data:
if len(data) >= MAX_FRAME_BYTES:
raise ValueError("frame-limit-exceeded")
remaining = deadline - time.monotonic()
if remaining <= 0:
raise socket.timeout("authentication frame timed out")
connection.settimeout(remaining)
# MAX_FRAME_BYTES limits the JSON payload; the terminating LF is
# framing and may be the one byte beyond that payload budget.
part = connection.recv(min(65536, MAX_FRAME_BYTES + 1 - len(data)))
if not part:
raise ValueError("incomplete authentication frame")
scanned = len(data)
data.extend(part)
# The delimiter is one byte, so it cannot straddle a chunk.
# Search only the bytes just received.
newline = data.find(b"\n", scanned)
if newline > MAX_FRAME_BYTES:
raise _frame_limit_exceeded()
if newline != len(data) - 1:
line, remainder = bytes(data).split(b"\n", 1)
if len(line) > MAX_FRAME_BYTES or remainder:
raise ValueError("invalid framed request")
value = json.loads(bytes(data[:newline]))
value = json.loads(line)
return value if isinstance(value, dict) else {}
def _send_authenticated(self, connection: socket.socket, value: dict[str, Any], client_nonce: str, server_nonce: str) -> None:
if self._deliver_authenticated(connection, value, client_nonce, server_nonce):
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
if encoded is None:
value = self._truncated_response(value)
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
if encoded is None:
# This fixed-size fallback is checked through the same signed framing path.
value = {
"ok": False,
"error": "response-limit-exceeded",
"execution_state": value.get("execution_state", "unknown"),
"output": None,
"output_truncated": True,
"message": "The result exceeded the bridge frame limit; command completion state is preserved.",
}
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
if encoded is None:
return
# Keep the recovery payload. Include the observed path when it
# fits. An explicit null path means unknown, so the client does
# not pair this generation with the descriptor's older path.
bounded = self._bounded_status_response(value)
if bounded is not None:
observed_path = value.get("current_file")
if isinstance(observed_path, str) and observed_path:
with_path = dict(bounded)
with_path["current_file"] = observed_path
with_path.pop("message", None)
if self._deliver_authenticated(
connection, with_path, client_nonce, server_nonce
):
return
if self._deliver_authenticated(
connection, bounded, client_nonce, server_nonce
):
return
command = self._bounded_command_response(value)
if command is not None:
observed_path = value.get("current_file")
if isinstance(observed_path, str) and observed_path:
with_path = dict(command)
with_path["current_file"] = observed_path
with_path.pop("message", None)
if self._deliver_authenticated(
connection, with_path, client_nonce, server_nonce
):
return
if self._deliver_authenticated(
connection, command, client_nonce, server_nonce
):
return
value = self._truncated_response(value)
if self._deliver_authenticated(connection, value, client_nonce, server_nonce):
return
# This fixed-size fallback is checked through the same signed framing path.
self._deliver_authenticated(connection, {
"ok": False,
"error": "response-limit-exceeded",
"execution_state": value.get("execution_state", "unknown"),
"output": None,
"output_truncated": True,
"message": "The result exceeded the bridge frame limit; command completion state is preserved.",
}, client_nonce, server_nonce)
def _deliver_authenticated(
self, connection: socket.socket, value: dict[str, Any], client_nonce: str, server_nonce: str
) -> bool:
proof = self._authenticated_proof(value, client_nonce, server_nonce)
if proof is None:
return False
try:
self._write_authenticated_frame(connection, value, proof)
connection.sendall(encoded)
except OSError:
# A disconnected caller must not fall through to another payload.
pass
return True
def _authenticated_proof(
self, value: dict[str, Any], client_nonce: str, server_nonce: str
) -> str | None:
def _encode_authenticated_response(self, value: dict[str, Any], client_nonce: str, server_nonce: str) -> bytes | None:
if not _within_response_limit(value):
return None
# The payload budget excludes the terminating LF. Fragments are
# measured and signed here, then discarded. The write pass encodes
# them again so a maximum-sized reply is not held in several copies.
frame_size = _AUTHENTICATED_FRAME_OVERHEAD
proof = hmac.new(self.token.encode(), digestmod=hashlib.sha256)
proof.update(
f"rea-cutter-response-v2:{self.session_id}:{client_nonce}:{server_nonce}:".encode()
)
for fragment in _response_fragments(value):
frame_size += len(_escape_json_fragment(fragment))
prefix = b'{"response":"'
suffix = b'","proof":"' + (b"0" * 64) + b'"}\n'
frame_size = len(prefix) + len(suffix)
response_parts: list[bytes] = []
escaped_parts: list[bytes] = []
for fragment in json.JSONEncoder(ensure_ascii=False, separators=(",", ":")).iterencode(value):
raw = fragment.encode("utf-8")
escaped = json.dumps(fragment, ensure_ascii=False)[1:-1].encode("utf-8")
frame_size += len(escaped)
if frame_size > MAX_FRAME_BYTES:
return None
proof.update(fragment.encode("utf-8"))
return proof.hexdigest()
def _encode_authenticated_response(
self, value: dict[str, Any], client_nonce: str, server_nonce: str
) -> bytes | None:
"""Return one exact frame. The socket path writes the same pieces."""
proof = self._authenticated_proof(value, client_nonce, server_nonce)
if proof is None:
return None
return b"".join(self._iter_authenticated_frame(value, proof))
def _iter_authenticated_frame(self, value: dict[str, Any], proof: str):
yield _AUTHENTICATED_PREFIX
for fragment in _response_fragments(value):
yield _escape_json_fragment(fragment)
yield _AUTHENTICATED_PROOF_MARK + proof.encode("ascii") + b'"}\n'
def _write_authenticated_frame(
self, connection: socket.socket, value: dict[str, Any], proof: str
) -> None:
pending = bytearray()
for chunk in self._iter_authenticated_frame(value, proof):
pending.extend(chunk)
if len(pending) >= _AUTHENTICATED_WRITE_BUFFER_BYTES:
connection.sendall(pending)
pending.clear()
if pending:
connection.sendall(pending)
@staticmethod
def _bounded_status_response(value: dict[str, Any]) -> dict[str, Any] | None:
if "completed_commands" not in value and "completed_connection" not in value:
return None
bounded: dict[str, Any] = {
"ok": value.get("ok") is True,
"session_id": value.get("session_id"),
"document_generation": value.get("document_generation"),
"identity_status": value.get("identity_status", "partial"),
"completed_commands": value.get("completed_commands"),
"current_file": None,
"message": (
"current_file is unknown because the status frame exceeded "
"REA_BRIDGE_MAX_REQUEST_BYTES. Do not reuse an older descriptor "
"path with this document_generation."
),
}
if "completed_connection" in value:
bounded["completed_connection"] = value["completed_connection"]
return bounded
@staticmethod
def _bounded_command_response(value: dict[str, Any]) -> dict[str, Any] | None:
if "execution_state" not in value and "document_generation" not in value:
return None
bounded: dict[str, Any] = {
"ok": value.get("ok") is True,
"output": None,
"execution_state": value.get("execution_state", "unknown"),
"output_truncated": True,
"current_file": None,
"message": (
"current_file is unknown because the result frame exceeded "
"REA_BRIDGE_MAX_REQUEST_BYTES. The command completion state and "
"document_generation are preserved. Do not reuse an older "
"descriptor path with this document_generation."
),
}
generation = value.get("document_generation")
if isinstance(generation, int) and not isinstance(generation, bool):
bounded["document_generation"] = generation
if value.get("identity_status") == "partial":
bounded["identity_status"] = "partial"
if value.get("ok") is not True and isinstance(value.get("error"), str):
bounded["error"] = value["error"]
return bounded
response_parts.append(raw)
escaped_parts.append(escaped)
proof = hmac.new(self.token.encode(), digestmod=hashlib.sha256)
proof.update(f"rea-cutter-response-v2:{self.session_id}:{client_nonce}:{server_nonce}:".encode())
for part in response_parts:
proof.update(part)
return prefix + b"".join(escaped_parts) + b'","proof":"' + proof.hexdigest().encode() + b'"}\n'
@staticmethod
def _truncated_response(value: dict[str, Any]) -> dict[str, Any]:
@@ -1088,17 +741,6 @@ class _BridgeServer(threading.Thread):
self.server.close()
except OSError:
pass
with self._connections_lock:
connections = list(self._connections)
for connection in connections:
try:
connection.shutdown(socket.SHUT_RDWR)
except OSError:
pass
try:
connection.close()
except OSError:
pass
class ReaCutterPlugin(cutter.CutterPlugin):
@@ -1132,128 +774,6 @@ def create_cutter_plugin() -> ReaCutterPlugin:
return ReaCutterPlugin()
def _json_object_key(key: Any) -> str:
"""Match CPython's JSON object-key text before quoting."""
if isinstance(key, str):
return key
if key is True:
return "true"
if key is False:
return "false"
if key is None:
return "null"
if isinstance(key, int):
return str(key)
if isinstance(key, float):
return json.dumps(key, separators=(",", ":"))
raise TypeError("keys must be str, int, float, bool or None")
def _response_fragments(value: Any):
"""Yield compact JSON pieces whose concatenation is the encoded value.
String contents are sliced before encoding so a large output never becomes
one frame-sized fragment. Each piece is at most ``_JSON_CHUNK_CHARS``
characters, or the six-byte ``\\u00XX`` expansion of one such slice.
"""
if isinstance(value, str):
yield '"'
for start in range(0, len(value), _JSON_CHUNK_CHARS):
yield json.dumps(
value[start:start + _JSON_CHUNK_CHARS], ensure_ascii=False
)[1:-1]
yield '"'
return
if isinstance(value, dict):
yield "{"
first = True
for key, item in value.items():
if not first:
yield ","
first = False
yield from _response_fragments(_json_object_key(key))
yield ":"
yield from _response_fragments(item)
yield "}"
return
if isinstance(value, (list, tuple)):
yield from _list_fragments(value)
return
token = _scalar_token(value)
if token is None:
raise TypeError("response value is not JSON")
yield token
def _scalar_token(value: Any) -> str | None:
"""Compact JSON for one scalar. None means the value needs structure."""
if type(value) is int:
return str(value)
if value is None:
return "null"
if value is True:
return "true"
if value is False:
return "false"
if isinstance(value, float):
return json.dumps(value, separators=(",", ":"))
return None
def _list_fragments(value: list[Any] | tuple[Any, ...]):
"""Encode a list, batching scalars into bounded pieces.
One ``json.dumps`` per number makes a dense array miss the client
deadline. A batch is at most ``_JSON_CHUNK_CHARS`` of compact JSON.
"""
yield "["
first = True
batch: list[str] = []
batch_len = 0
def flush_batch() -> str | None:
nonlocal first, batch, batch_len
if not batch:
return None
piece = ",".join(batch)
batch = []
batch_len = 0
if first:
first = False
return piece
return "," + piece
for item in value:
token = _scalar_token(item)
if token is None:
emitted = flush_batch()
if emitted is not None:
yield emitted
if first:
first = False
else:
yield ","
yield from _response_fragments(item)
continue
extra = len(token) if not batch else len(token) + 1
if batch and batch_len + extra > _JSON_CHUNK_CHARS:
emitted = flush_batch()
if emitted is not None:
yield emitted
extra = len(token)
batch.append(token)
batch_len += extra
emitted = flush_batch()
if emitted is not None:
yield emitted
yield "]"
def _escape_json_fragment(fragment: str) -> bytes:
"""Escape one raw JSON piece for the outer response string."""
return json.dumps(fragment, ensure_ascii=False)[1:-1].encode("utf-8")
def _within_response_limit(value: Any) -> bool:
pending = [value]
seen: set[int] = set()
@@ -1262,37 +782,20 @@ def _within_response_limit(value: Any) -> bool:
current = pending.pop()
if isinstance(current, str):
size += 2
if len(current) > MAX_FRAME_BYTES - size:
return False
if current.isascii():
size += len(current)
size += current.count('"') + current.count("\\")
size += sum(5 for character in current if ord(character) < 0x20)
else:
# Count UTF-8 bytes without allocating a copy of the whole output.
try:
size += sum(
len(current[start:start + 65536].encode("utf-8"))
for start in range(0, len(current), 65536)
)
except UnicodeEncodeError:
# Unencodable output also needs the signed omission fallback.
return False
size += current.count('"') + current.count("\\")
# A short escape is two JSON bytes and was already counted as one.
# Other controls expand to a six-byte \\u00XX sequence.
size += sum(
1 if ord(character) in (0x08, 0x09, 0x0A, 0x0C, 0x0D) else 5
for character in current
if ord(character) < 0x20
)
size += 4 * len(current)
size += current.count('"') + current.count("\\")
size += sum(5 for character in current if ord(character) < 0x20)
elif isinstance(current, dict):
identity = id(current)
if identity in seen:
return False
seen.add(identity)
count = len(current)
# Braces, one colon per entry, and commas between entries.
# Keys and values are counted when they are visited.
size += 2 + count + max(0, count - 1)
size += 2 + 4 * len(current)
if size > MAX_FRAME_BYTES:
return False
pending.extend(current.keys())
@@ -1302,29 +805,12 @@ def _within_response_limit(value: Any) -> bool:
if identity in seen:
return False
seen.add(identity)
count = len(current)
# Brackets and commas between items. Each item is counted below.
size += 2 + max(0, count - 1)
size += 2 + 2 * len(current)
if size > MAX_FRAME_BYTES:
return False
pending.extend(current)
elif current is None:
size += 4
elif current is True:
size += 4
elif current is False:
size += 5
elif isinstance(current, int):
size += len(str(current))
elif isinstance(current, float):
size += len(json.dumps(current, separators=(",", ":")))
else:
try:
size += len(
json.dumps(current, ensure_ascii=False, separators=(",", ":"))
)
except (TypeError, ValueError):
return False
size += 16
if size > MAX_FRAME_BYTES:
return False
return True
+12 -65
View File
@@ -1,91 +1,38 @@
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
import ghidra.app.plugin.core.analysis.AutoAnalysisManager;
import ghidra.app.services.Analyzer;
import ghidra.app.util.headless.HeadlessScript;
import ghidra.program.model.listing.Program;
import ghidra.util.classfinder.ClassSearcher;
/**
* Disable the named auto-analyzers before default auto-analysis.
*
* REA_GHIDRA_DISABLED_ANALYZERS is process-wide, but Ghidra registers only the
* installed analyzers whose canAnalyze(program) accepts this program. A name an
* installed analyzer provides without being registered here is not applicable
* and is recorded instead of rejected; a name no installed analyzer provides
* aborts the import. The outcome is stored as a program property that the
* bridge reports during its handshake.
*/
/** Disable the named auto-analyzers before default auto-analysis. */
public final class ReaGhidraAnalysisOptions extends HeadlessScript {
static final String REPORT_OPTION = "REA disabled analyzers";
private static final String SETTING = "REA_GHIDRA_DISABLED_ANALYZERS";
@Override
public void run() throws Exception {
String[] arguments = getScriptArgs();
if (arguments.length != 1 || arguments[0].isBlank()) {
throw new IllegalArgumentException(
"ReaGhidraAnalysisOptions requires one comma-separated " + SETTING + " argument");
"ReaGhidraAnalysisOptions requires one comma-separated analyzer-name argument");
}
String[] names = arguments[0].split(",");
boolean[] registered = new boolean[names.length];
AutoAnalysisManager analysis = AutoAnalysisManager.getAnalysisManager(currentProgram);
Map<String, String> options = getCurrentAnalysisOptionsAndValues(currentProgram);
Set<String> installed = null;
for (int index = 0; index < names.length; index++) {
for (String requestedName : names) {
monitor.checkCancelled();
String name = names[index].trim();
names[index] = name;
if (analysis.getAnalyzer(name) != null) {
if (!options.containsKey(name)) {
throw new IllegalStateException(SETTING + " names " + name +
", which Ghidra registered for this program without an enablement option");
}
registered[index] = true;
continue;
}
if (installed == null) installed = installedAnalyzerNames();
if (!installed.contains(name)) {
throw new IllegalArgumentException(SETTING + " names " + name +
", which is not an analyzer in this Ghidra installation");
String name = requestedName.trim();
if (analysis.getAnalyzer(name) == null || !options.containsKey(name)) {
throw new IllegalArgumentException("Unknown Ghidra analyzer for this program: " + name);
}
}
for (int index = 0; index < names.length; index++) {
for (String requestedName : names) {
monitor.checkCancelled();
if (registered[index]) setAnalysisOption(currentProgram, names[index], "false");
setAnalysisOption(currentProgram, requestedName.trim(), "false");
}
options = getCurrentAnalysisOptionsAndValues(currentProgram);
JsonArray analyzers = new JsonArray();
for (int index = 0; index < names.length; index++) {
for (String requestedName : names) {
monitor.checkCancelled();
String name = names[index];
if (registered[index] && !"false".equals(options.get(name))) {
throw new IllegalStateException(SETTING + " names " + name +
", which remained enabled for this program");
String name = requestedName.trim();
if (!"false".equals(options.get(name))) {
throw new IllegalStateException("Ghidra analyzer remained enabled: " + name);
}
JsonObject analyzer = new JsonObject();
analyzer.addProperty("name", name);
analyzer.addProperty("status", registered[index] ? "disabled" : "not_applicable");
analyzers.add(analyzer);
}
JsonObject report = new JsonObject();
report.addProperty("format", "rea-ghidra-disabled-analyzers-v1");
report.add("analyzers", analyzers);
currentProgram.getOptions(Program.PROGRAM_INFO).setString(REPORT_OPTION, report.toString());
println("REA disabled analyzers: " + report);
}
/** The same analyzer universe AutoAnalysisManager filters by canAnalyze(program). */
private static Set<String> installedAnalyzerNames() {
Set<String> names = new HashSet<>();
for (Analyzer analyzer : ClassSearcher.getInstances(Analyzer.class)) {
names.add(analyzer.getName());
}
return names;
}
}
File diff suppressed because it is too large. Load diff
-193
View File
@@ -1,193 +0,0 @@
import ghidra.app.script.GhidraScript;
import ghidra.program.database.mem.FileBytes;
import ghidra.program.model.address.Address;
import ghidra.program.model.address.AddressSpace;
import ghidra.program.model.listing.Program;
import ghidra.program.model.mem.Memory;
import ghidra.program.model.mem.MemoryBlock;
import ghidra.program.model.mem.MemoryBlockSourceInfo;
import java.nio.file.Files;
import java.nio.file.LinkOption;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
/**
* Rebuild a GameCube/Wii DOL memory map from the validated load ranges REA's TypeScript
* admission wrote. BinaryLoader maps the whole file once at address 0; this
* script replaces that block with text, data, and BSS blocks at their load addresses.
*/
public final class ReaGhidraPrepareDol extends GhidraScript {
// Canonical v2 header, 18 section rows, 12 gap rows and entry, using uint32 hex.
private static final int MAX_PLAN_BYTES = 13 + 18 * 49 + 12 * 30 + 17;
private static final class Section {
final String kind;
final int index;
final long fileOffset;
final long address;
final long size;
Section(String kind, int index, long fileOffset, long address, long size) {
this.kind = kind;
this.index = index;
this.fileOffset = fileOffset;
this.address = address;
this.size = size;
}
}
private record BssGap(long address, long size) {}
@Override
public void run() throws Exception {
String[] arguments = getScriptArgs();
if (arguments.length != 1) {
throw new IllegalArgumentException(
"GameCube DOL preparation requires exactly one section table path"
);
}
if (!currentProgram.getExecutableFormat().equals("Raw Binary") ||
!currentProgram.getLanguageID().getIdAsString().startsWith("PowerPC:BE:32:")) {
throw new IllegalArgumentException(
"GameCube DOL preparation requires a BinaryLoader import in big-endian 32-bit PowerPC"
);
}
Path tablePath = Path.of(arguments[0]);
if (!Files.isRegularFile(tablePath, LinkOption.NOFOLLOW_LINKS)) {
throw new IllegalArgumentException("GameCube DOL section table is not a regular file");
}
if (Files.size(tablePath) > MAX_PLAN_BYTES) {
throw new IllegalArgumentException("GameCube DOL plan exceeds the canonical format's byte bound");
}
List<Section> sections = new ArrayList<>();
List<BssGap> bssGaps = new ArrayList<>();
int expectedSections = -1;
int expectedGaps = -1;
long entry = -1;
for (String line : Files.readAllLines(tablePath)) {
String[] fields = line.trim().split("\\s+");
switch (fields[0]) {
case "plan" -> {
requireFields(fields, 4, line);
if (expectedSections >= 0 || !fields[1].equals("2")) {
throw new IllegalArgumentException("Duplicate or unsupported GameCube DOL plan header");
}
expectedSections = Integer.parseInt(fields[2]);
expectedGaps = Integer.parseInt(fields[3]);
// A DOL has 18 slots, including at most 11 data slots and 12 BSS gaps.
if (expectedSections < 1 || expectedSections > 18 || expectedGaps < 0 || expectedGaps > 12) {
throw new IllegalArgumentException("GameCube DOL plan counts exceed the format's slot bounds");
}
}
case "section" -> {
requireFields(fields, 6, line);
if (expectedSections < 0 || sections.size() >= expectedSections) {
throw new IllegalArgumentException("GameCube DOL section exceeds its declared count");
}
sections.add(new Section(
fields[1],
Integer.parseInt(fields[2]),
Long.decode(fields[3]),
Long.decode(fields[4]),
Long.decode(fields[5])
));
}
case "bss-gap" -> {
requireFields(fields, 3, line);
if (expectedGaps < 0 || bssGaps.size() >= expectedGaps) {
throw new IllegalArgumentException("GameCube DOL BSS gap exceeds its declared count");
}
long address = Long.decode(fields[1]);
long size = Long.decode(fields[2]);
if (address < 0 || size <= 0) {
throw new IllegalArgumentException("GameCube DOL BSS gap is not a positive range");
}
bssGaps.add(new BssGap(address, size));
}
case "entry" -> {
requireFields(fields, 2, line);
entry = Long.decode(fields[1]);
}
default -> throw new IllegalArgumentException("Unknown GameCube DOL table line: " + line);
}
}
if (expectedSections != sections.size() || expectedGaps != bssGaps.size() || entry < 0) {
throw new IllegalArgumentException("GameCube DOL section table is incomplete");
}
Memory memory = currentProgram.getMemory();
List<FileBytes> files = memory.getAllFileBytes();
if (files.size() != 1) {
throw new IllegalStateException("GameCube DOL preparation requires exactly one imported file image");
}
FileBytes fileBytes = files.get(0);
AddressSpace space = currentProgram.getAddressFactory().getDefaultAddressSpace();
// Remove the whole-file block before creating section blocks, so no two blocks claim one address.
for (MemoryBlock block : memory.getBlocks()) {
for (MemoryBlockSourceInfo source : block.getSourceInfos()) {
if (source.getFileBytes().isPresent() && source.getFileBytes().get().equals(fileBytes)) {
memory.removeBlock(block, monitor);
break;
}
}
}
for (Section section : sections) {
if (section.fileOffset + section.size > fileBytes.getSize()) {
throw new IllegalArgumentException(
"GameCube DOL " + section.kind + " " + section.index + " exceeds the imported file"
);
}
boolean text = section.kind.equals("text");
if (!text && !section.kind.equals("data")) {
throw new IllegalArgumentException("Unknown GameCube DOL section kind: " + section.kind);
}
MemoryBlock block = memory.createInitializedBlock(
"." + section.kind + section.index,
space.getAddress(section.address),
fileBytes,
section.fileOffset,
section.size,
false
);
block.setRead(true);
block.setWrite(!text);
block.setExecute(text);
}
// Apply the domain owner's exact uninitialized ranges; do not reinterpret BSS here.
int gapIndex = 0;
for (BssGap gap : bssGaps) {
MemoryBlock block = memory.createUninitializedBlock(
".bss" + gapIndex++,
space.getAddress(gap.address()),
gap.size(),
false
);
block.setRead(true);
block.setWrite(true);
block.setExecute(false);
}
Address entryAddress = space.getAddress(entry);
MemoryBlock entryBlock = memory.getBlock(entryAddress);
if (entryBlock == null || !entryBlock.isExecute()) {
throw new IllegalArgumentException("GameCube DOL entry must lie in an executable section");
}
addEntryPoint(entryAddress);
if (!disassemble(entryAddress) || createFunction(entryAddress, "__start") == null) {
throw new IllegalStateException("GameCube DOL entry cannot be decoded as a function");
}
currentProgram.getOptions(Program.PROGRAM_INFO).setBoolean("REA raw binary prepared", true);
currentProgram.getOptions(Program.PROGRAM_INFO).setBoolean("REA GameCube DOL prepared", true);
}
private static void requireFields(String[] fields, int count, String line) {
if (fields.length != count) {
throw new IllegalArgumentException("Malformed GameCube DOL table line: " + line);
}
}
}
@@ -1,283 +0,0 @@
import ghidra.app.script.GhidraScript;
import ghidra.program.database.SpecExtension;
import ghidra.program.model.data.DataType;
import ghidra.program.model.data.DoubleDataType;
import ghidra.program.model.data.Float16DataType;
import ghidra.program.model.data.FloatDataType;
import ghidra.program.model.data.FunctionDefinition;
import ghidra.program.model.data.IntegerDataType;
import ghidra.program.model.data.StructureDataType;
import ghidra.program.model.lang.CompilerSpec;
import ghidra.program.model.lang.Language;
import ghidra.program.model.lang.PrototypeModel;
import ghidra.program.model.lang.Register;
import ghidra.program.model.listing.Function;
import ghidra.program.model.listing.Parameter;
import ghidra.program.model.listing.Program;
import ghidra.program.model.listing.VariableStorage;
import ghidra.program.model.pcode.Varnode;
import ghidra.program.model.pcode.XmlEncode;
import java.io.StringReader;
import java.io.StringWriter;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.regex.Pattern;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.InputSource;
/**
* Apply the committed RISC-V calling-convention policy after analysis. Ghidra's generic RV32
* compiler specification models ilp32d. Each policy derives a program-specific prototype from the
* program's own default prototype, as its {@link Policy} row and docs/providers/ghidra-riscv.md state. The
* script installs the prototype as a specification extension and assigns it, with dynamic
* storage, to every non-thunk function, external function and function definition.
*/
public final class ReaGhidraRiscvCallingConvention extends GhidraScript {
static final String REPORT_KEY = "REA RISC-V calling convention";
/** RISC-V floating-point registers by ABI name: ft0-ft11, fs0-fs11 and fa0-fa7. */
private static final Pattern FLOAT_REGISTER = Pattern.compile("f[ast]\\d+");
/**
* What a policy changes in the default ilp32d prototype. Every policy also passes values of 9
* bytes or more, such as {@code long double}, by reference.
*
* @param floatEntryBytes the largest value a floating-point register may pass or return; 0
* removes those entries and every rule that assigns floating-point storage
* @param preservesFloatRegisters whether fs0-fs11 stay preserved across calls
*/
private record Policy(
String id, String prototype, int floatEntryBytes, boolean preservesFloatRegisters) {}
private static final List<Policy> POLICIES =
List.of(new Policy("ilp32-prototype-v1", "__rea_ilp32", 0, false));
@Override
public void run() throws Exception {
String[] arguments = getScriptArgs();
Policy policy =
POLICIES.stream()
.filter(candidate -> arguments.length == 1 && candidate.id().equals(arguments[0]))
.findFirst()
.orElseThrow(
() ->
new IllegalArgumentException(
"REA RISC-V calling convention requires one of the policies "
+ POLICIES.stream().map(Policy::id).toList()));
Language language = currentProgram.getLanguage();
if (!language.getProcessor().toString().equals("RISCV")
|| currentProgram.getDefaultPointerSize() != 4)
throw new IllegalStateException(
"REA " + policy.id() + " requires a 32-bit RISC-V program, not "
+ language.getLanguageID());
CompilerSpec spec = currentProgram.getCompilerSpec();
new SpecExtension(currentProgram)
.addReplaceCompilerSpecExtension(
derive(spec.getDefaultCallingConvention(), spec, language, policy), monitor);
PrototypeModel model =
currentProgram.getCompilerSpec().getCallingConvention(policy.prototype());
if (model == null)
throw new IllegalStateException("REA " + policy.prototype() + " was not installed");
selfCheck(model, policy, language);
int functions = 0;
List<Function> callables = new ArrayList<>();
currentProgram.getFunctionManager().getFunctions(true).forEach(callables::add);
currentProgram.getFunctionManager().getExternalFunctions().forEach(callables::add);
for (Function function : callables) {
// A thunk's signature is its target's; the target is in this list.
if (function.isThunk()) continue;
if (function.hasCustomVariableStorage()) function.setCustomVariableStorage(false);
function.setCallingConvention(policy.prototype());
if (policy.floatEntryBytes() == 0) {
List<VariableStorage> storage = new ArrayList<>();
storage.add(function.getReturn().getVariableStorage());
for (Parameter parameter : function.getParameters())
storage.add(parameter.getVariableStorage());
// An untyped function has no assigned storage yet; the decompiler applies the
// prototype when it recovers one.
requireIntegerStorage(
storage.toArray(VariableStorage[]::new), function.getName(true), true);
}
functions += 1;
}
int definitions = 0;
Iterator<DataType> types = currentProgram.getDataTypeManager().getAllDataTypes();
while (types.hasNext()) {
if (!(types.next() instanceof FunctionDefinition definition)) continue;
definition.setCallingConvention(policy.prototype());
definitions += 1;
}
// Written last: any failure above leaves no report, and REA refuses the session.
currentProgram.getOptions(Program.PROGRAM_INFO).setString(
REPORT_KEY,
"{\"policy\":\"" + policy.id() + "\",\"functions\":" + functions
+ ",\"definitions\":" + definitions + "}");
println(
"REA RISC-V " + policy.prototype() + ": functions=" + functions
+ " definitions=" + definitions);
}
/** The default ilp32d prototype changed as the policy states. */
private static String derive(
PrototypeModel base, CompilerSpec spec, Language language, Policy policy) throws Exception {
XmlEncode encoder = new XmlEncode();
base.encode(encoder, spec.getPcodeInjectLibrary());
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
Document document =
factory.newDocumentBuilder().parse(new InputSource(new StringReader(encoder.toString())));
Element root = document.getDocumentElement();
if (!root.getTagName().equals("prototype"))
throw new IllegalStateException("Unexpected default prototype encoding " + root.getTagName());
root.setAttribute("name", policy.prototype());
List<Element> floatEntries = new ArrayList<>();
NodeList entries = root.getElementsByTagName("pentry");
for (int index = 0; index < entries.getLength(); index += 1) {
Element entry = (Element) entries.item(index);
if (entry.getAttribute("storage").equals("float")) floatEntries.add(entry);
}
if (floatEntries.isEmpty())
throw new IllegalStateException("Default prototype has no floating-point entries");
if (policy.floatEntryBytes() == 0) {
List<Node> removed = new ArrayList<>(floatEntries);
NodeList rules = root.getElementsByTagName("rule");
for (int index = 0; index < rules.getLength(); index += 1) {
Element rule = (Element) rules.item(index);
if (usesFloatStorage(rule)) removed.add(rule);
}
if (removed.size() == floatEntries.size())
throw new IllegalStateException("Default prototype has no floating-point rules to remove");
for (Node node : removed) node.getParentNode().removeChild(node);
} else
for (Element entry : floatEntries)
entry.setAttribute("maxsize", Integer.toString(policy.floatEntryBytes()));
// Values over 2 x XLEN go by reference. Returns need no rule: a value that the return
// registers cannot hold already becomes a caller buffer passed in a0.
NodeList inputs = root.getElementsByTagName("input");
if (inputs.getLength() != 1)
throw new IllegalStateException("Default prototype has no single input list");
Element input = (Element) inputs.item(0);
Element last = null;
for (Node child = input.getFirstChild(); child != null; child = child.getNextSibling())
if (child instanceof Element element && element.getTagName().equals("rule")) last = element;
if (last == null || !joinsAnyValue(last))
throw new IllegalStateException("Default prototype's input list does not end with a join");
Element byReference = document.createElement("rule");
Element datatype = document.createElement("datatype");
datatype.setAttribute("name", "any");
datatype.setAttribute("minsize", "9");
byReference.appendChild(datatype);
byReference.appendChild(document.createElement("convert_to_ptr"));
input.insertBefore(byReference, last);
if (!policy.preservesFloatRegisters()) {
NodeList killedLists = root.getElementsByTagName("killedbycall");
Element killed = killedLists.getLength() == 0 ? null : (Element) killedLists.item(0);
if (killed == null) {
killed = document.createElement("killedbycall");
root.appendChild(killed);
}
List<Element> moved = new ArrayList<>();
NodeList preservedLists = root.getElementsByTagName("unaffected");
for (int list = 0; list < preservedLists.getLength(); list += 1) {
NodeList preserved = preservedLists.item(list).getChildNodes();
for (int index = 0; index < preserved.getLength(); index += 1) {
if (preserved.item(index) instanceof Element varnode
&& isFloatRegister(varnode, language))
moved.add(varnode);
}
}
for (Element varnode : moved) {
varnode.getParentNode().removeChild(varnode);
killed.appendChild(varnode);
}
}
StringWriter xml = new StringWriter();
var transformer = TransformerFactory.newInstance().newTransformer();
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
transformer.transform(new DOMSource(document), new StreamResult(xml));
return xml.toString();
}
/** The installed prototype places each class of value as its policy states. */
private void selfCheck(PrototypeModel model, Policy policy, Language language) {
StructureDataType pair = new StructureDataType("rea_float_pair", 0);
pair.add(FloatDataType.dataType);
pair.add(FloatDataType.dataType);
VariableStorage[] storage =
model.getStorageLocations(
currentProgram,
new DataType[] {
DoubleDataType.dataType, FloatDataType.dataType, DoubleDataType.dataType,
IntegerDataType.dataType, pair, Float16DataType.dataType
},
false);
VariableStorage quad = storage[storage.length - 1];
if (quad.size() != currentProgram.getDefaultPointerSize() || quad.getRegister() == null)
throw new IllegalStateException(
"REA " + policy.prototype() + " does not pass a 16-byte value by reference: " + quad);
if (policy.floatEntryBytes() == 0)
requireIntegerStorage(storage, policy.prototype() + " self-check", false);
if (policy.preservesFloatRegisters()) return;
for (Varnode preserved : model.getUnaffectedList()) {
Register register = language.getRegister(preserved.getAddress(), preserved.getSize());
if (register != null && FLOAT_REGISTER.matcher(register.getName()).matches())
throw new IllegalStateException(
"REA " + policy.prototype() + " still preserves " + register.getName());
}
}
private static boolean joinsAnyValue(Element rule) {
List<Element> children = new ArrayList<>();
for (Node child = rule.getFirstChild(); child != null; child = child.getNextSibling())
if (child instanceof Element element) children.add(element);
return children.size() == 2
&& children.get(0).getTagName().equals("datatype")
&& children.get(0).getAttribute("name").equals("any")
&& children.get(1).getTagName().equals("join");
}
/** Whether a rule's action, at any depth, assigns floating-point storage. */
private static boolean usesFloatStorage(Element element) {
NodeList children = element.getChildNodes();
for (int index = 0; index < children.getLength(); index += 1) {
if (!(children.item(index) instanceof Element child)) continue;
if (child.getAttribute("storage").equals("float") || usesFloatStorage(child)) return true;
}
return false;
}
private static boolean isFloatRegister(Element varnode, Language language) {
if (!varnode.getTagName().equals("varnode")
|| !varnode.getAttribute("space").equals("register"))
return false;
Register register =
language.getRegister(
language.getAddressFactory().getRegisterSpace()
.getAddress(Long.decode(varnode.getAttribute("offset"))),
Integer.parseInt(varnode.getAttribute("size")));
return register != null && FLOAT_REGISTER.matcher(register.getName()).matches();
}
private static void requireIntegerStorage(
VariableStorage[] storage, String owner, boolean allowUnassigned) {
for (VariableStorage item : storage) {
if (item.isBadStorage() || (!allowUnassigned && item.isUnassignedStorage()))
throw new IllegalStateException("REA RISC-V storage unresolved for " + owner);
List<Register> registers = item.getRegisters();
if (registers == null) continue; // stack-only storage
for (Register register : registers) {
if (FLOAT_REGISTER.matcher(register.getName()).matches())
throw new IllegalStateException(
"REA RISC-V storage still uses " + register.getName() + " for " + owner);
}
}
}
}
+36 -333
View File
@@ -8,103 +8,12 @@ API access on this thread: moving dispatch to a worker can deadlock Hopper.
import json
import hmac
import os
import re
import socket
from typing import Any, Optional, Protocol, Sequence
BAD_ADDRESSES = (-1, 0xFFFFFFFFFFFFFFFF, None)
# The native label setter truncates NSString names at this UTF-16 extent.
HOPPER_SYMBOL_NAME_UTF16_UNITS = 1024
# Request JSON payload budget in bytes, excluding the trailing LF.
# REA_BRIDGE_MAX_REQUEST_BYTES selects it. Unset stays 16777216.
# 7464 is the authenticated Cutter frame of the largest bounded symbol name.
# Cutter's 64-receipt recovery status is smaller with a null path. A path
# that does not fit is null, not an older descriptor path.
# 536870887 is Node's 536870888-character string ceiling minus the framing LF.
# The ceiling is the encoded payload; JSON expansion is measured before it.
# Encoded JSON allowance, in bytes, shared by one instruction list, one
# native-caller list, and one caller-scan page. This release has no runtime override.
CALLER_SCAN_ALLOWANCE_BYTES = 512 * 1024
_BRIDGE_MAX_REQUEST_BYTES_SETTING = "REA_BRIDGE_MAX_REQUEST_BYTES"
DEFAULT_BRIDGE_MAX_REQUEST_BYTES = 16 * 1024 * 1024
MINIMUM_BRIDGE_MAX_REQUEST_BYTES = 7464
MAXIMUM_BRIDGE_MAX_REQUEST_BYTES = 536870887
def _bridge_max_request_bytes_constraint():
return (
"REA_BRIDGE_MAX_REQUEST_BYTES must be a positive decimal integer "
"from %d to %d bytes; the default is %d bytes (16 MiB). "
"A value below %d bytes cannot carry the largest admitted "
"symbol-name command, and a larger value is refused so the encoded "
"payload plus its framing LF stays one string under Node's "
"single-string ceiling."
% (
MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
DEFAULT_BRIDGE_MAX_REQUEST_BYTES,
MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
)
)
def _parse_bridge_max_request_bytes(raw):
"""Refuse a non-decimal or out-of-range ceiling. Never fall back."""
constraint = _bridge_max_request_bytes_constraint()
if isinstance(raw, bool):
raise RuntimeError(constraint)
if isinstance(raw, int):
value = raw
elif isinstance(raw, str):
# ASCII digits only. str.isdigit() accepts numerals such as U+0668
# that the TypeScript setting rejects.
if re.fullmatch(r"[1-9][0-9]*", raw) is None:
raise RuntimeError(constraint)
value = int(raw)
else:
raise RuntimeError(constraint)
if (
value < MINIMUM_BRIDGE_MAX_REQUEST_BYTES
or value > MAXIMUM_BRIDGE_MAX_REQUEST_BYTES
):
raise RuntimeError(constraint)
return value
def _resolve_bridge_max_request_bytes():
"""Prefer the bootstrap injection, then the process environment, then 16 MiB."""
injected = globals().get(_BRIDGE_MAX_REQUEST_BYTES_SETTING)
if injected is not None:
return _parse_bridge_max_request_bytes(injected), "setting"
raw = os.environ.get(_BRIDGE_MAX_REQUEST_BYTES_SETTING)
if raw is None:
return DEFAULT_BRIDGE_MAX_REQUEST_BYTES, "default"
return _parse_bridge_max_request_bytes(raw), "setting"
MAX_REQUEST_BYTES, BRIDGE_MAX_REQUEST_BYTES_SOURCE = (
_resolve_bridge_max_request_bytes()
)
def _bridge_max_request_bytes_recovery():
return (
"The effective REA_BRIDGE_MAX_REQUEST_BYTES value is %d bytes "
"(source %s; default %d, floor %d, maximum %d). "
"Set REA_BRIDGE_MAX_REQUEST_BYTES in the environment that starts REA "
"to a decimal integer from %d to %d, then restart REA. "
"Restart Hopper or Cutter from that same environment so the bridge "
"uses this ceiling."
% (
MAX_REQUEST_BYTES,
BRIDGE_MAX_REQUEST_BYTES_SOURCE,
DEFAULT_BRIDGE_MAX_REQUEST_BYTES,
MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
)
)
_rea_session_document = None
_search_inventory_cache = {}
_pseudocode_cache = {}
@@ -235,9 +144,6 @@ EXHAUSTIVE_ANALYSIS_METHODS = frozenset(
"procedure_assembly",
"procedure_callees",
"procedure_callers",
"rea_caller_scan_page",
"rea_caller_scan_target",
"rea_caller_base",
"procedure_info",
"procedure_pseudo_code",
"procedure_references",
@@ -330,27 +236,11 @@ def _name_matches(document, address, name):
return observed == name or (not name and observed is None)
def _bookmark_api(document, name):
method = getattr(document, name, None)
if not callable(method):
raise CapabilityUnavailableError(
"Hopper's public Python API does not expose %s" % name
)
return method
def _bookmark_name(document, address):
return _bookmark_api(document, "getBookmarkName")(address)
def _address_selection(document, value=None):
"""Own the coordinate and whether it was selected by symbol or address."""
def _address(document, value=None):
"""Keep explicit hexadecimal coordinates distinct from symbol names."""
if value is None:
return document.getCurrentAddress(), "cursor"
return document.getCurrentAddress()
_api_text(value, "Address")
if value.startswith("file-offset:"):
raise InvalidRequestError("Hopper does not support file-offset: selectors; select Ghidra for source-file coordinates or use a Hopper analysis address")
source = "address"
if value.lower().startswith("0x"):
try:
address = int(value, 16)
@@ -361,8 +251,6 @@ def _address_selection(document, value=None):
# Prefer an observed symbol, retaining bare-hex compatibility only
# when Hopper has no symbol with the supplied name.
address = document.getAddressForName(value)
if address not in BAD_ADDRESSES:
source = "symbol"
if address in BAD_ADDRESSES:
try:
address = int(value, 16)
@@ -370,11 +258,7 @@ def _address_selection(document, value=None):
raise InvalidRequestError("Unknown Hopper address or name: %s" % value) from error
if not 0 <= address < 0xFFFFFFFFFFFFFFFF:
raise InvalidRequestError("Address is outside Hopper's usable unsigned 64-bit range: %s" % value)
return address, source
def _address(document, value=None):
return _address_selection(document, value)[0]
return address
def _segment(document, address):
@@ -385,15 +269,10 @@ def _segment(document, address):
def _procedure(document, value=None):
address, source = _address_selection(document, value)
address = document.getCurrentAddress() if value is None else _address(document, value)
result = _segment(document, address).getProcedureAtAddress(address)
if result is None:
raise InvalidRequestError("No procedure exists at the requested address")
if source == "symbol" and address != result.getEntryPoint():
raise InvalidRequestError(
"Symbol %s at %s is inside procedure %s at %s; select its entry name/address or use resolve_containing_procedure for explicit containment"
% (value, _hex(address), _procedure_name(result), _hex(result.getEntryPoint()))
)
return result
@@ -406,7 +285,6 @@ def _procedure_identity(procedure):
return {
"address": _hex(procedure.getEntryPoint()),
"name": _procedure_name(procedure),
"symbol": None,
"classification": None,
"body": {
"available": False,
@@ -491,30 +369,19 @@ def _instruction_addresses(procedure):
return result
def _iter_native_calls(procedure, direction):
def _native_calls(procedure, direction):
references = procedure.getAllCallees() if direction == "outgoing" else procedure.getAllCallers()
kinds = {0: "none", 1: "unknown", 2: "direct", 3: "objective_c"}
calls = {}
unresolved = []
for reference in references:
source, target, kind = reference.fromAddress(), reference.toAddress(), reference.type()
if source in BAD_ADDRESSES:
yield None, None, None
continue
if target in BAD_ADDRESSES:
yield source, None, {"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind}
unresolved.append({"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind})
continue
yield source, target, {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
def _native_calls(procedure, direction):
calls = {}
unresolved = []
for source, target, detail in _iter_native_calls(procedure, direction):
if detail is None:
continue
if target is None:
unresolved.append(detail)
else:
calls[(source, target)] = detail
calls[(source, target)] = {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
return calls, sorted(unresolved, key=lambda item: int(item["address"], 16))
@@ -564,134 +431,8 @@ def _procedure_map(document):
return result
def _caller_target_data(procedure):
addresses = []
seen = set()
size = 0
for block in procedure.basicBlockIterator():
for address in _block_instruction_addresses(procedure, block):
if address in seen:
continue
encoded = _hex(address)
size += len(encoded) + 4
if size > CALLER_SCAN_ALLOWANCE_BYTES:
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": False}
addresses.append(encoded)
seen.add(address)
return {"target": _procedure_identity(procedure), "instructions": addresses, "instructions_complete": True}
def _caller_scan_target(document, params):
return _caller_target_data(_procedure(document, params.get("procedure")))
def _caller_base(document, params):
procedure = _procedure(document, params.get("procedure"))
callers = []
size = 0
complete = True
for item in procedure.getAllCallerProcedures():
address = _hex(item.getEntryPoint())
size += len(address) + 4
if size > CALLER_SCAN_ALLOWANCE_BYTES:
complete = False
break
callers.append(address)
return {
**_caller_target_data(procedure),
"callers": sorted(callers, key=lambda value: int(value, 16)),
"callers_complete": complete,
}
def _caller_scan_page(document, params):
"""Observe forward relationships in one bounded, stateless bridge request."""
start = params.get("start")
if isinstance(start, bool) or not isinstance(start, int) or start < 0:
raise InvalidRequestError("Caller scan start must be a non-negative integer")
segments = document.getSegmentsList()
counts = [segment.getProcedureCount() for segment in segments]
total = sum(counts)
end = min(start + 64, total)
if start > total:
raise InvalidRequestError("Caller scan start exceeds the procedure inventory")
items = []
item_bytes = 0
page_limit = CALLER_SCAN_ALLOWANCE_BYTES
oversized = False
identities = {}
containing = {}
def identity(procedure):
entry = procedure.getEntryPoint()
if entry not in identities:
identities[entry] = _procedure_identity(procedure)
return identities[entry]
def owner(address):
if address not in containing:
procedure, _ = _containing_procedure(document, address)
containing[address] = identity(procedure) if procedure is not None else None
return containing[address]
segment_start = 0
for segment, count in zip(segments, counts):
begin, stop = max(start - segment_start, 0), min(end - segment_start, count)
segment_start += count
for index in range(begin, max(begin, stop)):
candidate = segment.getProcedureAtIndex(index)
item = {"caller": identity(candidate), "callees": [], "references": []}
size = len(json.dumps(item, ensure_ascii=True).encode("utf-8")) + 1
remaining = page_limit - item_bytes
too_large = size > remaining
if not too_large:
for callee in candidate.getAllCalleeProcedures():
address = _hex(callee.getEntryPoint())
size += len(address) + 4
if size > remaining:
too_large = True
break
item["callees"].append(address)
if not too_large:
callee_entries = set(item["callees"])
for source, destination, call in _iter_native_calls(candidate, "outgoing"):
# Account for every native record, even when its target cannot
# be associated with an observed procedure.
size += 32
if size > remaining:
too_large = True
break
if destination is None:
continue
source_procedure = item["caller"] if _hex(source) == item["caller"]["address"] else owner(source)
target_address = _hex(destination)
if target_address in callee_entries:
target_entry = target_address
else:
target_procedure = owner(destination)
if target_procedure is None:
continue
target_entry = target_procedure["address"]
edge = {
"source_address": _hex(source), "target_address": _hex(destination),
"source_procedure": "caller" if source_procedure is not None and source_procedure["address"] == item["caller"]["address"] else source_procedure,
"target_procedure": target_entry,
"call": call,
}
size += len(json.dumps(edge, ensure_ascii=True).encode("utf-8")) + 1
if size > remaining:
too_large = True
break
item["references"].append(edge)
if too_large:
oversized = not items
break
items.append(item)
item_bytes += size
else:
continue
break
return {"start": start, "next": start + len(items), "total": total, "items": items, "oversized": oversized}
def _strings(document):
return {address: record["value"] for address, record in _string_inventory(document).items()}
def _string_record(segment, address, display):
@@ -780,7 +521,7 @@ def _search_inventory(document, kind):
if kind == "procedure":
values = _procedure_map(document)
elif kind == "string":
values = _string_inventory(document)
values = _strings(document)
elif kind == "name":
values = {_hex(address): value for address, value in _name_map(document).items()}
else:
@@ -807,17 +548,23 @@ def _search_results(document, kind, params):
else:
raise CapabilityUnavailableError("Regex matching requires REA's supervised worker; the native bridge only executes literal searches")
selected = []
for item in _search_inventory(document, kind):
if not matches(item[1]):
continue
selected.append(item)
if not selected:
return []
if kind == "string":
return [
{"address": address, **record}
for address, record in _search_inventory(document, kind)
if matches(record["value"])
]
records = _string_inventory(document)
return [{"address": address, **records[address]} for address, _ in selected]
return [
{"address": address, "value": value}
for address, value in _search_inventory(document, kind)
if matches(value)
]
{
"address": address,
"value": value,
}
for address, value in selected
]
def _unavailable(reason):
@@ -994,7 +741,6 @@ def _dispatch(method, params):
"version": "1.0.0",
"run_id": REA_RUN_ID,
"process_id": os.getpid(),
"max_request_bytes": MAX_REQUEST_BYTES,
}
if method in ("shutdown", "shutdown_document"):
document = _session_document()
@@ -1034,15 +780,6 @@ def _dispatch(method, params):
if method in EXHAUSTIVE_ANALYSIS_METHODS:
_api().require_analysis_complete(document, method)
if method == "rea_caller_scan_target":
return _caller_scan_target(document, params)
if method == "rea_caller_base":
return _caller_base(document, params)
if method == "rea_caller_scan_page":
return _caller_scan_page(document, params)
if method == "analyze_function":
return _analyze_function(document, params)
if method == "read_function_instructions":
@@ -1227,7 +964,6 @@ def _dispatch(method, params):
length = sum(max(0, _basic_block_end(procedure, block) - block.getStartingAddress()) for block in blocks)
return {
"name": _procedure_name(procedure),
"symbol": None,
"entrypoint": _hex(procedure.getEntryPoint()),
"basicblock_count": procedure.getBasicBlockCount(),
"length": length,
@@ -1268,41 +1004,23 @@ def _dispatch(method, params):
observed = getter(address)
return observed == comment or (comment == "" and observed is None)
if method == "list_bookmarks":
return [{"address": _hex(item), "name": _bookmark_name(document, item)} for item in _bookmark_api(document, "getBookmarks")()]
return [{"address": _hex(item), "name": document.getBookmarkName(item)} for item in document.getBookmarks()]
if method == "set_bookmark":
address = _address(document, params.get("address"))
_segment(document, address)
name = params.get("name")
if name is not None:
_api_text(name, "Bookmark name")
has_bookmark = _bookmark_api(document, "hasBookmarkAtAddress")
exists = has_bookmark(address)
if name is None and exists:
# Omission means preserve the existing bookmark and its name.
return True
_bookmark_api(document, "getBookmarkName")
if name == "" and exists:
rename = _bookmark_api(document, "renameBookmarkAtAddress")
rename(address, name)
else:
setter = _bookmark_api(document, "setBookmarkAtAddress")
setter(address, name)
if not has_bookmark(address):
return False
observed = _bookmark_name(document, address)
if name is None:
return observed is None or observed == ""
return observed == name or (name == "" and observed is None)
document.setBookmarkAtAddress(address, name)
return document.hasBookmarkAtAddress(address)
if method == "unset_bookmark":
address = _address(document, params.get("address"))
# Permit removing a legacy orphan bookmark, but do not report success
# for an unmapped address with no bookmark to remove.
has_bookmark = _bookmark_api(document, "hasBookmarkAtAddress")
remove_bookmark = _bookmark_api(document, "removeBookmarkAtAddress")
if not has_bookmark(address):
if not document.hasBookmarkAtAddress(address):
_segment(document, address)
remove_bookmark(address)
return not has_bookmark(address)
document.removeBookmarkAtAddress(address)
return not document.hasBookmarkAtAddress(address)
raise InvalidRequestError("Unknown bridge method")
@@ -1312,23 +1030,9 @@ def _serve_connection(connection):
should_stop = False
try:
while True:
# Bound the read itself. The payload budget excludes LF, so permit
# one additional byte for the delimiter while still refusing an
# oversized unterminated peer without waiting for EOF.
line = file.readline(MAX_REQUEST_BYTES + 1)
line = file.readline()
if not line:
break
if not line.endswith(b"\n"):
message = (
"Bridge request exceeds the %d-byte payload limit. %s"
% (MAX_REQUEST_BYTES, _bridge_max_request_bytes_recovery())
if len(line) > MAX_REQUEST_BYTES
else "Incomplete bridge request frame"
)
_write_message(file, {"id": 0, "error": _safe_diagnostic(
InvalidRequestError(message)
)})
break
request_id = None
authenticated = False
should_stop = False
@@ -1399,8 +1103,7 @@ def _serve_connection(connection):
break
except (ConnectionError, OSError):
# A lost client cannot confirm document closure or revoke this run.
# Keep the listener available for the owner to confirm cleanup.
should_stop = False
pass
finally:
try:
file.close()
+8 -13
View File
@@ -18,6 +18,7 @@ from mitmproxy import ctx, version
from mitmproxy.io import tnetstring
PROFILE = "12.2.3"
OUTPUT_BYTES = 96 * 1024 * 1024
CREDENTIALS = {b"authorization", b"proxy-authorization", b"cookie", b"set-cookie"}
@@ -203,7 +204,7 @@ def without_userinfo(value, base_scheme=None):
raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
def project_record(state, secrets, max_depth):
def project_record(state, secrets):
binaries, numbers, redactions = [], [], []
byte_secrets = []
for secret in secrets:
@@ -250,8 +251,8 @@ def project_record(state, secrets, max_depth):
del redactions[before[2]:]
def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
if depth > max_depth:
raise CaptureFailure("input-limit", f"Native capture exceeds max_nesting_depth ({max_depth} levels).", path)
if depth > 64:
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
if credential:
validate_omitted(value, path, depth)
redactions.append({"pointer": path, "reason": "transport-credential"})
@@ -336,12 +337,12 @@ def decode(request):
except CaptureFailure:
raise
except RecursionError:
raise CaptureFailure("unsupported", "Native parser reached Python's recursion limit at byte offset " + str(start) + ". max_nesting_depth does not raise that upstream limit.") from None
raise CaptureFailure("input-limit", "Native parser exceeded its recursion budget at byte offset " + str(start) + ".") from None
except (ValueError, IndexError, UnicodeError, EOFError):
raise CaptureFailure("format", "Malformed native record at byte offset " + str(start) + ".") from None
if not isinstance(state, dict):
raise CaptureFailure("format", "Native flow record must be a dictionary at byte offset " + str(start) + ".")
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"], request["budgets"]["depth"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
return {"decoder": {"id": "mitmproxy-native-tnetstring", "name": "REA offline native mitmproxy adapter", "version": PROFILE}, "container": {"reported": None, "numeric_literals": [], "redactions": [], "records_pointer": None}, "total_records": len(records), "records": records}
@@ -349,8 +350,6 @@ def failure_reply(error):
"""Keep producer constraints, resource exhaustion and adapter defects distinct."""
if isinstance(error, CaptureFailure):
return {"ok": False, "reason": error.reason, "message": str(error), "pointer": error.pointer}
if isinstance(error, RecursionError):
return {"ok": False, "reason": "unsupported", "message": "Native projection reached Python's recursion limit; max_nesting_depth does not raise that upstream limit.", "pointer": ""}
if isinstance(error, MemoryError):
return {"ok": False, "reason": "resource-limit", "message": "Native decoder exhausted its 768 MiB address-space budget.", "pointer": ""}
return {"ok": False, "reason": "decoder", "message": "Offline native decoder failed unexpectedly: " + type(error).__name__ + ".", "pointer": ""}
@@ -363,13 +362,9 @@ class OfflineCapture:
def running(self):
resource.setrlimit(resource.RLIMIT_AS, (768 * 1024 * 1024, 768 * 1024 * 1024))
resource.setrlimit(resource.RLIMIT_CPU, (30, 30))
resource.setrlimit(resource.RLIMIT_FSIZE, (OUTPUT_BYTES, OUTPUT_BYTES))
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
request = json.loads(Path(ctx.options.rea_request_path).read_text(encoding="utf-8"))
budgets = request["budgets"]
if any(type(budgets[key]) is not int or budgets[key] < 1 for key in ("inputBytes", "outputBytes", "depth")):
raise ValueError("Invalid admitted capture budgets")
output_bytes = budgets["outputBytes"]
resource.setrlimit(resource.RLIMIT_FSIZE, (output_bytes, output_bytes))
try:
response = {"ok": True, "value": decode(request)}
except Exception as error:
@@ -379,7 +374,7 @@ class OfflineCapture:
output = json.dumps(response, ensure_ascii=True, allow_nan=False, separators=(",", ":")).encode("ascii")
except Exception as error:
output = json.dumps(failure_reply(error), separators=(",", ":")).encode("ascii")
if len(output) > output_bytes:
if len(output) > OUTPUT_BYTES:
output = b'{"ok":false,"reason":"limit","message":"Native reply exceeds the complete-evidence output budget.","pointer":""}'
Path(request["reply_path"]).write_bytes(output)
finally:
+4 -4
View File
@@ -36,7 +36,7 @@ def location(value, scheme=None):
state = {"response": {"headers": [[b"Location", value]]}}
if scheme is not None:
state["request"] = {"scheme": scheme}
record = capture.project_record(state, [], 64)
record = capture.project_record(state, [])
sidecar = next(
field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
)
@@ -75,11 +75,11 @@ class TransportUserinfoTests(unittest.TestCase):
def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
with self.subTest(value=value):
record = capture.project_record({"request": {"path": value}}, [], 64)
record = capture.project_record({"request": {"path": value}}, [])
self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
self.assertEqual(record["binary_fields"][0]["state"], "retained")
self.assertEqual(record["redactions"], [])
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [], 64)
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
self.assertEqual(record["binary_fields"][0]["state"], "redacted")
@@ -110,7 +110,7 @@ class TransportUserinfoTests(unittest.TestCase):
"request": {"scheme": b"https"},
"response": {"headers": [[b"Location", b"https:invoice@customer"]]},
},
}, [], 64)
}, [])
self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
self.assertEqual(record["redactions"], [])
-39
View File
@@ -1,39 +0,0 @@
import Foundation
import CoreFoundation
struct AccessibilityValue {
let value: Any
let gap: String?
}
/** Preserve JSON scalars; report successful reads that JSON cannot represent faithfully. */
func captureAccessibilityValue(_ source: Any?) -> AccessibilityValue {
guard let source else { return AccessibilityValue(value: NSNull(), gap: nil) }
if let text = source as? String { return AccessibilityValue(value: text, gap: nil) }
if let number = source as? NSNumber {
if CFGetTypeID(number) == CFBooleanGetTypeID() {
return AccessibilityValue(value: number.boolValue, gap: nil)
}
let value = number.doubleValue
let encoding = String(cString: number.objCType)
let floating = encoding == "f" || encoding == "d"
if number is NSDecimalNumber || !value.isFinite || (value == 0 && value.sign == .minus) || (!floating && abs(value) > 9_007_199_254_740_991) {
return AccessibilityValue(value: NSNull(), gap: "AXValue number \(number.stringValue) cannot be represented faithfully as a JSON number")
}
return AccessibilityValue(value: number, gap: nil)
}
return AccessibilityValue(value: NSNull(), gap: "AXValue type \(type(of: source)) is not a supported JSON scalar")
}
/** Keep an unavailable action list distinct from a successful empty list. */
func captureActionNames(status: Int32, success: Int32, names: [String]?) -> (names: [String], error: Int32?) {
guard status == success else { return ([], status) }
return (names ?? [], nil)
}
/** AX absence is distinct from a failed attribute read. */
func captureAttributeValue<Value>(status: Int32, success: Int32, absent: [Int32], value: Value?) -> (value: Value?, error: Int32?) {
if status == success { return (value, nil) }
if absent.contains(status) { return (nil, nil) }
return (nil, status)
}
+23 -56
View File
@@ -18,10 +18,9 @@ struct Request: Decodable {
struct Action: Decodable { var kind: String; var path: [Int]?; var direction: String?; var text: String? }
struct BoundaryFailure: Error { var code: String; var message: String }
func fail(_ code: String, _ message: String) throws -> Never { throw BoundaryFailure(code: code, message: message) }
func attribute(_ element: AXUIElement, _ key: String) -> (value: CFTypeRef?, error: Int32?) {
func attribute(_ element: AXUIElement, _ key: String) -> CFTypeRef? {
var value: CFTypeRef?
let status = AXUIElementCopyAttributeValue(element, key as CFString, &value)
return captureAttributeValue(status: status.rawValue, success: AXError.success.rawValue, absent: [AXError.noValue.rawValue, AXError.attributeUnsupported.rawValue], value: value)
return AXUIElementCopyAttributeValue(element, key as CFString, &value) == .success ? value : nil
}
func childCount(_ element: AXUIElement) -> ChildCount {
var count = 0
@@ -35,24 +34,19 @@ func children(_ element: AXUIElement, count: Int) -> ChildBatch<AXUIElement> {
return value as? [AXUIElement]
}
}
func windowBounds(_ element: AXUIElement) -> (bounds: CGRect?, error: Int32?) {
let position = attribute(element, kAXPositionAttribute)
let size = attribute(element, kAXSizeAttribute)
if let error = position.error ?? size.error { return (nil, error) }
guard let p = position.value, let s = size.value,
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return (nil, nil) }
var point = CGPoint.zero; var dimensions = CGSize.zero
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &dimensions) else { return (nil, nil) }
return (CGRect(origin: point, size: dimensions), nil)
func text(_ element: AXUIElement, _ key: String) -> Any { (attribute(element, key) as? String) ?? NSNull() as Any }
func windowBounds(_ element: AXUIElement) -> CGRect? {
guard let p = attribute(element, kAXPositionAttribute), let s = attribute(element, kAXSizeAttribute),
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return nil }
var point = CGPoint.zero; var size = CGSize.zero
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &size) else { return nil }
return CGRect(origin: point, size: size)
}
func observe(_ request: Request) async throws -> [String: Any] {
guard let app = NSRunningApplication(processIdentifier: request.pid) else {
try fail("pid-not-application", "LaunchServices does not expose an application for the selected PID; this does not establish that the process is absent")
guard let app = NSRunningApplication(processIdentifier: request.pid), let url = app.executableURL else {
try fail("target-mismatch", "Selected PID does not expose an application executable URL")
}
guard let url = app.executableURL else {
try fail("application-executable-unavailable", "The selected application does not expose an executable URL; select a PID with an identifiable application executable")
}
guard let canonicalPointer = realpath(url.path, nil) else { try fail("executable-path-unavailable", "Running executable path cannot be resolved") }
guard let canonicalPointer = realpath(url.path, nil) else { try fail("target-mismatch", "Running executable path cannot be resolved") }
defer { free(canonicalPointer) }
let canonicalExecutable = String(cString: canonicalPointer)
guard canonicalExecutable == request.executable else {
@@ -68,10 +62,7 @@ func observe(_ request: Request) async throws -> [String: Any] {
guard SHA256.hash(data: executableBytes).map({ String(format: "%02x", $0) }).joined() == request.sha256 else {
try fail("target-mismatch", "Running executable bytes differ from the active target digest")
}
// .optionIncludingWindow omits a selected window while it is off screen, which
// is not absence: select from every window, and let capture report what it can.
guard let windows = (CGWindowListCopyWindowInfo([.optionAll], kCGNullWindowID) as? [[String: Any]])?
.filter({ ($0[kCGWindowNumber as String] as? NSNumber)?.uint32Value == request.window_id }),
guard let windows = CGWindowListCopyWindowInfo([.optionIncludingWindow], request.window_id) as? [[String: Any]],
windows.count == 1, let window = windows.first,
(window[kCGWindowOwnerPID as String] as? NSNumber)?.int32Value == request.pid,
(window[kCGWindowLayer as String] as? NSNumber)?.intValue == 0,
@@ -87,22 +78,11 @@ func observe(_ request: Request) async throws -> [String: Any] {
guard AXIsProcessTrusted() else { try fail("accessibility-denied", "Grant Accessibility to the REA host in System Settings > Privacy & Security, then retry") }
let application = AXUIElementCreateApplication(request.pid)
AXUIElementSetMessagingTimeout(application, 2)
let windowsAttribute = attribute(application, kAXWindowsAttribute)
if let error = windowsAttribute.error {
try fail("accessibility-unavailable", "Cannot read accessibility windows: AXError \(error)")
guard let axWindows = attribute(application, kAXWindowsAttribute) as? [AXUIElement] else {
try fail("accessibility-unavailable", "Selected application does not expose accessibility windows")
}
guard let axWindows = windowsAttribute.value as? [AXUIElement] else {
try fail("accessibility-unavailable", "Accessibility windows are unavailable for the selected application")
}
let candidates = axWindows.map { (element: $0, observation: windowBounds($0)) }
if let error = candidates.compactMap({ $0.observation.error }).first {
try fail("accessibility-unavailable", "Cannot read accessibility window geometry: AXError \(error)")
}
guard candidates.allSatisfy({ $0.observation.bounds != nil }) else {
try fail("accessibility-unavailable", "Accessibility window geometry is unavailable; the selected window cannot be identified")
}
let matches = candidates.filter { $0.observation.bounds == bounds }.map { $0.element }
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(candidates.map { String(describing: $0.observation.bounds) })") }
let matches = axWindows.filter { windowBounds($0) == bounds }
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(axWindows.map { String(describing: windowBounds($0)) })") }
selected = matches[0]
}
if let action = request.action, let root = selected {
@@ -113,16 +93,13 @@ func observe(_ request: Request) async throws -> [String: Any] {
guard let available = count.value else {
try fail("element-count-unavailable", "Cannot validate selected accessibility path because child count failed with AXError \(count.error ?? -1)")
}
guard index >= 0 && index < available else { try fail("element-missing", "Selected accessibility path no longer exists") }
let childStatus = AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild)
guard childStatus == .success else { try fail("accessibility-unavailable", "Cannot read selected accessibility child: AXError \(childStatus.rawValue)") }
guard let item = (selectedChild as? [AXUIElement])?.first else { try fail("accessibility-unavailable", "Selected accessibility child was not returned") }
guard index >= 0 && index < available,
AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild) == .success,
let item = (selectedChild as? [AXUIElement])?.first else { try fail("element-missing", "Selected accessibility path no longer exists") }
element = item
}
var owner: pid_t = 0
let ownerStatus = AXUIElementGetPid(element, &owner)
guard ownerStatus == .success else { try fail("accessibility-unavailable", "Cannot identify accessibility element owner: AXError \(ownerStatus.rawValue)") }
guard owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
guard AXUIElementGetPid(element, &owner) == .success && owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
let outcome: AXError
switch action.kind {
case "click": outcome = AXUIElementPerformAction(element, kAXPressAction as CFString)
@@ -139,18 +116,8 @@ func observe(_ request: Request) async throws -> [String: Any] {
if nodes.count >= request.max_nodes { truncated = true; break }
let childCountResult = childCount(element)
var actions: CFArray?
let actionStatus = AXUIElementCopyActionNames(element, &actions)
let observedActions = captureActionNames(status: actionStatus.rawValue, success: AXError.success.rawValue, names: actions as? [String])
if let error = observedActions.error { gaps.append("AX actions unavailable at path \(path): AXError \(error)"); truncated = true }
let role = attribute(element, kAXRoleAttribute)
let title = attribute(element, kAXTitleAttribute)
let value = attribute(element, kAXValueAttribute)
for (name, error) in [("role", role.error), ("title", title.error), ("value", value.error)] {
if let error { gaps.append("AX \(name) unavailable at path \(path): AXError \(error)"); truncated = true }
}
let observedValue = captureAccessibilityValue(value.value)
if let gap = observedValue.gap { gaps.append("\(gap) at path \(path)"); truncated = true }
nodes.append(["path": path, "role": (role.value as? String) ?? NSNull() as Any, "title": (title.value as? String) ?? NSNull() as Any, "value": observedValue.value, "actions": observedActions.names, "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
AXUIElementCopyActionNames(element, &actions)
nodes.append(["path": path, "role": text(element, kAXRoleAttribute), "title": text(element, kAXTitleAttribute), "value": text(element, kAXValueAttribute), "actions": actions as? [String] ?? [], "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
guard let totalChildren = childCountResult.value else {
truncated = true
gaps.append("AX child count unavailable at path \(path): AXError \(childCountResult.error ?? -1)")
+49 -61
View File
@@ -25,17 +25,25 @@ ARGUMENT_REGISTERS = {
"x86_64": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
"x86_64h": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
}
MAX_SELECTOR_BYTES = 1024
MAX_REPORTED_LOCATIONS = 64
MAX_OTHER_STOPS = 256
MAX_TRACE_BYTES = 8 * 1024 * 1024
MAX_TRACE_FRAMES = 65_536
MAX_OBSERVATION_JOURNAL_BYTES = MAX_TRACE_BYTES
MAX_BREAKPOINT_LOCATION_BYTES = 8 * 1024 * 1024
MAX_TARGET_OUTPUT_BYTES = 1024 * 1024
OUTPUT_DRAIN_BYTES_PER_PASS = 1024 * 1024
OUTPUT_EOF_WAIT_SECONDS = 1.0
OUTPUT_READER_JOIN_SECONDS = 0.5
LISTENER_NAME = "rea-native-call-observation"
def _can_retain_trace(current_bytes, current_frames, event_bytes, event_frames, limits):
def _can_retain_trace(current_bytes, current_frames, event_bytes, event_frames):
"""Whether a complete event fits both aggregate trace budgets."""
return (
current_bytes + event_bytes <= limits["trace_bytes"]
and current_frames + event_frames <= limits["trace_frames"]
current_bytes + event_bytes <= MAX_TRACE_BYTES
and current_frames + event_frames <= MAX_TRACE_FRAMES
)
@@ -64,7 +72,7 @@ def _fits_trace_budget(value, remaining_bytes):
class _ObservationJournal:
"""Flush a bounded JSONL prefix so observations survive tracer failure."""
def __init__(self, handle, max_bytes):
def __init__(self, handle, max_bytes=MAX_OBSERVATION_JOURNAL_BYTES):
self.handle = handle
self.max_bytes = max_bytes
self.bytes = 0
@@ -236,20 +244,6 @@ def _hex(value):
return "0x%x" % value
def _source_location(address):
"""Preserve available line-entry facets without acquiring source files."""
entry = address.GetLineEntry()
if not entry.IsValid():
return None
file_spec = entry.GetFileSpec()
file_path = str(file_spec) if file_spec.IsValid() else None
line = entry.GetLine() or None
column = entry.GetColumn() or None
if not file_path and line is None and column is None:
return None
return {"file_path": file_path or None, "line": line, "column": column}
def _location(address, target):
"""Load and file address, module and symbol of one code address."""
module = address.GetModule()
@@ -263,7 +257,6 @@ def _location(address, target):
"module": module.GetFileSpec().GetFilename() if module.IsValid() else None,
"module_path": str(module.GetFileSpec()) if module.IsValid() else None,
"symbol": symbol.GetName() if symbol.IsValid() else None,
"source_location": _source_location(address),
}
return location
@@ -364,7 +357,7 @@ def _record(context, thread, frame, index, sequence, location, remaining_bytes):
error = lldb.SBError()
text = process.ReadCStringFromMemory(
values[names[1]],
config["capture_limits"]["selector_bytes"],
MAX_SELECTOR_BYTES,
error,
)
selector = text if error.Success() and text else None
@@ -397,16 +390,10 @@ def _record(context, thread, frame, index, sequence, location, remaining_bytes):
_TRACE_CONTEXT = None
def _resource_limit(context, limit):
context["resource_limit"] = limit
return "resource-limit"
def _handle_breakpoint(context, frame, index):
"""Read one hit while LLDB's breakpoint callback owns the stopped frame."""
if not frame.IsValid() or frame.GetPC() == lldb.LLDB_INVALID_ADDRESS:
raise RuntimeError("LLDB breakpoint callback did not supply a readable frame")
limits = context["config"]["capture_limits"]
thread = frame.GetThread()
events = context["events"]
location = _location(frame.GetPCAddress(), context["target"])
@@ -423,50 +410,49 @@ def _handle_breakpoint(context, frame, index):
context["retained_frames"],
0,
event_frame_count,
limits,
):
return _resource_limit(context, "trace_frames")
return "resource-limit"
try:
remaining_bytes = limits["trace_bytes"] - context["retained_trace_bytes"]
remaining_bytes = MAX_TRACE_BYTES - context["retained_trace_bytes"]
record = _record(
context, thread, frame, index, len(events), location, remaining_bytes
)
except ValueError:
return _resource_limit(context, "trace_bytes")
return "resource-limit"
if record is not None:
if not _fits_trace_budget(record, remaining_bytes):
return _resource_limit(context, "trace_bytes")
return "resource-limit"
record_bytes = len(json.dumps(record).encode("utf-8"))
if context["retained_trace_bytes"] + record_bytes > limits["trace_bytes"]:
return _resource_limit(context, "trace_bytes")
if context["retained_trace_bytes"] + record_bytes > MAX_TRACE_BYTES:
return "resource-limit"
for depth in range(1, frame_count + 1):
try:
location = _location(
thread.GetFrameAtIndex(depth).GetPCAddress(), context["target"]
)
except ValueError:
return _resource_limit(context, "trace_bytes")
return "resource-limit"
if (
_estimated_json_size(location) + record_bytes + 1
> limits["trace_bytes"] - context["retained_trace_bytes"]
> MAX_TRACE_BYTES - context["retained_trace_bytes"]
):
return _resource_limit(context, "trace_bytes")
return "resource-limit"
location_bytes = len(json.dumps(location).encode("utf-8"))
addition = location_bytes + (1 if record["backtrace"] else 0)
if (
context["retained_trace_bytes"]
+ record_bytes
+ addition
> limits["trace_bytes"]
> MAX_TRACE_BYTES
):
return _resource_limit(context, "trace_bytes")
return "resource-limit"
record["backtrace"].append(location)
record_bytes += addition
journal = context.get("observation_journal")
if journal is not None and not journal.append(
{"kind": "event", "event": record}
):
return _resource_limit(context, "trace_bytes")
return "resource-limit"
events.append(record)
context["retained_frames"] += event_frame_count
context["retained_trace_bytes"] += record_bytes
@@ -498,9 +484,9 @@ def _handle_stop(context, other_stops):
if journal is not None and not journal.append(
{"kind": "other-stop", "reason": stop_description}
):
return _resource_limit(context, "trace_bytes")
return "resource-limit"
other_stops.append(stop_description)
if len(other_stops) >= context["config"]["capture_limits"]["other_stops"]:
if len(other_stops) >= MAX_OTHER_STOPS:
return "stop-limit"
return None
@@ -519,19 +505,20 @@ def _breakpoint_report(target, created, config):
if not _wanted(config["breakpoints"][index], location["symbol"] or ""):
continue
count += 1
remaining_bytes = (
config["capture_limits"]["breakpoint_location_bytes"] - retained_location_bytes
)
if not _fits_trace_budget(location, remaining_bytes):
locations_truncated = True
continue
location_bytes = len(json.dumps(location).encode("utf-8"))
addition = location_bytes + (1 if locations else 0)
if addition > remaining_bytes:
locations_truncated = True
continue
locations.append(location)
retained_location_bytes += addition
if len(locations) < MAX_REPORTED_LOCATIONS:
remaining_bytes = (
MAX_BREAKPOINT_LOCATION_BYTES - retained_location_bytes
)
if not _fits_trace_budget(location, remaining_bytes):
locations_truncated = True
continue
location_bytes = len(json.dumps(location).encode("utf-8"))
addition = location_bytes + (1 if locations else 0)
if addition > remaining_bytes:
locations_truncated = True
continue
locations.append(location)
retained_location_bytes += addition
report.append({"index": index, "location_count": count, "locations": locations})
return report, locations_truncated
@@ -651,7 +638,7 @@ def trace(debugger, config):
for breakpoint in breakpoints:
breakpoint_index[breakpoint.GetID()] = index
listener = lldb.SBListener(LISTENER_NAME)
output_budget = config["capture_limits"]["output_bytes"]
output_budget = config.get("max_output_bytes", MAX_TARGET_OUTPUT_BYTES)
stdout = _OutputPipe(
config["stdout_path"], config["stdout_capture_path"], output_budget
)
@@ -671,19 +658,21 @@ def trace(debugger, config):
observation_path = config.get("observation_path")
if observation_path:
observation_journal = _ObservationJournal(
open(observation_path, "wb"),
config["capture_limits"]["trace_bytes"],
open(observation_path, "wb")
)
process = target.Launch(_launch_info(target, config, listener), error)
process_created = process.IsValid()
if not error.Success() or not process_created:
if not error.Success() or not process.IsValid():
exit_observed = _retire_failed_launch(
process, config, listener, stdout, stderr
)
return {
"status": "launch-error",
"error": error.GetCString(),
"target_process_created": process_created,
**(
{"target_process_created": True}
if process.IsValid()
else {}
),
**(
{"target_process_exit_observed": True}
if exit_observed
@@ -863,7 +852,6 @@ def trace(debugger, config):
"breakpoint_locations_truncated": breakpoint_locations_truncated,
"events": events,
"resource_limit_reached": outcome == "resource-limit",
"resource_limit": context.get("resource_limit"),
"target_output": {
"stdout_bytes": stdout.bytes,
"stderr_bytes": stderr.bytes,
+40 -220
View File
@@ -25,113 +25,6 @@ spec.loader.exec_module(tracer)
class NativeTraceBudgetTests(unittest.TestCase):
def test_source_coordinates_preserve_partial_metadata_without_reading_source(self):
def address(valid=True, file_valid=True, line=23, column=7):
class SourceFileSpec:
def IsValid(self):
return file_valid
def __str__(self):
return "/missing/é space/fixture.pas"
file_spec = SourceFileSpec()
entry = types.SimpleNamespace(
IsValid=lambda: valid,
GetFileSpec=lambda: file_spec,
GetLine=lambda: line,
GetColumn=lambda: column,
)
return types.SimpleNamespace(GetLineEntry=lambda: entry)
self.assertEqual(
tracer._source_location(address()),
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": 7},
)
self.assertEqual(
tracer._source_location(address(column=0)),
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": None},
)
self.assertEqual(
tracer._source_location(address(file_valid=False)),
{"file_path": None, "line": 23, "column": 7},
)
self.assertEqual(
tracer._source_location(address(line=0)),
{"file_path": "/missing/é space/fixture.pas", "line": None, "column": 7},
)
self.assertIsNone(tracer._source_location(address(valid=False)))
self.assertIsNone(
tracer._source_location(address(file_valid=False, line=0, column=0))
)
def test_invalid_launch_process_reports_explicit_no_creation(self):
class InvalidProcess:
def IsValid(self):
return False
def Kill(self):
raise AssertionError("an invalid process cannot own target cleanup")
class LaunchError:
def Success(self):
return False
def GetCString(self):
return "No such file or directory: fixture/absent"
class Target:
def IsValid(self):
return True
def Launch(self, _launch_info, _error):
return InvalidProcess()
class Debugger:
def SetAsync(self, _enabled):
pass
def GetInstanceName(self):
return "fixture"
def CreateTarget(self, *_arguments):
return Target()
class Pipe:
def __init__(self, *_arguments):
self.closed = False
pipes.append(self)
def start(self):
pass
def close(self, expected_exit=False):
self.closed = True
pipes = []
debugger_api = types.SimpleNamespace(SetInternalVariable=lambda *_arguments: None)
with patch.object(tracer.lldb, "SBDebugger", debugger_api, create=True), patch.object(
tracer.lldb, "SBError", LaunchError, create=True
), patch.object(tracer.lldb, "SBListener", lambda _name: object(), create=True), patch.object(
tracer, "_hash_file", return_value="expected"
), patch.object(tracer, "_file_identity", return_value=(1, 2)), patch.object(
tracer, "_module_identity", return_value=("fixture", (1, 2), "fixture-uuid")
), patch.object(tracer, "_launch_info", return_value=object()), patch.object(
tracer, "_OutputPipe", Pipe
):
result = tracer.trace(Debugger(), {
"executable": "fixture", "architecture": "arm64", "expected_sha256": "expected",
"breakpoints": [], "working_directory": "fixture/absent",
"capture_limits": {"output_bytes": 1024},
"stdout_path": "stdout", "stdout_capture_path": "stdout-capture",
"stderr_path": "stderr", "stderr_capture_path": "stderr-capture",
})
self.assertEqual(result["status"], "launch-error")
self.assertEqual(result["error"], "No such file or directory: fixture/absent")
self.assertIs(result["target_process_created"], False)
self.assertNotIn("target_process_exit_observed", result)
self.assertEqual(len(pipes), 2)
self.assertTrue(all(pipe.closed for pipe in pipes))
def test_failed_launch_does_not_infer_exit_from_a_failed_kill(self):
class Process:
def IsValid(self):
@@ -321,44 +214,37 @@ class NativeTraceBudgetTests(unittest.TestCase):
return super().flush()
stream = CountingStream()
journal = tracer._ObservationJournal(stream, max_bytes=4096)
journal = tracer._ObservationJournal(stream)
journal.append({"kind": "event", "event": {"sequence": 0}})
journal.append({"kind": "other-stop", "reason": "signal"})
self.assertEqual((stream.writes, stream.flushes), (2, 2))
self.assertEqual(journal.bytes, len(stream.getvalue()))
def test_aggregate_trace_budget_uses_the_selected_admission(self):
limits = {"trace_bytes": 100, "trace_frames": 10}
self.assertTrue(tracer._can_retain_trace(90, 8, 10, 2, limits))
self.assertFalse(tracer._can_retain_trace(90, 8, 11, 2, limits))
self.assertFalse(tracer._can_retain_trace(90, 8, 10, 3, limits))
raised = {"trace_bytes": 101, "trace_frames": 11}
self.assertTrue(tracer._can_retain_trace(90, 8, 11, 3, raised))
def test_breakpoint_reports_exact_exhausted_budget_and_accepts_more_room(self):
frame = types.SimpleNamespace(
IsValid=lambda: True, GetPC=lambda: 1,
GetThread=lambda: types.SimpleNamespace(GetNumFrames=lambda: 1),
GetPCAddress=lambda: object(),
def test_aggregate_trace_budget_admits_exact_boundary_and_rejects_next_record(self):
self.assertTrue(
tracer._can_retain_trace(
tracer.MAX_TRACE_BYTES - 10,
tracer.MAX_TRACE_FRAMES - 2,
10,
2,
)
)
self.assertFalse(
tracer._can_retain_trace(
tracer.MAX_TRACE_BYTES - 10,
tracer.MAX_TRACE_FRAMES - 2,
11,
2,
)
)
self.assertFalse(
tracer._can_retain_trace(
tracer.MAX_TRACE_BYTES - 10,
tracer.MAX_TRACE_FRAMES - 2,
10,
3,
)
)
context = {
"target": object(), "events": [], "retained_frames": 1,
"retained_trace_bytes": 0,
"config": {"breakpoints": [{}], "backtrace_frames": 0, "max_events": 10,
"capture_limits": {"trace_bytes": 4096, "trace_frames": 1}},
}
with patch.object(tracer, "_location", return_value={"symbol": "main"}), patch.object(
tracer, "_wanted", return_value=True
), patch.object(tracer, "_record", return_value={"backtrace": []}):
self.assertEqual(tracer._handle_breakpoint(context, frame, 0), "resource-limit")
self.assertEqual(context["resource_limit"], "trace_frames")
context["config"]["capture_limits"] = {"trace_bytes": 8, "trace_frames": 2}
self.assertEqual(tracer._handle_breakpoint(context, frame, 0), "resource-limit")
self.assertEqual(context["resource_limit"], "trace_bytes")
self.assertEqual(context["events"], [])
context["config"]["capture_limits"]["trace_bytes"] = 4096
self.assertIsNone(tracer._handle_breakpoint(context, frame, 0))
self.assertEqual(context["events"], [{"backtrace": []}])
def test_oversized_producer_string_is_rejected_before_serialization(self):
self.assertTrue(tracer._fits_trace_budget("method", 1024))
@@ -437,9 +323,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
return FileSpec()
class Address:
def GetLineEntry(self):
return types.SimpleNamespace(IsValid=lambda: False)
def GetModule(self):
return Module()
@@ -467,7 +350,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
object(),
[[Breakpoint()]],
{
"capture_limits": {"breakpoint_location_bytes": 4096},
"breakpoints": [
{
"kind": "objc-method",
@@ -481,34 +363,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
self.assertEqual(report[0]["locations"], [])
self.assertFalse(locations_truncated)
def test_breakpoint_report_keeps_all_locations_under_aggregate_budget(self):
class Location:
def __init__(self, index):
self.index = index
def GetAddress(self):
return self.index
class Breakpoint:
def GetNumLocations(self):
return 65
def GetLocationAtIndex(self, index):
return Location(index)
with patch.object(
tracer,
"_location",
side_effect=lambda address, _target: {"address": address, "symbol": None},
), patch.object(tracer, "_wanted", return_value=True):
report, locations_truncated = tracer._breakpoint_report(
object(), [[Breakpoint()]], {"breakpoints": [{}], "capture_limits": {"breakpoint_location_bytes": 4096}}
)
self.assertEqual(report[0]["location_count"], 65)
self.assertEqual(len(report[0]["locations"]), 65)
self.assertFalse(locations_truncated)
def test_breakpoint_report_bounds_aggregate_location_json(self):
tracer.lldb.LLDB_INVALID_ADDRESS = -1
symbol_name = (
@@ -542,24 +396,7 @@ class NativeTraceBudgetTests(unittest.TestCase):
def GetFileSpec(self):
return FileSpec()
class SourceFileSpec:
def IsValid(self):
return True
def __str__(self):
return source_path
class Address:
def GetLineEntry(self):
if source_path is None:
return types.SimpleNamespace(IsValid=lambda: False)
return types.SimpleNamespace(
IsValid=lambda: True,
GetFileSpec=SourceFileSpec,
GetLine=lambda: 15,
GetColumn=lambda: 3,
)
def GetModule(self):
return Module()
@@ -583,39 +420,22 @@ class NativeTraceBudgetTests(unittest.TestCase):
def GetLocationAtIndex(self, index):
return Location()
breakpoint_count = 30
breakpoint_count = 1000
created = [[Breakpoint()] for _ in range(breakpoint_count)]
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count,
"capture_limits": {"breakpoint_location_bytes": 512 * 1024}}
for source_path in (None, "/tmp/" + "源🚀" * 3000 + "/calls.pas"):
with self.subTest(source_path_present=source_path is not None):
report, locations_truncated = tracer._breakpoint_report(
object(), created, config
)
retained_bytes = sum(
len(json.dumps(location).encode("utf-8"))
+ (1 if index > 0 else 0)
for row in report
for index, location in enumerate(row["locations"])
)
self.assertTrue(locations_truncated)
self.assertEqual(
sum(row["location_count"] for row in report), 1920
)
self.assertLess(
sum(len(row["locations"]) for row in report), 1920
)
self.assertLessEqual(
retained_bytes, 512 * 1024
)
if source_path is not None:
self.assertTrue(
any(
location["source_location"]["file_path"] == source_path
for row in report
for location in row["locations"]
)
)
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count}
report, locations_truncated = tracer._breakpoint_report(
object(), created, config
)
retained_bytes = sum(
len(json.dumps(location).encode("utf-8"))
+ (1 if index > 0 else 0)
for row in report
for index, location in enumerate(row["locations"])
)
self.assertTrue(locations_truncated)
self.assertEqual(sum(row["location_count"] for row in report), 64_000)
self.assertLess(sum(len(row["locations"]) for row in report), 64_000)
self.assertLessEqual(retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES)
if __name__ == "__main__":
+5 -7
View File
@@ -78,9 +78,14 @@ public final class NativeAotExtension {
new FrozenObjectAnnotator(program, manager).analyze(located.directory(), scan.getPointerLocations(), monitor, log);
}
monitor.checkCancelled();
report.addProperty("method_tables", manager.getMethodTableCount());
JsonObject coverage = auditFrozenObjects(program, scan.getPointerLocations(), monitor);
report.add("coverage", coverage);
if (coverage.get("frozen_objects_annotated").getAsInt() != coverage.get("frozen_object_candidates").getAsInt())
diagnostics.add("Some frozen-object candidates lack committed instance annotations; inspect coverage counts and Ghidra diagnostics.");
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
report.add("reason", JsonNull.INSTANCE);
report.add("derived_memory", derivedRange.deepCopy());
JsonArray tables = new JsonArray();
for (var mt : manager.getMethodTables()) {
monitor.checkCancelled();
@@ -89,13 +94,6 @@ public final class NativeAotExtension {
identity.addProperty("type", mt.getMTType().getPathName());
tables.add(identity);
}
// Publish recovery facts only after every fallible recovery and audit step.
// A failed transaction is rolled back, so it cannot report committed metadata.
report.addProperty("method_tables", manager.getMethodTableCount());
report.add("coverage", coverage);
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
report.add("reason", JsonNull.INSTANCE);
report.add("derived_memory", derivedRange.deepCopy());
report.add("types", tables);
commit = true;
return finish(log);
+41 -137
View File
@@ -12,32 +12,16 @@ struct ProcessIdentityObservation: Encodable {
let pid: Int32
let state: String
let identity: String?
let effective_uid: UInt32?
let lifecycle: String?
let relation: ProcessRelationObservation?
/** Display-only kernel accounting name (p_comm, at most MAXCOMLEN bytes). */
let name: String?
let reason: String?
}
struct ProcessRelationObservation: Encodable {
let state: String
let parent_pid: Int32
let process_group_id: Int32
}
struct ProcessMetadata {
private struct ProcessMetadata {
let identity: String
let effectiveUid: UInt32
let lifecycle: String
let parentPid: Int32
let processGroupId: Int32
let pointerSize: Int
let name: String
}
struct ProcessSnapshot {
let metadata: [Int32: ProcessMetadata]
fileprivate let metadata: [Int32: ProcessMetadata]
let failure: String?
}
@@ -91,12 +75,7 @@ func readProcessSnapshot(
process.kp_proc.p_pid,
ProcessMetadata(
identity: "\(start.tv_sec):\(start.tv_usec)",
effectiveUid: process.kp_eproc.e_ucred.cr_uid,
lifecycle: process.kp_proc.p_stat == SZOMB ? "zombie" : "non-zombie",
parentPid: process.kp_eproc.e_ppid,
processGroupId: process.kp_eproc.e_pgid,
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4,
name: processName(process)
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4
)
)
},
@@ -107,88 +86,25 @@ func readProcessSnapshot(
return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(ENOMEM)")
}
/** Decode the NUL-terminated accounting name; invalid UTF-8 is replaced for display. */
private func processName(_ process: kinfo_proc) -> String {
withUnsafeBytes(of: process.kp_proc.p_comm) { bytes in
String(decoding: bytes.prefix { $0 != 0 }, as: UTF8.self)
}
}
func readProcessIdentities(pids: [Int32]) -> [ProcessIdentityObservation] {
readProcessIdentities(pids: pids, snapshot: readProcessSnapshot())
}
func readProcessIdentities(
pids: [Int32],
snapshot: ProcessSnapshot
) -> [ProcessIdentityObservation] {
let snapshot = readProcessSnapshot()
return pids.map { pid in
guard pid > 0 else {
return unavailableIdentity(pid: pid, reason: "invalid_process_id")
}
guard let metadata = snapshot.metadata[pid] else {
if let failure = snapshot.failure {
return unavailableIdentity(pid: pid, reason: failure)
}
// This records absence from this successful KERN_PROC_ALL snapshot only.
// Unlike the Linux procfs path, it has no PID-specific kill(0)
// corroboration; callers must not treat it as universal or future exit
// proof.
return ProcessIdentityObservation(
return unavailableIdentity(
pid: pid,
state: "absent",
identity: nil,
effective_uid: nil,
lifecycle: nil,
relation: nil,
name: nil,
reason: "not-in-snapshot"
reason: snapshot.failure ?? "process_not_in_snapshot"
)
}
return ProcessIdentityObservation(
pid: pid,
state: "readable",
identity: metadata.identity,
effective_uid: metadata.effectiveUid,
lifecycle: metadata.lifecycle,
relation: ProcessRelationObservation(
state: "observed",
parent_pid: metadata.parentPid,
process_group_id: metadata.processGroupId
),
name: metadata.name,
reason: nil
)
return ProcessIdentityObservation(pid: pid, state: "readable", identity: metadata.identity, reason: nil)
}
}
private func unavailableIdentity(pid: Int32, reason: String) -> ProcessIdentityObservation {
ProcessIdentityObservation(
pid: pid,
state: "unavailable",
identity: nil,
effective_uid: nil,
lifecycle: nil,
relation: nil,
name: nil,
reason: reason
)
}
enum ProcargsMalformedStage: String, CaseIterable {
case header
case argumentCount = "argument_count"
case executablePath = "executable_path"
case pointerSize = "pointer_size"
case pathPadding = "path_padding"
case arguments
case terminator
case trailingEntries = "trailing_entries"
ProcessIdentityObservation(pid: pid, state: "unavailable", identity: nil, reason: reason)
}
enum RunTokenReadError: Error {
case invalidBuffer
case invalidBufferAt(ProcargsMalformedStage)
case duplicateToken
case invalidTokenEncoding
case environmentUnavailable
@@ -228,22 +144,6 @@ func processRunTokenObservation(
parse: Result<ParsedRunToken, RunTokenReadError>,
codeSigningFlags: () -> UInt32?
) -> RunTokenObservation {
// Native inspection restrictions apply even when argv is incomplete or
// overwritten, before parsing reaches the environment. Keep this fact at
// the native boundary; cleanup still decides whether the process is related.
func withheldEnvironment() -> RunTokenObservation? {
guard let flags = codeSigningFlags() else { return nil }
if (flags & csPlatformBinary) != 0 {
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
}
if (flags & csRestrict) != 0 {
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
}
return nil
}
func unavailableEnvironment(_ reason: String) -> RunTokenObservation {
withheldEnvironment() ?? unavailable(pid: pid, reason: reason)
}
switch parse {
case .success(let parsed):
if let runId = parsed.runId {
@@ -254,25 +154,38 @@ func processRunTokenObservation(
if parsed.environmentEntryCount > 0 {
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
}
if let withheld = withheldEnvironment() { return withheld }
let flags = codeSigningFlags()
if let flags, (flags & csPlatformBinary) != 0 {
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
}
if let flags, (flags & csRestrict) != 0 {
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
}
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
case .failure(let failure):
switch failure {
case .environmentUnavailable:
return unavailableEnvironment("environment_unavailable")
let flags = codeSigningFlags()
if let flags, (flags & csPlatformBinary) != 0 {
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
}
if let flags, (flags & csRestrict) != 0 {
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
}
return unavailable(pid: pid, reason: "environment_unavailable")
case .ambiguousEnvironmentBoundary:
return unavailableEnvironment("ambiguous_environment_boundary")
// These failures already establish that token bytes were available.
if let flags = codeSigningFlags(), (flags & csRestrict) != 0 {
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
}
return unavailable(pid: pid, reason: "ambiguous_environment_boundary")
case .duplicateToken:
return unavailable(pid: pid, reason: "duplicate_run_token")
case .invalidTokenEncoding:
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
case .appleVectorUnavailable:
return unavailableEnvironment("apple_vector_unavailable")
return unavailable(pid: pid, reason: "apple_vector_unavailable")
case .invalidBuffer:
return unavailableEnvironment("malformed_procargs")
case .invalidBufferAt(let stage):
return unavailableEnvironment("malformed_procargs_\(stage.rawValue)")
return unavailable(pid: pid, reason: "malformed_procargs")
}
}
}
@@ -331,30 +244,27 @@ func parseProcessEnvironment(
in bytes: [UInt8],
pointerSize: Int = MemoryLayout<UnsafeRawPointer>.size
) throws -> ParsedRunToken {
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBufferAt(.header) }
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBuffer }
let argc = bytes.withUnsafeBytes { $0.loadUnaligned(as: Int32.self).littleEndian }
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBufferAt(.argumentCount) }
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBuffer }
var offset = MemoryLayout<Int32>.size
guard let executablePath = readString(in: bytes, offset: &offset) else {
throw RunTokenReadError.invalidBufferAt(.executablePath)
throw RunTokenReadError.invalidBuffer
}
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBufferAt(.pointerSize) }
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBuffer }
let pathRecordSize = Array("executable_path=".utf8).count + executablePath.count + 1
let padding = (pointerSize - pathRecordSize % pointerSize) % pointerSize
guard offset + padding <= bytes.count,
bytes[offset..<(offset + padding)].allSatisfy({ $0 == 0 }) else {
throw RunTokenReadError.invalidBufferAt(.pathPadding)
throw RunTokenReadError.invalidBuffer
}
offset += padding
let tokenPrefix = Array("REA_PROCESS_RUN_ID=".utf8)
var tokenInArgument = false
for _ in 0..<argc {
guard let argument = readString(in: bytes, offset: &offset, allowEmpty: true) else {
throw RunTokenReadError.invalidBufferAt(.arguments)
guard readString(in: bytes, offset: &offset, allowEmpty: true) != nil else {
throw RunTokenReadError.invalidBuffer
}
tokenInArgument = tokenInArgument || argument.starts(with: tokenPrefix)
}
guard bytes.last == 0 else { throw RunTokenReadError.invalidBufferAt(.terminator) }
guard bytes.last == 0 else { throw RunTokenReadError.invalidBuffer }
if offset == bytes.count {
throw RunTokenReadError.environmentUnavailable
}
@@ -372,7 +282,7 @@ func parseProcessEnvironment(
continue
}
guard let entry = readString(in: bytes, offset: &offset) else {
throw RunTokenReadError.invalidBufferAt(.trailingEntries)
throw RunTokenReadError.invalidBuffer
}
trailingEntries.append(entry)
}
@@ -393,28 +303,22 @@ func parseProcessEnvironment(
throw RunTokenReadError.appleVectorUnavailable
}
if trailingEntries[appleVectorStart...].contains(where: {
$0.starts(with: tokenPrefix)
$0.starts(with: Array("REA_PROCESS_RUN_ID=".utf8))
}) {
throw RunTokenReadError.ambiguousEnvironmentBoundary
}
let environmentEntries = trailingEntries[..<appleVectorStart]
var found: String?
for entry in environmentEntries {
if entry.starts(with: tokenPrefix) {
if entry.starts(with: Array("REA_PROCESS_RUN_ID=".utf8)) {
guard found == nil else { throw RunTokenReadError.duplicateToken }
let value = entry.dropFirst(tokenPrefix.count)
let value = entry.dropFirst("REA_PROCESS_RUN_ID=".utf8.count)
guard let decoded = String(bytes: value, encoding: .utf8) else {
throw RunTokenReadError.invalidTokenEncoding
}
found = decoded
}
}
// A changed process title can overwrite the original argument boundaries,
// causing a real environment entry to be consumed as an argument. A real
// argument can have this form too, so it proves neither ownership nor absence.
if tokenInArgument {
throw RunTokenReadError.ambiguousEnvironmentBoundary
}
return ParsedRunToken(runId: found, environmentEntryCount: environmentEntries.count)
}
+5 -18
View File
@@ -5,6 +5,7 @@ from pathlib import Path
import resource
import gdb
OUTPUT_BYTES = 64 * 1024 * 1024
class ContextFailure(Exception):
@@ -19,9 +20,8 @@ def inspect(request):
root = Path(os.environ["REA_CORE_CONTEXT_REQUEST"]).parent
empty = root / "no-libraries"
empty.mkdir(mode=0o700)
for command in ["set auto-load off", "set debuginfod enabled off", "set auto-solib-add off", "set history save off", "set pagination off", "set target-file-system-kind unix",
"set sysroot ./no-libraries", "set solib-search-path ./no-libraries",
"set debug-file-directory ./no-libraries"]:
for command in ["set auto-load off", "set debuginfod enabled off", "set auto-solib-add off", "set history save off", "set pagination off",
"set sysroot ./no-libraries", "set solib-search-path ./no-libraries"]:
gdb.execute(command, to_string=True)
entry = request["gdbinit"]
try:
@@ -38,19 +38,7 @@ def inspect(request):
# relative filename in the owned cwd so no selected path becomes GDB syntax.
if Path(request["snapshot_path"]) != root.parent / "object.snapshot":
raise ContextFailure("decoder", "Core context request did not select the shared owned snapshot.")
# GDB 16 searches AT_EXECFN/NT_FILE outside sysroot when no executable BFD
# is selected. Occupy that slot with the already running, owned debugger;
# exec-file loads no symbols and never runs it. Clear it before any query.
# /proc/self/exe identifies this process, not the core's historical PID.
architecture = {"x86_64-little-endian": "i386:x86-64", "aarch64-little-endian": "aarch64"}.get(request["architecture"])
if architecture is None:
raise ContextFailure("unsupported", "Unsupported recorded core architecture.")
gdb.execute("exec-file /proc/self/exe", to_string=True)
# The placeholder must not select the core's architecture. Reuse the
# decoder's validated profile, including for cross-architecture GDB builds.
gdb.execute("set architecture " + architecture, to_string=True)
gdb.execute("core-file ../object.snapshot", to_string=True)
gdb.execute("file", to_string=True)
inferior = gdb.selected_inferior()
if inferior.connection is None or inferior.connection.type != "core" or gdb.current_progspace().filename is not None:
raise ContextFailure("decoder", "Requested core-only boundary was not established; no mapping query performed.")
@@ -78,7 +66,6 @@ def failure_exit(request_path, status, marker):
def main():
request_path = os.environ["REA_CORE_CONTEXT_REQUEST"]
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
output_bytes = request["producer_limits"]["output_bytes"]
reserve = bytearray(1024 * 1024)
try:
reply = {"ok": True, "value": inspect(request)}
@@ -94,8 +81,8 @@ def main():
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
try:
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
if len(encoded) > output_bytes:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds producer_limits.output_bytes=" + str(output_bytes) + "."}).encode("utf-8")
if len(encoded) > OUTPUT_BYTES:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds64 MiB reply budget."}).encode("utf-8")
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "wb") as handle:
handle.write(encoded)
-127
View File
@@ -1,127 +0,0 @@
"""Producer regressions with controlled GDB state, not native debugger qualification."""
import json
import os
from pathlib import Path
import runpy
import sys
import tempfile
import types
import unittest
from unittest.mock import patch
BRIDGE = Path(__file__).with_name("core_context.py")
class CoreContextTests(unittest.TestCase):
def inspect(self, failure=None, connection="core", symbols=None, architecture="x86_64-little-endian", objfile="recorded-executable", output_bytes=16384):
commands, reads, queries = [], [], []
state = {"executable": None, "symbols": None, "connection": None}
gdb = types.ModuleType("gdb")
gdb.VERSION = "16.3"
gdb.selected_inferior = lambda: types.SimpleNamespace(
connection=state["connection"], pid=123)
gdb.current_progspace = lambda: types.SimpleNamespace(filename=state["symbols"])
def execute(command, to_string):
self.assertTrue(to_string)
commands.append(command)
if command == failure:
raise RuntimeError("controlled command failure")
if command == "exec-file /proc/self/exe":
state["executable"] = "/proc/self/exe"
elif command == "core-file ../object.snapshot":
if state["executable"] is None:
reads.append("recorded-executable")
state["executable"] = state["symbols"] = "recorded-executable"
state["connection"] = types.SimpleNamespace(type=connection)
elif command == "file":
state["executable"] = None
state["symbols"] = symbols
gdb.execute = execute
pwndbg = types.ModuleType("pwndbg")
pwndbg.aglib = types.ModuleType("pwndbg.aglib")
pwndbg.aglib.vmmap = types.ModuleType("pwndbg.aglib.vmmap")
pwndbg.lib = types.ModuleType("pwndbg.lib")
pwndbg.lib.version = types.ModuleType("pwndbg.lib.version")
pwndbg.lib.version.__version__ = "2026.09.15"
def maps():
self.assertIsNone(state["executable"], "Helper executable must be discarded before mapping queries")
queries.append("maps")
return [types.SimpleNamespace(start=4096, end=8192, offset=0, flags=5, objfile=objfile)]
pwndbg.aglib.vmmap.get = maps
modules = {module.__name__: module for module in [
gdb, pwndbg, pwndbg.aglib, pwndbg.aglib.vmmap, pwndbg.lib, pwndbg.lib.version]}
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
stage = root / "debugger"
stage.mkdir()
plugin = root / "gdbinit.py"
plugin.write_text("")
request = stage / "request.json"
reply = stage / "reply.json"
request.write_text(json.dumps({"gdbinit": str(plugin), "architecture": architecture, "snapshot_path": str(root / "object.snapshot"), "reply_path": str(reply), "producer_limits": {"output_bytes": output_bytes}}))
with patch.dict(sys.modules, modules), patch.dict(os.environ, REA_CORE_CONTEXT_REQUEST=str(request)):
runpy.run_path(str(BRIDGE))
result = json.loads(reply.read_text())
return result, commands, reads, queries
def test_core_load_never_opens_recorded_executable_and_queries_only_after_discard(self):
result, commands, reads, queries = self.inspect()
self.assertTrue(result["ok"], result)
self.assertEqual(reads, [])
self.assertEqual(queries, ["maps"])
self.assertEqual(commands[-4:], ["exec-file /proc/self/exe", "set architecture i386:x86-64", "core-file ../object.snapshot", "file"])
self.assertLess(commands.index("set debug-file-directory ./no-libraries"), commands.index("core-file ../object.snapshot"))
self.assertIsNone(result["value"]["executable"])
self.assertEqual(result["value"]["maps"][0]["current_file_identity"], "unknown")
def test_failed_guard_core_load_or_discard_never_queries_mappings(self):
for command in ["exec-file /proc/self/exe", "set architecture i386:x86-64", "core-file ../object.snapshot", "file"]:
with self.subTest(command=command):
result, commands, reads, queries = self.inspect(failure=command)
self.assertFalse(result["ok"])
self.assertEqual(reads, [])
self.assertEqual(queries, [])
if command in ["exec-file /proc/self/exe", "set architecture i386:x86-64"]:
self.assertNotIn("core-file ../object.snapshot", commands)
def test_cross_architecture_uses_the_validated_decoder_profile(self):
result, commands, reads, queries = self.inspect(architecture="aarch64-little-endian")
self.assertTrue(result["ok"], result)
self.assertEqual(reads, [])
self.assertEqual(queries, ["maps"])
self.assertLess(commands.index("set architecture aarch64"), commands.index("core-file ../object.snapshot"))
def test_unknown_architecture_refuses_before_loading_core(self):
result, commands, reads, queries = self.inspect(architecture="unknown")
self.assertFalse(result["ok"])
self.assertEqual(result["reason"], "unsupported")
self.assertNotIn("core-file ../object.snapshot", commands)
self.assertEqual(reads, [])
self.assertEqual(queries, [])
def test_selected_output_limit_reports_recovery_and_accepts_more_headroom(self):
result, _, reads, queries = self.inspect(objfile="m" * 2048, output_bytes=1024)
self.assertFalse(result["ok"])
self.assertEqual(result["reason"], "output-limit")
self.assertIn("producer_limits.output_bytes=1024", result["message"])
self.assertEqual(reads, [])
self.assertEqual(queries, ["maps"])
recovered, _, _, _ = self.inspect(objfile="m" * 2048, output_bytes=4096)
self.assertTrue(recovered["ok"], recovered)
def test_wrong_connection_or_remaining_symbols_never_queries_mappings(self):
for options in [{"connection": "native"}, {"symbols": "unexpected-symbols"}]:
with self.subTest(options=options):
result, _, reads, queries = self.inspect(**options)
self.assertFalse(result["ok"])
self.assertEqual(reads, [])
self.assertEqual(queries, [])
if __name__ == "__main__":
unittest.main()
+9 -12
View File
@@ -5,12 +5,14 @@ from pathlib import Path
import resource
import sys
def lower_resource_limits(policy):
OUTPUT_BYTES = 64 * 1024 * 1024
def lower_resource_limits():
limits = {}
for name, kind, maximum in (
("address_space_bytes", resource.RLIMIT_AS, policy["address_space_bytes"]),
("cpu_seconds", resource.RLIMIT_CPU, policy["cpu_seconds"]),
("file_size_bytes", resource.RLIMIT_FSIZE, policy["file_bytes"]),
("address_space_bytes", resource.RLIMIT_AS, 3 * 1024**3),
("cpu_seconds", resource.RLIMIT_CPU, 30),
("file_size_bytes", resource.RLIMIT_FSIZE, OUTPUT_BYTES),
):
soft, hard = resource.getrlimit(kind)
finite = [maximum] + [value for value in (soft, hard) if value != resource.RLIM_INFINITY]
@@ -23,11 +25,6 @@ def lower_resource_limits(policy):
def main(request_path, inspect, failure_type, profile):
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
policy = request["producer_limits"]
for key in ("input_bytes", "output_bytes", "diagnostic_bytes", "address_space_bytes", "file_bytes", "cpu_seconds"):
value = policy[key]
if type(value) is not int or not 1 <= value <= 9007199254740991:
raise ValueError("producer_limits." + key + " must be a positive safe integer")
os.environ["PWNLIB_NOTERM"] = "1"
os.environ["PWNLIB_CACHE_DIR"] = str(Path(request_path).parent / "cache")
limits = None
@@ -35,7 +32,7 @@ def main(request_path, inspect, failure_type, profile):
# The outer exit-status contract also covers allocation failure during serialization.
memory_reserve = bytearray(1024 * 1024)
try:
limits = lower_resource_limits(policy)
limits = lower_resource_limits()
descriptor = os.open(Path(request_path).parent / "limits.json", os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
json.dump(limits, handle)
@@ -54,8 +51,8 @@ def main(request_path, inspect, failure_type, profile):
else:
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
if len(encoded) > policy["output_bytes"]:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete artifact result exceeds producer_limits.output_bytes=" + str(policy["output_bytes"]) + "."}).encode("utf-8")
if len(encoded) > OUTPUT_BYTES:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete artifact result exceeds the 64 MiB reply budget."}).encode("utf-8")
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "wb") as handle:
handle.write(encoded)
+12 -38
View File
@@ -15,11 +15,6 @@ import sys
PROFILE = "[email protected];[email protected];[email protected]"
PACKAGES = {"pwntools": "4.15.0", "pyelftools": "0.33", "unicorn": "2.1.2"}
OUTPUT_BYTES = 64 * 1024 * 1024
# (e_machine, ELF class bits, byte order) -> architecture pwntools must also report.
PROFILES = {(62, 64, "little"): "amd64", (183, 64, "little"): "aarch64", (3, 32, "little"): "i386",
(40, 32, "little"): "arm", (8, 32, "little"): "mips", (8, 32, "big"): "mips"}
PROFILE_SCOPE = ("Layout profile requires ELF64 x86-64, ELF64 AArch64 little-endian, ELF32 i386, "
"ELF32 ARM little-endian or ELF32 MIPS little- or big-endian objects.")
class LayoutFailure(Exception):
@@ -121,16 +116,13 @@ def inspect_elf(path, cache):
raise LayoutFailure("format", "Selected object does not contain an ELF header.")
if len(content) < 16:
raise LayoutFailure("format", "Selected ELF identification is truncated.")
if content[4] not in (1, 2) or content[5] not in (1, 2):
raise LayoutFailure("unsupported-target", "Selected ELF declares an unknown class or byte order.")
bits, header_size = (32, 52) if content[4] == 1 else (64, 64)
order = "little" if content[5] == 1 else "big"
if len(content) < header_size:
raise LayoutFailure("format", f"Selected ELF{bits} header is truncated.")
profile = (int.from_bytes(content[18:20], order), bits, order)
if profile not in PROFILES:
raise LayoutFailure("unsupported-target", PROFILE_SCOPE)
if int.from_bytes(content[16:18], order) not in (1, 2, 3):
if content[4] != 2 or content[5] != 1:
raise LayoutFailure("unsupported-target", "Initial layout profile requires ELF64 little-endian objects.")
if len(content) < 64:
raise LayoutFailure("format", "Selected ELF64 header is truncated.")
if int.from_bytes(content[18:20], "little") != 62:
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF objects.")
if int.from_bytes(content[16:18], "little") not in (1, 2, 3):
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
from pwnlib.context import context
with context.local(cache_dir=str(cache), log_level="warning"):
@@ -140,11 +132,6 @@ def inspect_elf(path, cache):
try:
headers = ELFFile(io.BytesIO(content))
section_count = headers.num_sections()
if section_count:
location(headers.header.e_shoff, section_count * headers.header.e_shentsize, len(content))
segment_count = headers.num_segments()
if segment_count:
location(headers.header.e_phoff, segment_count * headers.header.e_phentsize, len(content))
if headers.header.e_shstrndx == 0xffff and section_count == 0:
raise LayoutFailure("format", "ELF extended section-name index requires a declared section-zero header.")
raw_name_index = headers.header.e_shstrndx
@@ -160,21 +147,9 @@ def inspect_elf(path, cache):
if names_header.sh_type != "SHT_STRTAB":
raise LayoutFailure("format", f"ELF section-name table {section_name_index} is not SHT_STRTAB.")
location(names_header.sh_offset, names_header.sh_size, len(content))
except (ELFError, ValueError, OverflowError, AssertionError, IndexError) as error:
except ELFError as error:
raise LayoutFailure("format", str(error)) from error
class LayoutELF(ELF):
def search(self, needle, writable=False, executable=False):
# This adapter exposes structural facts, GOT/PLT and mitigation
# indicators, not kernel config/version discovery. Upstream's
# constructor searches virtual memory for those unrelated
# heuristics, materializing NOBITS tails and interpreting
# ordinary IKCFG_ST data as a compressed kernel config.
return iter(())
def _populate_kernel_version(self):
# Kernel banner decoding is outside this report too.
return None
def _get_section_name(self, header):
# SHN_UNDEF declares no names; upstream would otherwise read
# strings at the inactive section-zero offset.
@@ -193,8 +168,8 @@ def inspect_elf(path, cache):
if type is None or section.header.sh_type == type)
try:
image = LayoutELF(str(path), checksec=False)
if (image.arch, image.bits, image.endian) != (PROFILES[profile], bits, order):
raise LayoutFailure("unsupported-target", PROFILE_SCOPE)
if image.arch != "amd64" or image.bits != 64 or image.endian != "little":
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF64 little-endian objects.")
if image.header.e_type not in ("ET_EXEC", "ET_DYN", "ET_REL"):
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
length = len(content)
@@ -216,7 +191,7 @@ def inspect_elf(path, cache):
})
if h.sh_type in ("SHT_SYMTAB", "SHT_DYNSYM"):
if h.sh_entsize < image.structs.Elf_Sym.sizeof():
raise LayoutFailure("format", f"ELF symbol entry is smaller than the decoded Elf{bits}_Sym structure.")
raise LayoutFailure("format", "ELF symbol entry is smaller than the decoded Elf64_Sym structure.")
if h.sh_link >= len(all_sections) or all_sections[h.sh_link].header.sh_type != "SHT_STRTAB":
raise LayoutFailure("format", f"ELF symbol table {index} links to invalid string table {h.sh_link}.")
strings = all_sections[h.sh_link]
@@ -319,7 +294,6 @@ def inspect_elf(path, cache):
"Relocatable relocation targets with SHN_UNDEF or inactive SHT_NULL headers retain their reported section index/offset as unknown; they do not establish an associated target section.",
"GOT/PLT are derived upstream convenience maps; PLT inference can emulate selected instructions in Unicorn. They do not establish target runtime behavior; aliases may collapse and warnings may indicate incomplete coverage. Their completeness is unknown.",
"Mitigations are upstream static heuristics, not runtime protection. ET_DYN does not prove an executable; absent canary symbols do not prove every function unprotected.",
"Kernel configuration/version discovery is not performed; these memory-content scans do not supply the reported structural, linkage or mitigation facts.",
"This profile inspects ELF EXEC/DYN/REL layout only. It does not analyze recorded cores, resolve loaded libraries, launch the target as a host process or start a debugger.",
"Symbol/relocation inventories reflect original section tables. A sectionless image can still report dynamic dependency names; missing tables do not prove absence of dynamic symbols or relocations.",
*(["Absent section-name table: section-dependent RELRO/canary heuristics may be incomplete; protection coverage is unknown."] if all_sections and section_name_index == 0 else []),
@@ -328,7 +302,7 @@ def inspect_elf(path, cache):
}
except LayoutFailure:
raise
except (ELFError, ValueError, OverflowError, AssertionError, IndexError) as error:
except (ELFError, ValueError, AssertionError, IndexError) as error:
raise LayoutFailure("format", "Selected ELF failed unchanged upstream structural parsing: " + str(error)) from error
+8 -14
View File
@@ -88,18 +88,16 @@ def inspect_core(path, cache):
segments, notes, padding, threads, signals = [], [], [], [], []
for segment_index, segment in enumerate(image.iter_segments()):
h = segment.header
header_offset = header.e_phoff + segment_index * header.e_phentsize
type_id = int.from_bytes(content[header_offset:header_offset + 4], "little")
backing = type_id != 0 and h.p_filesz > 0
backing = h.p_type != "PT_NULL" and h.p_filesz > 0
if backing:
location(h.p_offset, h.p_filesz, len(content))
if type_id == 1 and h.p_filesz > h.p_memsz:
if h.p_type == "PT_LOAD" and h.p_filesz > h.p_memsz:
raise CoreFailure("format", "Recorded PT_LOAD file bytes exceed memory size.")
segments.append({"index": segment_index, "type": h.p_type, "type_id": type_id,
"header_location": location(header_offset, header.e_phentsize, len(content)),
segments.append({"index": segment_index, "type": h.p_type,
"header_location": location(header.e_phoff + segment_index * header.e_phentsize, header.e_phentsize, len(content)),
"offset": hex(h.p_offset), "file_size": hex(h.p_filesz), "memory_size": hex(h.p_memsz),
"virtual_address": hex(h.p_vaddr), "physical_address": hex(h.p_paddr), "alignment": hex(h.p_align), "flags": hex(h.p_flags), "file_backing": "file" if backing else "none"})
if type_id != 4 or h.p_filesz == 0:
if h.p_type != "PT_NOTE" or h.p_filesz == 0:
continue
position, end = h.p_offset, h.p_offset + h.p_filesz
header_size = image.structs.Elf_Nhdr.sizeof()
@@ -111,7 +109,6 @@ def inspect_core(path, cache):
padding.append({"segment_index": segment_index, "location": location(position, len(raw), end), "bytes_base64": encoded(raw)})
break
note = struct_parse(image.structs.Elf_Nhdr, image.stream, stream_pos=position)
note_type_id = int.from_bytes(content[position + 8:position + 12], "little")
owner_start = position + header_size
descriptor_start = owner_start + ((note.n_namesz + 3) // 4) * 4
note_end = descriptor_start + ((note.n_descsz + 3) // 4) * 4
@@ -122,10 +119,7 @@ def inspect_core(path, cache):
display = owner.split(b"\0", 1)[0].decode("utf-8", "replace") if owner else None
descriptor = content[descriptor_start:descriptor_start + note.n_descsz]
note_index = len(notes)
# pyelftools selects ET_CORE labels without owner context. Keep
# those labels only for exact CORE owners; other kinds are opaque.
note_type = note.n_type if owner == b"CORE\0" else note_type_id
notes.append({"index": note_index, "segment_index": segment_index, "type": note_type, "type_id": note_type_id,
notes.append({"index": note_index, "segment_index": segment_index, "type": note.n_type,
"location": location(position, note_end - position, end),
"owner_location": location(owner_start, len(owner), end), "owner_bytes_base64": encoded(owner), "owner_display": display,
"descriptor_location": location(descriptor_start, len(descriptor), end), "descriptor_bytes_base64": encoded(descriptor)})
@@ -133,7 +127,7 @@ def inspect_core(path, cache):
# Only the initial Linux CORE ABI is interpreted. Other owners retain exact bytes.
if owner != b"CORE\0":
continue
if note_type_id == 1:
if note.n_type == "NT_PRSTATUS":
if len(descriptor) < ctypes.sizeof(prstatus_type):
raise CoreFailure("format", "Linux " + profile_label + " NT_PRSTATUS descriptor is truncated.")
status = prstatus_type.from_buffer_copy(descriptor)
@@ -142,7 +136,7 @@ def inspect_core(path, cache):
relative = prstatus_type.pr_reg.offset + getattr(registers_type, name).offset
registers.append({"name": name, "value": hex(getattr(status.pr_reg, name)), "location": location(descriptor_start + relative, ctypes.sizeof(kind), end)})
threads.append({"note_index": note_index, "historical_pid": ctypes.c_int32(status.pr_pid).value, "recorded_current_signal": status.pr_cursig, "registers": registers})
elif note_type_id == 0x53494749:
elif note.n_type == "NT_SIGINFO":
if len(descriptor) < ctypes.sizeof(elf_siginfo_64):
raise CoreFailure("format", "Linux " + profile_label + " NT_SIGINFO descriptor is truncated.")
signal = elf_siginfo_64.from_buffer_copy(descriptor)
+66 -88
View File
@@ -1,18 +1,4 @@
import { defineConfig } from "vitepress";
import { dirname, relative, resolve, sep } from "node:path";
import { fileURLToPath } from "node:url";
import {
documentationRewrites,
documentationSections,
} from "./documentation.js";
const documentationRoot = fileURLToPath(new URL("..", import.meta.url));
const rewrites: Record<string, string> = {
...documentationRewrites,
"README.md": "documentation.md",
"specs/README.md": "specs/index.md",
"adr/README.md": "adr/index.md",
};
export default defineConfig({
title: "REA",
@@ -20,92 +6,84 @@ export default defineConfig({
"Reverse engineer binaries, applications, and runtime behavior with one CLI and MCP server.",
lang: "en-US",
base: "/rea/",
rewrites,
markdown: {
config(md) {
// Repository links resolve from their physical source. Render them from
// the stable public route instead, including links across categories.
md.core.ruler.after("inline", "documentation-source-links", (state) => {
const physicalPath =
state.env.realPath ??
(state.env.relativePath
? resolve(documentationRoot, state.env.relativePath)
: null);
if (physicalPath === null) return;
const renderedPath =
state.env.path ??
resolve(
documentationRoot,
rewrites[state.env.relativePath] ?? state.env.relativePath,
);
for (const block of state.tokens) {
for (const token of block.children ?? []) {
const attribute =
token.type === "link_open"
? "href"
: token.type === "image"
? "src"
: null;
if (attribute === null) continue;
const destination = token.attrGet(attribute);
if (
!destination ||
/^(?:[a-z][a-z\d+.-]*:|[/#])/iu.test(destination)
)
continue;
const separator = destination.search(/[?#]/u);
const path =
separator === -1 ? destination : destination.slice(0, separator);
const suffix = separator === -1 ? "" : destination.slice(separator);
const source = relative(
documentationRoot,
resolve(dirname(physicalPath), path),
)
.split(sep)
.join("/");
const rendered = relative(
dirname(renderedPath),
resolve(
documentationRoot,
source.startsWith("public/")
? source.slice("public/".length)
: (rewrites[source] ?? source),
),
)
.split(sep)
.join("/");
token.attrSet(attribute, rendered + suffix);
}
}
});
},
},
srcExclude: ["readme/**"],
themeConfig: {
nav: [
{ text: "Get started", link: "/installation" },
{ text: "Documentation", link: "/documentation" },
{ text: "Reference", link: "/mcp-contracts" },
{ text: "Roadmap", link: "/roadmap" },
],
sidebar: [
{
text: "Documentation directory",
text: "Getting started",
items: [
{ text: "All guides", link: "/documentation" },
{ text: "Authoring conventions", link: "/development/documentation" },
{ text: "Feature specifications", link: "/specs/" },
{ text: "Architecture decisions", link: "/adr/" },
{ text: "Installation and setup", link: "/installation" },
{ text: "Client configuration", link: "/client-configuration" },
{ text: "CLI and Evidence", link: "/cli" },
{ text: "Agent prompts", link: "/mcp-prompts" },
{ text: "Roadmap", link: "/roadmap" },
],
},
{
text: "Investigation guides",
items: [
{ text: "Native binaries", link: "/native-investigation" },
{ text: "Go build metadata", link: "/go-binaries" },
{ text: "Rust dependencies", link: "/rust-binaries" },
{
text: "JavaScript and Electron",
link: "/javascript-artifact-reconstruction",
},
{
text: "JavaScript workflows",
link: "/javascript-application-workflows",
},
{ text: "Browser observation", link: "/browser-observation" },
{
text: "Web network captures",
link: "/web-network-captures",
},
{ text: "Electron observation", link: "/electron-observation" },
{ text: "Electron fuses", link: "/electron-fuses" },
{ text: "Process capture", link: "/process-capture" },
{ text: ".NET assemblies", link: "/managed-code-analysis" },
{ text: "Android applications", link: "/android-analysis" },
{ text: "Apple applications", link: "/apple-application-analysis" },
{ text: "Firmware", link: "/firmware-analysis" },
{ text: "SQLite databases", link: "/sqlite-databases" },
{ text: "Packet captures", link: "/packet-captures" },
{ text: "Signal recordings", link: "/signal-recordings" },
],
},
{
text: "Provider guides",
collapsed: true,
items: [
{ text: "IDA", link: "/ida-provider" },
{ text: "Kuna", link: "/kuna-provider" },
{ text: "Ghidra on Windows", link: "/windows-ghidra-p0" },
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
{ text: "Ghidra raw binary", link: "/raw-binary" },
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
{ text: "Ghidra PowerPC", link: "/ghidra-ppc" },
{ text: "Ghidra semantics", link: "/ghidra-provider" },
],
},
{
text: "Reference and development",
collapsed: true,
items: [
{ text: "MCP contracts", link: "/mcp-contracts" },
{ text: "Tool design", link: "/tool-design" },
{ text: "Testing", link: "/testing" },
{ text: "Client verification", link: "/client-verification" },
{
text: "Reconstruction readiness",
link: "/reconstruction-readiness",
},
{ text: "Releasing", link: "/releasing" },
],
},
...documentationSections.map((section) => ({
text: section.text,
collapsed: section.text !== "Getting started",
items: section.items.map((item) => ({
text: item.text,
link: item.route,
})),
})),
],
search: { provider: "local" },
socialLinks: [
-684
View File
@@ -1,684 +0,0 @@
// Authored guide ownership: physical sources and stable public routes.
export const documentationSections = [
{
text: "Getting started",
items: [
{
text: "Installation and setup",
source: "getting-started/installation.md",
route: "/installation",
},
{
text: "Client configuration and troubleshooting",
source: "getting-started/client-configuration.md",
route: "/client-configuration",
},
{
text: "CLI and Evidence",
source: "getting-started/cli.md",
route: "/cli",
},
{
text: "Guided MCP prompts and completion",
source: "getting-started/mcp-prompts.md",
route: "/mcp-prompts",
},
],
},
{
text: "Native binaries",
items: [
{
text: "Native binary investigation",
source: "guides/native/native-investigation.md",
route: "/native-investigation",
},
{
text: "Go binary build metadata",
source: "guides/native/go-binaries.md",
route: "/go-binaries",
},
{
text: "Offline binary diagnostics",
source: "guides/native/binary-diagnostics.md",
route: "/binary-diagnostics",
},
{
text: "ELF and Mach-O initializer functions",
source: "guides/native/native-initializers.md",
route: "/native-initializers",
},
{
text: "ELF unwind function ranges",
source: "guides/native/elf-unwind-ranges.md",
route: "/elf-unwind-ranges",
},
{
text: "Linux kernel build config",
source: "guides/native/kernel-build-config.md",
route: "/kernel-build-config",
},
{
text: "Recorded crash evidence",
source: "guides/native/recorded-crashes.md",
route: "/recorded-crashes",
},
{
text: "Windows Shell Link shortcuts",
source: "guides/native/windows-shortcut.md",
route: "/windows-shortcut",
},
],
},
{
text: "JavaScript and Electron",
items: [
{
text: "JavaScript Application Graph",
source: "guides/javascript/javascript-application-graph.md",
route: "/javascript-application-graph",
},
{
text: "Cross-layer JavaScript application workflows",
source: "guides/javascript/javascript-application-workflows.md",
route: "/javascript-application-workflows",
},
{
text: "JavaScript artifact reconstruction",
source: "guides/javascript/javascript-artifact-reconstruction.md",
route: "/javascript-artifact-reconstruction",
},
{
text: "JavaScript source recovery",
source: "guides/javascript/javascript-recovery.md",
route: "/javascript-recovery",
},
{
text: "Passive Node and Electron runtime observation",
source: "guides/javascript/javascript-runtime-observation.md",
route: "/javascript-runtime-observation",
},
{
text: "JavaScript static/runtime reconciliation",
source: "guides/javascript/javascript-runtime-reconciliation.md",
route: "/javascript-runtime-reconciliation",
},
{
text: "Electron file-page observation",
source: "guides/javascript/electron-observation.md",
route: "/electron-observation",
},
{
text: "V8 code cache inspection",
source: "guides/javascript/v8-code-cache.md",
route: "/v8-code-cache",
},
],
},
{
text: "Browser and captured websites",
items: [
{
text: "Website observation with CDP",
source: "guides/web/browser-observation.md",
route: "/browser-observation",
},
{
text: "Browser scenario contract",
source: "guides/web/browser-scenario-contract.md",
route: "/browser-scenario-contract",
},
{
text: "Website runtime source attribution",
source: "guides/web/web-runtime.md",
route: "/web-runtime",
},
{
text: "Historical web network captures",
source: "guides/web/web-network-captures.md",
route: "/web-network-captures",
},
{
text: "HTTP/3 capture lab",
source: "http3-capture-lab.md",
route: "/http3-capture-lab",
},
{
text: "Captured website source locations",
source: "guides/web/web-source-location.md",
route: "/web-source-location",
},
{
text: "Captured website module relationships",
source: "guides/web/website-module-trace.md",
route: "/website-module-trace",
},
{
text: "Captured website scripts and static JavaScript analysis",
source: "guides/web/website-script-export.md",
route: "/website-script-export",
},
],
},
{
text: "Android applications",
items: [
{
text: "Static Android APK analysis",
source: "guides/android/android-analysis.md",
route: "/android-analysis",
},
{
text: "Apktool resource decoding",
source: "guides/android/apktool-resource-analysis.md",
route: "/apktool-resource-analysis",
},
{
text: "ADB device inspection and acquisition",
source: "guides/android/adb-device-analysis.md",
route: "/adb-device-analysis",
},
{
text: "Flutter build identification",
source: "guides/android/flutter-analysis.md",
route: "/flutter-analysis",
},
],
},
{
text: "Apple applications",
items: [
{
text: "Apple Application Inventory Projection",
source: "guides/apple/apple-application-analysis.md",
route: "/apple-application-analysis",
},
],
},
{
text: ".NET assemblies",
items: [
{
text: "Managed-code analysis",
source: "guides/managed/managed-code-analysis.md",
route: "/managed-code-analysis",
},
],
},
{
text: "Firmware",
items: [
{
text: "Firmware analysis",
source: "guides/firmware/firmware-analysis.md",
route: "/firmware-analysis",
},
],
},
{
text: "HarmonyOS packages",
items: [
{
text: "Static HarmonyOS package inventory",
source: "guides/harmony/harmony-analysis.md",
route: "/harmony-analysis",
},
],
},
{
text: "WebAssembly",
items: [
{
text: "Offline WebAssembly artifacts",
source: "guides/wasm/wasm-artifacts.md",
route: "/wasm-artifacts",
},
],
},
{
text: "EVM contracts",
items: [
{
text: "Offline EVM bytecode",
source: "guides/evm/evm-bytecode.md",
route: "/evm-bytecode",
},
],
},
{
text: "Databases and recorded packets",
items: [
{
text: "SQLite database snapshots",
source: "guides/data/sqlite-databases.md",
route: "/sqlite-databases",
},
{
text: "Recorded packet capture inspection",
source: "guides/data/packet-captures.md",
route: "/packet-captures",
},
{
text: "Packet capture summaries",
source: "guides/data/pcap-analysis.md",
route: "/pcap-analysis",
},
],
},
{
text: "Process capture and instrumentation",
items: [
{
text: "Process capture",
source: "guides/runtime/process-capture.md",
route: "/process-capture",
},
{
text: "Frida instrumentation",
source: "guides/runtime/frida-instrumentation.md",
route: "/frida-instrumentation",
},
],
},
{
text: "Provider setup and formats",
items: [
{
text: "Ghidra provider semantics",
source: "providers/ghidra-provider.md",
route: "/ghidra-provider",
},
{
text: "DOS MZ and COM analysis with Ghidra",
source: "providers/ghidra-dos.md",
route: "/ghidra-dos",
},
{
text: "Optional NativeAOT metadata recovery",
source: "providers/ghidra-nativeaot.md",
route: "/ghidra-nativeaot",
},
{
text: "MIPS ELF analysis through Ghidra",
source: "providers/ghidra-mips.md",
route: "/ghidra-mips",
},
{
text: "PSP static ELF analysis through Ghidra",
source: "providers/ghidra-psp.md",
route: "/ghidra-psp",
},
{
text: "RISC-V ELF analysis through Ghidra",
source: "providers/ghidra-riscv.md",
route: "/ghidra-riscv",
},
{
text: "Windows Ghidra P0",
source: "providers/windows-ghidra-p0.md",
route: "/windows-ghidra-p0",
},
{
text: "Headerless raw-binary analysis with Ghidra",
source: "providers/raw-binary.md",
route: "/raw-binary",
},
{
text: "IDA provider",
source: "providers/ida-provider.md",
route: "/ida-provider",
},
{
text: "JEB instrumentation and analysis",
source: "providers/jeb-analysis.md",
route: "/jeb-analysis",
},
],
},
{
text: "Contracts and evidence",
items: [
{
text: "MCP runtime contracts",
source: "reference/mcp-contracts.md",
route: "/mcp-contracts",
},
{
text: "Reconstruction obligation ledgers",
source: "reference/reconstruction-obligation-ledgers.md",
route: "/reconstruction-obligation-ledgers",
},
],
},
{
text: "Development and verification",
items: [
{
text: "Settings and recovery",
source: "design/limits-and-recovery.md",
route: "/design/limits-and-recovery",
},
{
text: "MCP tool design",
source: "development/tool-design.md",
route: "/tool-design",
},
{
text: "Testing REA",
source: "development/testing.md",
route: "/testing",
},
{
text: "Client and agent verification",
source: "development/client-verification.md",
route: "/client-verification",
},
{
text: "Reconstruction readiness conformance",
source: "development/reconstruction-readiness.md",
route: "/reconstruction-readiness",
},
{
text: "Releasing REA",
source: "development/releasing.md",
route: "/releasing",
},
{
text: "REA roadmap",
source: "development/roadmap.md",
route: "/roadmap",
},
],
},
{
text: "Release migrations",
items: [
{
text: "Upgrading from REA 6.1 to 6.2",
source: "migrations/migration-6.2.md",
route: "/migration-6.2",
},
],
},
{
text: "Signal recordings",
items: [
{
text: "Recorded signal inspection",
source: "guides/signals/signal-recordings.md",
route: "/signal-recordings",
},
],
},
{
text: "Additional guides",
items: [
{
text: "Offline API contract inventory",
source: "api-contracts.md",
route: "/api-contracts",
},
{
text: "Bun standalone executables",
source: "bun-standalone.md",
route: "/bun-standalone",
},
{
text: "Deno compile executables",
source: "deno-standalone.md",
route: "/deno-standalone",
},
{
text: "Capability composition and ownership",
source: "capability-composition.md",
route: "/capability-composition",
},
{
text: ".NET single-file bundles",
source: "dotnet-bundles.md",
route: "/dotnet-bundles",
},
{
text: "ESP flash images",
source: "esp-flash-image.md",
route: "/esp-flash-image",
},
{
text: "U-Boot FIT images",
source: "fit-images.md",
route: "/fit-images",
},
{
text: "Measure MCP catalog costs in Codex",
source: "mcp-catalog-evaluation.md",
route: "/mcp-catalog-evaluation",
},
{
text: "Offline Playwright traces",
source: "playwright-traces.md",
route: "/playwright-traces",
},
{
text: "Testing lanes and provider verification",
source: "testing-lanes.md",
route: "/testing-lanes",
},
{
text: "UF2 and DfuSe images",
source: "uf2-dfuse.md",
route: "/uf2-dfuse",
},
{
text: "WeChat mini-program wxapkg archives",
source: "wxapkg.md",
route: "/wxapkg",
},
{
text: "Binary hardening",
source: "binary-hardening.md",
route: "/binary-hardening",
},
{
text: "Desktop application capture",
source: "desktop-application-capture.md",
route: "/desktop-application-capture",
},
{
text: "Desktop application export",
source: "desktop-application-export.md",
route: "/desktop-application-export",
},
{
text: "Desktop application state",
source: "desktop-application-state.md",
route: "/desktop-application-state",
},
{
text: "Desktop application tasks",
source: "desktop-application-tasks.md",
route: "/desktop-application-tasks",
},
{
text: "Desktop build computer",
source: "desktop-build-computer.md",
route: "/desktop-build-computer",
},
{
text: "Desktop build witness",
source: "desktop-build-witness.md",
route: "/desktop-build-witness",
},
{
text: "iNES containers",
source: "ines-containers.md",
route: "/ines-containers",
},
{
text: "Build RL environments with REA",
source: "agent-environments-quickstart.md",
route: "/agent-environments-quickstart",
},
{
text: "Agent environments from recorded captures",
source: "agent-environments.md",
route: "/agent-environments",
},
{
text: "APK signature schemes",
source: "apk-signature.md",
route: "/apk-signature",
},
{
text: "Apple IPS crash reports",
source: "apple-ips.md",
route: "/apple-ips",
},
{
text: "Static analysis in GitHub Actions",
source: "ci-analysis.md",
route: "/ci-analysis",
},
{
text: "Desktop installation directories",
source: "desktop-applications.md",
route: "/desktop-applications",
},
{
text: "Flattened devicetree inspection",
source: "devicetree.md",
route: "/devicetree",
},
{
text: "Electron fuse state",
source: "electron-fuses.md",
route: "/electron-fuses",
},
{ text: "ELF note tables", source: "elf-notes.md", route: "/elf-notes" },
{
text: "Erlang BEAM files",
source: "erlang-beam.md",
route: "/erlang-beam",
},
{ text: "ESP images", source: "esp-image.md", route: "/esp-image" },
{
text: "GameCube DOL analysis through Ghidra",
source: "ghidra-dol.md",
route: "/ghidra-dol",
},
{
text: "PowerPC ELF analysis through Ghidra",
source: "ghidra-ppc.md",
route: "/ghidra-ppc",
},
{
text: "Godot PCK archives",
source: "godot-pck.md",
route: "/godot-pck",
},
{
text: "Static CUDA and HIP code objects",
source: "gpu-code-objects.md",
route: "/gpu-code-objects",
},
{
text: "Intel HEX and S-record images",
source: "hex-records.md",
route: "/hex-records",
},
{
text: "IL2CPP metadata",
source: "il2cpp-metadata.md",
route: "/il2cpp-metadata",
},
{
text: "Static JavaScript route inventory",
source: "javascript-route-inventory.md",
route: "/javascript-route-inventory",
},
{
text: "Linux kernel modules",
source: "kernel-modules.md",
route: "/kernel-modules",
},
{
text: "Kuna provider",
source: "kuna-provider.md",
route: "/kuna-provider",
},
{
text: "Model file declarations",
source: "model-artifacts.md",
route: "/model-artifacts",
},
{
text: "Match native functions across builds",
source: "native-function-matching.md",
route: "/native-function-matching",
},
{
text: "Node.js single executable applications",
source: "node-sea.md",
route: "/node-sea",
},
{
text: "PE Authenticode certificate tables",
source: "pe-certificates.md",
route: "/pe-certificates",
},
{
text: "PE import and delay-load tables",
source: "pe-imports.md",
route: "/pe-imports",
},
{
text: "PE TLS callbacks and load configuration",
source: "pe-load-config.md",
route: "/pe-load-config",
},
{
text: "Python packaged applications",
source: "python-packaged-applications.md",
route: "/python-packaged-applications",
},
{
text: "CPython pyc inspection",
source: "python-pyc.md",
route: "/python-pyc",
},
{
text: "Reconstruction coverage",
source: "reconstruction-coverage.md",
route: "/reconstruction-coverage",
},
{
text: "Reverse-engineering tool integrations",
source: "reverse-engineering-tools.md",
route: "/reverse-engineering-tools",
},
{
text: "Rust binary dependency metadata",
source: "rust-binaries.md",
route: "/rust-binaries",
},
{
text: "U-Boot legacy uImage containers",
source: "uimage.md",
route: "/uimage",
},
{
text: "UnityFS AssetBundle archives",
source: "unityfs.md",
route: "/unityfs",
},
],
},
];
export const documentationRewrites = Object.fromEntries(
documentationSections.flatMap((section) =>
section.items.flatMap((item) => {
const target = `${item.route.slice(1)}.md`;
return item.source === target ? [] : [[item.source, target]];
}),
),
);
-205
View File
@@ -1,205 +0,0 @@
# Documentation directory
REA documentation is organized by reader task and document owner. This directory
links every authored guide; the [site](https://rea.tools/) also serves separate
product pages. The VitePress documentation source is this `docs/` tree.
## Getting started
- [Installation and setup](getting-started/installation.md)
- [Client configuration and troubleshooting](getting-started/client-configuration.md)
- [CLI and Evidence](getting-started/cli.md)
- [Guided MCP prompts and completion](getting-started/mcp-prompts.md)
## Native binaries
- [Native binary investigation](guides/native/native-investigation.md)
- [Go binary build metadata](guides/native/go-binaries.md)
- [Offline binary diagnostics](guides/native/binary-diagnostics.md)
- [ELF and Mach-O initializer functions](guides/native/native-initializers.md)
- [ELF unwind function ranges](guides/native/elf-unwind-ranges.md)
- [Linux kernel build config](guides/native/kernel-build-config.md)
- [Recorded crash evidence](guides/native/recorded-crashes.md)
- [Windows Shell Link shortcuts](guides/native/windows-shortcut.md)
## JavaScript and Electron
- [JavaScript Application Graph](guides/javascript/javascript-application-graph.md)
- [Cross-layer JavaScript application workflows](guides/javascript/javascript-application-workflows.md)
- [JavaScript artifact reconstruction](guides/javascript/javascript-artifact-reconstruction.md)
- [JavaScript source recovery](guides/javascript/javascript-recovery.md)
- [Passive Node and Electron runtime observation](guides/javascript/javascript-runtime-observation.md)
- [JavaScript static/runtime reconciliation](guides/javascript/javascript-runtime-reconciliation.md)
- [Electron file-page observation](guides/javascript/electron-observation.md)
- [V8 code cache inspection](guides/javascript/v8-code-cache.md)
## Browser and captured websites
- [Website observation with CDP](guides/web/browser-observation.md)
- [Browser scenario contract](guides/web/browser-scenario-contract.md)
- [Website runtime source attribution](guides/web/web-runtime.md)
- [Historical web network captures](guides/web/web-network-captures.md)
- [HTTP/3 capture lab](http3-capture-lab.md)
- [Captured website source locations](guides/web/web-source-location.md)
- [Captured website module relationships](guides/web/website-module-trace.md)
- [Captured website scripts and static JavaScript analysis](guides/web/website-script-export.md)
## Android applications
- [Static Android APK analysis](guides/android/android-analysis.md)
- [Apktool resource decoding](guides/android/apktool-resource-analysis.md)
- [ADB device inspection and acquisition](guides/android/adb-device-analysis.md)
- [Flutter build identification](guides/android/flutter-analysis.md)
## Apple applications
- [Apple Application Inventory Projection](guides/apple/apple-application-analysis.md)
## .NET assemblies
- [Managed-code analysis](guides/managed/managed-code-analysis.md)
## Firmware
- [Firmware analysis](guides/firmware/firmware-analysis.md)
## HarmonyOS packages
- [Static HarmonyOS package inventory](guides/harmony/harmony-analysis.md)
## WebAssembly
- [Offline WebAssembly artifacts](guides/wasm/wasm-artifacts.md)
## EVM contracts
- [Offline EVM bytecode](guides/evm/evm-bytecode.md)
## Databases and recorded packets
- [SQLite database snapshots](guides/data/sqlite-databases.md)
- [Recorded packet capture inspection](guides/data/packet-captures.md)
- [Packet capture summaries](guides/data/pcap-analysis.md)
## Process capture and instrumentation
- [Process capture](guides/runtime/process-capture.md)
- [Frida instrumentation](guides/runtime/frida-instrumentation.md)
## Signal recordings
- [Recorded signal inspection](guides/signals/signal-recordings.md)
## Provider setup and formats
- [Ghidra provider semantics](providers/ghidra-provider.md)
- [DOS MZ and COM analysis with Ghidra](providers/ghidra-dos.md)
- [Optional NativeAOT metadata recovery](providers/ghidra-nativeaot.md)
- [MIPS ELF analysis through Ghidra](providers/ghidra-mips.md)
- [PSP static ELF analysis through Ghidra](providers/ghidra-psp.md)
- [RISC-V ELF analysis through Ghidra](providers/ghidra-riscv.md)
- [Windows Ghidra P0](providers/windows-ghidra-p0.md)
- [Headerless raw-binary analysis with Ghidra](providers/raw-binary.md)
- [IDA provider](providers/ida-provider.md)
- [JEB instrumentation and analysis](providers/jeb-analysis.md)
## Contracts and evidence
- [MCP runtime contracts](reference/mcp-contracts.md)
- [Reconstruction obligation ledgers](reference/reconstruction-obligation-ledgers.md)
## Development and verification
- [Settings and recovery](design/limits-and-recovery.md)
- [Flutter budget calibration](verification/flutter-budget-calibration.md)
- [Source-map budget calibration](verification/source-map-budget-calibration.md)
- [Source-location delivery qualification](verification/source-location-delivery.md)
- [JavaScript precision calibration](verification/javascript-precision-calibration.md)
- [MCP tool design](development/tool-design.md)
- [Testing REA](development/testing.md)
- [Client and agent verification](development/client-verification.md)
- [Reconstruction readiness conformance](development/reconstruction-readiness.md)
- [Releasing REA](development/releasing.md)
- [REA roadmap](development/roadmap.md)
## Release migrations
- [Upgrading from REA 6.1 to 6.2](migrations/migration-6.2.md)
## Architecture and authoring
- [Architecture map](https://github.com/morluto/rea/blob/main/docs/architecture/architecture.mermaid)
- [Documentation ownership and contribution model](development/documentation.md)
- [Feature specifications](specs/README.md)
- [Architectural decision records](adr/README.md)
## Machine-owned documentation
| Path | Owner and update command |
| ----------------------------------------- | ----------------------------------------------------------------------------------------- |
| `public/product-catalog.json` | Ignored contract projection; `npm run docs:generate` |
| `verification/managed-conformance-*.json` | Ignored managed conformance projections; `npm run evidence:generate` |
| `error-contract.schema.json` | Tracked schema from contracts; `npm run docs:error-schema` |
| `user-visible-errors.csv` | Authored error classification inventory; preserve its schema |
| `releases/<version>.contributions.json` | Durable release contribution record; follow the [release guide](development/releasing.md) |
| `assets/` | Shared authored images referenced by guides |
Do not edit generated catalogs or evidence to change product facts. Change their
canonical contracts or producer and regenerate. Historical release records retain
the exact commit messages and references from the release they describe.
## Additional guides
- [Offline API contract inventory](api-contracts.md)
- [Bun standalone executables](bun-standalone.md)
- [Deno compile executables](deno-standalone.md)
- [Capability composition and ownership](capability-composition.md)
- [.NET single-file bundles](dotnet-bundles.md)
- [ESP flash images](esp-flash-image.md)
- [U-Boot FIT images](fit-images.md)
- [Measure MCP catalog costs in Codex](mcp-catalog-evaluation.md)
- [Offline Playwright traces](playwright-traces.md)
- [Testing lanes and provider verification](testing-lanes.md)
- [UF2 and DfuSe images](uf2-dfuse.md)
- [WeChat mini-program wxapkg archives](wxapkg.md)
- [Binary hardening](binary-hardening.md)
- [Desktop application capture](desktop-application-capture.md)
- [Desktop application export](desktop-application-export.md)
- [Desktop application state](desktop-application-state.md)
- [Desktop application tasks](desktop-application-tasks.md)
- [Desktop build computer](desktop-build-computer.md)
- [Desktop build witness](desktop-build-witness.md)
- [iNES containers](ines-containers.md)
- [Build RL environments with REA](agent-environments-quickstart.md)
- [Agent environments from recorded captures](agent-environments.md)
- [APK signature schemes](apk-signature.md)
- [Apple IPS crash reports](apple-ips.md)
- [Static analysis in GitHub Actions](ci-analysis.md)
- [Desktop installation directories](desktop-applications.md)
- [Flattened devicetree inspection](devicetree.md)
- [Electron fuse state](electron-fuses.md)
- [ELF note tables](elf-notes.md)
- [Erlang BEAM files](erlang-beam.md)
- [ESP images](esp-image.md)
- [GameCube DOL analysis through Ghidra](ghidra-dol.md)
- [PowerPC ELF analysis through Ghidra](ghidra-ppc.md)
- [Godot PCK archives](godot-pck.md)
- [Static CUDA and HIP code objects](gpu-code-objects.md)
- [Intel HEX and S-record images](hex-records.md)
- [IL2CPP metadata](il2cpp-metadata.md)
- [Static JavaScript route inventory](javascript-route-inventory.md)
- [Linux kernel modules](kernel-modules.md)
- [Kuna provider](kuna-provider.md)
- [Model file declarations](model-artifacts.md)
- [Match native functions across builds](native-function-matching.md)
- [Node.js single executable applications](node-sea.md)
- [PE Authenticode certificate tables](pe-certificates.md)
- [PE import and delay-load tables](pe-imports.md)
- [PE TLS callbacks and load configuration](pe-load-config.md)
- [Python packaged applications](python-packaged-applications.md)
- [CPython pyc inspection](python-pyc.md)
- [Reconstruction coverage](reconstruction-coverage.md)
- [Reverse-engineering tool integrations](reverse-engineering-tools.md)
- [Rust binary dependency metadata](rust-binaries.md)
- [U-Boot legacy uImage containers](uimage.md)
- [UnityFS AssetBundle archives](unityfs.md)
+214
View File
@@ -0,0 +1,214 @@
# ADB device inspection and acquisition
REA inspects connected Android devices, reads device state, and transfers
files and packages through the caller's `adb` binary. Emulators, USB
devices, and network devices participate alike: REA neither starts nor
manages them and never installs platform tools. Every operation composes a
fixed adb argument vector — caller values occupy positional arguments only
and are quoted when adb forwards them to the device shell.
The adb client may start its local server on first device query; that is adb
client behavior, recorded in the tool result when observed.
`inspect_adb_client` probes only the binary and never contacts the server.
## Supply tools
Use any Android platform-tools `adb` (verified with 34.0.5). Select it
through the environment:
```sh
# Optional: absolute path to the adb binary; without it, adb resolves from PATH.
export REA_ADB_PATH=/absolute/path/to/adb
```
REA never installs the Android SDK, an emulator, or a system image.
Readiness is `rea inspect-adb-client`: it runs `adb version` and reports the
resolved binary, its version, and where it was selected from. A missing
binary reports `capability_unavailable` with the `REA_ADB_PATH` remediation
rather than attempting an installation.
The `binary_session` inventory uses the same bounded client-version probe; it
does not query devices, start the adb server, or establish that a requested
serial is reachable. Device state is checked by each device operation.
## Answer an analyst question
```sh
# Device identity
rea inspect-adb-client
rea list-adb-devices
rea inspect-adb-device emulator-5554
rea inspect-adb-display emulator-5554
rea list-adb-features emulator-5554
rea list-adb-services emulator-5554
# Find and acquire an app (the easy-get flow)
rea resolve-adb-packages emulator-5554 whatsapp
rea list-adb-packages emulator-5554 --scope third_party
rea inspect-adb-package emulator-5554 com.whatsapp
rea pull-adb-package emulator-5554 com.whatsapp --output-directory /tmp/acq
rea inspect-android-package /tmp/acq/com.whatsapp/base.apk
# Device state and files
rea list-adb-processes emulator-5554
rea inspect-adb-window emulator-5554
rea read-adb-setting emulator-5554 global adb_enabled
rea read-adb-logcat emulator-5554 --count 200
rea list-adb-directory emulator-5554 /data/data/com.whatsapp
rea pull-adb-file emulator-5554 /data/local/tmp/notes.txt --output-directory /tmp/acq
rea push-adb-file emulator-5554 /tools/frida-server /data/local/tmp/frida-server
rea capture-adb-screen emulator-5554 --output-directory /tmp/shots
rea collect-adb-bugreport emulator-5554 --output-directory /tmp/bugs
# Package lifecycle and dynamic analysis (device-mutating)
rea install-adb-package emulator-5554 /tmp/app.apk --replace
rea start-adb-app emulator-5554 com.example.app
rea read-adb-logcat emulator-5554 --pid 12345
rea start-adb-activity emulator-5554 android.intent.action.VIEW --data-uri https://example.com
rea stop-adb-app emulator-5554 com.example.app
rea uninstall-adb-package emulator-5554 com.example.app
```
Equivalent MCP methods use the snake_case names (`list_adb_devices`,
`pull_adb_package`, `resolve_adb_packages`, …). Device operations take
`serial` exactly as `adb devices` reports it. The pulled APK set feeds the
existing analysis families directly, for example
`rea inspect-android-package /tmp/acq/<package>/base.apk`.
## Operations
| Tool | Kind | What it does |
| ----------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `inspect_adb_client` | read | Resolved binary, version, install path; never contacts the server |
| `list_adb_devices` | read | Serial, state, transport, model metadata, inferred emulator/physical kind with basis |
| `inspect_adb_device` | read | Fixed `getprop` whitelist: build, SDK, patch, fingerprint, product, ABIs, debuggable, `ro.kernel.qemu` |
| `inspect_adb_display` | read | `wm size` and `wm density` as declared |
| `list_adb_features` | read | `pm list features` (hardware/software features, hex GL versions) |
| `list_adb_services` | read | `service list` binder services and interfaces |
| `list_adb_processes` | read | `ps -A`: user, pid, ppid, rss, state, name |
| `inspect_adb_window` | read | `dumpsys window` focus declarations (what is on screen) |
| `read_adb_setting` | read | One `settings get` value; device-side null is reported as such |
| `read_adb_logcat` | read | Bounded `logcat -d -t` dump for one buffer, optional pid filter |
| `list_adb_packages` | read | `pm list packages -f` with scope; base APK device paths |
| `resolve_adb_packages` | read | Case-insensitive package-name substring search with exact-match flag |
| `inspect_adb_package` | read | `dumpsys package` projection: versions, install times, installer, flags, permission count |
| `list_adb_directory` | read | NUL-framed immediate children, including hidden entries: kinds, permissions, owner, group, sizes, symlink targets |
| `pull_adb_package` | read | Complete split APK set from `pm path`, each file digested |
| `pull_adb_file` | read | One absolute device path pulled and digested |
| `capture_adb_screen` | read | One `screencap -p` frame (exec-out, no device-side file) with dimensions |
| `collect_adb_bugreport` | read | Device-generated bugreport zip, digested |
| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); checks for existing targets and refuses them without explicit overwrite (the check is not atomic); compares digests when device `sha256sum` is available |
| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves its Success/Failure verdict, records `replace_requested`, and leaves actual replacement `null` because prior package state is not observed |
| `uninstall_adb_package` | **mutates device** | Removes a package and its data for user 0 |
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
For `install_adb_package`, `replace_requested` records whether `-r` was sent.
The `replaced` result is `null`: the successful install response does not say
whether the package was installed fresh or replaced, and REA does not query the
device first. The install verdict and host APK digest remain available.
Push and install consume a private snapshot of the selected local input. The
reported size and SHA-256 identify those consumed bytes; `local_path` and
`apk_path` retain the original input path, whose contents may change later.
Snapshot creation needs temporary disk space proportional to the input size.
Failed snapshot cleanup remains owned and is retried before later operations
and during shutdown.
## Interpret the result
- **Directory names retain whitespace and newlines.** Symlink targets do too,
including trailing newlines. Records containing invalid UTF-8 remain as
labelled hex in `unparsed_lines` and make directory coverage partial.
- **Inferences carry their basis.** Device `kind` reports
`kind_basis` (serial prefix or USB transport); APK `role` reports
`role_basis: file_name`; `emulator_observed` is the build's own
`ro.kernel.qemu` value, `null` when absent.
- **Device paths are not host paths.** `base_apk_device_path` values live on
the device filesystem. Only pulls and captures produce host-local paths,
each paired with byte size and SHA-256 digest. A pulled file's host name is
the device path's final POSIX segment. REA preserves the original device
path and filename; a missing final segment or `.`/`..` is refused before
output creation. Windows hosts follow [Windows filename rules](https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file),
while POSIX hosts retain valid POSIX filenames such as names with spaces or
colons. A package pull checks reported names against the destination
filesystem before pulling, and refuses names that would share a local path.
- **Package path refusals are diagnosable.** Refusals from `pull_adb_package`
return `unreadable_output` with `retryable: false`, keep the `pm path` output
in `details.captured_output`, and name the refusal in `details.reason`.
`unsafe_device_path` lists refused entries in
`details.diagnostics.rejected_paths` as `{device_path, reason}`;
`duplicate_local_destination` reports the shared `file_name` and both
`device_paths`; `local_destination_unavailable` reports the `device_path`,
`file_name`, and filesystem `error_code`. Diagnostics retain raw device
strings, while the caller-facing message escapes invisible and bidirectional
characters so they cannot hide text or reorder the path.
- **Splits are part of the answer.** `pm path` enumerates the base APK and
every installed split; pulls cover the whole set. System packages whose
APKs keep other names report the `unknown` role rather than guessing.
- **Partial results keep facts.** Failed individual pulls return successful
artifacts with `coverage: "partial"` and per-file `failures`. Listings
that contain unrecognized lines report them as `unparsed_lines` and mark
partial coverage instead of dropping rows.
- **Mutations are explicit.** `push_adb_file`, `install_adb_package`, and
`uninstall_adb_package` declare `mutatesTarget` and never run implicitly:
pushes check for existing device files before transfer and refuse them
unless `overwrite` is set. The check is not atomic, so a different process
can create the path before `adb push` writes it. When available, the device's
`sha256sum` verifies the transfer; installs preserve the
device's `Failure [...]` reason verbatim.
- **Producer identity stays attached.** Every result embeds the producing
adb client (`binary_path`, `path_source`, `version`, `installed_path`).
## Resource and lifecycle limits
- Caller values reach adb only as positional arguments in fixed vectors;
no tool accepts an arbitrary shell command line, `input`, or `monkey`.
The app/activity lifecycle tools use `am start -W` and `am force-stop`.
- Per-invocation budgets: client/devices 10-20 s, shell reads 20-60 s with
64 KiB-8 MiB capture caps (logcat 4 MiB, screen 32 MiB), transfers and
bugreports 600 s. Exceeded budgets report `resource_constraint`; stopped
commands report `provider_timeout`.
- Pull and capture outputs are additive: the destination must not already
exist. Re-running a completed pull refuses the existing directory instead
of overwriting evidence.
- Cancellation during a package pull preserves caller-selected output. The
error includes partial observations identifying completed artifacts and the
interrupted local path; remaining APKs were not attempted.
- Screen capture checks cancellation before publishing its exclusive output.
If cancellation arrives after publication, the error retains the committed
file's path, digest and dimensions; the file remains available as evidence.
- Bugreports collect into isolated staging and publish with exclusive creation;
filename collisions select the next numeric suffix. If publication fails, the
completed archive remains in staging. The error's partial observation gives
its recoverable `local_path`, the original `adb_reported_path`, and an unknown
digest status. Later operations and shutdown preserve this retained evidence.
- Failed package acquisition rollback retains its directory identity and retries
cleanup before later work and during shutdown. Cleanup refuses a replacement
directory. Terminal shutdown stops admission and drains active operations.
- The provider holds no long-lived processes; every invocation is one
bounded adb run. The adb server belongs to the caller.
## Related capability and follow-up lanes
UI interaction beyond intent starts (tapping, typing, monkey fuzzing, screen
recording) remains a separate lane: candidates such as
[mobile-device-mcp](https://github.com/srmorete/mobile-device-mcp) install
driver software on the device and pair with the Frida instrumentation lane
(#1359) behind explicit mutation-authority contracts. Network transports
beyond already-connected devices (`adb connect`) are not exposed yet.
## Real verification
`npm run verify:adb` exercises every operation against a live device. It
requires `adb` (or `REA_ADB_PATH`) and any attached emulator or phone;
`--serial` selects one when several are attached, `--pull PACKAGE` pulls a
real package and verifies the returned digests against the files on disk,
and `--install-apk PATH` drives a full install → resolve → uninstall
lifecycle with a lane-owned probe.
Record: verified against adb 34.0.5 on Linux against an Android 14
(API 34) x86_64 emulator — all operations above, a 224-package inventory,
287 services, 92 features, a real two-APK split set pulled with byte-exact
digests, a verified push/pull roundtrip, a 1.3 MB screen capture, and the
complete package lifecycle; see `docs/testing.md` for the lane summary.
Loaded 100 of 7458 files, more files were not shown because too many files have changed in this diff. Show more