mirror of
https://github.com/morluto/rea.git
synced 2026-11-11 03:31:35 -10:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
11940ac24d | ||
|
|
4a3bc37b76 | ||
|
|
a16f5f4b5d | ||
|
|
8a2eb4ab61 | ||
|
|
d36ba58971 | ||
|
|
2d80427422 | ||
|
|
94dd7b72f8 | ||
|
|
20aeaf110f | ||
|
|
15d08235e1 | ||
|
|
e9f12d212c | ||
|
|
12a916b255 | ||
|
|
c80e6cd845 | ||
|
|
ca268ead0a | ||
|
|
073b4dcc41 | ||
|
|
87cd7070b6 | ||
|
|
3dced80300 | ||
|
|
a77e02788e | ||
|
|
47704ab093 | ||
|
|
930c56d670 | ||
|
|
cb17828dd1 | ||
|
|
d9537fc75c | ||
|
|
34025d2fcd | ||
|
|
98f796c991 | ||
|
|
389b69d097 | ||
|
|
913c9fe194 | ||
|
|
3eb502ae5d | ||
|
|
b4d501e714 | ||
|
|
50740bb05e | ||
|
|
fd9d362d98 | ||
|
|
2f87d3edb8 | ||
|
|
01707fcbde | ||
|
|
5b5990f7a9 | ||
|
|
61e3eb3ca7 | ||
|
|
a14a0c20ed | ||
|
|
d1895841a4 | ||
|
|
f7d564f436 | ||
|
|
0ece957589 | ||
|
|
66acbb3421 | ||
|
|
cb048c3ed5 | ||
|
|
dc5527b0bc | ||
|
|
8052f8daf0 | ||
|
|
3c0e9d8a02 | ||
|
|
01e8c6447a | ||
|
|
8cc2a1ae6e | ||
|
|
c71aded3ee | ||
|
|
b00138be64 | ||
|
|
9f5f9c1289 | ||
|
|
a896e35c47 | ||
|
|
37d2d207d2 | ||
|
|
67f74c5fb9 | ||
|
|
83f0a4d238 | ||
|
|
654feeef9c | ||
|
|
eff195d6aa | ||
|
|
13abfdadff | ||
|
|
2bb266b39b | ||
|
|
fcf8b99a77 | ||
|
|
7c0a658cb4 | ||
|
|
d3eb3f6478 | ||
|
|
b4654b493d | ||
|
|
65765eb495 | ||
|
|
46d79ecd78 | ||
|
|
e5fe737677 | ||
|
|
6151e13835 | ||
|
|
dd928466cb | ||
|
|
4d0f6d9b42 | ||
|
|
f6e31acd74 | ||
|
|
59e36cf86e | ||
|
|
e504e09aeb | ||
|
|
ea00ca2d46 | ||
|
|
aece2e7e16 | ||
|
|
1c7d7b6cc3 | ||
|
|
d54a41152e | ||
|
|
9e79188ba8 | ||
|
|
e7c4a26316 | ||
|
|
12a32d934a | ||
|
|
38a0dafa76 | ||
|
|
df25c923c4 | ||
|
|
3896f72552 | ||
|
|
44ef32c1c0 | ||
|
|
5b02119b69 | ||
|
|
7f0555b75e | ||
|
|
4812843e5b | ||
|
|
e55ce72d9d | ||
|
|
34d844dcad | ||
|
|
98bc82f1e1 | ||
|
|
f2e73ca8b6 | ||
|
|
876bc145a7 | ||
|
|
bb200646e5 | ||
|
|
17b91a9853 | ||
|
|
fcd364069a | ||
|
|
e9e106f580 | ||
|
|
fcc353518a | ||
|
|
2b570ae194 | ||
|
|
66498b4f7c | ||
|
|
e6114e81dc | ||
|
|
ecf6ac9f8d | ||
|
|
550b7df262 | ||
|
|
ca7c263dd6 | ||
|
|
2e67d57d0f | ||
|
|
81d4f6b42a | ||
|
|
dd522bf9a7 | ||
|
|
55c746afaf | ||
|
|
08d943ea29 | ||
|
|
27ced7bdef | ||
|
|
979552aa5c | ||
|
|
36334402de | ||
|
|
ba4fb6a806 | ||
|
|
4073cfc932 | ||
|
|
827b808248 | ||
|
|
4de8020745 | ||
|
|
9cd1eadfb7 | ||
|
|
8c40ba78de | ||
|
|
4a02ca95de | ||
|
|
c7a4869eb9 | ||
|
|
528c46ae92 | ||
|
|
626c705c99 | ||
|
|
c9e7aaca27 | ||
|
|
914363fbfb | ||
|
|
77100250de | ||
|
|
a7ec7271d6 | ||
|
|
a08e40b924 | ||
|
|
1f2a869e5b | ||
|
|
8e11a430cf | ||
|
|
ca70cdfcc0 | ||
|
|
ada8fcfe82 | ||
|
|
30f96aef39 | ||
|
|
b94d811500 | ||
|
|
2b1f725221 | ||
|
|
91456101f1 | ||
|
|
a2aea3f4c6 | ||
|
|
19f7eef513 | ||
|
|
43588f07ab | ||
|
|
e8fc501e06 | ||
|
|
42a02b846d | ||
|
|
10a5a60000 | ||
|
|
310993d4cb | ||
|
|
160d2ecdb5 | ||
|
|
1de3298eeb | ||
|
|
ff86e19e23 | ||
|
|
574aca127f | ||
|
|
1544fe8803 | ||
|
|
e33a207749 | ||
|
|
0e88049d59 | ||
|
|
6d8c71f8b3 | ||
|
|
4e96d64189 | ||
|
|
04e9654e11 | ||
|
|
4546673eec | ||
|
|
b99a7b6a21 | ||
|
|
5934ba1bf9 | ||
|
|
b3bab01b22 | ||
|
|
428590e9f2 | ||
|
|
1e80a96094 | ||
|
|
abc4f0f8b2 | ||
|
|
5c2917266d | ||
|
|
1371ba1e07 | ||
|
|
faf42f8f13 | ||
|
|
bf671f20a2 | ||
|
|
6de105bc60 | ||
|
|
7aaff2e4e4 | ||
|
|
a54e848302 | ||
|
|
6a2b690440 | ||
|
|
bae4ec0cda | ||
|
|
a8cd7cadc0 | ||
|
|
22a3ba263e | ||
|
|
09f58bf044 | ||
|
|
1a8a9b749a | ||
|
|
1f20c57627 | ||
|
|
c51ac43f9c | ||
|
|
e5a92d1848 | ||
|
|
710c353bff | ||
|
|
96086e9c1f | ||
|
|
f8ed4d16a0 | ||
|
|
699cb72c48 | ||
|
|
411804afbf | ||
|
|
81c09773be | ||
|
|
25d46f3a58 | ||
|
|
75871ebb8b | ||
|
|
40239b31e7 | ||
|
|
ffac2e0c99 | ||
|
|
fc9585a44f | ||
|
|
7771fb7123 | ||
|
|
2847464189 | ||
|
|
58edc31638 | ||
|
|
114b57678f | ||
|
|
4a8b157a28 | ||
|
|
f3629bae94 | ||
|
|
48ee3f02fd | ||
|
|
6e8cd0bc5e | ||
|
|
3c54f3ce4e | ||
|
|
55f344a77b | ||
|
|
01a7e2c3b7 | ||
|
|
2e34aa0433 | ||
|
|
c0b3c18b8a | ||
|
|
afed309f0f | ||
|
|
a3ff69f5ba | ||
|
|
a44d457745 | ||
|
|
2be301f894 | ||
|
|
73bc95d23f | ||
|
|
a18867e623 | ||
|
|
d3899717b2 | ||
|
|
755d58d29a | ||
|
|
b725586fbf | ||
|
|
e2bdfcee75 | ||
|
|
647ebc2645 | ||
|
|
5d0391b06e | ||
|
|
3eae278da5 | ||
|
|
133a849b2e | ||
|
|
a6e02657e6 | ||
|
|
72b07c58d4 | ||
|
|
4d711acffd | ||
|
|
5821506eb2 | ||
|
|
3c736dfe11 | ||
|
|
82abd9e6ca | ||
|
|
29b41a7d94 | ||
|
|
41bd808c1f | ||
|
|
847d59c947 | ||
|
|
d39699ad0c | ||
|
|
281db61890 | ||
|
|
2885347cde | ||
|
|
a6a2527ade | ||
|
|
f8fc572e4a | ||
|
|
ab9e5fa204 | ||
|
|
8d45cb45c4 | ||
|
|
7712e3185e | ||
|
|
466149169e | ||
|
|
1c2626edbd | ||
|
|
b16d18f198 | ||
|
|
14030e2f11 | ||
|
|
9505c73824 | ||
|
|
1a446cdade | ||
|
|
1ef7f02cae | ||
|
|
7b86474be7 | ||
|
|
5d10337709 | ||
|
|
8eadb872f2 | ||
|
|
f170175ec8 | ||
|
|
fbe09f84df | ||
|
|
ebb480ba22 | ||
|
|
da1f029c2f | ||
|
|
4db9c166ef | ||
|
|
29fd6a7d03 | ||
|
|
d7338876d6 | ||
|
|
9df3f7174f | ||
|
|
79e476204f | ||
|
|
9c95218374 | ||
|
|
d9d2641981 | ||
|
|
e2585aa7ce | ||
|
|
1d1c7f98ff | ||
|
|
b1f1b3eff3 | ||
|
|
c1d0f2bb45 | ||
|
|
04de5f3200 | ||
|
|
fe791ff159 | ||
|
|
4082375e73 | ||
|
|
dcee59f273 | ||
|
|
31018cf037 | ||
|
|
17f0c6944c | ||
|
|
360cd6f438 | ||
|
|
b129d39ac9 | ||
|
|
fbb785293b | ||
|
|
1b8a7ee7f6 | ||
|
|
86b1b1431a | ||
|
|
d5e2d59eb6 | ||
|
|
03e43c0f1b | ||
|
|
724211b468 | ||
|
|
fe1a5b1498 | ||
|
|
fc3f18b748 | ||
|
|
0a66fb2acf | ||
|
|
56d2ba36b8 | ||
|
|
0ea642843a | ||
|
|
0c37c22d51 | ||
|
|
b32da4aefa | ||
|
|
c8a52fd478 | ||
|
|
67f402fb81 | ||
|
|
1ef28be5bc | ||
|
|
de0c50ead7 | ||
|
|
fe4105187d | ||
|
|
b05f472592 | ||
|
|
38286d4bc3 | ||
|
|
15e3a73ba6 | ||
|
|
38dc9d246b | ||
|
|
6736b78693 | ||
|
|
b179755e4f | ||
|
|
42f61205c3 | ||
|
|
b4c8799123 | ||
|
|
fd584d4495 | ||
|
|
3c4be703cd | ||
|
|
439c197c3f | ||
|
|
a8e815d664 | ||
|
|
c1e3d7dc6e | ||
|
|
8661bb38d5 | ||
|
|
0ec73a8102 | ||
|
|
93dc8d044d | ||
|
|
6377a1831f | ||
|
|
a9882a58b2 | ||
|
|
25f50eea02 | ||
|
|
b899c35419 | ||
|
|
a4b4957346 | ||
|
|
ddb8abf855 | ||
|
|
99e79026c5 | ||
|
|
b4c3a028fe | ||
|
|
5e4055b39e | ||
|
|
5b34c4d3f5 | ||
|
|
db6400d775 | ||
|
|
dc7594e85f | ||
|
|
3345019fe1 | ||
|
|
9dfec80e93 | ||
|
|
005435ca78 | ||
|
|
dbfab4a65d | ||
|
|
b2f3dc81fa | ||
|
|
a4da4e7230 | ||
|
|
433fd9486d | ||
|
|
94ca4b656a | ||
|
|
d2f9b8b7de | ||
|
|
cb4e5cfbff | ||
|
|
55b6d2ca20 | ||
|
|
95ebd38d3f | ||
|
|
d4a7c20e04 | ||
|
|
27edc5c83f | ||
|
|
c52a9b9d79 | ||
|
|
8dfc47b107 | ||
|
|
e9b6c87c1c | ||
|
|
387bcb795d | ||
|
|
e5ddbe8909 | ||
|
|
018f19d8c8 | ||
|
|
fe2985a0b4 | ||
|
|
26d81afedd | ||
|
|
d71e35a5ae | ||
|
|
240f08d406 | ||
|
|
61ab3f0c4f | ||
|
|
fd39977c32 | ||
|
|
012861ce1d | ||
|
|
0dcd0dfc20 | ||
|
|
35f1c3517f | ||
|
|
eb8ba20c35 | ||
|
|
bdf7cc3b1e | ||
|
|
3b2522bd3b | ||
|
|
8bc01b48b4 | ||
|
|
deae5fbe6d | ||
|
|
fa0fcf35ce | ||
|
|
217b6fdcdc | ||
|
|
78343052c2 | ||
|
|
b105dae7c6 | ||
|
|
7a1d89a801 | ||
|
|
759817ccbe | ||
|
|
da3bb5aa7b | ||
|
|
ec7d09176b | ||
|
|
bf1811bdbc | ||
|
|
74201434e7 | ||
|
|
769d92025d | ||
|
|
ca1674473c | ||
|
|
22f28644fe | ||
|
|
43f06eb1e8 | ||
|
|
f21fda0fc0 | ||
|
|
7b9b6cc78b | ||
|
|
b1f473fcf8 | ||
|
|
24053f7ee2 | ||
|
|
a344650a7b | ||
|
|
9e0bc71e78 | ||
|
|
1f9bf8af0e | ||
|
|
419956f56e | ||
|
|
c5ee066022 | ||
|
|
3230f50b16 | ||
|
|
0a15246d32 | ||
|
|
9108d9a627 | ||
|
|
3c3d405c00 | ||
|
|
43d9c9d19b | ||
|
|
86059108b1 | ||
|
|
35fbec9df5 | ||
|
|
b394d50943 | ||
|
|
694edef951 | ||
|
|
d4b9a33348 | ||
|
|
b06df804d6 | ||
|
|
df2d018b30 | ||
|
|
6c1faaed75 | ||
|
|
1fcf50518e | ||
|
|
437e65d349 | ||
|
|
df5103e26d | ||
|
|
f328b49d22 | ||
|
|
70e4653feb | ||
|
|
1883578b67 | ||
|
|
97b5503f39 | ||
|
|
645735fc0f | ||
|
|
25885b05c0 | ||
|
|
2c27c2a43b | ||
|
|
ab1b9e35d0 | ||
|
|
bf271ba3fb | ||
|
|
dda2d40295 | ||
|
|
0b1a945d69 | ||
|
|
425581e838 | ||
|
|
f047624018 | ||
|
|
62a2814756 | ||
|
|
400a91b5f9 | ||
|
|
8dc2d7bec4 | ||
|
|
908d9c0726 | ||
|
|
6a24e801ea | ||
|
|
5e61ae0b0e | ||
|
|
7baed55130 | ||
|
|
512bcf5cdb | ||
|
|
31b6db72c6 | ||
|
|
3e1f70096b | ||
|
|
e8d1bbc98b | ||
|
|
e66e92904e | ||
|
|
8c64eb511f | ||
|
|
e304de99bc | ||
|
|
d307ece767 | ||
|
|
4fde832260 | ||
|
|
97fe58d3fe | ||
|
|
7c821cfea1 | ||
|
|
f46e9a8479 | ||
|
|
29768c6f53 | ||
|
|
cc13c5d095 | ||
|
|
897caa814d | ||
|
|
a874cef9af | ||
|
|
a6c10a28f4 | ||
|
|
9536ebb7ca | ||
|
|
8358cfcab3 | ||
|
|
94728785ea | ||
|
|
38be5fd9f2 | ||
|
|
f20f05536e | ||
|
|
4df630e16a | ||
|
|
c546e6d12c | ||
|
|
875f729070 | ||
|
|
dcf481c11a | ||
|
|
ca456efcbf | ||
|
|
891371e738 | ||
|
|
068c319e82 | ||
|
|
4a1c4a59ba | ||
|
|
95382c2ba0 | ||
|
|
5d5fe72218 | ||
|
|
bd0642e0a4 | ||
|
|
2c6f981667 | ||
|
|
84f3401dcf | ||
|
|
99d00d1041 | ||
|
|
228ca6a9da | ||
|
|
4b27c51724 | ||
|
|
b46f4fcebd | ||
|
|
78a84d75c0 | ||
|
|
c78c8aed40 | ||
|
|
6b3372e259 | ||
|
|
52fe342c73 | ||
|
|
4d7c6efe9a | ||
|
|
0b42dc56d1 | ||
|
|
0d3a2b815e | ||
|
|
a961db280f | ||
|
|
9d0e87f331 | ||
|
|
94182f331e | ||
|
|
5553acfe50 | ||
|
|
44d232fccb | ||
|
|
5a25d42f23 | ||
|
|
3602b64c45 | ||
|
|
d9c9e6ea67 | ||
|
|
9e78606e2b | ||
|
|
00a6524b9a | ||
|
|
90c95a6b64 | ||
|
|
4063bfda97 | ||
|
|
21a5179105 | ||
|
|
4b4abcc2ee | ||
|
|
d0fd1b05fc | ||
|
|
c0c6488ae9 | ||
|
|
25d3c7237f | ||
|
|
b4b881116f | ||
|
|
88cf498036 | ||
|
|
559083be29 | ||
|
|
5044e20171 | ||
|
|
08637b0c24 | ||
|
|
c5ed046d6e | ||
|
|
03a3fcbcea | ||
|
|
72d5e72e6d | ||
|
|
c818ca9042 | ||
|
|
2ac8578827 | ||
|
|
b81da90354 | ||
|
|
887b99888e | ||
|
|
0b028cc75a | ||
|
|
1a0e2c77f2 | ||
|
|
b40b0b8665 | ||
|
|
015a02d57b | ||
|
|
688dca3e90 | ||
|
|
197a296664 | ||
|
|
aa2c1c61ce | ||
|
|
2d25a09c0f | ||
|
|
9109e6f6a7 | ||
|
|
cc58263ee0 | ||
|
|
56fb5219ce | ||
|
|
f3c21665d8 | ||
|
|
ce45f92219 | ||
|
|
f912a68417 | ||
|
|
6ca4e1368c | ||
|
|
1071ad867c | ||
|
|
c06b53644b | ||
|
|
352a846127 | ||
|
|
5679ed66f4 | ||
|
|
a67525cdf1 | ||
|
|
d5b5808f91 | ||
|
|
cbd4bd4a66 | ||
|
|
31278c7b5a | ||
|
|
db0b9ac20c | ||
|
|
18cc16141c | ||
|
|
60f9a4c69a | ||
|
|
4bb53000a3 | ||
|
|
c928bb8d80 | ||
|
|
ee54b2e7cf | ||
|
|
70fd7923ef | ||
|
|
10de107fd5 | ||
|
|
407d7f698d | ||
|
|
79da7ea8fa | ||
|
|
ff542ada47 | ||
|
|
75ff321e9e | ||
|
|
33b4762d5c | ||
|
|
6cf4e768e8 | ||
|
|
ad597388a4 | ||
|
|
110932cd5e | ||
|
|
5cafea5ebe | ||
|
|
872890bef3 | ||
|
|
ab9dc4569f | ||
|
|
1596d6a275 | ||
|
|
80427af8cc | ||
|
|
e47bb152c2 | ||
|
|
81c09888bf | ||
|
|
106ba1730f | ||
|
|
f7dc5489b3 | ||
|
|
3e8a5f76d0 | ||
|
|
ac30fbd658 | ||
|
|
5d21bddb18 | ||
|
|
2e2f981ccf | ||
|
|
f8bb792f15 | ||
|
|
0ecf3a4cbf | ||
|
|
8842092671 |
No files matched your search
@@ -10,8 +10,8 @@ equivalent completed investigations. A useful outcome can also be a measured
|
||||
decision to leave an implementation alone.
|
||||
|
||||
Follow the checkout's [performance priorities](../../../AGENTS.md#performance),
|
||||
[tool design](../../../docs/tool-design.md), and
|
||||
[testing guide](../../../docs/testing.md). This skill supplies an optimization
|
||||
[tool design](../../../docs/development/tool-design.md), and
|
||||
[testing guide](../../../docs/development/testing.md). This skill supplies an optimization
|
||||
workflow, not additional permission to execute targets, publish, or merge.
|
||||
Use the current contracts, provider prerequisites, and user authorization.
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ description: Prepare or rewrite REA release changelogs and GitHub release notes
|
||||
# REA changelog update
|
||||
|
||||
Use this for release-note preparation and contributor attribution. Read
|
||||
[docs/releasing.md](../../../docs/releasing.md) from the repository root for
|
||||
[docs/development/releasing.md](../../../docs/development/releasing.md) from the repository root for
|
||||
publication authority and the existing checkpoint checks. This skill prepares
|
||||
notes; it does not authorize merging, tagging, or publishing.
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ Establish which checks still catch real regressions and which costs can be
|
||||
removed without losing that protection. The outcome may be a verified fix,
|
||||
an evidence-backed recommendation, or an explicit decision to stop.
|
||||
|
||||
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/testing.md),
|
||||
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/development/testing.md),
|
||||
and [contribution rules](../../../CONTRIBUTING.md). Use their current commands
|
||||
and pinned toolchain, rather than commands remembered from an earlier audit.
|
||||
Preserve the requested scope: investigation alone does not authorize edits;
|
||||
|
||||
@@ -5,7 +5,7 @@ description: Audit, investigate, design, or change REA investigation tools, CLI/
|
||||
|
||||
# REA tool design
|
||||
|
||||
Use the [tool-design guide](../../../docs/tool-design.md) for tool boundaries,
|
||||
Use the [tool-design guide](../../../docs/development/tool-design.md) for tool boundaries,
|
||||
contract semantics, provider ownership, and discoverability. Inspect the current
|
||||
contract and nearest existing tool relevant to the analyst question; source,
|
||||
documentation, and live catalogs may describe different versions.
|
||||
@@ -14,23 +14,30 @@ For provider-backed work, exercise the analyst's task through REA and compare
|
||||
relevant native provider and independent artifact observations where available.
|
||||
Check applicable partial results, recovery and session boundaries; report what
|
||||
was not exercised. For failure guidance, follow [design recovery around the
|
||||
failure](../../../docs/tool-design.md#design-recovery-around-the-failure).
|
||||
Use [workflow verification](../../../docs/testing.md#provider-backed-workflow-verification)
|
||||
failure](../../../docs/development/tool-design.md#design-recovery-around-the-failure).
|
||||
Use [workflow verification](../../../docs/development/testing.md#provider-backed-workflow-verification)
|
||||
to select evidence for the proposed contract instead of seeking provider API
|
||||
parity.
|
||||
|
||||
For an audit, follow [audit existing tools](../../../docs/tool-design.md#audit-existing-tools):
|
||||
For an audit, follow [audit existing tools](../../../docs/development/tool-design.md#audit-existing-tools):
|
||||
scope representative tasks, exercise the live workflow, and distinguish
|
||||
reproduced bugs, supported limitations, design opportunities and unverified
|
||||
claims. Check relevant rendered MCP prompts, guided workflows and packaged skill
|
||||
advice against the live contracts and exercise their suggested handoffs. Report
|
||||
working alternatives and evidence limits as well as findings.
|
||||
|
||||
For wording audits and agent trials, use
|
||||
[focused tool-description trials](../../../docs/development/client-verification.md#focused-tool-description-trials).
|
||||
Start with representative workflow boundaries and repeat failures or ambiguous
|
||||
outcomes; keep actual arguments and supported final claims separate. Reuse the
|
||||
repository's evaluation tooling and report actual runtime/model versions and
|
||||
synthetic versus production coverage rather than copying a harness into a skill.
|
||||
|
||||
For a design request, return the proposed boundary, input/result shape, Evidence
|
||||
semantics, nearest alternative, and verification approach. Implement when
|
||||
requested, preserving the user's scope and existing authorization.
|
||||
|
||||
For implementation or verification, consult [testing](../../../docs/testing.md)
|
||||
For implementation or verification, consult [testing](../../../docs/development/testing.md)
|
||||
for the affected consumer and provider lanes, and
|
||||
[contributing](../../../CONTRIBUTING.md) for generated-file ownership. Read
|
||||
provider/workflow guides only as the task requires.
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "60"
|
||||
version: "62"
|
||||
---
|
||||
|
||||
# REA
|
||||
@@ -17,9 +17,9 @@ readiness or provider commands.
|
||||
|
||||
Installing this skill supplies instructions; it does not register the REA MCP
|
||||
server, install analysis engines, or add tools to an already-running agent.
|
||||
The exception is a Claude Code or Codex plugin install: the plugin bundles
|
||||
this skill and registers its own `rea` server, outside the state that
|
||||
`rea setup` and `rea doctor` manage.
|
||||
The exception is a Claude Code, Codex, or Cursor Marketplace plugin install:
|
||||
the plugin bundles this skill and registers its own `rea` server, outside the
|
||||
state that `rea setup` and `rea doctor` manage.
|
||||
|
||||
If REA tools are available and their registration is not known to be stale,
|
||||
proceed directly to the target. Do not run diagnostics or setup before every
|
||||
@@ -97,6 +97,12 @@ explicit path or endpoint and do not need it.
|
||||
- Bun `--compile` executable: `open_binary`, then `inspect_artifact`; the Bun
|
||||
module graph is the application layer. Extract stored modules and analyze the
|
||||
JavaScript. See [Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
- Deno `deno compile` executable: `open_binary`, then `inspect_artifact`; the
|
||||
embedded virtual filesystem is the application layer. Extract stored files and
|
||||
analyze compatible JavaScript. Environment-file values and CA certificate bytes
|
||||
are retained in local Evidence with their counts and byte lengths. The npm
|
||||
snapshot length is reported and not decoded, and the executable is not run. See
|
||||
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
|
||||
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
|
||||
`open_bundle_resources`, then the selected resource tool. This does not
|
||||
require `CFBundleExecutable` or a deep provider. Resource sessions have no
|
||||
@@ -108,7 +114,7 @@ explicit path or endpoint and do not need it.
|
||||
- Firmware image: `inspect_firmware_regions` when advertised. Use
|
||||
`extract_firmware` when extraction is requested, with a caller-selected new
|
||||
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
|
||||
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
|
||||
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md).
|
||||
- ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact`
|
||||
without a provider; see the ESP section of `references/native-and-artifacts.md`.
|
||||
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
|
||||
@@ -140,6 +146,10 @@ explicit path or endpoint and do not need it.
|
||||
- Task for a desktop application's exported environment, from a
|
||||
`capture-desktop-application` folder: read
|
||||
[desktop application tasks](references/environments.md#tasks-for-a-desktop-application-environment).
|
||||
- Environment of a Linux desktop app with a task, from a request such as
|
||||
"build me an Inkscape environment where the agent changes a poster's
|
||||
headline": read [desktop builds](references/desktop-build.md). You drive the
|
||||
app on a build computer from `start_build_computer`, then export and witness.
|
||||
- Fiddler Everywhere traffic or a native `.saz` capture: read the
|
||||
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
|
||||
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
|
||||
@@ -150,7 +160,7 @@ explicit path or endpoint and do not need it.
|
||||
old-generation heap is `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` (default 192 MiB,
|
||||
floor 8, maximum 16384), set where REA starts. Host memory and `NODE_OPTIONS` do not
|
||||
raise it. See
|
||||
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
|
||||
[historical captures](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md) for the exact upstream
|
||||
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
|
||||
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
|
||||
or establish network loss. Prefer detail summary, then kind qlog trace/stream
|
||||
@@ -165,7 +175,7 @@ explicit path or endpoint and do not need it.
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and
|
||||
TShark 4.x on an absolute `PATH` entry or via the optional absolute
|
||||
`REA_TSHARK_COMMAND` override; see
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/guides/data/packet-captures.md).
|
||||
Packet limits count input packets before filtering. Output defaults to 50 matching
|
||||
frames; use `match_offset` / `match_limit` for later pages, or explicit null
|
||||
limit for complete matching output. Preserve the source digest across pages.
|
||||
@@ -173,7 +183,7 @@ explicit path or endpoint and do not need it.
|
||||
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
|
||||
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
|
||||
and capinfos, with Linux prlimit or Windows Job Objects; see
|
||||
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
|
||||
[packet summaries](https://github.com/morluto/rea/blob/main/docs/guides/data/pcap-analysis.md).
|
||||
Select an exact DNS question for address pages; page exhaustion does not prove
|
||||
network completeness. This reads recordings and does not acquire traffic.
|
||||
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
|
||||
@@ -181,7 +191,7 @@ explicit path or endpoint and do not need it.
|
||||
relevant details before choosing settings; this does not scan hardware.
|
||||
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
|
||||
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
|
||||
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
|
||||
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/guides/signals/signal-recordings.md).
|
||||
Output defaults to 50 literal lines; select annotation classes and page with
|
||||
`line_offset` / `line_limit`, or explicit null for all remaining output.
|
||||
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
|
||||
@@ -190,7 +200,7 @@ explicit path or endpoint and do not need it.
|
||||
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
|
||||
Select JavaScript glue and candidate paths explicitly and preserve URL/path
|
||||
ambiguity. No implicit fetch or target execution; WAT is not original source.
|
||||
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/wasm-artifacts.md).
|
||||
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/guides/wasm/wasm-artifacts.md).
|
||||
- Explicit local Android APK (or ZIP/AAB) signing: `inspect_apk_signature`.
|
||||
Prefer its summary or selected signer/digest/lineage page; retain complete
|
||||
Evidence and use `inspect_analysis_view` kind `apk-signature` for follow-ups.
|
||||
@@ -201,21 +211,27 @@ explicit path or endpoint and do not need it.
|
||||
See the [APK signature guide](https://github.com/morluto/rea/blob/main/docs/apk-signature.md).
|
||||
- Explicit local EVM bytecode carrier: `inspect_evm_interface` with caller-selected
|
||||
`raw` or `hex` encoding. The owned worker deadline is `timeout_ms` (default
|
||||
30000 ms, range 1..2147483647, next call). It does not raise the fixed byte,
|
||||
address-space, or CPU allowances. Preserve carrier/decoded digests and treat selectors,
|
||||
30000 ms, range 1..2147483647, next call). Select byte, address-space and CPU
|
||||
allowances separately through `producer_limits`; tighter inherited OS limits
|
||||
remain enforced, and the 256 MiB worker heap is independent. Preserve
|
||||
carrier/decoded digests and treat selectors,
|
||||
argument strings and mutability as inferred candidates; this performs no chain
|
||||
lookup or target execution. For the complete inferred interface without hex or
|
||||
duplicate stdout, discover support and select `detail: "summary"`; complete
|
||||
Evidence stays retained under its original identity. Use the returned
|
||||
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
|
||||
Candidate coverage does not establish interface completeness. See the
|
||||
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
|
||||
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
|
||||
with the explicit local file. This built-in target-free inspection requires no
|
||||
Go installation or disassembler. Preserve raw module bytes and source ranges;
|
||||
absent metadata does not prove a binary is not Go, and unparsed text records
|
||||
remain explicit. See the
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
|
||||
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/guides/evm/evm-bytecode.md).
|
||||
- Embedded Go compiler, module, build-setting metadata, or static `(ptr, len)`
|
||||
string literals: `inspect_go_binary` with the explicit local file (pass
|
||||
`include_string_literals: true` and optional `max_string_literals` for bounded
|
||||
inferred static string-header recovery, and page retained rows with
|
||||
`inspect_analysis_view` kind `go` collection `string_literals`). This built-in
|
||||
target-free inspection requires no Go installation or disassembler. Preserve
|
||||
raw module bytes and source ranges; absent metadata does not prove a binary is
|
||||
not Go, static string recovery excludes compiler-inlined stack or immediate
|
||||
strings, and unparsed text records remain explicit. See the
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/go-binaries.md).
|
||||
- Embedded CUDA/HIP code objects: `inspect_gpu_code_objects`; inventory targets
|
||||
first, then decode selected payloads and query retained GPU Evidence.
|
||||
- Bundled GGUF or safetensors model declarations: `inspect_model_artifact`
|
||||
@@ -268,6 +284,14 @@ explicit path or endpoint and do not need it.
|
||||
proof the image never imported APIs through another table. Bound snapshots
|
||||
and loaded IAT contents remain unobserved. See the
|
||||
[PE import guide](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
|
||||
- Windows Shell Link shortcuts: `inspect_windows_shortcut` with the explicit
|
||||
local `.lnk` file. This built-in target-free inspection requires no Windows
|
||||
SDK or COM. Preserve header flags and FILETIMEs, IDList items, LinkInfo
|
||||
volume and network names, StringData arguments and icon location, ExtraData
|
||||
signatures, and Tracker droid facts labelled derived. Unknown IDList items
|
||||
and ExtraData signatures stay raw with SHA-256. The target path is not
|
||||
resolved or executed. See the
|
||||
[Windows shortcut guide](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
|
||||
- PE TLS callbacks and load configuration: `inspect_pe_load_config` with the
|
||||
explicit local PE32/PE32+ file. This built-in target-free inspection requires
|
||||
no Windows SDK. Preserve callback VAs as strings with their RVA, file offset
|
||||
@@ -291,13 +315,32 @@ explicit path or endpoint and do not need it.
|
||||
build-id or ABI-tag fields. A missing table is observed absence of those
|
||||
declarations, not proof the image never carried notes. See the
|
||||
[ELF notes guide](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
- Code that runs before `main` or at exit in ELF or Mach-O:
|
||||
`inspect_native_initializers` with the explicit local file (and
|
||||
`architecture` for a universal Mach-O). This built-in inspection requires no
|
||||
disassembler. Preserve slot and target addresses, the `resolution` of each
|
||||
entry, and its limitations. An `unresolved` slot is not a zero pointer, and a
|
||||
missing table is not proof that no constructor exists. See the
|
||||
[initializer guide](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
|
||||
|
||||
- ELF unwind function ranges: `inspect_elf_unwind_ranges` with the explicit
|
||||
local ELF32/ELF64 file. This built-in target-free inspection requires no
|
||||
disassembler. Preserve FDE start, end, size, containing section, exact
|
||||
symbol name, CIE index, LSDA, and personality. Page with `offset` and
|
||||
`limit`. A missing `.eh_frame` is observed absence of that declaration,
|
||||
not proof the image has no functions. `.debug_frame` is reported present
|
||||
or absent and is not decoded. CFA instructions are counted and not
|
||||
evaluated. See the
|
||||
[ELF unwind ranges guide](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
|
||||
- Apple `.ips` crash reports: `inspect_apple_ips` with the explicit local
|
||||
file. This built-in target-free inspection requires no Xcode, atos, or
|
||||
pwntools. Preserve first-line metadata, `bug_type` as a string, process and
|
||||
exception facts, lossless integer registers, usedImages UUIDs, and
|
||||
documented classification matches. Only `bug_type` `"309"` yields a crash
|
||||
body; other types, including `"288"` stackshots, return no crash result.
|
||||
Opened Mach-O and dSYM UUID joins remain unobserved. See the
|
||||
Select an optional `join_path` for UUID-and-architecture matching to one
|
||||
local Mach-O or dSYM; preserve slice identity and unjoined reasons. Joined
|
||||
static addresses do not resolve function names or dSYM symbols. See the
|
||||
[Apple IPS guide](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
|
||||
- Linux kernel module metadata: `inspect_kernel_module` with the explicit local
|
||||
ELF32/ELF64 relocatable file. This built-in target-free inspection requires
|
||||
@@ -309,6 +352,16 @@ explicit path or endpoint and do not need it.
|
||||
zstd containers are unsupported; decompress into a separate file with an
|
||||
external codec utility, then inspect the decompressed `ET_REL` image. See the
|
||||
[kernel module guide](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
|
||||
- Linux kernel version and build config (`CONFIG_*` options such as
|
||||
`CONFIG_SECURITY_SELINUX`): `inspect_kernel_build_config` with the explicit
|
||||
local `vmlinux`, raw `Image`, x86 `bzImage`, or `configs.ko`. This built-in
|
||||
target-free inspection requires no Linux host or `extract-ikconfig`. Preserve
|
||||
the banner version, config SHA-256, option count, and value kinds; use `name`
|
||||
for one option. An `IKCFG_ST` string is a config only with a valid gzip member
|
||||
and `IKCFG_ED`. An absent config may mean the kernel was built without
|
||||
`CONFIG_IKCONFIG`. Non-gzip payloads are unsupported; decompress externally,
|
||||
then inspect the result. See the
|
||||
[kernel build config guide](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
|
||||
- CPython 3.7–3.13 `.pyc` header, nested marshal code objects and bytecode
|
||||
identity: `inspect_python_pyc` with the explicit local file. This built-in
|
||||
target-free inspection requires no Python installation. Preserve magic,
|
||||
@@ -332,17 +385,17 @@ explicit path or endpoint and do not need it.
|
||||
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
|
||||
report truncation and have unspecified order. Use a quiescent snapshot rather
|
||||
than assuming a live database/WAL pair was copied atomically. See the
|
||||
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
|
||||
- Explicit Linux ELF file for offline layout, symbols, relocations or static
|
||||
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
|
||||
caller-supplied pwntools without opening a disassembler database. Preserve its
|
||||
raw locations and inference/coverage limits; see the
|
||||
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
- Supplied Linux x86-64 ELF core: `inspect_recorded_crash`. Read every recorded
|
||||
thread, raw note and source-bound register without opening a live target.
|
||||
Historical PIDs do not select live processes. Request `include_debugger_context`
|
||||
only when core-only mapping candidates help; see the
|
||||
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md).
|
||||
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md).
|
||||
- PE resource type/name/language trees and static icon group references:
|
||||
`inspect_pe_resources` with an explicit path. This portable operation requires
|
||||
no active target or engine; opaque payloads retain file ranges and SHA-256.
|
||||
@@ -358,9 +411,9 @@ The legacy attached profile binds the already-open GUI input by SHA-256 and
|
||||
leaves its database open. The modern headless profile creates and closes an
|
||||
owned private database without saving. Installation and registration reuse
|
||||
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
|
||||
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
|
||||
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md).
|
||||
For migration to the separate native IDA MCP plugin, follow the guide's
|
||||
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/ida-provider.md#migrate-a-legacy-attached-registration).
|
||||
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md#migrate-a-legacy-attached-registration).
|
||||
Replace the upstream registration explicitly; modern mrexodia tools are not the
|
||||
native profile. Native attachment preserves the database but does not provide
|
||||
direct callers; check session availability and the guide's verification limits.
|
||||
@@ -539,6 +592,8 @@ as complete while required questions remain open.
|
||||
[references/environments.md](references/environments.md)
|
||||
- Tasks for a desktop application's exported environment:
|
||||
[references/environments.md](references/environments.md#tasks-for-a-desktop-application-environment)
|
||||
- A desktop app environment from a request, built and witnessed on build
|
||||
computers: [references/desktop-build.md](references/desktop-build.md)
|
||||
|
||||
## Finish the task
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ Preserve slot/relocation provenance and source coverage: matched means observed
|
||||
candidate presence; partial scans do not establish uniqueness or absence.
|
||||
This reads DEX/ELF bytes without loading the APK into the decompiler or executing
|
||||
target code. Runtime registration and loading remain unknown. See the
|
||||
[JNI guide](https://github.com/morluto/rea/blob/main/docs/android-analysis.md#link-native-declarations-to-jni-candidates).
|
||||
[JNI guide](https://github.com/morluto/rea/blob/main/docs/guides/android/android-analysis.md#link-native-declarations-to-jni-candidates).
|
||||
|
||||
For an R8/ProGuard release build with its `mapping.txt`, pass the absolute
|
||||
`mapping_path` to `inspect_android_class` or `inspect_android_method`. Prefer
|
||||
|
||||
@@ -11,7 +11,7 @@ main and an older release expose the same arguments. Tool-name prefixes belong
|
||||
to the client: find the canonical REA operation in its actual tool listing and
|
||||
server association before concluding that it is absent. A shared prefix alone
|
||||
does not establish which server an approval wildcard covers. See
|
||||
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/client-configuration.md#client-tool-names-and-discovery).
|
||||
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/getting-started/client-configuration.md#client-tool-names-and-discovery).
|
||||
|
||||
## Choose the remedy from the observed failure
|
||||
|
||||
@@ -118,19 +118,22 @@ is not a recommendation to change it. Do not invent a setting. Host memory and
|
||||
|
||||
## REA installed as a client plugin
|
||||
|
||||
A Claude Code or Codex plugin owns its `rea` server and this skill. Doctor's
|
||||
`--client` and `--skill` checks and `rea setup` read and write only
|
||||
setup-owned state, so for a plugin-only install they report `missing`, and
|
||||
applying setup would add a second `rea` server. If this skill was loaded from
|
||||
a plugin, or the client lists a `rea` plugin, do not prepare a setup plan for
|
||||
missing tools. Instead:
|
||||
A Claude Code, Codex, or Cursor Marketplace plugin owns its `rea` server and
|
||||
this skill. Doctor's `--client` and `--skill` checks and `rea setup` read and
|
||||
write only setup-owned state, so for a plugin-only install they report
|
||||
`missing`, and applying setup would add a second `rea` server. If this skill
|
||||
was loaded from a plugin, or the client lists a `rea` plugin, do not prepare a
|
||||
setup plan for missing tools. Instead:
|
||||
|
||||
- Check that the plugin is installed and enabled with the client's own
|
||||
listing: `claude plugin list` and `/mcp` in Claude Code, `codex plugin list`
|
||||
in Codex.
|
||||
in Codex, and Cursor's plugin settings for a Marketplace install.
|
||||
- Restart or reconnect the client, then read its launch error for the `rea`
|
||||
server if tools are still absent. The plugin starts its bundled runtime; a missing build or version mismatch
|
||||
requires a reinstall, or a fresh build when developing from source.
|
||||
server if tools are still absent. Claude Code and Codex start the plugin's
|
||||
bundled runtime; a missing build or version mismatch requires a reinstall,
|
||||
or a fresh build when developing from source. A Cursor Marketplace install
|
||||
starts `npx` at the version pinned in the plugin's `mcp.json`. A failed
|
||||
download needs network access to that published version.
|
||||
- Check analysis engines through the connected server's
|
||||
`binary_session.result.tool_availability`. `rea doctor --provider <id> --json`
|
||||
from a shell reports that shell's environment, which can differ from the
|
||||
|
||||
@@ -0,0 +1,860 @@
|
||||
# Desktop app environments from a request
|
||||
|
||||
Use this when someone asks for an environment of a Linux desktop app with a
|
||||
task, such as "Build me an Inkscape environment where the agent changes a
|
||||
poster's headline and saves it". You are the building agent: you drive the
|
||||
app on a build computer through REA's MCP tools, write the task, export the
|
||||
bundle and witness it in a fresh computer. REA runs no model. The tools are
|
||||
MCP-only and need a Linux host with Docker, agentenv and REA's packed
|
||||
runtime. When a start reports one missing, tell the person to run
|
||||
`rea doctor --provider desktop_build` and apply its fixes, or
|
||||
`rea setup --client <client> --desktop-build` and restart the client, and
|
||||
stop.
|
||||
|
||||
For the whole build:
|
||||
|
||||
- Drive the app only with `act_on_build_computer`. Never script it, run
|
||||
commands in the computer or write its files another way. The one command
|
||||
you write for the computer is a task's reader, which REA runs to check a
|
||||
file; it never makes the seed or the edit.
|
||||
- Ask the upfront questions once, then work on your own. Pause only when
|
||||
blocked.
|
||||
- The seed's content is synthetic. Never put real personal data in it.
|
||||
- Close every computer you start, on success and on failure.
|
||||
- Note the time you start; the report gives the build time.
|
||||
|
||||
## Ask once, upfront
|
||||
|
||||
In one message, ask only what the request leaves open, and keep the list for
|
||||
the report. Skip each question the request answers; when it answers them all,
|
||||
start without asking:
|
||||
|
||||
1. The app and the task, if either is unclear: what changes, in which
|
||||
document.
|
||||
2. Whether they have a starting file of their own; otherwise you make one.
|
||||
3. Whether they want the tasks measured after the witness: with which
|
||||
model, within which budget, and whether to harden a task until it lands
|
||||
in the band.
|
||||
4. If so, whether they want to review the tasks before those solves.
|
||||
|
||||
When you cannot ask, as in a headless `claude -p` run, do not stop: take
|
||||
these defaults for whatever the request leaves open, say which you took in
|
||||
your first message and in the report, and go on. Install from the Ubuntu
|
||||
archive, make a synthetic seed, skip the review, and measure each task
|
||||
within $4 in all, with the model the request names, without hardening.
|
||||
|
||||
## When you are blocked
|
||||
|
||||
Stop, say what you need and wait only for:
|
||||
|
||||
- a licence the app or its installer asks someone to accept: never accept it
|
||||
on your own; name it and where to read it, and go on only once the person
|
||||
says they accept it. Then record their words with `record_build_consent`,
|
||||
kind `licence`, and accept it on their behalf;
|
||||
- credentials: ask the person to write each secret, and nothing else, in a
|
||||
file named after a handle you choose, such as `keepassxc-master`, in
|
||||
`~/.config/rea/build-secrets/` on this host (under `$XDG_CONFIG_HOME` when
|
||||
it is set), readable only by them (`chmod 600`), and wait until they say it
|
||||
is there; then type it with `type_secret` (see Drive the app). Never ask
|
||||
for a password or key in chat, and never read, list or print that folder;
|
||||
- a signed-in state in the seed: by default the export withholds it (see
|
||||
Sign-in and licence state). Seed it only when the person says the bundle
|
||||
may hold it: record their words with `record_build_consent`, kind
|
||||
`seed_sign_in`, and export with `seed_sign_in: true`;
|
||||
- an install this flow cannot do, below.
|
||||
|
||||
Decide everything else yourself and note it in the report.
|
||||
|
||||
## Sign-in and licence state
|
||||
|
||||
What an app writes when someone signs in or activates a licence is withheld
|
||||
from the seed: the bundle's receiver supplies their own. Mark
|
||||
`pre-sign-in` with `mark_build_checkpoint` just before the sign-in or
|
||||
activation and `post-sign-in` just after, both before `seed`: every file
|
||||
under the owned paths that changed between them is withheld, and the draft
|
||||
leaves it out. A file the app writes at first launch, such as a trial's
|
||||
licence file, changes before you can mark anything: name it in
|
||||
`export_build_environment`'s `withhold`. The export lists each withheld file
|
||||
in `withheld_files` with the environment variable a deploy reads it from and
|
||||
the handle of the secret REA keeps its bytes in; a check may not name one.
|
||||
The export refuses a seed file holding a withheld file's exact bytes, such as
|
||||
a `.bak` copy, but a backup that differs from it is not caught: trash backups
|
||||
before `seed`.
|
||||
Explicitly withhold any other files the person identifies as sensitive;
|
||||
REA does not infer secrecy from a value's shape or a file name.
|
||||
|
||||
## Choose the install
|
||||
|
||||
The computer is Ubuntu 24.04 (noble) on this host's architecture only. Use
|
||||
the first source that has the app:
|
||||
|
||||
| Source | `install` |
|
||||
| ---------------------------- | ------------------------------------------------------------------------------------------- |
|
||||
| Ubuntu archive | `{"kind": "archive", "package": "inkscape"}` |
|
||||
| PPA or vendor apt repository | `{"kind": "apt_repository", "package", "uri", "suite", "components", "key_file"}` |
|
||||
| `.deb` by URL | `{"kind": "package_file", "url", "sha256", "architecture", "dependencies"}` |
|
||||
| AppImage by URL | `{"kind": "appimage", "url", "sha256", "architecture", "extra_packages"}` |
|
||||
| Tarball by URL | `{"kind": "tarball", "url", "sha256", "architecture", "program", "name", "extra_packages"}` |
|
||||
|
||||
- A PPA's `uri` is `https://ppa.launchpadcontent.net/OWNER/NAME/ubuntu`,
|
||||
`suite` `noble`, `components` `["main"]`. `key_file` is the repository's
|
||||
signing key saved on this host; for a PPA, fetch it from
|
||||
keyserver.ubuntu.com by the fingerprint its Launchpad page shows.
|
||||
- When the app exits at launch for want of a runtime piece its package does
|
||||
not install, start a new build computer with that package in
|
||||
`extra_packages`, up to 16 archive names; the export installs them too.
|
||||
MyPaint needs
|
||||
`{"kind": "archive", "package": "mypaint", "extra_packages": ["librsvg2-common"]}`
|
||||
for its SVG icons, and Freeplane `["openjdk-17-jre"]`, since its launcher
|
||||
refuses noble's Java 21. Name one up front, too, when a package the app
|
||||
only recommends holds its content, such as QElectroTech's element library
|
||||
`qelectrotech-data`, without which no component can be placed.
|
||||
- Take the `.deb` path only when the app is not in the archive at all, not
|
||||
to add a package. Ask the requester for its URL and SHA-256; do not fetch
|
||||
a vendor's download page yourself. List its `Depends` as `dependencies`,
|
||||
with any runtime piece it lacks: it installs with the network off.
|
||||
- Take an AppImage or a tarball only when no package has the app. Pin it by
|
||||
the release file's URL for this host's architecture and the SHA-256 its
|
||||
project publishes, such as a GitHub release asset's digest or the
|
||||
release's `SHA256SUMS`; ask the requester when the project publishes none.
|
||||
The computer downloads it when the bundle is put, refuses other bytes, and
|
||||
unpacks it under `/opt/rea-apps/<desktop_entry>` with the network off; the
|
||||
export downloads the same file.
|
||||
- An AppImage installs its own desktop entry, so `desktop_entry` is the
|
||||
`.desktop` file at the top of the image, such as FreeCAD's
|
||||
`org.freecad.FreeCAD`. A wrong name fails the start with the entries the
|
||||
image has: start again with one of them. The image brings its libraries,
|
||||
so `extra_packages` is rarely needed; FreeCAD 1.1.4's needs none.
|
||||
- A tarball has no entry of its own. `program` is the program's path inside
|
||||
the archive, such as `firefox/firefox`; REA writes an entry named
|
||||
`desktop_entry` that runs it with the files it opens, and `name` is what
|
||||
the Applications menu shows. A tarball uses the computer's libraries: name
|
||||
what its program lacks in `extra_packages`. gzip, bzip2 and xz archives
|
||||
are read, zstd ones are not.
|
||||
- Flatpak and snaps are not supported: say so and stop. Flatpak's sandbox
|
||||
needs user namespaces the computer's container does not grant, so a
|
||||
Flathub app installs but never opens; ask for the project's AppImage,
|
||||
tarball or `.deb` when it ships one. An archive package that only installs
|
||||
a snap, such as noble's `firefox`, counts as a snap. A package with no
|
||||
build for this host's architecture (some PPAs are amd64 only) cannot be
|
||||
installed either.
|
||||
- `desktop_entry` is the package's `.desktop` file name under
|
||||
`/usr/share/applications` without `.desktop`, such as
|
||||
`org.inkscape.Inkscape`. Add `window_class` when you know it; Inkscape's
|
||||
is `Inkscape`. AbiWord's entry is `abiword` and its class `Abiword`.
|
||||
- To find the entry, read the package's file list: its page on
|
||||
packages.ubuntu.com, or `apt-get download PKG && dpkg -c PKG_*.deb` where
|
||||
apt runs. The entry may ship in a companion `-data` or `-common` package,
|
||||
as Planner's in `planner-data`: read that package's list too. The entry's `StartupWMClass` may not be the window's real
|
||||
class, so do not take `window_class` from it: MuseScore 3's says
|
||||
`mscore3`, its window's is `MuseScore3`. The export uses the class the
|
||||
capture saw.
|
||||
|
||||
## Drive the app
|
||||
|
||||
- Pass `computer_id` and the action's fields at the top level: click
|
||||
`{x, y, count}`, scroll `{x, y, direction, amount}`, a screenshot region
|
||||
`[x0, y0, x1, y1]`. The rest are in
|
||||
[the build-computer guide](https://github.com/morluto/rea/blob/main/docs/desktop-build-computer.md).
|
||||
- Take a `screenshot` before pointing: points are pixels of the latest
|
||||
frame. If a response is too large, ask for a `region`, or `jpeg` with a
|
||||
lower `quality`. A screenshot with `region` does not capture the screen
|
||||
again; it crops the latest frame, so take a full screenshot, or a `wait`
|
||||
with `screenshot: true`, first. The crop then becomes the latest frame:
|
||||
take a full screenshot before pointing again.
|
||||
- A drawn caret does not prove keyboard focus. After using a toolbar
|
||||
control, click into the document, then type a few characters and look
|
||||
before a long text.
|
||||
- Confirm a dialog with `key` Return, not a trailing `\n` in `type`.
|
||||
- An action's own screenshot is taken at once. After opening a menu, a
|
||||
dialog or a window, look with
|
||||
`{"action": "wait", "ms": 500, "screenshot": true}`.
|
||||
- On a computer started from a request the app is not open at first, and
|
||||
`start_build_capture` opens it: never open it yourself before that. Do not
|
||||
close it while capturing; the capture ends when the app exits. When the
|
||||
app is closed at any other time, open it from Applications → Other → the
|
||||
environment's name.
|
||||
- Type a secret with `{"action": "type_secret", "secret": "<handle>"}`,
|
||||
only into a field that masks what is typed: check in a screenshot that it
|
||||
is a password field, which shows dots. A screenshot shows any field that
|
||||
shows its text, such as many licence-key boxes, so never type a secret into
|
||||
one: say the app needs a secret the build cannot type unseen, and stop.
|
||||
`type_secret` takes no `screenshot`; look with a `wait` after it. A refusal
|
||||
names the secret's file and what is wrong with it: tell the person, never
|
||||
the value.
|
||||
- Prefer shortcuts and typing to long pointer paths: `key` takes `keys`, an
|
||||
array holding one chord's keys, such as
|
||||
`{"action": "key", "keys": ["Control", "s"]}`, and `type` turns `\n` into
|
||||
Enter.
|
||||
- Before `type`, check in a screenshot that a text cursor or a focused
|
||||
field is there. In apps with single-key shortcuts, such as Inkscape, text
|
||||
typed without one runs as commands.
|
||||
- Save As differs by app. Check the dialog's current folder first: most
|
||||
open in the home, so click Documents in the sidebar, but Scribus opens in
|
||||
`~/Documents` and its sidebar has no Documents entry. Type only the file
|
||||
name. A dialog taller than the 1280×800 screen hides its buttons:
|
||||
maximize it from its title bar, or confirm it with `key` Return, as when
|
||||
HomeBank's Save button is off-screen. When a GTK Save dialog ignores
|
||||
clicks or runs off the screen, type `~/Documents/<name>` in its Name field
|
||||
and press Return.
|
||||
- Answer the app's save questions, such as keeping the format, then check
|
||||
the saved path in the title or status bar, or the dialog's folder when
|
||||
the app shows only the name, and that it shows no unsaved mark, before
|
||||
you mark a checkpoint or check. Where the title never shows one, as in
|
||||
HomeBank, a greyed Save or Revert, or the file reopened from recent
|
||||
files, confirms the save.
|
||||
- Qt spin boxes change their value on scroll: move the pointer off them
|
||||
before you scroll.
|
||||
- gLabels edits an object's text in Object properties → Text, not on the
|
||||
canvas.
|
||||
- In Grisbi's transaction form Return moves to the next field; only Return
|
||||
in the last field, Notes, records the transaction.
|
||||
- ghostwriter autosaves a new document as `~/Documents/untitled-1.md`: name
|
||||
it with File → Rename, not Save As; Ctrl+Shift+S does nothing, and Enter
|
||||
after a checklist item starts another `- [ ]` item.
|
||||
- KMyMoney's first launch opens the New File Setup wizard; Finish asks for a
|
||||
storage type (XML), then a Save As where typing `Documents/<name>.kmy`
|
||||
works. Its date field is a sectioned Qt editor: double-click the month and
|
||||
type each part, since Ctrl+A garbles it. A new transaction's payee comes
|
||||
prefilled and selected. Amounts are saved as reduced fractions, such as
|
||||
`-1749/20`, so compare them exactly. Untick Help → Tip of the Day → "Show
|
||||
tips on startup" before `seed`, or the tip appears at every boot.
|
||||
- QElectroTech: an element dropped from Collections follows the pointer until
|
||||
a click on the folio places it. Its title shows only the app, so it needs
|
||||
`state_paths`; its settings are in `~/.qet`.
|
||||
- Rosegarden: after a dialog its Transport window can take keyboard focus,
|
||||
so save with File → Save.
|
||||
- Pencil2D: the Tools dock is too narrow for its buttons, so use shortcuts
|
||||
(B for the brush); double-click a layer's name to rename it.
|
||||
- Umbrello: the title shows no unsaved mark, so confirm a save with a probe
|
||||
checkpoint. An Untitled document autosaves `~/autosave.xmi`: delete it
|
||||
before `seed`. Add attributes from the tree: right-click the class → New →
|
||||
Attribute.
|
||||
- Hydrogen: the computer has no audio device, so Hydrogen falls back to a
|
||||
null driver and its BPM field snaps back to 120 within about 100 ms.
|
||||
Before `seed`, set Preferences → Audio System → ALSA with device `null`
|
||||
and click "Apply and restart output", which `~/.hydrogen/hydrogen.conf`
|
||||
keeps for the seed; also set autosaves per hour to 0, and dismiss the
|
||||
development-version dialog with "Don't show again". `pulseaudio` as an
|
||||
extra package does not help.
|
||||
- Poedit cannot add entries by hand. Use File → New, pick the language and
|
||||
save to `Documents` first (Sources Paths stays locked until then), then
|
||||
Properties → Sources Paths → + → Add Files with a source such as
|
||||
`/usr/lib/python3.12/argparse.py`. It writes `fr.mo` beside the `.po`, and
|
||||
the language box's autocomplete popup takes the first OK click.
|
||||
- CherryTree: its first-run autosave "Choose Storage Type" dialog swallows
|
||||
keys, so save once early as `.ctd` and untick autosave.
|
||||
- VSCodium (its tarball, with `extra_packages: ["libnss3",
|
||||
"libasound2t64"]`) runs on outside REA's reach, so `stop_build_capture`
|
||||
fails to clean it up: after `end`, quit it with Ctrl+Q, then stop. Enter
|
||||
accepts its word suggestions, so press Escape after typing a line.
|
||||
- Glade: a new widget's id is empty until you set it, and a second save
|
||||
writes a `.glade~` backup, which shows in the draft as an unrelated
|
||||
change. A backup present at `seed` is seeded with the document, so trash
|
||||
it before `seed`. The draft names a button's `<child>` by the id
|
||||
its widget carries, `[name=child][>object@id=submit_button]`, which moving
|
||||
the button keeps; give the widget an id before `seed` so it can.
|
||||
- VYM: the title never names the map, so pass `Documents/<map>.vym` in
|
||||
`state_paths`. Its settings live in `.config/InSilmaril` and are written
|
||||
only at a clean exit. It writes `<map>.vym.lock` while a map is open, so
|
||||
close the map before `seed`. Press A only after clicking a branch on the
|
||||
canvas: keys typed in the tree editor run other shortcuts.
|
||||
- KmPlot: Create → Cartesian Plot, then edit the formula in the Function
|
||||
tab and press Return. It stores minus as U+2212 (`−`). Its file:// title
|
||||
still names the file.
|
||||
- RedNotebook keeps each month in `~/.rednotebook/data/YYYY-MM.txt`, YAML
|
||||
read as text, folded at about column 80: pick check phrases the fold keeps
|
||||
on one line.
|
||||
- Qt5 apps saving with QDom, such as KmPlot, write attributes in a random
|
||||
order in each process, so check their XML by element, never by whole line.
|
||||
- Gnumeric's desktop entry is `org.gnumeric.gnumeric`. Its Save As dialog
|
||||
runs off the 1280×800 screen: type `~/Documents/<name>` in Name and press
|
||||
Return. Rename a sheet with right-click on its tab, then Rename….
|
||||
- Parley's title shows the collection's title, so name the file in
|
||||
`state_paths`. File → New opens the new collection's Properties dialog:
|
||||
confirm it with OK, since Cancel keeps the old collection and its autosave
|
||||
lock open. A new collection starts with 15 blank entries: delete them.
|
||||
Pick languages from the dropdown; a typed name keeps the previous locale.
|
||||
While a document is open KDE keeps an autosave and its `.lock` in
|
||||
`~/.local/share/stalefiles/parley/`, and there is no File → Close: before
|
||||
`seed`, use File → New, then reopen the collection from Open Recent.
|
||||
- Sigil: pass `extra_packages: ["libqt5svg5"]`, or its SVG toolbar icons are
|
||||
blank. Its title names the book, so it needs no `state_paths`. It unpacks
|
||||
the open book into `.local/share/sigil-ebook/sigil/workspace/Sigil-XXXXXX/`
|
||||
and deletes it on exit, so never name the whole `sigil-ebook` folder: that
|
||||
seeds the stale copy.
|
||||
- What the app saved under the home before `seed`, such as a document in
|
||||
`~/Documents` or its settings, is part of the seed. A file you move to the
|
||||
Trash is left out, and the export lists it in `limitations`. Trash a stray
|
||||
file rather than starting over. JabRef 3.8 writes `<file>.bak` and
|
||||
`<file>.sav` beside a `.bib` it saves; in the owned folder they are seeded
|
||||
and show in the draft as noise, so trash them before `seed` if they should
|
||||
not ship.
|
||||
- Dismiss first-run dialogs, such as a welcome screen, before `seed`. A
|
||||
choice the app saves only later, such as Inkscape's "show this every
|
||||
time", may not reach the seed, so expect the dialog in the witness too.
|
||||
- Some apps write their settings late: Freeplane writes
|
||||
`~/.config/freeplane/1.7.x/auto.properties` only once Preferences is
|
||||
confirmed, and a seed without it reopens its documentation map. Before
|
||||
`seed`, open the app's preferences and confirm them, then read a probe
|
||||
checkpoint's `changed_files` to see the settings file written.
|
||||
- Before `seed`, click into the document to clear any selection the app
|
||||
would save: Gnote saves its cursor and selection with the note.
|
||||
- An app whose state folder is named after it, such as
|
||||
`~/.local/share/gnote`, or after it and its version, such as TreeLine's
|
||||
`~/.treeline-3.1`, is found without `state_paths`.
|
||||
- TreeLine: File → New opens a template window. Book List's CATEGORY makes
|
||||
AUTHOR children by default; change it under Data → Configure Data Types →
|
||||
Default Child Type. A new child opens an inline rename showing "", so
|
||||
press Escape and fill its fields in the Data Edit tab. Ctrl+A adds a child
|
||||
node even while a field has focus. Clicking a node already selected starts
|
||||
an inline rename, so Ctrl+X then cuts its text: cut a node from its
|
||||
context menu. It saves nodes sorted by their
|
||||
generated uid1, so their order changes from run to run. The draft names a
|
||||
new node by its format and a field of its `data`,
|
||||
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, and checks its other
|
||||
fields through that, so give each new node a title no other node has.
|
||||
- After an app crashes, capture again on a fresh computer, so the seed
|
||||
carries no crash-recovery state, such as MyPaint's "Recover interrupted
|
||||
work?".
|
||||
- Give what the task changes a stable name where the app has one, such as an
|
||||
id in Inkscape's Object Properties, so a check can select it.
|
||||
- Expect apps to split text into runs: Writer's revision ids split an
|
||||
edited paragraph into spans and may give it an automatic style such as
|
||||
`P1`, and AbiWord wraps each insertion in `<c>`. Check such text whole
|
||||
with `text_equals` (see Write the task), and find the paragraph by
|
||||
something the edit keeps, such as AbiWord's `xid`.
|
||||
- Freeplane's first launch opens a documentation map: close it with Ctrl+W.
|
||||
Its Ctrl+N asks for a template.
|
||||
- MyPaint 2.0.1 crashes on brush strokes on the build computer: build its
|
||||
seed with flood fill and layer operations.
|
||||
- GnuCash's Ctrl+S on a new book opens a Save dialog with no format choice:
|
||||
use File > Save As to pick XML. Close the book (File > New File) before
|
||||
`seed`, so its `.LCK` lock is not seeded.
|
||||
|
||||
## Build the seed and the edit
|
||||
|
||||
1. `start_build_computer` with
|
||||
`{"source": {"kind": "request", "desktop_entry": ..., "install": ...}}`.
|
||||
The first start of an app builds its image, and starts at the same time
|
||||
on this host wait for each other; either can take minutes. A failed
|
||||
start returns the build's error: fix the request, or stop when the
|
||||
install cannot work, and close any `computer_id` the error names.
|
||||
2. `start_build_capture`. It returns once the app's window is open. A
|
||||
`package_lists_error` may stop the capture finding the package's
|
||||
sources; the export says when that matters. When the app exits before
|
||||
its window opens, the error gives its exit code and the end of its
|
||||
stderr, which may name what it lacks, such as an icon loader or a Java
|
||||
runtime: close the computer and start a new one with that package in
|
||||
`extra_packages` (see Choose the install), or stop as for an install
|
||||
this flow cannot do.
|
||||
- REA counts a file as the app's document only when the app's window
|
||||
title shows its file name or `~/` path. When the title shows something
|
||||
else, such as MuseScore's score title, pass the path you will save the
|
||||
document at in `state_paths`, relative to the home:
|
||||
`{"computer_id": ..., "state_paths": ["Documents/MuseScore3/Scores/Morning_Song.mscz"]}`.
|
||||
It need not exist yet. Add the app's settings folder when its name
|
||||
differs from the app's, such as `.config/MuseScore`, so the seed keeps
|
||||
the dismissed first-run dialogs. Do not pass the document as a launch
|
||||
argument to make REA see it: `state_paths` is the way. gLabels' title
|
||||
leaves out the file's extension and Grisbi's shows the entity's name,
|
||||
so both need it.
|
||||
- An app whose document is a folder of pages, such as a Zim notebook
|
||||
(`Notebooks/Notes`, a `.txt` file per page), never shows a file in its
|
||||
title: pass the folder in `state_paths`. Every page in it is then a
|
||||
document, so the bundle opens none at launch and the app starts where
|
||||
it keeps its place; check the page the task changes.
|
||||
3. Make the seed through the app and save it from the app under
|
||||
`~/Documents`: build the document in the app, or write its source where
|
||||
the app edits its format as text, then save it again from the app. Make
|
||||
it look real but synthetic: the thing the task changes, plus a few things
|
||||
the change must leave alone. Keep it to one document: the bundle's app
|
||||
opens the seed's document at launch only when there is exactly one, and
|
||||
otherwise starts on its own screen. For a starting file of the requester's,
|
||||
read it here and rebuild it in the app with synthetic data in place of
|
||||
personal data; REA cannot copy a file into the computer.
|
||||
- An app that keeps its own data and saves after every change, such as
|
||||
Osmo's contacts in `~/.local/share/osmo/contacts_records.xml`, has no
|
||||
document: that state is the seed. Build it in the app and mark `seed`
|
||||
once the app has written. To see whether it has, mark a throwaway
|
||||
checkpoint, such as `probe-1`, and read its `changed_files`: up to ten
|
||||
files that changed since the checkpoint before, those a window title
|
||||
named first, then those under `state_paths` and those the app held
|
||||
open, caches last, with `more_changed_files` counting the rest.
|
||||
- The app's own compressed format works when it is gzip, such as
|
||||
Gnumeric's `.gnumeric`, Dia's `.dia` or Inkscape's `.svgz`: REA reads
|
||||
the document inside, and checks point into it as into the same
|
||||
document uncompressed. Compression may be off by default: Dia's
|
||||
"Compress diagram files" is unticked, which REA reads as plain XML. A format REA reads as binary, such as one
|
||||
compressed with xz, has no view: save in a format the app offers that
|
||||
REA reads, then reopen the file once and check that a plain Ctrl+S
|
||||
keeps that format. When the task is about the app's own binary
|
||||
format, such as GIMP's `.xcf` or Blender's `.blend`, keep it and check
|
||||
it through a reader (see Read a binary file with a reader).
|
||||
4. Only once the seed is saved, `mark_build_checkpoint` with name `seed`.
|
||||
5. Make the change the task asks for, as a user would, and save it.
|
||||
6. Only once the change is saved, `mark_build_checkpoint` with name `end`.
|
||||
For more tasks in this environment, make each next task's change on top
|
||||
of the last as a user would, save it, and mark `end-<task name>`, such as
|
||||
`end-add-bread`. Each task is checked at its own end, so its checks must
|
||||
not read what a later task changes, and a solving agent starts each from
|
||||
the seed, so a task must not need an earlier one done first.
|
||||
Then, for each task you can, make a near miss of its change on top, a
|
||||
plausible wrong edit such as a typo or a wrong number, save it and mark
|
||||
`near-<task name>`: the export refuses a task whose near miss passes every
|
||||
check. Take the near misses after the last end, since each later draft
|
||||
starts from the end before it.
|
||||
7. `stop_build_capture`. Read `draft.files` (each file that changed, with
|
||||
its net change and its `values`), `draft.task.checks` (suggested
|
||||
checks), `draft.guards`, `draft.notes` and `limitations`; each
|
||||
`later_drafts` entry is the same for one later task, from the end
|
||||
checkpoint before its own, so it shows only that task's change. When
|
||||
`draft` is null, read `draft_refusal` first: a missing `seed` or `end`
|
||||
checkpoint means capturing again on a new build computer, while a failed
|
||||
draft command names its own cause to fix. If a declared secret was used,
|
||||
automatic drafting is withheld: write the task manually and export it.
|
||||
When `draft.files` is empty,
|
||||
`draft.notes` says why:
|
||||
- `no_change`: nothing was saved between `seed` and `end`. Capture again
|
||||
on a new build computer, and save before marking `end`. When the app
|
||||
keeps the task's state only on screen, such as a calculator's result,
|
||||
judge the screen instead (see Judge the screen).
|
||||
- `unowned_changes` with `no_document`: the app saved your document
|
||||
where REA did not count it as the app's. Its `files` list what changed,
|
||||
with their roles; the document is among them. Close the computer and
|
||||
capture again on a new one with that path in `state_paths`; the export
|
||||
refuses a check on a file outside the owned paths.
|
||||
- `uncheckable_changes`: the app saved the change in dconf, as gedit does
|
||||
its preferences, and no check reads dconf. Saving again does not help:
|
||||
choose a change the app saves in a file, and capture again on a new
|
||||
build computer, or judge the screen (see Judge the screen).
|
||||
- `unreadable_changes`: a dconf database changed but REA could not read
|
||||
it; `error` says why. This is not evidence the app saved there: report
|
||||
the error rather than changing the task.
|
||||
|
||||
A note with a `file` says why that file's suggestions are thin:
|
||||
- `by_position`: only position names what changed; its `elements` lack
|
||||
a naming attribute. Name a record that keeps its fields as child
|
||||
elements by one of them, as Osmo's contact by
|
||||
`[name=contact][>first_name=Delphine]`, a JSON record by a field
|
||||
inside it, `[>data/Title=…]` (`~1` for the `/` inside a pointer), or
|
||||
an element by an id its child carries, `[>object@id=…]`; count records
|
||||
with `member_count`, `matching` `[name=contact]`; otherwise check an
|
||||
ancestor's text, or give the element an id in the app and capture again.
|
||||
- `view_state_only`: every change is view state, a renumbered id or a
|
||||
time the app stamps, and its `pointers` are the checks kept. The edit
|
||||
was likely not saved: capture again and save it before `end`.
|
||||
- `long_text`: the element's text is too long to check whole; check a
|
||||
phrase in it with `text_contains`.
|
||||
- `cut_view` or `unreadable_view`: REA's view holds only part of the
|
||||
file, or none a check can read, such as a binary format (step 3). For
|
||||
a binary file, read it with a reader (see Read a binary file with a
|
||||
reader).
|
||||
|
||||
## Write the task
|
||||
|
||||
Write `{"name", "prompt", "checks"}` from the draft and what you did, one
|
||||
per task, each with its own name.
|
||||
|
||||
- `name`: lowercase letters, digits and hyphens, not `smoke`.
|
||||
- `prompt`: what the requester wants, in their words and the app's terms,
|
||||
starting with the app's name, naming the document as a user would and
|
||||
ending with saving it: "In Inkscape, change the poster's headline in
|
||||
Documents/poster.svg to Summer Bake Sale and save it." Name the app: the
|
||||
solving agent's own shell cannot see the computer's files, and the app's
|
||||
name sends it to the screen. Leave saving out for an app that saves on its own, after every change
|
||||
or a few seconds after the last one, such as Osmo or Gnote: grading waits
|
||||
until the app's files have held for 3 s, so the solving agent need not
|
||||
wait for the save. A save more than 3 s after the agent stops is still
|
||||
missed; name such an app in the report. Never say how the file stores it,
|
||||
and never give the checks.
|
||||
- Checks are file-based: content checks that prove the edit, never only that
|
||||
a file exists or changed. Start from the suggestions; keep those that say
|
||||
what the user did and drop what follows from how the app saves.
|
||||
- Compare exact values, so a typo or a wrong number fails.
|
||||
- Read each file's `values` before writing checks: every value the change
|
||||
added, changed and removed, with `before` and `after` (an uncut scalar, or
|
||||
a preview cut at 600 characters) at the pointer a check names it by. Copy
|
||||
the pointer. Copy `after` into `value` only when it is an uncut scalar: a
|
||||
string without a trailing `…`, a number, or a boolean. Compact XML and a
|
||||
JSON array or object are preview strings, so a `value_equals` check needs
|
||||
the structured value from the file's view, and element text uses
|
||||
`text_equals` or `text_contains`.
|
||||
Only XML files, archives, JSON and key files have values: a text file's
|
||||
or SQLite database's `change` already lists its lines and rows.
|
||||
`refers_to` says what a number or id stands for, such as `category="44"`
|
||||
naming `<cat key="44" name="Groceries"/>`, or a new KMyMoney
|
||||
transaction's splits naming their account and payee; use it for the
|
||||
prompt's words. `alternatives` gives a changed value's pointer by the
|
||||
element's other names, such as a label beside an id, to pick another key
|
||||
without another export.
|
||||
Skip entries whose `note` is `by_position` (no check can use that
|
||||
pointer) or noise (`view_state`, `renumbered`, `volatile`,
|
||||
`timestamp`, `upkeep`, `layout`). `values_cut` means a limit left some out.
|
||||
- Select elements by what they hold, never by position: SVG text by id,
|
||||
`/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0`,
|
||||
or an ODF paragraph by style,
|
||||
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Title]/children/0`.
|
||||
A selector must match exactly one element, so give the target a unique id
|
||||
or style in the seed. The draft's pointers already do this: a name-like
|
||||
attribute such as Scribus's `ANNAME` before a coordinate, and two
|
||||
attributes together where one is not enough, as a spreadsheet cell's
|
||||
`[@Col=3][@Row=2]`. Only an element's text is read by position, as
|
||||
`children/0`.
|
||||
- When an app may split an element's text into runs, check it whole:
|
||||
`text_equals` (or `text_contains` for a value inside a sentence) on the
|
||||
element compares all its text, however it is split. Add
|
||||
`normalize_whitespace: true` when the app may reflow spaces. Example, an
|
||||
AbiWord paragraph by its stable id:
|
||||
`{"kind": "text_contains", "file": "Documents/memo.abw", "pointer": "/root/children/[name=section]/children/[name=p][@xid=6]", "value": "at 3:30 PM"}`.
|
||||
Point a text check at the paragraph or another element the edit keeps,
|
||||
never at a run such as AbiWord's `[name=c]`: a reopened document splits
|
||||
the text into several runs. The draft lists only what changed; an
|
||||
unchanged element is in `draft.guards` when it is a like sibling of a
|
||||
changed one, and otherwise use the app's usual numbering, such as another
|
||||
paragraph's `xid`, and let the export confirm it.
|
||||
- In an ODS spreadsheet no attribute names a row: the draft names it by its
|
||||
label cell, `[name=table:table-row][>table:table-cell=February]`. Where no
|
||||
cell names it alone, name the sheet in the seed, and check its whole
|
||||
text: `text_equals` on
|
||||
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:spreadsheet]/children/[name=table:table][@table:name=Q1 Sales]`,
|
||||
or a stretch of it with `text_contains`. Cell texts join with nothing
|
||||
between, so a row reads `February23100`.
|
||||
- For a line added to free text, such as an item on a note's list, the
|
||||
draft compares the whole text with `text_equals`. This rejects longer or
|
||||
negated items, but can reject a correct edit with different ordering or
|
||||
spacing. Report that limitation when the task permits such variations.
|
||||
- For a note placed by its own `x` and `y`, such as Xournal++'s `<text>`
|
||||
on a layer, the draft checks its text on the layer with `text_contains`,
|
||||
which passes wherever the user puts it, and guards each note it kept.
|
||||
- Some apps keep text in an attribute, one element per run, such as
|
||||
Scribus's `<ITEXT CH=...>`: add `"text_attribute": "CH"` to `text_equals`
|
||||
or `text_contains` on the frame,
|
||||
`/root/children/[name=DOCUMENT]/children/[name=PAGEOBJECT][@ANNAME=Headline]`,
|
||||
to read its runs' values joined.
|
||||
- When the draft shows no values or no checks, a scratch export can expose
|
||||
`envs/<env>/seed/` only after the task has at least one valid check: it must
|
||||
pass on the captured end state and fail on the seed. Before the first
|
||||
successful export, use `act_on_build_computer` to open the saved file in an
|
||||
available GUI text editor or viewer and inspect its actual representation
|
||||
without changing it. Use the visible saved content and the task's edit to
|
||||
write a check; an export refusal can also name the end state's value, such
|
||||
as `is "44", not "1"`. If the file cannot be inspected or expressed by a
|
||||
supported check, report the limitation instead of guessing a check.
|
||||
- Where easy, add one check that the rest is untouched, such as another
|
||||
text's value the edit must leave alone. `draft.guards` offers them at
|
||||
their seed values, which the end holds too: a field on the nearest like
|
||||
siblings, one the user sets, such as Planner's `work`, before a default
|
||||
they all share; the text an addition kept; a text file's lines beside its
|
||||
new one. Copy one or two into `checks`; they pass on the seed, which is
|
||||
fine, since the export needs only one check to fail there.
|
||||
- The draft suggests a decimal the app writes as text, such as `146.83`,
|
||||
as a number with `tolerance: 0.001`; keep the tolerance for any number
|
||||
you check that way.
|
||||
- When the draft shows an XML document's changes as `lines`, REA read it
|
||||
as text, for example because its DOCTYPE declares entities. Selectors are
|
||||
impossible there, and `line_count` with `contains` only approximates a
|
||||
value: `3:30` also matches `13:30`. Bound it with neighbouring text and
|
||||
name the check as approximate in the report's limitations.
|
||||
- Some apps save a document reloaded from disk differently from one typed
|
||||
in the same session: AbiWord wraps each new insertion in its own markup.
|
||||
Your build's end state hides this, which is why whole-text checks matter;
|
||||
expect the witness to show the real format.
|
||||
|
||||
The kinds and pointer rules are in
|
||||
[the desktop tasks section](environments.md#pick-the-checks).
|
||||
|
||||
### Read a binary file with a reader
|
||||
|
||||
When the draft notes `unreadable_view` on a binary document, add a reader to
|
||||
the task: the app's own headless command, which REA runs in the computer on a
|
||||
private copy of the file, and whose output a check reads instead of the file.
|
||||
Write it in the app's scripting mode, print each fact on a line starting
|
||||
with `REA ` (with the space), and name the copy `{file}`:
|
||||
|
||||
- GIMP `.xcf`, layer names, as `lines`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "layers",
|
||||
"file": "Documents/poster.xcf",
|
||||
"argv": ["gimp-console-2.10", "-i", "-d", "-f", "-b", "-"],
|
||||
"stdin": "(let ((image (car (gimp-file-load RUN-NONINTERACTIVE \"{file}\" \"{file}\")))) (for-each (lambda (layer) (display \"\\nREA layer \") (display (car (gimp-item-get-name layer)))) (vector->list (cadr (gimp-image-get-layers image)))) (newline))\n(gimp-quit 0)\n",
|
||||
"output": "lines"
|
||||
}
|
||||
```
|
||||
|
||||
checked with `{"kind": "lines_present", "file": "Documents/poster.xcf", "reader": "layers", "lines": ["layer Summer Sale"]}`.
|
||||
Script-Fu's prompt `ts> ` starts a line, so print a newline before each
|
||||
marked line, as `\nREA` does. A text layer's text is
|
||||
`(car (gimp-text-layer-get-text layer))`.
|
||||
|
||||
- Blender `.blend`, objects with their type and location, as `json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "objects",
|
||||
"file": "Documents/scene.blend",
|
||||
"argv": [
|
||||
"blender",
|
||||
"-b",
|
||||
"--factory-startup",
|
||||
"{file}",
|
||||
"--python-expr",
|
||||
"import bpy, json; print('REA ' + json.dumps({o.name: {'type': o.type, 'location': [round(v, 4) for v in o.location]} for o in bpy.data.objects}, sort_keys=True))"
|
||||
],
|
||||
"output": "json"
|
||||
}
|
||||
```
|
||||
|
||||
checked with `{"kind": "value_equals", "file": "Documents/scene.blend", "reader": "objects", "pointer": "/Crate/location/0", "value": 2, "tolerance": 0.001}`.
|
||||
Print every value a check compares as a number or string a check can
|
||||
name, rounded where the app stores floats.
|
||||
Blender 4 shows its Quick Setup splash at every launch until its
|
||||
preferences are saved: with `.config/blender` in `state_paths`, confirm
|
||||
Quick Setup with Continue, save the preferences (Edit → Preferences, then
|
||||
Save Preferences in the menu at its bottom left), and read a probe
|
||||
checkpoint's `changed_files` for `.config/blender/4.0/config/userpref.blend`
|
||||
before `seed`, so solves start on the scene. Blender often takes the first click on a
|
||||
menu or a dialog's button only to focus it: click again.
|
||||
|
||||
- A format LibreOffice opens but REA cannot read, such as a legacy `.xls`,
|
||||
converted with `--convert-to`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "cells",
|
||||
"file": "Documents/budget.xls",
|
||||
"argv": [
|
||||
"sh",
|
||||
"-c",
|
||||
"soffice --headless --convert-to csv --outdir out \"$1\" >/dev/null && sed 's/^/REA /' out/*.csv",
|
||||
"sh",
|
||||
"{file}"
|
||||
],
|
||||
"output": "lines",
|
||||
"timeout_s": 120
|
||||
}
|
||||
```
|
||||
|
||||
Pass `{file}` to `sh -c` as an argument, as here, rather than inside the
|
||||
script.
|
||||
|
||||
The reader runs without the display, in a fresh home with the app's default
|
||||
settings, so use the app's headless or batch mode; set `display: true` only
|
||||
for an app that has none. It must exit 0 within `timeout_s` (60 s by
|
||||
default, at most 300) and print at most 1 MiB. Pick facts the change sets and
|
||||
the near miss gets wrong, such as a name or a number, not the app's whole
|
||||
output.
|
||||
|
||||
Give the task the reader in `readers`, and each check through it `reader`
|
||||
and the reader's `file`. `json` readers serve `value_equals`, `value_absent`
|
||||
and `member_count`; `lines` readers `lines_present`, `lines_absent` and
|
||||
`line_count`. The export runs the reader in the build computer on the seed
|
||||
and the end: when a check through it fails on the end, the refusal shows
|
||||
what the reader printed, and a reader that fails shows its error and the end
|
||||
of its stderr. Fix the reader or the check from that and export again.
|
||||
|
||||
### Judge the screen
|
||||
|
||||
Only when no file shows the change, as when the draft notes `no_change`
|
||||
because the app keeps its state on screen, write the task's checks as
|
||||
screen judges: `{"id": "shown", "kind": "screen_judge", "question": "Does the
|
||||
calculator's display show 8.568?"}`. A model, the judge, answers the question
|
||||
on a screenshot three times, and the check passes when most answers are yes.
|
||||
|
||||
- REA keeps a screenshot at each checkpoint. Mark `seed` with the screen
|
||||
showing the state before the task, where the answer is no, and `end` with
|
||||
the change showing, where it is yes: the bundle ships both as the judge's
|
||||
examples.
|
||||
- Ask about what the change makes visible, with exact values, so the near
|
||||
miss is a no: a result, a label, a selected item. Avoid what a window
|
||||
moved or a cursor blinking can change.
|
||||
- The prompt asks to leave the result showing, since the judge sees only
|
||||
the screen when the agent stops.
|
||||
- A task with screen judges holds only screen judges and no readers. The
|
||||
export refuses one when a file shows the change, and checks none of its
|
||||
questions: the witness does, running the bundle's own judge with the
|
||||
runner's model endpoint (`LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
|
||||
`[model]` in `.agentenv/config.toml`) on a fresh screenshot at each stage.
|
||||
Without an endpoint the witness's check fails: say what to set and stop.
|
||||
- The judge's model is `[model.roles] judge`, else `[model] default`, else
|
||||
`anthropic/claude-sonnet-5-5`. `agent-env run --judge-model` does not
|
||||
reach it. Each grade costs about $0.02.
|
||||
|
||||
## Export
|
||||
|
||||
`export_build_environment` with `tasks` and `directory`, the absolute path of
|
||||
a new or empty folder here, such as `<home>/rea-builds/poster/bundle`. Each
|
||||
entry of `tasks` is `{"task": ..., "end_checkpoint": ...}`: leave
|
||||
`end_checkpoint` out for the task checked at `end`, and give
|
||||
`end-<task name>` for each later one. The
|
||||
export seeds what was saved at `seed`, replaces the computer's host name,
|
||||
`user@host` and home path (a bare user name, such as AbiWord's author
|
||||
`user`, stays), and checks each task against `seed`, its own end and its
|
||||
`near-<task name>` when you took one. It refuses a task
|
||||
unless at least one check fails on the seed, every check passes at its
|
||||
end and one fails at its near miss, and says which task and why, such as a
|
||||
pointer that names nothing at the end. A
|
||||
check that the rest is untouched passes on the seed too, which is fine. On
|
||||
a refusal, fix the task and export again, into another folder if that one
|
||||
is no longer empty. The bundle's eval lists every task, so a run of the
|
||||
bundle runs them all. A refusal naming a secret's handle means a file of the
|
||||
bundle holds that secret, most often an app's sign-in or licence state in
|
||||
the seed: do not work around it, but report it and stop. Keep
|
||||
`seed_files`, `scrubbed_files`, `unscrubbed_files` and `limitations` for the
|
||||
report.
|
||||
|
||||
The limitations name the folders the app wrote that the bundle neither owns
|
||||
nor seeds, such as `~/.config/MuseScore`, so the app starts without them:
|
||||
expect its first-run wizard, a tour or a "restore session?" question at every
|
||||
boot. If the witness shows one, capture again on a new build computer with
|
||||
those folders in `state_paths`, and dismiss the dialogs before `seed`. Lock
|
||||
files the app holds while it runs, such as LibreOffice's `.lock` and
|
||||
`.~lock.<name>#`, GnuCash's `<book>.LCK` or Grisbi's `.<file>.lock`, are
|
||||
left out of the seed and listed after "Left out of the seed". An app can
|
||||
still start fine yet refuse every save when a lock was seeded, so close the
|
||||
document before `seed` where the app lets you (apps other than LibreOffice,
|
||||
such as Rosegarden, also write `.~lock.<name>#` and autosaves beside a
|
||||
document, and closing it covers them too), and treat "file is locked"
|
||||
on a save in the witness as a seeded lock: delete it from the bundle's seed
|
||||
and witness again in a new computer. In LibreOffice, own the profile
|
||||
(`state_paths: [".config/libreoffice"]`) for a clean boot, and close the
|
||||
document before `seed`, then reopen it from Recent Documents to make the
|
||||
change: the export also leaves out the recovery list LibreOffice writes while
|
||||
a document is open, which would start the bundle into document recovery. A
|
||||
limitation naming `~/.config/dconf/user` as unread means the app's dconf
|
||||
settings, such as a first-run tip you turned off, are not seeded: capture
|
||||
again on a new build computer. The limitations also name seeded WebKit,
|
||||
Chromium or QtWebEngine storage and caches; when the witness's boot is clean,
|
||||
list them in the report.
|
||||
|
||||
Keep the build computer open until the witness holds: a task fix exports
|
||||
again from it.
|
||||
|
||||
## Witness
|
||||
|
||||
Witness each task in its own fresh computer of the bundle, from the prompt,
|
||||
as a solving agent would; a computer is witnessed once, so the steps below
|
||||
repeat for each task, each with its own record folder. Do not replay the build's actions or coordinates, and do not
|
||||
check between stages to steer the edit: a solving agent never sees the
|
||||
checks. Give every `check_build_task` the task exactly as exported.
|
||||
|
||||
1. `start_build_computer` with
|
||||
`{"source": {"kind": "bundle", "path": "<bundle>"}}`, adding
|
||||
`"settings": {"<variable>": "<handle>"}` for each of the export's
|
||||
`withheld_files`, or the deploy is refused for want of them, and agentenv
|
||||
keeps that bundle's build failed, so start from a fresh copy of it. The app starts
|
||||
with the seed's files in place and opens the seed's document when the
|
||||
seed holds exactly one and the app takes a file. When the export's
|
||||
`limitations` say it opens no document, open it from the prompt's path,
|
||||
as a solving agent would. Take a `wait` screenshot first, since the
|
||||
desktop may not have drawn yet. When the start fails because the
|
||||
computer never got ready, and its error names lock files in the bundle's
|
||||
seed, delete them from the bundle and start again.
|
||||
2. `check_build_task` with `stage: "seed"` and `directory`, the absolute
|
||||
path of a new or empty folder for the record. A check must fail.
|
||||
Recording starts. Later stages take no `directory`.
|
||||
3. Do the task from the prompt with `act_on_build_computer` and save.
|
||||
`check_build_task` with `stage: "after_edit"`: every check must pass.
|
||||
Each check waits until the app's files have held for 3 s, as the
|
||||
bundle's verifier does, so an app that saves on its own needs no wait of
|
||||
yours. A result with `settled: false` means the files were still
|
||||
changing at the 15 s bound, and `read_cut: true` that a slow read was
|
||||
still running there.
|
||||
4. Make a near miss on top of it: a plausible wrong edit, such as a typo or
|
||||
a wrong number, and save. `check_build_task` with `stage: "near_miss"`:
|
||||
a check must fail.
|
||||
5. `act_on_build_computer` with `action: "reset"`, then `check_build_task`
|
||||
with `stage: "after_reset"`: a check must fail again.
|
||||
6. The last stage ends the witness: `witness.record` has the verdict, the
|
||||
path of `witness.json` and the recording's segments in `directory`.
|
||||
`close_build_computer` on the witness computer.
|
||||
|
||||
A stage that does not hold ends the witness as `failed`. Read each check's
|
||||
result and decide:
|
||||
|
||||
- `after_edit` failed though the saved file is right: the check is brittle.
|
||||
Fix it, export again and witness in a new computer.
|
||||
- `after_edit` failed because your edit was wrong: witness again in a new
|
||||
computer; a computer is witnessed once.
|
||||
- `near_miss` passed: the checks are too loose. Tighten them, export again
|
||||
and witness again.
|
||||
- `after_reset` passed: the reset did not restore the seed. Report it as a
|
||||
limitation.
|
||||
|
||||
Stop after three witnesses of a task and report what still fails.
|
||||
|
||||
## Measure
|
||||
|
||||
Only if the requester asked for it, or by the headless default above. If
|
||||
they wanted to review the tasks first, show them each prompt and checks and
|
||||
wait for their approval. Then `measure_build_task` each task with `bundle`,
|
||||
`task`, `model` and `budget_usd`, its share of the budget. It runs the task
|
||||
in fresh computers, 3 times, then 2 more unless they passed none or all, and
|
||||
keeps a task that passes 1 to 3 of 5 (`band: "in_band"`). It stops before a
|
||||
solve the largest cost so far would take past the budget, and writes the
|
||||
record into the bundle's `difficulty.json` and README. The solver needs a
|
||||
model endpoint where REA starts: `LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
|
||||
`base_url` and `api_key` under `[model]` in `.agentenv/config.toml`; when the
|
||||
measure says a solve did not run for want of one, tell the person what to
|
||||
set and stop. Read the result's `failures_per_check` and each solve's
|
||||
`failed_checks` before calling a task hard: a check that fails in most failed
|
||||
solves while the app shows the right change is brittle, so fix the check,
|
||||
export and witness again, rather than count it as difficulty.
|
||||
|
||||
## Harder tasks
|
||||
|
||||
When the requester wants tasks worth evaluating or training on, harden a
|
||||
task that came out `too_easy` and measure it again, within the budget. The
|
||||
solver sees and fixes its slips on screen, so one edit with the value in the
|
||||
prompt is solved almost every time; so far, on TreeLine with Sonnet 5.5, so
|
||||
were one more dialog, a conditional move of seven records, and ten
|
||||
corrections read from a second document and applied across the list. Try,
|
||||
one at a time, measuring each:
|
||||
|
||||
- A longer chain of exact edits than that, each a value the checks compare
|
||||
whole, so one slip in the chain fails the task.
|
||||
- Facts read in one view and used in another: values the solver must read
|
||||
from a second document the app opens, or from another record, rather than
|
||||
from the prompt, such as "the same year as The Quiet Glacier's".
|
||||
- Distractors beside each target: records with near-identical names or
|
||||
values, and a guard check on each that it stays as it was.
|
||||
- Mistakes that do not show once saved: a value the app displays rounded or
|
||||
formatted unlike what it stores, or an edit the app keeps only when its
|
||||
field loses focus.
|
||||
|
||||
Stop when the budget left cannot cover 3 solves of the hardened task, and
|
||||
report the bands measured.
|
||||
|
||||
Each hardened task stays sound: its checks still fail on the seed and pass on
|
||||
its end, its near miss fails, and another valid way of doing it passes, so
|
||||
check values, never the order or method the solver used. Put the second
|
||||
document in the seed. Count records with `member_count` and check each
|
||||
value by a selector that names its record, since no check joins two records.
|
||||
|
||||
## Close and report
|
||||
|
||||
`close_build_computer` on every computer still open, also after a failure.
|
||||
If a close fails, call it again; if it still fails, tell the requester what
|
||||
is left.
|
||||
|
||||
Report:
|
||||
|
||||
- the bundle path and the environment name;
|
||||
- each task's name, prompt and end checkpoint;
|
||||
- each check, and why it proves the edit and fails the near miss, and each
|
||||
reader's command;
|
||||
- each task's witness stages with their results, the verdict, and the paths of
|
||||
`witness.json` and the recording;
|
||||
- the seed's contents: `seed_files` and what each holds;
|
||||
- `scrubbed_files` and `unscrubbed_files`; an unscrubbed file still names
|
||||
the host or user, so it needs review before sharing. `recompressed_files`
|
||||
are gzip seeds whose headers were cleared, their content unchanged: give
|
||||
their count;
|
||||
- limitations: the capture's, the export's and anything you could not do;
|
||||
- each measured task's band, passes of finished solves, turns, cost and
|
||||
failures per check, from `difficulty.json`;
|
||||
- the build time and the questions you asked.
|
||||
|
||||
Never publish or share the bundle yourself.
|
||||
@@ -294,7 +294,9 @@ Use this when the user has a folder from `rea capture-desktop-application` and
|
||||
wants a task for the environment `rea export-desktop-application` makes from
|
||||
it: a prompt for a computer-use agent and checks that grade what the agent
|
||||
left in the app's files. You write the prompt and the checks; REA validates
|
||||
the task against the capture and exports it. These commands are CLI only.
|
||||
the task against the capture and exports it. These commands are CLI only. To
|
||||
build the environment and its task from a request instead, follow
|
||||
[desktop builds](desktop-build.md).
|
||||
|
||||
### Describe the change
|
||||
|
||||
@@ -306,14 +308,27 @@ Run `rea describe-desktop-application-task CAPTURE --format json`. Read:
|
||||
and last version, with its `role`, the `baseline` and `final` versions and
|
||||
the net `change`. Text changes list each added and removed line once, and
|
||||
`line_counts` gives lines found at both ends whose count changed.
|
||||
- `values` in each of those files: what the change added, changed and
|
||||
removed, `before` and `after`, at the pointer a check names it by, with
|
||||
`refers_to` naming the element a key such as `category="44"` stands for.
|
||||
Skip entries noted `by_position` or as noise. Only XML files, archives,
|
||||
JSON and key files have values: a text file's or database's change
|
||||
already lists its lines and rows.
|
||||
- `task`: a draft whose `checks` are suggestions from the net change of the
|
||||
documents the user worked on, or of the app's own state when no document
|
||||
changed.
|
||||
- `guards`: optional checks that the rest kept its seed values, such as a
|
||||
like sibling's field or a text file's neighbouring lines; add one or two
|
||||
to show the rest is untouched.
|
||||
- `notes`: why `files` is empty, or why a file's suggestions are thin, such
|
||||
as `by_position` when no attribute names the changed element.
|
||||
|
||||
### Write the prompt
|
||||
|
||||
Say what the user wants done, in their words and the app's terms: "Add buy
|
||||
milk to my to-do list and save it". Never say how the file stores it: no
|
||||
Say what the user wants done, in their words and the app's terms, naming the
|
||||
app: "In Mousepad, add buy milk to my to-do list and save it". A solving
|
||||
agent's own shell cannot see the computer's files; the app's name sends it to
|
||||
the screen. Never say how the file stores it: no
|
||||
paths the user would not name, line formats, XML, keys or SQL, and never the
|
||||
checks. Replace the draft's placeholder; the export refuses it.
|
||||
|
||||
@@ -344,13 +359,18 @@ document saved as a zip archive, such as FreeCAD's `.FCStd`, reads as its
|
||||
entries: point into an entry's view, and name array members by selector
|
||||
rather than position, as
|
||||
`/entries/[name=Document.xml]/view/root/children/[name=Object][@name=Hole]`,
|
||||
so the check survives an app version that writes them in another order. Give
|
||||
so the check survives an app version that writes them in another order. A
|
||||
gzip-compressed document, such as Gnumeric's `.gnumeric`, reads as the
|
||||
document inside: point into it as into the same document uncompressed. Give
|
||||
`value_equals` a `tolerance` to compare a number the app writes as text, such
|
||||
as `"5.0000000000000000"`, with a number. Check what the user wrote in a
|
||||
document with `text_equals` on its paragraph element, not `value_equals` on a
|
||||
text node: apps such as AbiWord and Writer split a paragraph into runs as they
|
||||
save, so its first text node holds only part of it; use `text_contains` for a
|
||||
value within a sentence, such as a time. The
|
||||
value within a sentence, such as a time. A binary document, such as GIMP's
|
||||
`.xcf`, has no view: give the task a `readers` entry, the app's own headless
|
||||
command printing `REA ` lines, and a check naming it in `reader`, as
|
||||
[desktop builds](desktop-build.md#read-a-binary-file-with-a-reader) shows. The
|
||||
[desktop application tasks](https://github.com/morluto/rea/blob/main/docs/desktop-application-tasks.md)
|
||||
guide has the exact rules.
|
||||
|
||||
@@ -361,7 +381,8 @@ Write the task as JSON with a `name`, the `prompt` and the `checks`, then run
|
||||
the task only when its checks fail on the seed and all pass on the captured
|
||||
end state. A refusal names the check: one that fails on the end state does not
|
||||
match what the user left; "every check already passes on the seed" means no
|
||||
check captures the change.
|
||||
check captures the change. For several tasks from one capture, repeat
|
||||
`--task` and name each later task's end with `--task-end NAME=CHECKPOINT`.
|
||||
|
||||
When the export refuses an application that no apt repository installs, ask
|
||||
the user for the `.deb` file they installed it from and capture with
|
||||
@@ -372,5 +393,7 @@ user's to choose.
|
||||
|
||||
Tell the user the task's name, prompt and checks, and how to measure a solve
|
||||
rate: run `agent-env run DIRECTORY --task NAME --agent claude-code` with
|
||||
`--model MODEL` three times and count the runs reported `passed`. Do not run
|
||||
`--model MODEL` three times and count the runs reported `passed`. The runs
|
||||
need a model endpoint: `LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
|
||||
`base_url` and `api_key` under `[model]` in `.agentenv/config.toml`. Do not run
|
||||
agents unless asked. The bundle's seed is the user's real data: keep it local.
|
||||
@@ -47,7 +47,7 @@ member comparison CLI still takes two assembly paths; process comparison takes
|
||||
two capture Evidence JSON paths. Check command help for the installed version.
|
||||
An input bundle grants no implicit file lookup: `evidence-file` and historical
|
||||
graph file selectors are accepted only by contracts advertising them. See the
|
||||
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/cli.md#import-export-and-compare-evidence).
|
||||
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/getting-started/cli.md#import-export-and-compare-evidence).
|
||||
|
||||
Completed `inspect_artifact` inventory also supports a summary and
|
||||
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
|
||||
|
||||
@@ -6,7 +6,7 @@ For captured or bundled JavaScript that needs readable source recovery, use
|
||||
`additional_paths` when advertised, then analyze the returned `analysis_input`
|
||||
directory together. Each input retains its own snapshot, digest and byte-range
|
||||
provenance; inputs and recovered code are never executed. Recovery requires a
|
||||
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
|
||||
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/guides/javascript/javascript-recovery.md).
|
||||
|
||||
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
|
||||
extracted tree. When the question concerns known source paths and the connected
|
||||
|
||||
@@ -67,11 +67,18 @@ See [desktop installations](https://github.com/morluto/rea/blob/main/docs/deskto
|
||||
Use `inspect_go_binary` for an explicit local ELF, PE or Mach-O (universal
|
||||
files need `architecture`) when the
|
||||
question concerns the embedded compiler version, module/dependency versions,
|
||||
local replacements or build settings. No `open_binary`, Go installation or
|
||||
local replacements, build settings, or static Go string literals packed without
|
||||
NUL terminators. Pass `include_string_literals: true`
|
||||
(`--include-string-literals`, optional `max_string_literals` /
|
||||
`--max-string-literals`) to recover bounded inferred word-aligned `(ptr, len)`
|
||||
UTF-8 string slices from read-only data and page them through `go`
|
||||
`string_literals` views. No `open_binary`, Go installation or
|
||||
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
|
||||
Preserve artifact SHA-256, original file ranges, framed module bytes and every
|
||||
unparsed module-text line. A missing record is an unknown, not proof of a
|
||||
non-Go binary; observed metadata does not authenticate its claims. Function,
|
||||
non-Go binary; observed metadata does not authenticate its claims, and static
|
||||
`(ptr, len)` string recovery excludes compiler-inlined stack/immediate strings
|
||||
without proving runtime reachability. Function,
|
||||
source and type recovery remain a separate native-analysis question.
|
||||
|
||||
## Model files
|
||||
@@ -142,6 +149,20 @@ table or overlay. Bound snapshots, loaded IAT contents, and export-forward
|
||||
resolution remain unobserved. See
|
||||
[PE import and delay-load tables](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
|
||||
|
||||
## Windows Shell Link shortcuts
|
||||
|
||||
Use `inspect_windows_shortcut` for an explicit local `.lnk` file when the
|
||||
question concerns the recorded target path, arguments, working directory, icon
|
||||
location, volume serial, network share names, ExtraData blocks, or Tracker
|
||||
droid identifiers. No `open_binary`, Windows SDK, COM, or target execution is
|
||||
required. The CLI equivalent is `inspect-windows-shortcut PATH`. Preserve
|
||||
artifact SHA-256, FILETIME raw decimals, IDList type bytes, and ExtraData
|
||||
signatures. Unknown IDList items and ExtraData signatures remain raw with
|
||||
SHA-256; they are not guessed. Version-1 UUID MAC and timestamp values are
|
||||
labelled derived. A missing optional section is observed absence of that
|
||||
structure. See
|
||||
[Windows Shell Link shortcuts](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
|
||||
|
||||
## PE TLS callbacks and load configuration
|
||||
|
||||
Use `inspect_pe_load_config` for an explicit local PE32/PE32+ image when the
|
||||
@@ -178,6 +199,36 @@ absence of those declarations, not proof the image never carried notes in
|
||||
another container. See
|
||||
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
|
||||
## ELF and Mach-O initializer functions
|
||||
|
||||
Use `inspect_native_initializers` for an explicit local ELF or Mach-O image
|
||||
when the question is which functions run before `main` or at exit: ELF
|
||||
`DT_PREINIT_ARRAY`, `DT_INIT`, `DT_INIT_ARRAY`, `DT_FINI`, `DT_FINI_ARRAY` and
|
||||
legacy `.ctors`/`.dtors`, or Mach-O `__mod_init_func`, `__init_offsets` and
|
||||
`__mod_term_func`. Pass `architecture` for a universal Mach-O. The CLI
|
||||
equivalent is `inspect-native-initializers PATH`. PIE slots are resolved
|
||||
through RELATIVE, RELR and section relocations, and Mach-O slots through
|
||||
chained fixups. Keep each entry's `resolution`: `unresolved` means the value was
|
||||
not established, and `unresolved_import` names an external symbol. Page with
|
||||
`offset` and `limit`. See
|
||||
[ELF and Mach-O initializer functions](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
|
||||
|
||||
## ELF unwind function ranges
|
||||
|
||||
Use `inspect_elf_unwind_ranges` for an explicit local ELF32/ELF64 image when
|
||||
the question concerns function starts and lengths that survive `strip`, which
|
||||
functions have an LSDA, or which ranges have no exact symbol. No
|
||||
`open_binary` or target execution is required. The CLI equivalent is
|
||||
`inspect-elf-unwind-ranges PATH`. Preserve artifact SHA-256, FDE start and
|
||||
end, containing section, exact symbol name, CIE index, LSDA, and personality.
|
||||
Page later ranges with `offset` and `limit`. A missing `.eh_frame` is
|
||||
observed absence of that declaration and of `PT_GNU_EH_FRAME`, not proof the
|
||||
image has no functions. Header disagreement is a limitation and does not drop
|
||||
decoded ranges. `.debug_frame` is reported present or absent and is not
|
||||
decoded. CFA instructions are counted and not evaluated. Indirect EH pointers
|
||||
are not dereferenced. See
|
||||
[ELF unwind function ranges](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
|
||||
|
||||
## Apple IPS crash reports
|
||||
|
||||
Use `inspect_apple_ips` for an explicit local Apple `.ips` file when the
|
||||
@@ -187,7 +238,10 @@ Xcode, atos, or target execution is required. The CLI equivalent is
|
||||
`inspect-apple-ips PATH`. Preserve artifact SHA-256, `bug_type` as a string,
|
||||
uppercase image UUIDs, and integer literals that exceed IEEE-754 range. Only
|
||||
`bug_type` `"309"` yields a crash body; other DiagnosticReports types return
|
||||
no crash result. Opened Mach-O and dSYM UUID joins remain unobserved. See
|
||||
no crash result. Select `join_path` for one local Mach-O or dSYM to match
|
||||
report images to slices by UUID and architecture. Preserve the selected
|
||||
artifact identity, slice facts and unjoined reasons; a joined static address
|
||||
does not resolve function names or dSYM symbols. See
|
||||
[Apple IPS crash reports](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
|
||||
|
||||
## Linux kernel modules
|
||||
@@ -209,6 +263,27 @@ container. gzip, xz, and zstd whole-file containers are unsupported and name
|
||||
decompress into a separate file, then call `inspect_kernel_module` on that file. See
|
||||
[Linux kernel modules](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
|
||||
|
||||
## Linux kernel build config
|
||||
|
||||
Use `inspect_kernel_build_config` for an explicit local kernel image (`vmlinux`
|
||||
ELF, raw arm64 `Image`, x86 `bzImage`/`zImage`, or `configs.ko`) when the
|
||||
question is which kernel version it is or how it was configured. No
|
||||
`open_binary`, Linux host, or `extract-ikconfig` is required. The CLI
|
||||
equivalent is `inspect-kernel-build-config PATH`; pass `name` for one exact
|
||||
`CONFIG_*` option and `offset`/`limit` to page the rest. Preserve the banner
|
||||
version, builder, and compiler, the config SHA-256 and option count, and each
|
||||
entry's value kind (`y`, `m`, `string`, `number`, `not_set`). A config is
|
||||
accepted only when `IKCFG_ST` is immediately followed by a valid gzip member
|
||||
and `IKCFG_ED`, so an ordinary `IKCFG_ST` string is not a config. Offsets with
|
||||
`space: "inflated_payload"` are positions in an inflated bzImage payload, not
|
||||
file offsets. An absent config describes only the searched bytes; the kernel
|
||||
may have been built without `CONFIG_IKCONFIG` or with it as a module. Truncated
|
||||
payloads are incomplete, not absent. A framed blob that does not decode, has a
|
||||
bad CRC, or holds non-config text is `malformed`, not absent. xz, lz4, lzma, zstd, bzip2, and lzo
|
||||
payloads are unsupported and name `external_decompression`: decompress with an
|
||||
external utility, then call `inspect_kernel_build_config` on the result. See
|
||||
[Linux kernel build config](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
|
||||
|
||||
## CPython pyc files
|
||||
|
||||
Use `inspect_python_pyc` for an explicit local CPython 3.7–3.13 `.pyc` when the
|
||||
@@ -246,7 +321,7 @@ Choose `row_limit` when a finite sample is useful and keep truncation and
|
||||
unspecified row order explicit. Virtual tables, shadow tables and views are
|
||||
schema facts, not row-read targets. REA inspects private copies and leaves
|
||||
the source files untouched; obtain a coherent quiescent database/WAL snapshot
|
||||
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
|
||||
|
||||
## Recorded Linux crashes
|
||||
|
||||
@@ -261,7 +336,7 @@ The optional `include_debugger_context` facet adds mapping candidates through
|
||||
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
|
||||
establish current file identity; zero reported flags leave permissions unknown.
|
||||
Requested unavailable context fails with setup guidance. See
|
||||
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md)
|
||||
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md)
|
||||
for exact upstream profiles, bounds and verification coverage.
|
||||
|
||||
## Offline ELF layout
|
||||
@@ -325,7 +400,8 @@ on the first MCP call. It retains the complete observation and returns the
|
||||
original selection, text availability and length without the body. Use
|
||||
`normalized_result.parent_evidence_id` with `inspect_analysis_view` and
|
||||
`view: {kind: "native", facet: "pseudocode", offset: 0, limit: 20000}`;
|
||||
follow `coverage.next_offset` for later pages. Offsets and limits count UTF-16
|
||||
follow `coverage.next_offset` for later pages. Choose a smaller limit if one
|
||||
page exceeds the client or transport budget. Offsets and limits count UTF-16
|
||||
code units, not lines or tokens. Pages reuse the same observation without
|
||||
repeating decompilation. Null and empty text remain distinct. Omitted or
|
||||
`complete` detail preserves full inline delivery. Summary delivery requires a
|
||||
@@ -393,7 +469,7 @@ In ephemeral mode, choose a private absolute directory and set
|
||||
`REA_GHIDRA_CHECKPOINT_DIR` in the environment starting REA. Restart the MCP server
|
||||
after changing that launch configuration; no current tool call or runtime
|
||||
analysis-setting update enables it. Stock ELF/PE/Mach-O imports on Linux/macOS
|
||||
are supported; see the [checkpoint setup and limitations](https://github.com/morluto/rea/blob/main/docs/installation.md#automatically-checkpoint-an-owned-ghidra-program).
|
||||
are supported; see the [checkpoint setup and limitations](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program).
|
||||
|
||||
With the setting enabled, normal opens restore compatible saved Programs
|
||||
automatically. Initial completed analysis is saved, and `close_binary` saves
|
||||
@@ -414,7 +490,7 @@ import overrides and repair scripts are unavailable in this mode. Treat original
|
||||
analysis completeness/settings as unknown and prior edits as caller-project state.
|
||||
The program's recorded import digest must match the target; changed project content
|
||||
requires reopening for a fresh profile. See the
|
||||
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/installation.md#reuse-an-existing-ghidra-project).
|
||||
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#reuse-an-existing-ghidra-project).
|
||||
|
||||
## Ghidra startup deadline recovery
|
||||
|
||||
@@ -448,7 +524,7 @@ and memory controls in place. Disable specific expensive analyzers only when
|
||||
their missing coverage is acceptable and retain the resulting limitations.
|
||||
Exported C and embedded resources remain useful independent evidence, but a
|
||||
timeout does not establish complete analysis or absence of code. See the
|
||||
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
|
||||
## Bridge request ceiling
|
||||
|
||||
@@ -599,7 +675,7 @@ broader metadata recovery on Linux; it requires an explicitly configured
|
||||
NativeAOT JAR. These are separate routes. The built-in parser also runs on
|
||||
Windows Ghidra P0; read frozen literals from its report, because
|
||||
`search_strings` does not include them. See the
|
||||
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/ghidra-nativeaot.md)
|
||||
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-nativeaot.md)
|
||||
for setup, supported layouts, and host restrictions. Preserve binary identity
|
||||
and distinguish derived metadata from Ghidra database observations. Recovered
|
||||
types and native pseudocode are not original C# source. Windows Ghidra P0 has
|
||||
@@ -628,7 +704,7 @@ every executable in the bundle. It also lists missing or weak loads and earlier
|
||||
provides them.
|
||||
|
||||
Use `extract_artifact` when materialized files from a supported format are needed.
|
||||
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md).
|
||||
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md).
|
||||
It materializes all regular files into a fresh temporary directory chosen by
|
||||
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
|
||||
The result reports the materialized directory; callers do not choose the
|
||||
@@ -656,7 +732,7 @@ temporary, not a retained export of the image's contents.
|
||||
For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
|
||||
`--target-format dos-com`). Inspect the load image before trusting function
|
||||
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
|
||||
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
|
||||
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-dos.md).
|
||||
|
||||
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
|
||||
|
||||
@@ -670,7 +746,7 @@ otherwise. Big-endian ARM, Thumb-only, and Cortex-M variants are unsupported.
|
||||
`0x0000..0xffff`. 65C02, 65816, cartridge containers, bank switching, and mapper
|
||||
semantics are unsupported. `m68k` requires an even entry and decodes the 68040
|
||||
superset of the 68000 family; Amiga/Atari containers and vector-table entry
|
||||
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
|
||||
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/providers/raw-binary.md).
|
||||
|
||||
Intel HEX, S-record, UF2 and DfuSe files are containers, not raw images:
|
||||
`inspect_artifact` lists address regions (UF2 per familyID, DfuSe per element
|
||||
@@ -684,6 +760,11 @@ numbering gaps leave coverage unknown; a DfuSe suffix CRC mismatch follows
|
||||
Fresh-import sessions admit annotations; existing-project reuse is read-only.
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits. Normal close saves them when owned Program checkpoint mode is enabled; ephemeral mode discards them. CLI `annotate-native-function` follows the same setting when closing after its updated result. Select the [checkpoint workflow](#continue-editable-ghidra-analysis-across-processes) before edits that must survive a later process. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier. Ghidra rejects batch addresses that request the same name or a name another selected address holds; `raw_result.failures` gives each failure's reason.
|
||||
|
||||
A provider switch or different target can retire the active Ghidra database and discard
|
||||
its edits, even if the new provider fails on a later call. Same-target,
|
||||
same-profile reopen keeps the active database. Retained annotation Evidence
|
||||
records what happened; it does not restore the edits.
|
||||
|
||||
For `compare_functions`, obtain `analyze_function` Evidence before and after the
|
||||
edit while the MCP session is open; annotation Evidence is not a comparison
|
||||
input. Export the needed Evidence before close if comparison will happen later.
|
||||
@@ -704,7 +785,7 @@ code pages, payload hashes and static icon-group references. Same-language icon
|
||||
resolution is explicit; missing references and size mismatch remain visible.
|
||||
Other payloads are opaque, and this operation does not decode images or simulate
|
||||
Windows resource loading. Consult the budgets and supported tree profile in
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
|
||||
## Offline PE and ELF export tables
|
||||
|
||||
@@ -714,7 +795,7 @@ directory or ELF dynamic symbol table on any host without opening a native
|
||||
target. Ordinal-only, forwarded and hidden-version exports stay distinct;
|
||||
partial or unsupported coverage means the inventory is incomplete, not that
|
||||
exports are absent. Use `inspect_macho` for Mach-O. See
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
|
||||
For exact core-module import/export differences, use `compare_wasm_interfaces`
|
||||
with two complete inspection Evidence records (or current-session retained
|
||||
@@ -830,3 +911,24 @@ Layout labels describe record shapes, not a detected compiler version; unknown
|
||||
or ambiguous layouts are unsupported. Bytecode is retained, not decoded, and the
|
||||
executable is never run. See
|
||||
[Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
|
||||
## Deno compile executables
|
||||
|
||||
A `deno compile` executable stores its application in a Mach-O `__SUI,d3n0l4nd`
|
||||
section, an Intel Mach-O `<~sui-data~>` trailer, an ELF `SUI` note of type
|
||||
`0x53554901`, or a PE `RT_RCDATA` resource named `D3N0L4ND`.
|
||||
`open_binary` then `inspect_artifact` reports root format `deno-standalone`
|
||||
with the carrier, entry point, recorded options, and one occurrence per virtual
|
||||
file, including size and SHA-256. Transpiled output, CJS export analysis, and
|
||||
source maps are separate occurrences with the file's `index` and their own
|
||||
`role`. They normally use a `.deno-*` suffix on the file path and move to an
|
||||
indexed path on a collision, so use the reported `logical_path`. Symlink targets
|
||||
stay recorded path parts. A virtual filesystem that cannot be listed leaves
|
||||
coverage unknown.
|
||||
Environment-file values and CA certificate bytes are retained in local Evidence
|
||||
with their counts and byte lengths. The npm snapshot length is reported and not
|
||||
decoded. `extract-artifact` writes
|
||||
stored files through the safe output tree; route extracted JavaScript to
|
||||
`analyze_javascript_application`. Other layouts are unsupported, and the
|
||||
executable is never run. See
|
||||
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
|
||||
@@ -41,10 +41,16 @@ behavior unknown. Record truncation, unmatched breakpoints, changed target
|
||||
identity and cleanup uncertainty as limitations, rather than treating no retained
|
||||
event as proof that the behavior never occurs.
|
||||
|
||||
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/frida-instrumentation.md),
|
||||
For same-session MCP comparisons, pass each capture inline or reuse it as
|
||||
`{"kind":"retained-evidence","evidence_id":"ev_..."}`. Default comparison
|
||||
keeps capture-local timing as provenance without treating scheduler or observer
|
||||
latency as a behavior change; use an exact `trace_spec` event when timing is part
|
||||
of the analyst question.
|
||||
|
||||
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/guides/runtime/frida-instrumentation.md),
|
||||
[GDB/Rizin/Cutter](https://github.com/morluto/rea/blob/main/docs/reverse-engineering-tools.md),
|
||||
[process capture](https://github.com/morluto/rea/blob/main/docs/process-capture.md),
|
||||
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/native-investigation.md#native-desktop-observation).
|
||||
[process capture](https://github.com/morluto/rea/blob/main/docs/guides/runtime/process-capture.md),
|
||||
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/guides/native/native-investigation.md#native-desktop-observation).
|
||||
|
||||
## Browser and Electron
|
||||
|
||||
@@ -130,5 +136,5 @@ Inspect native `.saz` snapshots with `inspect_web_network_capture` and
|
||||
traffic analysis, connect Everywhere's separate MCP server and use its
|
||||
advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or
|
||||
`AgentCalls`). Preserve existing sessions and filters when inspecting a
|
||||
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md)
|
||||
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md)
|
||||
for the workflow and Telerik's public setup and traffic-debugging skills.
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"name": "rea",
|
||||
"version": "6.8.0",
|
||||
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
|
||||
"author": {
|
||||
"name": "morluto",
|
||||
"url": "https://github.com/morluto"
|
||||
},
|
||||
"homepage": "https://rea.tools/",
|
||||
"repository": "https://github.com/morluto/rea",
|
||||
"license": "MIT",
|
||||
"keywords": [
|
||||
"reverse-engineering",
|
||||
"binary-analysis",
|
||||
"decompiler",
|
||||
"mcp-server"
|
||||
],
|
||||
"skills": ".cursor-plugin/skills",
|
||||
"logo": "assets/rea-plugin.png"
|
||||
}
|
||||
@@ -0,0 +1,607 @@
|
||||
---
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "62"
|
||||
---
|
||||
|
||||
# REA
|
||||
|
||||
Use REA when a claim depends on a shipped binary or package, decompilation,
|
||||
requested runtime evidence, or comparison with
|
||||
behavior not established by available source. For ordinary analysis of a
|
||||
complete source repository, use normal repository tools and do not run REA
|
||||
readiness or provider commands.
|
||||
|
||||
## Connect only when needed
|
||||
|
||||
Installing this skill supplies instructions; it does not register the REA MCP
|
||||
server, install analysis engines, or add tools to an already-running agent.
|
||||
The exception is a Claude Code, Codex, or Cursor Marketplace plugin install:
|
||||
the plugin bundles this skill and registers its own `rea` server, outside the
|
||||
state that `rea setup` and `rea doctor` manage.
|
||||
|
||||
If REA tools are available and their registration is not known to be stale,
|
||||
proceed directly to the target. Do not run diagnostics or setup before every
|
||||
investigation. Use the connected server's actual tool list and input schemas;
|
||||
a skill installed from repository main may describe capabilities absent from
|
||||
an older npm release. Keep complete inline Evidence when that server does not
|
||||
advertise retained references.
|
||||
|
||||
If tools are absent, a call is blocked by the client, or a required provider is
|
||||
unavailable, read [connection and recovery](references/connection-and-recovery.md).
|
||||
Diagnose the observed blocker, continue useful supported work, and resume the
|
||||
original task after repair. Client approval, shell PATH, registration and engine
|
||||
availability have different remedies; avoid repeating setup for all of them.
|
||||
If the host times out without a REA result, do not repeat the unchanged call or
|
||||
assume work stopped. Read the recovery guide's deadline distinction before retrying.
|
||||
If the MCP connection drops, retry that call once only when it is safe to
|
||||
repeat or known not to have started. If it may already have changed the
|
||||
target, or it fails again, tell the user and ask them to restart the MCP
|
||||
client. If REA could not start its connection, retry once, then tell the user
|
||||
and ask them to restart the MCP client. Do not retry a startup failure that
|
||||
returns the same unchanged result. For plugin installs, follow
|
||||
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin).
|
||||
For setup-owned registrations, run `rea doctor` when REA could not start
|
||||
or tools stay missing after that restart. For `cleanup_incomplete`, a retryable
|
||||
capacity limit, or `configuration_invalid`, follow the recovery guide's ladder
|
||||
before asking for a restart. Read that guide before a second repair attempt.
|
||||
|
||||
Choose the installation owner before repairing registration. Plugin installs use
|
||||
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin);
|
||||
only setup-owned registrations use the guide's
|
||||
[registration repair plan](references/connection-and-recovery.md#repair-registration-when-needed).
|
||||
A plugin-only doctor result of `missing` does not call for `rea setup`.
|
||||
|
||||
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
|
||||
JADX, Kuna, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
|
||||
software to repair MCP registration. For an unsupported client, use manual stdio
|
||||
registration with an existing installation’s absolute runtime and entry script, or continue through the CLI.
|
||||
|
||||
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
|
||||
```
|
||||
|
||||
No MCP registration or native engine is required. Read the returned Evidence,
|
||||
graph, limitations, and unknowns with the same care as an MCP result; the CLI
|
||||
returns the Evidence record directly. This fallback does not establish that
|
||||
MCP is configured. Native CLI tasks still require their selected engine.
|
||||
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
|
||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place from a
|
||||
trusted directory outside the target project, with absolute target paths. Managed
|
||||
setup binds the current installation directly; retain its runtime and package
|
||||
cache entry, or restore that installation before repairing its registration.
|
||||
For setup-owned registration, configuration repair starts with a read-only setup plan; show paths, backups and
|
||||
skill replacement before applying authorized changes. Preserve existing approval
|
||||
and target-execution boundaries.
|
||||
|
||||
## Route the target first
|
||||
|
||||
Choose the first tool from the target the user supplied. Use `open_binary` for
|
||||
active-target native or archive workflows; target-free tools take their own
|
||||
explicit path or endpoint and do not need it.
|
||||
|
||||
- ASAR or extracted JavaScript/Electron tree:
|
||||
`analyze_javascript_application`.
|
||||
- Archive/package member inventory (ZIP/APK/IPA/MSIX/AppX, DMG, or a PyInstaller
|
||||
executable): `open_binary` with the supplied local path; use `inspect_artifact`
|
||||
when its graph and findings help answer the question. A PyInstaller CArchive
|
||||
is the application layer; the bootloader is only the native prefix. Start with
|
||||
`detail: "summary"` for large inventories, then use retained inventory views
|
||||
for selected member fields/pages. A complete small inventory can cost less
|
||||
than extra calls and schemas. Runtime options are metadata, not files.
|
||||
Compact delivery still inventories and hashes members. See the
|
||||
[Python packaged applications guide](https://github.com/morluto/rea/blob/main/docs/python-packaged-applications.md).
|
||||
- Bun `--compile` executable: `open_binary`, then `inspect_artifact`; the Bun
|
||||
module graph is the application layer. Extract stored modules and analyze the
|
||||
JavaScript. See [Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
- Deno `deno compile` executable: `open_binary`, then `inspect_artifact`; the
|
||||
embedded virtual filesystem is the application layer. Extract stored files and
|
||||
analyze compatible JavaScript. Environment-file values and CA certificate bytes
|
||||
are retained in local Evidence with their counts and byte lengths. The npm
|
||||
snapshot length is reported and not decoded, and the executable is not run. See
|
||||
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
|
||||
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
|
||||
`open_bundle_resources`, then the selected resource tool. This does not
|
||||
require `CFBundleExecutable` or a deep provider. Resource sessions have no
|
||||
executable digest or binary snapshot; use `open_binary` for native code.
|
||||
- Android APK code, classes, methods, or incoming references:
|
||||
`inspect_android_package`, then focused Android tools when advertised.
|
||||
Archive member inventory still uses the archive route above.
|
||||
- Managed PE/CLI assembly: `inspect_managed_artifact`.
|
||||
- Firmware image: `inspect_firmware_regions` when advertised. Use
|
||||
`extract_firmware` when extraction is requested, with a caller-selected new
|
||||
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
|
||||
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md).
|
||||
- ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact`
|
||||
without a provider; see the ESP section of `references/native-and-artifacts.md`.
|
||||
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
|
||||
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
|
||||
optional metadata recovery and its supported host/layout boundary.
|
||||
- Native program behavior, live calls, instrumentation, or command-run effects:
|
||||
choose the matching [native runtime workflow](references/runtime-observation.md#native-runtime-questions).
|
||||
`open_binary` and `trace_native_values` establish static facts; they do not
|
||||
observe execution. Runtime tools have distinct launch, attach, mutation and
|
||||
cleanup effects, and do not require a preliminary static session unless their
|
||||
own contract says so.
|
||||
- User-owned browser page already open: `list_browser_targets`.
|
||||
- Passive traffic capture of a SPA: the default scope is the page's own origin,
|
||||
and data APIs usually live on another origin. Discover API origins from
|
||||
absolute URLs in loaded bundles or script text via JavaScript analysis tools
|
||||
(another passive capture without those origins still scopes to the page
|
||||
origin, and `completeness.excluded` counts lack URLs). Pass both the page
|
||||
origin and those API origins in `allowed_origins` — the list replaces the
|
||||
default scope — and read `completeness.excluded` before reporting that an
|
||||
API is absent. For long captures with body shapes, use
|
||||
`export_evidence_bundle` instead of expecting an inline result.
|
||||
- Browser action the agent must perform (click, type, navigate):
|
||||
`capture_browser_scenario` with declared actions. See
|
||||
[runtime observation](references/runtime-observation.md) for connect/launch
|
||||
and the focused follow-up browser tools.
|
||||
- Agent environment (AgentEnv bundle) of a web application: read
|
||||
[environments](references/environments.md) before recording; it records with
|
||||
`capture_browser_scenario` and writes the bundle with `export_environment_bundle`.
|
||||
- Task for a desktop application's exported environment, from a
|
||||
`capture-desktop-application` folder: read
|
||||
[desktop application tasks](references/environments.md#tasks-for-a-desktop-application-environment).
|
||||
- Environment of a Linux desktop app with a task, from a request such as
|
||||
"build me an Inkscape environment where the agent changes a poster's
|
||||
headline": read [desktop builds](references/desktop-build.md). You drive the
|
||||
app on a build computer from `start_build_computer`, then export and witness.
|
||||
- Fiddler Everywhere traffic or a native `.saz` capture: read the
|
||||
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
|
||||
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
|
||||
connect Everywhere's separate MCP server for live traffic.
|
||||
- Retained HAR/SAZ/native mitmproxy/aioquic qlog capture: `inspect_web_network_capture`. Select
|
||||
original record ordinals when useful; inspect producer fields and byte/number
|
||||
sidecars without fetching recorded URLs or claiming live attribution. HAR/SAZ/qlog
|
||||
old-generation heap is `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` (default 192 MiB,
|
||||
floor 8, maximum 16384), set where REA starts. Host memory and `NODE_OPTIONS` do not
|
||||
raise it. See
|
||||
[historical captures](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md) for the exact upstream
|
||||
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
|
||||
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
|
||||
or establish network loss. Prefer detail summary, then kind qlog trace/stream
|
||||
pages; select one trace, stream ID and direction for contribution/range pages.
|
||||
Facts retain all-recorded-events-before-selection scope. Other qlog layouts
|
||||
and PCAP correlation are unsupported.
|
||||
- User-owned Electron runtime already open: `list_electron_targets`. Before an electron-main Inspector attempt, check packaged fuses with `inspect_electron_fuses`; disabled `EnableNodeCliInspectArguments` means the app ignores `--inspect`.
|
||||
- Packet-field discovery: `inspect_packet_fields` searches installed protocols
|
||||
and field names/types before choosing fields for `inspect_packet_capture`.
|
||||
Query/page the relevant records; registrations do not establish observed traffic.
|
||||
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and
|
||||
TShark 4.x on an absolute `PATH` entry or via the optional absolute
|
||||
`REA_TSHARK_COMMAND` override; see
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/guides/data/packet-captures.md).
|
||||
Packet limits count input packets before filtering. Output defaults to 50 matching
|
||||
frames; use `match_offset` / `match_limit` for later pages, or explicit null
|
||||
limit for complete matching output. Preserve the source digest across pages.
|
||||
This does not perform live capture.
|
||||
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
|
||||
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
|
||||
and capinfos, with Linux prlimit or Windows Job Objects; see
|
||||
[packet summaries](https://github.com/morluto/rea/blob/main/docs/guides/data/pcap-analysis.md).
|
||||
Select an exact DNS question for address pages; page exhaustion does not prove
|
||||
network completeness. This reads recordings and does not acquire traffic.
|
||||
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
|
||||
or selected decoder help, including channels and options. Query and page the
|
||||
relevant details before choosing settings; this does not scan hardware.
|
||||
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
|
||||
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
|
||||
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/guides/signals/signal-recordings.md).
|
||||
Output defaults to 50 literal lines; select annotation classes and page with
|
||||
`line_offset` / `line_limit`, or explicit null for all remaining output.
|
||||
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
|
||||
This reads existing files; it does not acquire hardware signals.
|
||||
- Explicit local WebAssembly artifact: `inspect_wasm_artifact`. Supply WABT 1.0.42
|
||||
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
|
||||
Select JavaScript glue and candidate paths explicitly and preserve URL/path
|
||||
ambiguity. No implicit fetch or target execution; WAT is not original source.
|
||||
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/guides/wasm/wasm-artifacts.md).
|
||||
- Explicit local Android APK (or ZIP/AAB) signing: `inspect_apk_signature`.
|
||||
Prefer its summary or selected signer/digest/lineage page; retain complete
|
||||
Evidence and use `inspect_analysis_view` kind `apk-signature` for follow-ups.
|
||||
Reports which schemes are present (v1 JAR, v2, v3, v3.1), each signer's
|
||||
certificate SHA-256 fingerprint (v1 included, from its PKCS#7 block), v3 key-rotation lineage, source stamp, and
|
||||
whether each signed digest still matches the bytes (self-consistency). This
|
||||
is identity and integrity, not a trust or authenticity verdict.
|
||||
See the [APK signature guide](https://github.com/morluto/rea/blob/main/docs/apk-signature.md).
|
||||
- Explicit local EVM bytecode carrier: `inspect_evm_interface` with caller-selected
|
||||
`raw` or `hex` encoding. The owned worker deadline is `timeout_ms` (default
|
||||
30000 ms, range 1..2147483647, next call). Select byte, address-space and CPU
|
||||
allowances separately through `producer_limits`; tighter inherited OS limits
|
||||
remain enforced, and the 256 MiB worker heap is independent. Preserve
|
||||
carrier/decoded digests and treat selectors,
|
||||
argument strings and mutability as inferred candidates; this performs no chain
|
||||
lookup or target execution. For the complete inferred interface without hex or
|
||||
duplicate stdout, discover support and select `detail: "summary"`; complete
|
||||
Evidence stays retained under its original identity. Use the returned
|
||||
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
|
||||
Candidate coverage does not establish interface completeness. See the
|
||||
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/guides/evm/evm-bytecode.md).
|
||||
- Embedded Go compiler, module, build-setting metadata, or static `(ptr, len)`
|
||||
string literals: `inspect_go_binary` with the explicit local file (pass
|
||||
`include_string_literals: true` and optional `max_string_literals` for bounded
|
||||
inferred static string-header recovery, and page retained rows with
|
||||
`inspect_analysis_view` kind `go` collection `string_literals`). This built-in
|
||||
target-free inspection requires no Go installation or disassembler. Preserve
|
||||
raw module bytes and source ranges; absent metadata does not prove a binary is
|
||||
not Go, static string recovery excludes compiler-inlined stack or immediate
|
||||
strings, and unparsed text records remain explicit. See the
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/go-binaries.md).
|
||||
- Embedded CUDA/HIP code objects: `inspect_gpu_code_objects`; inventory targets
|
||||
first, then decode selected payloads and query retained GPU Evidence.
|
||||
- Bundled GGUF or safetensors model declarations: `inspect_model_artifact`
|
||||
with the explicit local file. Default header hashing leaves full-file identity
|
||||
unknown; `hash: "full"` additionally reads all weights for SHA-256. MCP returns
|
||||
retained summaries; optional `view` selects one page in the initial call. Use
|
||||
`inspect_analysis_view` kind `model-artifact` for
|
||||
batched metadata keys, tensor names or type totals, selected fields and pages.
|
||||
Preserve header-scoped indices and array preview omissions; do not infer model
|
||||
behavior from declared architecture or tensor types. See the
|
||||
[model file guide](https://github.com/morluto/rea/blob/main/docs/model-artifacts.md).
|
||||
- Unity IL2CPP names, member owners and string literals from `global-metadata.dat`:
|
||||
`inspect_il2cpp_metadata` with the explicit local file; MCP defaults to compact
|
||||
counts. Query sections, images, types, methods, fields, properties, events,
|
||||
parameters or string_literals through `inspect_analysis_view`
|
||||
with the returned `parent_evidence_id`, using exact filters, batch indices and
|
||||
selected fields. Parameters accept `method_indices`, avoiding a scan of all
|
||||
parameter pages. Missing collections in historical Evidence remain unknown.
|
||||
Versions 24, 27, 29, 31 and 39; no Unity, IL2CPP
|
||||
binary or disassembler is run. Unrecognized magic (possibly encrypted) and
|
||||
other versions are unsupported. Optional matching `binary_path` enables static native addresses and type references; query `method_addresses` with indices or addresses/states, preserving every shared-body method and both artifact identities. Without a linked binary native counts remain unknown. Byte-pattern fallback is inferred; no runtime behavior or same-build pairing is proved. See the
|
||||
[IL2CPP metadata guide](https://github.com/morluto/rea/blob/main/docs/il2cpp-metadata.md).
|
||||
- Embedded cargo-auditable Rust dependency metadata: `inspect_rust_binary`
|
||||
with the explicit local ELF, PE, or thin Mach-O file. This built-in
|
||||
target-free inspection requires no Rust installation or disassembler. Preserve
|
||||
package order, dependency indices, and the section file range. A missing
|
||||
`.dep-v0` section is null metadata, not proof the binary has no dependencies.
|
||||
A rustc commit hash is an inference from a read-only `/rustc/<40-hex>/` path.
|
||||
For selected crates, use `view: {collection: "packages", names: [...],
|
||||
fields: ["name", "version", "source"]}` on the first call; explicitly include
|
||||
`dependencies` for direct index references. Use `detail: "summary"` for counts
|
||||
and ranges, or retained `inspect_analysis_view` Rust package/edge pages for
|
||||
subsequent filters. Keep the complete parent Evidence accessible; CLI compact
|
||||
delivery requires `--evidence_output`.
|
||||
See the
|
||||
[Rust binary guide](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
|
||||
- PE Authenticode certificate table: `inspect_pe_certificates` with the explicit
|
||||
local PE32/PE32+ file. This built-in target-free inspection requires no
|
||||
Windows SDK. Preserve WIN_CERTIFICATE ranges, types, payload SHA-256, and
|
||||
PKCS#7 signer/X.509 identity fields, including `subject_key_identifier_hex`
|
||||
when the signer uses that form. A missing security directory is observed
|
||||
absence of that table, not proof the image is unsigned in every catalog sense.
|
||||
Signatures are not verified. See the
|
||||
[PE certificate guide](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
- PE import and delay-load tables: `inspect_pe_imports` with the explicit
|
||||
local PE32/PE32+ file. This built-in target-free inspection requires no
|
||||
Windows SDK. Preserve DLL names, named and ordinal functions, ILT/IAT RVAs,
|
||||
delay-load descriptors, and the pefile import hash of the standard import
|
||||
table. A missing import directory is observed absence of that table, not
|
||||
proof the image never imported APIs through another table. Bound snapshots
|
||||
and loaded IAT contents remain unobserved. See the
|
||||
[PE import guide](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
|
||||
- Windows Shell Link shortcuts: `inspect_windows_shortcut` with the explicit
|
||||
local `.lnk` file. This built-in target-free inspection requires no Windows
|
||||
SDK or COM. Preserve header flags and FILETIMEs, IDList items, LinkInfo
|
||||
volume and network names, StringData arguments and icon location, ExtraData
|
||||
signatures, and Tracker droid facts labelled derived. Unknown IDList items
|
||||
and ExtraData signatures stay raw with SHA-256. The target path is not
|
||||
resolved or executed. See the
|
||||
[Windows shortcut guide](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
|
||||
- PE TLS callbacks and load configuration: `inspect_pe_load_config` with the
|
||||
explicit local PE32/PE32+ file. This built-in target-free inspection requires
|
||||
no Windows SDK. Preserve callback VAs as strings with their RVA, file offset
|
||||
and section, and the size-covered security cookie, SafeSEH, CFG,
|
||||
EH-continuation and XFG fields. Guard-flag mitigation claims are inferred
|
||||
linker declarations, not proof of enforcement. A missing directory is observed
|
||||
absence of that declaration. See the
|
||||
[PE load configuration guide](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
|
||||
- ELF, PE, and Mach-O hardening: `inspect_binary_hardening` with the explicit
|
||||
local file. This built-in target-free inspection requires no pwntools or other
|
||||
host tool. One image per Mach-O slice. Each property is `enabled`, `disabled`,
|
||||
`partial`, `unknown`, or `not_applicable`, with the header field or import
|
||||
behind it. A missing stack-protector or fortify import is `unknown`, not
|
||||
proof the protection was omitted. Authenticode and Mach-O code signatures
|
||||
report presence, not validity. Hardened runtime stays unknown and names
|
||||
`inspect_signature`. The image is not executed. See the
|
||||
[binary hardening guide](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
|
||||
- ELF note tables: `inspect_elf_notes` with the explicit local ELF32/ELF64
|
||||
file. This built-in target-free inspection requires no disassembler. Preserve
|
||||
PT_NOTE and SHT_NOTE ranges, owners, types, descriptor SHA-256, and GNU
|
||||
build-id or ABI-tag fields. A missing table is observed absence of those
|
||||
declarations, not proof the image never carried notes. See the
|
||||
[ELF notes guide](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
- Code that runs before `main` or at exit in ELF or Mach-O:
|
||||
`inspect_native_initializers` with the explicit local file (and
|
||||
`architecture` for a universal Mach-O). This built-in inspection requires no
|
||||
disassembler. Preserve slot and target addresses, the `resolution` of each
|
||||
entry, and its limitations. An `unresolved` slot is not a zero pointer, and a
|
||||
missing table is not proof that no constructor exists. See the
|
||||
[initializer guide](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
|
||||
|
||||
- ELF unwind function ranges: `inspect_elf_unwind_ranges` with the explicit
|
||||
local ELF32/ELF64 file. This built-in target-free inspection requires no
|
||||
disassembler. Preserve FDE start, end, size, containing section, exact
|
||||
symbol name, CIE index, LSDA, and personality. Page with `offset` and
|
||||
`limit`. A missing `.eh_frame` is observed absence of that declaration,
|
||||
not proof the image has no functions. `.debug_frame` is reported present
|
||||
or absent and is not decoded. CFA instructions are counted and not
|
||||
evaluated. See the
|
||||
[ELF unwind ranges guide](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
|
||||
- Apple `.ips` crash reports: `inspect_apple_ips` with the explicit local
|
||||
file. This built-in target-free inspection requires no Xcode, atos, or
|
||||
pwntools. Preserve first-line metadata, `bug_type` as a string, process and
|
||||
exception facts, lossless integer registers, usedImages UUIDs, and
|
||||
documented classification matches. Only `bug_type` `"309"` yields a crash
|
||||
body; other types, including `"288"` stackshots, return no crash result.
|
||||
Select an optional `join_path` for UUID-and-architecture matching to one
|
||||
local Mach-O or dSYM; preserve slice identity and unjoined reasons. Joined
|
||||
static addresses do not resolve function names or dSYM symbols. See the
|
||||
[Apple IPS guide](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
|
||||
- Linux kernel module metadata: `inspect_kernel_module` with the explicit local
|
||||
ELF32/ELF64 relocatable file. This built-in target-free inspection requires
|
||||
no Linux host or `modinfo`. Preserve `.modinfo` key order and duplicate keys,
|
||||
`__versions` CRC hex, section file ranges, and the appended signature
|
||||
trailer's PKCS#7 SHA-256. A missing section is observed absence when section
|
||||
names are known, not proof the module never carried that metadata.
|
||||
`struct module` is not decoded. The signature is not validated. gzip, xz, and
|
||||
zstd containers are unsupported; decompress into a separate file with an
|
||||
external codec utility, then inspect the decompressed `ET_REL` image. See the
|
||||
[kernel module guide](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
|
||||
- Linux kernel version and build config (`CONFIG_*` options such as
|
||||
`CONFIG_SECURITY_SELINUX`): `inspect_kernel_build_config` with the explicit
|
||||
local `vmlinux`, raw `Image`, x86 `bzImage`, or `configs.ko`. This built-in
|
||||
target-free inspection requires no Linux host or `extract-ikconfig`. Preserve
|
||||
the banner version, config SHA-256, option count, and value kinds; use `name`
|
||||
for one option. An `IKCFG_ST` string is a config only with a valid gzip member
|
||||
and `IKCFG_ED`. An absent config may mean the kernel was built without
|
||||
`CONFIG_IKCONFIG`. Non-gzip payloads are unsupported; decompress externally,
|
||||
then inspect the result. See the
|
||||
[kernel build config guide](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
|
||||
- CPython 3.7–3.13 `.pyc` header, nested marshal code objects and bytecode
|
||||
identity: `inspect_python_pyc` with the explicit local file. This built-in
|
||||
target-free inspection requires no Python installation. Preserve magic,
|
||||
PEP 552 hash or mtime/size, nested names, and bytecode SHA-256. Bytecode is
|
||||
not decompiled, and a decoded header does not prove the matching `.py` is
|
||||
present. See the
|
||||
[CPython pyc guide](https://github.com/morluto/rea/blob/main/docs/python-pyc.md).
|
||||
- Erlang BEAM IFF container: `inspect_erlang_beam` with the explicit local
|
||||
`.beam` file. This built-in target-free inspection requires no Erlang
|
||||
installation. Preserve chunk names, atom order, export/import/lambda tables,
|
||||
and the Code header. For selected facts, prefer `detail: "summary"` plus a
|
||||
`beam` view (collection, names/indices, fields and paging), retaining the full
|
||||
Evidence. CLI summary requires `--evidence_output`. Missing table counts stay
|
||||
null; decoded empty tables count zero. Bytecode is not executed. See the
|
||||
[Erlang BEAM guide](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
|
||||
|
||||
- Explicit local SQLite database snapshot: `inspect_sqlite_database`. Inspect
|
||||
with Node.js 24.x >=24.15 or 26+ providing native SQLite limits. Read schema
|
||||
first (compact inventory by default), or select an exact ordinary table
|
||||
and `columns` when rows are needed. Defaults are selected-table schema and
|
||||
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
|
||||
report truncation and have unspecified order. Use a quiescent snapshot rather
|
||||
than assuming a live database/WAL pair was copied atomically. See the
|
||||
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
|
||||
- Explicit Linux ELF file for offline layout, symbols, relocations or static
|
||||
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
|
||||
caller-supplied pwntools without opening a disassembler database. Preserve its
|
||||
raw locations and inference/coverage limits; see the
|
||||
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
- Supplied Linux x86-64 ELF core: `inspect_recorded_crash`. Read every recorded
|
||||
thread, raw note and source-bound register without opening a live target.
|
||||
Historical PIDs do not select live processes. Request `include_debugger_context`
|
||||
only when core-only mapping candidates help; see the
|
||||
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md).
|
||||
- PE resource type/name/language trees and static icon group references:
|
||||
`inspect_pe_resources` with an explicit path. This portable operation requires
|
||||
no active target or engine; opaque payloads retain file ranges and SHA-256.
|
||||
- PE export directories and ELF dynamic-symbol exports: `inspect_native_exports`
|
||||
with an explicit path; portable, no active target or engine.
|
||||
- Native executable, library, or analysis database: `open_binary`, then
|
||||
use focused analysis tools directly; call `binary_overview` when metadata or
|
||||
inventory context is useful and available from the selected provider.
|
||||
|
||||
For an existing IDA MCP configuration, use `open_binary` with the original
|
||||
input binary and `provider_id: "ida"`; do not pass an `.idb` or `.i64` database.
|
||||
The legacy attached profile binds the already-open GUI input by SHA-256 and
|
||||
leaves its database open. The modern headless profile creates and closes an
|
||||
owned private database without saving. Installation and registration reuse
|
||||
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
|
||||
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md).
|
||||
For migration to the separate native IDA MCP plugin, follow the guide's
|
||||
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md#migrate-a-legacy-attached-registration).
|
||||
Replace the upstream registration explicitly; modern mrexodia tools are not the
|
||||
native profile. Native attachment preserves the database but does not provide
|
||||
direct callers; check session availability and the guide's verification limits.
|
||||
Proceed directly to `analyze_function`, pseudocode, or function/string searches
|
||||
after opening; IDA does not supply `binary_overview`. Consult session availability
|
||||
when composing broader workflows. Modern direct callers and unsupported dossier
|
||||
facets remain unknown; live IDA results are not replayed from snapshots.
|
||||
|
||||
With `REA_KUNA_COMMAND` configured, `provider_id: "kuna"` answers procedure
|
||||
lists, lookups, pseudocode, assembly and `analyze_function` from Kuna's cached
|
||||
whole-program export. Callers/callees use a separate body-free graph shared by
|
||||
the session; `xrefs` returns exact-address incoming sites and filters forwarding
|
||||
aliases. These queries and inventory lookups do not trigger a C export. Indirect
|
||||
targets remain unknown and can make relationship coverage partial. Reference
|
||||
caches expire on close; a fresh CLI process repeats the native reference walk.
|
||||
Typed references, CFG, raw-memory tools and annotations remain unsupported. A cold first
|
||||
answer can come from a single-function run (its limitations say so); assembly
|
||||
waits for the export. For pseudocode and assembly, `raw_result.source` names the
|
||||
cached `.c`/`.asm` file and byte span, which you may search directly instead of
|
||||
calling per function. See the
|
||||
[Kuna provider guide](https://github.com/morluto/rea/blob/main/docs/kuna-provider.md).
|
||||
|
||||
If the user says only "my app" without a name, path, or target already selected
|
||||
for this task, ask which app before searching for artifacts. Paths in unrelated
|
||||
saved reports and nearby test fixtures do not select the user's target. Resolve
|
||||
an actual human-readable app name to one clear installed artifact when possible;
|
||||
ask when matches are ambiguous. Never choose an example app on the user's behalf.
|
||||
|
||||
In a target-free session, use `open_binary` to bind any archive/package or
|
||||
native target whose analysis tool operates on the active target. Do not call an
|
||||
unadvertised tool; inspect `binary_session` with `{}` and its
|
||||
`result.tool_availability` for availability reasons and remediation. Expand each
|
||||
group's `names` into entries, inheriting the inventory-level client requirements
|
||||
unless the group overrides them. The grouped name union is the complete tool
|
||||
inventory; use `tools/list` for canonical tool schemas. The default summary keeps
|
||||
each provider candidate's availability and target-support status, code and
|
||||
reason; `omitted_fields` lists what it leaves out. Request `{"detail": "complete"}`
|
||||
only when you need an unavailable candidate's remediation, provider
|
||||
diagnostics, candidate capability sets, or tool surface, effect and annotation
|
||||
flags; in those closed flag vocabularies, listed
|
||||
flags are true and omitted known flags are false. Provider candidate facts
|
||||
describe possible providers, while top-level session capabilities describe the
|
||||
current route and `analysis_provider_binding` names the selected provider. Do
|
||||
not treat those scopes as interchangeable.
|
||||
|
||||
For editable Ghidra analysis that must survive a later agent process, select
|
||||
the [owned Program checkpoint workflow](references/native-and-artifacts.md#continue-editable-ghidra-analysis-across-processes)
|
||||
before making metadata edits. It is separate from Evidence/Unknown persistence.
|
||||
|
||||
For a Ghidra startup timeout on a large native target, follow
|
||||
[deadline recovery](references/native-and-artifacts.md#ghidra-startup-deadline-recovery)
|
||||
before abandoning deep analysis or repeating the same cold import. The default
|
||||
330 seconds can be changed for new startup attempts through
|
||||
`update_analysis_settings` using `ghidra.startup_timeout_ms`. Changing the
|
||||
server's launch environment instead requires a new REA process.
|
||||
|
||||
## Work summary-first
|
||||
|
||||
Start with the default result and use its inline Evidence and graph context.
|
||||
For a JavaScript application or ELF layout whose size is unknown or large,
|
||||
call `analyze_javascript_application` or `inspect_binary_layout` with
|
||||
`"detail": "summary"`, then inspect the retained `parent_evidence_id` with
|
||||
`inspect_analysis_view`. Over MCP, `analyze_function` is summary-first without
|
||||
a flag: its default reply is the summary, and the complete dossier stays
|
||||
retained under `parent_evidence_id` for facet paging. Pass that ID as a
|
||||
`retained-evidence` reference to `compare_functions`, or inside
|
||||
`match_native_functions` `left_observations`/`right_observations` or
|
||||
`build_call_path.functions`; request `"detail": "complete"` only when the
|
||||
dossier must travel inline.
|
||||
Use the same exact-reference form in other advertised logical Evidence inputs,
|
||||
including optional sources, arrays and nested sources across domains. Each
|
||||
source can independently be inline or retained; preserve its source identity,
|
||||
coverage, partial facts and unknowns. A summary or inspection page has its own
|
||||
Evidence ID: use `normalized_result.parent_evidence_id` when the consumer needs
|
||||
the complete source. For portable CLI follow-ups, commands advertising
|
||||
`--evidence-bundle` resolve references from that selected exported bundle.
|
||||
See [Evidence composition](references/evidence-workflows.md) for scope and
|
||||
recovery; file selectors are supported only where explicitly advertised.
|
||||
Page module identities and select the returned `node_id` before requesting an
|
||||
item; a single file path can represent several graph roles. Do not repeat an
|
||||
identical analysis or rejected call without a changed input or resolved blocker.
|
||||
Recheck session state when a lifecycle change makes that status useful.
|
||||
Make a focused follow-up only when the
|
||||
returned result leaves a specific question unanswered.
|
||||
|
||||
If MCP reports `resource_constraint` with `details.resource: "transport"`, call
|
||||
`inspect_analysis_view` with its reported same-session Evidence reference for a
|
||||
summary, one section/module, or a stable page. Use `trace_application_feature`
|
||||
once a seed is known, or call `export_evidence_bundle` to write the complete
|
||||
session to a selected path. The analysis remains complete in the ledger; avoid
|
||||
repeating it merely to request the same oversized response. Complete CLI JSON
|
||||
output also streams without a single MCP frame.
|
||||
|
||||
When the retained parent ID is unknown, use `get_evidence_bundle` with
|
||||
`detail: "summary"` and exact filters for operation, target SHA-256,
|
||||
analysis-profile digest or canonical procedure address. Select its returned
|
||||
Evidence ID for the next view instead of requesting the complete bundle. When
|
||||
that record has no supported view (for example an annotation), read it with
|
||||
`detail: "records"` and an `evidence_id` filter; `missing_evidence_links`
|
||||
names linked sources this session does not retain.
|
||||
`retention: "complete-record"` means the whole record is stored; native
|
||||
value-flow truncation and unknown metadata remain separate. For an exported
|
||||
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
|
||||
`detail: "summary"` and the same `filters`. Reading that file still requires
|
||||
validating its full contents within the local memory budget.
|
||||
|
||||
A client can also truncate a successful tool result without a REA transport
|
||||
error. Treat that preview as incomplete; use the retained parent ID for smaller
|
||||
views or export when the full record is needed. Do not parse the preview as
|
||||
canonical JSON or repeat the producer. For JavaScript, read the
|
||||
[module-view and trace recipes](references/javascript-applications.md) before
|
||||
drilling into a large graph; a broad bidirectional trace can expand unrelated
|
||||
containment relationships even after summary analysis.
|
||||
|
||||
Every conclusion must distinguish observations, inferences, and unknowns. Cite
|
||||
Evidence IDs, preserve limitations and incomplete coverage, and never imply
|
||||
that static analysis observed execution. Runtime requests execute the declared
|
||||
target and lifecycle; do not broaden the target or action beyond those fields.
|
||||
|
||||
Within the user's requested investigation, call available analysis tools
|
||||
directly. REA does not require permission grants or per-call approval flags.
|
||||
The agent client can independently require approval. Follow the declared request
|
||||
scope and the host's actual access requirements; use the recovery guide for an
|
||||
observed client-policy rejection.
|
||||
|
||||
## Plan broader investigations
|
||||
|
||||
For requests that span multiple features or subsystems, use a staged workflow:
|
||||
|
||||
1. Turn the request into a checklist of questions and the evidence each answer
|
||||
needs. Resolve target identity and constraints from the conversation and
|
||||
workspace before asking for information again.
|
||||
2. Inspect the current REA session, artifact identity, saved analysis database,
|
||||
bookmarks, and prior evidence. Reuse matching state; do not open duplicate
|
||||
sessions or repeat identical analysis.
|
||||
3. Start with the smallest useful overview or inventory. Follow each question
|
||||
from its entry point through relevant data and state changes to its result.
|
||||
Batch related operations around a specific hypothesis, then expand only when
|
||||
the returned evidence leaves a concrete gap.
|
||||
4. Inspect relevant packaged resources and configuration alongside code when
|
||||
they affect the question. Use format-aware inventory and parsers; do not
|
||||
infer behavior from filenames, strings, or layout alone.
|
||||
5. Corroborate a conclusion with the evidence type it requires. Use runtime
|
||||
observation only when static evidence cannot answer the question and the
|
||||
required host runtime and OS access are available. For JavaScript behavior,
|
||||
run probes against the actual app through the available browser, Electron,
|
||||
or process capture workflow.
|
||||
6. Decompose work into independent questions. When parallel workers are
|
||||
available and the questions do not depend on one another, assign distinct
|
||||
scopes, point workers to existing evidence, and ask them to return sources,
|
||||
conclusions, and unresolved gaps. Otherwise, work sequentially.
|
||||
7. Keep a concise finding ledger linking each conclusion to Evidence IDs,
|
||||
confidence/evidence type, search boundary, and remaining unknowns. Update
|
||||
the shared index or investigation report so later passes can reuse results.
|
||||
|
||||
Before finishing, revisit the original checklist. Mark each question as
|
||||
answered, partially answered, or unresolved based on its evidence; keep
|
||||
bounded negative searches bounded, and do not describe a broad investigation
|
||||
as complete while required questions remain open.
|
||||
|
||||
## Read only the relevant guide
|
||||
|
||||
- Native binaries, managed assemblies, archives, and extraction:
|
||||
[references/native-and-artifacts.md](references/native-and-artifacts.md)
|
||||
- ASARs, extracted JavaScript, feature tracing, and version comparison:
|
||||
[references/javascript-applications.md](references/javascript-applications.md)
|
||||
- Android APK declarations, classes, methods, and static references:
|
||||
[references/android-applications.md](references/android-applications.md)
|
||||
- Native instrumentation, debugger and process capture; browser/Electron
|
||||
observation; static/runtime comparison:
|
||||
[references/runtime-observation.md](references/runtime-observation.md)
|
||||
- Evidence paging, comparisons, residual unknowns, and verification:
|
||||
[references/evidence-workflows.md](references/evidence-workflows.md)
|
||||
- Agent environments (AgentEnv bundles) of a web application, recorded from
|
||||
a live instance or a HAR:
|
||||
[references/environments.md](references/environments.md)
|
||||
- Tasks for a desktop application's exported environment:
|
||||
[references/environments.md](references/environments.md#tasks-for-a-desktop-application-environment)
|
||||
- A desktop app environment from a request, built and witnessed on build
|
||||
computers: [references/desktop-build.md](references/desktop-build.md)
|
||||
|
||||
## Finish the task
|
||||
|
||||
Explain findings in plain language and tie them to returned evidence. When the
|
||||
user asks to build something, use normal coding tools and separate observed
|
||||
behavior from design choices. Close an opened native session with
|
||||
`close_binary` when its engine is no longer needed. Evidence references and Unknown
|
||||
history remain in memory on the same MCP connection. End the investigation with
|
||||
`reset_investigation`; supply `save_path` to checkpoint the complete bundle before
|
||||
clearing. Import the saved bundle to restore original references and Unknown
|
||||
history in a new process. Retention is bounded and is not automatic persistence.
|
||||
@@ -0,0 +1,63 @@
|
||||
# Android application artifacts
|
||||
|
||||
Use the connected server's advertised Android tools. Repository main and npm
|
||||
4.1.0 include this family. A skill installation alone cannot add it to an
|
||||
older server.
|
||||
|
||||
Start with `inspect_android_package` on the caller's APK path for package
|
||||
identity, manifest declarations, coverage, and inline Evidence. Use
|
||||
`search_android_classes` to find candidates, `inspect_android_class` for one
|
||||
class's members, `inspect_android_method` for decompilation, and
|
||||
`trace_android_references` for incoming static references. Each call takes its
|
||||
own APK path and explicit selectors; do not open a native session first.
|
||||
Choose an `overload_index` from the class inventory when methods are ambiguous
|
||||
instead of guessing.
|
||||
|
||||
For JNI implementations, discover `inspect_android_jni_bindings`. Select raw
|
||||
DEX class/method identities and optional descriptors, ABI or logical library
|
||||
paths. Prefer `detail: "summary"`; complete Evidence remains retained and
|
||||
`android-jni` views page declarations, libraries, candidates or binding rows
|
||||
with exact fields. Global native-pair ambiguity survives class selection.
|
||||
Preserve slot/relocation provenance and source coverage: matched means observed
|
||||
candidate presence; partial scans do not establish uniqueness or absence.
|
||||
This reads DEX/ELF bytes without loading the APK into the decompiler or executing
|
||||
target code. Runtime registration and loading remain unknown. See the
|
||||
[JNI guide](https://github.com/morluto/rea/blob/main/docs/guides/android/android-analysis.md#link-native-declarations-to-jni-candidates).
|
||||
|
||||
For an R8/ProGuard release build with its `mapping.txt`, pass the absolute
|
||||
`mapping_path` to `inspect_android_class` or `inspect_android_method`. Prefer
|
||||
`detail: "summary"` to receive normalized symbols or the selected body without
|
||||
duplicate raw class metadata. Complete Evidence stays retained under the linked
|
||||
parent ID and can be exported; CLI summary delivery requires `--evidence_output`.
|
||||
Omitted or complete detail preserves the full inline response. Keep
|
||||
selecting by observed names; the separate `mapping` block reports each symbol
|
||||
as `mapped`, `ambiguous` or `unmapped` after checking DEX types and prototypes.
|
||||
REA does not verify that the mapping belongs to the APK build.
|
||||
|
||||
The provider requires a separately supplied headless JADX MCP JAR 0.7.1 and
|
||||
Java 17 or newer. `REA_JADX_MCP_JAR` selects the absolute JAR path; `JAVA_HOME`
|
||||
can select Java. Use `rea doctor --provider jadx --json` to check the JADX
|
||||
configuration and obtain remediation for missing prerequisites. Java and the
|
||||
headless JADX MCP JAR remain caller-supplied; REA setup does not install or
|
||||
configure them. Configuration readiness does not establish successful analysis
|
||||
of a particular APK.
|
||||
|
||||
Linux and macOS use the POSIX owned-process boundary. Windows x64 requires the
|
||||
matching bundled native controls and uses an owned Job Object with bidirectional
|
||||
stdio. A full JDK, including `jdk.compiler`, is required on every host; Windows
|
||||
`JAVA_HOME` selects `bin\java.exe`. Consult the Android guide for actual host
|
||||
verification coverage. Calls use one worker and provider deadlines; optional
|
||||
`REA_JADX_HEAP_MIB` selects the JVM heap and `REA_JADX_OPERATION_TIMEOUT_MS`
|
||||
the per-operation deadline (integer milliseconds from 1 to 2147483647, default
|
||||
120000). To change that deadline on a running MCP server, set
|
||||
`jadx.operation_timeout_ms` through `update_analysis_settings`, then retry after
|
||||
work and cleanup settle. Newly accepted operations use the override; accepted
|
||||
work keeps its budget. Changing the launch environment requires a new process.
|
||||
The MCP client's tool timeout must also allow the selected deadline. The APK is never launched,
|
||||
and no emulator is required.
|
||||
|
||||
Manifest entries and decompiled text are static observations, not execution or
|
||||
policy enforcement. Incoming references are recovered static relationships;
|
||||
reflection, dynamic loading, and unresolved calls remain unknown. Preserve
|
||||
artifact digests, locations, diagnostics, and incomplete coverage. For archive
|
||||
members rather than code analysis, use `open_binary` and `inspect_artifact`.
|
||||
+252
@@ -0,0 +1,252 @@
|
||||
# Connection and task recovery
|
||||
|
||||
Read this guide only for an observed connection, command, approval or provider
|
||||
blocker. Keep the user's target and question in view. Explain what prevented the
|
||||
operation and the next useful step; after repair, continue the investigation.
|
||||
|
||||
Commands below use an installed `rea`. If none is available, use the package
|
||||
invocation in `SKILL.md` when acquiring REA is authorized. Follow the connected
|
||||
server's schemas and the installed CLI's help rather than assuming repository
|
||||
main and an older release expose the same arguments. Tool-name prefixes belong
|
||||
to the client: find the canonical REA operation in its actual tool listing and
|
||||
server association before concluding that it is absent. A shared prefix alone
|
||||
does not establish which server an approval wildcard covers. See
|
||||
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/getting-started/client-configuration.md#client-tool-names-and-discovery).
|
||||
|
||||
## Choose the remedy from the observed failure
|
||||
|
||||
| Observation | Meaning and next step |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| No REA tools in the active session | If REA was installed as a client plugin, follow the plugin section below. Otherwise inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| MCP connection dropped or REA could not start | If the connection drops, retry that call once only when it is safe to repeat or known not to have started. If it may already have changed the target, or the same failure returns, tell the user REA's connection failed and ask them to restart the MCP client. If REA could not start its connection, retry once, then tell the user and ask them to restart the MCP client. Do not retry a startup failure that returns the same unchanged result. Run `rea doctor` when REA could not start or tools stay missing after restart. Doctor cannot restore a live connection by itself. |
|
||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. Request `{"detail": "complete"}` only for provider diagnostics. |
|
||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
||||
|
||||
If the client cannot launch its shell/sandbox helper or read the installed skill,
|
||||
inspect its launch diagnostic. For example, Codex can refuse helper aliases when
|
||||
CODEX_HOME is under `/tmp`; use an authorized profile outside the OS temporary
|
||||
directory. REA setup cannot repair the client's executor. Preserve the sandbox
|
||||
instead of silently disabling it to make an analysis command work.
|
||||
|
||||
Static JavaScript and ordinary ZIP inventory need neither Hopper nor Ghidra.
|
||||
HAR inspection is offline and does not require a live browser. Native
|
||||
decompilation, Android analysis and firmware extraction have their own
|
||||
bring-your-own prerequisites. Use supported inventory/static evidence when it
|
||||
answers part of the question, and name the unavailable evidence separately.
|
||||
Do not substitute another target. Label observations from ordinary shell tools
|
||||
as such; only actual REA output supplies REA Evidence.
|
||||
|
||||
## Distinguish client and server deadlines
|
||||
|
||||
A host message such as `timed out awaiting tools/call` with no REA result means
|
||||
the client's deadline expired before a result arrived. It does not establish
|
||||
that REA finished, canceled successfully, or released all resources. Do not
|
||||
repeat the unchanged analysis. Inspect available session/provider state and
|
||||
retained Evidence only when useful; an idle `binary_session` is not proof that
|
||||
every target-free operation stopped. Never invent a missing Evidence reference.
|
||||
|
||||
Allow the host tool deadline to cover execution, cancellation/cleanup, and
|
||||
response delivery. In Codex, the setting is
|
||||
`mcp_servers.<server-id>.tool_timeout_sec` in **seconds**, not
|
||||
`startup_timeout_sec`. Select a value for the actual operation budget and
|
||||
headroom, not a universal timeout. See the
|
||||
[official MCP configuration reference](https://learn.chatgpt.com/docs/extend/mcp?surface=cli).
|
||||
Apply configuration changes only within the authorized scope, preserving
|
||||
unrelated entries and backups; reconnect/restart as required by the host.
|
||||
When configuration is not editable, report the setting needed or use an
|
||||
authorized CLI fallback. Do not silently broaden permissions or switch targets.
|
||||
|
||||
A returned REA `provider_timeout` instead means the **server/provider** reached
|
||||
its deadline. Follow its `remediation.action`, reuse accessible partial Evidence,
|
||||
and resolve reported incomplete cleanup before retrying. Change a request
|
||||
field or a `REA_*` setting only when the returned remediation explicitly
|
||||
recommends that change. Restart REA when that text says the value is read at
|
||||
startup. A mention that a control does not raise the deadline is not a
|
||||
recommendation to change it. Do not invent a setting the remediation does not
|
||||
name. Raising only the client deadline does not change
|
||||
this server budget. For static JavaScript,
|
||||
`analysis_timeout_ms` (CLI `rea analyze-javascript-application <path>
|
||||
--analysis-timeout-ms <milliseconds>`) controls each source's
|
||||
startup, analysis and transfer deadline in **milliseconds**. Increase that
|
||||
budget or reduce semantic work / select exact inventoried `source_paths` when
|
||||
the task permits. Omit `source_paths` for all sources; `[]` is invalid.
|
||||
`detail: "summary"` reduces delivery, not analysis work. A per-source budget
|
||||
does not bound the elapsed time for an entire multi-source application.
|
||||
Partial-Evidence inspection can avoid recomputation; an unavailable reader is
|
||||
not itself a reason to block a corrected static-analysis retry.
|
||||
The shorthand `rea analyze <path>` does not expose this timeout flag; use the
|
||||
explicit JavaScript command when changing that budget.
|
||||
|
||||
For invalid arguments, use the connected tool's current input schema, or the
|
||||
installed CLI's command help. Refresh the catalog if stale; do not guess fields
|
||||
from an older skill or return the entire catalog to repair one field. Correct
|
||||
all reported fields before retrying.
|
||||
|
||||
## Recovery ladder
|
||||
|
||||
Follow the returned remediation. Change a request field or a `REA_*` setting
|
||||
only when that text explicitly recommends the change. Restart REA when it says
|
||||
the value is read at startup. A mention that a control does not raise a limit
|
||||
is not a recommendation to change it. Do not invent a setting. Host memory and
|
||||
`NODE_OPTIONS` do not raise a REA admission unless that remediation says so.
|
||||
|
||||
- MCP connection dropped: retry that call once only when it is safe to repeat
|
||||
or known not to have started. If it may already have changed the target, or
|
||||
it fails again, tell the user REA's connection failed and ask them to restart
|
||||
the MCP client. Do not run `rea doctor` for the drop itself.
|
||||
- REA could not start its connection: retry once, then tell the user and ask
|
||||
them to restart the MCP client. Run `rea doctor` when REA could not start or
|
||||
tools stay missing after that restart. Do not retry a startup failure that
|
||||
returns the same unchanged result.
|
||||
- `cleanup_incomplete`, or retained cleanup owners in the error details: retry
|
||||
that exact cleanup owner. Do not start new work that needs those resources
|
||||
until that owner confirms release. Repeating the original call is not that
|
||||
retry.
|
||||
- Capacity, when the result is `resource_constraint` and `retryable` is true:
|
||||
wait for the active admission to finish, then retry within the reported
|
||||
limits. Do not restart REA to free the slot.
|
||||
- `configuration_invalid`: correct the named setting in the environment that
|
||||
starts REA (the shell or the MCP registration), then restart REA. Run
|
||||
`rea doctor` to see the `runtime-configuration:` check. Re-run `rea setup`
|
||||
for that client when the registration must carry the new setting, then
|
||||
reconnect.
|
||||
- `invalid_request`: correct the reported fields from the current schema. Do
|
||||
not repeat the unchanged call.
|
||||
|
||||
## REA installed as a client plugin
|
||||
|
||||
A Claude Code, Codex, or Cursor Marketplace plugin owns its `rea` server and
|
||||
this skill. Doctor's `--client` and `--skill` checks and `rea setup` read and
|
||||
write only setup-owned state, so for a plugin-only install they report
|
||||
`missing`, and applying setup would add a second `rea` server. If this skill
|
||||
was loaded from a plugin, or the client lists a `rea` plugin, do not prepare a
|
||||
setup plan for missing tools. Instead:
|
||||
|
||||
- Check that the plugin is installed and enabled with the client's own
|
||||
listing: `claude plugin list` and `/mcp` in Claude Code, `codex plugin list`
|
||||
in Codex, and Cursor's plugin settings for a Marketplace install.
|
||||
- Restart or reconnect the client, then read its launch error for the `rea`
|
||||
server if tools are still absent. Claude Code and Codex start the plugin's
|
||||
bundled runtime; a missing build or version mismatch requires a reinstall,
|
||||
or a fresh build when developing from source. A Cursor Marketplace install
|
||||
starts `npx` at the version pinned in the plugin's `mcp.json`. A failed
|
||||
download needs network access to that published version.
|
||||
- Check analysis engines through the connected server's
|
||||
`binary_session.result.tool_availability`. `rea doctor --provider <id> --json`
|
||||
from a shell reports that shell's environment, which can differ from the
|
||||
environment the client gave the plugin server. If you still run it through
|
||||
the package runner, use the version pinned in the plugin manifest's server
|
||||
entry (`rea-agents@<version>`), not `@latest`, so it matches the server the
|
||||
plugin starts.
|
||||
|
||||
Use the setup repair below only when the user wants a setup-owned registration
|
||||
instead of the plugin, and remove or disable the plugin first.
|
||||
|
||||
## Repair registration when needed
|
||||
|
||||
For Codex:
|
||||
|
||||
```bash
|
||||
rea doctor --client codex --json
|
||||
rea setup --client codex --dry-run --json
|
||||
```
|
||||
|
||||
Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
|
||||
`cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`,
|
||||
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `trae`, `grok_build`, `omp`, `pi`,
|
||||
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, `freebuff`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
||||
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
|
||||
For unsupported clients, use manual stdio registration with the selected
|
||||
installation’s absolute runtime and entry script, or continue through the CLI.
|
||||
Acquire packages from a trusted directory outside the target project. Keep a
|
||||
package-runner installation and its runtime available while registrations use it.
|
||||
|
||||
Show the plan's exact paths, backups, changes and bundled-skill replacement.
|
||||
Obtain approval for writes when that concrete scope is not already authorized,
|
||||
then apply the same scope with `rea setup --client codex --yes`. Add
|
||||
`--install-hopper` only when separately needed and explicitly approved. Do not
|
||||
install or upgrade unrelated prerequisites. Restart/reconnect, verify tools
|
||||
actually appear, and resume the original task. If they remain absent, inspect
|
||||
the launch failure rather than looping through setup.
|
||||
|
||||
## Codex tool approval is separate from REA readiness
|
||||
|
||||
REA tools have no permission-grant handshake. Codex can still require approval
|
||||
because an operation creates owned resources, launches a helper or has cleanup
|
||||
effects. An advertised tool or successful doctor result does not override that
|
||||
client policy. `workspace-write` alone does not resolve an MCP approval
|
||||
rejection, and `approval_policy = "never"` does not mean every MCP tool is
|
||||
preapproved.
|
||||
|
||||
When interactive approval is available, use the client's approval flow for the
|
||||
requested operation. In an unattended run, continue through an authorized CLI
|
||||
workflow if the execution environment permits it, or report the blocked
|
||||
operation and the approval needed to proceed. Do not retry the same rejected
|
||||
call, silently broaden approvals, or rewrite MCP effects as read-only.
|
||||
|
||||
For an operator who explicitly wants to preapprove named operations in a
|
||||
disposable Codex profile, current Codex supports per-tool configuration. For
|
||||
example, opening and closing an analysis session can be scoped independently:
|
||||
|
||||
```toml
|
||||
[mcp_servers.rea.tools.open_binary]
|
||||
approval_mode = "approve"
|
||||
|
||||
[mcp_servers.rea.tools.close_binary]
|
||||
approval_mode = "approve"
|
||||
```
|
||||
|
||||
Use the actual server ID and only the operations whose effects were approved;
|
||||
this example does not approve inspection, extraction or runtime execution.
|
||||
Show the proposed config edit and preserve unrelated entries and backups before
|
||||
writing. Managed policy can constrain client settings. Consult the
|
||||
[Codex configuration reference](https://learn.chatgpt.com/docs/config-file/config-reference)
|
||||
for the installed client's supported settings. This is an optional recovery
|
||||
choice, not a prerequisite for every REA task.
|
||||
|
||||
## Repair an editable REA limit
|
||||
|
||||
If `details.limit` names an editable setting and its bounds, call
|
||||
`update_analysis_settings` directly with a `changes` entry containing `setting`,
|
||||
`action: "set"` and `value`. `inspect_analysis_settings` is optional when those
|
||||
facts are already present. Retry the same input after work and cleanup settle.
|
||||
Runtime defaults affect new work only; explicit per-call arguments take
|
||||
precedence and must be corrected directly when selected. Reset restores launch
|
||||
values. Updates survive target close/open but disappear on REA restart. They do
|
||||
not edit host deadlines, response buffers, registrations or fixed limits.
|
||||
|
||||
## Continue through an installed CLI
|
||||
|
||||
Check the command environment only if the launcher fails:
|
||||
|
||||
```bash
|
||||
command -v rea
|
||||
command -v node
|
||||
node --version
|
||||
rea --version
|
||||
```
|
||||
|
||||
Use the public launcher at its known installation path with a supported existing
|
||||
Node on PATH when the child shell loses a prefix installation. Avoid internal
|
||||
`dist/cli.js` entrypoints or global toolchain changes. MCP may have its own valid
|
||||
launcher even when a shell cannot resolve `rea`.
|
||||
|
||||
For a user-supplied JavaScript tree or ASAR, select output files in the authorized
|
||||
workspace and save complete Evidence before reading a view:
|
||||
|
||||
```bash
|
||||
rea analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
|
||||
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
|
||||
rea inspect-analysis-view app-view.json
|
||||
```
|
||||
|
||||
These commands do not register MCP or install an analysis engine. CLI JavaScript
|
||||
JSON is complete and can be very large; redirect it rather than loading it into
|
||||
context. Use an existing JSON processor if `jq` is absent, without installing
|
||||
one just for this recipe. Inspect a module page or item using the same inline
|
||||
source and the [JavaScript recipes](javascript-applications.md). Preserve graph
|
||||
coverage, limitations and unknowns. Native CLI analysis still requires its
|
||||
selected engine, and all CLI operations retain the host's execution permissions.
|
||||
@@ -0,0 +1,860 @@
|
||||
# Desktop app environments from a request
|
||||
|
||||
Use this when someone asks for an environment of a Linux desktop app with a
|
||||
task, such as "Build me an Inkscape environment where the agent changes a
|
||||
poster's headline and saves it". You are the building agent: you drive the
|
||||
app on a build computer through REA's MCP tools, write the task, export the
|
||||
bundle and witness it in a fresh computer. REA runs no model. The tools are
|
||||
MCP-only and need a Linux host with Docker, agentenv and REA's packed
|
||||
runtime. When a start reports one missing, tell the person to run
|
||||
`rea doctor --provider desktop_build` and apply its fixes, or
|
||||
`rea setup --client <client> --desktop-build` and restart the client, and
|
||||
stop.
|
||||
|
||||
For the whole build:
|
||||
|
||||
- Drive the app only with `act_on_build_computer`. Never script it, run
|
||||
commands in the computer or write its files another way. The one command
|
||||
you write for the computer is a task's reader, which REA runs to check a
|
||||
file; it never makes the seed or the edit.
|
||||
- Ask the upfront questions once, then work on your own. Pause only when
|
||||
blocked.
|
||||
- The seed's content is synthetic. Never put real personal data in it.
|
||||
- Close every computer you start, on success and on failure.
|
||||
- Note the time you start; the report gives the build time.
|
||||
|
||||
## Ask once, upfront
|
||||
|
||||
In one message, ask only what the request leaves open, and keep the list for
|
||||
the report. Skip each question the request answers; when it answers them all,
|
||||
start without asking:
|
||||
|
||||
1. The app and the task, if either is unclear: what changes, in which
|
||||
document.
|
||||
2. Whether they have a starting file of their own; otherwise you make one.
|
||||
3. Whether they want the tasks measured after the witness: with which
|
||||
model, within which budget, and whether to harden a task until it lands
|
||||
in the band.
|
||||
4. If so, whether they want to review the tasks before those solves.
|
||||
|
||||
When you cannot ask, as in a headless `claude -p` run, do not stop: take
|
||||
these defaults for whatever the request leaves open, say which you took in
|
||||
your first message and in the report, and go on. Install from the Ubuntu
|
||||
archive, make a synthetic seed, skip the review, and measure each task
|
||||
within $4 in all, with the model the request names, without hardening.
|
||||
|
||||
## When you are blocked
|
||||
|
||||
Stop, say what you need and wait only for:
|
||||
|
||||
- a licence the app or its installer asks someone to accept: never accept it
|
||||
on your own; name it and where to read it, and go on only once the person
|
||||
says they accept it. Then record their words with `record_build_consent`,
|
||||
kind `licence`, and accept it on their behalf;
|
||||
- credentials: ask the person to write each secret, and nothing else, in a
|
||||
file named after a handle you choose, such as `keepassxc-master`, in
|
||||
`~/.config/rea/build-secrets/` on this host (under `$XDG_CONFIG_HOME` when
|
||||
it is set), readable only by them (`chmod 600`), and wait until they say it
|
||||
is there; then type it with `type_secret` (see Drive the app). Never ask
|
||||
for a password or key in chat, and never read, list or print that folder;
|
||||
- a signed-in state in the seed: by default the export withholds it (see
|
||||
Sign-in and licence state). Seed it only when the person says the bundle
|
||||
may hold it: record their words with `record_build_consent`, kind
|
||||
`seed_sign_in`, and export with `seed_sign_in: true`;
|
||||
- an install this flow cannot do, below.
|
||||
|
||||
Decide everything else yourself and note it in the report.
|
||||
|
||||
## Sign-in and licence state
|
||||
|
||||
What an app writes when someone signs in or activates a licence is withheld
|
||||
from the seed: the bundle's receiver supplies their own. Mark
|
||||
`pre-sign-in` with `mark_build_checkpoint` just before the sign-in or
|
||||
activation and `post-sign-in` just after, both before `seed`: every file
|
||||
under the owned paths that changed between them is withheld, and the draft
|
||||
leaves it out. A file the app writes at first launch, such as a trial's
|
||||
licence file, changes before you can mark anything: name it in
|
||||
`export_build_environment`'s `withhold`. The export lists each withheld file
|
||||
in `withheld_files` with the environment variable a deploy reads it from and
|
||||
the handle of the secret REA keeps its bytes in; a check may not name one.
|
||||
The export refuses a seed file holding a withheld file's exact bytes, such as
|
||||
a `.bak` copy, but a backup that differs from it is not caught: trash backups
|
||||
before `seed`.
|
||||
Explicitly withhold any other files the person identifies as sensitive;
|
||||
REA does not infer secrecy from a value's shape or a file name.
|
||||
|
||||
## Choose the install
|
||||
|
||||
The computer is Ubuntu 24.04 (noble) on this host's architecture only. Use
|
||||
the first source that has the app:
|
||||
|
||||
| Source | `install` |
|
||||
| ---------------------------- | ------------------------------------------------------------------------------------------- |
|
||||
| Ubuntu archive | `{"kind": "archive", "package": "inkscape"}` |
|
||||
| PPA or vendor apt repository | `{"kind": "apt_repository", "package", "uri", "suite", "components", "key_file"}` |
|
||||
| `.deb` by URL | `{"kind": "package_file", "url", "sha256", "architecture", "dependencies"}` |
|
||||
| AppImage by URL | `{"kind": "appimage", "url", "sha256", "architecture", "extra_packages"}` |
|
||||
| Tarball by URL | `{"kind": "tarball", "url", "sha256", "architecture", "program", "name", "extra_packages"}` |
|
||||
|
||||
- A PPA's `uri` is `https://ppa.launchpadcontent.net/OWNER/NAME/ubuntu`,
|
||||
`suite` `noble`, `components` `["main"]`. `key_file` is the repository's
|
||||
signing key saved on this host; for a PPA, fetch it from
|
||||
keyserver.ubuntu.com by the fingerprint its Launchpad page shows.
|
||||
- When the app exits at launch for want of a runtime piece its package does
|
||||
not install, start a new build computer with that package in
|
||||
`extra_packages`, up to 16 archive names; the export installs them too.
|
||||
MyPaint needs
|
||||
`{"kind": "archive", "package": "mypaint", "extra_packages": ["librsvg2-common"]}`
|
||||
for its SVG icons, and Freeplane `["openjdk-17-jre"]`, since its launcher
|
||||
refuses noble's Java 21. Name one up front, too, when a package the app
|
||||
only recommends holds its content, such as QElectroTech's element library
|
||||
`qelectrotech-data`, without which no component can be placed.
|
||||
- Take the `.deb` path only when the app is not in the archive at all, not
|
||||
to add a package. Ask the requester for its URL and SHA-256; do not fetch
|
||||
a vendor's download page yourself. List its `Depends` as `dependencies`,
|
||||
with any runtime piece it lacks: it installs with the network off.
|
||||
- Take an AppImage or a tarball only when no package has the app. Pin it by
|
||||
the release file's URL for this host's architecture and the SHA-256 its
|
||||
project publishes, such as a GitHub release asset's digest or the
|
||||
release's `SHA256SUMS`; ask the requester when the project publishes none.
|
||||
The computer downloads it when the bundle is put, refuses other bytes, and
|
||||
unpacks it under `/opt/rea-apps/<desktop_entry>` with the network off; the
|
||||
export downloads the same file.
|
||||
- An AppImage installs its own desktop entry, so `desktop_entry` is the
|
||||
`.desktop` file at the top of the image, such as FreeCAD's
|
||||
`org.freecad.FreeCAD`. A wrong name fails the start with the entries the
|
||||
image has: start again with one of them. The image brings its libraries,
|
||||
so `extra_packages` is rarely needed; FreeCAD 1.1.4's needs none.
|
||||
- A tarball has no entry of its own. `program` is the program's path inside
|
||||
the archive, such as `firefox/firefox`; REA writes an entry named
|
||||
`desktop_entry` that runs it with the files it opens, and `name` is what
|
||||
the Applications menu shows. A tarball uses the computer's libraries: name
|
||||
what its program lacks in `extra_packages`. gzip, bzip2 and xz archives
|
||||
are read, zstd ones are not.
|
||||
- Flatpak and snaps are not supported: say so and stop. Flatpak's sandbox
|
||||
needs user namespaces the computer's container does not grant, so a
|
||||
Flathub app installs but never opens; ask for the project's AppImage,
|
||||
tarball or `.deb` when it ships one. An archive package that only installs
|
||||
a snap, such as noble's `firefox`, counts as a snap. A package with no
|
||||
build for this host's architecture (some PPAs are amd64 only) cannot be
|
||||
installed either.
|
||||
- `desktop_entry` is the package's `.desktop` file name under
|
||||
`/usr/share/applications` without `.desktop`, such as
|
||||
`org.inkscape.Inkscape`. Add `window_class` when you know it; Inkscape's
|
||||
is `Inkscape`. AbiWord's entry is `abiword` and its class `Abiword`.
|
||||
- To find the entry, read the package's file list: its page on
|
||||
packages.ubuntu.com, or `apt-get download PKG && dpkg -c PKG_*.deb` where
|
||||
apt runs. The entry may ship in a companion `-data` or `-common` package,
|
||||
as Planner's in `planner-data`: read that package's list too. The entry's `StartupWMClass` may not be the window's real
|
||||
class, so do not take `window_class` from it: MuseScore 3's says
|
||||
`mscore3`, its window's is `MuseScore3`. The export uses the class the
|
||||
capture saw.
|
||||
|
||||
## Drive the app
|
||||
|
||||
- Pass `computer_id` and the action's fields at the top level: click
|
||||
`{x, y, count}`, scroll `{x, y, direction, amount}`, a screenshot region
|
||||
`[x0, y0, x1, y1]`. The rest are in
|
||||
[the build-computer guide](https://github.com/morluto/rea/blob/main/docs/desktop-build-computer.md).
|
||||
- Take a `screenshot` before pointing: points are pixels of the latest
|
||||
frame. If a response is too large, ask for a `region`, or `jpeg` with a
|
||||
lower `quality`. A screenshot with `region` does not capture the screen
|
||||
again; it crops the latest frame, so take a full screenshot, or a `wait`
|
||||
with `screenshot: true`, first. The crop then becomes the latest frame:
|
||||
take a full screenshot before pointing again.
|
||||
- A drawn caret does not prove keyboard focus. After using a toolbar
|
||||
control, click into the document, then type a few characters and look
|
||||
before a long text.
|
||||
- Confirm a dialog with `key` Return, not a trailing `\n` in `type`.
|
||||
- An action's own screenshot is taken at once. After opening a menu, a
|
||||
dialog or a window, look with
|
||||
`{"action": "wait", "ms": 500, "screenshot": true}`.
|
||||
- On a computer started from a request the app is not open at first, and
|
||||
`start_build_capture` opens it: never open it yourself before that. Do not
|
||||
close it while capturing; the capture ends when the app exits. When the
|
||||
app is closed at any other time, open it from Applications → Other → the
|
||||
environment's name.
|
||||
- Type a secret with `{"action": "type_secret", "secret": "<handle>"}`,
|
||||
only into a field that masks what is typed: check in a screenshot that it
|
||||
is a password field, which shows dots. A screenshot shows any field that
|
||||
shows its text, such as many licence-key boxes, so never type a secret into
|
||||
one: say the app needs a secret the build cannot type unseen, and stop.
|
||||
`type_secret` takes no `screenshot`; look with a `wait` after it. A refusal
|
||||
names the secret's file and what is wrong with it: tell the person, never
|
||||
the value.
|
||||
- Prefer shortcuts and typing to long pointer paths: `key` takes `keys`, an
|
||||
array holding one chord's keys, such as
|
||||
`{"action": "key", "keys": ["Control", "s"]}`, and `type` turns `\n` into
|
||||
Enter.
|
||||
- Before `type`, check in a screenshot that a text cursor or a focused
|
||||
field is there. In apps with single-key shortcuts, such as Inkscape, text
|
||||
typed without one runs as commands.
|
||||
- Save As differs by app. Check the dialog's current folder first: most
|
||||
open in the home, so click Documents in the sidebar, but Scribus opens in
|
||||
`~/Documents` and its sidebar has no Documents entry. Type only the file
|
||||
name. A dialog taller than the 1280×800 screen hides its buttons:
|
||||
maximize it from its title bar, or confirm it with `key` Return, as when
|
||||
HomeBank's Save button is off-screen. When a GTK Save dialog ignores
|
||||
clicks or runs off the screen, type `~/Documents/<name>` in its Name field
|
||||
and press Return.
|
||||
- Answer the app's save questions, such as keeping the format, then check
|
||||
the saved path in the title or status bar, or the dialog's folder when
|
||||
the app shows only the name, and that it shows no unsaved mark, before
|
||||
you mark a checkpoint or check. Where the title never shows one, as in
|
||||
HomeBank, a greyed Save or Revert, or the file reopened from recent
|
||||
files, confirms the save.
|
||||
- Qt spin boxes change their value on scroll: move the pointer off them
|
||||
before you scroll.
|
||||
- gLabels edits an object's text in Object properties → Text, not on the
|
||||
canvas.
|
||||
- In Grisbi's transaction form Return moves to the next field; only Return
|
||||
in the last field, Notes, records the transaction.
|
||||
- ghostwriter autosaves a new document as `~/Documents/untitled-1.md`: name
|
||||
it with File → Rename, not Save As; Ctrl+Shift+S does nothing, and Enter
|
||||
after a checklist item starts another `- [ ]` item.
|
||||
- KMyMoney's first launch opens the New File Setup wizard; Finish asks for a
|
||||
storage type (XML), then a Save As where typing `Documents/<name>.kmy`
|
||||
works. Its date field is a sectioned Qt editor: double-click the month and
|
||||
type each part, since Ctrl+A garbles it. A new transaction's payee comes
|
||||
prefilled and selected. Amounts are saved as reduced fractions, such as
|
||||
`-1749/20`, so compare them exactly. Untick Help → Tip of the Day → "Show
|
||||
tips on startup" before `seed`, or the tip appears at every boot.
|
||||
- QElectroTech: an element dropped from Collections follows the pointer until
|
||||
a click on the folio places it. Its title shows only the app, so it needs
|
||||
`state_paths`; its settings are in `~/.qet`.
|
||||
- Rosegarden: after a dialog its Transport window can take keyboard focus,
|
||||
so save with File → Save.
|
||||
- Pencil2D: the Tools dock is too narrow for its buttons, so use shortcuts
|
||||
(B for the brush); double-click a layer's name to rename it.
|
||||
- Umbrello: the title shows no unsaved mark, so confirm a save with a probe
|
||||
checkpoint. An Untitled document autosaves `~/autosave.xmi`: delete it
|
||||
before `seed`. Add attributes from the tree: right-click the class → New →
|
||||
Attribute.
|
||||
- Hydrogen: the computer has no audio device, so Hydrogen falls back to a
|
||||
null driver and its BPM field snaps back to 120 within about 100 ms.
|
||||
Before `seed`, set Preferences → Audio System → ALSA with device `null`
|
||||
and click "Apply and restart output", which `~/.hydrogen/hydrogen.conf`
|
||||
keeps for the seed; also set autosaves per hour to 0, and dismiss the
|
||||
development-version dialog with "Don't show again". `pulseaudio` as an
|
||||
extra package does not help.
|
||||
- Poedit cannot add entries by hand. Use File → New, pick the language and
|
||||
save to `Documents` first (Sources Paths stays locked until then), then
|
||||
Properties → Sources Paths → + → Add Files with a source such as
|
||||
`/usr/lib/python3.12/argparse.py`. It writes `fr.mo` beside the `.po`, and
|
||||
the language box's autocomplete popup takes the first OK click.
|
||||
- CherryTree: its first-run autosave "Choose Storage Type" dialog swallows
|
||||
keys, so save once early as `.ctd` and untick autosave.
|
||||
- VSCodium (its tarball, with `extra_packages: ["libnss3",
|
||||
"libasound2t64"]`) runs on outside REA's reach, so `stop_build_capture`
|
||||
fails to clean it up: after `end`, quit it with Ctrl+Q, then stop. Enter
|
||||
accepts its word suggestions, so press Escape after typing a line.
|
||||
- Glade: a new widget's id is empty until you set it, and a second save
|
||||
writes a `.glade~` backup, which shows in the draft as an unrelated
|
||||
change. A backup present at `seed` is seeded with the document, so trash
|
||||
it before `seed`. The draft names a button's `<child>` by the id
|
||||
its widget carries, `[name=child][>object@id=submit_button]`, which moving
|
||||
the button keeps; give the widget an id before `seed` so it can.
|
||||
- VYM: the title never names the map, so pass `Documents/<map>.vym` in
|
||||
`state_paths`. Its settings live in `.config/InSilmaril` and are written
|
||||
only at a clean exit. It writes `<map>.vym.lock` while a map is open, so
|
||||
close the map before `seed`. Press A only after clicking a branch on the
|
||||
canvas: keys typed in the tree editor run other shortcuts.
|
||||
- KmPlot: Create → Cartesian Plot, then edit the formula in the Function
|
||||
tab and press Return. It stores minus as U+2212 (`−`). Its file:// title
|
||||
still names the file.
|
||||
- RedNotebook keeps each month in `~/.rednotebook/data/YYYY-MM.txt`, YAML
|
||||
read as text, folded at about column 80: pick check phrases the fold keeps
|
||||
on one line.
|
||||
- Qt5 apps saving with QDom, such as KmPlot, write attributes in a random
|
||||
order in each process, so check their XML by element, never by whole line.
|
||||
- Gnumeric's desktop entry is `org.gnumeric.gnumeric`. Its Save As dialog
|
||||
runs off the 1280×800 screen: type `~/Documents/<name>` in Name and press
|
||||
Return. Rename a sheet with right-click on its tab, then Rename….
|
||||
- Parley's title shows the collection's title, so name the file in
|
||||
`state_paths`. File → New opens the new collection's Properties dialog:
|
||||
confirm it with OK, since Cancel keeps the old collection and its autosave
|
||||
lock open. A new collection starts with 15 blank entries: delete them.
|
||||
Pick languages from the dropdown; a typed name keeps the previous locale.
|
||||
While a document is open KDE keeps an autosave and its `.lock` in
|
||||
`~/.local/share/stalefiles/parley/`, and there is no File → Close: before
|
||||
`seed`, use File → New, then reopen the collection from Open Recent.
|
||||
- Sigil: pass `extra_packages: ["libqt5svg5"]`, or its SVG toolbar icons are
|
||||
blank. Its title names the book, so it needs no `state_paths`. It unpacks
|
||||
the open book into `.local/share/sigil-ebook/sigil/workspace/Sigil-XXXXXX/`
|
||||
and deletes it on exit, so never name the whole `sigil-ebook` folder: that
|
||||
seeds the stale copy.
|
||||
- What the app saved under the home before `seed`, such as a document in
|
||||
`~/Documents` or its settings, is part of the seed. A file you move to the
|
||||
Trash is left out, and the export lists it in `limitations`. Trash a stray
|
||||
file rather than starting over. JabRef 3.8 writes `<file>.bak` and
|
||||
`<file>.sav` beside a `.bib` it saves; in the owned folder they are seeded
|
||||
and show in the draft as noise, so trash them before `seed` if they should
|
||||
not ship.
|
||||
- Dismiss first-run dialogs, such as a welcome screen, before `seed`. A
|
||||
choice the app saves only later, such as Inkscape's "show this every
|
||||
time", may not reach the seed, so expect the dialog in the witness too.
|
||||
- Some apps write their settings late: Freeplane writes
|
||||
`~/.config/freeplane/1.7.x/auto.properties` only once Preferences is
|
||||
confirmed, and a seed without it reopens its documentation map. Before
|
||||
`seed`, open the app's preferences and confirm them, then read a probe
|
||||
checkpoint's `changed_files` to see the settings file written.
|
||||
- Before `seed`, click into the document to clear any selection the app
|
||||
would save: Gnote saves its cursor and selection with the note.
|
||||
- An app whose state folder is named after it, such as
|
||||
`~/.local/share/gnote`, or after it and its version, such as TreeLine's
|
||||
`~/.treeline-3.1`, is found without `state_paths`.
|
||||
- TreeLine: File → New opens a template window. Book List's CATEGORY makes
|
||||
AUTHOR children by default; change it under Data → Configure Data Types →
|
||||
Default Child Type. A new child opens an inline rename showing "", so
|
||||
press Escape and fill its fields in the Data Edit tab. Ctrl+A adds a child
|
||||
node even while a field has focus. Clicking a node already selected starts
|
||||
an inline rename, so Ctrl+X then cuts its text: cut a node from its
|
||||
context menu. It saves nodes sorted by their
|
||||
generated uid1, so their order changes from run to run. The draft names a
|
||||
new node by its format and a field of its `data`,
|
||||
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, and checks its other
|
||||
fields through that, so give each new node a title no other node has.
|
||||
- After an app crashes, capture again on a fresh computer, so the seed
|
||||
carries no crash-recovery state, such as MyPaint's "Recover interrupted
|
||||
work?".
|
||||
- Give what the task changes a stable name where the app has one, such as an
|
||||
id in Inkscape's Object Properties, so a check can select it.
|
||||
- Expect apps to split text into runs: Writer's revision ids split an
|
||||
edited paragraph into spans and may give it an automatic style such as
|
||||
`P1`, and AbiWord wraps each insertion in `<c>`. Check such text whole
|
||||
with `text_equals` (see Write the task), and find the paragraph by
|
||||
something the edit keeps, such as AbiWord's `xid`.
|
||||
- Freeplane's first launch opens a documentation map: close it with Ctrl+W.
|
||||
Its Ctrl+N asks for a template.
|
||||
- MyPaint 2.0.1 crashes on brush strokes on the build computer: build its
|
||||
seed with flood fill and layer operations.
|
||||
- GnuCash's Ctrl+S on a new book opens a Save dialog with no format choice:
|
||||
use File > Save As to pick XML. Close the book (File > New File) before
|
||||
`seed`, so its `.LCK` lock is not seeded.
|
||||
|
||||
## Build the seed and the edit
|
||||
|
||||
1. `start_build_computer` with
|
||||
`{"source": {"kind": "request", "desktop_entry": ..., "install": ...}}`.
|
||||
The first start of an app builds its image, and starts at the same time
|
||||
on this host wait for each other; either can take minutes. A failed
|
||||
start returns the build's error: fix the request, or stop when the
|
||||
install cannot work, and close any `computer_id` the error names.
|
||||
2. `start_build_capture`. It returns once the app's window is open. A
|
||||
`package_lists_error` may stop the capture finding the package's
|
||||
sources; the export says when that matters. When the app exits before
|
||||
its window opens, the error gives its exit code and the end of its
|
||||
stderr, which may name what it lacks, such as an icon loader or a Java
|
||||
runtime: close the computer and start a new one with that package in
|
||||
`extra_packages` (see Choose the install), or stop as for an install
|
||||
this flow cannot do.
|
||||
- REA counts a file as the app's document only when the app's window
|
||||
title shows its file name or `~/` path. When the title shows something
|
||||
else, such as MuseScore's score title, pass the path you will save the
|
||||
document at in `state_paths`, relative to the home:
|
||||
`{"computer_id": ..., "state_paths": ["Documents/MuseScore3/Scores/Morning_Song.mscz"]}`.
|
||||
It need not exist yet. Add the app's settings folder when its name
|
||||
differs from the app's, such as `.config/MuseScore`, so the seed keeps
|
||||
the dismissed first-run dialogs. Do not pass the document as a launch
|
||||
argument to make REA see it: `state_paths` is the way. gLabels' title
|
||||
leaves out the file's extension and Grisbi's shows the entity's name,
|
||||
so both need it.
|
||||
- An app whose document is a folder of pages, such as a Zim notebook
|
||||
(`Notebooks/Notes`, a `.txt` file per page), never shows a file in its
|
||||
title: pass the folder in `state_paths`. Every page in it is then a
|
||||
document, so the bundle opens none at launch and the app starts where
|
||||
it keeps its place; check the page the task changes.
|
||||
3. Make the seed through the app and save it from the app under
|
||||
`~/Documents`: build the document in the app, or write its source where
|
||||
the app edits its format as text, then save it again from the app. Make
|
||||
it look real but synthetic: the thing the task changes, plus a few things
|
||||
the change must leave alone. Keep it to one document: the bundle's app
|
||||
opens the seed's document at launch only when there is exactly one, and
|
||||
otherwise starts on its own screen. For a starting file of the requester's,
|
||||
read it here and rebuild it in the app with synthetic data in place of
|
||||
personal data; REA cannot copy a file into the computer.
|
||||
- An app that keeps its own data and saves after every change, such as
|
||||
Osmo's contacts in `~/.local/share/osmo/contacts_records.xml`, has no
|
||||
document: that state is the seed. Build it in the app and mark `seed`
|
||||
once the app has written. To see whether it has, mark a throwaway
|
||||
checkpoint, such as `probe-1`, and read its `changed_files`: up to ten
|
||||
files that changed since the checkpoint before, those a window title
|
||||
named first, then those under `state_paths` and those the app held
|
||||
open, caches last, with `more_changed_files` counting the rest.
|
||||
- The app's own compressed format works when it is gzip, such as
|
||||
Gnumeric's `.gnumeric`, Dia's `.dia` or Inkscape's `.svgz`: REA reads
|
||||
the document inside, and checks point into it as into the same
|
||||
document uncompressed. Compression may be off by default: Dia's
|
||||
"Compress diagram files" is unticked, which REA reads as plain XML. A format REA reads as binary, such as one
|
||||
compressed with xz, has no view: save in a format the app offers that
|
||||
REA reads, then reopen the file once and check that a plain Ctrl+S
|
||||
keeps that format. When the task is about the app's own binary
|
||||
format, such as GIMP's `.xcf` or Blender's `.blend`, keep it and check
|
||||
it through a reader (see Read a binary file with a reader).
|
||||
4. Only once the seed is saved, `mark_build_checkpoint` with name `seed`.
|
||||
5. Make the change the task asks for, as a user would, and save it.
|
||||
6. Only once the change is saved, `mark_build_checkpoint` with name `end`.
|
||||
For more tasks in this environment, make each next task's change on top
|
||||
of the last as a user would, save it, and mark `end-<task name>`, such as
|
||||
`end-add-bread`. Each task is checked at its own end, so its checks must
|
||||
not read what a later task changes, and a solving agent starts each from
|
||||
the seed, so a task must not need an earlier one done first.
|
||||
Then, for each task you can, make a near miss of its change on top, a
|
||||
plausible wrong edit such as a typo or a wrong number, save it and mark
|
||||
`near-<task name>`: the export refuses a task whose near miss passes every
|
||||
check. Take the near misses after the last end, since each later draft
|
||||
starts from the end before it.
|
||||
7. `stop_build_capture`. Read `draft.files` (each file that changed, with
|
||||
its net change and its `values`), `draft.task.checks` (suggested
|
||||
checks), `draft.guards`, `draft.notes` and `limitations`; each
|
||||
`later_drafts` entry is the same for one later task, from the end
|
||||
checkpoint before its own, so it shows only that task's change. When
|
||||
`draft` is null, read `draft_refusal` first: a missing `seed` or `end`
|
||||
checkpoint means capturing again on a new build computer, while a failed
|
||||
draft command names its own cause to fix. If a declared secret was used,
|
||||
automatic drafting is withheld: write the task manually and export it.
|
||||
When `draft.files` is empty,
|
||||
`draft.notes` says why:
|
||||
- `no_change`: nothing was saved between `seed` and `end`. Capture again
|
||||
on a new build computer, and save before marking `end`. When the app
|
||||
keeps the task's state only on screen, such as a calculator's result,
|
||||
judge the screen instead (see Judge the screen).
|
||||
- `unowned_changes` with `no_document`: the app saved your document
|
||||
where REA did not count it as the app's. Its `files` list what changed,
|
||||
with their roles; the document is among them. Close the computer and
|
||||
capture again on a new one with that path in `state_paths`; the export
|
||||
refuses a check on a file outside the owned paths.
|
||||
- `uncheckable_changes`: the app saved the change in dconf, as gedit does
|
||||
its preferences, and no check reads dconf. Saving again does not help:
|
||||
choose a change the app saves in a file, and capture again on a new
|
||||
build computer, or judge the screen (see Judge the screen).
|
||||
- `unreadable_changes`: a dconf database changed but REA could not read
|
||||
it; `error` says why. This is not evidence the app saved there: report
|
||||
the error rather than changing the task.
|
||||
|
||||
A note with a `file` says why that file's suggestions are thin:
|
||||
- `by_position`: only position names what changed; its `elements` lack
|
||||
a naming attribute. Name a record that keeps its fields as child
|
||||
elements by one of them, as Osmo's contact by
|
||||
`[name=contact][>first_name=Delphine]`, a JSON record by a field
|
||||
inside it, `[>data/Title=…]` (`~1` for the `/` inside a pointer), or
|
||||
an element by an id its child carries, `[>object@id=…]`; count records
|
||||
with `member_count`, `matching` `[name=contact]`; otherwise check an
|
||||
ancestor's text, or give the element an id in the app and capture again.
|
||||
- `view_state_only`: every change is view state, a renumbered id or a
|
||||
time the app stamps, and its `pointers` are the checks kept. The edit
|
||||
was likely not saved: capture again and save it before `end`.
|
||||
- `long_text`: the element's text is too long to check whole; check a
|
||||
phrase in it with `text_contains`.
|
||||
- `cut_view` or `unreadable_view`: REA's view holds only part of the
|
||||
file, or none a check can read, such as a binary format (step 3). For
|
||||
a binary file, read it with a reader (see Read a binary file with a
|
||||
reader).
|
||||
|
||||
## Write the task
|
||||
|
||||
Write `{"name", "prompt", "checks"}` from the draft and what you did, one
|
||||
per task, each with its own name.
|
||||
|
||||
- `name`: lowercase letters, digits and hyphens, not `smoke`.
|
||||
- `prompt`: what the requester wants, in their words and the app's terms,
|
||||
starting with the app's name, naming the document as a user would and
|
||||
ending with saving it: "In Inkscape, change the poster's headline in
|
||||
Documents/poster.svg to Summer Bake Sale and save it." Name the app: the
|
||||
solving agent's own shell cannot see the computer's files, and the app's
|
||||
name sends it to the screen. Leave saving out for an app that saves on its own, after every change
|
||||
or a few seconds after the last one, such as Osmo or Gnote: grading waits
|
||||
until the app's files have held for 3 s, so the solving agent need not
|
||||
wait for the save. A save more than 3 s after the agent stops is still
|
||||
missed; name such an app in the report. Never say how the file stores it,
|
||||
and never give the checks.
|
||||
- Checks are file-based: content checks that prove the edit, never only that
|
||||
a file exists or changed. Start from the suggestions; keep those that say
|
||||
what the user did and drop what follows from how the app saves.
|
||||
- Compare exact values, so a typo or a wrong number fails.
|
||||
- Read each file's `values` before writing checks: every value the change
|
||||
added, changed and removed, with `before` and `after` (an uncut scalar, or
|
||||
a preview cut at 600 characters) at the pointer a check names it by. Copy
|
||||
the pointer. Copy `after` into `value` only when it is an uncut scalar: a
|
||||
string without a trailing `…`, a number, or a boolean. Compact XML and a
|
||||
JSON array or object are preview strings, so a `value_equals` check needs
|
||||
the structured value from the file's view, and element text uses
|
||||
`text_equals` or `text_contains`.
|
||||
Only XML files, archives, JSON and key files have values: a text file's
|
||||
or SQLite database's `change` already lists its lines and rows.
|
||||
`refers_to` says what a number or id stands for, such as `category="44"`
|
||||
naming `<cat key="44" name="Groceries"/>`, or a new KMyMoney
|
||||
transaction's splits naming their account and payee; use it for the
|
||||
prompt's words. `alternatives` gives a changed value's pointer by the
|
||||
element's other names, such as a label beside an id, to pick another key
|
||||
without another export.
|
||||
Skip entries whose `note` is `by_position` (no check can use that
|
||||
pointer) or noise (`view_state`, `renumbered`, `volatile`,
|
||||
`timestamp`, `upkeep`, `layout`). `values_cut` means a limit left some out.
|
||||
- Select elements by what they hold, never by position: SVG text by id,
|
||||
`/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0`,
|
||||
or an ODF paragraph by style,
|
||||
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Title]/children/0`.
|
||||
A selector must match exactly one element, so give the target a unique id
|
||||
or style in the seed. The draft's pointers already do this: a name-like
|
||||
attribute such as Scribus's `ANNAME` before a coordinate, and two
|
||||
attributes together where one is not enough, as a spreadsheet cell's
|
||||
`[@Col=3][@Row=2]`. Only an element's text is read by position, as
|
||||
`children/0`.
|
||||
- When an app may split an element's text into runs, check it whole:
|
||||
`text_equals` (or `text_contains` for a value inside a sentence) on the
|
||||
element compares all its text, however it is split. Add
|
||||
`normalize_whitespace: true` when the app may reflow spaces. Example, an
|
||||
AbiWord paragraph by its stable id:
|
||||
`{"kind": "text_contains", "file": "Documents/memo.abw", "pointer": "/root/children/[name=section]/children/[name=p][@xid=6]", "value": "at 3:30 PM"}`.
|
||||
Point a text check at the paragraph or another element the edit keeps,
|
||||
never at a run such as AbiWord's `[name=c]`: a reopened document splits
|
||||
the text into several runs. The draft lists only what changed; an
|
||||
unchanged element is in `draft.guards` when it is a like sibling of a
|
||||
changed one, and otherwise use the app's usual numbering, such as another
|
||||
paragraph's `xid`, and let the export confirm it.
|
||||
- In an ODS spreadsheet no attribute names a row: the draft names it by its
|
||||
label cell, `[name=table:table-row][>table:table-cell=February]`. Where no
|
||||
cell names it alone, name the sheet in the seed, and check its whole
|
||||
text: `text_equals` on
|
||||
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:spreadsheet]/children/[name=table:table][@table:name=Q1 Sales]`,
|
||||
or a stretch of it with `text_contains`. Cell texts join with nothing
|
||||
between, so a row reads `February23100`.
|
||||
- For a line added to free text, such as an item on a note's list, the
|
||||
draft compares the whole text with `text_equals`. This rejects longer or
|
||||
negated items, but can reject a correct edit with different ordering or
|
||||
spacing. Report that limitation when the task permits such variations.
|
||||
- For a note placed by its own `x` and `y`, such as Xournal++'s `<text>`
|
||||
on a layer, the draft checks its text on the layer with `text_contains`,
|
||||
which passes wherever the user puts it, and guards each note it kept.
|
||||
- Some apps keep text in an attribute, one element per run, such as
|
||||
Scribus's `<ITEXT CH=...>`: add `"text_attribute": "CH"` to `text_equals`
|
||||
or `text_contains` on the frame,
|
||||
`/root/children/[name=DOCUMENT]/children/[name=PAGEOBJECT][@ANNAME=Headline]`,
|
||||
to read its runs' values joined.
|
||||
- When the draft shows no values or no checks, a scratch export can expose
|
||||
`envs/<env>/seed/` only after the task has at least one valid check: it must
|
||||
pass on the captured end state and fail on the seed. Before the first
|
||||
successful export, use `act_on_build_computer` to open the saved file in an
|
||||
available GUI text editor or viewer and inspect its actual representation
|
||||
without changing it. Use the visible saved content and the task's edit to
|
||||
write a check; an export refusal can also name the end state's value, such
|
||||
as `is "44", not "1"`. If the file cannot be inspected or expressed by a
|
||||
supported check, report the limitation instead of guessing a check.
|
||||
- Where easy, add one check that the rest is untouched, such as another
|
||||
text's value the edit must leave alone. `draft.guards` offers them at
|
||||
their seed values, which the end holds too: a field on the nearest like
|
||||
siblings, one the user sets, such as Planner's `work`, before a default
|
||||
they all share; the text an addition kept; a text file's lines beside its
|
||||
new one. Copy one or two into `checks`; they pass on the seed, which is
|
||||
fine, since the export needs only one check to fail there.
|
||||
- The draft suggests a decimal the app writes as text, such as `146.83`,
|
||||
as a number with `tolerance: 0.001`; keep the tolerance for any number
|
||||
you check that way.
|
||||
- When the draft shows an XML document's changes as `lines`, REA read it
|
||||
as text, for example because its DOCTYPE declares entities. Selectors are
|
||||
impossible there, and `line_count` with `contains` only approximates a
|
||||
value: `3:30` also matches `13:30`. Bound it with neighbouring text and
|
||||
name the check as approximate in the report's limitations.
|
||||
- Some apps save a document reloaded from disk differently from one typed
|
||||
in the same session: AbiWord wraps each new insertion in its own markup.
|
||||
Your build's end state hides this, which is why whole-text checks matter;
|
||||
expect the witness to show the real format.
|
||||
|
||||
The kinds and pointer rules are in
|
||||
[the desktop tasks section](environments.md#pick-the-checks).
|
||||
|
||||
### Read a binary file with a reader
|
||||
|
||||
When the draft notes `unreadable_view` on a binary document, add a reader to
|
||||
the task: the app's own headless command, which REA runs in the computer on a
|
||||
private copy of the file, and whose output a check reads instead of the file.
|
||||
Write it in the app's scripting mode, print each fact on a line starting
|
||||
with `REA ` (with the space), and name the copy `{file}`:
|
||||
|
||||
- GIMP `.xcf`, layer names, as `lines`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "layers",
|
||||
"file": "Documents/poster.xcf",
|
||||
"argv": ["gimp-console-2.10", "-i", "-d", "-f", "-b", "-"],
|
||||
"stdin": "(let ((image (car (gimp-file-load RUN-NONINTERACTIVE \"{file}\" \"{file}\")))) (for-each (lambda (layer) (display \"\\nREA layer \") (display (car (gimp-item-get-name layer)))) (vector->list (cadr (gimp-image-get-layers image)))) (newline))\n(gimp-quit 0)\n",
|
||||
"output": "lines"
|
||||
}
|
||||
```
|
||||
|
||||
checked with `{"kind": "lines_present", "file": "Documents/poster.xcf", "reader": "layers", "lines": ["layer Summer Sale"]}`.
|
||||
Script-Fu's prompt `ts> ` starts a line, so print a newline before each
|
||||
marked line, as `\nREA` does. A text layer's text is
|
||||
`(car (gimp-text-layer-get-text layer))`.
|
||||
|
||||
- Blender `.blend`, objects with their type and location, as `json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "objects",
|
||||
"file": "Documents/scene.blend",
|
||||
"argv": [
|
||||
"blender",
|
||||
"-b",
|
||||
"--factory-startup",
|
||||
"{file}",
|
||||
"--python-expr",
|
||||
"import bpy, json; print('REA ' + json.dumps({o.name: {'type': o.type, 'location': [round(v, 4) for v in o.location]} for o in bpy.data.objects}, sort_keys=True))"
|
||||
],
|
||||
"output": "json"
|
||||
}
|
||||
```
|
||||
|
||||
checked with `{"kind": "value_equals", "file": "Documents/scene.blend", "reader": "objects", "pointer": "/Crate/location/0", "value": 2, "tolerance": 0.001}`.
|
||||
Print every value a check compares as a number or string a check can
|
||||
name, rounded where the app stores floats.
|
||||
Blender 4 shows its Quick Setup splash at every launch until its
|
||||
preferences are saved: with `.config/blender` in `state_paths`, confirm
|
||||
Quick Setup with Continue, save the preferences (Edit → Preferences, then
|
||||
Save Preferences in the menu at its bottom left), and read a probe
|
||||
checkpoint's `changed_files` for `.config/blender/4.0/config/userpref.blend`
|
||||
before `seed`, so solves start on the scene. Blender often takes the first click on a
|
||||
menu or a dialog's button only to focus it: click again.
|
||||
|
||||
- A format LibreOffice opens but REA cannot read, such as a legacy `.xls`,
|
||||
converted with `--convert-to`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "cells",
|
||||
"file": "Documents/budget.xls",
|
||||
"argv": [
|
||||
"sh",
|
||||
"-c",
|
||||
"soffice --headless --convert-to csv --outdir out \"$1\" >/dev/null && sed 's/^/REA /' out/*.csv",
|
||||
"sh",
|
||||
"{file}"
|
||||
],
|
||||
"output": "lines",
|
||||
"timeout_s": 120
|
||||
}
|
||||
```
|
||||
|
||||
Pass `{file}` to `sh -c` as an argument, as here, rather than inside the
|
||||
script.
|
||||
|
||||
The reader runs without the display, in a fresh home with the app's default
|
||||
settings, so use the app's headless or batch mode; set `display: true` only
|
||||
for an app that has none. It must exit 0 within `timeout_s` (60 s by
|
||||
default, at most 300) and print at most 1 MiB. Pick facts the change sets and
|
||||
the near miss gets wrong, such as a name or a number, not the app's whole
|
||||
output.
|
||||
|
||||
Give the task the reader in `readers`, and each check through it `reader`
|
||||
and the reader's `file`. `json` readers serve `value_equals`, `value_absent`
|
||||
and `member_count`; `lines` readers `lines_present`, `lines_absent` and
|
||||
`line_count`. The export runs the reader in the build computer on the seed
|
||||
and the end: when a check through it fails on the end, the refusal shows
|
||||
what the reader printed, and a reader that fails shows its error and the end
|
||||
of its stderr. Fix the reader or the check from that and export again.
|
||||
|
||||
### Judge the screen
|
||||
|
||||
Only when no file shows the change, as when the draft notes `no_change`
|
||||
because the app keeps its state on screen, write the task's checks as
|
||||
screen judges: `{"id": "shown", "kind": "screen_judge", "question": "Does the
|
||||
calculator's display show 8.568?"}`. A model, the judge, answers the question
|
||||
on a screenshot three times, and the check passes when most answers are yes.
|
||||
|
||||
- REA keeps a screenshot at each checkpoint. Mark `seed` with the screen
|
||||
showing the state before the task, where the answer is no, and `end` with
|
||||
the change showing, where it is yes: the bundle ships both as the judge's
|
||||
examples.
|
||||
- Ask about what the change makes visible, with exact values, so the near
|
||||
miss is a no: a result, a label, a selected item. Avoid what a window
|
||||
moved or a cursor blinking can change.
|
||||
- The prompt asks to leave the result showing, since the judge sees only
|
||||
the screen when the agent stops.
|
||||
- A task with screen judges holds only screen judges and no readers. The
|
||||
export refuses one when a file shows the change, and checks none of its
|
||||
questions: the witness does, running the bundle's own judge with the
|
||||
runner's model endpoint (`LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
|
||||
`[model]` in `.agentenv/config.toml`) on a fresh screenshot at each stage.
|
||||
Without an endpoint the witness's check fails: say what to set and stop.
|
||||
- The judge's model is `[model.roles] judge`, else `[model] default`, else
|
||||
`anthropic/claude-sonnet-5-5`. `agent-env run --judge-model` does not
|
||||
reach it. Each grade costs about $0.02.
|
||||
|
||||
## Export
|
||||
|
||||
`export_build_environment` with `tasks` and `directory`, the absolute path of
|
||||
a new or empty folder here, such as `<home>/rea-builds/poster/bundle`. Each
|
||||
entry of `tasks` is `{"task": ..., "end_checkpoint": ...}`: leave
|
||||
`end_checkpoint` out for the task checked at `end`, and give
|
||||
`end-<task name>` for each later one. The
|
||||
export seeds what was saved at `seed`, replaces the computer's host name,
|
||||
`user@host` and home path (a bare user name, such as AbiWord's author
|
||||
`user`, stays), and checks each task against `seed`, its own end and its
|
||||
`near-<task name>` when you took one. It refuses a task
|
||||
unless at least one check fails on the seed, every check passes at its
|
||||
end and one fails at its near miss, and says which task and why, such as a
|
||||
pointer that names nothing at the end. A
|
||||
check that the rest is untouched passes on the seed too, which is fine. On
|
||||
a refusal, fix the task and export again, into another folder if that one
|
||||
is no longer empty. The bundle's eval lists every task, so a run of the
|
||||
bundle runs them all. A refusal naming a secret's handle means a file of the
|
||||
bundle holds that secret, most often an app's sign-in or licence state in
|
||||
the seed: do not work around it, but report it and stop. Keep
|
||||
`seed_files`, `scrubbed_files`, `unscrubbed_files` and `limitations` for the
|
||||
report.
|
||||
|
||||
The limitations name the folders the app wrote that the bundle neither owns
|
||||
nor seeds, such as `~/.config/MuseScore`, so the app starts without them:
|
||||
expect its first-run wizard, a tour or a "restore session?" question at every
|
||||
boot. If the witness shows one, capture again on a new build computer with
|
||||
those folders in `state_paths`, and dismiss the dialogs before `seed`. Lock
|
||||
files the app holds while it runs, such as LibreOffice's `.lock` and
|
||||
`.~lock.<name>#`, GnuCash's `<book>.LCK` or Grisbi's `.<file>.lock`, are
|
||||
left out of the seed and listed after "Left out of the seed". An app can
|
||||
still start fine yet refuse every save when a lock was seeded, so close the
|
||||
document before `seed` where the app lets you (apps other than LibreOffice,
|
||||
such as Rosegarden, also write `.~lock.<name>#` and autosaves beside a
|
||||
document, and closing it covers them too), and treat "file is locked"
|
||||
on a save in the witness as a seeded lock: delete it from the bundle's seed
|
||||
and witness again in a new computer. In LibreOffice, own the profile
|
||||
(`state_paths: [".config/libreoffice"]`) for a clean boot, and close the
|
||||
document before `seed`, then reopen it from Recent Documents to make the
|
||||
change: the export also leaves out the recovery list LibreOffice writes while
|
||||
a document is open, which would start the bundle into document recovery. A
|
||||
limitation naming `~/.config/dconf/user` as unread means the app's dconf
|
||||
settings, such as a first-run tip you turned off, are not seeded: capture
|
||||
again on a new build computer. The limitations also name seeded WebKit,
|
||||
Chromium or QtWebEngine storage and caches; when the witness's boot is clean,
|
||||
list them in the report.
|
||||
|
||||
Keep the build computer open until the witness holds: a task fix exports
|
||||
again from it.
|
||||
|
||||
## Witness
|
||||
|
||||
Witness each task in its own fresh computer of the bundle, from the prompt,
|
||||
as a solving agent would; a computer is witnessed once, so the steps below
|
||||
repeat for each task, each with its own record folder. Do not replay the build's actions or coordinates, and do not
|
||||
check between stages to steer the edit: a solving agent never sees the
|
||||
checks. Give every `check_build_task` the task exactly as exported.
|
||||
|
||||
1. `start_build_computer` with
|
||||
`{"source": {"kind": "bundle", "path": "<bundle>"}}`, adding
|
||||
`"settings": {"<variable>": "<handle>"}` for each of the export's
|
||||
`withheld_files`, or the deploy is refused for want of them, and agentenv
|
||||
keeps that bundle's build failed, so start from a fresh copy of it. The app starts
|
||||
with the seed's files in place and opens the seed's document when the
|
||||
seed holds exactly one and the app takes a file. When the export's
|
||||
`limitations` say it opens no document, open it from the prompt's path,
|
||||
as a solving agent would. Take a `wait` screenshot first, since the
|
||||
desktop may not have drawn yet. When the start fails because the
|
||||
computer never got ready, and its error names lock files in the bundle's
|
||||
seed, delete them from the bundle and start again.
|
||||
2. `check_build_task` with `stage: "seed"` and `directory`, the absolute
|
||||
path of a new or empty folder for the record. A check must fail.
|
||||
Recording starts. Later stages take no `directory`.
|
||||
3. Do the task from the prompt with `act_on_build_computer` and save.
|
||||
`check_build_task` with `stage: "after_edit"`: every check must pass.
|
||||
Each check waits until the app's files have held for 3 s, as the
|
||||
bundle's verifier does, so an app that saves on its own needs no wait of
|
||||
yours. A result with `settled: false` means the files were still
|
||||
changing at the 15 s bound, and `read_cut: true` that a slow read was
|
||||
still running there.
|
||||
4. Make a near miss on top of it: a plausible wrong edit, such as a typo or
|
||||
a wrong number, and save. `check_build_task` with `stage: "near_miss"`:
|
||||
a check must fail.
|
||||
5. `act_on_build_computer` with `action: "reset"`, then `check_build_task`
|
||||
with `stage: "after_reset"`: a check must fail again.
|
||||
6. The last stage ends the witness: `witness.record` has the verdict, the
|
||||
path of `witness.json` and the recording's segments in `directory`.
|
||||
`close_build_computer` on the witness computer.
|
||||
|
||||
A stage that does not hold ends the witness as `failed`. Read each check's
|
||||
result and decide:
|
||||
|
||||
- `after_edit` failed though the saved file is right: the check is brittle.
|
||||
Fix it, export again and witness in a new computer.
|
||||
- `after_edit` failed because your edit was wrong: witness again in a new
|
||||
computer; a computer is witnessed once.
|
||||
- `near_miss` passed: the checks are too loose. Tighten them, export again
|
||||
and witness again.
|
||||
- `after_reset` passed: the reset did not restore the seed. Report it as a
|
||||
limitation.
|
||||
|
||||
Stop after three witnesses of a task and report what still fails.
|
||||
|
||||
## Measure
|
||||
|
||||
Only if the requester asked for it, or by the headless default above. If
|
||||
they wanted to review the tasks first, show them each prompt and checks and
|
||||
wait for their approval. Then `measure_build_task` each task with `bundle`,
|
||||
`task`, `model` and `budget_usd`, its share of the budget. It runs the task
|
||||
in fresh computers, 3 times, then 2 more unless they passed none or all, and
|
||||
keeps a task that passes 1 to 3 of 5 (`band: "in_band"`). It stops before a
|
||||
solve the largest cost so far would take past the budget, and writes the
|
||||
record into the bundle's `difficulty.json` and README. The solver needs a
|
||||
model endpoint where REA starts: `LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
|
||||
`base_url` and `api_key` under `[model]` in `.agentenv/config.toml`; when the
|
||||
measure says a solve did not run for want of one, tell the person what to
|
||||
set and stop. Read the result's `failures_per_check` and each solve's
|
||||
`failed_checks` before calling a task hard: a check that fails in most failed
|
||||
solves while the app shows the right change is brittle, so fix the check,
|
||||
export and witness again, rather than count it as difficulty.
|
||||
|
||||
## Harder tasks
|
||||
|
||||
When the requester wants tasks worth evaluating or training on, harden a
|
||||
task that came out `too_easy` and measure it again, within the budget. The
|
||||
solver sees and fixes its slips on screen, so one edit with the value in the
|
||||
prompt is solved almost every time; so far, on TreeLine with Sonnet 5.5, so
|
||||
were one more dialog, a conditional move of seven records, and ten
|
||||
corrections read from a second document and applied across the list. Try,
|
||||
one at a time, measuring each:
|
||||
|
||||
- A longer chain of exact edits than that, each a value the checks compare
|
||||
whole, so one slip in the chain fails the task.
|
||||
- Facts read in one view and used in another: values the solver must read
|
||||
from a second document the app opens, or from another record, rather than
|
||||
from the prompt, such as "the same year as The Quiet Glacier's".
|
||||
- Distractors beside each target: records with near-identical names or
|
||||
values, and a guard check on each that it stays as it was.
|
||||
- Mistakes that do not show once saved: a value the app displays rounded or
|
||||
formatted unlike what it stores, or an edit the app keeps only when its
|
||||
field loses focus.
|
||||
|
||||
Stop when the budget left cannot cover 3 solves of the hardened task, and
|
||||
report the bands measured.
|
||||
|
||||
Each hardened task stays sound: its checks still fail on the seed and pass on
|
||||
its end, its near miss fails, and another valid way of doing it passes, so
|
||||
check values, never the order or method the solver used. Put the second
|
||||
document in the seed. Count records with `member_count` and check each
|
||||
value by a selector that names its record, since no check joins two records.
|
||||
|
||||
## Close and report
|
||||
|
||||
`close_build_computer` on every computer still open, also after a failure.
|
||||
If a close fails, call it again; if it still fails, tell the requester what
|
||||
is left.
|
||||
|
||||
Report:
|
||||
|
||||
- the bundle path and the environment name;
|
||||
- each task's name, prompt and end checkpoint;
|
||||
- each check, and why it proves the edit and fails the near miss, and each
|
||||
reader's command;
|
||||
- each task's witness stages with their results, the verdict, and the paths of
|
||||
`witness.json` and the recording;
|
||||
- the seed's contents: `seed_files` and what each holds;
|
||||
- `scrubbed_files` and `unscrubbed_files`; an unscrubbed file still names
|
||||
the host or user, so it needs review before sharing. `recompressed_files`
|
||||
are gzip seeds whose headers were cleared, their content unchanged: give
|
||||
their count;
|
||||
- limitations: the capture's, the export's and anything you could not do;
|
||||
- each measured task's band, passes of finished solves, turns, cost and
|
||||
failures per check, from `difficulty.json`;
|
||||
- the build time and the questions you asked.
|
||||
|
||||
Never publish or share the bundle yourself.
|
||||
@@ -0,0 +1,399 @@
|
||||
# Agent environments from a live application
|
||||
|
||||
Use this when the user asks for an agent environment (an AgentEnv bundle) of a
|
||||
web application: a folder `agent-env` deploys, whose MCP tools answer from what
|
||||
one recording of the application observed and serve the writes that recording
|
||||
confirmed. The workflow records the application once with
|
||||
`capture_browser_scenario`, checks what the environment would serve, records
|
||||
again if a hint names a gap, then writes the bundle with
|
||||
`export_environment_bundle`. For a HAR the user already has, skip to
|
||||
[export](#export-and-hand-off) through `inspect_web_network_capture`, or use
|
||||
`rea environment-build <capture.har> <dir>`. For a task in a desktop
|
||||
application's exported environment, read
|
||||
[its section](#tasks-for-a-desktop-application-environment).
|
||||
|
||||
## Before recording
|
||||
|
||||
- Record only against an instance the user says is disposable or seeded for
|
||||
this purpose: every write step creates, edits or deletes its data. If the
|
||||
user has not said so, ask before any scenario that writes. REA does not
|
||||
start, seed or reset applications.
|
||||
- Credentials come from environment variables the user names. Declare each as a
|
||||
scenario secret, `{secret_id, environment_variable}`; the variable must be set
|
||||
in the environment REA's MCP server started with. Never ask for a password in
|
||||
chat. Declared secrets are redacted as the capture records.
|
||||
- Map the application first with `inspect_web_page` (and `capture_web_screenshot`
|
||||
when layout matters): the login form's locators, and the API calls behind the
|
||||
pages that list and edit the entities the user asked about.
|
||||
- `inspect_web_page` cannot sign in. To see pages behind sign-in, run a short
|
||||
`capture_browser_scenario` that signs in and opens them, with
|
||||
`"capture": {"after_each_step": ["accessibility"]}` (`"dom"` for markup).
|
||||
Read a step's tree with `inspect_analysis_view`, `view: {kind:
|
||||
"browser-scenario", collection: "text", step_index, artifact:
|
||||
"accessibility", offset: 0, limit: 2048}`, paging with `offset`. Capture
|
||||
options are only `after_each_step`, `at_end`, `events`, `network` and
|
||||
`limits`.
|
||||
|
||||
## Record a session an environment can learn writes from
|
||||
|
||||
One `capture_browser_scenario` records the whole session. Start from this
|
||||
shape, with the application's own locators, paths and headers:
|
||||
|
||||
```json
|
||||
{
|
||||
"browser": { "mode": "launch", "executable_path": "/usr/bin/chromium" },
|
||||
"start_url": { "url": "http://127.0.0.1:3000/login" },
|
||||
"secrets": [
|
||||
{ "secret_id": "admin_password", "environment_variable": "APP_PASSWORD" }
|
||||
],
|
||||
"capture": {
|
||||
"network": {
|
||||
"request_body": true,
|
||||
"response_body": true,
|
||||
"header_values": true
|
||||
}
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"step_id": "user",
|
||||
"action": "fill",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=username]" },
|
||||
"value": { "source": "literal", "value": "admin" }
|
||||
},
|
||||
{
|
||||
"step_id": "password",
|
||||
"action": "fill",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=password]" },
|
||||
"value": { "source": "secret", "secret_id": "admin_password" }
|
||||
},
|
||||
{
|
||||
"step_id": "sign-in",
|
||||
"action": "press",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=password]" },
|
||||
"key": "Enter"
|
||||
},
|
||||
{
|
||||
"step_id": "signed-in",
|
||||
"action": "wait_for",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "nav a[href='/settings']" },
|
||||
"state": "visible"
|
||||
},
|
||||
{
|
||||
"step_id": "folders-1",
|
||||
"action": "request",
|
||||
"timeout_ms": 15000,
|
||||
"method": "GET",
|
||||
"path": "/api/folders",
|
||||
"expected_status": [200]
|
||||
},
|
||||
{
|
||||
"step_id": "new-folder-1",
|
||||
"action": "click",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "role", "role": "button", "name": "New folder" }
|
||||
},
|
||||
{
|
||||
"step_id": "folder-title-1",
|
||||
"action": "fill",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "css", "selector": "input[name=title]" },
|
||||
"value": { "source": "literal", "value": "Scratch folder 1" }
|
||||
},
|
||||
{
|
||||
"step_id": "create-folder-1",
|
||||
"action": "click",
|
||||
"timeout_ms": 15000,
|
||||
"locator": { "kind": "role", "role": "button", "name": "Create" },
|
||||
"save_response": {
|
||||
"method": "POST",
|
||||
"path": "/api/folders",
|
||||
"save": [{ "name": "folder_1", "json_pointer": "/id" }]
|
||||
}
|
||||
},
|
||||
{
|
||||
"step_id": "read-folder-1",
|
||||
"action": "request",
|
||||
"timeout_ms": 15000,
|
||||
"method": "GET",
|
||||
"path": "/api/folders/{saved:folder_1}",
|
||||
"expected_status": [200]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Give each step that accepts `timeout_ms` a deadline. Without one, a step waits
|
||||
until it completes or the scenario is cancelled, so a locator that never
|
||||
matches hangs the capture and the client with it. The `wait_for_timeout` pause
|
||||
instead uses `duration_ms` and does not accept `timeout_ms`. Match `wait_for`
|
||||
locators to exactly one element.
|
||||
|
||||
1. Sign in through the UI: `fill` the user and password fields (the password
|
||||
as a secret reference), submit, then `wait_for` an element only a signed-in
|
||||
user sees.
|
||||
2. Read with `request` steps: each collection the environment needs, twice
|
||||
before any write, so fields that change on their own show up; then each
|
||||
object a list shows by id, up to ten per collection. An object known only
|
||||
from a list that leaves out fields its own read shows cannot be read by id
|
||||
or updated in the environment.
|
||||
3. Write the way the application's UI does: fill its forms and click its
|
||||
buttons, so each request carries the fields the UI sends, including ones
|
||||
the server would otherwise derive. Give the click or key press that submits
|
||||
a create `save_response` to keep the new object's ID, then `goto` its page
|
||||
(`/items/{saved:item_1}`) or read it with a `request` step. Never guess an
|
||||
ID, and never create a probe object to learn the next one. Write with
|
||||
`request` steps instead only when the UI cannot reach the operation (an
|
||||
API-only resource, or a UI that does not call the JSON API), or when coverage
|
||||
withholds the UI-shaped write as `unexplained-change` or `contradicted`;
|
||||
that counts as one of the two re-recordings. A `request` step write that
|
||||
coverage withholds that way usually lacks fields the UI sends: record it
|
||||
through the UI instead. If a journey cannot be scripted, leave the write
|
||||
unserved and report the limitation.
|
||||
Record two numbered scratch objects per write type,
|
||||
with varied free text. After each write, and before the next one, read the
|
||||
object back by id and then its list: create 1, read 1, list, create 2,
|
||||
read 2, list; never several writes before their reads.
|
||||
4. Leave more than a second between two creates and between a create and its
|
||||
first edit (`wait_for_timeout` with `duration_ms: 1100`), so lists sorted
|
||||
by creation time show them in order.
|
||||
5. Update both objects and read each back, changing the fields a person
|
||||
would change (a name, an email address, a status, a due date) rather than
|
||||
only a description: a served update accepts only the members a recorded
|
||||
update stored. Run an action that turns something on or off (star,
|
||||
bookmark, archive) twice in a row on one object, reading it back after
|
||||
each: the second call shows whether it sets or flips the field, and the
|
||||
environment serves the action only once a call has shown it. Delete both
|
||||
and read each back (expect 404, or the object marked deleted) and the list.
|
||||
6. Vary what the two creates send, however you write them: the first every
|
||||
field the application takes, the second only what a person must give (its
|
||||
title or name), with the other fields left out or empty. A served create
|
||||
requires each member both creates filled in and lets a caller leave out one
|
||||
a create left out or sent empty, so an agent that names only a title is not
|
||||
refused for fields it had no reason to set.
|
||||
|
||||
About `request` steps:
|
||||
|
||||
- `path` is on the start URL's origin; the page must be on that origin.
|
||||
- Header values are literals, declared secrets, a cookie the browser holds
|
||||
(`{source: "cookie", name}`), a local storage entry
|
||||
(`{source: "local_storage", key, json_pointer}`), a saved value
|
||||
(`{source: "saved", name}`), or Basic credentials
|
||||
(`{source: "basic_credentials", parts: [username, password]}`, sent as the
|
||||
base64 of `username:password` and redacted), with an optional `prefix` such as
|
||||
`"Bearer "` or `"Basic "`. Each credential part is a literal, secret, saved
|
||||
value, cookie or local storage entry, so a header the frontend builds from a
|
||||
login token (the username and the token) can be copied too; the password is
|
||||
a secret or saved value, never a literal. Copy the
|
||||
headers the application's own requests send (a CSRF header such as
|
||||
`X-CSRFToken` from the `csrftoken` cookie, a bearer token). Values read from the page are redacted from records made after they
|
||||
are read, not from the login response that returned them, and values
|
||||
shorter than 8 characters are not redacted at all.
|
||||
- `save: [{name, json_pointer, secret}]` keeps a string or number from the
|
||||
step's JSON response, such as the created object's `/id`, for later steps;
|
||||
`secret: true` redacts it, as for a token the API returns, so mark every
|
||||
saved token, key or session value secret. This also protects the selected
|
||||
source response before its body or headers are retained. Compressed or
|
||||
unknown-size UI responses cannot supply saved values. On a `click` or `press`,
|
||||
`save_response: {method, path, save}` keeps values from the response to the
|
||||
new request observed while the step waits (`{*}` in `path` matches one segment).
|
||||
Matching does not prove UI causality: choose a path distinct from polling;
|
||||
multiple observed matches fail as ambiguous. The step
|
||||
fails unless a 2xx JSON response arrives before its `timeout_ms`. Later
|
||||
steps send a saved value as `{saved:name}` in a request `path` (and its
|
||||
query) or a `goto` URL; in the `json` body, where a string that is only
|
||||
`{saved:name}` keeps the saved JSON type; or as `{source: "saved", name}` in
|
||||
a header or credential part.
|
||||
- Reading a page value makes later bodies costlier to keep: after a CSRF
|
||||
token, bodies over about 47 KiB are kept as `unavailable`. When the API
|
||||
returns larger responses, raise `capture.limits`
|
||||
(`max_network_body_allocation_bytes`, `max_total_event_allocation_bytes`).
|
||||
- Request steps follow no redirects: a redirect fails the step, and
|
||||
`expected_status` cannot name a 3xx status. Use the path the redirect names,
|
||||
such as the one with the trailing slash.
|
||||
- Set `expected_status` on every step, so a refused call fails its step and
|
||||
the later steps report not having run, instead of recording read-backs of
|
||||
nothing.
|
||||
|
||||
Keep it to about 60 to 160 requests. A scenario with no `request` step is
|
||||
recorded exactly as before.
|
||||
|
||||
The capture returns its whole Evidence, every recorded body included, which is
|
||||
often more than a client shows inline. Before checking coverage, confirm that
|
||||
every entry of `normalized_result.steps` has `status: "completed"`; if the
|
||||
client saved the result to a file, search it for `"status":"failed"` and read
|
||||
that step's `error`. A failed sign-in or write cancels every step after it.
|
||||
Use the result's `evidence_id` for coverage and export; do not paste the
|
||||
Evidence back into another tool.
|
||||
|
||||
## Check coverage
|
||||
|
||||
`inspect_analysis_view` with `source: {kind: "retained-evidence",
|
||||
evidence_id}` (the capture's id) and `view: {kind: "environment-coverage",
|
||||
template_rules: ["after-uid-literal", "opaque-token", "sibling-variance",
|
||||
"prefix-variance"]}`. It reports read tools
|
||||
answered from the state or replayed (`reads.recorded_reasons`), the served write
|
||||
tools, and each withheld write with its `reason` and `hint`. It does not apply
|
||||
the serve-time catalog and state size checks; the export reports those.
|
||||
|
||||
Re-record a withheld write at most twice, and only for a recording gap. Stop at
|
||||
once when the reason is application behaviour, and report the write as
|
||||
withheld with its hint:
|
||||
|
||||
| Reason | What to do |
|
||||
| --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `no-state-target` | Check the path template and recorded views; record missing reads. The route form may not be supported as a mutation target. |
|
||||
| `no-observation` | Re-record: read the entity back by id after the write; read it first for an update or delete. |
|
||||
| `unconfirmed` | Re-record: read the entity and its list after the write. |
|
||||
| `incomplete-entities` | Re-record: read the entity through every view that lists it. |
|
||||
| `unexplained-change` | Re-record once with a second, different call of the same write. |
|
||||
| `no-call` | Re-record once with a write that succeeds. |
|
||||
| `side-effect` | Stop: the application changes other entities. |
|
||||
| `contradicted` | Stop: a later read disagrees with the modelled effect. |
|
||||
| `body-unusable` | Re-record a UI write once with `request` steps (its body was not exposed); else stop. |
|
||||
|
||||
Use new scratch names on each attempt, and delete what an earlier attempt
|
||||
created before recording again (through the UI, or a `request` step with the
|
||||
id that capture shows): the environment starts from what the recording first
|
||||
reads, so a leftover object becomes seeded data that write tasks then find.
|
||||
New browser scenario captures include request times, so a fully timed capture
|
||||
can learn a write clock and serve updates that stamp modification times. Older
|
||||
captures or ones missing a request start keep record order and learn no clock.
|
||||
When coverage reports that limitation, record again with the current version
|
||||
before concluding that the timestamp change cannot be served.
|
||||
|
||||
## Export and hand off
|
||||
|
||||
`export_environment_bundle` with `source: {kind: "retained-evidence",
|
||||
evidence_id}` (or `{kind: "evidence-file", path}`), an absolute
|
||||
`output_directory` that does not exist yet and whose parent does, and a `name`
|
||||
(required when the application runs on `localhost` or an IP address). It returns
|
||||
the coverage again, `tool_count`, the written files and warnings; `dry_run:
|
||||
true` reports without writing. The bundle installs `rea-agents` of this REA's
|
||||
version from the registry; when the user runs an unreleased REA build, pass the
|
||||
`package_tarball` they give you (a packed `rea-agents` tarball) instead.
|
||||
|
||||
Tell the user how many tools answer from the state, which writes are served and
|
||||
which are withheld and why, and the run command (`agent-env run <folder>`). Say
|
||||
that the bundle ships the recorded responses and the capture's local paths
|
||||
(for a browser scenario, the browser executable's path), and
|
||||
list any `credential-named-members` warning pointers for review. Never publish
|
||||
or share the bundle yourself.
|
||||
|
||||
See
|
||||
[agent environments](https://github.com/morluto/rea/blob/main/docs/agent-environments.md)
|
||||
for what the state, replay and the bundle establish, and
|
||||
[evidence workflows](evidence-workflows.md) for the manifest and state views.
|
||||
|
||||
## Tasks for a desktop application environment
|
||||
|
||||
Use this when the user has a folder from `rea capture-desktop-application` and
|
||||
wants a task for the environment `rea export-desktop-application` makes from
|
||||
it: a prompt for a computer-use agent and checks that grade what the agent
|
||||
left in the app's files. You write the prompt and the checks; REA validates
|
||||
the task against the capture and exports it. These commands are CLI only. To
|
||||
build the environment and its task from a request instead, follow
|
||||
[desktop builds](desktop-build.md).
|
||||
|
||||
### Describe the change
|
||||
|
||||
Run `rea describe-desktop-application-task CAPTURE --format json`. Read:
|
||||
|
||||
- `owned_paths`: the only paths whose files the app serves, so the only files
|
||||
a check can read.
|
||||
- `files`: each file there whose state differs between the capture's first
|
||||
and last version, with its `role`, the `baseline` and `final` versions and
|
||||
the net `change`. Text changes list each added and removed line once, and
|
||||
`line_counts` gives lines found at both ends whose count changed.
|
||||
- `values` in each of those files: what the change added, changed and
|
||||
removed, `before` and `after`, at the pointer a check names it by, with
|
||||
`refers_to` naming the element a key such as `category="44"` stands for.
|
||||
Skip entries noted `by_position` or as noise. Only XML files, archives,
|
||||
JSON and key files have values: a text file's or database's change
|
||||
already lists its lines and rows.
|
||||
- `task`: a draft whose `checks` are suggestions from the net change of the
|
||||
documents the user worked on, or of the app's own state when no document
|
||||
changed.
|
||||
- `guards`: optional checks that the rest kept its seed values, such as a
|
||||
like sibling's field or a text file's neighbouring lines; add one or two
|
||||
to show the rest is untouched.
|
||||
- `notes`: why `files` is empty, or why a file's suggestions are thin, such
|
||||
as `by_position` when no attribute names the changed element.
|
||||
|
||||
### Write the prompt
|
||||
|
||||
Say what the user wants done, in their words and the app's terms, naming the
|
||||
app: "In Mousepad, add buy milk to my to-do list and save it". A solving
|
||||
agent's own shell cannot see the computer's files; the app's name sends it to
|
||||
the screen. Never say how the file stores it: no
|
||||
paths the user would not name, line formats, XML, keys or SQL, and never the
|
||||
checks. Replace the draft's placeholder; the export refuses it.
|
||||
|
||||
### Pick the checks
|
||||
|
||||
Checks must express only the observed change:
|
||||
|
||||
- Keep a suggestion only if it says what the user did. Drop those that follow
|
||||
from how the app saves: a format rewrite, renumbered lines, counters. A
|
||||
LibreCAD drawing gaining four holes was rewritten from DXF R12 to 2007, and
|
||||
the draft suggests counts of `SECTION`, `ENDSEC` and `CIRCLE`; only
|
||||
`CIRCLE` equals 9 is the user's change.
|
||||
- Never check unchanged content, whole files, or values the app stamps:
|
||||
times, ids it assigns, window sizes. REA refuses ISO 8601 and `asctime`
|
||||
date-times (`Thu Nov 5 09:20:31 2026`); the
|
||||
rest is your judgment.
|
||||
- Prefer a few robust checks: whole lines over positions, rows by meaningful
|
||||
columns over generated ids, `at_least` when the user may add more.
|
||||
|
||||
Kinds: `file_present` and `file_absent`; `lines_present` and `lines_absent`
|
||||
(`lines`); `line_count` (`exact` or `contains`, and `equals`, `at_least` or
|
||||
`at_most`); `value_equals` and `value_absent` (a JSON `pointer` into the view,
|
||||
such as `/groups/GROUP/KEY` for a key file); `text_equals` and
|
||||
`text_contains` (an XML element's whole text at a `pointer`, optional
|
||||
`normalize_whitespace`); `row_present` and `row_absent` (`table`, `where` by
|
||||
column) and `row_count`. Each names its `file` as a path under the home. A
|
||||
document saved as a zip archive, such as FreeCAD's `.FCStd`, reads as its
|
||||
entries: point into an entry's view, and name array members by selector
|
||||
rather than position, as
|
||||
`/entries/[name=Document.xml]/view/root/children/[name=Object][@name=Hole]`,
|
||||
so the check survives an app version that writes them in another order. A
|
||||
gzip-compressed document, such as Gnumeric's `.gnumeric`, reads as the
|
||||
document inside: point into it as into the same document uncompressed. Give
|
||||
`value_equals` a `tolerance` to compare a number the app writes as text, such
|
||||
as `"5.0000000000000000"`, with a number. Check what the user wrote in a
|
||||
document with `text_equals` on its paragraph element, not `value_equals` on a
|
||||
text node: apps such as AbiWord and Writer split a paragraph into runs as they
|
||||
save, so its first text node holds only part of it; use `text_contains` for a
|
||||
value within a sentence, such as a time. A binary document, such as GIMP's
|
||||
`.xcf`, has no view: give the task a `readers` entry, the app's own headless
|
||||
command printing `REA ` lines, and a check naming it in `reader`, as
|
||||
[desktop builds](desktop-build.md#read-a-binary-file-with-a-reader) shows. The
|
||||
[desktop application tasks](https://github.com/morluto/rea/blob/main/docs/desktop-application-tasks.md)
|
||||
guide has the exact rules.
|
||||
|
||||
### Export the task
|
||||
|
||||
Write the task as JSON with a `name`, the `prompt` and the `checks`, then run
|
||||
`rea export-desktop-application CAPTURE DIRECTORY --task TASK.json`. REA keeps
|
||||
the task only when its checks fail on the seed and all pass on the captured
|
||||
end state. A refusal names the check: one that fails on the end state does not
|
||||
match what the user left; "every check already passes on the seed" means no
|
||||
check captures the change. For several tasks from one capture, repeat
|
||||
`--task` and name each later task's end with `--task-end NAME=CHECKPOINT`.
|
||||
|
||||
When the export refuses an application that no apt repository installs, ask
|
||||
the user for the `.deb` file they installed it from and capture with
|
||||
`--package-file`, then ask where the computer may download that file and pass
|
||||
it as `--package-url`. Never fetch a vendor's download page yourself: such
|
||||
sites often refuse automated downloads, and the copy and its licence are the
|
||||
user's to choose.
|
||||
|
||||
Tell the user the task's name, prompt and checks, and how to measure a solve
|
||||
rate: run `agent-env run DIRECTORY --task NAME --agent claude-code` with
|
||||
`--model MODEL` three times and count the runs reported `passed`. The runs
|
||||
need a model endpoint: `LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
|
||||
`base_url` and `api_key` under `[model]` in `.agentenv/config.toml`. Do not run
|
||||
agents unless asked. The bundle's seed is the user's real data: keep it local.
|
||||
@@ -0,0 +1,176 @@
|
||||
# Evidence, comparison, and verification workflows
|
||||
|
||||
Evidence-producing tools return the complete Evidence record directly in MCP
|
||||
`structuredContent` and CLI JSON. Read `normalized_result` for operation data,
|
||||
and cite `evidence_id` when another tool or your explanation needs a stable
|
||||
reference. Preserve authority, limitations, coverage, and residual unknowns. If a result
|
||||
is incomplete or paginated, continue only when the remaining data matters to
|
||||
the task; do not fetch a bundle or resource merely to read a result already
|
||||
returned inline. When complete `inspect_binary_layout` or
|
||||
`analyze_javascript_application` Evidence is already retained, call
|
||||
`inspect_analysis_view` for a summary, one section/symbol/module, or a stable
|
||||
page instead of repeating the producer.
|
||||
|
||||
For any logical Evidence source that advertises retained references, pass
|
||||
`{kind: "retained-evidence", evidence_id}` instead of copying the complete
|
||||
record back into a call. The same contract covers individual, optional, array
|
||||
and nested sources across native, process, JavaScript, managed, mobile and WASM
|
||||
workflows. Sources can independently mix inline records and exact references.
|
||||
Resolution preserves the source's identity, provenance, coverage, partial facts
|
||||
and unknowns; it neither reruns analysis nor makes incomplete facts complete.
|
||||
Keep domain role and identity requirements: a native dossier cannot substitute
|
||||
for a managed member observation, for example.
|
||||
|
||||
When starting from a summary or inspection page, use its
|
||||
`normalized_result.parent_evidence_id` to reference the complete source.
|
||||
The view's own `evidence_id` names
|
||||
only that view. `compare_artifacts` explicitly supports artifact summary inputs
|
||||
and resolves their parents; do not assume every consumer does. When a workflow
|
||||
follows source links, keep those linked records available too.
|
||||
|
||||
MCP references belong to the current connection's ledger. If the ID is known,
|
||||
pass it directly; no discovery call or provider setup is needed. If it is missing,
|
||||
import the exported bundle or supply complete inline Evidence. Do not retry an
|
||||
unchanged reference or rerun a producer merely to move an existing record.
|
||||
Export before closing the session when the source must survive it. For CLI JSON
|
||||
workflows that advertise `--evidence-bundle`, the same reference JSON resolves
|
||||
against one explicitly selected authenticated bundle:
|
||||
|
||||
```bash
|
||||
rea trace-application-feature request.json --evidence-bundle /absolute/path/evidence.json --json
|
||||
```
|
||||
|
||||
This includes JavaScript traces/comparisons and runtime reconciliation, managed
|
||||
reconstruction/native verification/graph projection, Android/Apple/Harmony graph
|
||||
projection, WASM interface comparison and retained analysis views. The managed
|
||||
member comparison CLI still takes two assembly paths; process comparison takes
|
||||
two capture Evidence JSON paths. Check command help for the installed version.
|
||||
An input bundle grants no implicit file lookup: `evidence-file` and historical
|
||||
graph file selectors are accepted only by contracts advertising them. See the
|
||||
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/getting-started/cli.md#import-export-and-compare-evidence).
|
||||
|
||||
Completed `inspect_artifact` inventory also supports a summary and
|
||||
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
|
||||
Omit `logical_paths` to page all available occurrences; `[]` selects none.
|
||||
Other inventory collections are `nodes`, `edges` and
|
||||
`integrity_contradictions`. Keep `source_complete` separate from page exhaustion:
|
||||
a missing match in partial source Evidence does not establish absence. Use the
|
||||
returned parent Evidence ID for subsequent views and export before closing the
|
||||
session when the source must survive it.
|
||||
When `tools/list` advertises `inspect_artifact.detail`, request `"summary"` to
|
||||
retain its complete source and receive that view immediately. Without that
|
||||
advertised choice, keep the existing complete call. CLI inspection returns
|
||||
portable complete Evidence; save it before projecting a selected inline view.
|
||||
|
||||
For native procedure/name/string inventories, select `detail: "summary"` when
|
||||
the connected tool advertises it and the task needs counts or selected rows.
|
||||
Use its parent Evidence ID with
|
||||
`view: {kind: "native-inventory", offset: 0, limit: 32, fields: ["value"]}`.
|
||||
Search tools retain the existing pattern/mode/case/document selection. Omit
|
||||
`fields` for complete records; `record.address` and `source_index` remain on
|
||||
selected rows. Selecting `value` keeps its string/decoding qualifications.
|
||||
Parent Evidence ID plus source index preserves same-address
|
||||
alternatives. Follow `coverage.next_offset` only as far as the question needs;
|
||||
exhausting retained rows does not establish complete program discovery.
|
||||
|
||||
`batch_decompile` returns every body unless you select `detail: "summary"` when
|
||||
the connected tool advertises it. Use its parent Evidence ID with
|
||||
`view: {kind: "batch-decompile", collection: "items"}` for body-free rows and
|
||||
exact `where.indices`, `addresses`, `procedure_addresses` or `statuses`. Add
|
||||
`include_pseudocode: true` for several chosen bodies, or use
|
||||
`collection: "pseudocode"` with one row's `index` to page a large body. Batch
|
||||
limitations aggregate every item, so do not attribute them to one successful row.
|
||||
|
||||
Use `record_unknown` to track unresolved questions in the user's investigation
|
||||
and name the authority or environment still required. Supply supporting and
|
||||
contradicting evidence IDs. It updates the session registry directly.
|
||||
Use `update_unknown` with the current revision; reread after a stale revision
|
||||
instead of retrying blindly. Only qualifying observed evidence can verify a
|
||||
resolution.
|
||||
|
||||
Use comparisons with complete, compatible page sets when claiming equivalence
|
||||
or absence:
|
||||
|
||||
- `compare_artifacts` compares inventory pages by occurrence path, content,
|
||||
metadata, and graph relations. Name each side as
|
||||
`{kind: "retained-evidence", evidence_id}` (an `inspect_artifact` result,
|
||||
its nested inventory or its summary) or
|
||||
`{kind: "evidence-file", path, evidence_id?}` instead of resending Evidence.
|
||||
- `compare_functions` compares explicit function Evidence, not fuzzy
|
||||
whole-binary matches. It rejects an `analyze_function` summary reply; pass
|
||||
the summary's `parent_evidence_id` as a `retained-evidence` reference or
|
||||
the complete dossier.
|
||||
- `compare_bundles` compares canonical bundle membership and unknown history.
|
||||
- `find_changed_behavior` combines existing comparisons; static differences
|
||||
remain candidates, not causal proof. Its `comparisons` accept exact references.
|
||||
- `build_call_path` needs explicit function dossiers from one artifact/provider;
|
||||
pass complete parent IDs in `functions`. An incomplete frontier makes absence
|
||||
unknown.
|
||||
- `correlate_static_and_runtime` uses explicit mappings; matching patterns are
|
||||
hypotheses, not causality. Both comparison arrays accept exact references.
|
||||
- `verify_reconstruction` evaluates a finite typed specification. A pass covers
|
||||
only declared comparable claims, not global equivalence.
|
||||
|
||||
Process captures are opt-in behavioral evidence, not a security sandbox. V3
|
||||
captures cannot be upgraded to V4; rerun the original scenario. Distinguish root
|
||||
exit from descendant settlement and require freshness when the task needs it.
|
||||
A timed-out target that must write a final report needs `finalization_ms`: REA
|
||||
sends SIGTERM to the captured root, keeps observing, and sends SIGKILL once
|
||||
only if the target has not exited by the end of the interval.
|
||||
`exit.finalization.signals` lists each attempt with its delivery result; the
|
||||
observed exit signal stays in `exit.signal`, `exit.code` is null for every
|
||||
deadline exit, and the code of a finalizer that exited on its own is in
|
||||
`exit.finalization.exit_code` (null on a signal exit or when no exit was
|
||||
observed).
|
||||
|
||||
Use `inspect_analysis_view` with `view: {kind: "api-surface"}` on retained
|
||||
`inspect_web_network_capture` HAR Evidence to group observed request lines by
|
||||
method, origin and path template. Format matches are inferences, not handler
|
||||
identity or schemas. Group pages cite parent record ordinals; consult record
|
||||
coverage and exclusions before claiming completeness. No requests are replayed.
|
||||
When the connected schema advertises `include_query_parameter_names`, select
|
||||
it in the view for distinct observed names with original record ordinals. Query
|
||||
values are omitted; the names establish neither required parameters nor a
|
||||
complete server schema. Keep the original parent Evidence for exact URLs.
|
||||
|
||||
Use `inspect_analysis_view` with `view: {kind: "environment-manifest"}` on the
|
||||
same retained HAR Evidence to project the primary origin's request lines into
|
||||
named tools with path templates, parameters, observed query names, value-free
|
||||
body shapes and record pointers for an agent environment. Opt into
|
||||
`template_rules` only when recorded identifiers need them; every template,
|
||||
parameter and merged sibling is an inference, no header, cookie or body value is
|
||||
copied, and the three stated unknowns stay unresolved.
|
||||
|
||||
Use `view: {kind: "environment-state"}` with the manifest's `template_rules` to
|
||||
extract the data those reads observed instead: entity types and identity
|
||||
fields, entities with recorded values, documents, and per-tool render plans
|
||||
checked against the recordings. The state comes only from reads before the
|
||||
first related write; types, identity and list membership are inferences. The
|
||||
view also gives each state-changing tool a write plan, served only when a later
|
||||
recorded read confirms its effect and none contradicts it.
|
||||
`rea environment-serve manifest.json --evidence capture.json` derives that
|
||||
state at startup, answers read tools from it, serves the write tools whose plan
|
||||
is served, each applying its effect to that state, and serves `data/reset`,
|
||||
`data/add` and `data/get` on `/agentenv`; other write tools are not served, and
|
||||
each plan states why in its `reason`. It replays the capture instead with
|
||||
`--replay-only` or when the state is unavailable (not derivable, or a catalog
|
||||
over the discovery budget), and its stopped document's `state` says which
|
||||
applied.
|
||||
Use `view: {kind: "environment-coverage"}` with the same rules to see what an
|
||||
environment built from a capture would serve: read tools answered from the
|
||||
state or replayed with their reason, and each write tool served or withheld
|
||||
with a hint about what to record next; it does not apply the serve-time
|
||||
catalog and state size checks. `rea environment-build <capture.har> <dir>`
|
||||
inspects, projects and exports in one step and prints the same coverage, or
|
||||
`null` when the bundle serves replay.
|
||||
|
||||
For a PE/ELF export question, `inspect_native_exports` accepts `detail: "summary"`
|
||||
and retains the complete table. Query `inspect_analysis_view` with
|
||||
`view: {kind: "native-exports", collection: "exports", names: ["selected_name"],
|
||||
fields: ["name"], offset: 0, limit: 32}` and the reported parent Evidence ID.
|
||||
All same-name versions and aliases survive; use `source_index` with the parent ID
|
||||
for stable identity. `collection: "issues"` pages interpretation failures. Page
|
||||
coverage describes selected retained rows; `summary.coverage` or `exports.coverage`
|
||||
still describes the source table, so an empty partial page does not prove absence.
|
||||
CLI `inspect-native-exports --detail summary --evidence_output PATH` saves the
|
||||
complete bundle for later `inspect-analysis-view --evidence-bundle PATH` requests.
|
||||
+266
@@ -0,0 +1,266 @@
|
||||
# JavaScript and Electron application artifacts
|
||||
|
||||
For captured or bundled JavaScript that needs readable source recovery, use
|
||||
`recover_javascript_sources` with an absolute `path` and absent
|
||||
`output_directory`. Supply related runtime/shared/lazy files in
|
||||
`additional_paths` when advertised, then analyze the returned `analysis_input`
|
||||
directory together. Each input retains its own snapshot, digest and byte-range
|
||||
provenance; inputs and recovered code are never executed. Recovery requires a
|
||||
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/guides/javascript/javascript-recovery.md).
|
||||
|
||||
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
|
||||
extracted tree. When the question concerns known source paths and the connected
|
||||
schema advertises `source_paths`, select exact inventoried paths such as
|
||||
`["package.json", "main.js", "preload.js"]`. Include supporting manifests or
|
||||
source maps explicitly; dependencies are not implicitly analyzed. REA retains
|
||||
the complete artifact inventory and identity but reads source text and analyzes
|
||||
only selected files. The result and summary record `source_selection`; files
|
||||
outside scope and their relationships remain unknown, so partial application
|
||||
coverage does not mean the selected source failed. Use repeated `--source-path`
|
||||
on either static application CLI route, or `--source-paths-json` with a JSON
|
||||
file for selections too long for one command line. Omit the selection when the
|
||||
question requires complete application analysis; `source_paths: []` is invalid,
|
||||
not an unfiltered request. `detail: "summary"` reduces delivery
|
||||
and does not by itself reduce analysis work.
|
||||
|
||||
ASAR header/index admission defaults to 16 MiB / 16777216 bytes;
|
||||
an admission error names `asar.max_header_bytes` and its bounds (at most
|
||||
256 MiB). Call `update_analysis_settings` with that setting, `action: "set"`
|
||||
and enough bytes, then retry the same full-archive request. No inspection or
|
||||
REA restart is required. `REA_ASAR_MAX_HEADER_BYTES` seeds future REA processes.
|
||||
`source_paths`, `detail`, and `max_heap_mb` do not bypass or raise header admission.
|
||||
Runtime JavaScript timeout/heap defaults are also editable; explicit per-call
|
||||
`analysis_timeout_ms` and `max_heap_mb` take precedence.
|
||||
|
||||
Exact `#alias` specifiers resolve through the importer's
|
||||
enclosing package `imports` map to package-local files; nested `node_modules`
|
||||
packages use their own map. Wildcard, recursive, and external-package import
|
||||
targets stay unresolved with that limitation. When size is unknown or large and the connected schema advertises
|
||||
`detail`, request `detail: "summary"`. This returns an `inspect_analysis_view`
|
||||
Evidence record with a distinct ID and a `normalized_result.parent_evidence_id`
|
||||
for the complete retained application analysis. Source-map contents are part of
|
||||
that static analysis. Read findings, coverage, limitations and unknowns, then
|
||||
follow a specific unanswered question. Older servers without `detail` return
|
||||
the complete application Evidence inline when it fits.
|
||||
|
||||
When the connected schema advertises `max_heap_mb` and `analysis_timeout_ms`,
|
||||
these select the isolated source analyzer's old-space heap and per-source
|
||||
deadline. Defaults are 1024 MiB and 300000 ms. They do not change the MCP
|
||||
process heap or the client's tool deadline. For a host timeout without REA
|
||||
output, use the [deadline recovery guide](connection-and-recovery.md#distinguish-client-and-server-deadlines)
|
||||
instead of raising the server budget. A Bun-hosted REA does not enforce the heap choice; on Windows it
|
||||
instead limits the analyzer's committed memory to that choice plus a 512 MiB
|
||||
runtime reserve.
|
||||
Resource-limited analyses preserve completed static facts and
|
||||
report partial semantic coverage with specific unknown scopes; this does not
|
||||
mean the source is malformed. Inspect those limits before retrying a source.
|
||||
When expanded relationships exceed the parent's available memory, inspect the
|
||||
file's modules item view for retained `static_analysis` and
|
||||
`semantic_module_analysis`. Their facts remain available even when graph links
|
||||
are unknown. Cumulative result exhaustion can stop analysis before every file
|
||||
is examined; the inventory and remaining unknown scopes distinguish that case.
|
||||
Timeout or cancellation after source analysis begins retains partial Evidence
|
||||
in the same session. A returned `error.details.partial_observation` reference
|
||||
can be supplied directly to `inspect_analysis_view`. If the SDK cancels locally, retrieve its recorded ID from
|
||||
the Evidence bundle before inspecting the partial analysis.
|
||||
|
||||
## Inventory route candidates without returning the application graph
|
||||
|
||||
When the advertised schema supports `view`, use `detail: "summary"` with
|
||||
`view: {kind: "javascript-routes", collection: "routes", fields: ["path", "confidence"], limit: 20}`.
|
||||
The same view works with `analyze_web_bundle` on already selected captured sources.
|
||||
The complete source Evidence is retained; use its `parent_evidence_id` with
|
||||
`inspect_analysis_view` for `summary`, `routes`, `occurrences`, `sources`, or
|
||||
`unknowns`. Filter and batch IDs before requesting more fields, and follow
|
||||
`coverage.next_offset`. CLI selections require `--evidence_output` to keep the
|
||||
complete record. Static patterns, inferred deployment bases, heuristic system
|
||||
classification, and unavailable sources never establish route reachability or
|
||||
the complete application route denominator. See the
|
||||
[route inventory guide](https://github.com/morluto/rea/blob/main/docs/javascript-route-inventory.md).
|
||||
|
||||
For captured endpoint observations and inference facts, `analyze_web_bundle`
|
||||
also accepts `view: {kind: "web-bundle", collection: "endpoints", limit: 32}`
|
||||
with `detail: "summary"`. Read later `endpoints`, `inferences`, or `summary`
|
||||
pages from the retained parent. Filter before requesting detail; retain
|
||||
`source_index` together with parent ID and collection, and request `location`
|
||||
or `basis` when tracing sources. Page exhaustion does not mean producer
|
||||
completeness: check `completeness`, structured `source_unknowns`, and limitations.
|
||||
Endpoint candidates remain inferences rather than confirmed network behavior.
|
||||
|
||||
## Read a module without repeating analysis
|
||||
|
||||
Use the actual returned parent ID, on the same connection, as `source.evidence_id`.
|
||||
The following IDs are placeholders to replace with returned values. When the
|
||||
module's node ID is not already known, page identities before selecting an item.
|
||||
Pages return only node IDs, paths and roles; choose a useful page size and follow
|
||||
`coverage.next_offset` when another page matters:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": { "kind": "page", "collection": "modules", "offset": 0, "limit": 16 }
|
||||
}
|
||||
```
|
||||
|
||||
Select the relevant page entry by its exact `node_id` and recorded role:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": {
|
||||
"kind": "item",
|
||||
"collection": "modules",
|
||||
"selector": { "node_id": "jag_node_ACTUAL_NODE_ID" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Module pages include asset, bundled and source module roles, so one path can
|
||||
identify several entries. Use `selector.path` only when uniqueness is established;
|
||||
the returned node ID avoids needing a complete inventory to prove uniqueness. On ambiguity,
|
||||
select the matching node ID from the page or the error's candidates. Follow
|
||||
`next_offset` when another page matters. An item exposes that module node's
|
||||
recorded observations, identity, hashes, exports and source ranges; it does not
|
||||
include every child syntax observation, related edge or original source file.
|
||||
An absent item fact is not evidence of absence from the whole application.
|
||||
|
||||
For canonical identities and retained source-range counts, use a typed reference
|
||||
page instead of reading every full item:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": {
|
||||
"kind": "javascript",
|
||||
"collection": "modules",
|
||||
"projection": "references",
|
||||
"where": { "path": "main.js", "kind": "javascript-module" },
|
||||
"offset": 0,
|
||||
"limit": 16
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Filters match exact retained strings; omit a filter to include all values, use
|
||||
`path: null` for unavailable paths, and `path: ""` for a recorded empty path.
|
||||
Rows preserve canonical `identity` and distinct observation support states in
|
||||
`basis`. `retained_source_range_count` counts retained source-range observations,
|
||||
not runtime imports or unique reference sites. Use a row's `node_id` with the item
|
||||
view above for exact locations and full supporting observations. Follow
|
||||
`coverage.next_offset`; page exhaustion describes matching retained rows only.
|
||||
Check `source_coverage` for source selection, partial/unknown graph coverage and
|
||||
unknown counts before drawing a negative conclusion.
|
||||
|
||||
## Trace only the remaining question
|
||||
|
||||
Use `trace_application_feature` on the parent application Evidence for one
|
||||
literal node ID, route, string, API, IPC channel, module, or native export.
|
||||
Select `incoming` for relationships leading to the seed, `outgoing` for those
|
||||
leaving it, and `both` when the question needs both. Trace traversal includes
|
||||
containment relationships, so even a narrow seed can expand many nodes; a
|
||||
bidirectional trace can reach unrelated sibling modules. A module item or page
|
||||
is preferable when its observations already answer the question. Do not treat
|
||||
all reached nodes as participants in the feature. For a broad trace, pass
|
||||
`view: {"kind": "feature-trace", "collection": "paths"}` (or `summary`) to
|
||||
retain the complete trace and read terminal paths first; page `nodes` or `edges`
|
||||
with exact `where.node_ids` through `inspect_analysis_view` as needed.
|
||||
|
||||
Reuse retained parent Evidence when the server advertises it, or supply complete
|
||||
Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest
|
||||
matches exactly.
|
||||
|
||||
## Recover an oversized display
|
||||
|
||||
An oversized response reports `resource_constraint` and
|
||||
`details.resource: "transport"`. Reuse
|
||||
`details.reported_limits.evidence_reference` with `inspect_analysis_view` for a
|
||||
summary, one module, or a stable page of module identities.
|
||||
`export_evidence_bundle` writes the complete canonical session to a
|
||||
caller-selected file. These workflows keep the original Evidence and coverage;
|
||||
a broad follow-up can also exceed framing.
|
||||
|
||||
A client can truncate a successful result inside its text envelope without a
|
||||
REA `resource_constraint`. Treat a truncation notice or incomplete JSON as a
|
||||
display limitation; do not claim the full record was examined. Reuse the known
|
||||
parent ID for smaller views, or export complete Evidence to a selected file
|
||||
while the session remains open. If the parent ID was lost from the preview,
|
||||
use the prior analysis summary or explicit ledger entries to recover it; do
|
||||
not invent an ID or rerun analysis just to reproduce the same display.
|
||||
|
||||
BrowserWindow preferences, preload and contextBridge surfaces, IPC
|
||||
registrations, utility processes, and native binding requests are static syntax
|
||||
observations. Only a unique exact literal IPC channel match supports an inferred
|
||||
pairing. Dynamic or ambiguous channels remain unresolved. A requested `.node`
|
||||
member is not a verified native export. Never claim runtime reachability,
|
||||
registration, defaults, or policy enforcement from static analysis.
|
||||
|
||||
For version comparison, analyze each version once, then call
|
||||
`compare_application_versions`. Accept only its digest, source-map, structural
|
||||
fingerprint, or non-module semantic matches. Module ordinals and minified names
|
||||
are not persistent identity. Report added or removed only with complete
|
||||
opposite-side coverage; otherwise report unknown.
|
||||
|
||||
When the question asks how one exact exported callable's returned object shape
|
||||
changed, analyze each version once and then call
|
||||
`compare_javascript_export_shapes` with explicit module paths and export names.
|
||||
Use the returned IDs on the same connection, or complete inline Evidence
|
||||
records, from both analysis calls. Accept variant
|
||||
pairing only through the tool's unique exact literal discriminant. Read
|
||||
`property_inventories` and each change's `presence` before treating `unknown` as
|
||||
"the name was not observed": inventories list observed property names even when
|
||||
values stay unknown, with a `source_range` for each paired or unpaired variant.
|
||||
Array holes and mutation-invalidated slots are not observed properties.
|
||||
Complete parent coverage reports presence-only
|
||||
`added`/`removed` independently of unresolved values. Cite the
|
||||
comparison Evidence and report JSON Pointer changes; dynamic values, ambiguous
|
||||
variants, and incomplete parent-property coverage stay unknown. This is static
|
||||
inference, not runtime behavior. When runtime semantics are needed, run
|
||||
behavioral probes against the relevant application versions and capture them
|
||||
through the available browser, Electron, or process workflows.
|
||||
|
||||
## Reusing application Evidence
|
||||
|
||||
When advertised by the connected server, application trace and compare tools
|
||||
accept complete inline Evidence or
|
||||
`{"kind":"retained-evidence","evidence_id":"ev_<64 lowercase hex characters>"}`
|
||||
for their application input (`application`, or `left`/`right`). `inspect_analysis_view`
|
||||
uses the same retained-reference form in `source`, or portable inline Evidence,
|
||||
to project a summary, module page, or one module with its recorded observations.
|
||||
This notation is
|
||||
a template: replace it with the actual returned ID. Versions before 4.1.0 accept only full
|
||||
inline Evidence. Use the exact ID
|
||||
returned by the producer on the same MCP connection. Resolution does not run
|
||||
analysis or select a provider; findings remain inline. For summary or inspection
|
||||
replies, select the complete `normalized_result.parent_evidence_id`. Native
|
||||
observation arrays and runtime reconciliation's `static_layers[].analysis` and
|
||||
`runtime_observations[]` also accept exact references where advertised. Each
|
||||
source can independently be inline or retained, preserving identity, coverage
|
||||
and unknowns. CLI workflows advertising `--evidence-bundle` resolve these inputs
|
||||
from the explicitly selected bundle. `close_binary` releases the engine while
|
||||
retaining references and complete Unknown history on the same MCP connection.
|
||||
`reset_investigation` explicitly clears them; optional `save_path` checkpoints the
|
||||
complete bundle before clearing. Import that bundle in another connection to
|
||||
restore its original references. Retention is bounded, in-memory and never silently
|
||||
evicted; `binary_session.investigation` reports the budget. A missing reference
|
||||
includes its ID and recovery guidance.
|
||||
|
||||
For exact file/native metadata or processing gaps, use
|
||||
`view: {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"], limit: 32}`.
|
||||
Rows retain canonical node/observation identity and original Evidence. Paths
|
||||
match Evidence artifact-path/source-range locations; missing properties remain
|
||||
omitted and source coverage stays separate from page exhaustion. Batch paths and
|
||||
select only needed property keys; complete Evidence remains accessible by export.
|
||||
If names are unknown, opt into `include_available_property_keys: true`; it lists
|
||||
the original keys per returned observation without unselected values. Keep each
|
||||
row's node/observation identity paired when composing facts.
|
||||
These rows select application-graph observations; semantic relationships and
|
||||
semantic unknown records remain in the complete parent Evidence and trace tools.
|
||||
@@ -0,0 +1,934 @@
|
||||
# Native, managed, and packaged artifacts
|
||||
|
||||
For release comparisons on one MCP connection, retain `list_procedures` Evidence
|
||||
from each target and reuse its exact IDs after target switches or `close_binary`.
|
||||
For a handoff or new process, export the complete bundle and import it there,
|
||||
including its source closure. `reset_investigation` explicitly clears retained
|
||||
records; optional `save_path` checkpoints them before clearing.
|
||||
`match_native_functions` pairs unique exact names and, when supplied,
|
||||
complete observed body bytes. Reuse optional same-context `list_names`,
|
||||
`procedure_info`, `read_bytes` and existing `analyze_function` dossiers as
|
||||
`left_observations`/`right_observations`; an MCP `analyze_function` summary
|
||||
reply contributes its `parent_evidence_id` as a `retained-evidence` reference,
|
||||
not the summary itself. Avoid decompiling every function merely to match it. Its summary retains a parent for
|
||||
`inspect_analysis_view` `function-matches` collections `matches` and `functions`.
|
||||
Select columns, batch IDs and page individual facts; aliases remain unresolved.
|
||||
Unpaired observations do not prove additions/removals. Existing matched dossier
|
||||
pairs include inputs for `compare_functions`; otherwise analyze just chosen pairs.
|
||||
|
||||
Aggregate completed comparisons with `find_changed_behavior`, selecting each
|
||||
comparison's exact `retained-evidence` ID instead of resending full records.
|
||||
Inline comparison Evidence remains accepted. References belong to the current
|
||||
server: after handoff, import the original bundle including its source closure.
|
||||
Do not substitute dossier IDs or summary views. Static differences remain
|
||||
behavior candidates, with runtime behavior unknown unless separately observed.
|
||||
|
||||
For a matching representation refusal, inspect `details.limit` and raise
|
||||
`native.matching_representation_bytes` with `update_analysis_settings` before
|
||||
retrying the unchanged MCP request. CLI launch seed:
|
||||
`REA_NATIVE_MATCHING_REPRESENTATION_BYTES`. The default is 96 MiB; the policy
|
||||
range is 1 byte to 1 GiB and does not bound process RSS. Keep the complete
|
||||
inventories: splitting them changes global name/body uniqueness.
|
||||
|
||||
## Apple app resources
|
||||
|
||||
Use `open_bundle_resources` with the selected app path, then `inspect_plist`,
|
||||
`decode_interface_builder`, `inspect_asset_catalog`, or `inspect_keyed_archive`.
|
||||
The resource session reads the app without resolving `CFBundleExecutable` or
|
||||
starting a deep provider. Preserve `resource_root` provenance, per-resource
|
||||
hashes where reported, and null executable identity; a directory path is not
|
||||
an executable digest. Export retained Evidence before closing or switching
|
||||
roots. Binary analysis snapshots do not apply. Use `open_binary` to switch to
|
||||
executable analysis, and `close_binary` to release either session kind.
|
||||
|
||||
The four matching CLI commands accept the app path directly. For an archive
|
||||
inside the app, use `inspect-keyed-archive APP_PATH RELATIVE_ARCHIVE_PATH`.
|
||||
Host requirements for plist and compiled asset/Interface Builder tooling still
|
||||
apply; a resource-only session does not establish provider availability.
|
||||
|
||||
## Desktop installation directories
|
||||
|
||||
When an installation directory is supplied and its runtime is unknown, use
|
||||
`inspect_desktop_application` with its absolute path. Default MCP summary
|
||||
delivery retains complete Evidence. Read `files`, `omissions`, `candidates`, `components` or
|
||||
`next_probes` with `inspect_analysis_view` kind `desktop-application`, selecting
|
||||
exact paths, batched file/omission IDs or families and only needed fields and pages.
|
||||
Summary `probe_issues` retains counts and bounded previews. Select files with
|
||||
`has_issues: true` and fields `path`, `read_status`, `issues` to retrieve exact
|
||||
failures directly; preview text is display data, while `file_id` selects full facts. Preserve source
|
||||
coverage, read-range identity and unknowns; directory snapshot identity is not
|
||||
a digest of every file's contents. Component path markers do not establish
|
||||
runtime presence. For compact CLI output, use
|
||||
`inspect-desktop-application PATH --detail summary --evidence_output BUNDLE`.
|
||||
See [desktop installations](https://github.com/morluto/rea/blob/main/docs/desktop-applications.md).
|
||||
|
||||
## Go build metadata
|
||||
|
||||
Use `inspect_go_binary` for an explicit local ELF, PE or Mach-O (universal
|
||||
files need `architecture`) when the
|
||||
question concerns the embedded compiler version, module/dependency versions,
|
||||
local replacements, build settings, or static Go string literals packed without
|
||||
NUL terminators. Pass `include_string_literals: true`
|
||||
(`--include-string-literals`, optional `max_string_literals` /
|
||||
`--max-string-literals`) to recover bounded inferred word-aligned `(ptr, len)`
|
||||
UTF-8 string slices from read-only data and page them through `go`
|
||||
`string_literals` views. No `open_binary`, Go installation or
|
||||
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
|
||||
Preserve artifact SHA-256, original file ranges, framed module bytes and every
|
||||
unparsed module-text line. A missing record is an unknown, not proof of a
|
||||
non-Go binary; observed metadata does not authenticate its claims, and static
|
||||
`(ptr, len)` string recovery excludes compiler-inlined stack/immediate strings
|
||||
without proving runtime reachability. Function,
|
||||
source and type recovery remain a separate native-analysis question.
|
||||
|
||||
## Model files
|
||||
|
||||
Use `inspect_model_artifact` (`inspect-model-artifact PATH`) for local GGUF v2/v3
|
||||
and safetensors declarations without a model runtime. Header-only mode does not
|
||||
read weights or establish full-file identity. `hash: "full"` explicitly streams
|
||||
all bytes. Retained `model-artifact` views page `metadata`, `tensors` and derived
|
||||
`types`, with exact batch filters and selected fields. Arrays retain typed counts
|
||||
and eight-element previews; omitted values, unknown types and runtime behavior
|
||||
stay explicit. See [model files](https://github.com/morluto/rea/blob/main/docs/model-artifacts.md).
|
||||
|
||||
## IL2CPP metadata
|
||||
|
||||
Use `inspect_il2cpp_metadata` for an explicit local Unity `global-metadata.dat`
|
||||
(versions 24, 27, 29, 31 and 39) when the question concerns assembly images,
|
||||
names, member owners, string literals, namespaces or index ranges. No `open_binary`, Unity installation or target
|
||||
execution is required. The CLI equivalent is `inspect-il2cpp-metadata PATH`.
|
||||
MCP defaults to counts with complete retained Evidence. Query `sections`,
|
||||
`images`, `types`, `methods`, `fields`, `properties`, `events`, `parameters` or
|
||||
`string_literals` through `inspect_analysis_view`; use exact names,
|
||||
batch indices and selected fields. Row indices are scoped by artifact SHA-256
|
||||
and collection; page offsets count filtered matches. Parameters support batch
|
||||
`method_indices` to read one method's parameters directly. Historical reports
|
||||
without member collections retain unknown counts rather than empty inventories. Preserve
|
||||
artifact SHA-256 and index widths. Unrecognized magic may mean encryption and
|
||||
is unsupported, not decrypted. Linking names to `GameAssembly` or
|
||||
`UnityFramework` code addresses remains a separate native-analysis question.
|
||||
See [IL2CPP metadata](https://github.com/morluto/rea/blob/main/docs/il2cpp-metadata.md).
|
||||
|
||||
## Rust dependency metadata
|
||||
|
||||
Use `inspect_rust_binary` for an explicit local ELF, PE, or Mach-O (universal
|
||||
files need `architecture`) when
|
||||
the question concerns the cargo-auditable crate list embedded in `.dep-v0`
|
||||
(Mach-O: `__DATA,.dep-v0`). No `open_binary`, Rust installation, or target
|
||||
execution is required. The CLI equivalent is `inspect-rust-binary PATH`.
|
||||
Preserve artifact SHA-256, package order, dependency indices, and the section
|
||||
file range. A missing section is null metadata, not proof the binary has no
|
||||
dependencies. An omitted package kind stays null. A rustc commit hash is
|
||||
inferred from a read-only `/rustc/<40-hex>/` path and does not by itself
|
||||
identify the compiler. See [Rust binaries](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
|
||||
|
||||
## PE Authenticode certificate tables
|
||||
|
||||
Use `inspect_pe_certificates` for an explicit local PE32/PE32+ image when the
|
||||
question concerns the Authenticode certificate table, WIN_CERTIFICATE types, or
|
||||
embedded PKCS#7 signer and X.509 identity fields. No `open_binary`, Windows SDK,
|
||||
or target execution is required. The CLI equivalent is
|
||||
`inspect-pe-certificates PATH`. Preserve artifact SHA-256, table file ranges,
|
||||
payload hashes, original validity-time spellings, and
|
||||
`subject_key_identifier_hex` when the signer uses that form. A missing
|
||||
`IMAGE_DIRECTORY_ENTRY_SECURITY` directory is observed absence of that table,
|
||||
not proof the image is unsigned in every catalog or overlay sense. Signatures,
|
||||
chains, timestamps, and Authenticode digest coverage remain unobserved. See
|
||||
[PE Authenticode certificate tables](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
|
||||
## PE import and delay-load tables
|
||||
|
||||
Use `inspect_pe_imports` for an explicit local PE32/PE32+ image when the
|
||||
question concerns imported DLL and function names, ordinals, delay-load
|
||||
descriptors, or the pefile import hash. No `open_binary`, Windows SDK, or
|
||||
target execution is required. The CLI equivalent is `inspect-pe-imports PATH`.
|
||||
Preserve artifact SHA-256, directory file ranges, original DLL spellings, and
|
||||
ILT/IAT RVAs. A missing `IMAGE_DIRECTORY_ENTRY_IMPORT` directory is observed
|
||||
absence of that table, not proof the image never imported APIs through another
|
||||
table or overlay. Bound snapshots, loaded IAT contents, and export-forward
|
||||
resolution remain unobserved. See
|
||||
[PE import and delay-load tables](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
|
||||
|
||||
## Windows Shell Link shortcuts
|
||||
|
||||
Use `inspect_windows_shortcut` for an explicit local `.lnk` file when the
|
||||
question concerns the recorded target path, arguments, working directory, icon
|
||||
location, volume serial, network share names, ExtraData blocks, or Tracker
|
||||
droid identifiers. No `open_binary`, Windows SDK, COM, or target execution is
|
||||
required. The CLI equivalent is `inspect-windows-shortcut PATH`. Preserve
|
||||
artifact SHA-256, FILETIME raw decimals, IDList type bytes, and ExtraData
|
||||
signatures. Unknown IDList items and ExtraData signatures remain raw with
|
||||
SHA-256; they are not guessed. Version-1 UUID MAC and timestamp values are
|
||||
labelled derived. A missing optional section is observed absence of that
|
||||
structure. See
|
||||
[Windows Shell Link shortcuts](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
|
||||
|
||||
## PE TLS callbacks and load configuration
|
||||
|
||||
Use `inspect_pe_load_config` for an explicit local PE32/PE32+ image when the
|
||||
question concerns code that runs before the entry point through TLS callbacks,
|
||||
or the security cookie, SafeSEH, CFG, EH-continuation and XFG declarations in
|
||||
the load configuration. No `open_binary`, Windows SDK, or target execution is
|
||||
required. The CLI equivalent is `inspect-pe-load-config PATH`. Preserve artifact
|
||||
SHA-256, directory and callback-array file ranges, and callback VAs as
|
||||
hexadecimal strings. Fields outside the declared `Size` are `null`. Guard-flag
|
||||
mitigations carry `evidence_kind: "inferred"`; enforcement remains unobserved.
|
||||
See
|
||||
[PE TLS callbacks and load configuration](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
|
||||
|
||||
## Binary hardening
|
||||
|
||||
Use `inspect_binary_hardening` for an explicit local ELF, PE, or Mach-O file
|
||||
when the question concerns PIE or ASLR, NX, RELRO, stack-protector or FORTIFY
|
||||
signs, control-flow flags, or stripped state. No `open_binary`, pwntools, or
|
||||
target execution is required. The CLI equivalent is
|
||||
`inspect-binary-hardening PATH`. Universal Mach-O files return one image per
|
||||
slice. A missing import is `unknown`, not proof the protection was omitted.
|
||||
Authenticode and Mach-O code-signature rows report presence, not validity.
|
||||
`hardened_runtime` stays unknown and names `inspect_signature`. See
|
||||
[binary hardening](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
|
||||
|
||||
## ELF note tables
|
||||
|
||||
Use `inspect_elf_notes` for an explicit local ELF32/ELF64 image when the
|
||||
question concerns PT_NOTE/SHT_NOTE owners, GNU build-id, or GNU ABI-tag
|
||||
fields. No `open_binary` or target execution is required. The CLI equivalent
|
||||
is `inspect-elf-notes PATH`. Preserve artifact SHA-256, table file ranges,
|
||||
descriptor hashes, and original owner spellings. A missing table is observed
|
||||
absence of those declarations, not proof the image never carried notes in
|
||||
another container. See
|
||||
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
|
||||
## ELF and Mach-O initializer functions
|
||||
|
||||
Use `inspect_native_initializers` for an explicit local ELF or Mach-O image
|
||||
when the question is which functions run before `main` or at exit: ELF
|
||||
`DT_PREINIT_ARRAY`, `DT_INIT`, `DT_INIT_ARRAY`, `DT_FINI`, `DT_FINI_ARRAY` and
|
||||
legacy `.ctors`/`.dtors`, or Mach-O `__mod_init_func`, `__init_offsets` and
|
||||
`__mod_term_func`. Pass `architecture` for a universal Mach-O. The CLI
|
||||
equivalent is `inspect-native-initializers PATH`. PIE slots are resolved
|
||||
through RELATIVE, RELR and section relocations, and Mach-O slots through
|
||||
chained fixups. Keep each entry's `resolution`: `unresolved` means the value was
|
||||
not established, and `unresolved_import` names an external symbol. Page with
|
||||
`offset` and `limit`. See
|
||||
[ELF and Mach-O initializer functions](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
|
||||
|
||||
## ELF unwind function ranges
|
||||
|
||||
Use `inspect_elf_unwind_ranges` for an explicit local ELF32/ELF64 image when
|
||||
the question concerns function starts and lengths that survive `strip`, which
|
||||
functions have an LSDA, or which ranges have no exact symbol. No
|
||||
`open_binary` or target execution is required. The CLI equivalent is
|
||||
`inspect-elf-unwind-ranges PATH`. Preserve artifact SHA-256, FDE start and
|
||||
end, containing section, exact symbol name, CIE index, LSDA, and personality.
|
||||
Page later ranges with `offset` and `limit`. A missing `.eh_frame` is
|
||||
observed absence of that declaration and of `PT_GNU_EH_FRAME`, not proof the
|
||||
image has no functions. Header disagreement is a limitation and does not drop
|
||||
decoded ranges. `.debug_frame` is reported present or absent and is not
|
||||
decoded. CFA instructions are counted and not evaluated. Indirect EH pointers
|
||||
are not dereferenced. See
|
||||
[ELF unwind function ranges](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
|
||||
|
||||
## Apple IPS crash reports
|
||||
|
||||
Use `inspect_apple_ips` for an explicit local Apple `.ips` file when the
|
||||
question concerns crash metadata, exception and termination facts, threads,
|
||||
usedImages, lossless registers, or documented crash classes. No `open_binary`,
|
||||
Xcode, atos, or target execution is required. The CLI equivalent is
|
||||
`inspect-apple-ips PATH`. Preserve artifact SHA-256, `bug_type` as a string,
|
||||
uppercase image UUIDs, and integer literals that exceed IEEE-754 range. Only
|
||||
`bug_type` `"309"` yields a crash body; other DiagnosticReports types return
|
||||
no crash result. Select `join_path` for one local Mach-O or dSYM to match
|
||||
report images to slices by UUID and architecture. Preserve the selected
|
||||
artifact identity, slice facts and unjoined reasons; a joined static address
|
||||
does not resolve function names or dSYM symbols. See
|
||||
[Apple IPS crash reports](https://github.com/morluto/rea/blob/main/docs/apple-ips.md).
|
||||
|
||||
## Linux kernel modules
|
||||
|
||||
Use `inspect_kernel_module` for an explicit local ELF32/ELF64 relocatable
|
||||
object when the question concerns `.modinfo` license, vermagic, alias,
|
||||
depends, or parm strings, the `__versions` symbol CRCs, or an appended module
|
||||
signature trailer. No `open_binary`, Linux host, or `modinfo` program is
|
||||
required. The CLI equivalent is `inspect-kernel-module PATH`. Preserve
|
||||
artifact SHA-256, key order, duplicate keys, CRC hex, and section file ranges.
|
||||
A missing section is observed absence when section names are known, not proof
|
||||
the module never carried that metadata. `.gnu.linkonce.this_module` is reported
|
||||
as present or absent; `struct module` stays unknown because its layout changes
|
||||
with the kernel version. `__ksymtab` records are not decoded. The PKCS#7 range
|
||||
and its SHA-256 are observed; certificate, signer, and digest coverage remain
|
||||
unvalidated. An absent trailer is not proof the module is unsigned in another
|
||||
container. gzip, xz, and zstd whole-file containers are unsupported and name
|
||||
`external_decompression` as the next probe: use an external codec utility to
|
||||
decompress into a separate file, then call `inspect_kernel_module` on that file. See
|
||||
[Linux kernel modules](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
|
||||
|
||||
## Linux kernel build config
|
||||
|
||||
Use `inspect_kernel_build_config` for an explicit local kernel image (`vmlinux`
|
||||
ELF, raw arm64 `Image`, x86 `bzImage`/`zImage`, or `configs.ko`) when the
|
||||
question is which kernel version it is or how it was configured. No
|
||||
`open_binary`, Linux host, or `extract-ikconfig` is required. The CLI
|
||||
equivalent is `inspect-kernel-build-config PATH`; pass `name` for one exact
|
||||
`CONFIG_*` option and `offset`/`limit` to page the rest. Preserve the banner
|
||||
version, builder, and compiler, the config SHA-256 and option count, and each
|
||||
entry's value kind (`y`, `m`, `string`, `number`, `not_set`). A config is
|
||||
accepted only when `IKCFG_ST` is immediately followed by a valid gzip member
|
||||
and `IKCFG_ED`, so an ordinary `IKCFG_ST` string is not a config. Offsets with
|
||||
`space: "inflated_payload"` are positions in an inflated bzImage payload, not
|
||||
file offsets. An absent config describes only the searched bytes; the kernel
|
||||
may have been built without `CONFIG_IKCONFIG` or with it as a module. Truncated
|
||||
payloads are incomplete, not absent. A framed blob that does not decode, has a
|
||||
bad CRC, or holds non-config text is `malformed`, not absent. xz, lz4, lzma, zstd, bzip2, and lzo
|
||||
payloads are unsupported and name `external_decompression`: decompress with an
|
||||
external utility, then call `inspect_kernel_build_config` on the result. See
|
||||
[Linux kernel build config](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
|
||||
|
||||
## CPython pyc files
|
||||
|
||||
Use `inspect_python_pyc` for an explicit local CPython 3.7–3.13 `.pyc` when the
|
||||
question concerns the header magic, PEP 552 source hash or timestamp, nested
|
||||
code-object names, or bytecode identity. No `open_binary`, Python installation,
|
||||
or target execution is required. The CLI equivalent is `inspect-python-pyc PATH`.
|
||||
Preserve artifact SHA-256, the 16-byte header fields, nested code names, and
|
||||
bytecode SHA-256. Bytecode is not decompiled. A decoded header does not prove
|
||||
the matching `.py` is present. See [CPython pyc inspection](https://github.com/morluto/rea/blob/main/docs/python-pyc.md).
|
||||
|
||||
## Erlang BEAM files
|
||||
|
||||
Use `inspect_erlang_beam` for an explicit local `FOR1`/`BEAM` file when the
|
||||
question concerns IFF chunks, atom names, exports, imports, lambdas, or the
|
||||
Code header. No `open_binary`, Erlang installation, or bytecode execution is
|
||||
required. The CLI equivalent is `inspect-erlang-beam PATH`. Preserve artifact
|
||||
SHA-256, chunk file ranges, 1-based atom indices, and original atom spellings.
|
||||
Missing optional tables stay null. For a few facts, use `detail: "summary"` with
|
||||
a `beam` view selecting a collection, exact names/indices and fields; read later
|
||||
pages from `parent_evidence_id` or the complete Evidence bundle. Same-name
|
||||
exports retain every arity and original atom index. CLI summary delivery requires
|
||||
`--evidence_output`. Null table counts remain distinct from decoded empty tables. See
|
||||
[Erlang BEAM files](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
|
||||
|
||||
## SQLite snapshots
|
||||
|
||||
Use `inspect_sqlite_database` with an explicit local database snapshot for its
|
||||
schema and optional rows from one exact ordinary table. It requires Node.js
|
||||
24.x >=24.15 or 26+ with the native `DatabaseSync.limits` API. It also consumes the
|
||||
existing sibling `-wal`; retain both original digests when citing its result.
|
||||
Default output is a compact inventory; a selected table expands only its own
|
||||
schema and samples 20 rows. Select `columns` to avoid irrelevant or large cells,
|
||||
and request `schema_scope: "database"` only for complete schema details.
|
||||
Choose `row_limit` when a finite sample is useful and keep truncation and
|
||||
unspecified row order explicit. Virtual tables, shadow tables and views are
|
||||
schema facts, not row-read targets. REA inspects private copies and leaves
|
||||
the source files untouched; obtain a coherent quiescent database/WAL snapshot
|
||||
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
|
||||
|
||||
## Recorded Linux crashes
|
||||
|
||||
Use `inspect_recorded_crash` with an explicit Linux x86-64 ELF core path to
|
||||
inspect historical thread registers, signals, raw notes and original file
|
||||
ranges. It is independent of the active disassembler target. Source-bound
|
||||
register values and note bytes are observations; missing notes, signal-thread
|
||||
association and current executable/library identity remain unknown. PIDs are
|
||||
historical metadata and never authorize live attach or control.
|
||||
|
||||
The optional `include_debugger_context` facet adds mapping candidates through
|
||||
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
|
||||
establish current file identity; zero reported flags leave permissions unknown.
|
||||
Requested unavailable context fails with setup guidance. See
|
||||
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md)
|
||||
for exact upstream profiles, bounds and verification coverage.
|
||||
|
||||
## Offline ELF layout
|
||||
|
||||
Use `inspect_binary_layout` with an explicit local ELF path for file-backed
|
||||
sections, symbols, relocations, linkage names, and static mitigation inferences
|
||||
without opening a disassembler target. When that complete record is already
|
||||
retained, call `inspect_analysis_view` for a summary, mitigations or linkage
|
||||
facet, one section or symbol, or a stable page. Do not re-run the decoder to
|
||||
read a single object.
|
||||
|
||||
## Offline WASM
|
||||
|
||||
Use `inspect_wasm_artifact` with caller-supplied WABT for selected local bytes.
|
||||
For counts, validation and WAT identity, select `detail: "summary"` on the first
|
||||
MCP call. The complete original Evidence stays retained; use
|
||||
`normalized_result.parent_evidence_id` for exact exports or text pages. Omitted
|
||||
or complete detail keeps the existing complete result.
|
||||
For a completed result, use `inspect_analysis_view` with its reported retained
|
||||
reference for a summary or `view: {kind: "wasm", facet: "exports", offset: 0, limit: 32}`.
|
||||
Facets also include sections, imports, candidates, glue, WAT, headers, details
|
||||
and tool profile. Text offsets count UTF-16 code units; follow `next_offset`.
|
||||
The complete source remains retained after an oversized MCP reply. No target
|
||||
execution or network acquisition is implied, and WAT is decoded representation.
|
||||
|
||||
## Offline APK signatures
|
||||
|
||||
Use `inspect_apk_signature` for a selected local APK/ZIP/AAB. Default MCP summaries
|
||||
retain complete Evidence. Select `view.collection: signers/digests/lineage`,
|
||||
with scheme or ID batches, fields and paging; use `inspect_analysis_view` kind
|
||||
`apk-signature` for retained reads after deletion. IDs retain artifact/scheme
|
||||
scope and signer links; `fields: []` keeps identity links. Empty pages preserve
|
||||
partial decoding reasons and unknown authenticity. CLI compact delivery needs
|
||||
`--evidence_output` to preserve the complete parent.
|
||||
It reads the APK
|
||||
Signing Block (v2/v3/v3.1) and v1 `META-INF` signing without executing anything,
|
||||
and reports each signer's certificate facts (for v1, the certificate the PKCS#7
|
||||
block's SignerInfo names), the declared v3 rotation lineage,
|
||||
the declared source-stamp certificate and, for SHA-256/SHA-512 digests, whether
|
||||
the declared content digest matches the current bytes. That match is not a
|
||||
signature verification: anyone who changes the bytes can recompute the digest.
|
||||
`posture` summarises v1-only signing (which cannot tell a v1-only APK from one
|
||||
whose v2/v3 block was removed), observed signer agreement across schemes,
|
||||
whether the v1 signer equals the block-scheme signer (`false` is notable),
|
||||
whether every differing signer appears in a declared (unverified) rotation
|
||||
lineage, and the strongest scheme.
|
||||
It never claims authenticity: no signature or lineage is cryptographically
|
||||
verified.
|
||||
|
||||
## Native targets
|
||||
|
||||
After `open_binary`, use focused search, procedure, or function tools directly.
|
||||
Use `binary_overview` when target metadata or inventory context helps answer the
|
||||
question. Prefer literal search, names, decompilation, callers, callees, and
|
||||
cross-references. Addresses and recovered pseudocode are analysis observations,
|
||||
not original source. Provider unavailability and unsupported metadata remain
|
||||
unknown rather than false.
|
||||
|
||||
For large pseudocode, request `procedure_pseudo_code` with `detail: "summary"`
|
||||
on the first MCP call. It retains the complete observation and returns the
|
||||
original selection, text availability and length without the body. Use
|
||||
`normalized_result.parent_evidence_id` with `inspect_analysis_view` and
|
||||
`view: {kind: "native", facet: "pseudocode", offset: 0, limit: 20000}`;
|
||||
follow `coverage.next_offset` for later pages. Choose a smaller limit if one
|
||||
page exceeds the client or transport budget. Offsets and limits count UTF-16
|
||||
code units, not lines or tokens. Pages reuse the same observation without
|
||||
repeating decompilation. Null and empty text remain distinct. Omitted or
|
||||
`complete` detail preserves full inline delivery. Summary delivery requires a
|
||||
retaining session and does not reduce the work or memory of decompilation.
|
||||
|
||||
For a completed `analyze_function` dossier, use `inspect_analysis_view` with
|
||||
`view: {kind: "native", facet: "value_flow_operations", offset: 0, limit: 32}`
|
||||
and the exact retained Evidence ID to read selected facts without reanalysis.
|
||||
The default MCP reply is already a summary of that dossier: its
|
||||
`normalized_result.parent_evidence_id` is the retained ID to page, and its
|
||||
null counts mark facets the provider did not record or declared unavailable.
|
||||
Native facets also include procedure, pseudocode, assembly, basic blocks,
|
||||
comments, callers/callees, references, unresolved calls, referenced strings and
|
||||
names, the native API record, and value-flow summaries. Pseudocode offsets and limits count UTF-16
|
||||
code units; use the returned `next_offset` for continuation. Provider limitations
|
||||
and unavailable facts remain explicit; the complete parent Evidence is retained.
|
||||
|
||||
`inspect_macho` on a large framework returns megabytes of imports and exports.
|
||||
Select `detail: "summary"` for slice identity, UUID and per-collection counts,
|
||||
then page rows with `view: {kind: "macho", facet: "exports", offset: 0, limit: 64}`
|
||||
and `normalized_result.parent_evidence_id`. Facets are entrypoints,
|
||||
architectures, build_metadata, load_commands, dependencies, imports, exports,
|
||||
segments and provenance. `source_exhaustive: false` means unreturned symbols
|
||||
may exist.
|
||||
|
||||
To see which functions or Objective-C methods a Mach-O actually calls in one
|
||||
run, use `observe_native_calls` with explicit breakpoints and a bounded
|
||||
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
|
||||
runs and may change files or use the network. Hardened-runtime targets need the
|
||||
`get-task-allow` entitlement.
|
||||
Function breakpoints use exact LLDB runtime names: pass an `inspect_macho` export
|
||||
or import `runtime_name` (`main`), not its symbol-table `name` (`_main`). REA does
|
||||
not translate breakpoint names. Set `module` to the image
|
||||
file name when selecting that image only; otherwise loaded system images also
|
||||
match. Inspect retained locations and module paths before attributing a call.
|
||||
|
||||
Use `binary_session` with no arguments to check the open target, selected
|
||||
provider, and alignment. Its default summary `result.tool_availability` carries
|
||||
the complete tool inventory grouped by current status. Expand each group's
|
||||
`names` and inherit the inventory-level client requirements unless a group
|
||||
overrides them; availability, reasons, remediation, and modes remain inline.
|
||||
Provider candidates keep availability and target-support status, code, and
|
||||
reason. `omitted_fields` names what the summary leaves out; `{"detail":
|
||||
"complete"}` adds each unavailable candidate's `remediation`, provider
|
||||
diagnostics, candidate capability sets, and tool surface, `effects`, and
|
||||
`annotations`. Those arrays name true flags from fixed
|
||||
vocabularies, so an omitted known flag is false. `tools/list` remains the
|
||||
canonical source for full tool schemas when target or provider state changes.
|
||||
|
||||
Provider capability groups carry provider identity and ordered limitations.
|
||||
Prepend each set's `shared_limitations` to a group's `limitations` to recover the
|
||||
operation's full ordered list. Candidate capabilities, returned with complete
|
||||
detail, describe possible providers. Top-level session capabilities describe the current route, and
|
||||
`analysis_provider_binding` identifies the selected provider.
|
||||
|
||||
## Continue editable Ghidra analysis across processes
|
||||
|
||||
For a task spanning agent sessions, check `binary_session` after selecting
|
||||
Ghidra: the current provider's `capabilities[].shared_limitations` states whether
|
||||
Program checkpoints are enabled or the session is ephemeral. Successful native
|
||||
queries distinguish initial analysis from an actually restored packed Program;
|
||||
do not infer that restoration happened merely because checkpoint mode is enabled.
|
||||
|
||||
In ephemeral mode, choose a private absolute directory and set
|
||||
`REA_GHIDRA_CHECKPOINT_DIR` in the environment starting REA. Restart the MCP server
|
||||
after changing that launch configuration; no current tool call or runtime
|
||||
analysis-setting update enables it. Stock ELF/PE/Mach-O imports on Linux/macOS
|
||||
are supported; see the [checkpoint setup and limitations](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program).
|
||||
|
||||
With the setting enabled, normal opens restore compatible saved Programs
|
||||
automatically. Initial completed analysis is saved, and `close_binary` saves
|
||||
later metadata edits; no separate save/restore tool is needed. Repair a reported
|
||||
save failure and retry close. Cancellation or process failure can lose edits
|
||||
after the last successful save; unfinished initial analysis cannot resume.
|
||||
Keep the same checkpoint configuration for later processes. Evidence/Unknown
|
||||
history and immutable REA query snapshots have separate persistence workflows.
|
||||
|
||||
## Reuse caller-selected Ghidra analysis
|
||||
|
||||
When the caller supplies an expanded project, configure `REA_GHIDRA_PROJECT_PATH`
|
||||
(the `.gpr` marker) and optionally `REA_GHIDRA_PROGRAM_NAME` (one root program),
|
||||
then restart/reconnect before `open_binary`. CLI uses the same configuration.
|
||||
REA opens only a fingerprint-bound owned copy with `-process -noanalysis -readOnly`;
|
||||
original project files are not opened by the engine or saved. Database edits,
|
||||
import overrides and repair scripts are unavailable in this mode. Treat original
|
||||
analysis completeness/settings as unknown and prior edits as caller-project state.
|
||||
The program's recorded import digest must match the target; changed project content
|
||||
requires reopening for a fresh profile. See the
|
||||
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#reuse-an-existing-ghidra-project).
|
||||
|
||||
## Ghidra startup deadline recovery
|
||||
|
||||
`provider_timeout` during the first Ghidra query can cover import, auto-analysis,
|
||||
bridge startup, and readiness. The default 330,000 ms is configurable through
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS`; a smaller query still needs the same cold import.
|
||||
|
||||
When the error names `ghidra.startup_timeout_ms`, call
|
||||
`update_analysis_settings` with `action: "set"` and a larger value within the
|
||||
reported bounds. Resolve retained cleanup before the next startup attempt.
|
||||
No REA restart is needed; an already accepted startup keeps its original
|
||||
budget. Follow the error's session-specific restart guidance, and preserve useful Evidence before closing
|
||||
and reopening the target when remediation requires that lifecycle step.
|
||||
|
||||
For a per-function timeout, update `ghidra.decompile_timeout_seconds` and retry
|
||||
the same function on the existing Program. Explicit `null` removes that
|
||||
per-function deadline. Inspection is optional when the error supplies the
|
||||
setting and bounds. Runtime updates disappear when REA exits; environment
|
||||
changes only seed future REA processes.
|
||||
|
||||
The client's tool-call deadline must allow startup plus the requested query;
|
||||
server settings and progress notifications cannot override it. CLI analysis
|
||||
avoids the MCP client deadline, but retains the provider deadline. For example:
|
||||
|
||||
```sh
|
||||
REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000 rea analyze /absolute/path/to/app.exe --provider ghidra --json
|
||||
```
|
||||
|
||||
Thirty minutes is an example budget, not a completion guarantee. Keep host CPU
|
||||
and memory controls in place. Disable specific expensive analyzers only when
|
||||
their missing coverage is acceptable and retain the resulting limitations.
|
||||
Exported C and embedded resources remain useful independent evidence, but a
|
||||
timeout does not establish complete analysis or absence of code. See the
|
||||
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
|
||||
## Bridge request ceiling
|
||||
|
||||
Hopper and Cutter share `REA_BRIDGE_MAX_REQUEST_BYTES`. Unset, both sides use
|
||||
16777216 bytes. The payload budget excludes the terminating LF. A decimal
|
||||
integer from 7464 to 536870887 raises or lowers that ceiling. The floor fits
|
||||
the largest bounded symbol-name command. A tight Cutter status still returns
|
||||
its session id and up to 64 completion receipts. Its path is included when it
|
||||
fits, and is unknown when it does not, so an older descriptor path is not
|
||||
paired with the new generation. The maximum is the encoded payload, one byte under Node's string ceiling so
|
||||
the framing LF still fits. Cutter measures, signs, and writes a fitting
|
||||
response in bounded chunks. The bridge searches only the newly read bytes
|
||||
for the framing LF. The client unescapes each socket chunk into one response
|
||||
string before parsing it, so this ceiling does not keep the paged wire frame
|
||||
and another concatenated copy. Invalid values are configuration errors and do not fall back to 16777216. Set the variable in the environment that starts REA, restart REA,
|
||||
and start Hopper or Cutter from that same environment. An out-of-band bridge
|
||||
without the setting stays at 16777216 bytes. The client reports that mismatch
|
||||
during discovery only after the hello proves a live authenticated bridge, and
|
||||
does not send a status request or the command. A stale descriptor is ignored.
|
||||
Discovery fails if any such live ceiling differs, even when another live
|
||||
session matches. A command response discarded for that ceiling stays an
|
||||
unknown execution with `output_truncated`. A hello
|
||||
that never sends its authenticated frame does not hold the accept loop.
|
||||
At most 8 connection threads run; a further accept is closed, and a failure
|
||||
to start a thread does not stop the listener. `completed_connection` is the
|
||||
longest prefix of connections that have terminated. Dense scalar replies are
|
||||
encoded in bounded batches.
|
||||
The 25000 ms Cutter request deadline and the 256-candidate discovery budget
|
||||
still have no runtime override.
|
||||
|
||||
## Selected native facts and call graphs
|
||||
|
||||
For a large `get_call_graph`, or repeated `procedure_address`,
|
||||
`procedure_callees`, `procedure_callers` and `resolve_containing_procedure`
|
||||
queries, select `detail: "summary"` when the first 64 recorded observations
|
||||
suffice. Read further `native-observation` pages from the returned parent Evidence
|
||||
ID. Graph message IDs index that page's exact-text dictionary; keep level,
|
||||
source index, failed nodes, and partial/unknown relationship coverage.
|
||||
Read referenced source context through `native-context` facets `overview`,
|
||||
`analysis-profile`, `limitations` and `result-limitations`. Reuse inspected
|
||||
context/profile values only for the same digest; until caveats are read,
|
||||
interpretation beyond the returned facts remains unknown. Complete Evidence
|
||||
remains retained and exportable. Small complete replies can cost fewer tokens
|
||||
than additional context/page calls; count the completed task and discovery.
|
||||
|
||||
## Static memory and original file bytes
|
||||
|
||||
For Ghidra, `read_bytes` reads initialized analysis memory, including loader
|
||||
fixups; it does not read a running process. Check `returned_bytes` and `complete`
|
||||
before decoding a table. An incomplete read is missing coverage, not a zero-filled
|
||||
tail. Use `list_segments` to inspect the selected block's initialization and address-space
|
||||
facts, and retain unavailable facts as unknown.
|
||||
|
||||
Use `address_to_file_offset` to anchor one observed address to original file
|
||||
bytes. A mapping for the first address does not establish a contiguous mapping
|
||||
for an entire table, and loader fixups can make analysis bytes differ from source
|
||||
bytes. Keep source ranges and analysis-memory observations separate.
|
||||
|
||||
A pointer table can refer to writable slots populated by a constructor rather
|
||||
than directly to immutable resources. An uninitialized block or an observed zero
|
||||
does not establish the slot's later runtime value. Follow `xrefs` and the relevant
|
||||
writer's assembly or pseudocode; distinguish recorded assignments from inferred
|
||||
execution order. Do not replace unreadable bytes with zeros or conclude that a
|
||||
resource is absent without the necessary initialization evidence.
|
||||
|
||||
## DOS MZ with Ghidra
|
||||
|
||||
For an admitted DOS MZ target, use `inspect_native_load_image` to check measured
|
||||
loaded bytes, file mappings, relocations and the entry against the immutable
|
||||
snapshot. Keep `mismatch` and `unsupported` explicit. Verified import covers the
|
||||
reported static image, not runtime DOS or PC-98 hardware behavior.
|
||||
|
||||
For a large load-image report, select `detail: "summary"` and query its
|
||||
`parent_evidence_id` with `inspect_analysis_view` kind `native-load-image`.
|
||||
Choose a collection such as `relocations` or `mappings`, select only needed
|
||||
fields, and page with offset/limit. Original indices preserve same-address
|
||||
alternatives. Null collections remain unrecorded or inapplicable; paging does
|
||||
not upgrade unsupported verification. CLI requires `--evidence_output` for
|
||||
summary delivery so complete source Evidence remains accessible.
|
||||
|
||||
For Ghidra decompilation, `procedure_pseudo_code` also accepts `procedure: "file-offset:0x..."`; CLI `rea decompile <path> --file-offset=0x... --provider ghidra` resolves the source byte offset internally. An unresolved 0x-prefixed numeric procedure selector also falls back to a unique file mapping. Existing symbols and analysis addresses win; multiple mappings fail as ambiguous. Preserve the returned file-offset and address locations and raw selection observations. ELF virtual addresses are distinct from file offsets; never assume an arbitrary offset plus image base is an analysis address.
|
||||
|
||||
Complete function body ranges use inclusive ends and can contain gaps. Do not
|
||||
treat the enclosing span as code.
|
||||
For packed targets, retain the original and separately derived artifact identities;
|
||||
decompiling an unpacking stub does not recover the unpacked program.
|
||||
|
||||
When reading Objective-C dispatch metadata, distinguish class implementation
|
||||
methods from protocol requirements: class methods report null requirement
|
||||
flags, while decoded protocol declarations retain required and optional facts.
|
||||
Property attributes remain raw name/value tokens; use the property's
|
||||
`atomicity` field for the decoded `N` marker or the default atomic behavior.
|
||||
Null class/meta flags, ivar counts, atomicity, or read-only values mean the
|
||||
selected provider did not establish that fact. Symbol names alone do not prove
|
||||
that a class is a root class or has no ivars.
|
||||
|
||||
## Managed PE/CLI
|
||||
|
||||
Start with `inspect_managed_artifact`. REA's canonical managed inspection is
|
||||
execution-free: do not claim it loaded, reflected, executed, or resolved the
|
||||
assembly. Keep managed/native boundaries and unavailable reconstruction facts
|
||||
explicit. A bring-your-own reconstruction oracle is separate from the canonical
|
||||
parser and must not become an implicit setup dependency.
|
||||
|
||||
For identity/classification questions, `inspect_managed_artifact` can explicitly
|
||||
select `detail: "summary"` on the first MCP call. It retains complete metadata
|
||||
and returns a linked overview with full classification/evidence, nullable
|
||||
module/MVID, frameworks, metadata coverage/issues/limitations and returned-row
|
||||
counts. Default complete requests remain unchanged. Read original rows with
|
||||
`inspect_analysis_view`, source `normalized_result.parent_evidence_id`, and
|
||||
`view: {kind: "managed", collection: "attributes", offset: 0, limit: 16}`.
|
||||
Collections are `references`, `resources` and `attributes`. Paging does not
|
||||
repeat parsing. Returned-row exhaustion
|
||||
does not prove complete metadata or absence under partial/unavailable coverage;
|
||||
preserve build-local tokens and their artifact/MVID scope.
|
||||
|
||||
`inspect_managed_members` always returns the assembly-wide `member_refs`, even
|
||||
for one selected type. For questions about a selected type, select
|
||||
`detail: "summary"` with `type_names`; it returns the selection (with unmatched
|
||||
names), coverage and per-collection counts. Page rows with
|
||||
`view: {kind: "managed-members", collection: "call_edges", offset: 0, limit: 64}`
|
||||
over `types`, `exported_types`, `fields`, `methods`, `member_refs`, `call_edges`,
|
||||
`field_accesses` or `user_strings`.
|
||||
Use exact `names` and `declaring_types` filters before paging supported
|
||||
collections. For a compact method lookup, request `fields: ["name",
|
||||
"declaring_type", "signature"]`; omit `body` to avoid anchors. Identity tokens,
|
||||
original `source_index`, module/MVID and source coverage remain. Check
|
||||
`unresolved_filter_rows` before interpreting an empty match as absence.
|
||||
|
||||
### .NET NativeAOT
|
||||
|
||||
NativeAOT binaries contain native machine code, so ordinary CIL decompilers
|
||||
cannot recover the original method bodies. When managed metadata/CIL is absent,
|
||||
classify that as a native-analysis route rather than claiming source recovery.
|
||||
Open the native image with the selected deep provider and inspect functions,
|
||||
strings, calls, and references; describe decompiler output as pseudocode and
|
||||
inference.
|
||||
|
||||
NativeAOT images can be PE/COFF, ELF, or Mach-O executables and shared
|
||||
libraries. Use file bytes and loader metadata rather than suffixes to identify
|
||||
them. For packaged apps, inventory/extract with REA's artifact tools and retain
|
||||
the exact contained image identity. PDB, ELF debug, or dSYM sidecars are
|
||||
additional evidence only when the provider accepts and matches them.
|
||||
|
||||
REA has a built-in read-only snapshot parser for x64 PE targets with .NET 8 RTR
|
||||
9.1 metadata. An optional headless Ghidra adapter adds database annotations and
|
||||
broader metadata recovery on Linux; it requires an explicitly configured
|
||||
NativeAOT JAR. These are separate routes. The built-in parser also runs on
|
||||
Windows Ghidra P0; read frozen literals from its report, because
|
||||
`search_strings` does not include them. See the
|
||||
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-nativeaot.md)
|
||||
for setup, supported layouts, and host restrictions. Preserve binary identity
|
||||
and distinguish derived metadata from Ghidra database observations. Recovered
|
||||
types and native pseudocode are not original C# source. Windows Ghidra P0 has
|
||||
no database mutation authority.
|
||||
|
||||
## Packages and extraction
|
||||
|
||||
Use `inspect_artifact` for application bundles, archives, ZIP/APK/IPA/MSIX/AppX,
|
||||
ASAR, or DMG inputs when the artifact graph and findings help answer the
|
||||
question. It returns the complete artifact graph inline. Cite graph manifest
|
||||
IDs when using them.
|
||||
|
||||
For an IPA or a macOS `.app`, ZIP, or DMG, pass the inventory Evidence to
|
||||
`project_apple_application_graph`. It reports application roots and nested
|
||||
bundles by path convention: app extensions, XPC services, frameworks, login
|
||||
items, system and driver extensions, and plug-ins. It also lists privileged
|
||||
helpers, launchd plists, symlinks, and each bundle's `info_plist_path` and
|
||||
executable candidates. Roles are path conventions, not parsed plists; read the
|
||||
listed plists with `inspect_plist`.
|
||||
|
||||
After `open_binary` on a `.app` or Mach-O, use `trace_dylib_resolution` to see
|
||||
which file each `@rpath`, `@loader_path` and `@executable_path` load reaches for
|
||||
every executable in the bundle. It also lists missing or weak loads and earlier
|
||||
`@rpath` candidates that are absent. Narrow large bundles with `roots` or
|
||||
`architecture`. System paths stay undetermined because the dyld shared cache
|
||||
provides them.
|
||||
|
||||
Use `extract_artifact` when materialized files from a supported format are needed.
|
||||
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md).
|
||||
It materializes all regular files into a fresh temporary directory chosen by
|
||||
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
|
||||
The result reports the materialized directory; callers do not choose the
|
||||
destination. No separate permission grant is needed.
|
||||
|
||||
For classic iNES cartridges, `open_binary` recognizes the byte signature without
|
||||
an executable architecture. `inspect_artifact` retains header facts and each
|
||||
PRG/CHR file unit's offset, length and SHA-256; its nested inventory's
|
||||
`ines.reconstruction` verifies the actual ordered region bytes against the
|
||||
whole-file digest. Extraction includes the original header, optional trainer
|
||||
and uninterpreted trailing bytes. These file units are not mapper/CPU bank
|
||||
maps. NES 2.0, UNIF, dirty legacy headers and truncated payloads are explicitly
|
||||
refused. No emulator or 6502 analysis is performed. See the
|
||||
[iNES guide](https://github.com/morluto/rea/blob/main/docs/ines-containers.md).
|
||||
|
||||
Native DMG traversal automatically uses a read-only mount on supported macOS
|
||||
hosts, with an owned temporary mount directory and cleanup. It needs no approval
|
||||
flag. Unsupported hosts or mount failures remain explicit limitations; do not
|
||||
claim child inventory when only root identity is available. DMG extraction is
|
||||
unavailable through `extract_artifact`. To access DMG files on macOS, create a
|
||||
read-only mount with your own tools, select files from that mount for analysis,
|
||||
and detach the mount you created when finished. REA's inspection mount is
|
||||
temporary, not a retained export of the image's contents.
|
||||
|
||||
For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
|
||||
`--target-format dos-com`). Inspect the load image before trusting function
|
||||
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
|
||||
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-dos.md).
|
||||
|
||||
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
|
||||
|
||||
For other headerless images (embedded firmware without ELF/Mach-O/PE/MZ
|
||||
headers), open with `format: "raw-binary"`, `architecture` (`x86`, `x86_64`,
|
||||
`arm`, `arm64`, `6502`, or `m68k`), and `entry_point` (`0x` hex). File offset 0 is mapped at
|
||||
that address. Hopper and IDA refuse this interpretation; Ghidra uses
|
||||
`BinaryLoader` with a generic language: big-endian for `m68k`, little-endian
|
||||
otherwise. Big-endian ARM, Thumb-only, and Cortex-M variants are unsupported.
|
||||
`6502` selects NMOS 6502; the entry and complete file mapping must fit within
|
||||
`0x0000..0xffff`. 65C02, 65816, cartridge containers, bank switching, and mapper
|
||||
semantics are unsupported. `m68k` requires an even entry and decodes the 68040
|
||||
superset of the 68000 family; Amiga/Atari containers and vector-table entry
|
||||
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/providers/raw-binary.md).
|
||||
|
||||
Intel HEX, S-record, UF2 and DfuSe files are containers, not raw images:
|
||||
`inspect_artifact` lists address regions (UF2 per familyID, DfuSe per element
|
||||
with its alternate setting), and `extract_artifact` writes each region. Open an
|
||||
extracted region as `raw-binary` with its start address as `entry_point`. UF2
|
||||
numbering gaps leave coverage unknown; a DfuSe suffix CRC mismatch follows
|
||||
`integrity_policy`. See [UF2 and DfuSe](https://github.com/morluto/rea/blob/main/docs/uf2-dfuse.md).
|
||||
|
||||
### Function annotations in Ghidra
|
||||
|
||||
Fresh-import sessions admit annotations; existing-project reuse is read-only.
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits. Normal close saves them when owned Program checkpoint mode is enabled; ephemeral mode discards them. CLI `annotate-native-function` follows the same setting when closing after its updated result. Select the [checkpoint workflow](#continue-editable-ghidra-analysis-across-processes) before edits that must survive a later process. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier. Ghidra rejects batch addresses that request the same name or a name another selected address holds; `raw_result.failures` gives each failure's reason.
|
||||
|
||||
A provider switch or different target can retire the active Ghidra database and discard
|
||||
its edits, even if the new provider fails on a later call. Same-target,
|
||||
same-profile reopen keeps the active database. Retained annotation Evidence
|
||||
records what happened; it does not restore the edits.
|
||||
|
||||
For `compare_functions`, obtain `analyze_function` Evidence before and after the
|
||||
edit while the MCP session is open; annotation Evidence is not a comparison
|
||||
input. Export the needed Evidence before close if comparison will happen later.
|
||||
Authenticated same-artifact coordinates preserve renamed entries as
|
||||
`matched / same-entry`, with the canonical `entry_address` inline; name and
|
||||
exact-text changes remain separate dimensions. This requires compatible
|
||||
architecture, provider-owned mapping and selected-image commitments. Equal
|
||||
numeric addresses alone cannot establish correspondence; inspect retained
|
||||
source Evidence and coverage when the match remains ambiguous.
|
||||
|
||||
## Offline PE resource inventory
|
||||
|
||||
Use `inspect_pe_resources` with an absolute `path`, or
|
||||
`rea inspect-pe-resources ./selected.exe --json`, for portable PE32/PE32+
|
||||
resource inspection without opening a native target. Preserve returned Evidence:
|
||||
type/name/language IDs and UTF-16 names, original directory/data/payload ranges,
|
||||
code pages, payload hashes and static icon-group references. Same-language icon
|
||||
resolution is explicit; missing references and size mismatch remain visible.
|
||||
Other payloads are opaque, and this operation does not decode images or simulate
|
||||
Windows resource loading. Consult the budgets and supported tree profile in
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
|
||||
## Offline PE and ELF export tables
|
||||
|
||||
Use `inspect_native_exports` with an absolute `path`, or
|
||||
`rea inspect-native-exports ./libexample.so --json`, to read a PE export
|
||||
directory or ELF dynamic symbol table on any host without opening a native
|
||||
target. Ordinal-only, forwarded and hidden-version exports stay distinct;
|
||||
partial or unsupported coverage means the inventory is incomplete, not that
|
||||
exports are absent. Use `inspect_macho` for Mach-O. See
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
|
||||
|
||||
For exact core-module import/export differences, use `compare_wasm_interfaces`
|
||||
with two complete inspection Evidence records (or current-session retained
|
||||
references) and explicit scope. Preserve unknown types/duplicate names; static
|
||||
interface equality is not runtime compatibility. CLI accepts complete inline
|
||||
Evidence in `rea compare-wasm-interfaces input.json --json`, or exact references
|
||||
with `--evidence-bundle /absolute/path/evidence.json`. Use the complete parent
|
||||
ID when the inspection returned a summary.
|
||||
|
||||
## Godot PCK
|
||||
|
||||
For a standalone PCK, `open_binary` then `inspect_artifact` with
|
||||
`detail: "summary"` retains the complete static inventory. Use the returned
|
||||
`parent_evidence_id` with `inspect_analysis_view` inventory occurrences filtered
|
||||
by exact `logical_paths` or `occurrence_ids`, and join `artifact_id` to nodes
|
||||
filtered by `artifact_ids`. Select only needed fields; identity links remain.
|
||||
Prefer `inspect-artifact --detail summary --evidence_output PATH` for CLI tasks;
|
||||
the same CLI discovers embedded packs via their EOF trailer without opening a
|
||||
native analysis provider. Normal native opening remains unchanged.
|
||||
|
||||
Distinguish declared MD5 from observed SHA-256 and graph/page completeness from
|
||||
`source_coverage`. Encrypted directories and sparse bundles have unknown entry
|
||||
coverage; encrypted members have unknown content identity. An empty filtered
|
||||
page cannot prove absence in unknown coverage. Retained views are historical,
|
||||
not fresh filesystem checks. No decryption, Godot resource decoding, GDScript
|
||||
decompilation or target execution is provided. See
|
||||
[Godot PCK](https://github.com/morluto/rea/blob/main/docs/godot-pck.md).
|
||||
|
||||
UnityFS AssetBundle containers (`.bundle`, `.ab`, YooAsset packages and other
|
||||
byte-identified files) list `CAB-*` SerializedFiles and `.resS` members with
|
||||
header and node facts; none/LZ4/LZ4HC member blocks stream on demand. Bundles
|
||||
declaring asset-bundle encryption keep header facts with unknown directory
|
||||
coverage; LZMA member bytes retain named occurrences with unavailable hashes.
|
||||
Virtual member offsets are not physical carrier locations. Format 6–8 readers
|
||||
cache one decoded block and support retained filtered/paged views. No
|
||||
decryption or SerializedFile object decoding is provided. See
|
||||
[UnityFS](https://github.com/morluto/rea/blob/main/docs/unityfs.md).
|
||||
|
||||
WeChat mini-program packages (`.wxapkg` files with `0xBE`/`0xED` marks) list
|
||||
members with declared names and absolute carrier offsets; member bytes are
|
||||
stored verbatim. `V1MMWX` PC-client packages report `encrypted` directory
|
||||
status with unknown coverage and are refused on extraction; no decryption is
|
||||
attempted. See
|
||||
[wxapkg](https://github.com/morluto/rea/blob/main/docs/wxapkg.md).
|
||||
|
||||
U-Boot legacy uImages use the same inspection/extraction workflows. Prefer a
|
||||
retained summary and selected inventory views for header facts or component
|
||||
IDs. Header addresses/CPU/OS are declarations; payloads remain as stored.
|
||||
Multi-file components retain their aligned physical ranges; the full payload
|
||||
is also materialized. Unknown name text retains its original 32 bytes. See
|
||||
[U-Boot uImage](https://github.com/morluto/rea/blob/main/docs/uimage.md).
|
||||
|
||||
## ESP images
|
||||
|
||||
For an ESP32-family app or bootloader image, `open_binary` then
|
||||
`inspect_artifact` with `detail: "summary"`. `root_format_metadata` holds the
|
||||
observed `header` and `descriptor` (its `kind` is the app or bootloader role)
|
||||
and the derived `chip`, `checksum`, `appended_sha256` and
|
||||
`undescribed_trailing` ranges. Segment rows (`segment-NN.bin`) carry
|
||||
`load_addr`, `classification` (`mapped`, `loaded`, `not-loaded` or `unknown`,
|
||||
per ESP-IDF `should_map`/`should_load`; RAM ranges are not checked) and
|
||||
`source_location`; select them with `inspect_analysis_view` and
|
||||
`logical_paths`. Signature blocks are `unverified`; `public_key_sha256`
|
||||
matches eFuse key digests.
|
||||
|
||||
Unknown `source_coverage` means truncation or undescribed trailing bytes, such
|
||||
as the flash segments of a `--ram-only-header` image; do not infer absence
|
||||
from it. Erased flash is described. Integrity mismatches are facts, not
|
||||
refusals. ESP8266 images are refused, and flash-encrypted images are not
|
||||
recognized. Segments are not disassembled, and RISC-V or Xtensa segments
|
||||
cannot be opened as raw-binary. See
|
||||
[ESP images](https://github.com/morluto/rea/blob/main/docs/esp-image.md).
|
||||
|
||||
## ESP flash images
|
||||
|
||||
A whole-flash dump opens as `esp-flash-image`. `root_format_metadata` holds
|
||||
`partition_table` (its `method`: `default`, `caller` or a derived
|
||||
`scan`), `bootloader` and `apps` ESP image facts, the `otadata` records,
|
||||
`partitions_outside_dump` and `boot_selection`. Every table has a
|
||||
`boot_selection`: its `basis` names the bootloader's branch (`no-otadata`
|
||||
without otadata, `none` when the bootloader rejects the table, `unknown`
|
||||
when a record is missing), and it lists its `assumptions`;
|
||||
`running_partition` is always `unknown`. Each image names its entry
|
||||
`path`. With `detail: "summary"` the root kind is `esp-flash-image-summary`
|
||||
and each image is a compact `summary` (chip, entry point, segment count,
|
||||
classified load ranges, signature block count, descriptor identity,
|
||||
integrity); complete detail keeps every segment. Partition rows
|
||||
(`partitions/<label>.bin`) carry the declared `type`, `subtype`, names,
|
||||
`flash_address`, `size` and flags; `encrypted` is a storage flag, not
|
||||
proof of ciphertext. `written_offset`/`written_end` bound the bytes that
|
||||
are not erased: an app slot whose facts say `erased` can still hold an older
|
||||
image, and a limitation says so. For a dump read from another flash address, call
|
||||
`open_binary` with `format: "esp-flash-image"` and `flash_offset`; an
|
||||
`esp-image` whose `flash_dump` is set names that offset in a next probe; a
|
||||
relocated table can be named with `partition_table_offset`. No table found
|
||||
means unknown, often flash encryption. Extraction writes `nvs_keys` as
|
||||
ordinary data. See
|
||||
[ESP flash images](https://github.com/morluto/rea/blob/main/docs/esp-flash-image.md).
|
||||
|
||||
## Bun standalone executables
|
||||
|
||||
A `bun build --compile` executable stores its module graph in Mach-O
|
||||
`__BUN,__bun`, a PE or ELF `.bun` section, or an appended trailer.
|
||||
`open_binary` then `inspect_artifact` reports root format `bun-standalone` with
|
||||
the carrier, structural layout, module count, entry point, raw flags and
|
||||
compile exec-argv; each module keeps its `/$bunfs/...` source name, loader,
|
||||
encoding, sizes and SHA-256, with sourcemap, bytecode and module-info regions as
|
||||
separate children. Use summary root metadata `entry_point_logical_path` as the
|
||||
exact inventory `logical_paths` selector, then read the selected content node by
|
||||
its artifact ID; the reader owns virtual-name normalization and indexed fallback. `extract-artifact` writes stored contents through the safe
|
||||
output tree; route extracted JavaScript to `analyze_javascript_application`.
|
||||
Layout labels describe record shapes, not a detected compiler version; unknown
|
||||
or ambiguous layouts are unsupported. Bytecode is retained, not decoded, and the
|
||||
executable is never run. See
|
||||
[Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
|
||||
|
||||
## Deno compile executables
|
||||
|
||||
A `deno compile` executable stores its application in a Mach-O `__SUI,d3n0l4nd`
|
||||
section, an Intel Mach-O `<~sui-data~>` trailer, an ELF `SUI` note of type
|
||||
`0x53554901`, or a PE `RT_RCDATA` resource named `D3N0L4ND`.
|
||||
`open_binary` then `inspect_artifact` reports root format `deno-standalone`
|
||||
with the carrier, entry point, recorded options, and one occurrence per virtual
|
||||
file, including size and SHA-256. Transpiled output, CJS export analysis, and
|
||||
source maps are separate occurrences with the file's `index` and their own
|
||||
`role`. They normally use a `.deno-*` suffix on the file path and move to an
|
||||
indexed path on a collision, so use the reported `logical_path`. Symlink targets
|
||||
stay recorded path parts. A virtual filesystem that cannot be listed leaves
|
||||
coverage unknown.
|
||||
Environment-file values and CA certificate bytes are retained in local Evidence
|
||||
with their counts and byte lengths. The npm snapshot length is reported and not
|
||||
decoded. `extract-artifact` writes
|
||||
stored files through the safe output tree; route extracted JavaScript to
|
||||
`analyze_javascript_application`. Other layouts are unsupported, and the
|
||||
executable is never run. See
|
||||
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
|
||||
@@ -0,0 +1,140 @@
|
||||
# Runtime observation
|
||||
|
||||
## Native runtime questions
|
||||
|
||||
Start from the requested observation. State the selected target, input, expected
|
||||
result, failure signal and stopping condition before a run. Reuse static Evidence
|
||||
to choose a function or hypothesis; `trace_native_values` follows recovered
|
||||
static value dependencies and does not measure runtime values. Prefer static
|
||||
inspection or existing capture comparison when it already answers the question.
|
||||
|
||||
| Question | Existing workflow | Host and effects |
|
||||
| -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Which selected native entry is reached, with which integer argument registers? | `observe_native_calls` on the active Mach-O, with exact function/Objective-C breakpoints and a duration/event bound | macOS with LLDB and required debugger access. Launches a new owned process, stops at entries, records and continues; target execution may write files, show UI or use the network. It never attaches to an existing process. Inspect termination and partial coverage after the run. |
|
||||
| What does caller-authored instrumentation observe? | `list_frida_devices` / `list_frida_processes` when discovery is needed; `start_frida_session`, `load_frida_script`, then status/unload/close | Requires compatible installed Frida bindings and selected device access; the real REA verifier covers Linux. Attach selects an existing PID; spawn selects a program and starts it paused until `resume_frida_session`. Scripts execute in the target and can change memory, files, network activity and behavior. This is not passive inspection. |
|
||||
| Which debugger state or controlled execution explains the behavior? | `start_gdb_session` and `gdb_console`, or `start_rizin_debug_session` and `rizin_debug_command` | Persistent debugger workflows are Linux-only. GDB starts a debugger; console commands choose/load/attach/run targets. Rizin debug start launches the selected target and requires a durable `inferior_tty`. Both consoles are unrestricted and can change target memory/registers, run shell commands, write files and use the network. |
|
||||
| How do terminal output, process lifetime and selected files differ between runs? | `capture_process_scenario`, then `compare_process_captures` with compatible recorded inputs | New captures require Linux or macOS and a working native PTY backend; macOS also needs `xcrun`/Swift for ownership inspection. Capture launches an owned command with ordinary user permissions. Observation paths select snapshots, not a filesystem sandbox. Comparing retained captures does not execute targets and is also available on Windows. |
|
||||
| What does an existing desktop window show, or what changes after a declared UI action? | `observe_native_ui`; use `capture_native_ui_scenario` only for requested actions | macOS 14+ with required Screen Recording/Accessibility access. Select an existing PID/window. Observation does not launch or own the app; scenario actions can change its data or trigger network activity, and leave its state as-is. |
|
||||
|
||||
Close each persistent session with its matching tool. `close_frida_session`
|
||||
unloads scripts and detaches; it does not kill the target. A still-paused spawn
|
||||
is resumed only after instrumentation cleanup is confirmed. GDB and Rizin close
|
||||
can detach a live inferior and let it continue only after verifying its durable
|
||||
output terminal. When disposition or cleanup is unresolved, preserve the owned
|
||||
session and reported recovery; do not assume that closing the debugger terminated
|
||||
the target. Process capture and LLDB call observation instead own their launched
|
||||
processes and report their termination/cleanup outcome. Cancellation is not proof
|
||||
that cleanup completed.
|
||||
|
||||
Unrestricted GDB/Rizin commands and relevant start/close operations retire an
|
||||
active binary-analysis session in the same MCP server. Close an active Ghidra
|
||||
session explicitly first; after target effects settle, reopen the artifact before
|
||||
new static analysis. A Cutter bridge command is likewise unrestricted and may
|
||||
affect its GUI/target; it is not a passive native observation substitute.
|
||||
|
||||
To test a static hypothesis across two builds, retain compatible static
|
||||
comparison Evidence and `compare_process_captures` Evidence. Supply explicit
|
||||
finding/dimension mappings to `correlate_static_and_runtime`. It does not ingest
|
||||
arbitrary Frida, LLDB or debugger traces, auto-match names, or prove causality
|
||||
from cochange. Preserve those traces as their own Evidence and leave unobserved
|
||||
behavior unknown. Record truncation, unmatched breakpoints, changed target
|
||||
identity and cleanup uncertainty as limitations, rather than treating no retained
|
||||
event as proof that the behavior never occurs.
|
||||
|
||||
For same-session MCP comparisons, pass each capture inline or reuse it as
|
||||
`{"kind":"retained-evidence","evidence_id":"ev_..."}`. Default comparison
|
||||
keeps capture-local timing as provenance without treating scheduler or observer
|
||||
latency as a behavior change; use an exact `trace_spec` event when timing is part
|
||||
of the analyst question.
|
||||
|
||||
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/guides/runtime/frida-instrumentation.md),
|
||||
[GDB/Rizin/Cutter](https://github.com/morluto/rea/blob/main/docs/reverse-engineering-tools.md),
|
||||
[process capture](https://github.com/morluto/rea/blob/main/docs/guides/runtime/process-capture.md),
|
||||
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/guides/native/native-investigation.md#native-desktop-observation).
|
||||
|
||||
## Browser and Electron
|
||||
|
||||
Supply the existing browser or Electron instance's literal-port loopback CDP
|
||||
endpoint in the request. No separate permission configuration or per-call
|
||||
approval flag is needed. Browser discovery lists eligible HTTP(S) pages;
|
||||
`allowed_origins` optionally filters them. An individual capture defaults to the
|
||||
selected page's current origin. Electron discovery lists eligible local
|
||||
`file://` targets without a configured file-root allowlist.
|
||||
|
||||
Start passive browser work with `list_browser_targets`; start Electron runtime work with
|
||||
`list_electron_targets`. Before an electron-main Inspector attempt, check
|
||||
packaged fuses with `inspect_electron_fuses`. Inspect the target selected for
|
||||
the user's task. Observation with these tools is passive: never claim REA
|
||||
clicked, navigated, evaluated page JavaScript, invoked IPC, or captured prior
|
||||
activity. REA does not contain the page's network.
|
||||
|
||||
Passive captures omit raw header collections, credential header values, storage
|
||||
values, and raw WebSocket/JSON bodies. Selected response metadata, including
|
||||
content encoding, policy values, and Link URLs, is retained. Local URLs retain ordinary query
|
||||
values and fragments; userinfo credentials are removed. Browser accessibility
|
||||
text, console primitives, value-free shapes, script sources, source maps, and
|
||||
storage key names have request-specific capture options. Electron script
|
||||
sources are also optional. Use `capture_web_screenshot` when pixels are needed.
|
||||
Treat scope filtering, truncation, and attach-window coverage as limitations.
|
||||
Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
|
||||
|
||||
## Who performs the action
|
||||
|
||||
- The user performs it: `list_browser_targets`, then `inspect_web_page` or
|
||||
`observe_web_session` on the selected target.
|
||||
- The agent must perform it: one `capture_browser_scenario` with declared
|
||||
actions. Use connect mode with the loopback CDP endpoint and `target_id` for
|
||||
an existing browser, or launch mode with a caller-selected executable for a
|
||||
temporary profile. Connect mode first navigates the selected page to
|
||||
`start_url`, which can clear unsaved form values and other temporary state.
|
||||
Recreate the state needed for the investigation in the declared actions.
|
||||
Connect mode leaves the external browser open; launch mode owns a temporary
|
||||
browser and profile. Check `browser.cleanup` for the outcome.
|
||||
Request only the DOM, accessibility, or network details needed for the question.
|
||||
The default final capture contains the URL; step outcomes are reported separately.
|
||||
For multi-step flows with trees or events, select `detail: "summary"` and page
|
||||
exact steps, transactions, text slices or events with `inspect_analysis_view` kind `browser-scenario`.
|
||||
Add a wait action for the expected page state before capturing it. Report that
|
||||
REA performed the declared actions.
|
||||
|
||||
Scenario network records have no request initiator stacks. For that evidence,
|
||||
use passive capture while the user performs the action. Listener source locations
|
||||
identify candidate handlers; they do not establish which script sent a particular
|
||||
request. Avoid overlapping passive capture and scenario actions on the same target.
|
||||
|
||||
Focused follow-up tools:
|
||||
|
||||
- `observe_web_execution`: execution coverage during a window; it resets
|
||||
counters and temporarily disables optimization.
|
||||
- `inspect_web_event_listeners`: source locations for listeners registered
|
||||
directly on the first CSS match, without dispatching an event.
|
||||
- `compare_web_captures`: two normalized passive captures or recorded scenarios.
|
||||
- `discover_webmcp_tools`: inspect page-declared WebMCP inventory; never invoke
|
||||
the discovered tools.
|
||||
|
||||
Use `reconcile_javascript_runtime` only after static application Evidence and
|
||||
passive browser/Electron Evidence exist. Prefer captured-source digest identity;
|
||||
report path/digest disagreement as a mismatch and preserve ambiguity. Explicit
|
||||
path mappings are inference inputs and never broaden filesystem or CDP authority.
|
||||
A bundle observed at runtime does not prove every contained module executed.
|
||||
|
||||
## Fiddler Everywhere traffic debugging
|
||||
|
||||
For large saved captures, request `detail: "summary"`, then page retained Evidence
|
||||
with `inspect_analysis_view` and `view: {kind: "capture", detail: "preview", offset: 0, limit: 20}`.
|
||||
Select complete record pages only for needed headers, metadata or payloads; views
|
||||
read the historical parent without repeating capture decoding.
|
||||
|
||||
For selected protobuf/gRPC bodies, keep raw bytes as the source and use
|
||||
`inspect_analysis_view` kind `protobuf`, collection `bodies`, `frames` or
|
||||
`fields`. Batch exact record ordinals and field numbers/IDs, and select
|
||||
`schema_value` or `text` only when needed. Supply a local descriptor set and
|
||||
message type for schema interpretation; compressed payloads remain unknown.
|
||||
|
||||
Inspect native `.saz` snapshots with `inspect_web_network_capture` and
|
||||
`format: "saz"`; encrypted archives are unsupported. For live
|
||||
traffic analysis, connect Everywhere's separate MCP server and use its
|
||||
advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or
|
||||
`AgentCalls`). Preserve existing sessions and filters when inspecting a
|
||||
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md)
|
||||
for the workflow and Telerik's public setup and traffic-debugging skills.
|
||||
@@ -27,7 +27,7 @@ body:
|
||||
attributes:
|
||||
label: Documentation location
|
||||
description: Link the page, section, command, API, skill, or inline comment.
|
||||
placeholder: README.md#quick-start or docs/mcp-contracts.md
|
||||
placeholder: README.md#quick-start or docs/reference/mcp-contracts.md
|
||||
validations:
|
||||
required: true
|
||||
|
||||
|
||||
@@ -71,8 +71,8 @@ source, link, or rendered-document checks. -->
|
||||
- User-visible CLI/MCP output (if applicable):
|
||||
- Remaining proof gaps:
|
||||
|
||||
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/testing.md#boundary-regressions)
|
||||
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/testing.md#lifecycle-regressions)
|
||||
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#boundary-regressions)
|
||||
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#lifecycle-regressions)
|
||||
covered by the evidence above, and explain relevant gaps.
|
||||
|
||||
For evidence-bearing changes:
|
||||
@@ -99,7 +99,7 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
|
||||
its new behavior and migration, and why compatibility/deprecation cannot
|
||||
preserve it. Every release increments minor, including breaking changes;
|
||||
keep breaking markers and migration notes so callers can assess compatibility.
|
||||
See docs/releasing.md. -->
|
||||
See docs/development/releasing.md. -->
|
||||
- Real Hopper/Ghidra, browser, or OS coverage:
|
||||
- Package or release metadata impact:
|
||||
- Security, privacy, process, or containment review:
|
||||
|
||||
@@ -544,7 +544,7 @@ jobs:
|
||||
--coverage.reporter=text
|
||||
--coverage.reporter=json-summary
|
||||
--coverage.reporter=lcov
|
||||
--reporter=default
|
||||
--reporter=dot
|
||||
--reporter=junit
|
||||
--reporter=json
|
||||
--reporter=./scripts/ci/test-timing-reporter.mjs
|
||||
@@ -559,7 +559,7 @@ jobs:
|
||||
needs.test-source.result == 'success' || needs.test-source.result == 'failure')
|
||||
run: >-
|
||||
npm exec -- vitest --merge-reports
|
||||
--reporter=default
|
||||
--reporter=dot
|
||||
--reporter=junit
|
||||
--reporter=json
|
||||
--reporter=./scripts/ci/test-timing-reporter.mjs
|
||||
@@ -924,6 +924,7 @@ jobs:
|
||||
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
|
||||
tests/boundary/filesystem/typescriptCompiler.test.ts
|
||||
tests/boundary/filesystem/releaseProposal.test.ts
|
||||
tests/acceptance/byteEntropyWindowsIdentity.test.ts
|
||||
tests/boundary/filesystem/binaryTarget.test.ts
|
||||
src/contracts/sessionPathInputs.test.ts
|
||||
tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts
|
||||
@@ -1016,8 +1017,8 @@ jobs:
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- run: npm run docs:check
|
||||
- name: Render documentation and validate page links
|
||||
run: npm run docs:build
|
||||
- name: Verify published routes, source links and local anchors
|
||||
run: npm run verify:docs:routes
|
||||
- name: Generation leaves authored files unchanged
|
||||
run: git diff --exit-code
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
|
||||
@@ -25,7 +25,7 @@ jobs:
|
||||
node-version-file: .nvmrc
|
||||
- name: Validate the SDK pin against the provider's verified release
|
||||
run: node scripts/verify-ghidra-bridge-sources.mjs --check-sdk-pin >> "$GITHUB_ENV"
|
||||
# The bridge probe imports only source-owned ARM/Thumb bytes and never
|
||||
# The bridge probe imports only source-owned ARM, Thumb and MIPS bytes and never
|
||||
# executes target code. All SDK/project state stays in runner storage.
|
||||
- name: Acquire checksum-verified Ghidra SDK in isolated runner storage
|
||||
shell: bash
|
||||
@@ -46,7 +46,7 @@ jobs:
|
||||
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
|
||||
test -n "${!java_home_variable}"
|
||||
JAVA_HOME="${!java_home_variable}" npm run verify:ghidra:bridge-sources -- --verified-sdk
|
||||
- name: Inspect real ARM and Thumb instruction context through the bridge
|
||||
- name: Inspect real ARM, Thumb and MIPS instruction context through the bridge
|
||||
shell: bash
|
||||
run: |
|
||||
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
|
||||
@@ -58,10 +58,21 @@ jobs:
|
||||
-import "${probe_root}/arm-thumb.bin" -loader BinaryLoader -processor ARM:LE:32:v7 \
|
||||
-noanalysis -readOnly -deleteProject \
|
||||
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
-postScript ReaArmContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
-postScript ReaInstructionContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
|
||||
cat "${probe_root}/output.log"
|
||||
grep -F 'REA_ARM_CONTEXT_OK arm=0 thumb=1 unavailable=null (injected provider value)' "${probe_root}/output.log"
|
||||
mkdir -p "${probe_root}/mips-project"
|
||||
# MIPS32 addiu v0,a0,3; MIPS16 li v0,90. These bytes are never executed.
|
||||
printf '\003\000\202\044\132\152\000\000' > "${probe_root}/mips.bin"
|
||||
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/mips-project" MipsContext \
|
||||
-import "${probe_root}/mips.bin" -loader BinaryLoader -processor MIPS:LE:32:default \
|
||||
-noanalysis -readOnly -deleteProject \
|
||||
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
-postScript ReaInstructionContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" mips \
|
||||
> "${probe_root}/mips-output.log" 2>&1 || { cat "${probe_root}/mips-output.log"; exit 1; }
|
||||
cat "${probe_root}/mips-output.log"
|
||||
grep -F 'REA_MIPS_CONTEXT_OK standard=0 mips16=1' "${probe_root}/mips-output.log"
|
||||
# Each probe imports source-owned bytes at 0x1000, away from the Cortex-M
|
||||
# vector symbols that ARMCortex.pspec defines at 0, and decodes them from
|
||||
# the language's default context, so M-profile Thumb state must hold at
|
||||
@@ -95,3 +106,31 @@ jobs:
|
||||
decode cortex ARM:LE:32:Cortex mrs/4/msp bx/2/lr
|
||||
decode v8m ARM:LE:32:v8-m mrs/4/msplim bx/2/lr
|
||||
decode xtensa Xtensa:LE:32:default entry/3/a1 retw.n/2
|
||||
# A call through a writable slot keeps Ghidra's recorded target as an
|
||||
# observed callee while callee coverage stays partial; a direct call is
|
||||
# the complete control. The source-owned x86-64 bytes are never executed.
|
||||
- name: Keep computed-call targets non-exhaustive in callee coverage
|
||||
shell: bash
|
||||
run: |
|
||||
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
|
||||
export JAVA_HOME="${!java_home_variable}"
|
||||
probe_root="${RUNNER_TEMP}/rea-computed-call"
|
||||
mkdir -p "${probe_root}/project"
|
||||
{
|
||||
printf '\270\001\000\000\000\303\314\314' # 0x1000: mov eax, 1; ret
|
||||
printf '\270\002\000\000\000\303\314\314' # 0x1008: mov eax, 2; ret
|
||||
printf '\377\025\052\000\000\000\303\314' # 0x1010: call [rip + 0x2a]; ret
|
||||
printf '\377\320\303\314\314\314\314\314' # 0x1018: call rax; ret
|
||||
printf '\350\333\377\377\377\303\314\314' # 0x1020: call 0x1000; ret
|
||||
printf '\314%.0s' $(seq 24) # 0x1028: int3 padding
|
||||
printf '\000\020\000\000\000\000\000\000' # 0x1040: slot = 0x1000
|
||||
} > "${probe_root}/computed-call.bin"
|
||||
test "$(wc -c < "${probe_root}/computed-call.bin")" -eq 72
|
||||
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/project" ComputedCall \
|
||||
-import "${probe_root}/computed-call.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
|
||||
-processor x86:LE:64:default -noanalysis -readOnly -deleteProject \
|
||||
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
-postScript ReaComputedCallCoverageProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
|
||||
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
|
||||
cat "${probe_root}/output.log"
|
||||
grep -F 'REA_COMPUTED_CALL_COVERAGE_OK slot=observed-complete-discovery-unknown register=partial direct=complete callers=discovery-unknown' "${probe_root}/output.log"
|
||||
@@ -27,4 +27,4 @@ jobs:
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- run: npm run docs:build
|
||||
- run: npm run verify:docs:routes
|
||||
@@ -56,7 +56,8 @@ jobs:
|
||||
run: |
|
||||
capture_tool_root="${RUNNER_TEMP}/rea-mitmproxy-12.2.3"
|
||||
mkdir -p "${capture_tool_root}"
|
||||
curl --fail --location --retry 3 --max-time 180 \
|
||||
# This immutable GET may reset mid-body; retry it before checksum verification.
|
||||
curl --fail --location --retry 3 --retry-all-errors --max-time 180 \
|
||||
https://downloads.mitmproxy.org/12.2.3/mitmproxy-12.2.3-linux-x86_64.tar.gz \
|
||||
--output "${capture_tool_root}/mitmproxy.tar.gz"
|
||||
echo '2e95286b618fa6fd33e5e62a78c2e5112571d85f42ec2bac29b97ee242bdb5c5 mitmproxy.tar.gz' \
|
||||
|
||||
@@ -12,6 +12,8 @@
|
||||
"package-lock.json",
|
||||
".claude-plugin/plugin.json",
|
||||
".codex-plugin/plugin.json",
|
||||
".cursor-plugin/plugin.json",
|
||||
"mcp.json",
|
||||
"_reference/**",
|
||||
"third_party/**",
|
||||
".codex-plugin/mcp.json"
|
||||
|
||||
@@ -14,13 +14,13 @@ Preserve caller-selected inputs and local evidence, including output, URLs, path
|
||||
|
||||
## Architecture and Cleanup
|
||||
|
||||
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture.mermaid](docs/architecture.mermaid) when changing composition.
|
||||
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture/architecture.mermaid](docs/architecture/architecture.mermaid) when changing composition.
|
||||
|
||||
Give each interpretation and resource a clear owner. Consumers must use that owner's parsed facts rather than reinterpret raw source. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
|
||||
|
||||
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
|
||||
|
||||
For lifecycle changes, follow [ownership and completion invariants](docs/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
|
||||
For lifecycle changes, follow [ownership and completion invariants](docs/development/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/development/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
|
||||
|
||||
Before adding complexity for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode, likelihood, and complexity cost to the user and get their buy-in. Do not silently expand scope for theoretical robustness.
|
||||
|
||||
@@ -68,6 +68,12 @@ do not establish fewer tokens or faster analysis.
|
||||
|
||||
## Development
|
||||
|
||||
Documentation ownership and authoring conventions live in
|
||||
[docs/development/documentation.md](docs/development/documentation.md). A feature
|
||||
uses `docs/specs/<domain>/<feature-slug>/spec.md`, one `plan.md` and `tasks.md`;
|
||||
individual architectural decisions use `docs/adr/<domain>/<decision-slug>.md`.
|
||||
Consult the [documentation directory](docs/README.md) before adding a guide.
|
||||
|
||||
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`. Prefix `PATH="/usr/local/bin:$PATH"` before `node` and `npm`.
|
||||
|
||||
- `npm ci`: install locked dependencies.
|
||||
@@ -80,15 +86,15 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
|
||||
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
|
||||
|
||||
When changing installers or setup recovery, follow
|
||||
[Checking installers and publisher downloads](docs/testing.md#checking-installers-and-publisher-downloads).
|
||||
[Checking installers and publisher downloads](docs/development/testing.md#checking-installers-and-publisher-downloads).
|
||||
|
||||
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/testing.md#developer-commands).
|
||||
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/development/testing.md#developer-commands).
|
||||
|
||||
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
|
||||
|
||||
## Tool and Test Changes
|
||||
|
||||
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
|
||||
For tool design, follow [docs/development/tool-design.md](docs/development/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
|
||||
|
||||
For user- or agent-facing tool problems, prioritize fixing the owning behavior,
|
||||
contract, and actionable result or recovery at the point of use. Document the
|
||||
@@ -96,7 +102,7 @@ result in product docs when explanation is needed. Add skill guidance only for
|
||||
workflow judgment or context that the tool cannot reasonably convey; do not use
|
||||
a skill as the primary repair for a discoverability or contract defect.
|
||||
|
||||
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Select applicable [boundary regressions](docs/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
|
||||
For test changes or provider claims, use [docs/development/testing.md](docs/development/testing.md). Select applicable [boundary regressions](docs/development/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
|
||||
|
||||
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
|
||||
|
||||
|
||||
+7
-7
@@ -132,8 +132,8 @@ Full contribution record: [4 commits and 1 merged PR](https://github.com/morluto
|
||||
- **Kuna analysis:** Use a caller-installed Kuna for procedure listings, pseudocode, assembly, call relationships, exact-address xrefs and cached string searches. Project exports retain source spans and digests; repeated requests reuse completed work. Recorded real-provider coverage uses Kuna 1.833 on Linux; Windows, PE and universal Mach-O are unsupported. See [the provider guide](docs/kuna-provider.md). ([#3832](https://github.com/morluto/rea/pull/3832), [#3854](https://github.com/morluto/rea/pull/3854), [#4071](https://github.com/morluto/rea/pull/4071), [#4095](https://github.com/morluto/rea/pull/4095)). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Retained JavaScript facts:** Query exact application paths, selected properties and filtered module references through `inspect_analysis_view`, with paging and complete row Evidence. These views reuse completed analysis and preserve source selection and unknown coverage. ([#4265](https://github.com/morluto/rea/pull/4265), [morluto/rea#4190](https://github.com/morluto/rea/issues/4190), [#4097](https://github.com/morluto/rea/pull/4097), [morluto/rea#4092](https://github.com/morluto/rea/issues/4092)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **APK locale tables:** Request several resource locales in one decode with repeatable CLI `--locales` or MCP `locales`. Each table reports its status and coverage; existing single-locale fields remain available. ([#4804](https://github.com/morluto/rea/pull/4804), [morluto/rea#4777](https://github.com/morluto/rea/issues/4777)). Thanks @abhay-2108, @N0zoM1z0.
|
||||
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
|
||||
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
|
||||
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/providers/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
|
||||
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/providers/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
|
||||
- **JADX deadlines:** Set `REA_JADX_OPERATION_TIMEOUT_MS` to extend APK loading and upstream operation deadlines together. Doctor reports the effective value and setup forwards an explicit selection. ([#4466](https://github.com/morluto/rea/pull/4466), [morluto/rea#3918](https://github.com/morluto/rea/issues/3918)). Thanks @jaychou813, @SandstromPL.
|
||||
- **Electron window options:** Resolve stable local constant `BrowserWindow` options while retaining source provenance. Relevant writes and escapes keep the result unknown. ([#4400](https://github.com/morluto/rea/pull/4400), [morluto/rea#4366](https://github.com/morluto/rea/issues/4366)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Language editions:** Add the Greek website edition and Malayalam documentation, with Korean typography and translation corrections across the site. ([#3520](https://github.com/morluto/rea/pull/3520), [#4792](https://github.com/morluto/rea/pull/4792), [#3821](https://github.com/morluto/rea/pull/3821), [morluto/rea#3820](https://github.com/morluto/rea/issues/3820), [#3799](https://github.com/morluto/rea/pull/3799), [morluto/rea#3798](https://github.com/morluto/rea/issues/3798), [#4344](https://github.com/morluto/rea/pull/4344), [#3856](https://github.com/morluto/rea/pull/3856), [#4166](https://github.com/morluto/rea/pull/4166)). Thanks @gensokyo-dev, @hi2242, @kalayciburak, @kaoru0822-kitauji, @kraken-503, @kukinghan, @N0zoM1z0, @yeseong0412.
|
||||
@@ -192,7 +192,7 @@ Full contribution record: [571 commits and 236 merged PRs](https://github.com/mo
|
||||
|
||||
## [6.5.0](https://github.com/morluto/rea/compare/rea-agents-6.4.0...rea-agents-6.5.0) (2026-10-13)
|
||||
|
||||
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/testing.md).
|
||||
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/development/testing.md).
|
||||
|
||||
### Highlights
|
||||
|
||||
@@ -220,7 +220,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
|
||||
- **HarmonyOS packages:** inventory HAP/HSP/App Pack structures without executing code. The recorded real workflow covers one pinned HAP through repository-built CLI/MCP; installed-package and real HSP/App Pack coverage remain unverified. ([#1447](https://github.com/morluto/rea/pull/1447)). Thanks @BaaRai.
|
||||
- **Ghidra raw inputs and references:** compatible Linux/macOS Ghidra profiles accept caller-selected raw architecture/entry information and expose typed incoming references at exact addresses. Windows Ghidra P0 admits the raw NMOS 6502 profile described below; other raw targets and the incoming-reference facet remain outside Windows P0. ([#1868](https://github.com/morluto/rea/pull/1868), [issue morluto/rea#1836](https://github.com/morluto/rea/issues/1836), [#1715](https://github.com/morluto/rea/pull/1715), [issue morluto/rea#1245](https://github.com/morluto/rea/issues/1245)). Thanks @bugstan, @casesolved-co-uk, @mvanhorn, @N0zoM1z0.
|
||||
- **Ghidra MIPS:** Add bounded MIPS32r2/o32 relocatable objects, soft-float integer-register storage, admitted MIPS16 shared ELF, and a narrowly admitted recordless legacy MIPS32 executable profile. The latter pins an endian-matched 32-bit language and records inferred o32 as an analysis assumption. Admission follows each documented ABI/flag/relocation boundary. ([#1949](https://github.com/morluto/rea/pull/1949), [issue morluto/rea#1937](https://github.com/morluto/rea/issues/1937), [#1950](https://github.com/morluto/rea/pull/1950), [issue morluto/rea#1938](https://github.com/morluto/rea/issues/1938), [#1991](https://github.com/morluto/rea/pull/1991), [issue morluto/rea#1851](https://github.com/morluto/rea/issues/1851), [#1898](https://github.com/morluto/rea/pull/1898), [issue morluto/rea#1620](https://github.com/morluto/rea/issues/1620)). Thanks @AurelioB, @BaaRai, @bqw371, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
|
||||
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/providers/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
|
||||
- **PSP static ELF profile:** the Ghidra profile is Linux x64, bounded little-endian ELF32 `ET_EXEC` with the documented PSP/ABI declarations and a compatible caller-installed Allegrex extension. PRX, shared/relocatable images, PBP/ISO/CSO and runtime emulation are outside scope. Historical extension experiments are separate from public-provider verification; real public Ghidra PSP coverage remains unverified. ([#1599](https://github.com/morluto/rea/pull/1599)). Thanks @jaelliot.
|
||||
- **Ghidra analysis settings:** apply validated analyzer overrides before analysis while retaining the selected analysis profile. ([#1913](https://github.com/morluto/rea/pull/1913)). Thanks @akushonkamen, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Attached native IDA MCP:** add an optional native bridge alongside existing legacy/headless profiles. Recorded native runs use Linux IDA Free 9.4 with a navigation correction and Windows IDA Pro 9.3 with a loopback-listener change; stock native switching, native macOS and Windows IDA Free remain unverified. Attached databases remain read-only and are never saved or closed. ([#1544](https://github.com/morluto/rea/pull/1544)). Thanks @imissapixel, @N0zoM1z0.
|
||||
@@ -292,7 +292,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
|
||||
- **Source maps:** original-source and mapping rows preserve a null source name when unavailable. Handle null explicitly instead of expecting an invented unknown-source string; new index fields remain optional when reading older Evidence. ([ee50ed1](https://github.com/morluto/rea/commit/ee50ed121e38cc6e1a2ff0dc8b789f15f71c5402)).
|
||||
- **Streamed filtered JSON:** `--filter-output` now emits JSON `null` instead of `{}` when the selected value is null; consumers must distinguish a present null from a missing selection. Use `--full-output` when an object envelope is required. ([#2333](https://github.com/morluto/rea/pull/2333)). Thanks @Jackmeson1.
|
||||
- **Call graphs:** `get_call_graph.normalized_result` is now `{ graph, limitations }`; read traversal-depth groups under `graph` and retain relationship-specific caveats. Contract/fixture validation does not establish new Ghidra or IDA engine coverage. ([5dfb9d0](https://github.com/morluto/rea/commit/5dfb9d050c2cc44a61b183dae765bef4d22a53b7)).
|
||||
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/reference/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
|
||||
|
||||
- **Provider admission:** MIPS targets with unresolved GNU ABI attributes are refused. Supply supported ABI declarations. Saved Hopper databases on macOS are unavailable because the launcher cannot run the analysis bridge when opening a database; select an executable target instead. Database support requires a script-capable database launch route. ([d4bd65c](https://github.com/morluto/rea/commit/d4bd65c53c0254d0195cc7ab8da163dcadfd3b9b)).
|
||||
- **Segment extents:** `binary_overview` segment length may be null when Hopper coordinates cannot establish a finite extent in one address space. Preserve unknown rather than converting it to zero or a saturated value. ([36db19c](https://github.com/morluto/rea/commit/36db19c59)).
|
||||
@@ -372,7 +372,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/process-capture.md) and [browser scenario contract](docs/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
|
||||
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/guides/runtime/process-capture.md) and [browser scenario contract](docs/guides/web/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
|
||||
* **browser:** JSON shape paths now contain typed property and array-element segments. WebMCP input_schema_shape is replaced by input_schema and input_schema_sha256, retaining declared examples and defaults.
|
||||
* **process:** Filesystem digest omission entries now require system_code; budget and changed-file omissions use null, and unavailable-file omissions preserve the operating-system error code.
|
||||
* **contracts:** Provider analysis outputs must explicitly supply metadata previously repaired with defaults; unsupported nullable facets must be reported as null.
|
||||
@@ -540,7 +540,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
|
||||
Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
|
||||
|
||||
* **MCP / Evidence / snapshots / artifacts:** Success results are direct canonical Evidence; read `normalized_result` rather than the old `result`/`evidence` wrapper. Evidence requires nullable `analysis_profile`, session provider aliases are removed, snapshots require complete profile-bound query records, and artifact roles/permissions belong to occurrences. Recreate incompatible saved data; adding fields without recomputing their semantic identity is invalid. ([63bf914](https://github.com/morluto/rea/commit/63bf91405cf97b114a84f337fb1272d98a267999))
|
||||
* **Process captures:** Legacy v3 conversion is removed. Explicit executable identity and `event_journal` are required; retain old captures and recapture when current comparisons are needed instead of inventing launch identity or event order. ([2a77469](https://github.com/morluto/rea/commit/2a7746998d47446c4db6d4f524e0f183d6803f3f))
|
||||
|
||||
+27
-10
@@ -11,7 +11,7 @@ incorrect user instructions warrant standalone corrections. Collect other
|
||||
cosmetic-only suggestions in an issue rather than separate PRs.
|
||||
|
||||
For selected architectural owners and dependency paths, use the
|
||||
[architecture map](docs/architecture.mermaid). It is not a complete directory or
|
||||
[architecture map](docs/architecture/architecture.mermaid). It is not a complete directory or
|
||||
import inventory. For current provider wiring, start with
|
||||
[`src/composition/binary.ts`](src/composition/binary.ts) for deep candidates and
|
||||
the other `src/composition/` factories for tool services, including
|
||||
@@ -32,7 +32,7 @@ removing it. Preserve validation, lazy loading, caller policy, and resource
|
||||
acquisition and cleanup supplied by the old boundary. Published APIs, persisted
|
||||
formats, and independently versioned bridges require an explicit compatibility
|
||||
decision; an internal representation alone does not establish that obligation.
|
||||
See [interpreting boundary facts once](docs/tool-design.md#interpret-boundary-facts-once)
|
||||
See [interpreting boundary facts once](docs/development/tool-design.md#interpret-boundary-facts-once)
|
||||
for admission ownership and the distinction between repeated validation and
|
||||
independent checks.
|
||||
|
||||
@@ -44,7 +44,7 @@ Use the [analysis optimization skill](.agents/skills/rea-analysis-optimization/S
|
||||
to locate provider and shared-workflow bottlenecks, preserve Evidence, and
|
||||
compare equivalent real tasks before claiming an improvement.
|
||||
|
||||
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
|
||||
When adding or changing an MCP tool, follow the [tool design guide](docs/development/tool-design.md) and preserve the canonical contracts and generated catalog.
|
||||
When adding or changing an admission ceiling, budget, or deadline, follow
|
||||
[analysis settings, limits, and recovery](docs/design/limits-and-recovery.md)
|
||||
for ownership, operation snapshots, provider integration, and the steps for
|
||||
@@ -130,6 +130,19 @@ status checks alongside its existing checks. Deploy the workflow to each
|
||||
protected target branch before requiring it there. The workflow reads PR
|
||||
metadata using trusted default-branch code and writes only a commit status.
|
||||
|
||||
When an open PR may overlap another open change or a recently landed fix, note
|
||||
the related PR, issue, or commit in the description or a comment as soon as the
|
||||
overlap is known. Prefer one clear pointer over repeated status updates.
|
||||
|
||||
If an overlapping change merges while a PR awaits review, link that commit on
|
||||
the open PR and state what remains useful. Rebase or otherwise align with the
|
||||
current base when the remaining work is still wanted, narrow the PR to the
|
||||
uncovered portion, or close it when the change is fully superseded. Keep the
|
||||
description and validation current after that update. Maintainers may also note
|
||||
the expected disposition (rebase, narrow, or close) when they land or recognize
|
||||
an overlapping fix. Readiness still does not grant merge permission or a review
|
||||
date.
|
||||
|
||||
Maintainers may return a PR to the author or close it when the problem is
|
||||
unsupported, validation is missing, or the owner cannot explain the change.
|
||||
Contributors are responsible for completing this work before requesting review.
|
||||
@@ -234,7 +247,7 @@ guarantee.
|
||||
Use the development toolchain pinned in `.nvmrc` and
|
||||
`package.json#packageManager`. Run `nvm use` before installing dependencies;
|
||||
installed-package runtime support is documented in
|
||||
[installation](docs/installation.md#start-setup).
|
||||
[installation](docs/getting-started/installation.md#start-setup).
|
||||
Use Node.js and npm for source builds and verifier scripts; the installed
|
||||
package's Bun runtime support does not extend to those development scripts.
|
||||
|
||||
@@ -289,16 +302,20 @@ base revision, or run the suite on Linux or WSL, before attributing them to a
|
||||
change. The embedded-artifact parent-swap tests exercise the selected-path
|
||||
replacement when the rename succeeds; if Windows refuses it while the admitted
|
||||
file is open, they instead verify the refusal and cleanup. See
|
||||
[testing](docs/testing.md#developer-commands) for command scopes
|
||||
[testing](docs/development/testing.md#developer-commands) for command scopes
|
||||
and lane selection.
|
||||
|
||||
`git submodule update --init --recursive` fails at the pinned Unblob submodule
|
||||
on Windows and can leave later submodules unchecked out. Apply the
|
||||
[Unblob checkout workaround](docs/firmware-analysis.md#windows-submodule-checkout),
|
||||
[Unblob checkout workaround](docs/guides/firmware/firmware-analysis.md#windows-submodule-checkout),
|
||||
then rerun the command.
|
||||
|
||||
## Documentation website
|
||||
|
||||
Use the [documentation directory](docs/README.md) and
|
||||
[authoring guide](docs/development/documentation.md) for document ownership,
|
||||
domain-based feature specifications and architectural decision records.
|
||||
|
||||
The VitePress site uses the Markdown files in `docs/`. Run `npm run docs:dev`
|
||||
for live editing, `npm run docs:build` to check the production build and
|
||||
links, and `npm run docs:preview` to preview that build at `/rea/`.
|
||||
@@ -352,7 +369,7 @@ npm run check:fast
|
||||
`test:local` runs source tests without building. `check:changed` combines cached
|
||||
static checks with source tests affected since the merge base with `origin/main`.
|
||||
Import-graph selection can miss runtime registration, bridges, and generated data;
|
||||
select relevant boundary tests explicitly. See [testing](docs/testing.md#developer-commands)
|
||||
select relevant boundary tests explicitly. See [testing](docs/development/testing.md#developer-commands)
|
||||
for commands, test placement, pruning, and real-provider prerequisites.
|
||||
|
||||
Before requesting review, run relevant tests and `npm run check`; add
|
||||
@@ -427,8 +444,8 @@ Run `npm run verify:hopper` on a supported macOS host or
|
||||
`npm run verify:hopper:linux` on the supported Linux demo runner. These build
|
||||
native fixtures and verify the installed engine through CLI/MCP; package or
|
||||
simulated-provider checks do not establish engine behavior. See
|
||||
[real-toolchain verification](docs/testing.md#end-to-end-integration-and-golden-evidence)
|
||||
for prerequisites and scope, and [Hopper setup](docs/installation.md#hopper)
|
||||
[real-toolchain verification](docs/development/testing.md#end-to-end-integration-and-golden-evidence)
|
||||
for prerequisites and scope, and [Hopper setup](docs/getting-started/installation.md#hopper)
|
||||
for supported hosts and unattended startup.
|
||||
|
||||
Use the default `build/conformance/manifest.json` unless explicitly verifying
|
||||
@@ -437,7 +454,7 @@ Generated fixtures and manifests remain ignored.
|
||||
|
||||
## Maintainer release checklist
|
||||
|
||||
Use the [release guide](docs/releasing.md). Main pushes automatically refresh a
|
||||
Use the [release guide](docs/development/releasing.md). Main pushes automatically refresh a
|
||||
Release Please PR with the next version and changelog. Review its migration
|
||||
notes, wait for the current CI checks to pass, and merge it into main to publish.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md)
|
||||
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md) · [Română](docs/readme/README_ro.md)
|
||||
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md) · [Română](docs/readme/README_ro.md) · [Български](docs/readme/README_bg.md)
|
||||
|
||||
# REA: Reverse Engineer Anything
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
**See a feature you like. Understand how it works, down to the binary level.**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](docs/mcp-contracts.md#generated-catalog)
|
||||
[](docs/reference/mcp-contracts.md#generated-catalog)
|
||||
[](#what-you-can-analyze)
|
||||
[](https://skills.sh/morluto/rea/reverse-engineer-anything)
|
||||
[](LICENSE)
|
||||
@@ -72,7 +72,7 @@ Setup registers REA with your agent and installs matching workflow instructions.
|
||||
|
||||
On Linux, Ghidra, JADX, firmware tool launches and static JavaScript/Electron
|
||||
analysis require a procps-compatible `ps` on REA's `PATH` for process ownership
|
||||
checks. Recovery steps are in `docs/installation.md` under
|
||||
checks. Recovery steps are in `docs/getting-started/installation.md` under
|
||||
"Linux owned-process prerequisite".
|
||||
|
||||
### Set up your agent
|
||||
@@ -91,9 +91,10 @@ available. Run package acquisition from a trusted directory outside the target
|
||||
project. Installation and recovery details are in the setup guide below.
|
||||
|
||||
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
|
||||
[other agents](docs/installation.md#supported-agents). See
|
||||
[installation and setup](docs/installation.md) for provider configuration and
|
||||
manual MCP registration and Claude Code/Codex plugin installation.
|
||||
[other agents](docs/getting-started/installation.md#supported-agents). See
|
||||
[installation and setup](docs/getting-started/installation.md) for provider configuration,
|
||||
manual MCP registration, Claude Code/Codex plugins, and the Cursor Marketplace
|
||||
plugin.
|
||||
|
||||
MCP uses compact input schemas by default to reduce context overhead. Set
|
||||
`REA_MCP_INPUT_SCHEMA_PROFILE=full` in your MCP registration when you need the
|
||||
@@ -135,7 +136,7 @@ rea --help
|
||||
```
|
||||
|
||||
For native analysis, configure a provider first. See the
|
||||
[CLI and Evidence guide](docs/cli.md) for native commands, provider selection,
|
||||
[CLI and Evidence guide](docs/getting-started/cli.md) for native commands, provider selection,
|
||||
snapshots and scripting.
|
||||
|
||||
### Update REA
|
||||
@@ -191,45 +192,47 @@ CLI commands use the same workflows.
|
||||
REA requires Node.js 22.x (>=22.23.4), 24.x (>=24.22.1), or 26.12.1+, plus npm.
|
||||
Additional tools and host support depend on the target:
|
||||
|
||||
| Target | What REA returns | Requirements and guide |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/native-investigation.md) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
|
||||
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
|
||||
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
|
||||
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
|
||||
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
|
||||
| Binary hardening | PIE, NX, RELRO, CFG and related header flags for ELF, PE and Mach-O | Built-in static header inspection; [binary hardening](docs/binary-hardening.md) |
|
||||
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
|
||||
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
|
||||
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
|
||||
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
|
||||
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
|
||||
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
|
||||
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
|
||||
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
|
||||
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
|
||||
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
|
||||
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
|
||||
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
|
||||
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
|
||||
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
|
||||
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
|
||||
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
|
||||
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
|
||||
| Target | What REA returns | Requirements and guide |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/guides/native/native-investigation.md) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/guides/native/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
|
||||
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
|
||||
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
|
||||
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
|
||||
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
|
||||
| Windows shortcuts | Shell Link target, arguments, working dir, ExtraData and Tracker droid facts | Built-in static LNK inspection; [Windows shortcuts](docs/guides/native/windows-shortcut.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
|
||||
| Binary hardening | PIE, NX, RELRO, CFG and related header flags for ELF, PE and Mach-O | Built-in static header inspection; [binary hardening](docs/binary-hardening.md) |
|
||||
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
|
||||
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/guides/evm/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/guides/native/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/guides/javascript/javascript-application-workflows.md) |
|
||||
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/guides/web/browser-observation.md) |
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/guides/web/web-network-captures.md) |
|
||||
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/guides/managed/managed-code-analysis.md) |
|
||||
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/guides/android/android-analysis.md) |
|
||||
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
|
||||
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/guides/android/adb-device-analysis.md) |
|
||||
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/providers/jeb-analysis.md) |
|
||||
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/guides/android/apktool-resource-analysis.md) |
|
||||
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/guides/android/flutter-analysis.md) |
|
||||
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/guides/firmware/firmware-analysis.md) |
|
||||
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/guides/data/packet-captures.md) |
|
||||
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/guides/data/pcap-analysis.md) |
|
||||
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/guides/signals/signal-recordings.md) |
|
||||
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/guides/javascript/javascript-artifact-reconstruction.md), [Apple applications](docs/guides/apple/apple-application-analysis.md) |
|
||||
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/guides/runtime/process-capture.md) |
|
||||
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
|
||||
| Deno compile binaries | Virtual files, entry point, recorded options, npm snapshot length | None (static only); [Deno compile executables](docs/deno-standalone.md) |
|
||||
|
||||
Static JavaScript and .NET inspection read the supplied files without running
|
||||
the application. Runtime capture runs or interacts with the selected target
|
||||
@@ -243,15 +246,15 @@ or update either provider.
|
||||
<a id="choosing-a-deep-analysis-provider"></a>
|
||||
|
||||
Native formats and host support vary by provider. See
|
||||
[Hopper and Ghidra setup](docs/installation.md#hopper), the
|
||||
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), the
|
||||
[Hopper and Ghidra setup](docs/getting-started/installation.md#hopper), the
|
||||
[IDA guide](docs/providers/ida-provider.md) (including an optional native MCP bridge), the
|
||||
[Kuna decompiler guide](docs/kuna-provider.md), and
|
||||
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
|
||||
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
|
||||
[experimental Windows Ghidra support](docs/providers/windows-ghidra-p0.md).
|
||||
Ghidra also supports [16-bit DOS analysis](docs/providers/ghidra-dos.md).
|
||||
For large binaries, raise the provider's startup deadline with
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS` or `REA_HOPPER_STARTUP_TIMEOUT_MS`.
|
||||
For provider selection, see the [CLI guide](docs/cli.md#choose-a-provider).
|
||||
Check [release availability](docs/installation.md#released-package-and-main)
|
||||
For provider selection, see the [CLI guide](docs/getting-started/cli.md#choose-a-provider).
|
||||
Check [release availability](docs/getting-started/installation.md#released-package-and-main)
|
||||
for features added since the latest npm release.
|
||||
|
||||
## Showcases
|
||||
@@ -335,14 +338,20 @@ REA exports a runnable environment with a basic test to check that it starts and
|
||||
[Environment reference](docs/agent-environments.md) ·
|
||||
[Watch the demo on X](https://x.com/reatools/status/2112574822584918075?s=20)
|
||||
|
||||
### Desktop app environments
|
||||
|
||||
REA can also build an environment of a Linux desktop app, which an agent uses through screenshots, the mouse and the keyboard. From a one-line request, such as "Build me a Gnote environment where the agent adds olive oil to the shopping list note", your agent installs the app on a local build computer, writes the task, exports the environment and checks the task in a fresh computer. The machine that builds it needs Linux on amd64 or arm64, Docker and the agentenv framework version the guide pins.
|
||||
|
||||
[Set up desktop app environments](docs/desktop-build-computer.md)
|
||||
|
||||
## FAQ
|
||||
|
||||
<details>
|
||||
<summary><strong>Which agents can use REA?</strong></summary>
|
||||
|
||||
Any agent that supports local MCP servers. Setup configures the
|
||||
[supported agents](docs/installation.md#supported-agents); other clients can use
|
||||
[manual MCP registration](docs/installation.md#mcp-registry).
|
||||
[supported agents](docs/getting-started/installation.md#supported-agents); other clients can use
|
||||
[manual MCP registration](docs/getting-started/installation.md#mcp-registry).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -352,7 +361,7 @@ Any agent that supports local MCP servers. Setup configures the
|
||||
Deep native analysis uses Hopper, Ghidra, IDA or Kuna, depending on the target
|
||||
and operation. Static JavaScript and .NET inspection work without a native
|
||||
analysis engine. Setup can install Hopper after approval; Ghidra, IDA and Kuna
|
||||
use your existing installations. See [provider setup](docs/installation.md#hopper)
|
||||
use your existing installations. See [provider setup](docs/getting-started/installation.md#hopper)
|
||||
for the options and their supported targets.
|
||||
|
||||
</details>
|
||||
@@ -364,7 +373,7 @@ REA starts Hopper when an operation needs it. On macOS, a first-run dialog may
|
||||
ask you to choose demo mode or activate your license. Complete that prompt in the
|
||||
same user session before your first CLI/MCP operation or unattended job. Open
|
||||
Hopper manually if needed. See
|
||||
[Hopper startup and troubleshooting](docs/installation.md#launcher-paths-and-troubleshooting).
|
||||
[Hopper startup and troubleshooting](docs/getting-started/installation.md#launcher-paths-and-troubleshooting).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -373,7 +382,7 @@ Hopper manually if needed. See
|
||||
|
||||
The skill supplies investigation instructions for your agent. Use `npx rea-agents setup`
|
||||
to register REA's MCP server and install the matching instructions, then restart
|
||||
your agent. See [skill-only installation](docs/installation.md#skill-only-installation).
|
||||
your agent. See [skill-only installation](docs/getting-started/installation.md#skill-only-installation).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -422,12 +431,12 @@ with your REA version, target type, steps to reproduce and error output.
|
||||
Start with the website's [worked guides](https://rea.tools/guides/).
|
||||
For exact options, prerequisites and result contracts:
|
||||
|
||||
- [Installation and setup](docs/installation.md): agent registration, provider configuration, updates and uninstall.
|
||||
- [Readiness and troubleshooting](docs/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
|
||||
- [CLI and Evidence](docs/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
|
||||
- [MCP contracts](docs/mcp-contracts.md) and [agent prompts](docs/mcp-prompts.md): tool results, sessions and guided investigations.
|
||||
- [Tool catalog](docs/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
|
||||
- [Roadmap](docs/roadmap.md): planned work and capability trackers.
|
||||
- [Installation and setup](docs/getting-started/installation.md): agent registration, provider configuration, updates and uninstall.
|
||||
- [Readiness and troubleshooting](docs/getting-started/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
|
||||
- [CLI and Evidence](docs/getting-started/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
|
||||
- [MCP contracts](docs/reference/mcp-contracts.md) and [agent prompts](docs/getting-started/mcp-prompts.md): tool results, sessions and guided investigations.
|
||||
- [Tool catalog](docs/reference/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
|
||||
- [Roadmap](docs/development/roadmap.md): planned work and capability trackers.
|
||||
|
||||
Report vulnerabilities through [SECURITY.md](SECURITY.md).
|
||||
|
||||
@@ -462,8 +471,8 @@ report a bug or suggest a feature, or [send a pull request](https://github.com/m
|
||||
to improve the code or docs.
|
||||
|
||||
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
|
||||
[testing](docs/testing.md) for verification lanes, and the
|
||||
[architecture map](docs/architecture.mermaid) for selected layers and dependency paths.
|
||||
[testing](docs/development/testing.md) for verification lanes, and the
|
||||
[architecture map](docs/architecture/architecture.mermaid) for selected layers and dependency paths.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
+111
-27
@@ -4,6 +4,117 @@
|
||||
|
||||
Security fixes are provided for the latest published version of REA. Upgrade to the latest release before reporting a problem that may already have been resolved.
|
||||
|
||||
## Security boundary
|
||||
|
||||
REA is a local reverse-engineering and automation tool. CLI and MCP use the
|
||||
same application workflows and run with the host process's permissions.
|
||||
REA is not a general-purpose sandbox or a multi-tenant authorization service.
|
||||
|
||||
### Threat model
|
||||
|
||||
- **Trusted authority:** the operating-system account running REA, the caller
|
||||
or agent host that selects operations, and configured analysis providers,
|
||||
their executables and extensions, and service endpoints. An MCP client can
|
||||
invoke the operations exposed to it; its host must enforce the user's tool
|
||||
permissions and approvals.
|
||||
- **Untrusted data:** target binaries, archives, source files, project contents,
|
||||
captures, web pages and returned target content can be malicious. Selecting
|
||||
an input for static inspection does not authorize commands embedded in that
|
||||
input. Agents must treat target text and Evidence as data rather
|
||||
than instructions granting additional authority.
|
||||
- **Operation-specific effects:** static inspection parses target data and may
|
||||
launch an analysis engine. Runtime, debugger, instrumentation, browser,
|
||||
desktop and environment workflows can deliberately execute code, attach to
|
||||
processes, make network requests, write files or interact with applications
|
||||
and devices. Review the selected operation's effects and limitations; static
|
||||
inspection does not authorize launching or interacting with the target
|
||||
application. See
|
||||
[runtime and network responsibilities](DISCLAIMER.md#runtime-network-and-device-operations).
|
||||
|
||||
Standard client registrations use a dedicated stdio process. Opt-in
|
||||
[HTTP MCP](docs/reference/mcp-contracts.md#transports) uses plain HTTP without
|
||||
TLS or client authentication and shares provider/active-target state across
|
||||
protocol sessions. Its session ID is not a user-authentication or isolation
|
||||
boundary. Keep this endpoint within a trusted local or isolated environment.
|
||||
|
||||
### Intended security properties
|
||||
|
||||
Static JavaScript and .NET inspection do not run the target application.
|
||||
Unexpected execution of input code through a data-only analysis path is a
|
||||
security concern. Analysis engines themselves are executable software trusted
|
||||
with the host account's permissions; parsing an input in an external engine
|
||||
does not isolate that engine from the host.
|
||||
|
||||
REA authenticates its managed Hopper and Ghidra bridges with per-session random
|
||||
capability tokens transferred through private session files, rather than
|
||||
process arguments or environment variables. POSIX uses current-user Unix
|
||||
sockets. Windows Ghidra uses authenticated loopback TCP and a private NTFS
|
||||
runtime whose DACL admits only the current user and SYSTEM. These controls
|
||||
protect the managed session boundary, not every external provider or debugger
|
||||
endpoint. See [Windows Ghidra transport and lifecycle](docs/providers/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
|
||||
|
||||
Provider ownership and mutation rules remain specific to each workflow:
|
||||
|
||||
- Ghidra's working projects are temporary and owned by REA. Opt-in
|
||||
[checkpoints](docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program)
|
||||
retain REA-owned Program state. A
|
||||
[caller-selected existing project](docs/getting-started/installation.md#reuse-an-existing-ghidra-project)
|
||||
is fingerprinted and copied into an owned runtime for read-only processing;
|
||||
Ghidra does not open or save the original database through that workflow.
|
||||
- [IDA analysis](docs/providers/ida-provider.md) is read-only. An attached GUI
|
||||
database is never saved or closed by REA; owned headless sessions have a
|
||||
separate lifecycle.
|
||||
- Cleanup must stay within resources REA owns. Failed or unverified cleanup
|
||||
can retain a workspace or session for retry; closing a session
|
||||
does not imply that every investigation artifact has been erased.
|
||||
|
||||
`rea setup` shows planned configuration and installation changes and requires
|
||||
approval before applying them. Ghidra, Java and IDA are user-managed. Approved
|
||||
Linux Hopper installation authenticates the package's detached PGP signature
|
||||
against the pinned publisher identity and checks the signed launcher against
|
||||
the selected release's digest. The release feed's display version is separate
|
||||
from that authenticated identity. See [installation and publisher verification](docs/getting-started/installation.md#hopper).
|
||||
|
||||
### Limits and caller responsibilities
|
||||
|
||||
- **Host isolation:** private runtimes and capability tokens do not protect
|
||||
against malicious software already running as the same operating-system
|
||||
user, or a compromised trusted client or provider. Use an appropriately
|
||||
isolated account, VM or restricted environment when handling hostile targets.
|
||||
- **Resource bounds:** timeouts, cancellation, streaming and size limits are
|
||||
operation-specific. They are not a universal CPU, memory, disk or wall-time
|
||||
limit on every target and third-party tool. Enforce hard host resource limits
|
||||
externally when needed. Failures of a documented boundary or cancellation
|
||||
control still warrant investigation.
|
||||
- **Data and network exposure:** Evidence, caches, checkpoints, exports and
|
||||
captures can retain private target data, paths, URLs and memory contents.
|
||||
Local execution does not make these outputs safe to publish. Requested
|
||||
network operations, agent hosts and provider features such as
|
||||
[IDA Free's cloud decompiler](docs/providers/ida-provider.md) can communicate
|
||||
outside the host. Review their data handling and secure any exposed endpoint.
|
||||
- **Interpretation:** analysis, reconstruction and generated environments can
|
||||
be incomplete or incorrect. Identity checks, signatures and successful
|
||||
analysis do not establish that a target, provider or generated program is
|
||||
safe to execute. Review the Evidence and proposed actions before relying on
|
||||
them.
|
||||
|
||||
### Vulnerability scope
|
||||
|
||||
Please report suspected violations of these boundaries privately. Examples
|
||||
include unintended code execution during data-only inspection, escaping a
|
||||
documented extraction/output boundary, bypassing bridge authentication or
|
||||
target/session identity checks, unauthorized effects on an unrelated resource,
|
||||
and disclosure of transport credentials or explicitly marked sensitive values.
|
||||
An attacker-triggered bypass of a documented resource or cancellation control
|
||||
can also be security-relevant without code execution.
|
||||
|
||||
Incorrect analysis, unsupported formats, crashes, hangs and high resource use
|
||||
without a security-boundary impact are generally ordinary bug reports. Effects
|
||||
explicitly requested through an execution or interaction workflow are not by
|
||||
themselves a vulnerability. A dependency advisory needs its actual REA exposure
|
||||
assessed; a complete exploit or diagnosis is not required to report a concern.
|
||||
If you are unsure whether a boundary is affected, use the private channel below.
|
||||
|
||||
## Reporting a vulnerability
|
||||
|
||||
Do not disclose vulnerabilities, proof-of-concept exploits, credentials,
|
||||
@@ -21,30 +132,3 @@ unrelated local paths from logs or examples.
|
||||
The maintainer will acknowledge the report, assess its scope, and coordinate
|
||||
remediation and disclosure timing with the reporter. Please allow time for
|
||||
provider-dependent behavior to be reproduced safely.
|
||||
|
||||
## Security boundary
|
||||
|
||||
REA authenticates its Hopper and Ghidra bridges with per-session random
|
||||
capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses
|
||||
authenticated loopback TCP with a private NTFS runtime whose DACL admits only
|
||||
the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
|
||||
Tokens are transferred through private session descriptors rather than process
|
||||
arguments or environment variables.
|
||||
This is not a sandbox and does not protect against malicious processes already
|
||||
running as the same operating-system user. Opening an untrusted binary
|
||||
delegates parsing and analysis to the selected local provider with that user's
|
||||
permissions.
|
||||
|
||||
Ghidra sessions use a packaged Java `HeadlessScript`, an isolated temporary
|
||||
project, and private home, cache, and runtime directories. REA imports only the
|
||||
requested target, bounds startup and protocol messages, requests bounded CPU
|
||||
and heap settings, and deletes the temporary project when the session closes.
|
||||
These controls limit accidental persistence and resource use; they do not make
|
||||
Ghidra a sandbox. REA does not open or modify user-owned Ghidra projects.
|
||||
|
||||
`GHIDRA_INSTALL_DIR` and optional `JAVA_HOME` identify user-managed
|
||||
installations. Setup can add those values to detected client configuration only
|
||||
after showing its plan and receiving approval. REA does not download, install,
|
||||
upgrade, or modify Ghidra or Java.
|
||||
|
||||
On Linux, REA accepts release metadata only from Hopper's HTTPS endpoint, restricts package URLs to Hopper's public download origin, bounds the package size, and compares the downloaded bytes with Hopper's published checksum before requesting installation. The published SHA-1 value is a corruption check, not a modern package signature; HTTPS origin validation remains part of the trust boundary. Dependency installation is delegated without shell evaluation to `apt-get`, `dnf`, or `pacman`, directly as root or through `pkexec`. REA never invokes `sudo`.
|
||||
+21
-21
@@ -54,27 +54,27 @@ REA's adapters do not grant, replace, or waive these licenses. The upstream
|
||||
executables and their dependency graphs are supplied separately. Git submodules
|
||||
pin inspection sources; they are not included in the npm package.
|
||||
|
||||
| Project | License and source | REA relationship |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
|
||||
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
|
||||
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
|
||||
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
|
||||
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
|
||||
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
|
||||
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
|
||||
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
|
||||
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
|
||||
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
|
||||
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
|
||||
| Project | License and source | REA relationship |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/getting-started/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
|
||||
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
|
||||
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
|
||||
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
|
||||
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
|
||||
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
|
||||
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
|
||||
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
|
||||
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
|
||||
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
|
||||
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
|
||||
|
||||
Ghidra's original notice credits software developed at the National Security
|
||||
Agency. It explains that US-government-created portions are not subject to US
|
||||
|
||||
+267
-13
@@ -1,7 +1,9 @@
|
||||
"""Run REA inside an agentenv build computer for REA on the host: capture the app, draft its task and export it.
|
||||
|
||||
REA sends this file's source through the computer's ``computer-exec/v1`` route as
|
||||
``python3 -I -c <source> <command> <JSON arguments>``; each command prints one JSON line and exits. ``start``
|
||||
``python3 -I -c <source> <command> <JSON arguments>``. JSON that would exceed the bytes Linux
|
||||
passes as one string travels as several arguments, which this file joins before parsing. Each command prints
|
||||
one JSON line and exits. ``start``
|
||||
leaves the capture running on its own: its stdin is a FIFO it holds open for reading and writing, so it never
|
||||
sees the end of its input, and ``mark`` and ``stop`` write their requests there and wait for the capture's own
|
||||
acknowledgement on its progress log. A small shell around the capture writes its exit status to a file, the
|
||||
@@ -91,6 +93,13 @@ def _exit_code(capture: str):
|
||||
return None
|
||||
|
||||
|
||||
def _ended(capture: str, code: int) -> dict:
|
||||
"""A capture that has exited: its status, the end of its progress log and its output's size, which REA reads
|
||||
whole for the Evidence or failure it ended with, whichever step was waiting on it."""
|
||||
return {"exited": code, "progress": _tail(os.path.join(capture, "progress.log")),
|
||||
"output_bytes": os.path.getsize(os.path.join(capture, "capture.out"))}
|
||||
|
||||
|
||||
def _event_line(path: str, offset: int, kinds: set, name):
|
||||
"""The first progress line after ``offset`` whose event is of one of ``kinds``, for ``name`` when given."""
|
||||
try:
|
||||
@@ -119,17 +128,49 @@ def _wait(capture: str, offset: int, kinds: set, name, timeout: float) -> dict:
|
||||
return {"line": line}
|
||||
code = _exit_code(capture)
|
||||
if code is not None:
|
||||
return {"exited": code, "progress": _tail(log), "output": _tail(os.path.join(capture, "capture.out"))}
|
||||
return _ended(capture, code)
|
||||
if time.monotonic() > deadline:
|
||||
return {"timed_out": True, "progress": _tail(log)}
|
||||
time.sleep(POLL_S)
|
||||
|
||||
|
||||
def _within(path: str, folder: str) -> bool:
|
||||
return path == folder or path.startswith(folder + os.sep)
|
||||
|
||||
|
||||
def _state_paths(values: list, work: str):
|
||||
"""Each state path as an absolute path strictly under the home and outside REA's work folder, both as named
|
||||
and with its links resolved, or the refusal of the first that is not. The capture resolves the links again
|
||||
and retains what they lead to, so a link out of the home is refused here. Each passes on as named, so the
|
||||
capture keeps what its spelling means, such as a hidden settings folder."""
|
||||
home = os.path.normpath(os.path.expanduser("~"))
|
||||
real_home, real_work = os.path.realpath(home), os.path.realpath(work)
|
||||
spellings = {home, real_home}
|
||||
paths = []
|
||||
for value in values:
|
||||
relative = value[2:] if value.startswith("~/") else value
|
||||
path = os.path.normpath(os.path.join(home, relative))
|
||||
if ".." in relative.split("/") or not any(path.startswith(spelling + os.sep) for spelling in spellings):
|
||||
return None, f"The state path {value} is not under the computer's home, {home}."
|
||||
# Every link on the way is followed, a dangling one too; a path that does not exist yet resolves
|
||||
# through its nearest existing folder.
|
||||
resolved = os.path.realpath(path)
|
||||
if not resolved.startswith(real_home + os.sep):
|
||||
return None, f"The state path {value} leads through a link to {resolved}, which is not under the computer's home, {home}."
|
||||
if _within(path, work) or _within(resolved, real_work):
|
||||
return None, f"The state path {value} is in REA's work folder {work}, which every capture leaves out."
|
||||
paths.append(path)
|
||||
return paths, None
|
||||
|
||||
|
||||
def start(arguments: dict) -> int:
|
||||
work, capture = os.path.abspath(arguments["work"]), os.path.abspath(arguments["capture"])
|
||||
rea = shutil.which("rea")
|
||||
if rea is None:
|
||||
return _say({"error": "REA is not installed in this computer; start the computer from a request, which installs it."})
|
||||
state_paths, refusal = _state_paths(arguments.get("state_paths", []), work)
|
||||
if refusal is not None:
|
||||
return _say({"refused": refusal})
|
||||
if os.path.exists(capture):
|
||||
return _say({"error": f"the capture folder {capture} is already used"})
|
||||
apt = _apt(work, arguments.get("apt_timeout_s", 600))
|
||||
@@ -143,20 +184,38 @@ def start(arguments: dict) -> int:
|
||||
"--output", os.path.join(capture, "capture"), "--exclude", work, "--json"]
|
||||
for value in arguments.get("arguments", []):
|
||||
argv += ["--argument", value]
|
||||
for path in state_paths:
|
||||
argv += ["--state-path", path]
|
||||
if arguments.get("package_file"):
|
||||
argv += ["--package-file", arguments["package_file"]]
|
||||
# Reading and writing, so the capture never reads the end of its input between requests.
|
||||
reader = os.open(control, os.O_RDWR)
|
||||
process = None
|
||||
try:
|
||||
with open(os.path.join(capture, "capture.out"), "wb") as out, \
|
||||
open(os.path.join(capture, "progress.log"), "wb") as progress:
|
||||
process = subprocess.Popen(
|
||||
["sh", "-c", 'done=$1; shift; "$@"; code=$?; printf "%s" "$code" > "$done.tmp" && mv "$done.tmp" "$done"',
|
||||
"sh", os.path.join(capture, "exit"), *argv],
|
||||
cwd=os.path.expanduser("~"), env=_environment(work), stdin=reader, stdout=out, stderr=progress,
|
||||
start_new_session=True)
|
||||
try:
|
||||
with open(os.path.join(capture, "capture.out"), "wb") as out, \
|
||||
open(os.path.join(capture, "progress.log"), "wb") as progress:
|
||||
process = subprocess.Popen(
|
||||
["sh", "-c", 'done=$1; shift; "$@"; code=$?; printf "%s" "$code" > "$done.tmp" && mv "$done.tmp" "$done"',
|
||||
"sh", os.path.join(capture, "exit"), *argv],
|
||||
cwd=os.path.expanduser("~"), env=_environment(work), stdin=reader, stdout=out, stderr=progress,
|
||||
start_new_session=True)
|
||||
except Exception as cause:
|
||||
# Nothing will write exit. Record one and drop the FIFO, so stop does not wait on it.
|
||||
try:
|
||||
with open(os.path.join(capture, "exit"), "w", encoding="utf-8") as handle:
|
||||
handle.write("1")
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
os.unlink(control)
|
||||
except OSError:
|
||||
pass
|
||||
return _say({"error": f"the capture did not start: {cause}"})
|
||||
finally:
|
||||
os.close(reader)
|
||||
if process is None:
|
||||
return _say({"error": "the capture did not start"})
|
||||
waited = _wait(capture, 0, {"window_open"}, None, arguments.get("timeout_s", 120))
|
||||
return _say({"pid": process.pid, "apt": apt, **waited})
|
||||
|
||||
@@ -175,11 +234,13 @@ def _request(capture: str, line: str) -> bool:
|
||||
|
||||
|
||||
def mark(arguments: dict) -> int:
|
||||
"""Request a checkpoint and answer with the capture's acknowledgement line as it wrote it: the checkpoint's
|
||||
counts and up to ten of the files that changed, home-relative, with a count of the rest."""
|
||||
capture, name = os.path.abspath(arguments["capture"]), arguments["name"]
|
||||
code = _exit_code(capture)
|
||||
log = os.path.join(capture, "progress.log")
|
||||
if code is not None:
|
||||
return _say({"exited": code, "progress": _tail(log)})
|
||||
return _say(_ended(capture, code))
|
||||
# A checkpoint already taken, whose answer was lost, is answered from the capture's own log.
|
||||
taken = _event_line(log, 0, {"checkpoint"}, name)
|
||||
if taken is not None:
|
||||
@@ -213,6 +274,7 @@ def run(arguments: dict) -> int:
|
||||
rea = shutil.which("rea")
|
||||
if rea is None:
|
||||
return _say({"error": "REA is not installed in this computer"})
|
||||
os.makedirs(os.path.dirname(arguments["stdout"]), exist_ok=True)
|
||||
for path, text in arguments.get("files", {}).items():
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "w", encoding="utf-8") as handle:
|
||||
@@ -228,6 +290,187 @@ def run(arguments: dict) -> int:
|
||||
"stderr": done.stderr[-TAIL_BYTES:].decode("utf-8", "replace")})
|
||||
|
||||
|
||||
# The controller is sent with python -c, so a detached supervisor needs its
|
||||
# own source. It alone holds the child process handle and signals its group.
|
||||
LONG_RUN = r'''
|
||||
import json, os, signal, subprocess, sys, threading, time
|
||||
job = sys.argv[1]
|
||||
result = {}
|
||||
child = None
|
||||
drainer = None
|
||||
stderr_tail, stderr_errors = bytearray(), []
|
||||
|
||||
def drain(stream):
|
||||
try:
|
||||
while chunk := stream.read1(65536):
|
||||
stderr_tail.extend(chunk)
|
||||
del stderr_tail[:-4096]
|
||||
except OSError as error:
|
||||
stderr_errors.append(str(error))
|
||||
finally:
|
||||
try:
|
||||
stream.close()
|
||||
except OSError as error:
|
||||
stderr_errors.append(str(error))
|
||||
|
||||
def timed_out_marker():
|
||||
try:
|
||||
open(os.path.join(job, "timed_out"), "x").close()
|
||||
except FileExistsError:
|
||||
pass
|
||||
|
||||
try:
|
||||
request = json.loads("".join(sys.argv[2:]))
|
||||
if os.path.exists(os.path.join(job, "cancel")):
|
||||
result = {"absent": True}
|
||||
else:
|
||||
with open(request["stdout"], "wb") as out:
|
||||
child = subprocess.Popen(request["argv"], cwd=request["home"], stdin=subprocess.DEVNULL,
|
||||
stdout=out, stderr=subprocess.PIPE, start_new_session=True)
|
||||
drainer = threading.Thread(target=drain, args=(child.stderr,), daemon=True)
|
||||
drainer.start()
|
||||
deadline = time.monotonic() + request["timeout_s"]
|
||||
interrupted = False
|
||||
timed_out = False
|
||||
while child.poll() is None:
|
||||
timed_out = timed_out or time.monotonic() >= deadline
|
||||
if not interrupted and (timed_out or os.path.exists(os.path.join(job, "cancel"))):
|
||||
interrupted = True
|
||||
if timed_out:
|
||||
timed_out_marker()
|
||||
try:
|
||||
os.killpg(child.pid, signal.SIGINT)
|
||||
except OSError:
|
||||
pass
|
||||
time.sleep(0.2)
|
||||
# A descendant can still hold stderr after the leader exits. Do
|
||||
# not publish completion until that stream has actually closed.
|
||||
while drainer.is_alive():
|
||||
drainer.join(0.2)
|
||||
if time.monotonic() >= deadline:
|
||||
timed_out = True
|
||||
timed_out_marker()
|
||||
result = {"exit_code": child.returncode, "stdout_bytes": os.path.getsize(request["stdout"]),
|
||||
"timed_out": timed_out, "cancelled": interrupted and not timed_out,
|
||||
"stderr": bytes(stderr_tail).decode("utf-8", "replace")}
|
||||
if stderr_errors:
|
||||
result["error"] = f"The REA command's stderr could not be read: {stderr_errors[0]}"
|
||||
except Exception as error:
|
||||
result["error"] = f"The REA command supervisor failed: {error}"
|
||||
# Keep the child handle until it exits, including when cancellation arrives
|
||||
# after an unexpected supervisor error.
|
||||
while child is not None and child.poll() is None:
|
||||
if os.path.exists(os.path.join(job, "cancel")):
|
||||
try:
|
||||
os.killpg(child.pid, signal.SIGINT)
|
||||
except OSError:
|
||||
pass
|
||||
time.sleep(0.2)
|
||||
if drainer is not None and drainer.ident is not None:
|
||||
drainer.join()
|
||||
if child is not None:
|
||||
result.setdefault("exit_code", child.returncode)
|
||||
result.setdefault("stderr", bytes(stderr_tail).decode("utf-8", "replace"))
|
||||
try:
|
||||
result.setdefault("stdout_bytes", os.path.getsize(request["stdout"]))
|
||||
except OSError:
|
||||
pass
|
||||
if child is None or child.poll() is not None:
|
||||
temporary = os.path.join(job, "result.tmp")
|
||||
with open(temporary, "w", encoding="utf-8") as out:
|
||||
json.dump(result, out)
|
||||
os.replace(temporary, os.path.join(job, "result.json"))
|
||||
'''
|
||||
|
||||
|
||||
def run_begin(arguments: dict) -> int:
|
||||
"""Start one long REA command in its own capture-owned job folder."""
|
||||
rea = shutil.which("rea")
|
||||
if rea is None:
|
||||
return _say({"error": "REA is not installed in this computer"})
|
||||
job = os.path.abspath(arguments["job"])
|
||||
os.makedirs(os.path.dirname(job), exist_ok=True)
|
||||
try:
|
||||
os.mkdir(job, 0o700)
|
||||
except FileExistsError:
|
||||
return _say({"refused": "the command job folder is already used"})
|
||||
process = None
|
||||
try:
|
||||
with open(os.path.join(job, "owner"), "x", encoding="ascii") as out:
|
||||
out.write(arguments["token"])
|
||||
for path, text in arguments.get("files", {}).items():
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "w", encoding="utf-8") as handle:
|
||||
handle.write(text)
|
||||
os.makedirs(os.path.dirname(arguments["stdout"]), exist_ok=True)
|
||||
request = json.dumps({"argv": [rea, *arguments["argv"]], "stdout": arguments["stdout"],
|
||||
"home": os.path.expanduser("~"), "timeout_s": arguments["timeout_s"]})
|
||||
chunks = [request[i:i + 60000] for i in range(0, len(request), 60000)]
|
||||
with open(os.path.join(job, "supervisor.log"), "wb") as log:
|
||||
process = subprocess.Popen([sys.executable, "-I", "-c", LONG_RUN, job, *chunks], cwd=os.path.expanduser("~"),
|
||||
env=_environment(os.path.abspath(arguments["work"])), stdin=subprocess.DEVNULL,
|
||||
stdout=log, stderr=log, start_new_session=True)
|
||||
except (OSError, ValueError) as error:
|
||||
stage = "did not start" if process is None else "started but its setup did not finish"
|
||||
result = {"error": f"The REA command supervisor {stage}: {error}"}
|
||||
if process is None:
|
||||
temporary = os.path.join(job, "result.tmp")
|
||||
with open(temporary, "w", encoding="utf-8") as out:
|
||||
json.dump(result, out)
|
||||
os.replace(temporary, os.path.join(job, "result.json"))
|
||||
return _say(result)
|
||||
return _say({"started": True})
|
||||
|
||||
|
||||
def _run_result(job: str):
|
||||
try:
|
||||
with open(os.path.join(job, "result.json"), encoding="utf-8") as source:
|
||||
return json.load(source)
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
|
||||
|
||||
def _owns_run(job: str, token: str) -> bool:
|
||||
try:
|
||||
with open(os.path.join(job, "owner"), encoding="ascii") as owner:
|
||||
return owner.read() == token
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def run_poll(arguments: dict) -> int:
|
||||
job = os.path.abspath(arguments["job"])
|
||||
if not _owns_run(job, arguments["token"]):
|
||||
return _say({"error": "the command job ownership is not confirmed"})
|
||||
deadline = time.monotonic() + arguments.get("timeout_s", 30)
|
||||
while (result := _run_result(job)) is None and time.monotonic() < deadline:
|
||||
time.sleep(POLL_S)
|
||||
if result is None:
|
||||
pending = {"pending": True, "timed_out": os.path.exists(os.path.join(job, "timed_out"))}
|
||||
progress = _tail(os.path.join(job, "supervisor.log"))
|
||||
return _say({**pending, **({"progress": progress} if progress else {})})
|
||||
return _say({"finished": True, **result})
|
||||
|
||||
|
||||
def run_cancel(arguments: dict) -> int:
|
||||
job = os.path.abspath(arguments["job"])
|
||||
os.makedirs(os.path.dirname(job), exist_ok=True)
|
||||
try:
|
||||
os.mkdir(job, 0o700)
|
||||
except FileExistsError:
|
||||
pass
|
||||
else:
|
||||
# A late begin cannot launch into a folder cancellation claimed.
|
||||
with open(os.path.join(job, "owner"), "x", encoding="ascii") as owner:
|
||||
owner.write(arguments["token"])
|
||||
return _say({"absent": True})
|
||||
if not _owns_run(job, arguments["token"]):
|
||||
return _say({"error": "the command job ownership is not confirmed"})
|
||||
with open(os.path.join(job, "cancel"), "a", encoding="utf-8"):
|
||||
pass
|
||||
return run_poll(arguments)
|
||||
|
||||
|
||||
def listing(arguments: dict) -> int:
|
||||
"""Every regular file under a folder, by its path relative to the home, with its size and SHA-256.
|
||||
|
||||
@@ -236,8 +479,18 @@ def listing(arguments: dict) -> int:
|
||||
"""
|
||||
home = os.path.expanduser("~")
|
||||
files, skipped = [], []
|
||||
for directory, folders, names in os.walk(os.path.abspath(arguments["root"])):
|
||||
|
||||
def fail_walk(error: OSError) -> None:
|
||||
raise error
|
||||
|
||||
for directory, folders, names in os.walk(os.path.abspath(arguments["root"]), onerror=fail_walk):
|
||||
folders.sort()
|
||||
# os.walk does not follow directory symlinks; disclose them to the consumer.
|
||||
for name in folders[:]:
|
||||
path = os.path.join(directory, name)
|
||||
if os.path.islink(path):
|
||||
skipped.append(os.path.relpath(path, home))
|
||||
folders.remove(name)
|
||||
for name in sorted(names):
|
||||
path = os.path.join(directory, name)
|
||||
relative = os.path.relpath(path, home)
|
||||
@@ -256,7 +509,8 @@ def listing(arguments: dict) -> int:
|
||||
return _say({"listing_bytes": os.path.getsize(output)})
|
||||
|
||||
|
||||
COMMANDS = {"start": start, "mark": mark, "stop": stop, "run": run, "list": listing}
|
||||
COMMANDS = {"start": start, "mark": mark, "stop": stop, "run": run, "list": listing,
|
||||
"run-begin": run_begin, "run-poll": run_poll, "run-cancel": run_cancel}
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(COMMANDS[sys.argv[1]](json.loads(sys.argv[2])))
|
||||
sys.exit(COMMANDS[sys.argv[1]](json.loads("".join(sys.argv[2:]))))
|
||||
@@ -0,0 +1,132 @@
|
||||
"""Run one solve of a bundle's task in a fresh agentenv computer for REA and print how it ended.
|
||||
|
||||
REA runs ``solve.py <bundle folder> <task> <model> <agent>`` with the agentenv runner's own interpreter, so
|
||||
``agent_env`` imports. It runs the task once on this machine with ``run_bundle``, the solver deployed as the agent
|
||||
and asked on the model, the endpoint taken from the environment or agentenv's config as a run takes it, and the
|
||||
computer torn down as the run ends. It prints one JSON line: the run's outcome, instance id, seconds, each check's
|
||||
result, the solver's turns and cost from its trajectory when that is a local file, and agentenv's version; or
|
||||
``{"error": ...}`` when the task could not run. Everything else, agentenv's progress included, goes to stderr. It exits
|
||||
0 after a run and 2 when none ran.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import sys
|
||||
from importlib import metadata
|
||||
from pathlib import Path
|
||||
|
||||
#: The most characters of a check's description or a run's error REA keeps.
|
||||
MAX_TEXT = 400
|
||||
|
||||
|
||||
def _protocol():
|
||||
"""A private copy of stdout for the JSON line; fd 1 then points at stderr, as do children that inherit it."""
|
||||
protocol = os.fdopen(os.dup(1), "w", encoding="utf-8")
|
||||
os.dup2(2, 1)
|
||||
sys.stdout = sys.stderr
|
||||
return protocol
|
||||
|
||||
|
||||
def _say(protocol, value: dict) -> None:
|
||||
protocol.write(json.dumps(value) + "\n")
|
||||
protocol.flush()
|
||||
|
||||
|
||||
def _field(value, name: str):
|
||||
"""A field of agentenv's dataclasses, or of the dicts a stored context holds."""
|
||||
return value.get(name) if isinstance(value, dict) else getattr(value, name, None)
|
||||
|
||||
|
||||
def _trajectory_end(context) -> tuple:
|
||||
"""The solver's turns and cost from the last record of its local trajectory, or None for each."""
|
||||
for response in reversed(_field(context, "prompt_responses") or []):
|
||||
url = _field(response, "agent_trajectory_object_url") or ""
|
||||
if not url.startswith("file://"):
|
||||
continue
|
||||
try:
|
||||
records = json.loads(Path(url[len("file://"):]).read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return None, None
|
||||
last = records[-1] if isinstance(records, list) and records and isinstance(records[-1], dict) else {}
|
||||
turns, usd = last.get("num_turns"), last.get("total_cost_usd")
|
||||
return (turns if isinstance(turns, int) else None,
|
||||
float(usd) if isinstance(usd, (int, float)) and not isinstance(usd, bool) else None)
|
||||
return None, None
|
||||
|
||||
|
||||
def _solve_facts(run, task: str) -> tuple[str, list]:
|
||||
"""The selected verifier owns checks and score; run failures still take precedence."""
|
||||
verifications = (_field(run.context, "metadata") or {}).get("verifications") or {}
|
||||
found = verifications.get(task)
|
||||
results = found.get("results") if isinstance(found, dict) else None
|
||||
checks = [{"id": str(result.get("id")), "passed": result.get("result") is True,
|
||||
"description": str(result.get("description", ""))[:MAX_TEXT]}
|
||||
for result in results or [] if isinstance(result, dict)]
|
||||
outcome = run.outcome.value
|
||||
if outcome not in ("failed", "cancelled"):
|
||||
score = found.get("score") if isinstance(found, dict) else None
|
||||
outcome = "unscored"
|
||||
# Match TaskRun's score >= 1 rule without borrowing another verifier's
|
||||
# score or claiming success when no selected checks were observed.
|
||||
if checks and type(score) in (int, float) and math.isfinite(score):
|
||||
outcome = "passed" if score >= 1 else "scored below 1"
|
||||
return outcome, checks
|
||||
|
||||
|
||||
def _version() -> str | None:
|
||||
try:
|
||||
return metadata.version("agentenv-framework")
|
||||
except metadata.PackageNotFoundError:
|
||||
return None
|
||||
|
||||
|
||||
def _cleanup(result) -> dict:
|
||||
"""Only a returned teardown report can confirm what agentenv released."""
|
||||
if result is None:
|
||||
return {"unknown": True, "sandboxes": []}
|
||||
unknown, sandboxes = False, []
|
||||
for run in result.runs:
|
||||
report = getattr(run, "torn_down", None)
|
||||
if report is None:
|
||||
unknown = True
|
||||
else:
|
||||
sandboxes.extend({"id": item.sandbox_id, "type": item.sandbox_type} for item in report.still_up)
|
||||
return {"unknown": unknown, "sandboxes": sandboxes}
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
protocol = _protocol()
|
||||
bundle, task, model, agent = argv[1:5]
|
||||
try:
|
||||
from agent_env.bundle import run_bundle
|
||||
except ImportError as error:
|
||||
_say(protocol, {"error": f"agent_env does not import with {sys.executable}: {error}",
|
||||
"cleanup": {"unknown": False, "sandboxes": []}})
|
||||
return 2
|
||||
try:
|
||||
result = run_bundle(bundle, tasks=[task], model=model, agent=agent, sandbox="local", emulate=False,
|
||||
on_progress=lambda line: print(line, file=sys.stderr, flush=True))
|
||||
except BaseException as error: # noqa: BLE001 - a signal's RunInterrupted too; run_bundle tore its run down
|
||||
_say(protocol, {"error": f"{type(error).__name__}: {error}"[:MAX_TEXT * 4],
|
||||
"cleanup": _cleanup(getattr(error, "result", None))})
|
||||
return 2
|
||||
run = result.runs[0] if result.runs else None
|
||||
if run is None:
|
||||
_say(protocol, {"error": "the bundle ran no task", "cleanup": _cleanup(result)})
|
||||
return 2
|
||||
turns, usd = _trajectory_end(run.context)
|
||||
outcome, checks = _solve_facts(run, task)
|
||||
error = getattr(run, "error", None)
|
||||
_say(protocol, {"outcome": outcome, "instance_id": run.instance_id,
|
||||
"seconds": round(float(run.duration), 1), "checks": checks,
|
||||
"turns": turns, "usd": usd, "agentenv": _version(),
|
||||
"error": None if error is None else f"{type(error).__name__}: {error}"[:MAX_TEXT],
|
||||
"cleanup": _cleanup(result)})
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Recover local sandboxes under one stopped solve's private agentenv root."""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _docker(*args: str) -> str:
|
||||
command = os.environ.get("REA_DOCKER_COMMAND") or "docker"
|
||||
result = subprocess.run([command, *args], capture_output=True, text=True, check=False)
|
||||
if result.returncode:
|
||||
raise RuntimeError(f"Docker {' '.join(args[:2])} failed: {result.stderr[-1000:]}")
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
async def cleanup(root: Path) -> None:
|
||||
# A helper has already stopped. agentenv lists the sandboxes under the root
|
||||
# (AGENT_ENV_LOCAL_SANDBOX_DIR); anything else there remains owned for inspection.
|
||||
if not any(root.iterdir()):
|
||||
return
|
||||
from agent_env.providers.sandbox_providers.local_docker.provider import (
|
||||
LocalSandboxProvider, local_sandbox_ids, remove_local_work_dir,
|
||||
)
|
||||
for sandbox_id in local_sandbox_ids():
|
||||
await LocalSandboxProvider().terminate_sandbox(sandbox_id)
|
||||
# The pinned provider reports incomplete teardown and retains the folder.
|
||||
# Keep REA's independent Docker/Linux checks before releasing that owner.
|
||||
if _docker("ps", "-aq", "--filter", f"name=^agent-{sandbox_id}$"):
|
||||
raise RuntimeError(f"Sandbox {sandbox_id} still has its agent container")
|
||||
if _docker("ps", "-aq", "--filter", f"label=agentenv.sandbox={sandbox_id}"):
|
||||
raise RuntimeError(f"Sandbox {sandbox_id} still has labeled containers")
|
||||
if _docker("network", "ls", "-q", "--filter", f"label=agentenv.sandbox={sandbox_id}"):
|
||||
raise RuntimeError(f"Sandbox {sandbox_id} still has labeled networks")
|
||||
if sys.platform != "linux":
|
||||
raise RuntimeError(f"Cannot verify host processes for sandbox {sandbox_id} on {sys.platform}")
|
||||
from agent_env.providers.sandbox_providers.local_docker.provider import _marked_pids
|
||||
if _marked_pids(sandbox_id):
|
||||
raise RuntimeError(f"Sandbox {sandbox_id} still has marked host processes")
|
||||
if remove_local_work_dir(sandbox_id) is None:
|
||||
raise RuntimeError(f"Could not confirm removal of sandbox {sandbox_id}'s work folder")
|
||||
leftover = sorted(entry.name for entry in root.iterdir())
|
||||
if leftover:
|
||||
raise RuntimeError(f"Unexpected sandbox root entries: {', '.join(leftover)}")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
try:
|
||||
with contextlib.redirect_stdout(sys.stderr):
|
||||
asyncio.run(cleanup(Path(os.environ["AGENT_ENV_LOCAL_SANDBOX_DIR"])))
|
||||
print(json.dumps({"ok": True}))
|
||||
return 0
|
||||
except Exception as error:
|
||||
print(json.dumps({"ok": False, "error": f"{type(error).__name__}: {error}"}))
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,232 @@
|
||||
"""Exercise the capture controller's filesystem inventory boundary."""
|
||||
|
||||
import json
|
||||
import io
|
||||
from contextlib import redirect_stdout
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from bridge.agentenv import capture
|
||||
|
||||
|
||||
CONTROLLER = Path(__file__).with_name("capture.py")
|
||||
|
||||
|
||||
class CaptureListingTest(unittest.TestCase):
|
||||
def run_listing(self, root, output):
|
||||
return subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
str(CONTROLLER),
|
||||
"list",
|
||||
json.dumps({"root": str(root), "output": str(output)}),
|
||||
],
|
||||
text=True,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
def test_directory_symlink_is_disclosed_without_following_it(self):
|
||||
with tempfile.TemporaryDirectory() as parent:
|
||||
parent = Path(parent)
|
||||
root = parent / "root"
|
||||
target = parent / "target"
|
||||
output = parent / "listing.json"
|
||||
root.mkdir()
|
||||
target.mkdir()
|
||||
(root / "present.txt").write_text("present")
|
||||
(target / "outside.txt").write_text("outside")
|
||||
try:
|
||||
(root / "linked").symlink_to(target, target_is_directory=True)
|
||||
except OSError as error:
|
||||
self.skipTest(f"directory symlinks unavailable: {error}")
|
||||
result = self.run_listing(root, output)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
listing = json.loads(output.read_text())
|
||||
self.assertEqual(
|
||||
[item["path"] for item in listing["files"]],
|
||||
[os.path.relpath(root / "present.txt", Path.home())],
|
||||
)
|
||||
self.assertEqual(
|
||||
listing["skipped"], [os.path.relpath(root / "linked", Path.home())]
|
||||
)
|
||||
|
||||
def test_missing_root_does_not_produce_an_empty_successful_inventory(self):
|
||||
with tempfile.TemporaryDirectory() as parent:
|
||||
parent = Path(parent)
|
||||
output = parent / "listing.json"
|
||||
result = self.run_listing(parent / "missing", output)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertFalse(output.exists())
|
||||
|
||||
|
||||
@unittest.skipIf(sys.platform == "win32", "desktop build computers use Linux")
|
||||
class LongRunTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
self.root = Path(self.tmp.name)
|
||||
self.bin = self.root / "bin"
|
||||
self.bin.mkdir()
|
||||
rea = self.bin / "rea"
|
||||
rea.write_text(
|
||||
f"#!{sys.executable}\n"
|
||||
"import os, signal, subprocess, sys, time\n"
|
||||
"signal.signal(signal.SIGINT, lambda *_: sys.exit(130))\n"
|
||||
"if len(sys.argv) > 1 and sys.argv[1] == 'wait':\n"
|
||||
" open(sys.argv[2], 'w').close()\n"
|
||||
" time.sleep(30)\n"
|
||||
"if len(sys.argv) > 1 and sys.argv[1] == 'ignore':\n"
|
||||
" signal.signal(signal.SIGINT, signal.SIG_IGN)\n"
|
||||
" open(sys.argv[2], 'w').close()\n"
|
||||
" while not os.path.exists(sys.argv[3]): time.sleep(0.02)\n"
|
||||
"if len(sys.argv) > 1 and sys.argv[1] == 'stderr':\n"
|
||||
" sys.stderr.write('x' * 100000 + 'TAIL')\n"
|
||||
"if len(sys.argv) > 1 and sys.argv[1] == 'hold':\n"
|
||||
" subprocess.Popen([sys.executable, '-c', 'import os,sys,time\\nwhile not os.path.exists(sys.argv[1]): time.sleep(0.02)', sys.argv[2]], stdout=subprocess.DEVNULL, stderr=sys.stderr, start_new_session=True)\n"
|
||||
" sys.exit(0)\n"
|
||||
"print(os.environ.get('AGENTENV_CONTROL_TOKEN', 'no token'))\n"
|
||||
)
|
||||
rea.chmod(0o755)
|
||||
self.environment = {**os.environ, "HOME": str(self.root), "PATH": f"{self.bin}{os.pathsep}{os.environ.get('PATH', '')}",
|
||||
"AGENTENV_CONTROL_TOKEN": "not-for-the-child"}
|
||||
self.job = self.root / "job"
|
||||
self.output = self.root / "out.json"
|
||||
|
||||
def command(self, name, **values):
|
||||
result = subprocess.run([sys.executable, str(CONTROLLER), name, json.dumps(values)],
|
||||
env=self.environment, text=True, capture_output=True, timeout=10, check=True)
|
||||
return json.loads(result.stdout)
|
||||
|
||||
def begin(self, argv, timeout=60):
|
||||
return self.command("run-begin", job=str(self.job), work=str(self.root), stdout=str(self.output),
|
||||
token="owner-token", argv=argv, files={}, timeout_s=timeout)
|
||||
|
||||
def test_runs_and_reports_its_output(self):
|
||||
self.assertEqual(self.begin([]), {"started": True})
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertEqual(result["exit_code"], 0)
|
||||
self.assertEqual(self.output.read_text().strip(), "no token")
|
||||
|
||||
def test_passes_large_command_without_persisting_its_arguments(self):
|
||||
self.assertEqual(self.begin(["done", *("x" * 4000 for _ in range(20))]), {"started": True})
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertEqual(result["exit_code"], 0)
|
||||
self.assertFalse((self.job / "request.json").exists())
|
||||
|
||||
def test_keeps_only_a_bounded_stderr_tail(self):
|
||||
self.assertEqual(self.begin(["stderr"]), {"started": True})
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertEqual(len(result["stderr"]), 4096)
|
||||
self.assertTrue(result["stderr"].endswith("TAIL"))
|
||||
|
||||
def test_cancels_a_running_child_and_confirms_exit(self):
|
||||
ready = self.root / "ready"
|
||||
self.assertEqual(self.begin(["wait", str(ready)]), {"started": True})
|
||||
import time
|
||||
deadline = time.monotonic() + 5
|
||||
while not ready.exists() and time.monotonic() < deadline:
|
||||
time.sleep(0.02)
|
||||
self.assertTrue(ready.exists())
|
||||
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertTrue(result["cancelled"])
|
||||
self.assertEqual(result["exit_code"], 130)
|
||||
|
||||
def test_cancel_before_late_start_claims_the_job(self):
|
||||
self.assertEqual(self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=1), {"absent": True})
|
||||
self.assertIn("already used", self.begin([])["refused"])
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_cancel_refuses_a_folder_without_the_matching_owner(self):
|
||||
self.job.mkdir()
|
||||
(self.job / "owner").write_text("someone-else")
|
||||
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=1)
|
||||
self.assertIn("ownership is not confirmed", result["error"])
|
||||
self.assertFalse((self.job / "cancel").exists())
|
||||
|
||||
def test_reports_supervisor_output_if_completion_is_unconfirmed(self):
|
||||
self.job.mkdir()
|
||||
(self.job / "owner").write_text("owner-token")
|
||||
(self.job / "supervisor.log").write_text("supervisor failed")
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=0)
|
||||
self.assertEqual(result["progress"], "supervisor failed")
|
||||
self.assertTrue(result["pending"])
|
||||
|
||||
def test_reports_a_timed_out_child_that_has_not_stopped(self):
|
||||
ready, release = self.root / "ready", self.root / "release"
|
||||
self.assertEqual(self.begin(["ignore", str(ready), str(release)], timeout=3), {"started": True})
|
||||
try:
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=4)
|
||||
self.assertTrue(result["pending"])
|
||||
self.assertTrue(result["timed_out"])
|
||||
self.assertTrue(ready.exists())
|
||||
finally:
|
||||
release.touch()
|
||||
self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
|
||||
def test_waits_for_an_inherited_stderr_pipe_after_the_child_exits(self):
|
||||
release = self.root / "release"
|
||||
self.assertEqual(self.begin(["hold", str(release)], timeout=3), {"started": True})
|
||||
try:
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=4)
|
||||
self.assertTrue(result["pending"])
|
||||
self.assertTrue(result["timed_out"])
|
||||
finally:
|
||||
release.touch()
|
||||
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertTrue(result["finished"])
|
||||
|
||||
def test_failure_closing_supervisor_log_after_launch_does_not_claim_exit(self):
|
||||
ready = self.root / "ready"
|
||||
original_open = open
|
||||
original_popen = subprocess.Popen
|
||||
supervisors = []
|
||||
|
||||
class FailingClose:
|
||||
def __init__(self, file):
|
||||
self.file = file
|
||||
|
||||
def __enter__(self):
|
||||
return self.file
|
||||
|
||||
def __exit__(self, *args):
|
||||
self.file.close()
|
||||
raise OSError("log close failed")
|
||||
|
||||
def opening(path, *args, **kwargs):
|
||||
file = original_open(path, *args, **kwargs)
|
||||
return FailingClose(file) if str(path).endswith("supervisor.log") else file
|
||||
|
||||
def spawning(*args, **kwargs):
|
||||
process = original_popen(*args, **kwargs)
|
||||
supervisors.append(process)
|
||||
def cleanup():
|
||||
if process.poll() is None:
|
||||
(self.job / "cancel").touch()
|
||||
process.wait(timeout=5)
|
||||
self.addCleanup(cleanup)
|
||||
return process
|
||||
|
||||
arguments = {"job": str(self.job), "work": str(self.root), "stdout": str(self.output),
|
||||
"token": "owner-token", "argv": ["wait", str(ready)], "files": {}, "timeout_s": 60}
|
||||
with patch.dict(os.environ, self.environment, clear=True), patch("builtins.open", side_effect=opening), patch.object(capture.subprocess, "Popen", side_effect=spawning), redirect_stdout(io.StringIO()) as output:
|
||||
capture.run_begin(arguments)
|
||||
self.assertIn("started but its setup did not finish", json.loads(output.getvalue())["error"])
|
||||
import time
|
||||
deadline = time.monotonic() + 5
|
||||
while not ready.exists() and time.monotonic() < deadline:
|
||||
time.sleep(0.02)
|
||||
self.assertTrue(ready.exists())
|
||||
self.assertFalse((self.job / "result.json").exists())
|
||||
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=5)
|
||||
self.assertTrue(result["cancelled"])
|
||||
supervisors[0].wait(timeout=5)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,72 @@
|
||||
"""Project selected verifier facts without importing or running agentenv."""
|
||||
|
||||
import unittest
|
||||
from types import SimpleNamespace
|
||||
|
||||
from bridge.agentenv.solve import _solve_facts
|
||||
|
||||
|
||||
def run_with(verifications, outcome="passed"):
|
||||
return SimpleNamespace(context={"metadata": {"verifications": verifications}},
|
||||
outcome=SimpleNamespace(value=outcome))
|
||||
|
||||
|
||||
class SolveFactsTest(unittest.TestCase):
|
||||
def test_matching_verifier_success_and_failure(self):
|
||||
for passed, score, outcome in ((True, 1.0, "passed"), (False, 0.0, "scored below 1")):
|
||||
with self.subTest(passed=passed):
|
||||
run = run_with({"wanted": {"results": [{"id": "selected", "result": passed}],
|
||||
"score": score}})
|
||||
self.assertEqual(_solve_facts(run, "wanted"),
|
||||
(outcome, [{"id": "selected", "passed": passed, "description": ""}]))
|
||||
|
||||
def test_missing_empty_and_unscored_selected_verifier_do_not_borrow_success(self):
|
||||
other = {"results": [{"id": "unrelated", "result": True}], "score": 1.0}
|
||||
for selected in (None, {}, {"results": [], "score": 1.0},
|
||||
{"results": [{"id": 0, "result": False}]}):
|
||||
for reversed_order in (False, True):
|
||||
with self.subTest(selected=selected, reversed_order=reversed_order):
|
||||
entries = [("other", other)]
|
||||
if selected is not None:
|
||||
entries.append(("wanted", selected))
|
||||
if reversed_order:
|
||||
entries.reverse()
|
||||
expected = ([{"id": "0", "passed": False, "description": ""}]
|
||||
if selected and selected.get("results") else [])
|
||||
self.assertEqual(_solve_facts(run_with(dict(entries)), "wanted"),
|
||||
("unscored", expected))
|
||||
|
||||
def test_selected_score_owns_outcome_and_checks_preserve_false_and_zero(self):
|
||||
# A weighted/any-pass verifier's recorded score can pass despite a failed
|
||||
# criterion; an unrelated failing score cannot override the selected one.
|
||||
rows = [{"id": 0, "result": False}, {"id": "zero", "result": 0},
|
||||
{"id": "yes", "result": True}]
|
||||
checks = [{"id": "0", "passed": False, "description": ""},
|
||||
{"id": "zero", "passed": False, "description": ""},
|
||||
{"id": "yes", "passed": True, "description": ""}]
|
||||
for score, expected in ((0, "scored below 1"), (0.5, "scored below 1"),
|
||||
(1, "passed"), (1.5, "passed")):
|
||||
with self.subTest(score=score):
|
||||
verifications = {"other": {"score": 0},
|
||||
"wanted": {"results": rows, "score": score}}
|
||||
self.assertEqual(_solve_facts(run_with(verifications, "scored below 1"), "wanted"),
|
||||
(expected, checks))
|
||||
|
||||
def test_failure_and_cancellation_preserve_partial_named_checks(self):
|
||||
for outcome in ("failed", "cancelled"):
|
||||
with self.subTest(outcome=outcome):
|
||||
run = run_with({"wanted": {"results": [{"id": "partial", "result": True}],
|
||||
"score": 1.0}}, outcome)
|
||||
self.assertEqual(_solve_facts(run, "wanted"),
|
||||
(outcome, [{"id": "partial", "passed": True, "description": ""}]))
|
||||
|
||||
def test_non_numeric_or_nonfinite_scores_remain_unscored(self):
|
||||
for score in (None, True, "1", float("nan"), float("inf")):
|
||||
with self.subTest(score=score):
|
||||
run = run_with({"wanted": {"results": [{"id": "yes", "result": True}],
|
||||
"score": score}})
|
||||
self.assertEqual(_solve_facts(run, "wanted")[0], "unscored")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,108 @@
|
||||
"""Exercise cleanup decisions without a live agentenv provider."""
|
||||
import asyncio
|
||||
import tempfile
|
||||
import types
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from bridge.agentenv import solve_cleanup
|
||||
|
||||
|
||||
class SolveCleanupTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.folder = self.root / "agent-env-local-a1b2c3d4-abcdef12"
|
||||
self.folder.mkdir()
|
||||
self.terminated = []
|
||||
self.module = types.ModuleType("agent_env.providers.sandbox_providers.local_docker.provider")
|
||||
test = self
|
||||
|
||||
class Provider:
|
||||
async def terminate_sandbox(self, sandbox_id):
|
||||
test.terminated.append(sandbox_id)
|
||||
|
||||
self.module.LocalSandboxProvider = Provider
|
||||
self.module.local_sandbox_ids = self._listed
|
||||
self.module.remove_local_work_dir = lambda sandbox_id: self._remove(sandbox_id)
|
||||
self.module._marked_pids = lambda _sandbox_id: []
|
||||
self.patch_module = patch.dict("sys.modules", {self.module.__name__: self.module})
|
||||
self.patch_module.start()
|
||||
self.addCleanup(self.patch_module.stop)
|
||||
|
||||
def _listed(self):
|
||||
# As agentenv lists them: `agent-env-<id>-<suffix>` folders, whatever the id's shape.
|
||||
found = set()
|
||||
for entry in self.root.iterdir():
|
||||
sandbox_id, _, suffix = entry.name.removeprefix("agent-env-").rpartition("-")
|
||||
if entry.name.startswith("agent-env-") and sandbox_id and suffix and entry.is_dir():
|
||||
found.add(sandbox_id)
|
||||
return sorted(found)
|
||||
|
||||
def _remove(self, sandbox_id):
|
||||
folder = next(self.root.glob(f"agent-env-{sandbox_id}-*"))
|
||||
folder.rmdir()
|
||||
return folder
|
||||
|
||||
def run_cleanup(self):
|
||||
asyncio.run(solve_cleanup.cleanup(self.root))
|
||||
|
||||
def test_terminates_owned_sandbox_before_removing_its_folder(self):
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
self.run_cleanup()
|
||||
self.assertEqual(self.terminated, ["local-a1b2c3d4"])
|
||||
self.assertFalse(self.folder.exists())
|
||||
|
||||
def test_retains_folder_when_docker_reports_container_then_retries(self):
|
||||
with patch.object(solve_cleanup, "_docker", return_value="still-running"), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
with self.assertRaisesRegex(RuntimeError, "still has its agent container"):
|
||||
self.run_cleanup()
|
||||
self.assertTrue(self.folder.exists())
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
self.run_cleanup()
|
||||
self.assertFalse(self.folder.exists())
|
||||
self.assertEqual(len(self.terminated), 2)
|
||||
|
||||
def test_partial_cleanup_and_late_folder_are_retried(self):
|
||||
later = self.root / "agent-env-local-f1223344-abcdef12"
|
||||
later.mkdir()
|
||||
def docker(*args):
|
||||
return "still-running" if "local-f1223344" in " ".join(args) else ""
|
||||
with patch.object(solve_cleanup, "_docker", side_effect=docker), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
with self.assertRaisesRegex(RuntimeError, "still has its agent container"):
|
||||
self.run_cleanup()
|
||||
self.assertTrue(later.exists())
|
||||
self.assertFalse(self.folder.exists())
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
self.run_cleanup()
|
||||
self.assertEqual(list(self.root.iterdir()), [])
|
||||
|
||||
def test_macos_keeps_folder_when_host_process_cleanup_cannot_be_verified(self):
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "darwin"):
|
||||
with self.assertRaisesRegex(RuntimeError, "Cannot verify host processes"):
|
||||
self.run_cleanup()
|
||||
self.assertTrue(self.folder.exists())
|
||||
|
||||
def test_cleans_up_each_sandbox_agentenv_lists_whatever_its_id_looks_like(self):
|
||||
self.folder.rmdir()
|
||||
listed = self.root / "agent-env-local-a1b2c3d4e5-abcdef12"
|
||||
listed.mkdir()
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
self.run_cleanup()
|
||||
self.assertEqual(self.terminated, ["local-a1b2c3d4e5"])
|
||||
self.assertFalse(listed.exists())
|
||||
|
||||
def test_unexpected_entry_remains_owned(self):
|
||||
self.folder.rmdir()
|
||||
(self.root / "unexpected").mkdir()
|
||||
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
|
||||
with self.assertRaisesRegex(RuntimeError, "Unexpected sandbox root entries: unexpected"):
|
||||
self.run_cleanup()
|
||||
self.assertEqual(self.terminated, [])
|
||||
self.assertTrue((self.root / "unexpected").exists())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -119,9 +119,9 @@ import ghidra.program.util.VarnodeContext;
|
||||
public final class ReaGhidraBridge extends HeadlessScript {
|
||||
// At most 24 KiB after worst-case JSON escaping (six bytes per UTF-16 unit).
|
||||
private static final int PDB_ATTRIBUTE_UTF16_BUDGET = 4096;
|
||||
private static final int MAX_HIGH_PCODE_OPS = 3000;
|
||||
private static final int MAX_HIGH_PCODE_INPUTS_PER_OP = 64;
|
||||
private static final int MAX_HIGH_PCODE_DEF_USE_EDGES = 12000;
|
||||
private static final int DEFAULT_HIGH_PCODE_OPS = 3000;
|
||||
private static final int DEFAULT_HIGH_PCODE_INPUTS_PER_OP = 64;
|
||||
private static final int DEFAULT_HIGH_PCODE_DEF_USE_EDGES = 12000;
|
||||
private static final Gson GSON = new GsonBuilder().serializeNulls().create();
|
||||
private final Map<FileBytes, Boolean> targetSourceIdentity = new IdentityHashMap<>();
|
||||
private String admittedTargetSha256;
|
||||
@@ -179,10 +179,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private List<FunctionEntry> procedureInventory;
|
||||
private List<InventoryItem> stringInventory;
|
||||
private DecompInterface decompiler;
|
||||
private boolean existingProject;
|
||||
// A refused Program may only report its admission failure to the client.
|
||||
private String identityAdmissionFailure;
|
||||
/** Per-function native decompiler limit from the session; 0 leaves it unbounded. */
|
||||
private int decompileTimeoutSeconds;
|
||||
private Integer decompileMaxPayloadMiB;
|
||||
// DecompileProcess converts MiB to bytes using a signed-int left shift.
|
||||
private static final int MAX_DECOMPILE_PAYLOAD_MIB = Integer.MAX_VALUE >> 20;
|
||||
private ValueFlowLimits valueFlowLimits = defaultValueFlowLimits();
|
||||
// DecompileProcess converts seconds to an int millisecond timer delay.
|
||||
private static final int MAX_DECOMPILE_TIMEOUT_SECONDS = Integer.MAX_VALUE / 1000;
|
||||
private final ReaGhidraExtensions analysisExtensions = new ReaGhidraExtensions();
|
||||
@@ -199,6 +204,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
Path descriptorPath = Path.of(arguments[0]);
|
||||
checkpointDirectory = descriptorPath.toAbsolutePath().normalize().getParent();
|
||||
SessionDescriptor descriptor = readDescriptor(descriptorPath);
|
||||
existingProject = descriptor.existingProject;
|
||||
decompileTimeoutSeconds = descriptor.decompileTimeoutSeconds();
|
||||
// Windows retains a native immutable bearer descriptor lease until
|
||||
// runtime_close; POSIX removes it after this read. Normal cleanup removes
|
||||
@@ -424,6 +430,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
decompileTimeoutSeconds = request.decompileTimeoutSeconds() == null
|
||||
? descriptor.decompileTimeoutSeconds() : request.decompileTimeoutSeconds();
|
||||
valueFlowLimits = request.valueFlowLimits();
|
||||
applyDecompilePayloadLimit(request.decompileMaxPayloadMiB());
|
||||
writeSuccess(writer, request.id, handleRequest(request, descriptor));
|
||||
}
|
||||
catch (RequestFailure failure) {
|
||||
@@ -876,6 +884,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
JsonArray blocks = new JsonArray();
|
||||
JsonArray skipped = new JsonArray();
|
||||
JsonArray notLoaded = new JsonArray();
|
||||
JsonArray overlays = new JsonArray();
|
||||
List<ByteSearchBlock> eligible = new ArrayList<>();
|
||||
for (MemoryBlock block : ordered) {
|
||||
monitor.checkCancelled();
|
||||
@@ -883,10 +892,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
Address high = block.getEnd();
|
||||
Address bound = start == null ? end : start;
|
||||
if (bound != null && !bound.getAddressSpace().equals(low.getAddressSpace())) continue;
|
||||
if (bound == null && (!block.isLoaded() || block.isOverlay())) {
|
||||
// Without bounds, search loaded non-overlay memory. Unloaded file
|
||||
// parts and loaded overlays stay reachable through explicit bounds.
|
||||
if (bound == null && !block.isLoaded()) {
|
||||
notLoaded.add(byteSearchCoverage(block, low, high));
|
||||
continue;
|
||||
}
|
||||
if (bound == null && block.isOverlay()) {
|
||||
overlays.add(byteSearchCoverage(block, low, high));
|
||||
continue;
|
||||
}
|
||||
if (start != null && low.compareTo(start) < 0) low = start;
|
||||
if (end != null && high.compareTo(end) > 0) high = end;
|
||||
if (low.compareTo(high) > 0) continue;
|
||||
@@ -960,6 +975,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
searched.addProperty("bytes", covered);
|
||||
searched.add("skipped_uninitialized", skipped);
|
||||
searched.add("skipped_not_loaded", notLoaded);
|
||||
searched.add("skipped_overlays", overlays);
|
||||
JsonObject result = new JsonObject();
|
||||
result.add("matches", matches);
|
||||
result.add("searched", searched);
|
||||
@@ -1028,11 +1044,26 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return result;
|
||||
}
|
||||
|
||||
private void initializeDecompiler() {
|
||||
private DecompileOptions decompileOptions(Integer payloadMiB) {
|
||||
DecompileOptions options = new DecompileOptions();
|
||||
options.setMaxInstructions(Integer.MAX_VALUE);
|
||||
if (payloadMiB != null) options.setMaxPayloadMBytes(payloadMiB);
|
||||
return options;
|
||||
}
|
||||
|
||||
private void applyDecompilePayloadLimit(Integer payloadMiB) {
|
||||
if (java.util.Objects.equals(decompileMaxPayloadMiB, payloadMiB)) return;
|
||||
// DecompInterface supports live setOptions: it updates the existing
|
||||
// DecompileProcess result-size bound before sending native options.
|
||||
if (!decompiler.setOptions(decompileOptions(payloadMiB)))
|
||||
throw new RequestFailure("decompile_options_failed", "Ghidra could not apply the selected native decompiler payload limit");
|
||||
decompiler.flushCache();
|
||||
decompileMaxPayloadMiB = payloadMiB;
|
||||
}
|
||||
|
||||
private void initializeDecompiler() {
|
||||
decompiler = new DecompInterface();
|
||||
decompiler.setOptions(options);
|
||||
decompiler.setOptions(decompileOptions(decompileMaxPayloadMiB));
|
||||
decompiler.toggleCCode(true);
|
||||
decompiler.toggleSyntaxTree(true);
|
||||
decompiler.toggleJumpLoads(true);
|
||||
@@ -1414,8 +1445,29 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
flow.addProperty("conditional", false);
|
||||
flow.addProperty("computed", false);
|
||||
if (instruction == null) {
|
||||
result.addProperty("status", !currentProgram.getMemory().contains(address) ? "outside-memory" : currentProgram.getListing().getInstructionContaining(address) != null
|
||||
? "not-instruction-boundary" : currentProgram.getListing().getDefinedDataContaining(address) != null ? "data" : "not-in-listing");
|
||||
Instruction containing = currentProgram.getListing().getInstructionContaining(address);
|
||||
boolean inMemory = currentProgram.getMemory().contains(address);
|
||||
Data containingData = inMemory && containing == null
|
||||
? currentProgram.getListing().getDefinedDataContaining(address) : null;
|
||||
result.addProperty("status", !inMemory ? "outside-memory" : containing != null
|
||||
? "not-instruction-boundary" : containingData != null ? "data" : "not-in-listing");
|
||||
if (inMemory && containing == null && containingData != null) {
|
||||
JsonObject parentData = new JsonObject();
|
||||
parentData.addProperty("address", canonicalAddress(containingData.getAddress()));
|
||||
parentData.addProperty("length", containingData.getLength());
|
||||
parentData.addProperty("type_id", containingData.getDataType().getPathName());
|
||||
parentData.addProperty("provenance", "ghidra-listing");
|
||||
result.add("containing_data", parentData);
|
||||
}
|
||||
if (containing != null && currentProgram.getMemory().contains(address)) {
|
||||
JsonObject containingInstruction = new JsonObject();
|
||||
containingInstruction.addProperty("address", canonicalAddress(containing.getAddress()));
|
||||
containingInstruction.addProperty("length", containing.getLength());
|
||||
containingInstruction.addProperty("mnemonic", containing.getMnemonicString());
|
||||
containingInstruction.addProperty("raw_disassembly", containing.toString());
|
||||
containingInstruction.addProperty("provenance", "ghidra-listing");
|
||||
result.add("containing_instruction", containingInstruction);
|
||||
}
|
||||
for (String key : List.of("bytes", "length", "mnemonic", "raw_disassembly")) result.add(key, JsonNull.INSTANCE);
|
||||
} else {
|
||||
result.addProperty("status", "decoded");
|
||||
@@ -1825,7 +1877,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
while (operations.hasNext()) {
|
||||
monitor.checkCancelled();
|
||||
PcodeOpAST operation = operations.next();
|
||||
if (selected.size() >= MAX_HIGH_PCODE_OPS) {
|
||||
if (selected.size() >= valueFlowLimits.maxOperations()) {
|
||||
omittedOperationsLowerBound = 1;
|
||||
break;
|
||||
}
|
||||
@@ -1853,6 +1905,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
JsonArray effects = new JsonArray();
|
||||
int omittedInputs = 0;
|
||||
int omittedEdges = 0;
|
||||
boolean edgeLimitReached = false;
|
||||
for (PcodeOpAST operation : selected) {
|
||||
monitor.checkCancelled();
|
||||
String operationId = operationIds.get(operation);
|
||||
@@ -1868,14 +1921,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
value.addProperty("block_membership", operation.getParent() == null ? "detached" : "member");
|
||||
JsonArray inputs = new JsonArray();
|
||||
int inputCount = operation.getNumInputs();
|
||||
int capturedInputs = Math.min(inputCount, MAX_HIGH_PCODE_INPUTS_PER_OP);
|
||||
int capturedInputs = Math.min(inputCount, valueFlowLimits.maxInputsPerOperation());
|
||||
for (int inputIndex = 0; inputIndex < capturedInputs; inputIndex += 1) {
|
||||
Varnode input = operation.getInput(inputIndex);
|
||||
inputs.add(varnodeValue(input));
|
||||
if (input instanceof VarnodeAST astInput) {
|
||||
if (astInput.getHigh() instanceof HighParam parameter) {
|
||||
if (parameterUses.size() >= MAX_HIGH_PCODE_DEF_USE_EDGES) omittedEdges++;
|
||||
else {
|
||||
if (parameterUses.size() >= valueFlowLimits.maxEdges()) {
|
||||
edgeLimitReached = true;
|
||||
omittedEdges++;
|
||||
} else {
|
||||
JsonObject use = new JsonObject();
|
||||
use.addProperty("ordinal", parameter.getSlot());
|
||||
use.addProperty("use", operationId);
|
||||
@@ -1886,7 +1941,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
PcodeOp definition = astInput.getDef();
|
||||
if (definition instanceof PcodeOpAST astDefinition) {
|
||||
String definitionId = operationIds.get(astDefinition);
|
||||
if (definitionId == null || defUse.size() >= MAX_HIGH_PCODE_DEF_USE_EDGES) {
|
||||
if (definitionId == null || defUse.size() >= valueFlowLimits.maxEdges()) {
|
||||
if (defUse.size() >= valueFlowLimits.maxEdges()) edgeLimitReached = true;
|
||||
omittedEdges += 1;
|
||||
} else {
|
||||
JsonObject edge = new JsonObject();
|
||||
@@ -1909,6 +1965,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("available", true);
|
||||
result.addProperty("provenance", "ghidra-high-pcode");
|
||||
JsonObject limits = new JsonObject();
|
||||
limits.addProperty("max_operations", valueFlowLimits.maxOperations());
|
||||
limits.addProperty("max_inputs_per_operation", valueFlowLimits.maxInputsPerOperation());
|
||||
limits.addProperty("max_edges", valueFlowLimits.maxEdges());
|
||||
result.add("limits", limits);
|
||||
JsonArray limitsReached = new JsonArray();
|
||||
if (omittedOperationsLowerBound > 0) limitsReached.add("max_operations");
|
||||
if (omittedInputs > 0) limitsReached.add("max_inputs_per_operation");
|
||||
if (edgeLimitReached) limitsReached.add("max_edges");
|
||||
result.add("limits_reached", limitsReached);
|
||||
result.add("operations", operationValues);
|
||||
result.add("def_use", defUse);
|
||||
result.add("parameters", parameterValues);
|
||||
@@ -1930,19 +1996,42 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
limitations.add(
|
||||
"Def-use links do not cross function boundaries and may omit memory aliasing, call side effects, or unresolved indirect flow."
|
||||
);
|
||||
limitations.add(
|
||||
"Memory read/write effects cover high-p-code LOAD and STORE operations only; direct memory address varnodes in other operations are not included. An empty effect list does not establish that no memory is accessed. Recorded instruction references may provide complementary evidence."
|
||||
);
|
||||
limitations.add(
|
||||
"Memory read/write effects include stack and temporary memory operations; they do not identify persistent state or business meaning."
|
||||
);
|
||||
limitations.add(
|
||||
"Operation, input, and def-use limits can truncate large functions; inspect truncation, omitted lower bounds, and known omitted counts before drawing conclusions."
|
||||
"Operation, input, and def-use limits can truncate large functions; inspect the effective limits, limits_reached and omitted counts before drawing conclusions."
|
||||
);
|
||||
limitations.add(
|
||||
"Known omitted-edge counts cover links omitted from retained operations; omitted operations may contain additional uncounted links."
|
||||
);
|
||||
if (omittedOperationsLowerBound > 0)
|
||||
limitations.add(valueFlowLimitRecovery("max_operations", valueFlowLimits.maxOperations()));
|
||||
if (omittedInputs > 0)
|
||||
limitations.add(valueFlowLimitRecovery("max_inputs_per_operation", valueFlowLimits.maxInputsPerOperation()));
|
||||
if (edgeLimitReached)
|
||||
limitations.add(valueFlowLimitRecovery("max_edges", valueFlowLimits.maxEdges()));
|
||||
result.add("limitations", limitations);
|
||||
return result;
|
||||
}
|
||||
|
||||
private static String valueFlowLimitRecovery(String field, int effective) {
|
||||
String setting = "ghidra.value_flow_" + field;
|
||||
String environment = "REA_GHIDRA_VALUE_FLOW_" + field.toUpperCase(Locale.ROOT);
|
||||
String adjustment = effective < Integer.MAX_VALUE
|
||||
? "In MCP, use update_analysis_settings with setting " + setting +
|
||||
", action set and a larger integer value from 1 to " + Integer.MAX_VALUE +
|
||||
", then repeat the selected function analysis on this Program without restarting REA. " +
|
||||
"For a one-shot CLI retry, set " + environment +
|
||||
" to a larger count in the environment that starts the next rea invocation, then repeat the selected analysis; the previous invocation's Program is no longer active."
|
||||
: "The setting is at its Java integer maximum; select a smaller function for this projection.";
|
||||
return setting + " reached its effective value of " + effective + ". " + adjustment +
|
||||
" These controls do not raise the decompile deadline, bridge transport or MCP client limits.";
|
||||
}
|
||||
|
||||
private String pcodeOperationId(PcodeOpAST operation) {
|
||||
return canonicalAddress(operation.getSeqnum().getTarget()) + "#" +
|
||||
operation.getSeqnum().getTime();
|
||||
@@ -2376,7 +2465,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (!match.matched()) {
|
||||
continue;
|
||||
}
|
||||
JsonObject item = resultItem(entry.item, null);
|
||||
JsonObject item = resultItem(entry.item, "procedure");
|
||||
if (!match.aliases().isEmpty()) {
|
||||
JsonArray aliases = new JsonArray();
|
||||
for (String alias : match.aliases()) aliases.add(alias);
|
||||
@@ -2391,7 +2480,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (!matcher.matches(item.value)) {
|
||||
continue;
|
||||
}
|
||||
items.add(resultItem(item, null));
|
||||
items.add(resultItem(item, procedureSearch ? "procedure" : "string"));
|
||||
}
|
||||
return items;
|
||||
}
|
||||
@@ -3665,12 +3754,26 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (!results.decompileCompleted()) {
|
||||
throw new RequestFailure(
|
||||
"decompile_failed",
|
||||
"Ghidra decompilation failed: " + diagnosticMessage(results.getErrorMessage())
|
||||
"Ghidra decompilation failed: " + diagnosticMessage(results.getErrorMessage()) +
|
||||
". The effective native decompiler payload allowance is " +
|
||||
decompiler.getOptions().getMaxPayloadMBytes() +
|
||||
" MiB. " + decompilePayloadRecovery()
|
||||
);
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
private String decompilePayloadRecovery() {
|
||||
int effective = decompiler.getOptions().getMaxPayloadMBytes();
|
||||
if (effective >= MAX_DECOMPILE_PAYLOAD_MIB)
|
||||
return "If the diagnostic reports payload exhaustion, the native payload allowance is already at its supported maximum; no higher value is accepted. Use analyze_function for available listing facts or explicitly select a smaller function for decompilation.";
|
||||
return "If the diagnostic reports payload exhaustion, in MCP raise ghidra.decompile_max_payload_mib with update_analysis_settings " +
|
||||
"(integer MiB from 1 to " + MAX_DECOMPILE_PAYLOAD_MIB +
|
||||
") and repeat the selected operation on this Program without restarting REA. " +
|
||||
"For a one-shot CLI retry, set REA_GHIDRA_DECOMPILE_MAX_PAYLOAD_MIB to a larger MiB value in the environment that starts the next rea invocation, then repeat the selected operation; the previous invocation's Program is no longer active. " +
|
||||
"Runtime null or an omitted launch seed restores the installed Ghidra default. This does not raise the decompile deadline, high-pcode projection budgets or MCP client limits.";
|
||||
}
|
||||
|
||||
private static String decompiledText(DecompileResults results) {
|
||||
if (results == null) {
|
||||
return null;
|
||||
@@ -3716,9 +3819,23 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return result;
|
||||
}
|
||||
|
||||
private static JsonObject callRelationshipCoverage(
|
||||
private JsonObject callRelationshipCoverage(
|
||||
CallBoundaryObservation calls, Integer missingCallers) {
|
||||
JsonObject coverage = completeResolvedCallCoverage();
|
||||
JsonObject discovery = new JsonObject();
|
||||
if (missingCallers != null) {
|
||||
JsonObject callers = new JsonObject();
|
||||
callers.addProperty("status", "unknown");
|
||||
callers.addProperty("reason", "Recorded incoming references do not establish additional computed callers elsewhere in the program.");
|
||||
discovery.add("callers", callers);
|
||||
}
|
||||
if (calls != null && !calls.computedCallSites().isEmpty()) {
|
||||
JsonObject callees = new JsonObject();
|
||||
callees.addProperty("status", "unknown");
|
||||
callees.addProperty("reason", computedCallTargetLimitation(calls.computedCallSites()));
|
||||
discovery.add("callees", callees);
|
||||
}
|
||||
if (!discovery.isEmpty()) coverage.add("discovery", discovery);
|
||||
Integer omitted = calls == null ? null : calls.unrepresentedCalleeCalls();
|
||||
JsonArray directions = new JsonArray();
|
||||
if (missingCallers != null && missingCallers > 0) directions.add("callers");
|
||||
@@ -3733,20 +3850,49 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (calls == null && missingCallers == null) directions.add("callers");
|
||||
directions.add("callees");
|
||||
coverage.add("limited_directions", directions);
|
||||
return coverage;
|
||||
return qualifyProjectCallCoverage(coverage, calls, missingCallers);
|
||||
}
|
||||
JsonArray limitations = new JsonArray();
|
||||
if (callerLimitation != null) limitations.add(callerLimitation);
|
||||
if (omitted != null && omitted > 0) directions.add("callees");
|
||||
if (omitted != null && omitted > 0) {
|
||||
limitations.add("Ghidra's callee inventory omits targets of " + omitted +
|
||||
" call reference(s) or decoded call(s) without destinations.");
|
||||
directions.add("callees");
|
||||
}
|
||||
if (!limitations.isEmpty()) {
|
||||
coverage.addProperty("status", "partial");
|
||||
coverage.add("limitations", limitations);
|
||||
coverage.add("limited_directions", directions);
|
||||
}
|
||||
return qualifyProjectCallCoverage(coverage, calls, missingCallers);
|
||||
}
|
||||
|
||||
private JsonObject qualifyProjectCallCoverage(
|
||||
JsonObject coverage, CallBoundaryObservation calls, Integer missingCallers) {
|
||||
if (!existingProject) return coverage;
|
||||
String reason = "The selected project's prior analysis completeness is unknown; " +
|
||||
"the recorded call relationships may omit undiscovered callers or callees.";
|
||||
String status = coverage.get("status").getAsString();
|
||||
// A single-direction read with known omissions is still partial; an
|
||||
// unknown project history adds a limitation rather than erasing them.
|
||||
if (status.equals("partial") && (calls == null || missingCallers == null)) {
|
||||
coverage.getAsJsonArray("limitations").add(reason);
|
||||
return coverage;
|
||||
}
|
||||
if (status.equals("unknown")) reason += " " + coverage.get("reason").getAsString();
|
||||
if (status.equals("partial")) {
|
||||
for (JsonElement limitation : coverage.getAsJsonArray("limitations"))
|
||||
reason += " " + limitation.getAsString();
|
||||
}
|
||||
JsonArray directions = new JsonArray();
|
||||
if (missingCallers != null || (calls == null && missingCallers == null))
|
||||
directions.add("callers");
|
||||
if (calls != null || (calls == null && missingCallers == null))
|
||||
directions.add("callees");
|
||||
coverage.addProperty("status", "unknown");
|
||||
coverage.addProperty("reason", reason);
|
||||
coverage.remove("limitations");
|
||||
coverage.add("limited_directions", directions);
|
||||
return coverage;
|
||||
}
|
||||
|
||||
@@ -3761,6 +3907,24 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return coverage;
|
||||
}
|
||||
|
||||
/** Name a few sites inline; reference reads and call-target queries recover the rest. */
|
||||
private static final int NAMED_COMPUTED_CALL_SITES = 4;
|
||||
|
||||
/**
|
||||
* A computed call's ReferenceManager targets are static candidates: Ghidra
|
||||
* records no proof that other values cannot reach the call, so the callee
|
||||
* set stays incomplete while the recorded candidates remain observed.
|
||||
*/
|
||||
private static String computedCallTargetLimitation(List<String> sites) {
|
||||
if (sites.isEmpty()) return null;
|
||||
int named = Math.min(sites.size(), NAMED_COMPUTED_CALL_SITES);
|
||||
String listed = String.join(", ", sites.subList(0, named)) +
|
||||
(sites.size() > named ? " and " + (sites.size() - named) + " more" : "");
|
||||
return "Ghidra records call reference targets at " + sites.size() +
|
||||
" computed call site(s) (" + listed + "); these static references do not establish" +
|
||||
" every possible target, so additional callees remain unknown.";
|
||||
}
|
||||
|
||||
private boolean unrepresentedCaller(Reference reference, Set<Function> callers) {
|
||||
if (!reference.getReferenceType().isCall()) return false;
|
||||
Function caller = functionAtOrContaining(reference.getFromAddress());
|
||||
@@ -3795,9 +3959,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
monitor.checkCancelled();
|
||||
calls.observe(reference);
|
||||
}
|
||||
List<String> computedCallSites = new ArrayList<>();
|
||||
for (Instruction instruction : instructions) {
|
||||
boundary.observe(instruction);
|
||||
if (!instruction.getFlowType().isCall()) continue;
|
||||
FlowType flow = instruction.getFlowType();
|
||||
if (!flow.isCall()) continue;
|
||||
boolean destination = false;
|
||||
for (Reference reference : instruction.getReferencesFrom()) {
|
||||
if (!reference.getReferenceType().isCall()) continue;
|
||||
@@ -3809,9 +3975,13 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
calls.unresolved(instruction.getAddress(),
|
||||
"Instruction decoder identifies a call but ReferenceManager supplies no call destination");
|
||||
}
|
||||
// The same decoded flow classifies resolve_native_call_targets'
|
||||
// mechanism; its recorded targets are candidates, not a closed set.
|
||||
else if (flow.isComputed()) computedCallSites.add(canonicalAddress(instruction.getAddress()));
|
||||
}
|
||||
return new CallBoundaryObservation(boundary.values(),
|
||||
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls());
|
||||
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls(),
|
||||
List.copyOf(computedCallSites));
|
||||
}
|
||||
|
||||
private final class FunctionBoundaryLimitations {
|
||||
@@ -5055,7 +5225,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
Set<String> keys = new HashSet<>(REQUEST_KEYS);
|
||||
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("decompile_timeout_seconds"))
|
||||
keys.add("decompile_timeout_seconds");
|
||||
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("value_flow_limits"))
|
||||
keys.add("value_flow_limits");
|
||||
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("decompile_max_payload_mib"))
|
||||
keys.add("decompile_max_payload_mib");
|
||||
JsonObject object = requireObject(parsed, keys);
|
||||
Integer payloadMiB = null;
|
||||
if (object.has("decompile_max_payload_mib") && !object.get("decompile_max_payload_mib").isJsonNull()) {
|
||||
payloadMiB = requireInteger(object, "decompile_max_payload_mib");
|
||||
if (payloadMiB < 1 || payloadMiB > MAX_DECOMPILE_PAYLOAD_MIB)
|
||||
throw new IllegalArgumentException("Invalid per-request native decompiler payload limit");
|
||||
}
|
||||
ValueFlowLimits flowLimits = defaultValueFlowLimits();
|
||||
if (object.has("value_flow_limits")) {
|
||||
JsonObject limits = requireObject(object.get("value_flow_limits"),
|
||||
Set.of("max_operations", "max_inputs_per_operation", "max_edges"));
|
||||
int operations = requireInteger(limits, "max_operations");
|
||||
int inputs = requireInteger(limits, "max_inputs_per_operation");
|
||||
int edges = requireInteger(limits, "max_edges");
|
||||
if (operations < 1 || inputs < 1 || edges < 1)
|
||||
throw new IllegalArgumentException("Invalid per-request value-flow limits");
|
||||
flowLimits = new ValueFlowLimits(operations, inputs, edges);
|
||||
}
|
||||
Integer timeout = null;
|
||||
if (object.has("decompile_timeout_seconds")) {
|
||||
JsonElement value = object.get("decompile_timeout_seconds");
|
||||
@@ -5071,7 +5262,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (params == null || !params.isJsonObject()) {
|
||||
throw new IllegalArgumentException("Bridge request parameters are invalid");
|
||||
}
|
||||
return new Request(id, requireString(object, "method"), params.getAsJsonObject(), timeout);
|
||||
return new Request(id, requireString(object, "method"), params.getAsJsonObject(), timeout, payloadMiB, flowLimits);
|
||||
}
|
||||
|
||||
private static JsonObject requireObject(JsonElement element, Set<String> keys) {
|
||||
@@ -5257,7 +5448,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
}
|
||||
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage, boolean decompilerFailed) {}
|
||||
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
|
||||
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls, List<String> computedCallSites) {}
|
||||
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
|
||||
private record SwiftLiteralLayout(
|
||||
String representation,
|
||||
@@ -5279,7 +5470,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
int decompileTimeoutSeconds,
|
||||
boolean existingProject
|
||||
) {}
|
||||
private record Request(int id, String method, JsonObject params, Integer decompileTimeoutSeconds) {}
|
||||
private static ValueFlowLimits defaultValueFlowLimits() {
|
||||
return new ValueFlowLimits(DEFAULT_HIGH_PCODE_OPS, DEFAULT_HIGH_PCODE_INPUTS_PER_OP, DEFAULT_HIGH_PCODE_DEF_USE_EDGES);
|
||||
}
|
||||
private record ValueFlowLimits(int maxOperations, int maxInputsPerOperation, int maxEdges) {}
|
||||
private record Request(int id, String method, JsonObject params, Integer decompileTimeoutSeconds, Integer decompileMaxPayloadMiB, ValueFlowLimits valueFlowLimits) {}
|
||||
|
||||
private static final class RequestFailure extends RuntimeException {
|
||||
private final String code;
|
||||
|
||||
@@ -37,7 +37,7 @@ import org.xml.sax.InputSource;
|
||||
/**
|
||||
* Apply the committed RISC-V calling-convention policy after analysis. Ghidra's generic RV32
|
||||
* compiler specification models ilp32d. Each policy derives a program-specific prototype from the
|
||||
* program's own default prototype, as its {@link Policy} row and docs/ghidra-riscv.md state. The
|
||||
* program's own default prototype, as its {@link Policy} row and docs/providers/ghidra-riscv.md state. The
|
||||
* script installs the prototype as a specification extension and assigns it, with dynamic
|
||||
* storage, to every non-thunk function, external function and function definition.
|
||||
*/
|
||||
|
||||
@@ -1399,7 +1399,8 @@ def _serve_connection(connection):
|
||||
break
|
||||
except (ConnectionError, OSError):
|
||||
# A lost client cannot confirm document closure or revoke this run.
|
||||
pass
|
||||
# Keep the listener available for the owner to confirm cleanup.
|
||||
should_stop = False
|
||||
finally:
|
||||
try:
|
||||
file.close()
|
||||
|
||||
@@ -18,7 +18,6 @@ from mitmproxy import ctx, version
|
||||
from mitmproxy.io import tnetstring
|
||||
|
||||
PROFILE = "12.2.3"
|
||||
OUTPUT_BYTES = 96 * 1024 * 1024
|
||||
CREDENTIALS = {b"authorization", b"proxy-authorization", b"cookie", b"set-cookie"}
|
||||
|
||||
|
||||
@@ -204,7 +203,7 @@ def without_userinfo(value, base_scheme=None):
|
||||
raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
|
||||
|
||||
|
||||
def project_record(state, secrets):
|
||||
def project_record(state, secrets, max_depth):
|
||||
binaries, numbers, redactions = [], [], []
|
||||
byte_secrets = []
|
||||
for secret in secrets:
|
||||
@@ -251,8 +250,8 @@ def project_record(state, secrets):
|
||||
del redactions[before[2]:]
|
||||
|
||||
def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
|
||||
if depth > 64:
|
||||
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
|
||||
if depth > max_depth:
|
||||
raise CaptureFailure("input-limit", f"Native capture exceeds max_nesting_depth ({max_depth} levels).", path)
|
||||
if credential:
|
||||
validate_omitted(value, path, depth)
|
||||
redactions.append({"pointer": path, "reason": "transport-credential"})
|
||||
@@ -337,12 +336,12 @@ def decode(request):
|
||||
except CaptureFailure:
|
||||
raise
|
||||
except RecursionError:
|
||||
raise CaptureFailure("input-limit", "Native parser exceeded its recursion budget at byte offset " + str(start) + ".") from None
|
||||
raise CaptureFailure("unsupported", "Native parser reached Python's recursion limit at byte offset " + str(start) + ". max_nesting_depth does not raise that upstream limit.") from None
|
||||
except (ValueError, IndexError, UnicodeError, EOFError):
|
||||
raise CaptureFailure("format", "Malformed native record at byte offset " + str(start) + ".") from None
|
||||
if not isinstance(state, dict):
|
||||
raise CaptureFailure("format", "Native flow record must be a dictionary at byte offset " + str(start) + ".")
|
||||
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
|
||||
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"], request["budgets"]["depth"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
|
||||
return {"decoder": {"id": "mitmproxy-native-tnetstring", "name": "REA offline native mitmproxy adapter", "version": PROFILE}, "container": {"reported": None, "numeric_literals": [], "redactions": [], "records_pointer": None}, "total_records": len(records), "records": records}
|
||||
|
||||
|
||||
@@ -350,6 +349,8 @@ def failure_reply(error):
|
||||
"""Keep producer constraints, resource exhaustion and adapter defects distinct."""
|
||||
if isinstance(error, CaptureFailure):
|
||||
return {"ok": False, "reason": error.reason, "message": str(error), "pointer": error.pointer}
|
||||
if isinstance(error, RecursionError):
|
||||
return {"ok": False, "reason": "unsupported", "message": "Native projection reached Python's recursion limit; max_nesting_depth does not raise that upstream limit.", "pointer": ""}
|
||||
if isinstance(error, MemoryError):
|
||||
return {"ok": False, "reason": "resource-limit", "message": "Native decoder exhausted its 768 MiB address-space budget.", "pointer": ""}
|
||||
return {"ok": False, "reason": "decoder", "message": "Offline native decoder failed unexpectedly: " + type(error).__name__ + ".", "pointer": ""}
|
||||
@@ -362,9 +363,13 @@ class OfflineCapture:
|
||||
def running(self):
|
||||
resource.setrlimit(resource.RLIMIT_AS, (768 * 1024 * 1024, 768 * 1024 * 1024))
|
||||
resource.setrlimit(resource.RLIMIT_CPU, (30, 30))
|
||||
resource.setrlimit(resource.RLIMIT_FSIZE, (OUTPUT_BYTES, OUTPUT_BYTES))
|
||||
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
|
||||
request = json.loads(Path(ctx.options.rea_request_path).read_text(encoding="utf-8"))
|
||||
budgets = request["budgets"]
|
||||
if any(type(budgets[key]) is not int or budgets[key] < 1 for key in ("inputBytes", "outputBytes", "depth")):
|
||||
raise ValueError("Invalid admitted capture budgets")
|
||||
output_bytes = budgets["outputBytes"]
|
||||
resource.setrlimit(resource.RLIMIT_FSIZE, (output_bytes, output_bytes))
|
||||
try:
|
||||
response = {"ok": True, "value": decode(request)}
|
||||
except Exception as error:
|
||||
@@ -374,7 +379,7 @@ class OfflineCapture:
|
||||
output = json.dumps(response, ensure_ascii=True, allow_nan=False, separators=(",", ":")).encode("ascii")
|
||||
except Exception as error:
|
||||
output = json.dumps(failure_reply(error), separators=(",", ":")).encode("ascii")
|
||||
if len(output) > OUTPUT_BYTES:
|
||||
if len(output) > output_bytes:
|
||||
output = b'{"ok":false,"reason":"limit","message":"Native reply exceeds the complete-evidence output budget.","pointer":""}'
|
||||
Path(request["reply_path"]).write_bytes(output)
|
||||
finally:
|
||||
|
||||
@@ -36,7 +36,7 @@ def location(value, scheme=None):
|
||||
state = {"response": {"headers": [[b"Location", value]]}}
|
||||
if scheme is not None:
|
||||
state["request"] = {"scheme": scheme}
|
||||
record = capture.project_record(state, [])
|
||||
record = capture.project_record(state, [], 64)
|
||||
sidecar = next(
|
||||
field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
|
||||
)
|
||||
@@ -75,11 +75,11 @@ class TransportUserinfoTests(unittest.TestCase):
|
||||
def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
|
||||
for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
|
||||
with self.subTest(value=value):
|
||||
record = capture.project_record({"request": {"path": value}}, [])
|
||||
record = capture.project_record({"request": {"path": value}}, [], 64)
|
||||
self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
|
||||
self.assertEqual(record["binary_fields"][0]["state"], "retained")
|
||||
self.assertEqual(record["redactions"], [])
|
||||
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
|
||||
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [], 64)
|
||||
self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
|
||||
self.assertEqual(record["binary_fields"][0]["state"], "redacted")
|
||||
|
||||
@@ -110,7 +110,7 @@ class TransportUserinfoTests(unittest.TestCase):
|
||||
"request": {"scheme": b"https"},
|
||||
"response": {"headers": [[b"Location", b"https:invoice@customer"]]},
|
||||
},
|
||||
}, [])
|
||||
}, [], 64)
|
||||
self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
|
||||
self.assertEqual(record["redactions"], [])
|
||||
|
||||
|
||||
@@ -24,3 +24,16 @@ func captureAccessibilityValue(_ source: Any?) -> AccessibilityValue {
|
||||
}
|
||||
return AccessibilityValue(value: NSNull(), gap: "AXValue type \(type(of: source)) is not a supported JSON scalar")
|
||||
}
|
||||
|
||||
/** Keep an unavailable action list distinct from a successful empty list. */
|
||||
func captureActionNames(status: Int32, success: Int32, names: [String]?) -> (names: [String], error: Int32?) {
|
||||
guard status == success else { return ([], status) }
|
||||
return (names ?? [], nil)
|
||||
}
|
||||
|
||||
/** AX absence is distinct from a failed attribute read. */
|
||||
func captureAttributeValue<Value>(status: Int32, success: Int32, absent: [Int32], value: Value?) -> (value: Value?, error: Int32?) {
|
||||
if status == success { return (value, nil) }
|
||||
if absent.contains(status) { return (nil, nil) }
|
||||
return (nil, status)
|
||||
}
|
||||
@@ -18,9 +18,10 @@ struct Request: Decodable {
|
||||
struct Action: Decodable { var kind: String; var path: [Int]?; var direction: String?; var text: String? }
|
||||
struct BoundaryFailure: Error { var code: String; var message: String }
|
||||
func fail(_ code: String, _ message: String) throws -> Never { throw BoundaryFailure(code: code, message: message) }
|
||||
func attribute(_ element: AXUIElement, _ key: String) -> CFTypeRef? {
|
||||
func attribute(_ element: AXUIElement, _ key: String) -> (value: CFTypeRef?, error: Int32?) {
|
||||
var value: CFTypeRef?
|
||||
return AXUIElementCopyAttributeValue(element, key as CFString, &value) == .success ? value : nil
|
||||
let status = AXUIElementCopyAttributeValue(element, key as CFString, &value)
|
||||
return captureAttributeValue(status: status.rawValue, success: AXError.success.rawValue, absent: [AXError.noValue.rawValue, AXError.attributeUnsupported.rawValue], value: value)
|
||||
}
|
||||
func childCount(_ element: AXUIElement) -> ChildCount {
|
||||
var count = 0
|
||||
@@ -34,13 +35,15 @@ func children(_ element: AXUIElement, count: Int) -> ChildBatch<AXUIElement> {
|
||||
return value as? [AXUIElement]
|
||||
}
|
||||
}
|
||||
func text(_ element: AXUIElement, _ key: String) -> Any { (attribute(element, key) as? String) ?? NSNull() as Any }
|
||||
func windowBounds(_ element: AXUIElement) -> CGRect? {
|
||||
guard let p = attribute(element, kAXPositionAttribute), let s = attribute(element, kAXSizeAttribute),
|
||||
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return nil }
|
||||
var point = CGPoint.zero; var size = CGSize.zero
|
||||
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &size) else { return nil }
|
||||
return CGRect(origin: point, size: size)
|
||||
func windowBounds(_ element: AXUIElement) -> (bounds: CGRect?, error: Int32?) {
|
||||
let position = attribute(element, kAXPositionAttribute)
|
||||
let size = attribute(element, kAXSizeAttribute)
|
||||
if let error = position.error ?? size.error { return (nil, error) }
|
||||
guard let p = position.value, let s = size.value,
|
||||
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return (nil, nil) }
|
||||
var point = CGPoint.zero; var dimensions = CGSize.zero
|
||||
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &dimensions) else { return (nil, nil) }
|
||||
return (CGRect(origin: point, size: dimensions), nil)
|
||||
}
|
||||
func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard let app = NSRunningApplication(processIdentifier: request.pid) else {
|
||||
@@ -84,11 +87,22 @@ func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard AXIsProcessTrusted() else { try fail("accessibility-denied", "Grant Accessibility to the REA host in System Settings > Privacy & Security, then retry") }
|
||||
let application = AXUIElementCreateApplication(request.pid)
|
||||
AXUIElementSetMessagingTimeout(application, 2)
|
||||
guard let axWindows = attribute(application, kAXWindowsAttribute) as? [AXUIElement] else {
|
||||
try fail("accessibility-unavailable", "Selected application does not expose accessibility windows")
|
||||
let windowsAttribute = attribute(application, kAXWindowsAttribute)
|
||||
if let error = windowsAttribute.error {
|
||||
try fail("accessibility-unavailable", "Cannot read accessibility windows: AXError \(error)")
|
||||
}
|
||||
let matches = axWindows.filter { windowBounds($0) == bounds }
|
||||
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(axWindows.map { String(describing: windowBounds($0)) })") }
|
||||
guard let axWindows = windowsAttribute.value as? [AXUIElement] else {
|
||||
try fail("accessibility-unavailable", "Accessibility windows are unavailable for the selected application")
|
||||
}
|
||||
let candidates = axWindows.map { (element: $0, observation: windowBounds($0)) }
|
||||
if let error = candidates.compactMap({ $0.observation.error }).first {
|
||||
try fail("accessibility-unavailable", "Cannot read accessibility window geometry: AXError \(error)")
|
||||
}
|
||||
guard candidates.allSatisfy({ $0.observation.bounds != nil }) else {
|
||||
try fail("accessibility-unavailable", "Accessibility window geometry is unavailable; the selected window cannot be identified")
|
||||
}
|
||||
let matches = candidates.filter { $0.observation.bounds == bounds }.map { $0.element }
|
||||
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(candidates.map { String(describing: $0.observation.bounds) })") }
|
||||
selected = matches[0]
|
||||
}
|
||||
if let action = request.action, let root = selected {
|
||||
@@ -99,13 +113,16 @@ func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard let available = count.value else {
|
||||
try fail("element-count-unavailable", "Cannot validate selected accessibility path because child count failed with AXError \(count.error ?? -1)")
|
||||
}
|
||||
guard index >= 0 && index < available,
|
||||
AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild) == .success,
|
||||
let item = (selectedChild as? [AXUIElement])?.first else { try fail("element-missing", "Selected accessibility path no longer exists") }
|
||||
guard index >= 0 && index < available else { try fail("element-missing", "Selected accessibility path no longer exists") }
|
||||
let childStatus = AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild)
|
||||
guard childStatus == .success else { try fail("accessibility-unavailable", "Cannot read selected accessibility child: AXError \(childStatus.rawValue)") }
|
||||
guard let item = (selectedChild as? [AXUIElement])?.first else { try fail("accessibility-unavailable", "Selected accessibility child was not returned") }
|
||||
element = item
|
||||
}
|
||||
var owner: pid_t = 0
|
||||
guard AXUIElementGetPid(element, &owner) == .success && owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
|
||||
let ownerStatus = AXUIElementGetPid(element, &owner)
|
||||
guard ownerStatus == .success else { try fail("accessibility-unavailable", "Cannot identify accessibility element owner: AXError \(ownerStatus.rawValue)") }
|
||||
guard owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
|
||||
let outcome: AXError
|
||||
switch action.kind {
|
||||
case "click": outcome = AXUIElementPerformAction(element, kAXPressAction as CFString)
|
||||
@@ -122,10 +139,18 @@ func observe(_ request: Request) async throws -> [String: Any] {
|
||||
if nodes.count >= request.max_nodes { truncated = true; break }
|
||||
let childCountResult = childCount(element)
|
||||
var actions: CFArray?
|
||||
AXUIElementCopyActionNames(element, &actions)
|
||||
let observedValue = captureAccessibilityValue(attribute(element, kAXValueAttribute))
|
||||
let actionStatus = AXUIElementCopyActionNames(element, &actions)
|
||||
let observedActions = captureActionNames(status: actionStatus.rawValue, success: AXError.success.rawValue, names: actions as? [String])
|
||||
if let error = observedActions.error { gaps.append("AX actions unavailable at path \(path): AXError \(error)"); truncated = true }
|
||||
let role = attribute(element, kAXRoleAttribute)
|
||||
let title = attribute(element, kAXTitleAttribute)
|
||||
let value = attribute(element, kAXValueAttribute)
|
||||
for (name, error) in [("role", role.error), ("title", title.error), ("value", value.error)] {
|
||||
if let error { gaps.append("AX \(name) unavailable at path \(path): AXError \(error)"); truncated = true }
|
||||
}
|
||||
let observedValue = captureAccessibilityValue(value.value)
|
||||
if let gap = observedValue.gap { gaps.append("\(gap) at path \(path)"); truncated = true }
|
||||
nodes.append(["path": path, "role": text(element, kAXRoleAttribute), "title": text(element, kAXTitleAttribute), "value": observedValue.value, "actions": actions as? [String] ?? [], "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
|
||||
nodes.append(["path": path, "role": (role.value as? String) ?? NSNull() as Any, "title": (title.value as? String) ?? NSNull() as Any, "value": observedValue.value, "actions": observedActions.names, "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
|
||||
guard let totalChildren = childCountResult.value else {
|
||||
truncated = true
|
||||
gaps.append("AX child count unavailable at path \(path): AXError \(childCountResult.error ?? -1)")
|
||||
|
||||
@@ -25,24 +25,17 @@ ARGUMENT_REGISTERS = {
|
||||
"x86_64": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
|
||||
"x86_64h": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
|
||||
}
|
||||
MAX_SELECTOR_BYTES = 1024
|
||||
MAX_OTHER_STOPS = 256
|
||||
MAX_TRACE_BYTES = 8 * 1024 * 1024
|
||||
MAX_TRACE_FRAMES = 65_536
|
||||
MAX_OBSERVATION_JOURNAL_BYTES = MAX_TRACE_BYTES
|
||||
MAX_BREAKPOINT_LOCATION_BYTES = 8 * 1024 * 1024
|
||||
MAX_TARGET_OUTPUT_BYTES = 1024 * 1024
|
||||
OUTPUT_DRAIN_BYTES_PER_PASS = 1024 * 1024
|
||||
OUTPUT_EOF_WAIT_SECONDS = 1.0
|
||||
OUTPUT_READER_JOIN_SECONDS = 0.5
|
||||
LISTENER_NAME = "rea-native-call-observation"
|
||||
|
||||
|
||||
def _can_retain_trace(current_bytes, current_frames, event_bytes, event_frames):
|
||||
def _can_retain_trace(current_bytes, current_frames, event_bytes, event_frames, limits):
|
||||
"""Whether a complete event fits both aggregate trace budgets."""
|
||||
return (
|
||||
current_bytes + event_bytes <= MAX_TRACE_BYTES
|
||||
and current_frames + event_frames <= MAX_TRACE_FRAMES
|
||||
current_bytes + event_bytes <= limits["trace_bytes"]
|
||||
and current_frames + event_frames <= limits["trace_frames"]
|
||||
)
|
||||
|
||||
|
||||
@@ -71,7 +64,7 @@ def _fits_trace_budget(value, remaining_bytes):
|
||||
class _ObservationJournal:
|
||||
"""Flush a bounded JSONL prefix so observations survive tracer failure."""
|
||||
|
||||
def __init__(self, handle, max_bytes=MAX_OBSERVATION_JOURNAL_BYTES):
|
||||
def __init__(self, handle, max_bytes):
|
||||
self.handle = handle
|
||||
self.max_bytes = max_bytes
|
||||
self.bytes = 0
|
||||
@@ -243,6 +236,20 @@ def _hex(value):
|
||||
return "0x%x" % value
|
||||
|
||||
|
||||
def _source_location(address):
|
||||
"""Preserve available line-entry facets without acquiring source files."""
|
||||
entry = address.GetLineEntry()
|
||||
if not entry.IsValid():
|
||||
return None
|
||||
file_spec = entry.GetFileSpec()
|
||||
file_path = str(file_spec) if file_spec.IsValid() else None
|
||||
line = entry.GetLine() or None
|
||||
column = entry.GetColumn() or None
|
||||
if not file_path and line is None and column is None:
|
||||
return None
|
||||
return {"file_path": file_path or None, "line": line, "column": column}
|
||||
|
||||
|
||||
def _location(address, target):
|
||||
"""Load and file address, module and symbol of one code address."""
|
||||
module = address.GetModule()
|
||||
@@ -256,6 +263,7 @@ def _location(address, target):
|
||||
"module": module.GetFileSpec().GetFilename() if module.IsValid() else None,
|
||||
"module_path": str(module.GetFileSpec()) if module.IsValid() else None,
|
||||
"symbol": symbol.GetName() if symbol.IsValid() else None,
|
||||
"source_location": _source_location(address),
|
||||
}
|
||||
return location
|
||||
|
||||
@@ -356,7 +364,7 @@ def _record(context, thread, frame, index, sequence, location, remaining_bytes):
|
||||
error = lldb.SBError()
|
||||
text = process.ReadCStringFromMemory(
|
||||
values[names[1]],
|
||||
MAX_SELECTOR_BYTES,
|
||||
config["capture_limits"]["selector_bytes"],
|
||||
error,
|
||||
)
|
||||
selector = text if error.Success() and text else None
|
||||
@@ -389,10 +397,16 @@ def _record(context, thread, frame, index, sequence, location, remaining_bytes):
|
||||
_TRACE_CONTEXT = None
|
||||
|
||||
|
||||
def _resource_limit(context, limit):
|
||||
context["resource_limit"] = limit
|
||||
return "resource-limit"
|
||||
|
||||
|
||||
def _handle_breakpoint(context, frame, index):
|
||||
"""Read one hit while LLDB's breakpoint callback owns the stopped frame."""
|
||||
if not frame.IsValid() or frame.GetPC() == lldb.LLDB_INVALID_ADDRESS:
|
||||
raise RuntimeError("LLDB breakpoint callback did not supply a readable frame")
|
||||
limits = context["config"]["capture_limits"]
|
||||
thread = frame.GetThread()
|
||||
events = context["events"]
|
||||
location = _location(frame.GetPCAddress(), context["target"])
|
||||
@@ -409,49 +423,50 @@ def _handle_breakpoint(context, frame, index):
|
||||
context["retained_frames"],
|
||||
0,
|
||||
event_frame_count,
|
||||
limits,
|
||||
):
|
||||
return "resource-limit"
|
||||
return _resource_limit(context, "trace_frames")
|
||||
try:
|
||||
remaining_bytes = MAX_TRACE_BYTES - context["retained_trace_bytes"]
|
||||
remaining_bytes = limits["trace_bytes"] - context["retained_trace_bytes"]
|
||||
record = _record(
|
||||
context, thread, frame, index, len(events), location, remaining_bytes
|
||||
)
|
||||
except ValueError:
|
||||
return "resource-limit"
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
if record is not None:
|
||||
if not _fits_trace_budget(record, remaining_bytes):
|
||||
return "resource-limit"
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
record_bytes = len(json.dumps(record).encode("utf-8"))
|
||||
if context["retained_trace_bytes"] + record_bytes > MAX_TRACE_BYTES:
|
||||
return "resource-limit"
|
||||
if context["retained_trace_bytes"] + record_bytes > limits["trace_bytes"]:
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
for depth in range(1, frame_count + 1):
|
||||
try:
|
||||
location = _location(
|
||||
thread.GetFrameAtIndex(depth).GetPCAddress(), context["target"]
|
||||
)
|
||||
except ValueError:
|
||||
return "resource-limit"
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
if (
|
||||
_estimated_json_size(location) + record_bytes + 1
|
||||
> MAX_TRACE_BYTES - context["retained_trace_bytes"]
|
||||
> limits["trace_bytes"] - context["retained_trace_bytes"]
|
||||
):
|
||||
return "resource-limit"
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
location_bytes = len(json.dumps(location).encode("utf-8"))
|
||||
addition = location_bytes + (1 if record["backtrace"] else 0)
|
||||
if (
|
||||
context["retained_trace_bytes"]
|
||||
+ record_bytes
|
||||
+ addition
|
||||
> MAX_TRACE_BYTES
|
||||
> limits["trace_bytes"]
|
||||
):
|
||||
return "resource-limit"
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
record["backtrace"].append(location)
|
||||
record_bytes += addition
|
||||
journal = context.get("observation_journal")
|
||||
if journal is not None and not journal.append(
|
||||
{"kind": "event", "event": record}
|
||||
):
|
||||
return "resource-limit"
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
events.append(record)
|
||||
context["retained_frames"] += event_frame_count
|
||||
context["retained_trace_bytes"] += record_bytes
|
||||
@@ -483,9 +498,9 @@ def _handle_stop(context, other_stops):
|
||||
if journal is not None and not journal.append(
|
||||
{"kind": "other-stop", "reason": stop_description}
|
||||
):
|
||||
return "resource-limit"
|
||||
return _resource_limit(context, "trace_bytes")
|
||||
other_stops.append(stop_description)
|
||||
if len(other_stops) >= MAX_OTHER_STOPS:
|
||||
if len(other_stops) >= context["config"]["capture_limits"]["other_stops"]:
|
||||
return "stop-limit"
|
||||
return None
|
||||
|
||||
@@ -505,7 +520,7 @@ def _breakpoint_report(target, created, config):
|
||||
continue
|
||||
count += 1
|
||||
remaining_bytes = (
|
||||
MAX_BREAKPOINT_LOCATION_BYTES - retained_location_bytes
|
||||
config["capture_limits"]["breakpoint_location_bytes"] - retained_location_bytes
|
||||
)
|
||||
if not _fits_trace_budget(location, remaining_bytes):
|
||||
locations_truncated = True
|
||||
@@ -636,7 +651,7 @@ def trace(debugger, config):
|
||||
for breakpoint in breakpoints:
|
||||
breakpoint_index[breakpoint.GetID()] = index
|
||||
listener = lldb.SBListener(LISTENER_NAME)
|
||||
output_budget = config.get("max_output_bytes", MAX_TARGET_OUTPUT_BYTES)
|
||||
output_budget = config["capture_limits"]["output_bytes"]
|
||||
stdout = _OutputPipe(
|
||||
config["stdout_path"], config["stdout_capture_path"], output_budget
|
||||
)
|
||||
@@ -656,7 +671,8 @@ def trace(debugger, config):
|
||||
observation_path = config.get("observation_path")
|
||||
if observation_path:
|
||||
observation_journal = _ObservationJournal(
|
||||
open(observation_path, "wb")
|
||||
open(observation_path, "wb"),
|
||||
config["capture_limits"]["trace_bytes"],
|
||||
)
|
||||
process = target.Launch(_launch_info(target, config, listener), error)
|
||||
process_created = process.IsValid()
|
||||
@@ -847,6 +863,7 @@ def trace(debugger, config):
|
||||
"breakpoint_locations_truncated": breakpoint_locations_truncated,
|
||||
"events": events,
|
||||
"resource_limit_reached": outcome == "resource-limit",
|
||||
"resource_limit": context.get("resource_limit"),
|
||||
"target_output": {
|
||||
"stdout_bytes": stdout.bytes,
|
||||
"stderr_bytes": stderr.bytes,
|
||||
|
||||
@@ -25,6 +25,45 @@ spec.loader.exec_module(tracer)
|
||||
|
||||
|
||||
class NativeTraceBudgetTests(unittest.TestCase):
|
||||
def test_source_coordinates_preserve_partial_metadata_without_reading_source(self):
|
||||
def address(valid=True, file_valid=True, line=23, column=7):
|
||||
class SourceFileSpec:
|
||||
def IsValid(self):
|
||||
return file_valid
|
||||
|
||||
def __str__(self):
|
||||
return "/missing/é space/fixture.pas"
|
||||
|
||||
file_spec = SourceFileSpec()
|
||||
entry = types.SimpleNamespace(
|
||||
IsValid=lambda: valid,
|
||||
GetFileSpec=lambda: file_spec,
|
||||
GetLine=lambda: line,
|
||||
GetColumn=lambda: column,
|
||||
)
|
||||
return types.SimpleNamespace(GetLineEntry=lambda: entry)
|
||||
|
||||
self.assertEqual(
|
||||
tracer._source_location(address()),
|
||||
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": 7},
|
||||
)
|
||||
self.assertEqual(
|
||||
tracer._source_location(address(column=0)),
|
||||
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": None},
|
||||
)
|
||||
self.assertEqual(
|
||||
tracer._source_location(address(file_valid=False)),
|
||||
{"file_path": None, "line": 23, "column": 7},
|
||||
)
|
||||
self.assertEqual(
|
||||
tracer._source_location(address(line=0)),
|
||||
{"file_path": "/missing/é space/fixture.pas", "line": None, "column": 7},
|
||||
)
|
||||
self.assertIsNone(tracer._source_location(address(valid=False)))
|
||||
self.assertIsNone(
|
||||
tracer._source_location(address(file_valid=False, line=0, column=0))
|
||||
)
|
||||
|
||||
def test_invalid_launch_process_reports_explicit_no_creation(self):
|
||||
class InvalidProcess:
|
||||
def IsValid(self):
|
||||
@@ -82,6 +121,7 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
result = tracer.trace(Debugger(), {
|
||||
"executable": "fixture", "architecture": "arm64", "expected_sha256": "expected",
|
||||
"breakpoints": [], "working_directory": "fixture/absent",
|
||||
"capture_limits": {"output_bytes": 1024},
|
||||
"stdout_path": "stdout", "stdout_capture_path": "stdout-capture",
|
||||
"stderr_path": "stderr", "stderr_capture_path": "stderr-capture",
|
||||
})
|
||||
@@ -281,37 +321,44 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
return super().flush()
|
||||
|
||||
stream = CountingStream()
|
||||
journal = tracer._ObservationJournal(stream)
|
||||
journal = tracer._ObservationJournal(stream, max_bytes=4096)
|
||||
journal.append({"kind": "event", "event": {"sequence": 0}})
|
||||
journal.append({"kind": "other-stop", "reason": "signal"})
|
||||
self.assertEqual((stream.writes, stream.flushes), (2, 2))
|
||||
self.assertEqual(journal.bytes, len(stream.getvalue()))
|
||||
|
||||
def test_aggregate_trace_budget_admits_exact_boundary_and_rejects_next_record(self):
|
||||
self.assertTrue(
|
||||
tracer._can_retain_trace(
|
||||
tracer.MAX_TRACE_BYTES - 10,
|
||||
tracer.MAX_TRACE_FRAMES - 2,
|
||||
10,
|
||||
2,
|
||||
)
|
||||
)
|
||||
self.assertFalse(
|
||||
tracer._can_retain_trace(
|
||||
tracer.MAX_TRACE_BYTES - 10,
|
||||
tracer.MAX_TRACE_FRAMES - 2,
|
||||
11,
|
||||
2,
|
||||
)
|
||||
)
|
||||
self.assertFalse(
|
||||
tracer._can_retain_trace(
|
||||
tracer.MAX_TRACE_BYTES - 10,
|
||||
tracer.MAX_TRACE_FRAMES - 2,
|
||||
10,
|
||||
3,
|
||||
)
|
||||
def test_aggregate_trace_budget_uses_the_selected_admission(self):
|
||||
limits = {"trace_bytes": 100, "trace_frames": 10}
|
||||
self.assertTrue(tracer._can_retain_trace(90, 8, 10, 2, limits))
|
||||
self.assertFalse(tracer._can_retain_trace(90, 8, 11, 2, limits))
|
||||
self.assertFalse(tracer._can_retain_trace(90, 8, 10, 3, limits))
|
||||
raised = {"trace_bytes": 101, "trace_frames": 11}
|
||||
self.assertTrue(tracer._can_retain_trace(90, 8, 11, 3, raised))
|
||||
|
||||
def test_breakpoint_reports_exact_exhausted_budget_and_accepts_more_room(self):
|
||||
frame = types.SimpleNamespace(
|
||||
IsValid=lambda: True, GetPC=lambda: 1,
|
||||
GetThread=lambda: types.SimpleNamespace(GetNumFrames=lambda: 1),
|
||||
GetPCAddress=lambda: object(),
|
||||
)
|
||||
context = {
|
||||
"target": object(), "events": [], "retained_frames": 1,
|
||||
"retained_trace_bytes": 0,
|
||||
"config": {"breakpoints": [{}], "backtrace_frames": 0, "max_events": 10,
|
||||
"capture_limits": {"trace_bytes": 4096, "trace_frames": 1}},
|
||||
}
|
||||
with patch.object(tracer, "_location", return_value={"symbol": "main"}), patch.object(
|
||||
tracer, "_wanted", return_value=True
|
||||
), patch.object(tracer, "_record", return_value={"backtrace": []}):
|
||||
self.assertEqual(tracer._handle_breakpoint(context, frame, 0), "resource-limit")
|
||||
self.assertEqual(context["resource_limit"], "trace_frames")
|
||||
context["config"]["capture_limits"] = {"trace_bytes": 8, "trace_frames": 2}
|
||||
self.assertEqual(tracer._handle_breakpoint(context, frame, 0), "resource-limit")
|
||||
self.assertEqual(context["resource_limit"], "trace_bytes")
|
||||
self.assertEqual(context["events"], [])
|
||||
context["config"]["capture_limits"]["trace_bytes"] = 4096
|
||||
self.assertIsNone(tracer._handle_breakpoint(context, frame, 0))
|
||||
self.assertEqual(context["events"], [{"backtrace": []}])
|
||||
|
||||
def test_oversized_producer_string_is_rejected_before_serialization(self):
|
||||
self.assertTrue(tracer._fits_trace_budget("method", 1024))
|
||||
@@ -390,6 +437,9 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
return FileSpec()
|
||||
|
||||
class Address:
|
||||
def GetLineEntry(self):
|
||||
return types.SimpleNamespace(IsValid=lambda: False)
|
||||
|
||||
def GetModule(self):
|
||||
return Module()
|
||||
|
||||
@@ -417,6 +467,7 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
object(),
|
||||
[[Breakpoint()]],
|
||||
{
|
||||
"capture_limits": {"breakpoint_location_bytes": 4096},
|
||||
"breakpoints": [
|
||||
{
|
||||
"kind": "objc-method",
|
||||
@@ -451,7 +502,7 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
side_effect=lambda address, _target: {"address": address, "symbol": None},
|
||||
), patch.object(tracer, "_wanted", return_value=True):
|
||||
report, locations_truncated = tracer._breakpoint_report(
|
||||
object(), [[Breakpoint()]], {"breakpoints": [{}]}
|
||||
object(), [[Breakpoint()]], {"breakpoints": [{}], "capture_limits": {"breakpoint_location_bytes": 4096}}
|
||||
)
|
||||
|
||||
self.assertEqual(report[0]["location_count"], 65)
|
||||
@@ -491,7 +542,24 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
def GetFileSpec(self):
|
||||
return FileSpec()
|
||||
|
||||
class SourceFileSpec:
|
||||
def IsValid(self):
|
||||
return True
|
||||
|
||||
def __str__(self):
|
||||
return source_path
|
||||
|
||||
class Address:
|
||||
def GetLineEntry(self):
|
||||
if source_path is None:
|
||||
return types.SimpleNamespace(IsValid=lambda: False)
|
||||
return types.SimpleNamespace(
|
||||
IsValid=lambda: True,
|
||||
GetFileSpec=SourceFileSpec,
|
||||
GetLine=lambda: 15,
|
||||
GetColumn=lambda: 3,
|
||||
)
|
||||
|
||||
def GetModule(self):
|
||||
return Module()
|
||||
|
||||
@@ -515,22 +583,39 @@ class NativeTraceBudgetTests(unittest.TestCase):
|
||||
def GetLocationAtIndex(self, index):
|
||||
return Location()
|
||||
|
||||
breakpoint_count = 1000
|
||||
breakpoint_count = 30
|
||||
created = [[Breakpoint()] for _ in range(breakpoint_count)]
|
||||
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count}
|
||||
report, locations_truncated = tracer._breakpoint_report(
|
||||
object(), created, config
|
||||
)
|
||||
retained_bytes = sum(
|
||||
len(json.dumps(location).encode("utf-8"))
|
||||
+ (1 if index > 0 else 0)
|
||||
for row in report
|
||||
for index, location in enumerate(row["locations"])
|
||||
)
|
||||
self.assertTrue(locations_truncated)
|
||||
self.assertEqual(sum(row["location_count"] for row in report), 64_000)
|
||||
self.assertLess(sum(len(row["locations"]) for row in report), 64_000)
|
||||
self.assertLessEqual(retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES)
|
||||
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count,
|
||||
"capture_limits": {"breakpoint_location_bytes": 512 * 1024}}
|
||||
for source_path in (None, "/tmp/" + "源🚀" * 3000 + "/calls.pas"):
|
||||
with self.subTest(source_path_present=source_path is not None):
|
||||
report, locations_truncated = tracer._breakpoint_report(
|
||||
object(), created, config
|
||||
)
|
||||
retained_bytes = sum(
|
||||
len(json.dumps(location).encode("utf-8"))
|
||||
+ (1 if index > 0 else 0)
|
||||
for row in report
|
||||
for index, location in enumerate(row["locations"])
|
||||
)
|
||||
self.assertTrue(locations_truncated)
|
||||
self.assertEqual(
|
||||
sum(row["location_count"] for row in report), 1920
|
||||
)
|
||||
self.assertLess(
|
||||
sum(len(row["locations"]) for row in report), 1920
|
||||
)
|
||||
self.assertLessEqual(
|
||||
retained_bytes, 512 * 1024
|
||||
)
|
||||
if source_path is not None:
|
||||
self.assertTrue(
|
||||
any(
|
||||
location["source_location"]["file_path"] == source_path
|
||||
for row in report
|
||||
for location in row["locations"]
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -78,14 +78,9 @@ public final class NativeAotExtension {
|
||||
new FrozenObjectAnnotator(program, manager).analyze(located.directory(), scan.getPointerLocations(), monitor, log);
|
||||
}
|
||||
monitor.checkCancelled();
|
||||
report.addProperty("method_tables", manager.getMethodTableCount());
|
||||
JsonObject coverage = auditFrozenObjects(program, scan.getPointerLocations(), monitor);
|
||||
report.add("coverage", coverage);
|
||||
if (coverage.get("frozen_objects_annotated").getAsInt() != coverage.get("frozen_object_candidates").getAsInt())
|
||||
diagnostics.add("Some frozen-object candidates lack committed instance annotations; inspect coverage counts and Ghidra diagnostics.");
|
||||
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
|
||||
report.add("reason", JsonNull.INSTANCE);
|
||||
report.add("derived_memory", derivedRange.deepCopy());
|
||||
JsonArray tables = new JsonArray();
|
||||
for (var mt : manager.getMethodTables()) {
|
||||
monitor.checkCancelled();
|
||||
@@ -94,6 +89,13 @@ public final class NativeAotExtension {
|
||||
identity.addProperty("type", mt.getMTType().getPathName());
|
||||
tables.add(identity);
|
||||
}
|
||||
// Publish recovery facts only after every fallible recovery and audit step.
|
||||
// A failed transaction is rolled back, so it cannot report committed metadata.
|
||||
report.addProperty("method_tables", manager.getMethodTableCount());
|
||||
report.add("coverage", coverage);
|
||||
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
|
||||
report.add("reason", JsonNull.INSTANCE);
|
||||
report.add("derived_memory", derivedRange.deepCopy());
|
||||
report.add("types", tables);
|
||||
commit = true;
|
||||
return finish(log);
|
||||
|
||||
@@ -5,7 +5,6 @@ from pathlib import Path
|
||||
import resource
|
||||
import gdb
|
||||
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
|
||||
|
||||
class ContextFailure(Exception):
|
||||
@@ -79,6 +78,7 @@ def failure_exit(request_path, status, marker):
|
||||
def main():
|
||||
request_path = os.environ["REA_CORE_CONTEXT_REQUEST"]
|
||||
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
|
||||
output_bytes = request["producer_limits"]["output_bytes"]
|
||||
reserve = bytearray(1024 * 1024)
|
||||
try:
|
||||
reply = {"ok": True, "value": inspect(request)}
|
||||
@@ -94,8 +94,8 @@ def main():
|
||||
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
|
||||
try:
|
||||
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
|
||||
if len(encoded) > OUTPUT_BYTES:
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds the 64 MiB reply budget."}).encode("utf-8")
|
||||
if len(encoded) > output_bytes:
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds producer_limits.output_bytes=" + str(output_bytes) + "."}).encode("utf-8")
|
||||
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(descriptor, "wb") as handle:
|
||||
handle.write(encoded)
|
||||
|
||||
@@ -14,7 +14,7 @@ BRIDGE = Path(__file__).with_name("core_context.py")
|
||||
|
||||
|
||||
class CoreContextTests(unittest.TestCase):
|
||||
def inspect(self, failure=None, connection="core", symbols=None, architecture="x86_64-little-endian", objfile="recorded-executable"):
|
||||
def inspect(self, failure=None, connection="core", symbols=None, architecture="x86_64-little-endian", objfile="recorded-executable", output_bytes=16384):
|
||||
commands, reads, queries = [], [], []
|
||||
state = {"executable": None, "symbols": None, "connection": None}
|
||||
gdb = types.ModuleType("gdb")
|
||||
@@ -63,7 +63,7 @@ class CoreContextTests(unittest.TestCase):
|
||||
plugin.write_text("")
|
||||
request = stage / "request.json"
|
||||
reply = stage / "reply.json"
|
||||
request.write_text(json.dumps({"gdbinit": str(plugin), "architecture": architecture, "snapshot_path": str(root / "object.snapshot"), "reply_path": str(reply)}))
|
||||
request.write_text(json.dumps({"gdbinit": str(plugin), "architecture": architecture, "snapshot_path": str(root / "object.snapshot"), "reply_path": str(reply), "producer_limits": {"output_bytes": output_bytes}}))
|
||||
with patch.dict(sys.modules, modules), patch.dict(os.environ, REA_CORE_CONTEXT_REQUEST=str(request)):
|
||||
runpy.run_path(str(BRIDGE))
|
||||
result = json.loads(reply.read_text())
|
||||
@@ -104,14 +104,15 @@ class CoreContextTests(unittest.TestCase):
|
||||
self.assertEqual(reads, [])
|
||||
self.assertEqual(queries, [])
|
||||
|
||||
def test_output_limit_message_separates_the_words_and_names_the_limit(self):
|
||||
result, _, reads, queries = self.inspect(objfile="m" * (64 * 1024 * 1024))
|
||||
def test_selected_output_limit_reports_recovery_and_accepts_more_headroom(self):
|
||||
result, _, reads, queries = self.inspect(objfile="m" * 2048, output_bytes=1024)
|
||||
self.assertFalse(result["ok"])
|
||||
self.assertEqual(result["reason"], "output-limit")
|
||||
self.assertNotIn("exceeds64", result["message"])
|
||||
self.assertEqual(result["message"], "Complete debugger context exceeds the 64 MiB reply budget.")
|
||||
self.assertIn("producer_limits.output_bytes=1024", result["message"])
|
||||
self.assertEqual(reads, [])
|
||||
self.assertEqual(queries, ["maps"])
|
||||
recovered, _, _, _ = self.inspect(objfile="m" * 2048, output_bytes=4096)
|
||||
self.assertTrue(recovered["ok"], recovered)
|
||||
|
||||
def test_wrong_connection_or_remaining_symbols_never_queries_mappings(self):
|
||||
for options in [{"connection": "native"}, {"symbols": "unexpected-symbols"}]:
|
||||
|
||||
@@ -5,14 +5,12 @@ from pathlib import Path
|
||||
import resource
|
||||
import sys
|
||||
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
|
||||
def lower_resource_limits():
|
||||
def lower_resource_limits(policy):
|
||||
limits = {}
|
||||
for name, kind, maximum in (
|
||||
("address_space_bytes", resource.RLIMIT_AS, 3 * 1024**3),
|
||||
("cpu_seconds", resource.RLIMIT_CPU, 30),
|
||||
("file_size_bytes", resource.RLIMIT_FSIZE, OUTPUT_BYTES),
|
||||
("address_space_bytes", resource.RLIMIT_AS, policy["address_space_bytes"]),
|
||||
("cpu_seconds", resource.RLIMIT_CPU, policy["cpu_seconds"]),
|
||||
("file_size_bytes", resource.RLIMIT_FSIZE, policy["file_bytes"]),
|
||||
):
|
||||
soft, hard = resource.getrlimit(kind)
|
||||
finite = [maximum] + [value for value in (soft, hard) if value != resource.RLIM_INFINITY]
|
||||
@@ -25,6 +23,11 @@ def lower_resource_limits():
|
||||
|
||||
def main(request_path, inspect, failure_type, profile):
|
||||
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
|
||||
policy = request["producer_limits"]
|
||||
for key in ("input_bytes", "output_bytes", "diagnostic_bytes", "address_space_bytes", "file_bytes", "cpu_seconds"):
|
||||
value = policy[key]
|
||||
if type(value) is not int or not 1 <= value <= 9007199254740991:
|
||||
raise ValueError("producer_limits." + key + " must be a positive safe integer")
|
||||
os.environ["PWNLIB_NOTERM"] = "1"
|
||||
os.environ["PWNLIB_CACHE_DIR"] = str(Path(request_path).parent / "cache")
|
||||
limits = None
|
||||
@@ -32,7 +35,7 @@ def main(request_path, inspect, failure_type, profile):
|
||||
# The outer exit-status contract also covers allocation failure during serialization.
|
||||
memory_reserve = bytearray(1024 * 1024)
|
||||
try:
|
||||
limits = lower_resource_limits()
|
||||
limits = lower_resource_limits(policy)
|
||||
descriptor = os.open(Path(request_path).parent / "limits.json", os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
|
||||
json.dump(limits, handle)
|
||||
@@ -51,8 +54,8 @@ def main(request_path, inspect, failure_type, profile):
|
||||
else:
|
||||
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
|
||||
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
|
||||
if len(encoded) > OUTPUT_BYTES:
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete artifact result exceeds the 64 MiB reply budget."}).encode("utf-8")
|
||||
if len(encoded) > policy["output_bytes"]:
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete artifact result exceeds producer_limits.output_bytes=" + str(policy["output_bytes"]) + "."}).encode("utf-8")
|
||||
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(descriptor, "wb") as handle:
|
||||
handle.write(encoded)
|
||||
|
||||
@@ -135,13 +135,27 @@ def inspect_elf(path, cache):
|
||||
from pwnlib.context import context
|
||||
with context.local(cache_dir=str(cache), log_level="warning"):
|
||||
from elftools.common.exceptions import ELFError
|
||||
from elftools.common.utils import struct_parse
|
||||
from elftools.elf.elffile import ELFFile
|
||||
from pwnlib.elf import ELF
|
||||
try:
|
||||
headers = ELFFile(io.BytesIO(content))
|
||||
section_count = headers.num_sections()
|
||||
if section_count:
|
||||
if headers.header.e_shentsize < headers.structs.Elf_Shdr.sizeof():
|
||||
raise LayoutFailure("format", "ELF section-header entry is shorter than the required structure.")
|
||||
location(headers.header.e_shoff, section_count * headers.header.e_shentsize, len(content))
|
||||
# Upstream symbol, relocation and dynamic access reads stored stream
|
||||
# bytes, while this profile reports logical names and file coordinates.
|
||||
# A string table is read only when it names sections or symbols.
|
||||
raw_headers = []
|
||||
for index in range(section_count):
|
||||
raw_header = struct_parse(headers.structs.Elf_Shdr, headers.stream,
|
||||
stream_pos=headers.header.e_shoff + index * headers.header.e_shentsize)
|
||||
raw_headers.append(raw_header)
|
||||
if raw_header.sh_flags & 0x800 and raw_header.sh_type in (
|
||||
"SHT_SYMTAB", "SHT_DYNSYM", "SHT_REL", "SHT_RELA", "SHT_RELR", "SHT_DYNAMIC"):
|
||||
raise LayoutFailure("unsupported-target", f"ELF section {index} ({raw_header.sh_type}) is SHF_COMPRESSED; logical metadata and its source coordinates are not decoded by this profile.")
|
||||
segment_count = headers.num_segments()
|
||||
if segment_count:
|
||||
location(headers.header.e_phoff, segment_count * headers.header.e_phentsize, len(content))
|
||||
@@ -153,12 +167,22 @@ def inspect_elf(path, cache):
|
||||
section_name_index = headers.get_shstrndx()
|
||||
if raw_name_index == 0xffff and section_name_index < 0xff00:
|
||||
raise LayoutFailure("format", f"ELF SHN_XINDEX resolves to ordinary index {section_name_index}; the escape requires an index at least SHN_LORESERVE (0xff00).")
|
||||
consumed_strings = set()
|
||||
if section_name_index != 0:
|
||||
if section_name_index >= section_count:
|
||||
raise LayoutFailure("format", f"ELF section-name table index {section_name_index} lies outside the declared section count {section_count}.")
|
||||
names_header = headers._get_section_header(section_name_index)
|
||||
if names_header.sh_type != "SHT_STRTAB":
|
||||
raise LayoutFailure("format", f"ELF section-name table {section_name_index} is not SHT_STRTAB.")
|
||||
consumed_strings.add(section_name_index)
|
||||
for header in raw_headers:
|
||||
if header.sh_type in ("SHT_SYMTAB", "SHT_DYNSYM"):
|
||||
consumed_strings.add(header.sh_link)
|
||||
for index, raw_header in enumerate(raw_headers):
|
||||
if (raw_header.sh_flags & 0x800 and raw_header.sh_type == "SHT_STRTAB"
|
||||
and index in consumed_strings):
|
||||
raise LayoutFailure("unsupported-target", f"ELF section {index} (SHT_STRTAB) is SHF_COMPRESSED; logical metadata and its source coordinates are not decoded by this profile.")
|
||||
if section_name_index != 0:
|
||||
location(names_header.sh_offset, names_header.sh_size, len(content))
|
||||
except (ELFError, ValueError, OverflowError, AssertionError, IndexError) as error:
|
||||
raise LayoutFailure("format", str(error)) from error
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
"""Producer admission regressions; no pwntools imports or target execution."""
|
||||
from contextlib import nullcontext
|
||||
import importlib.metadata
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
|
||||
layout = runpy.run_path(str(Path(__file__).with_name("layout_impl.py")))
|
||||
|
||||
|
||||
class UpstreamReached(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class LayoutStorageTests(unittest.TestCase):
|
||||
def inspect(self, section_type, flags, section_stride=64, sections=None, shstrndx=0):
|
||||
described = sections if sections is not None else [
|
||||
types.SimpleNamespace(sh_type="SHT_NULL", sh_flags=0, sh_link=0),
|
||||
types.SimpleNamespace(sh_type=section_type, sh_flags=flags, sh_link=0),
|
||||
]
|
||||
class Headers:
|
||||
header = types.SimpleNamespace(e_shoff=64, e_shentsize=section_stride, e_shstrndx=shstrndx)
|
||||
structs = types.SimpleNamespace(Elf_Shdr=types.SimpleNamespace(sizeof=lambda: 64))
|
||||
stream = object()
|
||||
|
||||
def num_sections(self): return len(described)
|
||||
def num_segments(self): return 0
|
||||
def get_shstrndx(self): return self.header.e_shstrndx
|
||||
def _get_section_header(self, index):
|
||||
return described[index]
|
||||
|
||||
class Elf:
|
||||
def __init__(self, *_args, **_kwargs):
|
||||
raise UpstreamReached("No artifact decoding or emulation is allowed in this test")
|
||||
|
||||
modules = {name: types.ModuleType(name) for name in (
|
||||
"pwnlib", "pwnlib.context", "pwnlib.elf", "elftools", "elftools.common",
|
||||
"elftools.common.exceptions", "elftools.common.utils", "elftools.elf", "elftools.elf.elffile")}
|
||||
modules["pwnlib.context"].context = types.SimpleNamespace(local=lambda **_kwargs: nullcontext())
|
||||
modules["pwnlib.elf"].ELF = Elf
|
||||
modules["elftools.common.exceptions"].ELFError = type("ELFError", (Exception,), {})
|
||||
modules["elftools.elf.elffile"].ELFFile = lambda _stream: Headers()
|
||||
modules["elftools.common.utils"].struct_parse = lambda _struct, _stream, stream_pos: described[(stream_pos - 64) // max(section_stride, 1)]
|
||||
count = len(described)
|
||||
content = bytearray(64 + count * max(section_stride, 64))
|
||||
content[:7] = b"\x7fELF\x02\x01\x01"
|
||||
content[16:18] = (1).to_bytes(2, "little") # ET_REL
|
||||
content[18:20] = (62).to_bytes(2, "little")
|
||||
content[40:48] = (64).to_bytes(8, "little")
|
||||
content[58:60] = section_stride.to_bytes(2, "little")
|
||||
content[60:62] = count.to_bytes(2, "little")
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
snapshot = Path(directory) / "object.snapshot"
|
||||
snapshot.write_bytes(content)
|
||||
with patch.dict(sys.modules, modules), patch.object(importlib.metadata, "version", side_effect=layout["PACKAGES"].__getitem__):
|
||||
return layout["inspect_elf"](snapshot, Path(directory) / "cache")
|
||||
|
||||
def test_compressed_interpreted_storage_refuses_before_upstream_decoding(self):
|
||||
for section_type in ("SHT_SYMTAB", "SHT_DYNSYM", "SHT_REL", "SHT_RELA", "SHT_RELR", "SHT_DYNAMIC"):
|
||||
with self.subTest(section_type=section_type):
|
||||
with self.assertRaises(layout["LayoutFailure"]) as raised:
|
||||
self.inspect(section_type, 0x800)
|
||||
self.assertEqual(raised.exception.reason, "unsupported-target")
|
||||
self.assertEqual(raised.exception.args[0], f"ELF section 1 ({section_type}) is SHF_COMPRESSED; logical metadata and its source coordinates are not decoded by this profile.")
|
||||
with self.assertRaises(UpstreamReached):
|
||||
self.inspect(section_type, 0)
|
||||
|
||||
def test_unreferenced_compressed_string_table_remains_admitted(self):
|
||||
with self.assertRaises(UpstreamReached):
|
||||
self.inspect("SHT_STRTAB", 0x800)
|
||||
|
||||
def test_consumed_compressed_string_tables_are_refused(self):
|
||||
name_table = [
|
||||
types.SimpleNamespace(sh_type="SHT_NULL", sh_flags=0, sh_link=0),
|
||||
types.SimpleNamespace(sh_type="SHT_STRTAB", sh_flags=0x800, sh_link=0),
|
||||
]
|
||||
linked = [
|
||||
types.SimpleNamespace(sh_type="SHT_NULL", sh_flags=0, sh_link=0),
|
||||
types.SimpleNamespace(sh_type="SHT_SYMTAB", sh_flags=0, sh_link=2),
|
||||
types.SimpleNamespace(sh_type="SHT_STRTAB", sh_flags=0x800, sh_link=0),
|
||||
]
|
||||
cases = (
|
||||
("section-name", name_table, 1, 1),
|
||||
("symbol-link", linked, 0, 2),
|
||||
)
|
||||
for label, sections, shstrndx, index in cases:
|
||||
with self.subTest(label=label):
|
||||
with self.assertRaises(layout["LayoutFailure"]) as raised:
|
||||
self.inspect("SHT_NULL", 0, sections=sections, shstrndx=shstrndx)
|
||||
self.assertEqual(raised.exception.reason, "unsupported-target")
|
||||
self.assertEqual(
|
||||
raised.exception.args[0],
|
||||
f"ELF section {index} (SHT_STRTAB) is SHF_COMPRESSED; logical metadata and its source coordinates are not decoded by this profile.",
|
||||
)
|
||||
|
||||
def test_uninterpreted_compressed_payload_does_not_change_admission(self):
|
||||
with self.assertRaises(UpstreamReached):
|
||||
self.inspect("SHT_PROGBITS", 0x800)
|
||||
|
||||
def test_short_section_header_stride_refuses_before_raw_header_scan(self):
|
||||
for stride in (0, 1, 63):
|
||||
with self.subTest(stride=stride):
|
||||
with self.assertRaises(layout["LayoutFailure"]) as raised:
|
||||
self.inspect("SHT_PROGBITS", 0, stride)
|
||||
self.assertEqual(raised.exception.reason, "format")
|
||||
self.assertIn("section-header entry", str(raised.exception))
|
||||
for stride in (64, 96):
|
||||
with self.subTest(stride=stride):
|
||||
with self.assertRaises(UpstreamReached):
|
||||
self.inspect("SHT_PROGBITS", 0, stride)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+88
-117
@@ -1,4 +1,18 @@
|
||||
import { defineConfig } from "vitepress";
|
||||
import { dirname, relative, resolve, sep } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import {
|
||||
documentationRewrites,
|
||||
documentationSections,
|
||||
} from "./documentation.js";
|
||||
|
||||
const documentationRoot = fileURLToPath(new URL("..", import.meta.url));
|
||||
const rewrites: Record<string, string> = {
|
||||
...documentationRewrites,
|
||||
"README.md": "documentation.md",
|
||||
"specs/README.md": "specs/index.md",
|
||||
"adr/README.md": "adr/index.md",
|
||||
};
|
||||
|
||||
export default defineConfig({
|
||||
title: "REA",
|
||||
@@ -6,135 +20,92 @@ export default defineConfig({
|
||||
"Reverse engineer binaries, applications, and runtime behavior with one CLI and MCP server.",
|
||||
lang: "en-US",
|
||||
base: "/rea/",
|
||||
rewrites,
|
||||
markdown: {
|
||||
config(md) {
|
||||
// Repository links resolve from their physical source. Render them from
|
||||
// the stable public route instead, including links across categories.
|
||||
md.core.ruler.after("inline", "documentation-source-links", (state) => {
|
||||
const physicalPath =
|
||||
state.env.realPath ??
|
||||
(state.env.relativePath
|
||||
? resolve(documentationRoot, state.env.relativePath)
|
||||
: null);
|
||||
if (physicalPath === null) return;
|
||||
const renderedPath =
|
||||
state.env.path ??
|
||||
resolve(
|
||||
documentationRoot,
|
||||
rewrites[state.env.relativePath] ?? state.env.relativePath,
|
||||
);
|
||||
for (const block of state.tokens) {
|
||||
for (const token of block.children ?? []) {
|
||||
const attribute =
|
||||
token.type === "link_open"
|
||||
? "href"
|
||||
: token.type === "image"
|
||||
? "src"
|
||||
: null;
|
||||
if (attribute === null) continue;
|
||||
const destination = token.attrGet(attribute);
|
||||
if (
|
||||
!destination ||
|
||||
/^(?:[a-z][a-z\d+.-]*:|[/#])/iu.test(destination)
|
||||
)
|
||||
continue;
|
||||
const separator = destination.search(/[?#]/u);
|
||||
const path =
|
||||
separator === -1 ? destination : destination.slice(0, separator);
|
||||
const suffix = separator === -1 ? "" : destination.slice(separator);
|
||||
const source = relative(
|
||||
documentationRoot,
|
||||
resolve(dirname(physicalPath), path),
|
||||
)
|
||||
.split(sep)
|
||||
.join("/");
|
||||
const rendered = relative(
|
||||
dirname(renderedPath),
|
||||
resolve(
|
||||
documentationRoot,
|
||||
source.startsWith("public/")
|
||||
? source.slice("public/".length)
|
||||
: (rewrites[source] ?? source),
|
||||
),
|
||||
)
|
||||
.split(sep)
|
||||
.join("/");
|
||||
token.attrSet(attribute, rendered + suffix);
|
||||
}
|
||||
}
|
||||
});
|
||||
},
|
||||
},
|
||||
srcExclude: ["readme/**"],
|
||||
themeConfig: {
|
||||
nav: [
|
||||
{ text: "Get started", link: "/installation" },
|
||||
{ text: "Documentation", link: "/documentation" },
|
||||
{ text: "Reference", link: "/mcp-contracts" },
|
||||
{ text: "Roadmap", link: "/roadmap" },
|
||||
],
|
||||
sidebar: [
|
||||
{
|
||||
text: "Getting started",
|
||||
text: "Documentation directory",
|
||||
items: [
|
||||
{ text: "Installation and setup", link: "/installation" },
|
||||
{ text: "Client configuration", link: "/client-configuration" },
|
||||
{ text: "CLI and Evidence", link: "/cli" },
|
||||
{ text: "Agent prompts", link: "/mcp-prompts" },
|
||||
{ text: "Roadmap", link: "/roadmap" },
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Investigation guides",
|
||||
items: [
|
||||
{ text: "Native binaries", link: "/native-investigation" },
|
||||
{ text: "Go build metadata", link: "/go-binaries" },
|
||||
{ text: "IL2CPP metadata", link: "/il2cpp-metadata" },
|
||||
{ text: "Model artifacts", link: "/model-artifacts" },
|
||||
{ text: "Rust dependencies", link: "/rust-binaries" },
|
||||
{ text: "PE certificates", link: "/pe-certificates" },
|
||||
{ text: "PE imports", link: "/pe-imports" },
|
||||
{ text: "PE TLS/load config", link: "/pe-load-config" },
|
||||
{ text: "ELF notes", link: "/elf-notes" },
|
||||
{ text: "Binary hardening", link: "/binary-hardening" },
|
||||
{ text: "Apple IPS crash reports", link: "/apple-ips" },
|
||||
{ text: "Linux kernel modules", link: "/kernel-modules" },
|
||||
{ text: "CPython pyc files", link: "/python-pyc" },
|
||||
{ text: "Erlang BEAM", link: "/erlang-beam" },
|
||||
{
|
||||
text: "JavaScript and Electron",
|
||||
link: "/javascript-artifact-reconstruction",
|
||||
},
|
||||
{
|
||||
text: "JavaScript workflows",
|
||||
link: "/javascript-application-workflows",
|
||||
},
|
||||
{ text: "Browser observation", link: "/browser-observation" },
|
||||
{
|
||||
text: "Web network captures",
|
||||
link: "/web-network-captures",
|
||||
},
|
||||
{
|
||||
text: "Build RL environments",
|
||||
link: "/agent-environments-quickstart",
|
||||
},
|
||||
{ text: "Environment reference", link: "/agent-environments" },
|
||||
{ text: "Electron observation", link: "/electron-observation" },
|
||||
{ text: "Electron fuses", link: "/electron-fuses" },
|
||||
{ text: "Process capture", link: "/process-capture" },
|
||||
{
|
||||
text: "Desktop application capture",
|
||||
link: "/desktop-application-capture",
|
||||
},
|
||||
{
|
||||
text: "Desktop application state",
|
||||
link: "/desktop-application-state",
|
||||
},
|
||||
{
|
||||
text: "Desktop application export",
|
||||
link: "/desktop-application-export",
|
||||
},
|
||||
{
|
||||
text: "Desktop application tasks",
|
||||
link: "/desktop-application-tasks",
|
||||
},
|
||||
{
|
||||
text: "Desktop build computer",
|
||||
link: "/desktop-build-computer",
|
||||
},
|
||||
{
|
||||
text: "Building and witnessing",
|
||||
link: "/desktop-build-witness",
|
||||
},
|
||||
{ text: ".NET assemblies", link: "/managed-code-analysis" },
|
||||
{
|
||||
text: "Python packaged applications",
|
||||
link: "/python-packaged-applications",
|
||||
},
|
||||
{ text: "Android applications", link: "/android-analysis" },
|
||||
{ text: "Apple applications", link: "/apple-application-analysis" },
|
||||
{ text: "Desktop installations", link: "/desktop-applications" },
|
||||
{ text: "Firmware", link: "/firmware-analysis" },
|
||||
{ text: "iNES containers", link: "/ines-containers" },
|
||||
{ text: "SQLite databases", link: "/sqlite-databases" },
|
||||
{ text: "Packet captures", link: "/packet-captures" },
|
||||
{ text: "Signal recordings", link: "/signal-recordings" },
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Provider guides",
|
||||
collapsed: true,
|
||||
items: [
|
||||
{ text: "IDA", link: "/ida-provider" },
|
||||
{ text: "Kuna", link: "/kuna-provider" },
|
||||
{ text: "Ghidra on Windows", link: "/windows-ghidra-p0" },
|
||||
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
|
||||
{ text: "Ghidra raw binary", link: "/raw-binary" },
|
||||
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
|
||||
{ text: "Ghidra PowerPC", link: "/ghidra-ppc" },
|
||||
{ text: "Ghidra GameCube DOL", link: "/ghidra-dol" },
|
||||
{ text: "Ghidra semantics", link: "/ghidra-provider" },
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Reference and development",
|
||||
collapsed: true,
|
||||
items: [
|
||||
{ text: "MCP contracts", link: "/mcp-contracts" },
|
||||
{ text: "Tool design", link: "/tool-design" },
|
||||
{
|
||||
text: "Admission limits and recovery",
|
||||
link: "/design/limits-and-recovery",
|
||||
},
|
||||
{ text: "Testing", link: "/testing" },
|
||||
{ text: "Client verification", link: "/client-verification" },
|
||||
{
|
||||
text: "Reconstruction readiness",
|
||||
link: "/reconstruction-readiness",
|
||||
},
|
||||
{ text: "Releasing", link: "/releasing" },
|
||||
{ text: "All guides", link: "/documentation" },
|
||||
{ text: "Authoring conventions", link: "/development/documentation" },
|
||||
{ text: "Feature specifications", link: "/specs/" },
|
||||
{ text: "Architecture decisions", link: "/adr/" },
|
||||
],
|
||||
},
|
||||
...documentationSections.map((section) => ({
|
||||
text: section.text,
|
||||
collapsed: section.text !== "Getting started",
|
||||
items: section.items.map((item) => ({
|
||||
text: item.text,
|
||||
link: item.route,
|
||||
})),
|
||||
})),
|
||||
],
|
||||
search: { provider: "local" },
|
||||
socialLinks: [
|
||||
|
||||
@@ -0,0 +1,689 @@
|
||||
// Authored guide ownership: physical sources and stable public routes.
|
||||
export const documentationSections = [
|
||||
{
|
||||
text: "Getting started",
|
||||
items: [
|
||||
{
|
||||
text: "Installation and setup",
|
||||
source: "getting-started/installation.md",
|
||||
route: "/installation",
|
||||
},
|
||||
{
|
||||
text: "Client configuration and troubleshooting",
|
||||
source: "getting-started/client-configuration.md",
|
||||
route: "/client-configuration",
|
||||
},
|
||||
{
|
||||
text: "CLI and Evidence",
|
||||
source: "getting-started/cli.md",
|
||||
route: "/cli",
|
||||
},
|
||||
{
|
||||
text: "Guided MCP prompts and completion",
|
||||
source: "getting-started/mcp-prompts.md",
|
||||
route: "/mcp-prompts",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Native binaries",
|
||||
items: [
|
||||
{
|
||||
text: "Native binary investigation",
|
||||
source: "guides/native/native-investigation.md",
|
||||
route: "/native-investigation",
|
||||
},
|
||||
{
|
||||
text: "Go binary build metadata",
|
||||
source: "guides/native/go-binaries.md",
|
||||
route: "/go-binaries",
|
||||
},
|
||||
{
|
||||
text: "Offline binary diagnostics",
|
||||
source: "guides/native/binary-diagnostics.md",
|
||||
route: "/binary-diagnostics",
|
||||
},
|
||||
{
|
||||
text: "ELF and Mach-O initializer functions",
|
||||
source: "guides/native/native-initializers.md",
|
||||
route: "/native-initializers",
|
||||
},
|
||||
{
|
||||
text: "ELF unwind function ranges",
|
||||
source: "guides/native/elf-unwind-ranges.md",
|
||||
route: "/elf-unwind-ranges",
|
||||
},
|
||||
{
|
||||
text: "Linux kernel build config",
|
||||
source: "guides/native/kernel-build-config.md",
|
||||
route: "/kernel-build-config",
|
||||
},
|
||||
{
|
||||
text: "Recorded crash evidence",
|
||||
source: "guides/native/recorded-crashes.md",
|
||||
route: "/recorded-crashes",
|
||||
},
|
||||
{
|
||||
text: "Windows Shell Link shortcuts",
|
||||
source: "guides/native/windows-shortcut.md",
|
||||
route: "/windows-shortcut",
|
||||
},
|
||||
{
|
||||
source: "guides/native/ntfs-metadata.md",
|
||||
route: "/ntfs-metadata",
|
||||
text: "Acquired NTFS metadata",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "JavaScript and Electron",
|
||||
items: [
|
||||
{
|
||||
text: "JavaScript Application Graph",
|
||||
source: "guides/javascript/javascript-application-graph.md",
|
||||
route: "/javascript-application-graph",
|
||||
},
|
||||
{
|
||||
text: "Cross-layer JavaScript application workflows",
|
||||
source: "guides/javascript/javascript-application-workflows.md",
|
||||
route: "/javascript-application-workflows",
|
||||
},
|
||||
{
|
||||
text: "JavaScript artifact reconstruction",
|
||||
source: "guides/javascript/javascript-artifact-reconstruction.md",
|
||||
route: "/javascript-artifact-reconstruction",
|
||||
},
|
||||
{
|
||||
text: "JavaScript source recovery",
|
||||
source: "guides/javascript/javascript-recovery.md",
|
||||
route: "/javascript-recovery",
|
||||
},
|
||||
{
|
||||
text: "Passive Node and Electron runtime observation",
|
||||
source: "guides/javascript/javascript-runtime-observation.md",
|
||||
route: "/javascript-runtime-observation",
|
||||
},
|
||||
{
|
||||
text: "JavaScript static/runtime reconciliation",
|
||||
source: "guides/javascript/javascript-runtime-reconciliation.md",
|
||||
route: "/javascript-runtime-reconciliation",
|
||||
},
|
||||
{
|
||||
text: "Electron file-page observation",
|
||||
source: "guides/javascript/electron-observation.md",
|
||||
route: "/electron-observation",
|
||||
},
|
||||
{
|
||||
text: "V8 code cache inspection",
|
||||
source: "guides/javascript/v8-code-cache.md",
|
||||
route: "/v8-code-cache",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Browser and captured websites",
|
||||
items: [
|
||||
{
|
||||
text: "Website observation with CDP",
|
||||
source: "guides/web/browser-observation.md",
|
||||
route: "/browser-observation",
|
||||
},
|
||||
{
|
||||
text: "Browser scenario contract",
|
||||
source: "guides/web/browser-scenario-contract.md",
|
||||
route: "/browser-scenario-contract",
|
||||
},
|
||||
{
|
||||
text: "Website runtime source attribution",
|
||||
source: "guides/web/web-runtime.md",
|
||||
route: "/web-runtime",
|
||||
},
|
||||
{
|
||||
text: "Historical web network captures",
|
||||
source: "guides/web/web-network-captures.md",
|
||||
route: "/web-network-captures",
|
||||
},
|
||||
{
|
||||
text: "HTTP/3 capture lab",
|
||||
source: "http3-capture-lab.md",
|
||||
route: "/http3-capture-lab",
|
||||
},
|
||||
{
|
||||
text: "Captured website source locations",
|
||||
source: "guides/web/web-source-location.md",
|
||||
route: "/web-source-location",
|
||||
},
|
||||
{
|
||||
text: "Captured website module relationships",
|
||||
source: "guides/web/website-module-trace.md",
|
||||
route: "/website-module-trace",
|
||||
},
|
||||
{
|
||||
text: "Captured website scripts and static JavaScript analysis",
|
||||
source: "guides/web/website-script-export.md",
|
||||
route: "/website-script-export",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Android applications",
|
||||
items: [
|
||||
{
|
||||
text: "Static Android APK analysis",
|
||||
source: "guides/android/android-analysis.md",
|
||||
route: "/android-analysis",
|
||||
},
|
||||
{
|
||||
text: "Apktool resource decoding",
|
||||
source: "guides/android/apktool-resource-analysis.md",
|
||||
route: "/apktool-resource-analysis",
|
||||
},
|
||||
{
|
||||
text: "ADB device inspection and acquisition",
|
||||
source: "guides/android/adb-device-analysis.md",
|
||||
route: "/adb-device-analysis",
|
||||
},
|
||||
{
|
||||
text: "Flutter build identification",
|
||||
source: "guides/android/flutter-analysis.md",
|
||||
route: "/flutter-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Apple applications",
|
||||
items: [
|
||||
{
|
||||
text: "Apple Application Inventory Projection",
|
||||
source: "guides/apple/apple-application-analysis.md",
|
||||
route: "/apple-application-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: ".NET assemblies",
|
||||
items: [
|
||||
{
|
||||
text: "Managed-code analysis",
|
||||
source: "guides/managed/managed-code-analysis.md",
|
||||
route: "/managed-code-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Firmware",
|
||||
items: [
|
||||
{
|
||||
text: "Firmware analysis",
|
||||
source: "guides/firmware/firmware-analysis.md",
|
||||
route: "/firmware-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "HarmonyOS packages",
|
||||
items: [
|
||||
{
|
||||
text: "Static HarmonyOS package inventory",
|
||||
source: "guides/harmony/harmony-analysis.md",
|
||||
route: "/harmony-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "WebAssembly",
|
||||
items: [
|
||||
{
|
||||
text: "Offline WebAssembly artifacts",
|
||||
source: "guides/wasm/wasm-artifacts.md",
|
||||
route: "/wasm-artifacts",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "EVM contracts",
|
||||
items: [
|
||||
{
|
||||
text: "Offline EVM bytecode",
|
||||
source: "guides/evm/evm-bytecode.md",
|
||||
route: "/evm-bytecode",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Databases and recorded packets",
|
||||
items: [
|
||||
{
|
||||
text: "SQLite database snapshots",
|
||||
source: "guides/data/sqlite-databases.md",
|
||||
route: "/sqlite-databases",
|
||||
},
|
||||
{
|
||||
text: "Recorded packet capture inspection",
|
||||
source: "guides/data/packet-captures.md",
|
||||
route: "/packet-captures",
|
||||
},
|
||||
{
|
||||
text: "Packet capture summaries",
|
||||
source: "guides/data/pcap-analysis.md",
|
||||
route: "/pcap-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Process capture and instrumentation",
|
||||
items: [
|
||||
{
|
||||
text: "Process capture",
|
||||
source: "guides/runtime/process-capture.md",
|
||||
route: "/process-capture",
|
||||
},
|
||||
{
|
||||
text: "Frida instrumentation",
|
||||
source: "guides/runtime/frida-instrumentation.md",
|
||||
route: "/frida-instrumentation",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Provider setup and formats",
|
||||
items: [
|
||||
{
|
||||
text: "Ghidra provider semantics",
|
||||
source: "providers/ghidra-provider.md",
|
||||
route: "/ghidra-provider",
|
||||
},
|
||||
{
|
||||
text: "DOS MZ and COM analysis with Ghidra",
|
||||
source: "providers/ghidra-dos.md",
|
||||
route: "/ghidra-dos",
|
||||
},
|
||||
{
|
||||
text: "Optional NativeAOT metadata recovery",
|
||||
source: "providers/ghidra-nativeaot.md",
|
||||
route: "/ghidra-nativeaot",
|
||||
},
|
||||
{
|
||||
text: "MIPS ELF analysis through Ghidra",
|
||||
source: "providers/ghidra-mips.md",
|
||||
route: "/ghidra-mips",
|
||||
},
|
||||
{
|
||||
text: "PSP static ELF analysis through Ghidra",
|
||||
source: "providers/ghidra-psp.md",
|
||||
route: "/ghidra-psp",
|
||||
},
|
||||
{
|
||||
text: "RISC-V ELF analysis through Ghidra",
|
||||
source: "providers/ghidra-riscv.md",
|
||||
route: "/ghidra-riscv",
|
||||
},
|
||||
{
|
||||
text: "Windows Ghidra P0",
|
||||
source: "providers/windows-ghidra-p0.md",
|
||||
route: "/windows-ghidra-p0",
|
||||
},
|
||||
{
|
||||
text: "Headerless raw-binary analysis with Ghidra",
|
||||
source: "providers/raw-binary.md",
|
||||
route: "/raw-binary",
|
||||
},
|
||||
{
|
||||
text: "IDA provider",
|
||||
source: "providers/ida-provider.md",
|
||||
route: "/ida-provider",
|
||||
},
|
||||
{
|
||||
text: "JEB instrumentation and analysis",
|
||||
source: "providers/jeb-analysis.md",
|
||||
route: "/jeb-analysis",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Contracts and evidence",
|
||||
items: [
|
||||
{
|
||||
text: "MCP runtime contracts",
|
||||
source: "reference/mcp-contracts.md",
|
||||
route: "/mcp-contracts",
|
||||
},
|
||||
{
|
||||
text: "Reconstruction obligation ledgers",
|
||||
source: "reference/reconstruction-obligation-ledgers.md",
|
||||
route: "/reconstruction-obligation-ledgers",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Development and verification",
|
||||
items: [
|
||||
{
|
||||
text: "Settings and recovery",
|
||||
source: "design/limits-and-recovery.md",
|
||||
route: "/design/limits-and-recovery",
|
||||
},
|
||||
{
|
||||
text: "MCP tool design",
|
||||
source: "development/tool-design.md",
|
||||
route: "/tool-design",
|
||||
},
|
||||
{
|
||||
text: "Testing REA",
|
||||
source: "development/testing.md",
|
||||
route: "/testing",
|
||||
},
|
||||
{
|
||||
text: "Client and agent verification",
|
||||
source: "development/client-verification.md",
|
||||
route: "/client-verification",
|
||||
},
|
||||
{
|
||||
text: "Reconstruction readiness conformance",
|
||||
source: "development/reconstruction-readiness.md",
|
||||
route: "/reconstruction-readiness",
|
||||
},
|
||||
{
|
||||
text: "Releasing REA",
|
||||
source: "development/releasing.md",
|
||||
route: "/releasing",
|
||||
},
|
||||
{
|
||||
text: "REA roadmap",
|
||||
source: "development/roadmap.md",
|
||||
route: "/roadmap",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Release migrations",
|
||||
items: [
|
||||
{
|
||||
text: "Upgrading from REA 6.1 to 6.2",
|
||||
source: "migrations/migration-6.2.md",
|
||||
route: "/migration-6.2",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Signal recordings",
|
||||
items: [
|
||||
{
|
||||
text: "Recorded signal inspection",
|
||||
source: "guides/signals/signal-recordings.md",
|
||||
route: "/signal-recordings",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
text: "Additional guides",
|
||||
items: [
|
||||
{
|
||||
text: "Offline API contract inventory",
|
||||
source: "api-contracts.md",
|
||||
route: "/api-contracts",
|
||||
},
|
||||
{
|
||||
text: "Bun standalone executables",
|
||||
source: "bun-standalone.md",
|
||||
route: "/bun-standalone",
|
||||
},
|
||||
{
|
||||
text: "Deno compile executables",
|
||||
source: "deno-standalone.md",
|
||||
route: "/deno-standalone",
|
||||
},
|
||||
{
|
||||
text: "Capability composition and ownership",
|
||||
source: "capability-composition.md",
|
||||
route: "/capability-composition",
|
||||
},
|
||||
{
|
||||
text: ".NET single-file bundles",
|
||||
source: "dotnet-bundles.md",
|
||||
route: "/dotnet-bundles",
|
||||
},
|
||||
{
|
||||
text: "ESP flash images",
|
||||
source: "esp-flash-image.md",
|
||||
route: "/esp-flash-image",
|
||||
},
|
||||
{
|
||||
text: "U-Boot FIT images",
|
||||
source: "fit-images.md",
|
||||
route: "/fit-images",
|
||||
},
|
||||
{
|
||||
text: "Measure MCP catalog costs in Codex",
|
||||
source: "mcp-catalog-evaluation.md",
|
||||
route: "/mcp-catalog-evaluation",
|
||||
},
|
||||
{
|
||||
text: "Offline Playwright traces",
|
||||
source: "playwright-traces.md",
|
||||
route: "/playwright-traces",
|
||||
},
|
||||
{
|
||||
text: "Testing lanes and provider verification",
|
||||
source: "testing-lanes.md",
|
||||
route: "/testing-lanes",
|
||||
},
|
||||
{
|
||||
text: "UF2 and DfuSe images",
|
||||
source: "uf2-dfuse.md",
|
||||
route: "/uf2-dfuse",
|
||||
},
|
||||
{
|
||||
text: "WeChat mini-program wxapkg archives",
|
||||
source: "wxapkg.md",
|
||||
route: "/wxapkg",
|
||||
},
|
||||
{
|
||||
text: "Binary hardening",
|
||||
source: "binary-hardening.md",
|
||||
route: "/binary-hardening",
|
||||
},
|
||||
{
|
||||
text: "Desktop application capture",
|
||||
source: "desktop-application-capture.md",
|
||||
route: "/desktop-application-capture",
|
||||
},
|
||||
{
|
||||
text: "Desktop application export",
|
||||
source: "desktop-application-export.md",
|
||||
route: "/desktop-application-export",
|
||||
},
|
||||
{
|
||||
text: "Desktop application state",
|
||||
source: "desktop-application-state.md",
|
||||
route: "/desktop-application-state",
|
||||
},
|
||||
{
|
||||
text: "Desktop application tasks",
|
||||
source: "desktop-application-tasks.md",
|
||||
route: "/desktop-application-tasks",
|
||||
},
|
||||
{
|
||||
text: "Desktop build computer",
|
||||
source: "desktop-build-computer.md",
|
||||
route: "/desktop-build-computer",
|
||||
},
|
||||
{
|
||||
text: "Desktop build witness",
|
||||
source: "desktop-build-witness.md",
|
||||
route: "/desktop-build-witness",
|
||||
},
|
||||
{
|
||||
text: "iNES containers",
|
||||
source: "ines-containers.md",
|
||||
route: "/ines-containers",
|
||||
},
|
||||
{
|
||||
text: "Build RL environments with REA",
|
||||
source: "agent-environments-quickstart.md",
|
||||
route: "/agent-environments-quickstart",
|
||||
},
|
||||
{
|
||||
text: "Agent environments from recorded captures",
|
||||
source: "agent-environments.md",
|
||||
route: "/agent-environments",
|
||||
},
|
||||
{
|
||||
text: "APK signature schemes",
|
||||
source: "apk-signature.md",
|
||||
route: "/apk-signature",
|
||||
},
|
||||
{
|
||||
text: "Apple IPS crash reports",
|
||||
source: "apple-ips.md",
|
||||
route: "/apple-ips",
|
||||
},
|
||||
{
|
||||
text: "Static analysis in GitHub Actions",
|
||||
source: "ci-analysis.md",
|
||||
route: "/ci-analysis",
|
||||
},
|
||||
{
|
||||
text: "Desktop installation directories",
|
||||
source: "desktop-applications.md",
|
||||
route: "/desktop-applications",
|
||||
},
|
||||
{
|
||||
text: "Flattened devicetree inspection",
|
||||
source: "devicetree.md",
|
||||
route: "/devicetree",
|
||||
},
|
||||
{
|
||||
text: "Electron fuse state",
|
||||
source: "electron-fuses.md",
|
||||
route: "/electron-fuses",
|
||||
},
|
||||
{ text: "ELF note tables", source: "elf-notes.md", route: "/elf-notes" },
|
||||
{
|
||||
text: "Erlang BEAM files",
|
||||
source: "erlang-beam.md",
|
||||
route: "/erlang-beam",
|
||||
},
|
||||
{ text: "ESP images", source: "esp-image.md", route: "/esp-image" },
|
||||
{
|
||||
text: "GameCube DOL analysis through Ghidra",
|
||||
source: "ghidra-dol.md",
|
||||
route: "/ghidra-dol",
|
||||
},
|
||||
{
|
||||
text: "PowerPC ELF analysis through Ghidra",
|
||||
source: "ghidra-ppc.md",
|
||||
route: "/ghidra-ppc",
|
||||
},
|
||||
{
|
||||
text: "Godot PCK archives",
|
||||
source: "godot-pck.md",
|
||||
route: "/godot-pck",
|
||||
},
|
||||
{
|
||||
text: "Static CUDA and HIP code objects",
|
||||
source: "gpu-code-objects.md",
|
||||
route: "/gpu-code-objects",
|
||||
},
|
||||
{
|
||||
text: "Intel HEX and S-record images",
|
||||
source: "hex-records.md",
|
||||
route: "/hex-records",
|
||||
},
|
||||
{
|
||||
text: "IL2CPP metadata",
|
||||
source: "il2cpp-metadata.md",
|
||||
route: "/il2cpp-metadata",
|
||||
},
|
||||
{
|
||||
text: "Static JavaScript route inventory",
|
||||
source: "javascript-route-inventory.md",
|
||||
route: "/javascript-route-inventory",
|
||||
},
|
||||
{
|
||||
text: "Linux kernel modules",
|
||||
source: "kernel-modules.md",
|
||||
route: "/kernel-modules",
|
||||
},
|
||||
{
|
||||
text: "Kuna provider",
|
||||
source: "kuna-provider.md",
|
||||
route: "/kuna-provider",
|
||||
},
|
||||
{
|
||||
text: "Model file declarations",
|
||||
source: "model-artifacts.md",
|
||||
route: "/model-artifacts",
|
||||
},
|
||||
{
|
||||
text: "Match native functions across builds",
|
||||
source: "native-function-matching.md",
|
||||
route: "/native-function-matching",
|
||||
},
|
||||
{
|
||||
text: "Node.js single executable applications",
|
||||
source: "node-sea.md",
|
||||
route: "/node-sea",
|
||||
},
|
||||
{
|
||||
text: "PE Authenticode certificate tables",
|
||||
source: "pe-certificates.md",
|
||||
route: "/pe-certificates",
|
||||
},
|
||||
{
|
||||
text: "PE import and delay-load tables",
|
||||
source: "pe-imports.md",
|
||||
route: "/pe-imports",
|
||||
},
|
||||
{
|
||||
text: "PE TLS callbacks and load configuration",
|
||||
source: "pe-load-config.md",
|
||||
route: "/pe-load-config",
|
||||
},
|
||||
{
|
||||
text: "Python packaged applications",
|
||||
source: "python-packaged-applications.md",
|
||||
route: "/python-packaged-applications",
|
||||
},
|
||||
{
|
||||
text: "CPython pyc inspection",
|
||||
source: "python-pyc.md",
|
||||
route: "/python-pyc",
|
||||
},
|
||||
{
|
||||
text: "Reconstruction coverage",
|
||||
source: "reconstruction-coverage.md",
|
||||
route: "/reconstruction-coverage",
|
||||
},
|
||||
{
|
||||
text: "Reverse-engineering tool integrations",
|
||||
source: "reverse-engineering-tools.md",
|
||||
route: "/reverse-engineering-tools",
|
||||
},
|
||||
{
|
||||
text: "Rust binary dependency metadata",
|
||||
source: "rust-binaries.md",
|
||||
route: "/rust-binaries",
|
||||
},
|
||||
{
|
||||
text: "U-Boot legacy uImage containers",
|
||||
source: "uimage.md",
|
||||
route: "/uimage",
|
||||
},
|
||||
{
|
||||
text: "UnityFS AssetBundle archives",
|
||||
source: "unityfs.md",
|
||||
route: "/unityfs",
|
||||
},
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
export const documentationRewrites = Object.fromEntries(
|
||||
documentationSections.flatMap((section) =>
|
||||
section.items.flatMap((item) => {
|
||||
const target = `${item.route.slice(1)}.md`;
|
||||
return item.source === target ? [] : [[item.source, target]];
|
||||
}),
|
||||
),
|
||||
);
|
||||
+206
@@ -0,0 +1,206 @@
|
||||
# Documentation directory
|
||||
|
||||
REA documentation is organized by reader task and document owner. This directory
|
||||
links every authored guide; the [site](https://rea.tools/) also serves separate
|
||||
product pages. The VitePress documentation source is this `docs/` tree.
|
||||
|
||||
## Getting started
|
||||
|
||||
- [Installation and setup](getting-started/installation.md)
|
||||
- [Client configuration and troubleshooting](getting-started/client-configuration.md)
|
||||
- [CLI and Evidence](getting-started/cli.md)
|
||||
- [Guided MCP prompts and completion](getting-started/mcp-prompts.md)
|
||||
|
||||
## Native binaries
|
||||
|
||||
- [Native binary investigation](guides/native/native-investigation.md)
|
||||
- [Go binary build metadata](guides/native/go-binaries.md)
|
||||
- [Offline binary diagnostics](guides/native/binary-diagnostics.md)
|
||||
- [ELF and Mach-O initializer functions](guides/native/native-initializers.md)
|
||||
- [ELF unwind function ranges](guides/native/elf-unwind-ranges.md)
|
||||
- [Linux kernel build config](guides/native/kernel-build-config.md)
|
||||
- [Recorded crash evidence](guides/native/recorded-crashes.md)
|
||||
- [Windows Shell Link shortcuts](guides/native/windows-shortcut.md)
|
||||
- [Acquired NTFS metadata and change journals](guides/native/ntfs-metadata.md)
|
||||
|
||||
## JavaScript and Electron
|
||||
|
||||
- [JavaScript Application Graph](guides/javascript/javascript-application-graph.md)
|
||||
- [Cross-layer JavaScript application workflows](guides/javascript/javascript-application-workflows.md)
|
||||
- [JavaScript artifact reconstruction](guides/javascript/javascript-artifact-reconstruction.md)
|
||||
- [JavaScript source recovery](guides/javascript/javascript-recovery.md)
|
||||
- [Passive Node and Electron runtime observation](guides/javascript/javascript-runtime-observation.md)
|
||||
- [JavaScript static/runtime reconciliation](guides/javascript/javascript-runtime-reconciliation.md)
|
||||
- [Electron file-page observation](guides/javascript/electron-observation.md)
|
||||
- [V8 code cache inspection](guides/javascript/v8-code-cache.md)
|
||||
|
||||
## Browser and captured websites
|
||||
|
||||
- [Website observation with CDP](guides/web/browser-observation.md)
|
||||
- [Browser scenario contract](guides/web/browser-scenario-contract.md)
|
||||
- [Website runtime source attribution](guides/web/web-runtime.md)
|
||||
- [Historical web network captures](guides/web/web-network-captures.md)
|
||||
- [HTTP/3 capture lab](http3-capture-lab.md)
|
||||
- [Captured website source locations](guides/web/web-source-location.md)
|
||||
- [Captured website module relationships](guides/web/website-module-trace.md)
|
||||
- [Captured website scripts and static JavaScript analysis](guides/web/website-script-export.md)
|
||||
|
||||
## Android applications
|
||||
|
||||
- [Static Android APK analysis](guides/android/android-analysis.md)
|
||||
- [Apktool resource decoding](guides/android/apktool-resource-analysis.md)
|
||||
- [ADB device inspection and acquisition](guides/android/adb-device-analysis.md)
|
||||
- [Flutter build identification](guides/android/flutter-analysis.md)
|
||||
|
||||
## Apple applications
|
||||
|
||||
- [Apple Application Inventory Projection](guides/apple/apple-application-analysis.md)
|
||||
|
||||
## .NET assemblies
|
||||
|
||||
- [Managed-code analysis](guides/managed/managed-code-analysis.md)
|
||||
|
||||
## Firmware
|
||||
|
||||
- [Firmware analysis](guides/firmware/firmware-analysis.md)
|
||||
|
||||
## HarmonyOS packages
|
||||
|
||||
- [Static HarmonyOS package inventory](guides/harmony/harmony-analysis.md)
|
||||
|
||||
## WebAssembly
|
||||
|
||||
- [Offline WebAssembly artifacts](guides/wasm/wasm-artifacts.md)
|
||||
|
||||
## EVM contracts
|
||||
|
||||
- [Offline EVM bytecode](guides/evm/evm-bytecode.md)
|
||||
|
||||
## Databases and recorded packets
|
||||
|
||||
- [SQLite database snapshots](guides/data/sqlite-databases.md)
|
||||
- [Recorded packet capture inspection](guides/data/packet-captures.md)
|
||||
- [Packet capture summaries](guides/data/pcap-analysis.md)
|
||||
|
||||
## Process capture and instrumentation
|
||||
|
||||
- [Process capture](guides/runtime/process-capture.md)
|
||||
- [Frida instrumentation](guides/runtime/frida-instrumentation.md)
|
||||
|
||||
## Signal recordings
|
||||
|
||||
- [Recorded signal inspection](guides/signals/signal-recordings.md)
|
||||
|
||||
## Provider setup and formats
|
||||
|
||||
- [Ghidra provider semantics](providers/ghidra-provider.md)
|
||||
- [DOS MZ and COM analysis with Ghidra](providers/ghidra-dos.md)
|
||||
- [Optional NativeAOT metadata recovery](providers/ghidra-nativeaot.md)
|
||||
- [MIPS ELF analysis through Ghidra](providers/ghidra-mips.md)
|
||||
- [PSP static ELF analysis through Ghidra](providers/ghidra-psp.md)
|
||||
- [RISC-V ELF analysis through Ghidra](providers/ghidra-riscv.md)
|
||||
- [Windows Ghidra P0](providers/windows-ghidra-p0.md)
|
||||
- [Headerless raw-binary analysis with Ghidra](providers/raw-binary.md)
|
||||
- [IDA provider](providers/ida-provider.md)
|
||||
- [JEB instrumentation and analysis](providers/jeb-analysis.md)
|
||||
|
||||
## Contracts and evidence
|
||||
|
||||
- [MCP runtime contracts](reference/mcp-contracts.md)
|
||||
- [Reconstruction obligation ledgers](reference/reconstruction-obligation-ledgers.md)
|
||||
|
||||
## Development and verification
|
||||
|
||||
- [Settings and recovery](design/limits-and-recovery.md)
|
||||
- [Flutter budget calibration](verification/flutter-budget-calibration.md)
|
||||
- [Source-map budget calibration](verification/source-map-budget-calibration.md)
|
||||
- [Source-location delivery qualification](verification/source-location-delivery.md)
|
||||
- [JavaScript precision calibration](verification/javascript-precision-calibration.md)
|
||||
- [MCP tool design](development/tool-design.md)
|
||||
- [Testing REA](development/testing.md)
|
||||
- [Client and agent verification](development/client-verification.md)
|
||||
- [Reconstruction readiness conformance](development/reconstruction-readiness.md)
|
||||
- [Releasing REA](development/releasing.md)
|
||||
- [REA roadmap](development/roadmap.md)
|
||||
|
||||
## Release migrations
|
||||
|
||||
- [Upgrading from REA 6.1 to 6.2](migrations/migration-6.2.md)
|
||||
|
||||
## Architecture and authoring
|
||||
|
||||
- [Architecture map](https://github.com/morluto/rea/blob/main/docs/architecture/architecture.mermaid)
|
||||
- [Documentation ownership and contribution model](development/documentation.md)
|
||||
- [Feature specifications](specs/README.md)
|
||||
- [Architectural decision records](adr/README.md)
|
||||
|
||||
## Machine-owned documentation
|
||||
|
||||
| Path | Owner and update command |
|
||||
| ----------------------------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| `public/product-catalog.json` | Ignored contract projection; `npm run docs:generate` |
|
||||
| `verification/managed-conformance-*.json` | Ignored managed conformance projections; `npm run evidence:generate` |
|
||||
| `error-contract.schema.json` | Tracked schema from contracts; `npm run docs:error-schema` |
|
||||
| `user-visible-errors.csv` | Authored error classification inventory; preserve its schema |
|
||||
| `releases/<version>.contributions.json` | Durable release contribution record; follow the [release guide](development/releasing.md) |
|
||||
| `assets/` | Shared authored images referenced by guides |
|
||||
|
||||
Do not edit generated catalogs or evidence to change product facts. Change their
|
||||
canonical contracts or producer and regenerate. Historical release records retain
|
||||
the exact commit messages and references from the release they describe.
|
||||
|
||||
## Additional guides
|
||||
|
||||
- [Offline API contract inventory](api-contracts.md)
|
||||
- [Bun standalone executables](bun-standalone.md)
|
||||
- [Deno compile executables](deno-standalone.md)
|
||||
- [Capability composition and ownership](capability-composition.md)
|
||||
- [.NET single-file bundles](dotnet-bundles.md)
|
||||
- [ESP flash images](esp-flash-image.md)
|
||||
- [U-Boot FIT images](fit-images.md)
|
||||
- [Measure MCP catalog costs in Codex](mcp-catalog-evaluation.md)
|
||||
- [Offline Playwright traces](playwright-traces.md)
|
||||
- [Testing lanes and provider verification](testing-lanes.md)
|
||||
- [UF2 and DfuSe images](uf2-dfuse.md)
|
||||
- [WeChat mini-program wxapkg archives](wxapkg.md)
|
||||
- [Binary hardening](binary-hardening.md)
|
||||
- [Desktop application capture](desktop-application-capture.md)
|
||||
- [Desktop application export](desktop-application-export.md)
|
||||
- [Desktop application state](desktop-application-state.md)
|
||||
- [Desktop application tasks](desktop-application-tasks.md)
|
||||
- [Desktop build computer](desktop-build-computer.md)
|
||||
- [Desktop build witness](desktop-build-witness.md)
|
||||
- [iNES containers](ines-containers.md)
|
||||
- [Build RL environments with REA](agent-environments-quickstart.md)
|
||||
- [Agent environments from recorded captures](agent-environments.md)
|
||||
- [APK signature schemes](apk-signature.md)
|
||||
- [Apple IPS crash reports](apple-ips.md)
|
||||
- [Static analysis in GitHub Actions](ci-analysis.md)
|
||||
- [Desktop installation directories](desktop-applications.md)
|
||||
- [Flattened devicetree inspection](devicetree.md)
|
||||
- [Electron fuse state](electron-fuses.md)
|
||||
- [ELF note tables](elf-notes.md)
|
||||
- [Erlang BEAM files](erlang-beam.md)
|
||||
- [ESP images](esp-image.md)
|
||||
- [GameCube DOL analysis through Ghidra](ghidra-dol.md)
|
||||
- [PowerPC ELF analysis through Ghidra](ghidra-ppc.md)
|
||||
- [Godot PCK archives](godot-pck.md)
|
||||
- [Static CUDA and HIP code objects](gpu-code-objects.md)
|
||||
- [Intel HEX and S-record images](hex-records.md)
|
||||
- [IL2CPP metadata](il2cpp-metadata.md)
|
||||
- [Static JavaScript route inventory](javascript-route-inventory.md)
|
||||
- [Linux kernel modules](kernel-modules.md)
|
||||
- [Kuna provider](kuna-provider.md)
|
||||
- [Model file declarations](model-artifacts.md)
|
||||
- [Match native functions across builds](native-function-matching.md)
|
||||
- [Node.js single executable applications](node-sea.md)
|
||||
- [PE Authenticode certificate tables](pe-certificates.md)
|
||||
- [PE import and delay-load tables](pe-imports.md)
|
||||
- [PE TLS callbacks and load configuration](pe-load-config.md)
|
||||
- [Python packaged applications](python-packaged-applications.md)
|
||||
- [CPython pyc inspection](python-pyc.md)
|
||||
- [Reconstruction coverage](reconstruction-coverage.md)
|
||||
- [Reverse-engineering tool integrations](reverse-engineering-tools.md)
|
||||
- [Rust binary dependency metadata](rust-binaries.md)
|
||||
- [U-Boot legacy uImage containers](uimage.md)
|
||||
- [UnityFS AssetBundle archives](unityfs.md)
|
||||
@@ -0,0 +1,9 @@
|
||||
# Architectural decision records
|
||||
|
||||
Store one actual decision per `docs/adr/<domain>/<decision-slug>.md`, without a
|
||||
global sequence number. Use meaningful names and the Michael Nygard sections:
|
||||
Status, Context, Decision and Consequences. Link each decision from its owning
|
||||
feature specification.
|
||||
|
||||
See [the authoring guide](../development/documentation.md#feature-lifecycle) for
|
||||
the distinction between specifications, roadmaps and isolated decisions.
|
||||
@@ -11,14 +11,14 @@ with your RL framework to set task goals, check rewards, and run training.
|
||||
|
||||
## Before you start
|
||||
|
||||
Install [REA and its agent integration](installation.md). To record an app,
|
||||
Install [REA and its agent integration](getting-started/installation.md). To record an app,
|
||||
you also need a running test instance and a Chrome-family browser. Choose an
|
||||
instance where the agent is allowed to create, edit, and delete data. REA
|
||||
records the app you provide; you start it and prepare its test data.
|
||||
|
||||
To run the exported environment, install the
|
||||
[AgentEnv framework](https://github.com/scaleapi/agentenv-framework#install)
|
||||
and start Docker. REA's [compatibility check](testing.md#exported-environment-bundles)
|
||||
and start Docker. REA's [compatibility check](development/testing.md#exported-environment-bundles)
|
||||
uses AgentEnv version `0.9.1431` with Python 3.12.
|
||||
|
||||
There are two ways to build an environment: ask your agent to record the app,
|
||||
@@ -52,7 +52,7 @@ work and which need more evidence. The app is used during recording; the
|
||||
exported environment runs from the saved data.
|
||||
|
||||
After your agent exports the folder, skip to
|
||||
[run the environment](#3-run-the-environment). The next section is an
|
||||
[run the environment](#run-the-environment). The next section is an
|
||||
alternative for an existing recording.
|
||||
|
||||
## 2. Or build from a saved recording
|
||||
@@ -91,6 +91,8 @@ A successful build means the folder was created. Check the report to confirm
|
||||
it supports the operations your tasks need. `coverage: null` means the build
|
||||
could not prepare state for this environment and it will use replay.
|
||||
|
||||
<a id="run-the-environment"></a>
|
||||
|
||||
## 3. Run the environment
|
||||
|
||||
Whether your agent exported the folder or you built it from a HAR, run it
|
||||
|
||||
+23
-17
@@ -30,7 +30,7 @@ rea environment-serve manifest.json --evidence capture.json --port 8765
|
||||
```
|
||||
|
||||
The first command inspects the capture; the second projects the
|
||||
[environment manifest](web-network-captures.md#environment-manifest) view from
|
||||
[environment manifest](guides/web/web-network-captures.md#environment-manifest) view from
|
||||
it, where `view.json` wraps the capture Evidence as the inline source:
|
||||
|
||||
```json
|
||||
@@ -107,7 +107,10 @@ refusal names its stage: `environment-build` for the build's own checks,
|
||||
`inspect_web_network_capture` and `inspect_analysis_view` for inspection and
|
||||
projection, `environment-export` for the bundle. SIGINT cancels the build
|
||||
(`cancelled`); before the bundle stage nothing is written, and a cancellation
|
||||
during the write names the bundle it wrote. Stage progress is logged to stderr
|
||||
after publication preserves the bundle receipt. Before publication, a failed
|
||||
write attempts to remove partial output and reports cleanup failures. See
|
||||
[bundle publication and recovery](getting-started/cli.md#export-an-environment-bundle).
|
||||
Stage progress is logged to stderr
|
||||
at info level; the report is the only output on stdout:
|
||||
|
||||
```json
|
||||
@@ -163,7 +166,7 @@ at info level; the report is the only output on stdout:
|
||||
|
||||
`coverage` is the state's read views and write plans summarized, and `null`
|
||||
when the state is unavailable and the bundle serves replay. The
|
||||
[environment coverage](web-network-captures.md#environment-coverage) view
|
||||
[environment coverage](guides/web/web-network-captures.md#environment-coverage) view
|
||||
reports the same summary for any capture without building, so an agent can
|
||||
check it after each recording. The view summarizes the state as derived; it
|
||||
does not apply the serve-time admission checks, so for an environment whose
|
||||
@@ -209,9 +212,9 @@ warning does not establish that these bodies were checked. A browser scenario's
|
||||
saved values (`save` and `save_response`) named like credentials but not marked
|
||||
`secret: true` are recorded unredacted; `credential-named-saved-values` lists up
|
||||
to 20 of them by parameter pointer. Names do not establish sensitivity,
|
||||
so the build never refuses on them. Capture
|
||||
so the build never refuses on them.
|
||||
|
||||
Evidence also records the capture file's absolute path (`subject.local_path`,
|
||||
Capture Evidence also records the capture file's absolute path (`subject.local_path`,
|
||||
`parameters.capture_path`, `locations`, `artifact.path`), and the bundle
|
||||
carries it: build from a neutral folder when that path should not ship.
|
||||
|
||||
@@ -235,7 +238,7 @@ name `export_environment_bundle`; the others name their stage as
|
||||
## From a browser scenario
|
||||
|
||||
The capture can also come from REA itself: a
|
||||
[browser scenario](browser-scenario-contract.md) records a scripted session
|
||||
[browser scenario](guides/web/browser-scenario-contract.md) records a scripted session
|
||||
in a browser REA launches, and its Evidence is a manifest parent like a HAR
|
||||
inspection. Select the `network` event family and response bodies. A replayed
|
||||
reply carries up to 256 KiB of a recorded body, but the scenario keeps at most
|
||||
@@ -321,7 +324,7 @@ declared secret replaced by its redaction marker (a body that redaction had to
|
||||
empty replays no body).
|
||||
|
||||
To record writes, sign in through the UI and then send the application's API
|
||||
calls with [request steps](browser-scenario-contract.md#request-steps): each
|
||||
calls with [request steps](guides/web/browser-scenario-contract.md#request-steps): each
|
||||
sends one request to the start origin through the page's own `fetch`, with the
|
||||
session's cookies, and a header can carry the CSRF cookie or stored token the
|
||||
application's own requests send. A write is served only when the capture shows
|
||||
@@ -332,9 +335,12 @@ scenario is written and cannot use a value an earlier response returned, so an
|
||||
id must be known before the run: one the caller chooses, such as a `uid` in the
|
||||
create body, or one an earlier scenario created. When the server assigns ids,
|
||||
read the list after each create (the create's response or the list shows the
|
||||
new entity), and record updates and deletes in a second scenario that reads
|
||||
the first one's objects by id before editing them, or make them through the
|
||||
UI. The environment coverage view names each
|
||||
new entity). A second scenario can record updates and deletes using those
|
||||
known ids, but produces a separate environment: recordings are not combined.
|
||||
That scenario must include its own before/after reads and every action the
|
||||
environment should support; it does not inherit the first scenario's create
|
||||
evidence. To capture create and subsequent edits together, make them through
|
||||
the UI in one scenario. The environment coverage view names each
|
||||
write it withholds and what to record next. A scenario with write requests
|
||||
changes the application's data: record against an instance whose data may
|
||||
change.
|
||||
@@ -481,7 +487,7 @@ captured text without changing the tool names or schema fields clients use.
|
||||
|
||||
Replay answers each call from one recording, so the answers reflect whichever
|
||||
moment of the capture recorded them. The
|
||||
[environment state](web-network-captures.md#environment-state) view extracts
|
||||
[environment state](guides/web/web-network-captures.md#environment-state) view extracts
|
||||
instead the data the capture's reads observed before the first related write:
|
||||
entity types with an identity field, entities with their recorded values,
|
||||
documents for reads that are not entities, and a render plan per read tool,
|
||||
@@ -503,14 +509,14 @@ see [Serve from the state](#serve-from-the-state).
|
||||
|
||||
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the
|
||||
environment, then to `127.0.0.1` and an ephemeral port, exactly as
|
||||
[`rea mcp --http`](cli.md#serve-mcp-over-streamable-http) resolves them. The
|
||||
[`rea mcp --http`](getting-started/cli.md#serve-mcp-over-streamable-http) resolves them. The
|
||||
bound MCP URL is written to stderr as one JSON log line
|
||||
(`"msg":"MCP transport listening"` with `url`); `REA_LOG_LEVEL` overrides that
|
||||
log level. `ENVIRONMENT_NAME` renames the served environment; otherwise the
|
||||
manifest's `name` is used, and a set but blank value is refused as
|
||||
`configuration_invalid` before binding rather than silently falling back. The
|
||||
listener serves one MCP client at a time with the same session, `Host` header
|
||||
and shutdown contract as [transports](mcp-contracts.md#transports) describes.
|
||||
and shutdown contract as [transports](reference/mcp-contracts.md#transports) describes.
|
||||
It runs until SIGINT or SIGTERM, then closes the listener, prints one
|
||||
`{"status":"stopped", …}` document whose `state` member says whether it served
|
||||
from the state, and exits `130` or `143`.
|
||||
@@ -585,7 +591,7 @@ names the method and path template, then what the tool returns, one level deep:
|
||||
`returns a JSON object with fields …`, `returns a JSON array of objects with
|
||||
fields …` or `returns a JSON object whose <member> lists objects with fields …`,
|
||||
where a list is an array of objects found by the
|
||||
[environment state](web-network-captures.md#environment-state)'s list rule (the
|
||||
[environment state](guides/web/web-network-captures.md#environment-state)'s list rule (the
|
||||
root array, or the one array of objects an envelope carries). Field and member
|
||||
names are written as JSON strings, so a captured name cannot read as more of
|
||||
the description; a member name too long to show whole is a quoted prefix marked
|
||||
@@ -641,7 +647,7 @@ rea environment-serve manifest.json --evidence capture.json
|
||||
```
|
||||
|
||||
Before it binds, the server derives the
|
||||
[environment state](web-network-captures.md#environment-state) the capture
|
||||
[environment state](guides/web/web-network-captures.md#environment-state) the capture
|
||||
observed: it runs the `environment-state` view on the capture in-process with
|
||||
the manifest's `template_rules` in the same order, exactly as
|
||||
`inspect-analysis-view` projects it from the same capture, so the state
|
||||
@@ -733,7 +739,7 @@ query model may describe supported paging; an empty later page cannot start
|
||||
returning earlier entities merely because it was empty in the capture.
|
||||
|
||||
`POST`, `PUT`, `PATCH` and `DELETE` tools are served only when the state's
|
||||
[write plan](web-network-captures.md#environment-state) for them is served:
|
||||
[write plan](guides/web/web-network-captures.md#environment-state) for them is served:
|
||||
the capture's later reads confirmed the effect the plan models. Other write
|
||||
tools are not served, so no call claims a change it does not make; each one's
|
||||
plan states why in its `reason`. A served write changes the live state, which
|
||||
@@ -1187,7 +1193,7 @@ read, not that the write had no other effect.
|
||||
## Compatibility verification
|
||||
|
||||
`npm run verify:agentenv:environment` is the hosted `real-agentenv` lane
|
||||
described in [testing](testing.md#exported-environment-bundles). It projects
|
||||
described in [testing](development/testing.md#exported-environment-bundles). It projects
|
||||
the Grafana-shaped source-owned capture through the built CLI, packs the
|
||||
checkout, exports a bundle with `--package-tarball`, runs it with the pinned
|
||||
`agentenv-framework` release on the runner's Docker and requires the smoke task
|
||||
|
||||
+29
-4
@@ -85,10 +85,35 @@ names are retained. Original bytes remain in the caller's artifact; complete
|
||||
inventory Evidence does not embed the entire source JSON. Duplicate JSON members
|
||||
use the shared parser's last-value-wins behavior, recorded in source coverage.
|
||||
|
||||
If decoding or later validation fails after a verified read, the typed error
|
||||
retains a compact `file-snapshot-hash` observation with the exact path, digest
|
||||
and byte length. Syntax diagnostics do not quote source JSON. Cancellation
|
||||
does not replace a parser failure that has already completed.
|
||||
If decoding or validation fails after a verified read and before any operation,
|
||||
reference, or diagnostic row is retained, the typed error keeps a compact
|
||||
`file-snapshot-hash` observation with the exact path, digest and byte length.
|
||||
Syntax diagnostics do not quote source JSON. Cancellation does not replace a
|
||||
parser failure that has already completed.
|
||||
|
||||
When cancellation or the retained-facts limit stops OpenAPI traversal after at
|
||||
least one operation, reference, or diagnostic row, that observation is
|
||||
authenticated Evidence for the partial inventory rather than the hash. The
|
||||
normalized result uses the same inventory shape, `source_coverage.status` is
|
||||
`partial`, and only the examined rows are present. Reuse those rows with
|
||||
`inspect_analysis_view`. A session that already retained the Evidence uses:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": { "kind": "retained-evidence", "evidence_id": "<evidence_id>" },
|
||||
"view": {
|
||||
"kind": "api-contract",
|
||||
"collection": "operations",
|
||||
"offset": 0,
|
||||
"limit": 64
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
An inline record uses the same view with
|
||||
`source: { "kind": "inline", "evidence": <complete Evidence> }`. Do not reread
|
||||
the file to reconstruct the rows. A later hash is not substituted once this
|
||||
Evidence is attached.
|
||||
|
||||
## Resource bounds and verification
|
||||
|
||||
|
||||
+67
-4
@@ -85,9 +85,49 @@ canonical hexadecimal form), `EXC_GUARD` subtypes,
|
||||
pointer-authentication suffixes, and dyld missing-library strings. A report
|
||||
that matches none of those rules is `unclassified`.
|
||||
|
||||
Opened Mach-O and dSYM UUID joins remain unobserved. Symbol names printed
|
||||
inside the report are recorded claims and are not verified. Linux ELF cores
|
||||
use the separate [recorded crash](recorded-crashes.md) workflow.
|
||||
## Joining frames to the crashed Mach-O or dSYM
|
||||
|
||||
Pass `join_path` (`--join-path` in the CLI) to select one local Mach-O
|
||||
executable or dSYM. UUID bytes in the report and the artifact's `LC_UUID` load
|
||||
commands are compared in hexadecimal, so an uppercase `dwarfdump --uuid` value
|
||||
and a lowercase report `uuid` match.
|
||||
|
||||
```bash
|
||||
rea inspect-apple-ips /absolute/path/to/report.ips \
|
||||
--join-path /absolute/path/to/App.app/Contents/MacOS/App --json
|
||||
```
|
||||
|
||||
Shared-cache image sources `S` and `C` remain unjoined: UUID agreement does not
|
||||
establish that a standalone image has the cached layout.
|
||||
|
||||
For a FAT container every slice is examined. An image joins only when exactly
|
||||
one slice has that UUID **and** an architecture that agrees with the report's
|
||||
`arch`; `arm64` and `arm64e` stay distinct, and an all-zero UUID never joins. A
|
||||
UUID match with a different architecture, more than one compatible slice, or a
|
||||
slice without a readable `__TEXT` segment is reported as a rejected image with
|
||||
its reason and stays unjoined rather than being guessed.
|
||||
An unknown UUID in another slice prevents a unique join unless its architecture
|
||||
rules out a match. Missing or unrecognized architecture and unsupported CPU
|
||||
subtypes remain unknown; display-name fallbacks do not establish agreement.
|
||||
The parsed slice facts remain available even when no match can be derived.
|
||||
|
||||
`join.status` is `selected` with the artifact path, SHA-256, byte length, every
|
||||
slice's offset, arch, CPU type and subtype, UUID, and `__TEXT` vmaddr, plus
|
||||
`image_matches`, per-frame results, and join-specific limitations. A joined
|
||||
frame reports `static_address`, the slice `__TEXT` vmaddr plus the frame's
|
||||
recorded `imageOffset`, in decimal and hexadecimal. An unjoined frame reports
|
||||
a reason instead; an unknown image UUID, an absent `imageIndex` row, and a
|
||||
frame without `imageOffset` all stay unjoined rather than treated as aligned.
|
||||
|
||||
An artifact that is not a Mach-O image or FAT container reports
|
||||
`join.status: "unreadable"` with a typed reason and leaves every report fact
|
||||
intact. Symbol names printed inside the report stay recorded claims. Function
|
||||
or symbol lookup at a joined static address, and matching dSYM symbol tables,
|
||||
are not performed by this step.
|
||||
|
||||
Symbol names printed inside the report are recorded claims and are not
|
||||
verified. Linux ELF cores use the separate
|
||||
[recorded crash](guides/native/recorded-crashes.md) workflow.
|
||||
|
||||
## Limits and failures
|
||||
|
||||
@@ -120,4 +160,27 @@ success or truncated integer facts.
|
||||
rea inspect-apple-ips /absolute/path/to/report.ips --max-integer-text-bytes 16777216 --json
|
||||
```
|
||||
|
||||
This workflow does not execute a target, launch atos, or search for dSYMs.
|
||||
`max_join_bytes` (`--max-join-bytes`) independently bounds the join artifact
|
||||
read, defaulting to 64 MiB. This workflow does not execute a target, launch
|
||||
atos, or search for dSYMs.
|
||||
|
||||
## Join admission and failures
|
||||
|
||||
The join uses the shared Mach-O container, slice, command-table and segment
|
||||
readers. Slice extents and `sizeofcmds` bound every identity observation; duplicate
|
||||
UUIDs or malformed commands cannot select the first plausible value. Unreadable
|
||||
join results retain the successfully read artifact's path, SHA-256 and byte count.
|
||||
|
||||
The integer-text allowance spans the report and join, including repeated integer
|
||||
facts and derived static addresses. Reservation precedes derived address
|
||||
expansion. If a selected join read, interpretation or allowance fails, the error
|
||||
retains the completed IPS report. Join read limits and recovery use
|
||||
`max_join_bytes`, even when the report and join paths are identical. Cancellation
|
||||
retains the caller's reason independently of those completed facts.
|
||||
|
||||
When a failure follows a completed join-file read, `partial_observation` retains
|
||||
both the decoded report and a `join_artifact` receipt with the independently
|
||||
observed path, SHA-256 and byte count. Frame rejection reasons reference
|
||||
`join.limitations`; source-sized architecture diagnostics are retained once per
|
||||
image rather than copied into every frame. Universal-image admission includes
|
||||
the shared reader's supported hidden ARM64 row and rejects overlapping slices.
|
||||
File renamed without changes.
+18
-3
@@ -24,7 +24,7 @@ entry-point status, sizes and SHA-256. Sourcemaps, bytecode and module-info regi
|
||||
are separate children. Source names such as `/$bunfs/root/main.js` remain in
|
||||
metadata; extraction uses a relative path. Unsafe or non-UTF-8 names receive an
|
||||
indexed path and retain their source spelling or bytes. Virtual names deeper
|
||||
than 256 logical path parts also use an indexed path with an explicit limitation,
|
||||
than the selected `path_parts` allowance (default 256) also use an indexed path with an explicit limitation,
|
||||
preventing unbounded path-tree work while retaining the source fact. Duplicate names remain
|
||||
separate occurrences; ordinary extraction collision rules apply.
|
||||
|
||||
@@ -43,12 +43,27 @@ compatible source files; bytecode is retained as evidence, not decoded.
|
||||
Module parsing reads exact metadata ranges rather than loading stored asset
|
||||
bodies or whole pages around small pointers. A cache retains at most 256 KiB
|
||||
and 1,024 ranges for one reader preparation. Cumulative uncached metadata reads
|
||||
are limited to 256 MiB, checked before allocation or I/O. The module table,
|
||||
default to 256 MiB, checked before allocation or I/O. The module table,
|
||||
decoded strings and combined logical path tree have separate admission checks;
|
||||
at most 250,000 distinct path prefixes are admitted before inventory construction.
|
||||
by default at most 250,000 distinct path prefixes are admitted before inventory construction.
|
||||
Stored bodies are range-streamed for hashing and extraction. Header/suffix observations are reused
|
||||
only for the same admitted file state, with selected-path checks before publication.
|
||||
|
||||
Both artifact operations accept `artifact_limits.bun`. Its independent controls
|
||||
are `payload_bytes`, `module_count`, `representation_bytes`, `metadata_read_bytes`,
|
||||
`recognition_table_bytes`, `recognition_load_command_bytes`, `path_parts` and
|
||||
`path_nodes`. Failed recognition remains explicitly limited instead of proving
|
||||
absence. ELF section names are examined at their actual offsets, including names
|
||||
beyond a large section-name prefix. Select `artifact_limits.expanded_bytes` to
|
||||
change the aggregate referenced-plaintext work allowance.
|
||||
|
||||
For a reported metadata-read refusal, keep the same carrier and retry with
|
||||
`--artifact-limits '{"bun":{"metadata_read_bytes":536870912}}'` or the equivalent
|
||||
MCP `artifact_limits` object. Omitted fields keep defaults. The response names
|
||||
the effective limit and exact repair; accepted overrides are reported separately
|
||||
from caller selections. Changes apply on the next call without restarting REA.
|
||||
Summary and selected extraction do not change the producer's admission.
|
||||
|
||||
See [limits and recovery](design/limits-and-recovery.md) for admission ceilings.
|
||||
Synthetic tests cover the carriers and malformed declarations. Run
|
||||
`node scripts/verify-bun-standalone.mjs` after `npm run build:cached` for local Bun
|
||||
|
||||
+2
-2
@@ -65,14 +65,14 @@ cancellation can still prevent upload.
|
||||
that analysis was complete. A successful operation can retain partial analysis,
|
||||
unknowns, and limitations. Exit zero is not a security verdict, proof of complete
|
||||
coverage, or a policy gate for those findings. Inspect the saved Evidence and
|
||||
coverage before making such a decision. See [output and exit status](cli.md#output-and-exit-status).
|
||||
coverage before making such a decision. See [output and exit status](getting-started/cli.md#output-and-exit-status).
|
||||
|
||||
A nonzero exit can represent invalid input, unavailable tooling, an unsupported
|
||||
host/capability, or another operation failure. Keep the original JSON error (when
|
||||
emitted), stderr, and exit code, and use their category/message to diagnose it.
|
||||
An early startup failure may leave `analysis.json` empty; do not parse that as
|
||||
successful analysis. Select a supported operation or explicitly provision its
|
||||
[documented prerequisites](installation.md); do not silently switch providers,
|
||||
[documented prerequisites](getting-started/installation.md); do not silently switch providers,
|
||||
execute the artifact, or ignore the failure to obtain a green job.
|
||||
|
||||
To exercise the failure path, temporarily set `ARTIFACT_PATH` to
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
# Deno compile executables
|
||||
|
||||
`inspect-artifact` inventories a `deno compile` executable. `extract-artifact` writes the embedded virtual files through the existing safe output tree. Neither operation runs the executable or the Deno runtime.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./app --json
|
||||
rea extract-artifact ./app --json
|
||||
```
|
||||
|
||||
In MCP, open the executable with `open_binary`, then call `inspect_artifact`. Use `detail: "summary"` for a large virtual filesystem and `inspect_analysis_view` to page retained Evidence.
|
||||
|
||||
The recognized carriers are a Mach-O `__SUI` section named `d3n0l4nd`, an Intel Mach-O `<~sui-data~>` trailer, an ELF `PT_NOTE` or `SHT_NOTE` owned by `SUI` with type `0x53554901` and descriptor name `d3n0l4nd`, and a PE `RT_RCDATA` resource named `D3N0L4ND`. These are structural layouts. The metadata JSON has no Deno version field, so the compiler version stays unknown unless another record establishes it. Other layouts, including older eszip trailers, are unsupported.
|
||||
|
||||
Root metadata reports the carrier, entry point, argv, V8 flags, permission options, unstable configuration, application name, self-extracting marker, virtual-filesystem case sensitivity, and the npm snapshot byte length. The npm snapshot is not decoded. Each virtual file is one occurrence with its logical path, size, and SHA-256. Transpiled output, CJS export analysis, and source maps are separate occurrences beside that file. Each has the same `index` as the file and its own `role`. A region normally has the logical path of the file plus a `.deno-*` suffix. When that path belongs to another stored file, the region moves to an indexed path such as `deno-vfs/0.vfs-transpiled`, so read the reported `logical_path` instead of building it from the file name. They are not linked by `parent_occurrence_id`, which follows directories and expanded containers. Symlink targets stay recorded path parts and are never followed.
|
||||
|
||||
`Metadata.env_vars_from_env_file` holds values from a compile-time `--env-file`, and `ca_data` holds certificate bytes. Inventory retains the variable names and values, certificate bytes, counts, and byte lengths in local Evidence, so sharing that Evidence shares them. Extraction does not write them as files.
|
||||
|
||||
A virtual filesystem that cannot be listed reports `vfs_listing` as `unsupported` and leaves inventory and extraction coverage unknown, because its files are not examined.
|
||||
|
||||
Remote modules, redirects, and specifiers contribute counts and logical paths when their stored form is unambiguous. When it is not, they stay present with a byte length and an `unsupported` member listing. A module whose specifier is missing uses an indexed `deno-remote/<index>` path, which still exposes its stored bytes, and modules with resolved specifiers keep their paths. Colons, backslashes, and control characters in a specifier are percent-encoded in the logical path, so `http://localhost:8000/mod.ts` becomes `deno-remote/localhost%3A8000/mod.ts`. Percent escapes in a logical path use uppercase hex, and a specifier that reaches a path already taken uses an indexed path. `source_path` keeps the original specifier. A remote specifier is indexed the same way when a path component exceeds 255 UTF-8 bytes or the logical path exceeds 512 UTF-8 bytes, counting the `.deno-*` suffix of its stored regions. Unsafe, deep, or over-long virtual names use `deno-vfs/<index>` under the same rule and retain the original spelling when it fits the name allowance. A descendant does not present its own name as a complete source path when an ancestor spelling was dropped.
|
||||
|
||||
A length that overruns the section, or a closing marker that does not match, is malformed. An opening marker that is absent is unsupported. A confirmed section that cannot be parsed fails closed and does not fall through to a plain native inventory. An Intel Mach-O sentinel inside a confirmed trailer payload is data of that payload, so an Intel Deno executable stored as a virtual file does not replace the app that stores it. Trailers side by side with different bytes fail closed. A file is confirmed only by its carrier name, so an unrelated native file with malformed tables keeps its native inventory. That inventory does not establish that the file holds no Deno section.
|
||||
|
||||
See [limits and recovery](design/limits-and-recovery.md) for the named allowances. Synthetic tests cover the carriers and malformed declarations. Run `npm run verify:deno:standalone` for a local Deno compile of the host and, when the compiler can produce them, Linux, Windows, and Intel macOS targets. Deno is a fixture producer only and is not installed by REA. The produced executable is not run.
|
||||
|
||||
Layout reference: Deno `cli/lib/standalone/binary.rs` and `cli/lib/standalone/virtual_fs.rs` at 2.9.8, with libsui 0.16.4 section placement.
|
||||
@@ -3,7 +3,7 @@
|
||||
This guide explains the runtime settings design for maintainers and developers:
|
||||
who owns each value, when an update takes effect, how providers enforce it, and
|
||||
how to extend the system. The public tool contract is in
|
||||
[MCP contracts](../mcp-contracts.md#editable-analysis-settings); recorded runtime
|
||||
[MCP contracts](../reference/mcp-contracts.md#editable-analysis-settings); recorded runtime
|
||||
checks and their limitations are in the
|
||||
[verification report](../verification/runtime-analysis-settings.md).
|
||||
|
||||
@@ -172,17 +172,20 @@ checks all happen before publishing the replacement snapshot; a rejected batch
|
||||
cannot partially apply. Clients coordinating writes can supply
|
||||
`expected_revision`; without it, valid updates apply to the current state.
|
||||
|
||||
| Runtime setting | Application point |
|
||||
| ------------------------------------------------------------- | ---------------------------------------------------------------------- |
|
||||
| `asar.max_header_bytes` | Next archive operation, including full JavaScript application analysis |
|
||||
| `ghidra.startup_timeout_ms`, `hopper.startup_timeout_ms` | Next provider startup attempt |
|
||||
| `ghidra.decompile_timeout_seconds` | Next operation on the existing Ghidra Program; `null` means unbounded |
|
||||
| `jadx.operation_timeout_ms` | Next JADX operation |
|
||||
| `regex.matching_deadline_ms`, `regex.worker_heap_request_mib` | Next search / new regex worker |
|
||||
| `capture.decoder_old_generation_mib` | Next owned capture decoder |
|
||||
| `javascript.analysis_timeout_ms`, `javascript.max_heap_mb` | Next source analysis / new JavaScript worker |
|
||||
| `wasm.command_timeout_ms` | Next WABT operation |
|
||||
| `native.matching_representation_bytes` | Next `match_native_functions` request |
|
||||
| Runtime setting | Application point |
|
||||
| --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
|
||||
| `asar.max_header_bytes` | Next archive operation, including full JavaScript application analysis |
|
||||
| `asar.max_listing_bytes` | Next expanded ASAR listing, independently of packed header bytes |
|
||||
| `ghidra.startup_timeout_ms`, `hopper.startup_timeout_ms` | Next provider startup attempt |
|
||||
| `ghidra.decompile_timeout_seconds` | Next operation on the existing Ghidra Program; `null` means unbounded |
|
||||
| `ghidra.decompile_max_payload_mib` | Next native decompile; `null` preserves the selected provider's payload default |
|
||||
| `ghidra.value_flow_max_operations`, `ghidra.value_flow_max_inputs_per_operation`, `ghidra.value_flow_max_edges` | Next function value-flow projection on the existing Program |
|
||||
| `jadx.operation_timeout_ms` | Next JADX operation |
|
||||
| `regex.matching_deadline_ms`, `regex.worker_heap_request_mib` | Next search / new regex worker |
|
||||
| `capture.decoder_old_generation_mib` | Next owned capture decoder |
|
||||
| `javascript.analysis_timeout_ms`, `javascript.max_heap_mb` | Next source analysis / new JavaScript worker |
|
||||
| `wasm.command_timeout_ms` | Next WABT operation |
|
||||
| `native.matching_representation_bytes` | Next `match_native_functions` request |
|
||||
|
||||
### Precedence and lifetime
|
||||
|
||||
@@ -202,6 +205,48 @@ for an unknown or omitted value elsewhere.
|
||||
A write does not extend active work, relaunch a provider, clear retained
|
||||
cleanup, or invalidate successful Evidence.
|
||||
|
||||
Expanded ASAR listing admission defaults to 64 MiB and has its own launch seed,
|
||||
`REA_ASAR_MAX_LISTING_BYTES`. Raising `asar.max_header_bytes` alone cannot admit
|
||||
expanded paths and entries that exceed this separate budget. The listing bound
|
||||
uses exact safe-integer byte accounting; the host heap and transport constraints
|
||||
remain independent. A lower runtime value is enforced on the next listing even
|
||||
when the archive header was already examined.
|
||||
|
||||
Ghidra value-flow controls default to 3,000 operations, 64 inputs per operation,
|
||||
and 12,000 edges. They accept positive Java integer counts and use the respective
|
||||
`REA_GHIDRA_VALUE_FLOW_MAX_OPERATIONS`,
|
||||
`REA_GHIDRA_VALUE_FLOW_MAX_INPUTS_PER_OPERATION`, and
|
||||
`REA_GHIDRA_VALUE_FLOW_MAX_EDGES` launch seeds. Truncation names the exhausted
|
||||
control. Completed Evidence remains accessible, but a new function dossier is
|
||||
observed with the accepted controls instead of replaying an earlier projection.
|
||||
|
||||
The native decompiler payload control is separate from those projections and
|
||||
from decompile time. Its launch seed is
|
||||
`REA_GHIDRA_DECOMPILE_MAX_PAYLOAD_MIB`. Accepted explicit values are 1..2047 MiB:
|
||||
the supported Ghidra implementation shifts that value into a signed integer byte
|
||||
capacity. Omission or runtime `null` keeps the provider's own default; REA does
|
||||
not guess that default from a different provider version. Changing payload
|
||||
options takes effect before the next native decompile.
|
||||
|
||||
### Per-call resource policies and selection
|
||||
|
||||
An agent can also use controls declared directly by an operation. These do not
|
||||
require a runtime setter or restart. Keep omitted fields omitted in Evidence;
|
||||
report the resolved policy separately where needed. See the owning guides for
|
||||
[Apktool string pages](../guides/android/apktool-resource-analysis.md),
|
||||
[source maps](../guides/web/web-source-location.md),
|
||||
[historical captures](../guides/web/web-network-captures.md),
|
||||
[JavaScript precision](../guides/javascript/javascript-application-graph.md),
|
||||
[managed inspection](../guides/managed/managed-code-analysis.md), and
|
||||
[Frida capture](../guides/runtime/frida-instrumentation.md).
|
||||
|
||||
The same accepted policy must reach acquisition, parsing, worker requests,
|
||||
serialization, cached-result admission and generated consumers where those
|
||||
stages exist. Desktop state views and emitted AgentEnv Python share one policy;
|
||||
delivery pagination never substitutes for producer selection. Capture limits
|
||||
must be selected before an experiment: another experiment cannot recover bytes
|
||||
that were dropped from a previous live observation.
|
||||
|
||||
Overrides belong to one REA runtime, survive target close/open, and disappear
|
||||
when REA exits. Connections sharing that runtime share its overrides. No files,
|
||||
ambient environment, registrations or upstream software are edited. Overrides
|
||||
@@ -399,7 +444,7 @@ application points, and resulting behavior. Check that adapters deliver the
|
||||
canonical recovery guidance, with independent assertions for its underlying
|
||||
facts. Avoid freezing prose or deriving all expected values from the same
|
||||
registry used by the implementation; either can hide the wrong failure.
|
||||
See [test design](../testing.md#test-design-and-pruning) for the assertion policy.
|
||||
See [test design](../development/testing.md#test-design-and-pruning) for the assertion policy.
|
||||
|
||||
Runtime checks cover atomic validation, stale writes, reset, runtime isolation,
|
||||
accepted-operation snapshots, and explicit argument precedence. Public MCP
|
||||
@@ -419,14 +464,107 @@ editable setting or active worker telemetry.
|
||||
|
||||
## Bun standalone module graphs
|
||||
|
||||
`MAX_BUN_MODULE_COUNT` admits at most 100,000 module records.
|
||||
`MAX_BUN_RETAINED_BYTES` admits at most 256 MiB of serialized graph payload and
|
||||
separately bounds retained entry metadata. Physical metadata read work is also
|
||||
limited to 256 MiB per preparation, including repeated uncached ranges. A
|
||||
256 KiB / 1,024-entry exact-range cache avoids full-page amplification. The
|
||||
combined logical path tree admits at most 250,000 distinct prefixes; this is
|
||||
checked before adding nodes, with shared prefixes counted once. The shared
|
||||
artifact expansion budget also bounds the sum of referenced member lengths. Refusals name the boundary.
|
||||
There is no runtime override; additional host memory or `NODE_OPTIONS` does not
|
||||
raise these ceilings. Use a smaller producer fixture or a separately reviewed
|
||||
reader for larger graphs. No partial extraction is claimed on admission failure.
|
||||
`inspect_artifact` and `extract_artifact` accept per-call `artifact_limits`.
|
||||
The `bun` group defaults to 100,000 modules, independent 256 MiB payload,
|
||||
retained-representation and physical metadata-read allowances, 256 path parts
|
||||
and 250,000 logical path nodes. Recognition separately admits a 1 MiB ELF
|
||||
section table and 16 MiB Mach-O load commands. Exhausted recognition remains
|
||||
unknown rather than proving the carrier has no Bun graph. Inventory preserves
|
||||
native-root facts or another confirmed pack and reports the exact unexamined
|
||||
alternative inline; it does not assert Bun presence. ELF section names
|
||||
are examined with exact bounded reads, including names beyond the former
|
||||
prefix window.
|
||||
|
||||
For example, keep the same complete carrier and retry a reported recognition
|
||||
limitation with `{"artifact_limits":{"bun":{"recognition_table_bytes":2097152}}}`.
|
||||
The CLI spelling is `--artifact-limits '{"bun":{"recognition_table_bytes":2097152}}'`.
|
||||
No REA restart is needed. A smaller later selection is re-admitted; previously
|
||||
parsed facts cannot bypass it. The 256 KiB / 1,024-entry exact-range cache
|
||||
avoids full-page amplification and remains an internal optimization, not an agent
|
||||
resource setting. Shared logical path prefixes count once before nodes are added.
|
||||
|
||||
## Artifact reader admission
|
||||
|
||||
The same request provides `dotnet_bundle`, `node_sea` and `godot` groups. Their
|
||||
controls apply when recognition or inventory encounters that format; supplying
|
||||
a policy does not assert the format is present. Unsupported keys are rejected.
|
||||
Each group follows one domain-owned default policy. See [.NET bundles](../dotnet-bundles.md),
|
||||
[Node SEA](../node-sea.md), [Godot PCK](../godot-pck.md) and
|
||||
[Bun standalone](../bun-standalone.md) for format semantics.
|
||||
|
||||
`artifact_limits.expanded_bytes` selects aggregate plaintext work for ZIP, Bun,
|
||||
SEA and PCK readers, including repeated readable logical occurrences. Omission
|
||||
retains each archive's encoded-size-derived default. The .NET bundle group's
|
||||
own `expanded_bytes` controls its independently enforced bundle expansion.
|
||||
|
||||
Failures preserve `details.reader_limit` with the exact parameter, effective
|
||||
value, units, supported range, required value when known, next-call scope and
|
||||
`can_increase`. When `can_increase` is true, raise that field and retry the same
|
||||
complete artifact. At the supported maximum, or when the required representation
|
||||
exceeds it, the failure is not retryable by increasing that budget; inspect the
|
||||
declared dimensions and input validity, or explicitly choose a suitable reader.
|
||||
Summary, pagination and
|
||||
selected extraction do not bypass producing admission. Intrinsic format,
|
||||
safe-integer, path validity and runtime representation constraints remain fixed.
|
||||
|
||||
Evidence parameters retain the exact selected object, including omission or
|
||||
`{}`. When an object was selected, inventory and extraction report the resolved
|
||||
`effective_limits`; summaries preserve those facts. These are accepted policies,
|
||||
not process-memory telemetry or a claim that every format reader ran. Complete
|
||||
source identities and graph manifests remain content-addressed independently
|
||||
of delivery selection. No partial extraction is claimed on admission failure.
|
||||
|
||||
## Deno compile sections
|
||||
|
||||
`MAX_DENO_METADATA_JSON_BYTES` admits at most 8 MiB of metadata JSON.
|
||||
`MAX_DENO_VFS_HEADER_JSON_BYTES` admits at most 32 MiB of virtual-filesystem
|
||||
header JSON. Specifier, redirect, and remote tables each admit at most 100,000
|
||||
records. Specifier text admits at most 8 MiB, and remote module bodies admit at
|
||||
most 256 MiB. The virtual filesystem admits at most 100,000 directory, file,
|
||||
and symlink records and at most 512 nested levels (`MAX_DENO_VFS_DEPTH`). A
|
||||
name longer than 4,096 UTF-16 code units is indexed and is not retained.
|
||||
`MAX_DENO_VFS_RETAINED_UNITS` admits at most 32 million UTF-16 code units across
|
||||
the source path built for every entry, directories included, and retained
|
||||
symlink parts. A logical path deeper than 256 parts, a name that is not a safe
|
||||
relative path, a path component longer than 255 UTF-8 bytes
|
||||
(`MAX_DENO_PATH_COMPONENT_BYTES`, POSIX `NAME_MAX`), or a logical path longer
|
||||
than 512 UTF-8 bytes (`MAX_DENO_LOGICAL_PATH_BYTES`, half of the Darwin
|
||||
`PATH_MAX`) uses an indexed path and keeps the original spelling when that
|
||||
spelling is within the name allowance. Component and path lengths include the
|
||||
`.deno-*` suffix of the stored regions a member has. A descendant has no source
|
||||
path when an ancestor spelling was not retained. Cancellation is checked before
|
||||
the virtual-filesystem walk. The walk does not yield, so it does not observe a
|
||||
cancellation that arrives while it is running; the retained allowance bounds its
|
||||
work. The combined logical path tree admits at most 250,000 distinct prefixes.
|
||||
`MAX_DENO_RETAINED_BYTES` admits at most 256 MiB of retained header and entry
|
||||
metadata. The shared artifact expansion budget also bounds the sum of referenced
|
||||
member lengths, with a 256 MiB floor.
|
||||
|
||||
Carrier location claims a file only after it confirms the carrier name: a
|
||||
Mach-O `__SUI` segment, a `SUI` ELF note descriptor named `d3n0l4nd`, a PE
|
||||
`D3N0L4ND` resource, or an Intel trailer opening with `d3n0l4nd`. Before that,
|
||||
malformed tables and exhausted search budgets leave the file with native
|
||||
inventory, and that inventory makes no claim that the file holds no Deno
|
||||
section. ELF location reads ELF and note headers, not the whole image, and does
|
||||
not hash note descriptors. An ELF larger than 512 MiB is still located when
|
||||
those headers fit. Program and section tables are read at their canonical record
|
||||
size, so the 16-bit header count bounds each read at 65,535 records. A table
|
||||
that cannot be read, or that uses extended program numbering, is skipped and
|
||||
another table can still hold the note. At most 256 notes are examined
|
||||
(`MAX_INSPECT_ELF_NOTE_ENTRIES`). A damaged table or the note ceiling is a
|
||||
refusal only when a `SUI` note was also confirmed. Intel Mach-O trailer search
|
||||
reads each scanned byte at most once and examines at most 64 `<~sui-data~>`
|
||||
sentinels (`MAX_DENO_INTEL_TRAILER_SENTINELS`). The search continues past a
|
||||
confirmed trailer, because an enclosing trailer can hold it as a stored file, so
|
||||
reaching that allowance after a confirmation is a refusal. PE location walks the resource directories to the named payload and
|
||||
does not read the image whole, so a PE of any size is located when the
|
||||
`D3N0L4ND` payload is within the 512 MiB resource reader ceiling. A larger
|
||||
payload, or a confirmed resource that cannot be resolved, is a named refusal.
|
||||
Refusals name the boundary. There is no runtime override; additional host memory
|
||||
or `NODE_OPTIONS` does not raise these ceilings. Duplicate-section verification
|
||||
rejects unequal lengths before hashing, stops at the first digest mismatch,
|
||||
and reads at most the carrier file size in aggregate
|
||||
payload bytes. Overlapping copies that exceed that allowance are refused.
|
||||
Environment-file values and CA certificate bytes are retained in local Evidence
|
||||
with their counts and byte lengths. The npm snapshot length is retained and its
|
||||
bytes are not decoded.
|
||||
@@ -47,6 +47,17 @@ computer would not have that entry. An `--executable` runs from
|
||||
the current directory, as it would from a shell, and a relative path to it is
|
||||
resolved there.
|
||||
|
||||
A nested desktop file uses a dashed ID: `vendor/app.desktop` has ID
|
||||
`vendor-app`. REA checks path spellings that can encode that ID within each
|
||||
XDG root before trying the next root. Unrelated directories do not affect
|
||||
selection. If an applicable path cannot be examined or the bounded search
|
||||
cannot establish absence, capture reports an incomplete desktop-entry search
|
||||
rather than selecting a different application from a later root.
|
||||
|
||||
Desktop-entry string values retain literal trailing spaces. For example,
|
||||
`Path=/tmp/workspace ` and `Path=/tmp/workspace\s` name the same directory
|
||||
ending in a space. Use the escaped spelling to make such a suffix visible.
|
||||
|
||||
Before writing anything the command refuses to start if a window of the
|
||||
application is already open, or if the window list cannot be read. A launch
|
||||
beside a running instance can hand off to it, and that instance's unsaved
|
||||
@@ -56,6 +67,9 @@ environment, and reads commands from stdin, one per line. The application gets
|
||||
REA's environment without the `AGENTENV_*` variables an agent environment sets,
|
||||
such as its control token: an app the exported computer launches never has
|
||||
them, and an app captured inside a computer must not hold a credential to it.
|
||||
It also gets `LIBC_FATAL_STDERR_=1`, so glibc writes a fatal error, such as
|
||||
an abort's reason, to the app's stderr rather than its terminal, and an app
|
||||
that exits at launch reports it.
|
||||
|
||||
| Line | Effect |
|
||||
| ------------------------------------------- | ----------------------------------------------------------- |
|
||||
@@ -90,12 +104,20 @@ Progress lines on stderr are JSON, `{"rea_progress": {"phase":
|
||||
"desktop_capture", "message": ...}}`. Those a driver waits on also carry an
|
||||
`event`: `{"kind": "window_open"}` once the window opens; `{"kind":
|
||||
"checkpoint", "name", "trigger", "at_ms", "entries", "changes",
|
||||
"retained_files", "retained_bytes"}` once a checkpoint has been taken, the
|
||||
baseline, `stop` and `exit` included, with the paths walked, the paths
|
||||
changed since the checkpoint before and the files retained; and `{"kind":
|
||||
"checkpoint_refused", "name", "reason"}` for a requested name it refused. A
|
||||
driver that sends `checkpoint <name>` waits for the event naming it rather
|
||||
than for a fixed time.
|
||||
"retained_files", "retained_bytes", "changed_files", "more_changed_files"}`
|
||||
once a checkpoint has been taken, the baseline, `stop` and `exit` included,
|
||||
with the paths walked, the paths changed since the checkpoint before, the
|
||||
files retained, and up to ten of the files (not folders) that changed,
|
||||
home-relative: those under a `--state-path` or absolute `--argument` first,
|
||||
then those the application held open, then those in its state directories,
|
||||
then the rest, and last caches (a folder whose name ends in `cache`, such as
|
||||
`~/.cache`) and WebKit or Chromium storage, each by path, with a count of the
|
||||
rest. An application can rewrite dozens of cache files on a save, and KDE's
|
||||
holds its service cache open, so the saved document would otherwise fall past
|
||||
the ten;
|
||||
and `{"kind": "checkpoint_refused", "name", "reason"}` for a requested name it
|
||||
refused. A driver that sends `checkpoint <name>` waits for the event naming it
|
||||
rather than for a fixed time.
|
||||
|
||||
When the session ends the capture sends SIGTERM to the launched process, its
|
||||
process group and every process that carries the run token, waits up to five
|
||||
@@ -107,7 +129,11 @@ remain. Interrupting the command (Ctrl-C) ends the session the same way
|
||||
without a further checkpoint; the capture is still written with end reason
|
||||
`cancelled`, and the command exits with status 130. Interrupted before the
|
||||
application is launched, the command writes no capture and removes the
|
||||
content it had retained.
|
||||
content it had retained. If cancellation arrives during publication or final
|
||||
cleanup, the already-written capture can instead retain its earlier end
|
||||
reason. Inspect `capture.json` and the error's `details.partial_observation`
|
||||
for the retained capture Evidence. Cleanup failures also report resources
|
||||
that could not be released.
|
||||
|
||||
If the capture fails for any other reason before `capture.json` is written,
|
||||
the application is closed the same way and the content already retained is
|
||||
@@ -159,8 +185,9 @@ package that installed it. A Debian package records its name, version,
|
||||
architecture and the apt sources that list the installed version, with the
|
||||
origin (`o=`) apt reports for each source in `source_origins`, such as `Ubuntu`
|
||||
for the Ubuntu archive and its mirrors or `LP-PPA-<name>` for a PPA; an empty
|
||||
source list means the version came from a local `.deb` or the package lists
|
||||
are missing, which `package_lists_present` tells apart. apt's lists folder,
|
||||
source list means apt reports no source for the installed version.
|
||||
`package_lists_present` reports whether the lists are present. It is `null`
|
||||
when REA cannot read the lists directory. apt's lists folder,
|
||||
and the trusted keys below, are where `apt-config` resolves
|
||||
`Dir::State::Lists`, `Dir::Etc::TrustedParts` and `Dir::Etc::Trusted` under
|
||||
`Dir`, so an `APT_CONFIG` file that moves them is followed: on a computer
|
||||
@@ -188,9 +215,9 @@ does not bring it. A program link that dpkg does not own counts as owned when
|
||||
dpkg owns the file it resolves to, as an alternatives link does. An
|
||||
`--executable` launch, or an entry no package owns, such as
|
||||
one in `~/.local/share/applications`, records the executable's package. When
|
||||
`dpkg-query` does not finish or exits with an error for the executable or the
|
||||
entry, the package is `unknown` and the limitations say which path, since a
|
||||
failed lookup is not proof that no package owns it. The sources, vendor
|
||||
`dpkg-query` fails for the executable or entry, or the selected path contains
|
||||
characters `dpkg-query -S` would interpret as a pattern, package ownership is
|
||||
`unknown` and the limitations name the path. The sources, vendor
|
||||
repositories and `--package-file` match below are the recorded package's; the
|
||||
library walk still reads the executable.
|
||||
|
||||
@@ -217,7 +244,9 @@ writes it on Ubuntu 24.04, has that key as its only candidate, as apt trusts
|
||||
no other for it: REA reads the field's continuation lines as deb822 writes
|
||||
them, a `.` line standing for an empty one, from the key block's `BEGIN` line
|
||||
to its last `END` line, and records the key's path as the entry's sources
|
||||
file. A key counts when gpgv reports a valid signature by it and no bad,
|
||||
file. A fingerprint restriction in `Signed-By` leaves `signing_key` null.
|
||||
The reason names the restriction that REA could not verify. A key counts
|
||||
when gpgv reports a valid signature by it and no bad,
|
||||
expired, revoked or unverifiable signature by it, whatever gpgv's exit status:
|
||||
a release also signed by a key the host lacks makes gpgv fail. Each candidate
|
||||
is copied, an armored key decoded to a binary keyring, into a private
|
||||
@@ -246,8 +275,16 @@ looks each soname up as ld.so does: in the RPATH of the needing image and of
|
||||
each image up its chain of loaders to the executable when the needing image
|
||||
has no RUNPATH, then in its RUNPATH (with `$ORIGIN` expanded and paths
|
||||
normalized), and then in the `ldconfig -p` cache entries for the executable's
|
||||
machine. It asks `dpkg-query -S` for each library's package. Libraries in the
|
||||
executable's own folder belong to the application: they are walked but not
|
||||
machine. Each candidate's ELF machine, class and byte order must match the
|
||||
needing image before it is selected or its dependencies are walked. A foreign
|
||||
search-path candidate is skipped in favor of later directories or the cache; a
|
||||
cache architecture flag alone does not establish compatibility. If a candidate's
|
||||
identity cannot be read, that lookup stays unresolved, without selecting its
|
||||
package or guessing that a later candidate would load. A compatible identity
|
||||
remains available if a later header or dependency-table check fails; its package
|
||||
is retained with an incompleteness reason. REA asks `dpkg-query -S` for each
|
||||
selected library's package. Libraries in the executable's own folder belong to
|
||||
the application: they are walked but not
|
||||
listed, and neither is the application's own package. One batched
|
||||
`apt-cache policy` then keeps only the packages the Ubuntu 24.04 archive or a
|
||||
recorded vendor repository with a verified key offers, in any component its
|
||||
@@ -262,8 +299,14 @@ found nowhere are listed in
|
||||
`missing_libraries`, and libraries no package owns in `unpackaged_libraries`.
|
||||
`library_packages_reason` says why the lists are incomplete: the executable is
|
||||
not an ELF image (a launcher script, for example), `ldconfig`, `dpkg-query` or
|
||||
`apt-cache` is missing, a library could not be read or names more than 512
|
||||
libraries, or the walk stopped after 512 libraries.
|
||||
`apt-cache` is missing, `dpkg-query -S` could not tell which package owns a
|
||||
library (it exits 2 rather than 1, which only says no package owns the paths
|
||||
it did not name), a library could not be read or exceeds the selected dependency
|
||||
inventory budget, or the walk reached that budget. A failed owner query does
|
||||
not list the libraries it covered in `unpackaged_libraries`, since a lookup
|
||||
that could not finish is not proof that no package owns them. Completed batches
|
||||
retain their package and unowned-library facts. Paths containing package-query
|
||||
pattern characters stay unknown unless another candidate identifies an owner.
|
||||
|
||||
A sources file, key or library whose handle REA cannot close fails the capture
|
||||
with the file as its cleanup resource. When the repository check and the
|
||||
@@ -296,20 +339,29 @@ dot-folder named after the package, the executable, the desktop entry's full
|
||||
id and its last part, and the entry's name, with whether each existed at the
|
||||
start. A folder in those places whose name is one of these in another letter
|
||||
case joins them, such as GIMP's `~/.config/GIMP` for the `gimp` package; only
|
||||
the whole name is compared, so `~/.config/GIMPshop` does not. Dotfiles in the
|
||||
the whole name is compared, so `~/.config/GIMPshop` does not. A file under a
|
||||
folder the application creates during the capture beside one of them, with
|
||||
the same name in another letter case or with a version added (`-3.1` or
|
||||
`3`), as TreeLine's `~/.treeline-3.1`, is its state too. Dotfiles in the
|
||||
home named after the application (`.<name>rc`, or
|
||||
`.<name>` followed by `-`, `.` or `_`, for a name of three or more characters)
|
||||
that exist at the start join them, since an application may only read such a
|
||||
file during the capture. A `flatpak run` launch is named after its application
|
||||
file during the capture. So does KDE's settings file in the config folder,
|
||||
`<name>rc` such as Tellico's `~/.config/tellicorc`, for each such name of three
|
||||
or more characters, whether or not it exists yet. A `flatpak run` launch is named after its application
|
||||
id rather than the `flatpak` launcher, and adds `~/.var/app/<id>`.
|
||||
|
||||
**Session.** When it started, how long it ran and why it ended; the launched
|
||||
process's exit status; every process seen in the application's tree; every
|
||||
client window with its class pair, titles, owner pid and type, marked when it
|
||||
belongs to the application; when the first application window appeared; the
|
||||
belongs to the application. Titles and class names decode xprop's quoted
|
||||
delimiters, newline and tab escapes, and three-digit octal byte escapes;
|
||||
unsupported escape spellings leave that string unknown rather than inventing
|
||||
text. The session also records when the first application window appeared; the
|
||||
most common class pair of the application's windows, for matching its window
|
||||
later; files the application's processes held open under the walked roots; the first
|
||||
8 KiB of the launched process's stderr; and whether cleanup was verified. Each
|
||||
later; files the application's processes held open under the walked roots; the last
|
||||
8 KiB of the launched process's stderr, where a failing app's last words are,
|
||||
with `stderr_truncated` when earlier output was dropped; and whether cleanup was verified. Each
|
||||
process records whether the capture started it: the launched process, a
|
||||
process carrying the capture's run token, or a descendant of one, even when
|
||||
its environment cannot be read, as for a zombie or a sandboxed renderer.
|
||||
@@ -335,7 +387,9 @@ directory, which are named in the limitations instead; a selected path that is
|
||||
not a directory brings the `-wal` and `-journal` paths beside it, so a database
|
||||
selected on its own is captured with its logs), the state directories
|
||||
as walked (their physical spelling, which differs from the inventory's when one
|
||||
is a symlink), the paths every walk left out (the output folder and
|
||||
is a symlink), the selected paths (each `--state-path` and content-root
|
||||
`--argument` as named, absolute, beside the physical path walked, so a
|
||||
symlinked settings folder keeps what its name means), the paths every walk left out (the output folder and
|
||||
`--exclude` values), and the checkpoints in order: `baseline` before
|
||||
launch, each requested checkpoint, `stop` or `time-limit`, and `exit` after the
|
||||
application has gone. Each checkpoint records how many paths its walk saw,
|
||||
@@ -358,9 +412,17 @@ remaining content budget is not read, unless stored content of the same size
|
||||
could make it a duplicate. Files that could not be
|
||||
retained are listed with the reason: too large, over the content budget,
|
||||
changed since the walk, changed during the read, or unreadable.
|
||||
The baseline also records the paths and types of non-file entries it observed,
|
||||
including symlinks, without following them. This is required to distinguish an
|
||||
unreadable SQLite sidecar from an absent one. Captures made before this field
|
||||
was added must be made again before state inspection, task drafting or export.
|
||||
Directories the application held files open in, but whose starting content was
|
||||
not retained, are listed so a second capture can pass them with
|
||||
`--state-path`.
|
||||
`--state-path`, or `state_paths` on a build computer's `start_build_capture`,
|
||||
and a limitation names them, spelled from `~/` under the home. A lock file
|
||||
the application held open, such as Poedit's `~/poedit.lock` or Lucene's
|
||||
`write.lock`, makes no folder such a directory, since it holds nothing to
|
||||
retain.
|
||||
|
||||
**Limitations.** What the capture could not establish, such as a truncated
|
||||
walk or a missing window.
|
||||
@@ -377,3 +439,18 @@ evidence for attribution, not proof.
|
||||
|
||||
Clicks and keystrokes are not recorded. Window titles are sampled every half
|
||||
second, so short-lived titles can be missed.
|
||||
|
||||
Dependency inventory headroom is selected for each capture with
|
||||
`--library-policy '{"libraries":1024,"string_bytes":8192}'`. Unset fields retain
|
||||
their defaults: `libraries: 512` bounds both the closure walk and distinct
|
||||
DT_NEEDED offsets in each ELF image; `search_path_entries: 16` bounds distinct
|
||||
DT_RUNPATH and DT_RPATH offsets per kind; `string_bytes: 4096` includes the NUL
|
||||
terminator of each decoded string; `table_bytes: 1048576` bounds each program
|
||||
header or dynamic table. Values must be positive safe integers. A budget refusal
|
||||
names the field that needs headroom; partial package facts and unresolved
|
||||
coverage remain visible. The policy applies during this capture's admission,
|
||||
without restarting REA, and explicit selections are retained in Evidence. Raising
|
||||
content retention or a worker heap does not raise these inventory budgets.
|
||||
Choosing a new capture still launches the selected application; changing these
|
||||
read budgets does not add execution authority or recover an earlier capture's
|
||||
omitted inventory.
|
||||
@@ -15,23 +15,45 @@ agent-env run ./mousepad-bundle --dry-run
|
||||
|
||||
The export reads the capture and its [state view](desktop-application-state.md);
|
||||
it does not run the application. `--name` sets the environment name; otherwise
|
||||
it comes from the desktop entry id or the program, with an `app-` prefix when
|
||||
it comes from the desktop entry id or the program: a reverse-DNS id's last
|
||||
part (`org.xfce.mousepad` gives `mousepad`), or else the whole id without a
|
||||
trailing version (`glabels-3.0` gives `glabels`), with an `app-` prefix when
|
||||
that name would not be a portable bundle name (for example all digits). The
|
||||
output directory must be new or empty. `--task` adds a task an agent wrote for
|
||||
the capture, once REA has validated it against the capture; see
|
||||
[desktop application tasks](desktop-application-tasks.md). `--package-url`
|
||||
the capture, once REA has validated it against the capture; repeat it for
|
||||
several tasks, each checked at its own end checkpoint, which
|
||||
`--task-end NAME=CHECKPOINT` names; see
|
||||
[desktop application tasks](desktop-application-tasks.md#several-tasks). `--package-url`
|
||||
says where the computer downloads the package of an application installed from
|
||||
a `.deb` file; see [Applications from a .deb file](#applications-from-a-deb-file).
|
||||
`--download-install FILE` says which AppImage or tarball a build computer
|
||||
unpacked the application from; see
|
||||
[Applications from an AppImage or tarball](#applications-from-an-appimage-or-tarball).
|
||||
`--extra-package NAME`, repeatable, installs another Ubuntu 24.04 archive
|
||||
package beside the application, such as a runtime piece its package does not
|
||||
install; a [build computer](desktop-build-witness.md) passes its request's
|
||||
`extra_packages`, or a `.deb` request's `dependencies`, this way. A name that
|
||||
is not a Debian package name is refused (`extra_package`). Names the bundle
|
||||
installs anyway, the application's own package and the dependencies it names
|
||||
for a package installed from its `.deb` file, are not repeated; the rest are
|
||||
listed in the limitations and the README. A package the captured `.deb` lists
|
||||
as one alternative, such as `openjdk-17-jre | java-runtime`, is kept when apt
|
||||
installs the application by name, since apt may satisfy it with another.
|
||||
`--seed-checkpoint` names the checkpoint whose state seeds the bundle, the
|
||||
baseline by default, and `--end-checkpoint` the one a task must pass on, the
|
||||
capture's last by default; see [The seed checkpoint](#the-seed-checkpoint).
|
||||
`--synthetic-seed` says the seed was made for the task, as a build agent makes
|
||||
it in a [build computer](desktop-build-witness.md), rather than taken from the
|
||||
user's own data; see [A seed made for the task](#a-seed-made-for-the-task).
|
||||
|
||||
The command returns Evidence with operation `export_desktop_application` and
|
||||
predicate type `rea.desktop-application-export`, listing the environment name,
|
||||
the seed and end `checkpoints`, owned paths, seed files, the seeds it
|
||||
rewrote to remove the capture host's name and user (`scrubbed_files`), the
|
||||
bundle files that still hold them (`unscrubbed_files`), and written files,
|
||||
the last three by their path in the bundle. A failed export, cancellation
|
||||
gzip seeds it compressed again only to clear their headers
|
||||
(`recompressed_files`), the bundle files that still hold them
|
||||
(`unscrubbed_files`), and written files, the last four by their path in the
|
||||
bundle. A failed export, cancellation
|
||||
included, removes the bundle it had not yet published, so no partial copy of the
|
||||
user's data remains. If the bundle was published but its output handles could
|
||||
not be closed, the command reports a cleanup error that carries the export
|
||||
@@ -39,17 +61,19 @@ Evidence; the bundle itself is complete.
|
||||
|
||||
## The bundle
|
||||
|
||||
| Path | Contents |
|
||||
| -------------------------------- | -------------------------------------------------------------------------------------------------- |
|
||||
| `envs/<name>/env.toml` | `type = "computer_app"`, the module and class, `computer = "ubuntu-24.04"`, `driver = "reference"` |
|
||||
| `envs/<name>/<file>.py` | The `ComputerAppEnvironment` class |
|
||||
| `envs/<name>/SKILL.md` | What the app is, its window class and where its files live |
|
||||
| `envs/<name>/seed/` | The application's files at the seed checkpoint, by home-relative path |
|
||||
| `envs/<name>/apt-keys/` | The signing key of each vendor apt repository the app installs from |
|
||||
| `envs/<name>/install-package.sh` | With `--package-url`: the run script that installs the downloaded package with dpkg |
|
||||
| `tasks/smoke.json` | A task that deploys the app |
|
||||
| `tasks/<task>.json` | With `--task`: the agent's task, its verifier in `artifacts/` and the eval in `evals/` |
|
||||
| `README.md` | What was exported, from which capture, and what to keep in mind |
|
||||
| Path | Contents |
|
||||
| --------------------------------- | -------------------------------------------------------------------------------------------------- |
|
||||
| `envs/<name>/env.toml` | `type = "computer_app"`, the module and class, `computer = "ubuntu-24.04"`, `driver = "reference"` |
|
||||
| `envs/<name>/<file>.py` | The `ComputerAppEnvironment` class |
|
||||
| `envs/<name>/SKILL.md` | What the app is, its window class and where its files live |
|
||||
| `envs/<name>/seed/` | The application's files at the seed checkpoint, by home-relative path |
|
||||
| `envs/<name>/apt-keys/` | The signing key of each vendor apt repository the app installs from |
|
||||
| `envs/<name>/install-package.sh` | With `--package-url`: the run script that installs the downloaded package with dpkg |
|
||||
| `envs/<name>/install-download.sh` | With `--download-install`: the run script that unpacks the downloaded AppImage or tarball |
|
||||
| `envs/<name>/applications/` | With a tarball's `--download-install`: the desktop entry REA wrote for its program |
|
||||
| `tasks/smoke.json` | A task that deploys the app |
|
||||
| `tasks/<task>.json` | With `--task`: each of the agent's tasks, its verifier in `artifacts/` and the eval in `evals/` |
|
||||
| `README.md` | What was exported, from which capture, and what to keep in mind |
|
||||
|
||||
The class file is `<name>_app.py` with hyphens as underscores, for example
|
||||
`mousepad_app.py`: agentenv imports it under its stem, so a plain `<name>.py`
|
||||
@@ -59,9 +83,10 @@ an `App` suffix.
|
||||
|
||||
The class declares:
|
||||
|
||||
- `Packages(apt=[...])` with the package the capture found. A vendor package
|
||||
is named `<package>/<release>`, its repository's release (the Suite apt
|
||||
reports as `a=`, or else its Codename), so apt takes it from that repository
|
||||
- `Packages(apt=[...])` with the package the capture found, then the
|
||||
`--extra-package` names. A vendor package is named `<package>/<release>`,
|
||||
its repository's release (the Suite apt reports as `a=`, or else its
|
||||
Codename), so apt takes it from that repository
|
||||
even when the Ubuntu archive offers a higher version or epoch, as noble's
|
||||
transitional `firefox` does over packages.mozilla.org's; it is named alone
|
||||
when the release names neither, or when the capture was written before
|
||||
@@ -101,7 +126,8 @@ The class declares:
|
||||
reason (a capture
|
||||
written before restrictions were recorded counts as unrestricted); when
|
||||
another usable repository lists the same version it is left out instead,
|
||||
with a limitation. The export also refuses snaps, flatpaks, AppImages, and a
|
||||
with a limitation. The export also refuses snaps, flatpaks, an AppImage or
|
||||
tarball not named with `--download-install`, and a
|
||||
package no recorded repository lists unless it was installed from a `.deb`
|
||||
file the capture recorded and `--package-url` is given, since the computer
|
||||
could not install them by name, and a capture written before credentials
|
||||
@@ -124,6 +150,17 @@ The class declares:
|
||||
desktop entry without `Path`), which a relative argument or program path
|
||||
such as `./app` also needs, and still starts if that folder is missing. The
|
||||
export refuses a capture that saw no application window.
|
||||
- The seed's document, opened at launch, so the app shows it rather than its
|
||||
start screen: when the seed holds exactly one file the state view gives the
|
||||
`document` role, and the desktop entry's `Exec` takes files (a `%f`, `%F`,
|
||||
`%u` or `%U` field code), its path under `$HOME` is appended to the command.
|
||||
A command that already names a document, absolute or from its working
|
||||
directory, as one captured with `--argument <file>` does, is kept as it is;
|
||||
a settings path under the home does not count. A seed with no document launches
|
||||
as captured. With several, no desktop entry or an `Exec` that takes no file,
|
||||
the app launches as captured and the README and the result's limitations say
|
||||
why. A reset
|
||||
relaunches the app, so it opens the document again.
|
||||
- For an Electron application (the capture found its fuse sentinel or a
|
||||
`resources/app.asar`), `--no-sandbox` right after the program unless the
|
||||
command already has it: Chromium's sandbox needs user namespaces the
|
||||
@@ -131,7 +168,8 @@ The class declares:
|
||||
README and the result's limitations say the sandbox is off.
|
||||
- `paths`: the state directories holding the application's state files and the
|
||||
folders holding its documents, nested folders collapsed. A folder beside a
|
||||
state directory whose name differs from it only in letter case is owned
|
||||
state directory whose name differs from it only in letter case, or that
|
||||
adds a version, as TreeLine's `~/.treeline-3.1` beside `~/.treeline`, is owned
|
||||
whole in the files' spelling, so a `~/.config/GIMP` profile GIMP created
|
||||
during the capture beside the candidate `~/.config/gimp` is owned as
|
||||
`.config/GIMP`, not as the folder of its first file. The home itself and
|
||||
@@ -140,9 +178,13 @@ The class declares:
|
||||
Files the capture recorded under a symlinked state directory's physical
|
||||
spelling are mapped back to the innermost such directory's home spelling,
|
||||
unless that spelling is outside the captured home, as when the home itself is
|
||||
a symbolic link. When two files map to one home path, the export uses the one
|
||||
present before the capture, else the first by path, and a limitation names
|
||||
the other.
|
||||
a symbolic link. When a selected link names an application state directory,
|
||||
the bundle seeds the application's state path and restores the selected
|
||||
alias at boot. Targets recorded as directories are created even when the seed
|
||||
has no files there; file targets and targets of unknown kind are not turned
|
||||
into directories. Selections nested under that alias are seeded under the same
|
||||
state path. An existing incompatible alias stops setup without replacing it. When two files map to one home path, the export uses the one present
|
||||
before the capture, else the first by path, and a limitation names the other.
|
||||
- Data hooks, because declaring any hook switches off agentenv's default data
|
||||
plane: the class seeds the home once per boot and keeps it as the initial
|
||||
state with `keep_initial`, `data/add` and `data/reset` call agentenv's `add`
|
||||
@@ -155,10 +197,76 @@ retain is not seeded: one in the state view without retained content at that
|
||||
checkpoint, or one the capture omitted (for example too large or past the
|
||||
content budget). The result lists them in `unseeded_files` and the README
|
||||
names them. When the application held files open in folders whose starting
|
||||
content the capture did not retain, a limitation names those folders. The
|
||||
desktop Trash, `~/.local/share/Trash`, is never the application's, so a file
|
||||
the user moved there is neither owned nor seeded, even when a window title
|
||||
names it; a limitation lists the files the Trash held at the seed.
|
||||
content the capture did not retain, a limitation names those folders, spelled
|
||||
from `~/` under the home: a file there is seeded only when the capture
|
||||
retained it, as it does one written during the capture, and the limitation
|
||||
names the seed files there or says there are none. The desktop Trash,
|
||||
`~/.local/share/Trash`, is never the application's, so a file the user moved
|
||||
there is neither owned nor seeded, even when a window title names it; a
|
||||
limitation lists the files the Trash held at the seed.
|
||||
|
||||
Files that changed during the capture outside the owned paths are neither
|
||||
owned nor seeded, so the app starts without what it saved there: MuseScore
|
||||
keeps its settings in `~/.config/MuseScore`, which its name rules miss, and
|
||||
showed its first-run wizard on every boot. A limitation and the README name
|
||||
those folders, leaving out desktop services' files, each collapsed to its
|
||||
folder in `~/.config`, `~/.local/share`, `~/.local/state` or `~/.local`, the
|
||||
hidden folder of the home it lies in, or else its own folder, up to eight and
|
||||
a count of the rest. Capture again naming them with `--state-path`, or
|
||||
`state_paths` on a build computer, to own them.
|
||||
|
||||
Lock files the app holds while it runs are left out of the seed: a copied one
|
||||
makes the app find its profile or document in use, and a LibreOffice Calc
|
||||
bundle whose seed held its profile's `.lock` showed no window at boot. They
|
||||
are `lockfile`, `lock`, `.lock`, `.parentlock`, `parent.lock`,
|
||||
`SingletonLock` and Lucene's index lock `write.lock` anywhere, as Poedit's
|
||||
translation memory holds, LibreOffice's document lock `.~lock.<name>#`, a
|
||||
`<file>.lck`, `.<file>.lock`, `.<file>.lck` or `<file>.<ext>.lock` beside a
|
||||
file the capture recorded in the same folder, such as GnuCash's
|
||||
`household.gnucash.LCK`, Grisbi's empty `.household.gsb.lock`, which made it
|
||||
refuse every save, and VYM's `plan.vym.lock` (a project's `Gemfile.lock`
|
||||
beside its `Gemfile` has no extension to its file, so it is seeded), and
|
||||
`*.lock` or `*.lck` directly in the home, as wxWidgets' single-instance
|
||||
`~/poedit.lock`, or directly in one of the application's state directories;
|
||||
a lone `notes.lck` or `.notes.lock` elsewhere is seeded. A lock
|
||||
never makes a path the application's own, so `~/poedit.lock` is neither owned
|
||||
nor named among the folders the bundle does not own.
|
||||
Elsewhere such a file may be a project's own, such as `Cargo.lock` or
|
||||
`yarn.lock`, which pin dependencies, and is seeded, even under a folder whose
|
||||
name starts with a dot. A file the state view makes a document, or one named
|
||||
itself by `--state-path` or an absolute `--argument`, is never left out,
|
||||
whatever its name. A limitation and the README list them, up to
|
||||
eight and a count, after "Left out of the seed". They stay under the owned
|
||||
paths, so `data/get` returns the lock the running app holds, and `data/reset`
|
||||
restores the seed without them.
|
||||
|
||||
A seed file keeps the permission bits it had at the seed when the capture saw
|
||||
it change, such as a `0600` file an app checks: the class module lists those
|
||||
other than `0644` in `SEED_MODES`, and the first boot sets them when it copies
|
||||
the seed. A file the capture never saw change has no recorded mode and is
|
||||
written `0644`.
|
||||
|
||||
Files in app state are retained regardless of a `.log` extension or `log.<n>`
|
||||
name. They can hold conversation history or database writes as well as
|
||||
diagnostics. The normal host-identity scrub applies to retained text; filenames
|
||||
alone do not identify disposable diagnostics. Locks beside these files follow
|
||||
the lock rules above.
|
||||
|
||||
Other seed files the app keeps for its own
|
||||
running are seeded as captured and named, collapsed to the folder that holds
|
||||
them: WebKit or Chromium storage (such as `hsts-storage.sqlite`, `storage/`,
|
||||
`mediakeys/`, `Local Storage/`, `IndexedDB/`, `Cookies`, `Visited Links`,
|
||||
`Platform Notifications/`, and a whole `QtWebEngine/`
|
||||
profile), which may hold the app's own data, and caches (a folder whose name ends in `cache`); a stale one
|
||||
can change how the app starts. A seeded LibreOffice profile's
|
||||
`registrymodifications.xcu` ships without its `RecoveryList` items, which
|
||||
LibreOffice writes for the documents it has open and which would start the
|
||||
bundle into document recovery instead of its window; a limitation and the
|
||||
README name the file. REA finds the items with its XML parser, as `item`
|
||||
elements whose `oor:path` is `/org.openoffice.Office.Recovery/RecoveryList`
|
||||
however the tags are spelled, and cuts exactly those elements, so text in a
|
||||
comment or CDATA and every other byte stay as captured. A registry that is
|
||||
not well-formed XML ships unchanged.
|
||||
|
||||
The application's dconf keys at the seed, if any, are written once when the
|
||||
app first starts, also for an application that owns no files, and the class adds
|
||||
@@ -168,7 +276,14 @@ existing user database through a `file-db:` profile, then runs `dconf compile`
|
||||
on that dump and the captured keys, in that name order so the captured keys
|
||||
win, and replaces `~/.config/dconf/user`. If that fails, the server prints a
|
||||
one-line warning and the app starts without the captured keys. `data/reset` does
|
||||
not restore them.
|
||||
not restore them. No file of the bundle holds them, so `data/get` does not
|
||||
return them and no task check reads them; a draft whose only change between
|
||||
its checkpoints is to the application's dconf keys says so with
|
||||
`uncheckable_changes`, and one whose dconf database changed but could not be
|
||||
read says so with `unreadable_changes`. When a dconf database changed by the
|
||||
seed checkpoint but its version there could not be read, as without
|
||||
`dconf-cli`, the export seeds no keys from it and a limitation names it with
|
||||
the read error.
|
||||
|
||||
## The seed checkpoint
|
||||
|
||||
@@ -185,6 +300,52 @@ The export refuses a name the capture does not hold exactly once, and an end
|
||||
checkpoint taken before the seed checkpoint; `inspect-desktop-application-state`
|
||||
lists the names in `capture.checkpoints`.
|
||||
|
||||
## A seed made for the task
|
||||
|
||||
By default the seed is the user's real data: the README and the result's
|
||||
limitations say to keep the bundle local and not publish it. With
|
||||
`--synthetic-seed` they say instead that the seed was made for the task, and
|
||||
to read `envs/<name>/seed/` before publishing the bundle; the export cannot
|
||||
tell the two apart, so the flag is the caller's word. `export_build_environment`
|
||||
passes it, since the building agent makes the seed in the build computer. The
|
||||
Evidence records it as `synthetic_seed`.
|
||||
|
||||
## Withheld sign-in state
|
||||
|
||||
`--withhold-from <checkpoint> --withhold-to <checkpoint>` withholds every file
|
||||
under the owned paths that changed after the first checkpoint up to the
|
||||
second, as the capture recorded its changes; `--withhold-path <file>` adds a
|
||||
home-relative file, such as one written at first launch. The draft
|
||||
(`describe-desktop-application-task`) takes the same options, leaves those
|
||||
files out and lists them as `withheld_files`; the export refuses a check that
|
||||
names one. Each withheld file present at the seed is left out of
|
||||
`envs/<name>/seed/` and becomes a secret setting:
|
||||
|
||||
- the class declares `Settings` with a `SecretStr` field per file, and
|
||||
`WITHHELD`, each field's home path;
|
||||
- `env.toml` sets each field to `env:REA_<APP>_<FILE>`, so agentenv refuses
|
||||
the deploy, naming the variable, until it is set;
|
||||
- at first boot, before keeping the initial state, the class writes each file
|
||||
from its setting, the file's bytes in base64, mode `0600`, so `data/reset`
|
||||
restores it, and `data/get` serves it neither as a part nor in the views;
|
||||
- the README names each variable, with
|
||||
`export REA_<APP>_<FILE>="$(base64 -w0 < ~/<file>)"` from the receiver's
|
||||
own signed-in copy of the app.
|
||||
|
||||
A withheld file larger than 48 KiB is refused, since its base64 must fit one
|
||||
environment variable. `--withheld-copy <folder>` copies the withheld files,
|
||||
as captured, into a new or empty folder outside the bundle. The result lists
|
||||
them as `withheld_files`, each with its `setting`, `variable` and `handle`.
|
||||
REA resolves both destinations before writing either. A path that cannot be
|
||||
resolved, including a dangling link, is refused; the copy and bundle cannot
|
||||
contain one another, including through symbolic links.
|
||||
|
||||
`--consent <file>` ships the person's consents, a `consent.json` that
|
||||
`record_build_consent` records, as the bundle's own `consent.json`.
|
||||
`--seed-withheld` seeds the withheld files after all, refused unless that
|
||||
file holds a `seed_sign_in` consent; the result lists them as
|
||||
`consented_files`, and the README says they are there with consent.
|
||||
|
||||
## Host name and user
|
||||
|
||||
A seed can hold the capture host's name and the user's: Geany saved its mail
|
||||
@@ -228,6 +389,28 @@ export removes them from what the bundle ships:
|
||||
boot, as it did the captured home before. The replacements read as the
|
||||
computer's own: its user is `user`, with its home in `/home/user`, and
|
||||
`localhost` is the name every host has for itself.
|
||||
- A gzip-compressed seed, such as a Gnumeric workbook, is decompressed, up to
|
||||
256 MiB, scrubbed as the format the state view read its document as, and
|
||||
compressed again whether or not its document changed, so no file name,
|
||||
comment or time its header held ships: with no file name, comment or extra
|
||||
field, modification time 0 and the operating system unknown, so the same
|
||||
document gives the same bytes on every host. It is deflated at the fastest
|
||||
level, or stored when deflating would make it expand more than the 100 times
|
||||
a view reads, so a seed that read still reads. It is listed in
|
||||
`scrubbed_files` when its document changed, and otherwise in
|
||||
`recompressed_files`, which the README counts rather than lists, so the
|
||||
example and template files an app copies on first run do not bury the
|
||||
document scrubbed; one already in that form ships as captured. The class copies it at
|
||||
boot as it is, so the home it holds stays `/home/user`. One that does not
|
||||
decompress within the scrub limit ships as captured, its bytes searched.
|
||||
A document the state view did not read can still be decompressed and
|
||||
recompressed under the separate 256 MiB scrub limit; its content is searched
|
||||
even when its format cannot be scrubbed. One whose document needs no edit but a member of
|
||||
which has an extra field, where BGZF (`.vcf.gz`, `.bam`) and dictzip
|
||||
(`.dict.dz`) files index their blocks, also ships as captured, its bytes
|
||||
searched, so its blocks stay where its index says; one whose document
|
||||
changed is still compressed again as one member, losing them, as the
|
||||
export's limitation says.
|
||||
- Binary files, SQLite databases among them, and text files over 256 MiB are
|
||||
copied as captured. Then every file the bundle ships, the class module with
|
||||
its dconf keys and the README included, is searched by a looser rule than
|
||||
@@ -294,6 +477,73 @@ declared vendor repository installs the package already; when the URL is not
|
||||
written as `http://` or `https://` or holds a user name or password; and when
|
||||
the file is built for an architecture other than `amd64`, `arm64` or `all`.
|
||||
|
||||
## Applications from an AppImage or tarball
|
||||
|
||||
A [build computer](desktop-build-computer.md) can install an application from
|
||||
its project's AppImage or tarball, pinned by URL and SHA-256, under
|
||||
`/opt/rea-apps/<desktop entry>`. Neither holds a Debian package, so the
|
||||
capture records the package as `unknown`, and the export takes the install
|
||||
from `--download-install`, a JSON file the build computer writes from its
|
||||
request:
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": "appimage",
|
||||
"desktop_entry": "org.freecad.FreeCAD",
|
||||
"url": "https://github.com/FreeCAD/FreeCAD/releases/download/1.1.4/FreeCAD_1.1.4-Linux-aarch64-py311.AppImage",
|
||||
"sha256": "675d1a4295b2183e0e1d816a4b5557dcb63aeced4e60aaa28d6c3686f4c71dc9",
|
||||
"architecture": "arm64"
|
||||
}
|
||||
```
|
||||
|
||||
A tarball's adds `program`, its path inside the archive, and optionally
|
||||
`name`. The class then declares
|
||||
`downloads=[Download(url=..., sha256=..., to="vendor/app.AppImage", arch=...)]`
|
||||
(`vendor/app.tar` for a tarball) and `run=[RunScript(script="install-download.sh")]`,
|
||||
the same script the build computer ran, with no apt package for the app. The
|
||||
bundle does not contain the application and supports only the download's
|
||||
architecture; the URL must keep serving those bytes, and the application's
|
||||
licence is the operator's to check.
|
||||
|
||||
The export refuses the file (`download_install`) unless it names an
|
||||
`appimage` or `tarball` with an `http://` or `https://` URL and a SHA-256,
|
||||
the capture launched its `desktop_entry`, the captured program lies under its
|
||||
folder in `/opt/rea-apps`, and the capture ran on its architecture; it
|
||||
refuses `--package-url` beside it.
|
||||
|
||||
## Grading once the app has saved
|
||||
|
||||
A task's verifier, `artifacts/<task>-check/verify.py`, grades the app's files
|
||||
only once they have settled, because an app can save on its own timer after the
|
||||
last change. Gnote saves a note 4 s after the last key, so a read taken as
|
||||
soon as the agent stops finds the text on screen but not yet in the file. The
|
||||
verifier reads the app's `data/get` every 0.5 s. It compares each file's bytes
|
||||
between reads by their SHA-256, as `data/get` returns them, and grades once
|
||||
every file has held for 3 s. The 3 s count from when a read returned the
|
||||
current bytes, not when it began, so a slow read that brings a change late
|
||||
does not count its own run as quiet. 15 s after the first read began, the
|
||||
bound, the verifier grades the last completed read: no read starts at or
|
||||
after the bound, and a read still running then is cut. Each result records
|
||||
whether the files settled in `settled`, and whether the bound cut a read in
|
||||
`read_cut`.
|
||||
|
||||
The window has a cost and a limit:
|
||||
|
||||
- An app whose files do not change during the check, such as one that saved
|
||||
at once, is graded 3 s after the first read returns.
|
||||
- A save that lands more than 3 s after the first read returns is still
|
||||
missed. An app that saves 4 s after the last key is caught only when the
|
||||
agent stops at least a second after that key.
|
||||
- Files the app rewrites more often than every 3 s, such as a log, never
|
||||
settle, so every check waits until the bound.
|
||||
- The bound counts the reads too, so a slow `data/get` ends the check at
|
||||
15 s. Only the first read, which every grade needs, runs to its own
|
||||
timeout.
|
||||
|
||||
The bound and window are `SETTLE_BOUND_S` and `SETTLE_WINDOW_S` in the
|
||||
verifier. `check_build_task` waits the same way before it grades each
|
||||
witness stage; see [witnessing](desktop-build-witness.md#witness).
|
||||
|
||||
## Typed views
|
||||
|
||||
The views follow the [state view](desktop-application-state.md)'s formats and
|
||||
@@ -308,13 +558,30 @@ as their entries, read with Python's `zipfile` from the file in place, with
|
||||
REA's entry, size, compression ratio and archive limits and its wording for an
|
||||
entry it does not read. The module reads the central directory itself, as REA
|
||||
does, and shows `zipfile` only the records it lists, reading each entry in
|
||||
bounded chunks no further than a byte past its declared size. A JSON file
|
||||
bounded chunks no further than a byte past its declared size. A gzip file is
|
||||
the view of the document inside it, with `"compression": "gzip"` beside its
|
||||
format, decompressed with Python's `zlib` within the same entry size and ratio
|
||||
limits, its members joined as REA joins them. A JSON file
|
||||
holding a number JavaScript cannot read exactly, such as an integer above 2^53,
|
||||
or nested more than 500 levels deep, gets an `error` instead of a view, as in
|
||||
REA, while text that is not one JSON
|
||||
or whose retained value nests more than 500 levels deep, gets an `error` instead of a view, as in
|
||||
REA. Duplicate keys keep their last value before the depth check. Text that is not one JSON
|
||||
document, such as JSON Lines, stays text. Format detection follows REA's
|
||||
rules. Files larger than 64 MiB get an `error` instead of a view
|
||||
and are not read. XML is read with Python's `expat` twice: with namespace
|
||||
and are not read. The selected `file_bytes` also applies to scrubbed seed task
|
||||
views before admission, independently of how much seed data the export can
|
||||
copy. SQLite table views have a shared `sqlite_view_bytes` allowance, default
|
||||
16 MiB of compact UTF-8 JSON per table, checked before retaining each row.
|
||||
An oversized table keeps its error while other tables remain readable. Raising
|
||||
this view allowance does not raise the independent 16 MiB native SQLite
|
||||
value/record limit or REA worker reply limit; worker facts include additional
|
||||
schema and typed-cell metadata and may be refused earlier. The TypeScript
|
||||
and exported Python readers count their respective native compact JSON encodings;
|
||||
float and exponent spelling can make their exact byte thresholds differ even
|
||||
when they report equal numeric facts. The Python byte counter quotes large
|
||||
strings in bounded chunks before retention.
|
||||
Exported SQLite readers require Python 3.11 or later for the native connection limits; the
|
||||
AgentEnv compatibility workflow uses Python 3.12.
|
||||
XML is read with Python's `expat` twice: with namespace
|
||||
processing, which refuses what REA's parser refuses, such as an undeclared
|
||||
prefix, then without it, so element and attribute names keep the prefixes the
|
||||
document spells and namespace declarations stay attributes, as in REA's view;
|
||||
@@ -337,6 +604,6 @@ when a DOCTYPE names a DTD. Known differences remain:
|
||||
and then gives the file an `error`, and so does a table or column name that
|
||||
is not UTF-8.
|
||||
|
||||
The seed is the user's real data. Keep the bundle local and do not publish it.
|
||||
The class was checked against agentenv-framework 0.9.1431 with
|
||||
`python -m agentenv_protocol.computer._declare` and `agent-env run --dry-run`.
|
||||
Unless it was [made for the task](#a-seed-made-for-the-task), the seed is the
|
||||
user's real data. Keep the bundle local and do not publish it.
|
||||
The class targets agentenv-framework 0.9.1485.
|
||||
@@ -11,6 +11,65 @@ again.
|
||||
rea inspect-desktop-application-state ./mousepad-capture --format json
|
||||
```
|
||||
|
||||
Views use bounded defaults. When a view reports truncation or a byte-budget
|
||||
refusal, retry the same capture with `--view-policy` JSON overrides:
|
||||
|
||||
```bash
|
||||
rea inspect-desktop-application-state ./mousepad-capture \
|
||||
--state-path /home/ada/Documents/notes.txt \
|
||||
--view-policy '{"text_lines":10000,"file_bytes":134217728}' --format json
|
||||
```
|
||||
|
||||
`--state-path` selects an exact original captured path and can be repeated.
|
||||
Omitting it examines every captured path. Selection preserves the captured
|
||||
checkpoint and digest; it does not substitute the file's current contents.
|
||||
A path absent from the capture is an input error.
|
||||
|
||||
`--view-policy` accepts positive safe-integer overrides for `file_bytes`,
|
||||
`sqlite_tables`, `sqlite_rows`, `sqlite_view_bytes`, `text_lines`, `xml_nodes`, `xml_depth`,
|
||||
`parsed_bytes`, `json_depth`, `zip_entries`, `zip_entry_bytes`, `zip_bytes`,
|
||||
and `zip_ratio`. Unspecified controls keep the defaults described below.
|
||||
`xml_depth` admits at most 494 and `json_depth` at most 991: these bounds
|
||||
follow the fixed Evidence JSON nesting limit, including the captured-version
|
||||
and ZIP-member wrappers and XML's element/children representation. A request
|
||||
above either bound is rejected before reading the capture.
|
||||
Evidence parameters retain only the supplied `view_policy` overrides; omission
|
||||
and an explicitly empty selection remain distinct. State inspection and task
|
||||
drafts report the resolved policy separately as
|
||||
`normalized_result.effective_view_policy`. Changes apply on this call, without
|
||||
a restart. File and ZIP byte counts refer to retained file bytes or uncompressed
|
||||
ZIP member bytes, not transport size. Gzip document views share `zip_entry_bytes`
|
||||
and `zip_ratio` with ZIP entry views; `zip_bytes` bounds a ZIP archive, not a
|
||||
gzip stream. These selections also apply to the decompressed document's format
|
||||
view. `sqlite_view_bytes` instead bounds each
|
||||
projected table's compact UTF-8 JSON, including column names, primary keys,
|
||||
rows and truncation metadata; it defaults to 16 MiB. Rows are charged before
|
||||
retention. A table that exceeds this allowance keeps an actionable `error`,
|
||||
while other tables remain readable. Increase this control and retry the same
|
||||
capture for a projected-view refusal. It does not raise the SQLite worker's
|
||||
separate fixed value/record or complete-reply limits, which may refuse earlier.
|
||||
Each reader counts its native compact JSON encoding. Float spelling can differ:
|
||||
Python emits `16.0` where JavaScript emits `16`, and exponent spelling also
|
||||
varies. Identical numeric facts can therefore cross this byte threshold at
|
||||
different settings. This is not a common transport-byte guarantee.
|
||||
|
||||
The same option is available on `describe-desktop-application-task` and
|
||||
`export-desktop-application`. Export uses the accepted policy for task
|
||||
validation and emits it into the Python readers used by the environment's
|
||||
`data/get`; re-export with another policy to change those reader budgets.
|
||||
Task checks apply `file_bytes` to the actual scrubbed seed bytes, including any
|
||||
growth from replacing the capture home with `/home/user`. A seed over that
|
||||
view allowance may still be exported under the independent acquisition limit,
|
||||
but its unreadable view cannot establish a known task failure.
|
||||
The TypeScript and exported Python readers share the policy's defaults.
|
||||
These desktop capture workflows remain CLI-only.
|
||||
|
||||
Raising a view budget cannot recover content the capture never retained,
|
||||
change numeric exactness, enable encrypted or unsupported archive formats,
|
||||
or raise SQLite worker, Evidence nesting, process-memory or transport limits.
|
||||
Keep increases specific to the exhausted budget. Truncation and incomplete
|
||||
comparison coverage remain visible until the requested facts were examined.
|
||||
|
||||
The result is Evidence with operation `inspect_desktop_application_state` and
|
||||
predicate type `rea.desktop-application-state`. Its confidence is `inferred`,
|
||||
because roles come from heuristics; the views themselves are read from the
|
||||
@@ -25,7 +84,7 @@ changed later. Each path records the evidence the capture holds about it:
|
||||
| ------------------------ | ------------------------------------------------------------------------- |
|
||||
| `state_directory` | It lies in one of the application's state directories, in either spelling |
|
||||
| `opened_by_application` | An application process held it open during a sample |
|
||||
| `named_in_window_title` | An application window title spelled its name or `~/` path on its own |
|
||||
| `named_in_window_title` | An application window title spelled its name, `~/` or absolute path whole |
|
||||
| `selected_path` | It lies under a `--state-path` or absolute `--argument` of the capture |
|
||||
| `dconf_application_keys` | It is a dconf database holding groups whose path names the application |
|
||||
| `desktop_service` | It lies where desktop services and toolkits keep shared files, or caches |
|
||||
@@ -40,11 +99,26 @@ The role follows from the reasons, in this order: `shared_desktop` for a file
|
||||
in the desktop Trash, which the user deleted, whatever else names it;
|
||||
`application_state` for application keys in dconf, or a state directory outside shared desktop
|
||||
locations; `document` for a file named in a window title or selected by the
|
||||
caller, even if the application holds it open; `application_state` for another
|
||||
caller, even if the application holds it open, except that a selected file
|
||||
under a hidden folder of the home, such as `~/.config/MuseScore`, a dotfile
|
||||
there, or a lock an application holds, such as LibreOffice's `.~lock.<name>#`
|
||||
beside a document, is `application_state`, judged by the path as the caller
|
||||
named it, so a file reached through `~/.config/MuseScore` stays the application's
|
||||
when that folder is a link to `~/settings/MuseScore`; the innermost selected
|
||||
path holding a file decides, so a score selected itself inside the link's
|
||||
target keeps its own name and is a document (a capture written before it
|
||||
recorded selected paths is judged by their physical spelling);
|
||||
`application_state` for another
|
||||
open file outside shared locations; `shared_desktop` for other files in shared
|
||||
locations; `unrelated` otherwise. A window title names a file only when no
|
||||
other file name character adjoins the name, so `day.txt` does not match
|
||||
`today.txt - Notes`.
|
||||
`today.txt - Notes`, and never by a folder of a path the title shows: in
|
||||
CherryTree's `home-notebook.ctd - /home/user/Documents - CherryTree 1.1.2`,
|
||||
`user` names no file. A name without an extension, such as `Makefile`, must
|
||||
also stand on its own rather than end a longer path, as in
|
||||
`Makefile - Editor`. A file under `~/.cache` or `~/.config/dconf` is never
|
||||
named by a title. The capture's checkpoint acknowledgements rank files by the
|
||||
same rule.
|
||||
|
||||
Application names, which select dconf groups, come from the package, the
|
||||
executable, the desktop entry's id and `StartupWMClass`, and the window match.
|
||||
@@ -70,11 +144,21 @@ whether the file was present, the SHA-256 of its retained bytes, and a view:
|
||||
| `dconf` | The `GVariant` header | Groups whose path names the application, plus the names of the other groups |
|
||||
| `zip` | A zip local file header, `PK\x03\x04`, whatever the name | Its file entries in archive order, each with its own format and view |
|
||||
| `json` | A `.json` name or a leading `{` or `[` that parses | The parsed value, from files up to 4 MiB nested up to 500 levels |
|
||||
| `xml` | An XML name or declaration that parses | An element tree (`name`, `attributes`, `children`); text kept exactly, DTDs never read |
|
||||
| `xml` | An XML name, a declaration or a leading element that parses | An element tree (`name`, `attributes`, `children`); text kept exactly, DTDs never read |
|
||||
| `keyfile` | Only `[group]`, `key=value` and comment lines, with at least one group | Groups of `key` → string value, from files up to 4 MiB |
|
||||
| `text` | Other UTF-8 text | Lines, up to 5,000 |
|
||||
| `binary` | NUL bytes or invalid UTF-8 | None |
|
||||
|
||||
A gzip file is read as the document inside it; see
|
||||
[compressed documents](#compressed-documents).
|
||||
|
||||
Text that opens with an element, such as a Freeplane map's
|
||||
`<map version="freeplane 1.7.0">` with no declaration, or with a DOCTYPE that
|
||||
has no internal subset and then an element, such as a KmPlot plot's
|
||||
`<!DOCTYPE kmpdoc>` before `<kmpdoc version="4">`, is XML when the whole text
|
||||
is well formed, whatever the file is called, a `.txt` file included;
|
||||
otherwise it stays text. The exported verifier makes the same choice.
|
||||
|
||||
A JSON value holding a number JavaScript would round, such as an integer above
|
||||
2^53, is not read rather than shown rounded.
|
||||
|
||||
@@ -128,7 +212,15 @@ could misstate its committed rows. The capture records them under its content
|
||||
roots, which include both beside a database selected on its own, when its
|
||||
baseline walk listed every directory, read every path, entered no other
|
||||
device, did not stop early and recorded what it left out, and wherever they change; an excluded log, and any elsewhere, is
|
||||
unknown, since an unchanged file and a missing one look alike. A table
|
||||
unknown, since an unchanged file and a missing one look alike. After an
|
||||
incomplete checkpoint, a database or sidecar without an explicit
|
||||
later change remains unconfirmed even if a later walk completes. REA does not
|
||||
combine that older file with a newly recorded member of the SQLite set. The
|
||||
historical versions remain listed; the affected SQLite view has a `view_error`.
|
||||
A baseline non-file at a sidecar path, or a non-directory ancestor, cannot
|
||||
establish an absent file. Captures without the required baseline non-file
|
||||
inventory must be made again before inspection.
|
||||
A table
|
||||
that cannot be read keeps its `error` in place of its rows, and changes to that
|
||||
database are marked incomplete. dconf
|
||||
databases are read with `dconf dump /` against a private file profile, so the
|
||||
@@ -185,6 +277,37 @@ so an archive of hundreds of thousands of entries costs no more to read
|
||||
than one of a thousand. An archive whose central directory cannot be read has a
|
||||
`view_error` of its own.
|
||||
|
||||
## Compressed documents
|
||||
|
||||
Gnumeric's `.gnumeric`, Dia's `.dia`, Xournal++'s `.xopp`, GnuCash's default
|
||||
book and Inkscape's `.svgz` are XML compressed with gzip. A file that starts
|
||||
with gzip's magic bytes, `1f 8b 08`, whatever its name, is read as the
|
||||
document inside it: its members joined, then given a format and view by the
|
||||
rules above under the file's own name, so a pointer or check into it is
|
||||
written exactly as for the same document uncompressed, and the task draft is
|
||||
the same. The version records the compression beside the format:
|
||||
|
||||
```json
|
||||
{
|
||||
"format": "xml",
|
||||
"compression": "gzip",
|
||||
"view": {
|
||||
"root": { "name": "gnm:Workbook", "attributes": {}, "children": [] },
|
||||
"truncated": false
|
||||
},
|
||||
"view_error": null
|
||||
}
|
||||
```
|
||||
|
||||
The document is read only when it is at most 16 MiB and at most 100 times the
|
||||
file's size, an archive entry's limits, so decompressing stops there; past
|
||||
them, or when the stream is damaged or cut short, the version has `format`
|
||||
null and a `view_error`, and a check into it is undecided. A SQLite or dconf
|
||||
database inside keeps its format and has a `view_error`, an archive inside is
|
||||
read as an archive, and another gzip stream inside is binary. A gzip file
|
||||
inside an archive reads as binary. Other compression, such as xz, Zstandard
|
||||
or bzip2, is not read: such a file is binary.
|
||||
|
||||
## Kinds of change
|
||||
|
||||
Between consecutive versions a change is `created`, `deleted`, or a
|
||||
|
||||
@@ -3,11 +3,12 @@
|
||||
A [desktop application export](desktop-application-export.md) can carry one
|
||||
task: a prompt for a computer-use agent and checks that grade what the agent
|
||||
left in the application's files. An agent, not REA, writes the prompt and the
|
||||
checks from the change the capture recorded. REA validates the task against
|
||||
the capture and keeps it only when its checks fail on the seed and all pass on
|
||||
the captured end state. REA runs no model and no code the agent wrote: the
|
||||
checks are declarative, and the exported verifier evaluates them the same way
|
||||
REA does.
|
||||
checks from the change the capture recorded. For file checks, REA validates
|
||||
the task against the capture and keeps it only when its checks fail on the
|
||||
seed and all pass on the captured end state. The checks are declarative;
|
||||
optional [readers](#readers) run commands to interpret files, and
|
||||
[screen judges](#screen-judges) use a model when no file shows the change.
|
||||
The exported verifier uses the same checks and reader runner as REA.
|
||||
|
||||
```sh
|
||||
rea describe-desktop-application-task ./mousepad-capture --format json > draft.json
|
||||
@@ -42,35 +43,457 @@ operation `describe_desktop_application_task` and predicate type
|
||||
database is compared by its rows, since its hash leaves out the `-wal`
|
||||
file. Text changes list each line
|
||||
once and add `line_counts`, the lines found at both ends whose number of
|
||||
occurrences changed. Views appear only for the files the suggestions cover,
|
||||
with arrays past 200 items and strings past 2,000 characters cut
|
||||
occurrences changed. An XML file's change names its members by the
|
||||
[selectors](#pointers-and-selectors) checks use, as an archive's already
|
||||
does, and by position only where nothing names a member or where only the
|
||||
members' order changed. Views appear only for the files the suggestions
|
||||
cover, with arrays past 200 items and strings past 2,000 characters cut
|
||||
(`view_cut`).
|
||||
- `values`, in each of those files: the values its change added, changed and
|
||||
removed, each with the `pointer` a check names it by, its `kind` (`added`,
|
||||
`changed` or `removed`), and `before` and `after`: the scalar, an element as
|
||||
compact XML with the text the XML reader kept, spaces between runs included,
|
||||
or a JSON value as compact JSON, cut at 600 characters, and null at the end
|
||||
that lacks it. A file created or deleted lists what its one end holds,
|
||||
`added` or `removed`: an archive's entries, an XML file's root element, a
|
||||
JSON file's members or a key file's keys, each whole once. A value only
|
||||
position reaches has `note: "by_position"`, and its pointer cannot go into a
|
||||
check; noise an app writes as it saves has `view_state`, `renumbered`,
|
||||
`volatile`, `timestamp`, `upkeep` or `layout` (see `task`). A record of
|
||||
text fields added to a list of its kind, as GnuCash's
|
||||
`<gnc:transaction>`, is listed field by field, and a field that lists
|
||||
records, as its `<trn:splits>`, record by record, so the cut never hides
|
||||
one. `refers_to` names up to
|
||||
five other elements a value equals the `key` or `id` (`xmi:id` among them)
|
||||
of, outside a view the app keeps of the document, as Umbrello's
|
||||
`listview` mirroring its model by id, with the
|
||||
`attribute` holding it, as HomeBank's `category="44"` names
|
||||
`<cat key="44" name="Groceries"/>`, or the id a child element holds, as a
|
||||
GnuCash split's `<split:account>` names the account whose `<act:id>` it
|
||||
is, shown with its name, `<gnc:account …><act:name>Groceries</act:name>…`;
|
||||
an element added whole lists those of the records of attributes it nests
|
||||
too, as a KMyMoney transaction's splits name their `<ACCOUNT>` and
|
||||
`<PAYEE>`, each once. Where several elements share that key, the one
|
||||
whose tag shares the attribute's stem, as `payee` and `<pay>`, and of
|
||||
those the one named for the attribute, as `account` an `<ACCOUNT>` before
|
||||
an `<ACCOUNTID>`. A plain number names an element only through an
|
||||
attribute so named or ending in `id`, `ref` or `key`, so a list's
|
||||
`count="5"` names no `<FIXVERSION id="5"/>`. A value that changed lists,
|
||||
in `alternatives`, its pointer naming the element by each other identifier
|
||||
or name-like attribute it holds alone among its tag's members, up to
|
||||
three, as an Inkscape text's `[@inkscape:label=Headline]` beside its
|
||||
`[@id=text12]`, for a check to name it by instead; an element its tag
|
||||
alone names inside the first one a condition names, as an EPUB chapter's
|
||||
`h1`, by its `[@id=chapter-two]`. What a check can
|
||||
name comes first, text an edit changed before the rest, so no cut drops
|
||||
the edit, then what only position reaches, then noise. A file lists
|
||||
at most 20 values and 8 KB, the draft 24 KB, and `values_cut` says when a
|
||||
limit left some out. Values cover XML, archives, JSON and key files; a text
|
||||
or SQLite file's change and suggestions already carry its lines and rows.
|
||||
- `task`: a draft with the name `change`, a placeholder prompt the export
|
||||
refuses, and suggested checks derived from the net change of the files with
|
||||
role `document`, or of the app's own state files when no document changed.
|
||||
role `document`, or of the app's own state files when no document changed,
|
||||
excluding locks the export leaves out of the seed. App state files ending
|
||||
in `.log` remain eligible for checks.
|
||||
A text file gains `lines_present` for its new lines and `lines_absent` for
|
||||
lines it lost (when either is at most ten), and `line_count` for each line
|
||||
holding a letter whose count changed; JSON and key files gain
|
||||
`value_equals` and `value_absent` by pointer; SQLite tables gain rows added,
|
||||
holding a letter whose count changed; a new line holding an XML attribute
|
||||
REA recognizes as generated, such as a named epoch time or `ID_329023692`, gets a
|
||||
`line_count` of the lines `contains` its text up to that value, as
|
||||
`<node TEXT="Water filter" ID="`, at the end's number, where both views
|
||||
hold the whole file, instead of a `lines_present` another save would
|
||||
fail, or a `volatile_lines` note when no text comes before it. UUID-shaped
|
||||
text stays exact because its shape alone does not show that it was generated;
|
||||
JSON and key files gain
|
||||
`value_equals` and `value_absent` by pointer, a JSON array's records named
|
||||
by an id field each holds alone at both ends (`id`, `uid`, `uuid`, `guid`,
|
||||
`key` or a name ending in `_id` or `Id`), as `/lists/[id=chores]`,
|
||||
whatever order the app saves them in; where the app generated that id, a
|
||||
record both ends hold is named by its kind field and its first name-like
|
||||
string field (a key ending in `name`, `title`, `label` or `caption`),
|
||||
nearest first and up to three keys deep, that holds the same string at
|
||||
both ends and names it alone at each, as TreeLine's category by
|
||||
`/nodes/[format=CATEGORY][>data~1NAME=Science]`, and by its id where none
|
||||
does, as `/nodes/[uid=7584d494d9f511f1a873edd4f1920549]` once its name
|
||||
changed; and a record only one end holds is named by such a field that
|
||||
names it alone at its end and nothing at the other, as TreeLine's new book by
|
||||
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, its other fields
|
||||
checked through that selector and the fields the selector reads not, nor
|
||||
a field holding what every like record (one of its kind) at the end holds
|
||||
there, as the empty `children` of every book; and
|
||||
by position only where nothing names a record, as a new book whose title
|
||||
another book holds, with a `by_position` note listing those checks;
|
||||
never a value that is a uid the app generates (a
|
||||
GUID, or 32 hex digits as TreeLine's uuid1), nor a check the seed already
|
||||
passes, as a new record's `format` where another record of that kind held
|
||||
its place; a removed record is checked gone by its own pointer, as
|
||||
`/lists/[id=chores]`, and only where the end holds nothing there, so a
|
||||
record only position reached, whose place another took, is left to the
|
||||
count; `member_count` checks of the
|
||||
arrays whose members were added or removed, of the members' kind where
|
||||
records share a `type`, `kind`, `format` or `category` field, as `/nodes`
|
||||
matching `[format=BOOK]`, and a count guard of the innermost array an edit
|
||||
went through, as for XML; SQLite tables gain rows added,
|
||||
changed and removed, by primary key, and their new row count. An XML file,
|
||||
and the JSON, key file and XML views of a zip archive's entries, gain checks
|
||||
by [selectors](#pointers-and-selectors) and never through an element by
|
||||
position: in XML, first [`text_equals`](#whole-text) for each element whose
|
||||
text changed, then an element added whole that it exists, one removed whole
|
||||
that it is absent, then values changed, added and removed, an entry or a
|
||||
JSON value added whole through its members, and an empty array or object as
|
||||
it is. The text check names the paragraph rather than a run inside it: it
|
||||
goes up from an element sitting in its parent's text, and from one named by
|
||||
its tag or position alone to the nearest ancestor named by an attribute, as
|
||||
AbiWord's `p` by its `xid`; the text nodes and runs inside it get no other
|
||||
check, but an element holding no text that is added or removed there, such
|
||||
as an image or a line break, keeps its own, since the text check never
|
||||
reads it. Whitespace an app lays out between elements is not checked, and
|
||||
values that look like a time the app stamps, in ISO 8601 or as C's
|
||||
`asctime` writes it (`Thu Nov 5 09:20:31 2026`), are left out.
|
||||
text changed, then elements added or removed whole, then values changed,
|
||||
added and removed, an entry or a JSON value added whole through its
|
||||
members, and an empty array or object as it is.
|
||||
- A selector names a member by its tag, then by its identifiers (`name`,
|
||||
`id`, `xid`, `xml:id`), then by one other attribute, a name-like key such
|
||||
as Scribus's `ANNAME` or `inkscape:label`, a worded value or an integer of
|
||||
up to five digits first, then by two such attributes together, as
|
||||
Gnumeric's cells by `[@Col=3][@Row=2]`, then by a coordinate, a long
|
||||
number, a length with its unit, as `svg:height="2.629cm"`, an `xml:id`
|
||||
or `draw:id` of the form `id2` that LibreOffice numbers anew as it saves,
|
||||
so a Draw shape is named by its `draw:name`, or a token,
|
||||
such as `XPOS` or `ItemID`, or an id of random case, as Umbrello's
|
||||
`ubcgF2AWfBmHh` (`u` and twelve letters and digits), an automatic style name an app numbers as it
|
||||
saves and renumbers on reload, as LibreOffice's
|
||||
`presentation:style-name="pr1"` (`P1`, `T1`, `MT1`, `L2` on a
|
||||
`*:style-name`), so a slide's frames are named by `presentation:class`,
|
||||
or an attribute naming
|
||||
view state, such as which layer is `selected`, and last by a child
|
||||
element's text, `[>CHILD=VALUE]`, a name-like tag first, as Osmo's
|
||||
record by `[>first_name=Delphine]`, MuseScore's `<Text>` by
|
||||
`[>style=Title]`, a spreadsheet row by its label cell, or GnuCash's
|
||||
transaction by `[>trn:description=Greenleaf Market]`: text of at most 64
|
||||
characters, never a time or a token or id a program generates; then by
|
||||
an identifier a child carries, `[>CHILD@ATTR=VALUE]`, as a GtkBuilder
|
||||
`<child>` by `[>object@id=submit_button]`, never one an app generates or
|
||||
numbers anew as it saves; and then by text a child joins from elements
|
||||
of its own, as Glade's `[>packing=TrueTrue1]` from a widget's packing
|
||||
properties, which moving the widget changes. A pair
|
||||
counts every member holding both its values, whichever attributes that
|
||||
member ranks first. An element with identifiers is named by those alone;
|
||||
an id the app generates, as Freeplane's `ID_329023692`, or a GUID in
|
||||
any attribute, braced or not, as VYM's `uuid="{18eadb5d-…}"`, ranks
|
||||
last, after a child's text, so a new node is named by its `TEXT` and a
|
||||
new VYM branch by `[>heading=Sunset boat tour]`, while a member
|
||||
both versions hold pairs by such an id first, so records that keep their
|
||||
UUIDs and swap titles read as two title edits. A value that stands for
|
||||
none, as Grisbi's `Id="(null)"` on every transaction, names nothing, so a
|
||||
new transaction is named by its `Nb`. A child's text is
|
||||
counted on every sibling, one an identifier names included, so a title
|
||||
two records share names neither. The selector must name the
|
||||
member alone at both ends, with the same values; a child's text names a
|
||||
member that is in one version only when the list's length changed, so a
|
||||
row whose one cell an edit changed stays that row. An element whose one
|
||||
identifier changed, at its place among its tag's members, with every
|
||||
other attribute and its children alike, is that element renamed, as a
|
||||
MyPaint layer renamed from `Layer` to `Sun`: it is paired by its first
|
||||
other attribute naming it alone at both ends, as
|
||||
`[@src=data/sun.png]`, so the rename reads as its `name` changed, and
|
||||
the old name gets a check that it names nothing, unless its change is
|
||||
noise, as an id the app renumbered. Where that attribute's values number
|
||||
the members in document order, as MyPaint 2 writes `src` as
|
||||
`data/layer-00.png`, `data/layer-01.png` and on along the stack and
|
||||
renumbers them when a layer moves, each version is named by its own
|
||||
value of the renamed identifier instead, `[@name=Sun]` and
|
||||
`[@name=Layer]`.
|
||||
- A child of the document's root that its tag alone names, and that holds
|
||||
a list (two or more elements of one tag and no text of its own) and has
|
||||
an identifier, also gets a condition that keeps it named if the app adds
|
||||
a sibling of its tag, as Dia adds an empty `Background` layer each time
|
||||
it reopens and saves a diagram: an id other than `name` both ends hold,
|
||||
as Inkscape's `[name=g][@id=layer1]`, which a new sibling would not
|
||||
share; else the text of a child both ends hold, as Dia's
|
||||
`[name=dia:layer][>dia:object=#Receive order#]`, which an empty sibling
|
||||
lacks; else its `name`, which a new one may repeat. Deeper, a member of
|
||||
a list named for its tag, as Gnumeric's `gnm:Sheet` in `gnm:Sheets`,
|
||||
holding elements and its tag alone naming it, gets an id, else a
|
||||
name-like child's text, `[name=gnm:Sheet][>gnm:Name=Budget]`, else its
|
||||
`name`. Other steps, and elements with neither, stay as they are.
|
||||
An identifier the app numbers by position names nothing: where the
|
||||
members of a tag hold it as 1..N (or 0..N-1) in document order at both
|
||||
ends, members were inserted or removed, and every member another
|
||||
attribute names alone at both ends kept its number or moved it the way
|
||||
the count did, one at least, as Planner's task `id` after a task is
|
||||
inserted, members are paired by their other attributes, so an insertion
|
||||
reads as one task added. A value two members swap, or an id appended at
|
||||
the end, as Tellico's next `<entry id="4">`, leaves the ids identity. An
|
||||
identifier numbering the members 0..N-1 in document order at both ends,
|
||||
where members came or went, as Parley numbers its `<entry id="6">` by
|
||||
where it saves it, ranks after a child's text where a member has one, so
|
||||
a new entry is `[>translation=library]` and its kept neighbours are named
|
||||
alike; Tellico's ids, which count from 1, Tiled's and Rosegarden's,
|
||||
whose members hold no text, stay identity. An element a change added that
|
||||
holds nothing but the id of a record it added elsewhere, as Parley's
|
||||
lesson membership `<entry id="6"/>`, is `upkeep`, listed with `refers_to`
|
||||
on that record, never a check or a count.
|
||||
- The text check names the paragraph rather than a run inside it: it goes up
|
||||
from an element sitting in its parent's text, and from one named by its
|
||||
tag or position alone to the nearest ancestor named by an attribute, as
|
||||
AbiWord's `p` by its `xid`, unless it is a field its tag names in a
|
||||
record of two or more fields holding text, as QElectroTech's `<text>`
|
||||
beside `<info_name>` in a `<dynamic_elmt_text>`, or KmPlot's
|
||||
`<equation-0>` beside `<arg-min>`, which is checked itself rather than
|
||||
the record whose text joins its fields; one line of text the app lays
|
||||
out between line breaks, as KmPlot's formula before the next element, is
|
||||
checked, and its siblings guarded, with `normalize_whitespace: true`,
|
||||
and text whose whitespace alone changed, as a formula the app moved to
|
||||
an indented line of its own, gets no check; where no attribute names one, to the nearest
|
||||
ancestor within three levels its tag names with no step by position, as
|
||||
MuseScore's `VBox` around untitled `<Text>` elements; and where only
|
||||
position reaches the element, to the ancestor above the outermost step by
|
||||
position, by an attribute of its own that names it alone where one does,
|
||||
as an ODS sheet's `table:table[@table:name=Q1 Sales]` above its rows. The
|
||||
text nodes and runs inside it get no other check, but an element holding
|
||||
no text that is added or removed there, such as an image or a line break,
|
||||
keeps its own, since the text check never reads it. A decimal that is an
|
||||
element's only text, such as a spreadsheet cell's, is checked by its
|
||||
number instead. Free text that gains a line, such as a shopping list,
|
||||
uses `text_equals` on the whole text: the current XML checks cannot
|
||||
compare an unstructured line as a whole item. This can reject a correct
|
||||
edit that puts the item elsewhere or changes spacing. Text an app keeps in an attribute, one element per run, as Scribus's
|
||||
`<StoryText><ITEXT CH="Spring "/><ITEXT CH="Fair"/>`, gets a `text_equals`
|
||||
with `text_attribute` on the nearest ancestor an attribute names, as
|
||||
`[name=PAGEOBJECT][@ANNAME=Headline]` with `"text_attribute": "CH"`, where
|
||||
that ancestor holds no text nodes and the attribute's holders are runs,
|
||||
none inside another and none carrying an identifier or a name-like
|
||||
attribute beside it, as Umbrello's `<ownedAttribute xmi:id="…"
|
||||
name="email" visibility="private"/>`, a record whose attributes are
|
||||
checked one by one, two or more or one holding words with a space: an
|
||||
element's own attribute, as a Freeplane node's `TEXT` beside the nodes in
|
||||
it, or a lone value, as `<Bool value="false"/>`, is checked where it is.
|
||||
The check covers that attribute and the runs holding it; another change
|
||||
in the frame, as a run's `FONT`, keeps its own check. A record added
|
||||
or removed whole, an element holding only elements, its fields, such as
|
||||
Tellico's `<entry>` or a record of repeated `<field>` elements, never
|
||||
makes the list holding records of its kind a check, whose text would hold
|
||||
the records the edit left alone, except a record removed whole that only
|
||||
position names, whose absence only the list's text shows. Text made of
|
||||
runs is no record: an element whose tag names a paragraph or a piece of
|
||||
text (`p`, Writer's `text:p` and `text:h`, `h1` to `h6`, `li`, `para`,
|
||||
SVG's `text` and `tspan`, a `span` or an `a`), one inside such an
|
||||
element, as Word's `<w:r>` run in its `<w:p>`, one whose elements are all
|
||||
inline formatting (`b`, `i`, `u`, `em`, `strong`, `bold`, `italic`, a
|
||||
`span`, `a`, `code`, `sub`, `sup` and the like, Gnote's `size:` and
|
||||
`link:` runs), as `<list-item><bold>olive oil</bold></list-item>`, or
|
||||
whose runs split a sentence at a space, as `Meet ` and a linked
|
||||
`Monday`. Nor does an element holding a date the app stamps in parts
|
||||
take a text check.
|
||||
|
||||
- An element added to a list, where another element of its tag is at either
|
||||
end, gets a `value_equals` for each of its attributes, best ranked first,
|
||||
at most six, leaving out its selector's attributes, identifiers, times,
|
||||
volatile values, view state and values standing for none, as HomeBank's
|
||||
new `<ope>` gets its `amount`, `category` and `payee`; a value two or
|
||||
more like siblings all hold, as Grisbi's `Cu="1"`, comes after those
|
||||
that vary, as its `Pa`, `Ca`, `Dt` and `Am`. Records of attributes
|
||||
alone that it nests, as a KMyMoney `<TRANSACTION>` its `<SPLITS>` of
|
||||
`<SPLIT>`, get a `value_equals` of each attribute whose values vary among
|
||||
the like records, its own and those under its like siblings, at most
|
||||
three a record and six in all, as each split's `value` and `payee`, but
|
||||
not one every split holds alike, as `price="1/1"`, nor one each holds
|
||||
equal to an earlier one, as `shares` repeats `value`. Each record is
|
||||
named by an identifier that the app neither generates nor numbers anew
|
||||
in every record holding such a list, as KMyMoney's `S0001` in every
|
||||
transaction, else by its best ranked attribute short of a coordinate or
|
||||
an amount, as `[name=SPLIT][@account=A000001]`. An element whose other attributes, two
|
||||
or more, a like sibling holds every one of, as a new Planner task copies
|
||||
its neighbour's defaults, and any other element added whole, get a check
|
||||
that it exists; of those copied, an attribute naming an element the seed
|
||||
holds by an id that is no plain number, as an Umbrello attribute's
|
||||
`type` its data type, is the user's choice and keeps its check. A record added to a list that a selector names, as
|
||||
Tellico's `<entry id="4">` or Osmo's `<record>` by its first name, gets
|
||||
a `text_equals` for each field, at most five, each by its selector among
|
||||
the record's fields, as `[name=title]`, leaving out a field repeating the
|
||||
record's identifier, the field its selector names it by unless it is
|
||||
the only one left, as a new VYM branch's `<heading>`, a value every
|
||||
like record holds in that field, as Osmo's `<group>None</group>`, noise,
|
||||
a field holding ids of its own, a date the app stamps in parts, and blank
|
||||
text or times; a field that lists records, as GnuCash's `<trn:splits>`,
|
||||
gets its records' fields instead, each record by its own selector. An
|
||||
attribute every like record holds alike, as GnuCash's `version`, is the
|
||||
format's and gets no check. A record only position names also gets a
|
||||
`text_contains` for each new field of three or more characters, at most
|
||||
five, at the nearest ancestor a check can name, each absent from that
|
||||
ancestor's text at the seed.
|
||||
- Where members of a list were added or removed, a `member_count` of the
|
||||
list's members of their tag, and of their kind where they share one
|
||||
(`type`, `kind` or `shape`, as `[name=dia:object][@type=Flowchart - Box]`),
|
||||
unless every member of the tag holds a value of its own, as Tiled's
|
||||
objects a free-form `type` (`well`, `door`, `gate`), which labels each
|
||||
rather than grouping them,
|
||||
equal to the end's number, which differs from the seed's: at most two a
|
||||
file, the shallowest lists first, after the other checks. None for a
|
||||
list of one member at both ends, as a style an app adds for a run, for
|
||||
members of an element a text check reads, which are runs of its text,
|
||||
for an element in mixed content, or for noise.
|
||||
- A decimal written as text, such as `146.83`, is suggested as a number
|
||||
with `tolerance: 0.001`, unless the seed's value, read as the evaluator
|
||||
reads a number (`10` and `1e1` included), is that close too. A
|
||||
`value_equals` that would pass on the seed's value is not suggested.
|
||||
- An element placed by its own position, holding numeric `x` and `y` as a
|
||||
like sibling does, as Xournal++ places each `<text>` note on a layer, is
|
||||
an item of its own, never a run of the others' text: one added whole
|
||||
gets a `member_count` matching its whole text with whitespace collapsed,
|
||||
which passes wherever it is placed, and that list's `member_count`.
|
||||
- Noise leaves the suggestions when the file's change holds anything
|
||||
else, a change only position reaches included: view state, which a tag
|
||||
or attribute on the way names (the cursor, a selection, as VYM's
|
||||
`<select>`, the layer, frame,
|
||||
view or colour the app has current, as Pencil2D's `currentLayer`,
|
||||
scrolling, zoom, the viewport, windows, splitters, a list or tree view
|
||||
mirroring the document, as Umbrello's `listview`, Inkscape's `namedview`, view and
|
||||
configuration settings, the printer, a save's history, edit time,
|
||||
editing duration and cycles, last-saved time, the generator and
|
||||
document statistics, and a preview or thumbnail the app renders, as
|
||||
Xournal++'s `<preview>`), or the value of a name-like attribute or `key` on
|
||||
the way, when it is one token of several words, as LibreOffice's
|
||||
`config:config-item[@config:name=CursorPositionX]` inside
|
||||
`[@config:name=ooo:view-settings]`, or a settings path ending in how a
|
||||
panel shows (`visible`, `hidden`, `expanded`, `geometry`, `size` and the
|
||||
like), as VYM's `<setting key="/treeeditor/visible">`; an identifier the
|
||||
app numbers by position. An `Id` suffix alone does not show that an
|
||||
attribute is generated, so relationship fields such as `accountId` are
|
||||
retained; other save-generated IDs may need manual review; volatile
|
||||
values, UUIDs and epoch times, binary data encoded as base64 (a PNG's
|
||||
`iVBORw0KGgo`, or 256 or more base64 characters mixing cases and
|
||||
digits), an id counter (`uniqueid`, `nextobjectid`), a value equal to an
|
||||
id the change gave a new element where a counter or that element holds
|
||||
it, as Umbrello's `lastobject` holds the new attribute's `xmi:id`, but
|
||||
never a reference on an element both ends hold, as a KMyMoney split
|
||||
moved to a new account, and durations and counts under a document's metadata; times the app stamps, also as a date written in
|
||||
parts whose tag, or one it sits in, names when the app made, changed or
|
||||
saved something, as Tellico's `<cdate>` of `<year>`, `<month>` and
|
||||
`<day>` (`cdate`, `mdate`, `created`, `modified`, `saved`, `updated`,
|
||||
`entered`), or written whole under such a tag, as GnuCash's
|
||||
`<trn:date-entered><ts:date>2026-11-06 08:53:08 +0000`,
|
||||
which no check ever reads, while a date a user enters in parts, as a
|
||||
birthday, is an edit like any other; an id the app assigns, an `id`
|
||||
element with `type="guid"`, as GnuCash's `<trn:id>`, and text of control
|
||||
characters, as its unset options hold; `upkeep`, a change inside an
|
||||
app's `slots`, as GnuCash's `book:slots`, which it adds to and fills on
|
||||
reopening a book, a count an app keeps of its members, an attribute
|
||||
whose key ends in `count` or an element whose tag names a count, whose
|
||||
integer changed and at each end equals how many elements of one tag it,
|
||||
or the element holding the text, holds there, the counter left out, as
|
||||
VYM's root `branchCount`, KMyMoney's `<TRANSACTIONS count="5">` or
|
||||
GnuCash's `<gnc:count-data>`, or, for a counter the change added, how
|
||||
many of one tag, more than one, it holds as children, as FreeCAD's
|
||||
`<ObjectDeps Count="2">`, while a count a user enters, as an item's
|
||||
`<count>`, is an edit, and the `loext:tab-stop-distance` attributes
|
||||
LibreOffice fills in `styles.xml` while another entry holds the edit,
|
||||
and, while a change outside them is the edit, any
|
||||
change in `styles.xml`, in a document's style containers or its master,
|
||||
handout and notes pages, or to an attribute naming an automatic style
|
||||
(`P1`, `MT1`) at both ends, as Impress after a reload wraps its master pages' fields in a new `MT1` style and
|
||||
rewrites list style `L2`, and a text showing another field that changed
|
||||
alike, its element beside a field naming that one's key, as
|
||||
QElectroTech's `<dynamic_elmt_text>` shows a component's label in its
|
||||
`<text>` beside `<info_name>label</info_name>`; and `layout`, attributes holding a list of
|
||||
numbers, as Dia's `obj_bb` `6.3225,6.15;12.5775,8.15`, or a number a
|
||||
name on its way calls layout (a position, point, box, corner, width,
|
||||
height, side or offset, as `elem_width`), that changed in an element up
|
||||
to three levels below one whose text changed, below the root's own
|
||||
children, while their own element's text did not, as Dia moves and
|
||||
resizes a box around its new text, or below an element whose tag names a
|
||||
diagram, widget, canvas or scene while the model change is elsewhere, as
|
||||
Umbrello grows a class's `classwidget` `height` when an attribute is
|
||||
added; an order's `<amount value="12"/>` beside its new description is
|
||||
content. Geometry that changed with
|
||||
no text beside it, as a CAD sketch's, is content. However much noise
|
||||
comes first, it never takes a check's place. When the change holds
|
||||
nothing else, the checks stay and a note flags them: up to three in a
|
||||
document, and as many as fit in the app's own state, where a view
|
||||
setting may be the task.
|
||||
- Whitespace an app lays out between elements is not checked, and values
|
||||
that look like a time the app stamps, in ISO 8601, as C's `asctime`
|
||||
writes it (`Thu Nov 5 09:20:31 2026`) or as a time of day to the
|
||||
second (`05:16:21.675302540`), are left out; noise never makes its
|
||||
ancestor's whole text a check.
|
||||
- `guards`: checks an author may add to show the edit left the rest alone,
|
||||
kept out of `task.checks`, with ids `guard-1` and on: for each suggested
|
||||
`value_equals` or `text_equals` on a member of an XML list, or a
|
||||
`value_equals` through a JSON record added whole, as TreeLine's new book's
|
||||
`data/Author` gives
|
||||
`/nodes/[format=BOOK][>data~1Title=Small Engines of Life]/data/Author`
|
||||
(its like records are those of its kind, named as both ends pair them,
|
||||
and it reads only the check's own field), a field on up to
|
||||
two like siblings, those sharing most of the attributes its selector
|
||||
names it by and its type, as a cell's `Row` or `Col` and `ValueType`,
|
||||
then the nearest first, at its seed value, which the end holds
|
||||
too, trying at most the 16 nearest, so a guard costs the same among
|
||||
thousands of siblings; for a member named as a field, by a name-like attribute alone, as
|
||||
QElectroTech's `elementInformation[@name=label]`, the like siblings of
|
||||
the nearest record out that hold that field too come first, so the other
|
||||
components' labels guard a component's label rather than its other
|
||||
fields: the check's own field where like siblings hold different values in
|
||||
it, and otherwise one that varies among them, as Planner's `work` rather
|
||||
than the `type="normal"` every task shares, but not one numbering the
|
||||
members in order, as MyPaint's `src`; never through a sibling only
|
||||
position pairs. A guard never selects its sibling by
|
||||
what it reads: a sibling its selector names by the attribute the guard
|
||||
checks, or by a value that is or holds the text it reads, as Glade's
|
||||
`[>object=CancelTrueTrueTrue]` for a button's `Cancel` label, is named by
|
||||
another identifier, name-like attribute or child's text it holds alone
|
||||
at both ends instead, as Tiled's objects by `[@id=2]` rather than the
|
||||
`name` the guard reads, or Glade's button by `[>packing=TrueTrue0]`, or
|
||||
else the guard reads another field, or there is none. A `text_equals` with `text_attribute`
|
||||
reads its siblings the same way. A text file gets a `lines_present` of the
|
||||
nearest two unchanged lines of its first new line's kind, list items with the same
|
||||
marker, as Zim's other `[ ]` action items, or lines the same command
|
||||
opens, as the other headings after LyX's `\begin_layout Section`, or lines
|
||||
of the same keyword or `key:` in other records, as a gettext catalogue's
|
||||
other `msgstr` lines rather than the edited entry's own `msgid`, or else
|
||||
its nearest lines with words in them; each text both ends hold once, never
|
||||
a line with no text of its own, a command such as `\end_layout`, tags
|
||||
alone such as `<fonts>` or `</node>`, or a key alone such as a RedNotebook
|
||||
day's `24:`, nor a line that repeats; an item placed by its own position
|
||||
that an edit added, a `member_count` matching each of up to four kept items'
|
||||
text, nearest first. Where members were only edited, a
|
||||
`member_count` of the innermost list holding the edited element that an
|
||||
edit goes through by a member with like siblings, a record rather than a
|
||||
field of one (a member named by a name-like attribute alone that the
|
||||
record's like siblings hold too), of the edited member's tag and kind, at
|
||||
the seed's number, as Dia's boxes in their layer, MyPaint's layers in
|
||||
their stack or Glade's buttons in their button box: no member of the kind
|
||||
added or removed; never of runs in an element a text check reads, and
|
||||
only where the end holds the same number. A guard of a coordinate comes
|
||||
after those of text. At most four a file besides the count guard, which
|
||||
sits outside that cap, so sibling guards never squeeze it out. They pass
|
||||
on the seed, which the export allows, since one check failing there is enough.
|
||||
- `notes`: why `files` is empty, or why a file's suggestions are thin. Each
|
||||
note has a `code` and a `message`. For a draft with no files: `no_change`
|
||||
when no file under the home changed between the two checkpoints and nor
|
||||
did the application's dconf keys; `unowned_changes` when files changed but
|
||||
none under the owned paths, with up to five of them in `files`, each with
|
||||
its home path and role (desktop services' files last), and how many more
|
||||
in `more`; `uncheckable_changes` when the application's keys in a dconf
|
||||
database changed, with the database in `files`: no check reads dconf, and
|
||||
the export only writes the keys as they were at the seed checkpoint, so
|
||||
make the task's change one the app saves in a file; `unreadable_changes`
|
||||
when a dconf database changed but REA could not read it, with the read
|
||||
failure in `error` (install `dconf-cli` or set `REA_DCONF_COMMAND` when it
|
||||
is missing), since whose keys changed is then unknown; and `no_document`
|
||||
when no file has the document role, as when no window title shows the
|
||||
file's name. A capture again with `--state-path` naming the file, or
|
||||
`state_paths` on a build computer's `start_build_capture`, makes it the
|
||||
app's. For a drafted file, named in `file`: `by_position` when
|
||||
changes reach their element only by position, the tags no attribute names
|
||||
alone in `elements`, or when only the elements' order changed;
|
||||
`view_state_only` when every change is noise, the checks kept in
|
||||
`pointers`; `long_text` when an element's text changed past the 2,000
|
||||
characters a text check compares, the elements in `pointers`; `cut_view`
|
||||
when the views hold only part of the file; `unreadable_view` when no
|
||||
check can read it, as a document REA reads as binary;
|
||||
`volatile_lines` when a text file gained lines holding a value the app
|
||||
generates with no text before it to count them by; `generated_reference`
|
||||
when an array references a newly generated record ID that cannot be checked
|
||||
exactly across runs, with up to five affected `pointers`; and `read_as_text`
|
||||
when a text file opens like XML, with a declaration, a DOCTYPE or an
|
||||
element, but reads as text because its DOCTYPE has an internal subset or
|
||||
it is not well formed, so its checks compare lines that hold attributes in
|
||||
the order the app wrote them.
|
||||
|
||||
The suggestions are a starting point. Keep the ones that say what the user
|
||||
did; drop the ones that only follow from how the app saves its file. A LibreCAD
|
||||
did; drop the ones that only follow from how the app saves its file. Read
|
||||
`values` for what changed and its pointers, and `refers_to` for what a number
|
||||
stands for, before writing the prompt and any check of your own. A LibreCAD
|
||||
capture that added four holes to a drawing rewrote it from DXF R12 to DXF 2007,
|
||||
so most of its lines changed; the draft suggests counts of `SECTION`, `ENDSEC`
|
||||
and `CIRCLE`, and only `CIRCLE` going from 5 to 9 is the user's change.
|
||||
@@ -80,7 +503,7 @@ and `CIRCLE`, and only `CIRCLE` going from 5 to 9 is the user's change.
|
||||
```json
|
||||
{
|
||||
"name": "add-milk",
|
||||
"prompt": "Add buy milk to my to-do list in Mousepad and save it.",
|
||||
"prompt": "In Mousepad, add buy milk to my to-do list and save it.",
|
||||
"checks": [
|
||||
{
|
||||
"id": "milk",
|
||||
@@ -93,31 +516,35 @@ and `CIRCLE`, and only `CIRCLE` going from 5 to 9 is the user's change.
|
||||
```
|
||||
|
||||
`name` is a portable bundle name other than `smoke` (lowercase letters, digits
|
||||
and hyphens); `prompt` is what the user asks for; `checks` holds 1 to 32
|
||||
checks. Each check has a `kind`, a `file` given as a path under the home such as
|
||||
and hyphens); `prompt` is what the user asks for, naming the app, since a
|
||||
solving agent's own shell cannot see the computer's files and the app's name
|
||||
sends it to the screen; `checks` holds 1 to 32
|
||||
checks. Each file check has a `kind`, a `file` given as a path under the home such as
|
||||
`Documents/todo.txt`, and an optional `id` (`check-1`, `check-2` and so on by
|
||||
position otherwise). A check reads the typed view `data/get` returns for that
|
||||
file, in the shapes of the state view.
|
||||
|
||||
| Kind | Fields | Passes when |
|
||||
| --------------- | ------------------------------------------------------------ | ------------------------------------------------------------------------ |
|
||||
| `file_present` | | The file exists |
|
||||
| `file_absent` | | The file does not exist |
|
||||
| `lines_present` | `lines` | Every listed line is a whole line of the text view |
|
||||
| `lines_absent` | `lines` | None of the listed lines is |
|
||||
| `line_count` | `exact` or `contains`; `equals`, `at_least` or `at_most` | The number of lines equal to `exact`, or holding `contains`, compares so |
|
||||
| `value_equals` | `pointer`, `value`; optional `tolerance` | The pointer names a value equal to `value`, or within `tolerance` of it |
|
||||
| `value_absent` | `pointer` | The pointer names no value |
|
||||
| `text_equals` | `pointer`, `value`; optional `normalize_whitespace` | The XML element's [whole text](#whole-text) equals `value` |
|
||||
| `text_contains` | `pointer`, `value`; optional `normalize_whitespace` | The XML element's whole text holds `value` |
|
||||
| `row_present` | `table`, `where` | A row of the SQLite table has every column in `where` equal |
|
||||
| `row_absent` | `table`, `where` | No row does |
|
||||
| `row_count` | `table`; optional `where`; `equals`, `at_least` or `at_most` | The number of rows, or of rows matching `where`, compares so |
|
||||
| Kind | Fields | Passes when |
|
||||
| --------------- | --------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
|
||||
| `file_present` | | The file exists |
|
||||
| `file_absent` | | The file does not exist |
|
||||
| `lines_present` | `lines` | Every listed line is a whole line of the text view |
|
||||
| `lines_absent` | `lines` | None of the listed lines is |
|
||||
| `line_count` | `exact` or `contains`; `equals`, `at_least` or `at_most` | The number of lines equal to `exact`, or holding `contains`, compares so |
|
||||
| `value_equals` | `pointer`, `value`; optional `tolerance` | The pointer names a value equal to `value`, or within `tolerance` of it |
|
||||
| `value_absent` | `pointer` | The pointer names no value |
|
||||
| `text_equals` | `pointer`, `value`; optional `normalize_whitespace`, `text_attribute` | The XML element's [whole text](#whole-text) equals `value` |
|
||||
| `text_contains` | `pointer`, `value`; optional `normalize_whitespace`, `text_attribute` | The XML element's whole text holds `value` |
|
||||
| `member_count` | `pointer`; optional `matching`; `equals`, `at_least` or `at_most` | The number of [members of the array](#counting-members), or of those `matching` names, compares so |
|
||||
| `row_present` | `table`, `where` | A row of the SQLite table has every column in `where` equal |
|
||||
| `row_absent` | `table`, `where` | No row does |
|
||||
| `row_count` | `table`; optional `where`; `equals`, `at_least` or `at_most` | The number of rows, or of rows matching `where`, compares so |
|
||||
|
||||
The rules both evaluators follow:
|
||||
|
||||
- An absent file reads as empty: no lines, values or rows. So `lines_absent`,
|
||||
`value_absent` and `row_absent` pass on it and a count is 0.
|
||||
`value_absent` and `row_absent` pass on it and a count of lines or rows is
|
||||
0; `member_count` fails, since its pointer names no array.
|
||||
- A file that has no view (binary, over the read limit, or unreadable) fails
|
||||
every check but `file_present` and `file_absent`. Line checks need a text
|
||||
view, row checks a SQLite view and text checks an XML view, in a file or an
|
||||
@@ -129,7 +556,9 @@ The rules both evaluators follow:
|
||||
`{"groups": {"GROUP": {"KEY": "VALUE"}}}`, so a setting is
|
||||
`/groups/GROUP/KEY`; XML as `{"root": {"name", "attributes", "children"}}`;
|
||||
SQLite as `{"tables": {"TABLE": {"columns", "rows"}}}`; a zip archive as
|
||||
`{"entries": [{"name", "format", "view", …}]}`.
|
||||
`{"entries": [{"name", "format", "view", …}]}`; a gzip file, such as a
|
||||
Gnumeric `.gnumeric`, as the document inside it, so its pointers are those
|
||||
of the same document uncompressed.
|
||||
- With `tolerance`, a non-negative number, `value` must be a number, and the
|
||||
value found passes when it is a number, or a string that is a decimal
|
||||
numeral such as `"5.0000000000000000"`, at most `tolerance` from `value`.
|
||||
@@ -141,20 +570,106 @@ The rules both evaluators follow:
|
||||
- A check decided by what a view leaves out (`lines_absent`, `value_absent`,
|
||||
`row_absent`, the counts, `value_equals` of a whole object or array, and
|
||||
the text checks of an element) fails when that view is truncated; a check
|
||||
satisfied by what the view holds is decided on it.
|
||||
satisfied by what the view holds is decided on it. An element's text is not:
|
||||
the reader may have dropped text from between the runs it kept, so
|
||||
`text_contains` fails on a truncated view even where the kept runs hold its
|
||||
value. A text node or attribute value is whole, and decided.
|
||||
|
||||
The export refuses, naming the check, a file outside the owned paths, a file
|
||||
spelled with a leading slash, `.` or `..`, a pointer into a file that is text
|
||||
at the seed, as the bundle ships it, or at the end checkpoint, the empty
|
||||
pointer (the whole file), a value that names the captured home, or the
|
||||
pointer (the whole file) except in `member_count`, a `matching` that is no
|
||||
selector, a value that names the captured home, or the
|
||||
capture host's name or `user@host` anywhere (which the
|
||||
[export removes from the seed](desktop-application-export.md#host-name-and-user)), a
|
||||
`tolerance` with a value that is not a number, an empty `text_contains`
|
||||
value, or one of only whitespace with `normalize_whitespace`, and a value that
|
||||
looks like a date and time, in ISO 8601 or `asctime`'s form. Other volatile
|
||||
value, or one of only whitespace with `normalize_whitespace`, an empty
|
||||
`text_attribute`, a count without exactly one of `equals`, `at_least` and
|
||||
`at_most`, and a value that looks like a date and time, in ISO 8601 or
|
||||
`asctime`'s form. Other volatile
|
||||
content, such as a
|
||||
counter the app bumps on every save, is for the author to leave out.
|
||||
|
||||
## Readers
|
||||
|
||||
For a file REA cannot interpret itself, such as an app's binary document,
|
||||
the task can define `readers`: 1 to 8 commands using the app's own reading
|
||||
capability. Each reader has a unique `id` of lowercase letters, digits and
|
||||
hyphens, a home-relative `file`, an `argv` array and `output`, either `json`
|
||||
or `lines`. Its `argv`, or optional `stdin` text, must contain `{file}`;
|
||||
the runner replaces it with the path of a private copy of that file.
|
||||
The command must be installed where the export runs and in the exported
|
||||
computer where the task will be graded.
|
||||
|
||||
The command prints its answer in lines starting with `REA `; other stdout
|
||||
is ignored. With `output: "json"`, those marked lines join into one JSON
|
||||
value for `value_equals`, `value_absent` and `member_count`. With
|
||||
`output: "lines"`, they supply the lines for `lines_present`, `lines_absent`
|
||||
and `line_count`. A check selects a reader with `"reader": "READER-ID"`
|
||||
and must give that reader's file as its `file`. Every reader must be used by
|
||||
at least one check.
|
||||
|
||||
Readers execute commands, including any code in `argv` or `stdin`; review
|
||||
them before exporting or running a task. The runner uses a private copy, a
|
||||
fresh home, no session bus or computer control token, and no display unless
|
||||
the reader requests `display: true`. This is execution isolation from the
|
||||
running app's normal profile, not a filesystem sandbox. `timeout_s` defaults
|
||||
to 60 seconds and accepts 1 to 300. A failed command, timeout, malformed
|
||||
answer or missing marked output reports a reader error; it does not establish
|
||||
that the document lacks the expected value. Correct the command or its output and retry; increase
|
||||
`timeout_s` within its accepted range when the reader needs longer.
|
||||
|
||||
A source inspection failure, such as an inaccessible parent or a symlink loop,
|
||||
also reports a reader error and leaves affected checks undecided. Only a missing
|
||||
source or a source that is not a regular file is reported as absent.
|
||||
|
||||
Export runs readers on both retained ends, using the seed bytes as the bundle
|
||||
actually ships them. The witness and exported verifier run them in the
|
||||
computer after the app's files settle. Cleanup failures retain the reader's
|
||||
reported resources: retry the check or close the build computer to retry
|
||||
cleanup; when the command's outcome is unknown, confirm it stopped before
|
||||
removing its folder.
|
||||
|
||||
## Screen judges
|
||||
|
||||
A task whose result exists only on screen can use `screen_judge` checks:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "show-list",
|
||||
"prompt": "In Notes, show the shopping list.",
|
||||
"checks": [
|
||||
{
|
||||
"id": "shown",
|
||||
"kind": "screen_judge",
|
||||
"question": "Does the window show the shopping list?"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Each question must be a nonempty yes-or-no question of at most 2,000
|
||||
characters, false on the seed and true when the task is done. A screen task
|
||||
contains only screen checks and takes no readers. Export refuses it when an
|
||||
owned file's readable change could be checked instead, or when the build
|
||||
computer's seed or end screenshot is missing or exceeds 2 MiB. Capture those
|
||||
checkpoints in a build computer before exporting.
|
||||
|
||||
Export ships the two screenshots as labelled failing and passing examples;
|
||||
it does not call the judge or establish that the questions distinguish them.
|
||||
The [build witness](desktop-build-witness.md) runs the exported judge on a
|
||||
fresh screenshot at each stage. The verifier takes a screenshot after the
|
||||
solving agent stops. For each question, the judge sees both examples and
|
||||
votes three times; a majority of yes answers passes the check.
|
||||
|
||||
The judge uses `[model.roles] judge` in the runner's
|
||||
`.agentenv/config.toml`, then `[model] default`, then
|
||||
`anthropic/claude-sonnet-5-5`. Configure its endpoint with
|
||||
`LITELLM_BASE_URL` and `LITELLM_API_KEY`, or `[model] base_url` and `api_key`.
|
||||
`agent-env run --judge-model` does not reach this verifier. Without an
|
||||
endpoint, grading raises and the run fails rather than scoring the agent;
|
||||
configure the endpoint and rerun the witness or task.
|
||||
|
||||
## Pointers and selectors
|
||||
|
||||
An app that rewrites a document moves things: FreeCAD 1.0 records more
|
||||
@@ -168,9 +683,34 @@ rather than by its position:
|
||||
`[name=Object]`.
|
||||
- `[@KEY=VALUE]` names the member whose `attributes` hold `KEY="VALUE"`, as
|
||||
the XML view keeps an element's attributes.
|
||||
- `[>PATH=VALUE]` names the member that holds `VALUE` at a path of one to
|
||||
three `/`-separated steps inside it. `>` reads XML and JSON differently:
|
||||
|
||||
For XML, `[>text()=VALUE]` compares the member's own whole text, with
|
||||
whitespace collapsed, to `VALUE`. In JSON, `text()` remains a field name.
|
||||
|
||||
| Form | In an XML element | In a JSON record |
|
||||
| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- |
|
||||
| `[>CHILD=VALUE]` | a child element `CHILD` whose text, every text node in it with each run of whitespace collapsed to one space and none at either end, is `VALUE` | the string field `CHILD`, compared whole |
|
||||
| `[>A/B=VALUE]`, up to `[>A/B/C=VALUE]` | a child `A` with a child `B` of that text | the string at `member.A.B`, compared whole |
|
||||
| `[>A@ATTR=VALUE]`, `[>A/B@ATTR=VALUE]` | a child `A` (or its child `B`) whose attribute `ATTR` is `VALUE` | the field named `A@ATTR`: `@` is part of a JSON key |
|
||||
|
||||
Records that keep their fields as elements are named by a child's text:
|
||||
Osmo's `<contact><first_name>Delphine</first_name>…` is
|
||||
`[name=contact][>first_name=Delphine]`. A GtkBuilder `<child>` is named by
|
||||
the id its widget carries, `[name=child][>object@id=submit_button]`, and a
|
||||
TreeLine node by a field of its `data`,
|
||||
`[format=BOOK][>data/Title=The Silk Roads]`. Any element on the way will
|
||||
do; a path with more than three steps, or an empty one, is no selector;
|
||||
and a missing step, or a value at its end that is not a string, matches
|
||||
nothing.
|
||||
|
||||
- Conditions can follow one another, `[name=Object][@name=Hole]`, and all must
|
||||
hold. They compare strings only, the selector is escaped like any pointer
|
||||
segment (`~1` for `/`), and a value cannot hold `][`.
|
||||
segment, so a path's `/` is `~1` inside a pointer,
|
||||
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, and a `/` in a key
|
||||
cannot be a step; a value cannot hold `][`. A JSON field whose key begins
|
||||
with `@` or `>` is selected through a path, `[>@type=Book]`.
|
||||
|
||||
A selector must name exactly one member. When several match, or none does
|
||||
before the last segment, the check fails and says which selector matched what.
|
||||
@@ -184,7 +724,7 @@ FreeCAD 0.21 and 1.0 alike:
|
||||
```json
|
||||
{
|
||||
"name": "cut-hole",
|
||||
"prompt": "Cut a 10 mm hole through the middle of the plate and save it.",
|
||||
"prompt": "In FreeCAD, cut a 10 mm hole through the middle of the plate and save it.",
|
||||
"checks": [
|
||||
{
|
||||
"id": "cut",
|
||||
@@ -291,11 +831,56 @@ read the element's whole text instead:
|
||||
and trims both ends before comparing, as XPath's `normalize-space()` does,
|
||||
for an app that reflows text or a value written across lines. A no-break
|
||||
space is not whitespace there.
|
||||
- `text_attribute` reads text an app keeps in an attribute: the element's
|
||||
text is then that attribute's values on it and its descendants, joined in
|
||||
document order, and its text nodes are left out. Scribus keeps a text
|
||||
frame's text as `<StoryText><ITEXT CH="Spring "/><ITEXT CH="Fair"/></StoryText>`,
|
||||
one `ITEXT` per run of formatting, so the check below reads `Spring Fair`
|
||||
however the runs are split. The pointer must name an element; a text node
|
||||
or attribute value fails.
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "headline",
|
||||
"kind": "text_equals",
|
||||
"file": "Documents/flyer.sla",
|
||||
"pointer": "/root/children/[name=DOCUMENT]/children/[name=PAGEOBJECT][@ANNAME=Headline]",
|
||||
"value": "Spring Fair",
|
||||
"text_attribute": "CH"
|
||||
}
|
||||
```
|
||||
|
||||
## Counting members
|
||||
|
||||
`member_count` counts the members of the array its pointer names: a JSON
|
||||
array, or an XML element's `children`. `matching`, a selector of one or more
|
||||
conditions as [above](#pointers-and-selectors), `[>PATH=VALUE]` included,
|
||||
counts only the members it matches, however many; without it every member
|
||||
counts. An element's children hold its text nodes too, so give `matching`
|
||||
for XML, such as `[name=entry]`. A Tellico task that adds one book to a
|
||||
collection of 12:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "one-more-book",
|
||||
"kind": "member_count",
|
||||
"file": "Documents/books.tc",
|
||||
"pointer": "/entries/[name=tellico.xml]/view/root/children/[name=collection]/children",
|
||||
"matching": "[name=entry]",
|
||||
"equals": 13
|
||||
}
|
||||
```
|
||||
|
||||
It takes exactly one of `equals`, `at_least` and `at_most`. The empty
|
||||
pointer `""` names the whole file, for a JSON file whose top level is an
|
||||
array. A pointer that names no array, an absent file's included, fails and
|
||||
says what it found there. Like the other counts, it is undecided on a view
|
||||
that may hold only part of the array.
|
||||
|
||||
## Validation and export
|
||||
|
||||
`rea export-desktop-application <capture> <directory> --task <task.json>`
|
||||
evaluates the checks on the capture's state view: on each file as it was at
|
||||
evaluates file checks on the capture's state view: on each file as it was at
|
||||
the seed checkpoint, which is the seed, as the bundle ships it with the
|
||||
capture host's name and user removed, and at the end checkpoint, the
|
||||
captured end state. Those are the capture's first and last checkpoints unless
|
||||
@@ -304,9 +889,7 @@ It refuses a task whose every check passes on the seed, and a check that fails
|
||||
on the end state, with that check's reason. Only a failure on what the seed's
|
||||
view shows counts: a task whose checks fail on the seed only because a view
|
||||
could not be read or may have left out what they look for, such as a line past
|
||||
the 5,000 a text view keeps, is refused, naming each failure. A capture yields
|
||||
one task, so the option is taken once. The result's `task` names the task, its
|
||||
number of checks and those that fail on what the seed shows. A refused task
|
||||
the 5,000 a text view keeps, is refused, naming each failure. A refused task
|
||||
writes no bundle.
|
||||
|
||||
The bundle then holds, besides the app and `tasks/smoke.json`:
|
||||
@@ -315,17 +898,56 @@ The bundle then holds, besides the app and `tasks/smoke.json`:
|
||||
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| `tasks/<name>.json` | Deploy the app, deploy the agent and its skills, record the screen, prompt the agent, run the verifier, stop recording |
|
||||
| `artifacts/<name>-check/verify.py` | Reads the app's `data/get` through the computer and evaluates the checks, one result per check |
|
||||
| `evals/<env>.toml` | The task, so a run of the bundle runs it; the smoke task runs with `--task smoke` |
|
||||
| `evals/<env>.toml` | Every task, so a run of the bundle runs them all; the smoke task runs with `--task smoke` |
|
||||
|
||||
The verifier finds the app among the computer's child environments, calls its
|
||||
`data/get` under the run's grant of the computer's control token, and reads
|
||||
the `state` data part. It raises when the app cannot be reached, so the run
|
||||
fails rather than scoring the agent. The README lists the task, its prompt, its
|
||||
the `state` data part. It reads again until the app's files settle, as
|
||||
[grading once the app has saved](desktop-application-export.md#grading-once-the-app-has-saved)
|
||||
describes, and each result's `settled` says whether they did, and `read_cut`
|
||||
whether the bound cut a read still running. It raises when the app cannot be reached, so the run
|
||||
fails rather than scoring the agent. The README lists each task, its prompt, its
|
||||
checks and the recipe below.
|
||||
|
||||
### Several tasks
|
||||
|
||||
One capture can yield several tasks: repeat `--task`, one file per task, each
|
||||
with its own name. Do each task's change on top of the last one in the same
|
||||
capture and take a checkpoint once it is saved; the first task's end is the
|
||||
end checkpoint, and `--task-end NAME=CHECKPOINT`, repeatable, names another
|
||||
task's. Each task is checked at its own end, so a task whose check fails there
|
||||
is refused, naming the task and the checkpoint, even when it passes at the
|
||||
last one; its checks should not read what a later task changes. A checkpoint
|
||||
named `near-NAME`, when the capture took one, is that task's near miss: a
|
||||
plausible wrong edit on top of its change, such as a typo. The export refuses
|
||||
the task unless one of its checks fails on what the near miss's views show.
|
||||
|
||||
```sh
|
||||
rea export-desktop-application ./notes-capture ./notes-bundle \
|
||||
--seed-checkpoint seed --end-checkpoint end \
|
||||
--task add-eggs.json --task add-bread.json --task-end add-bread=end-add-bread
|
||||
```
|
||||
|
||||
Two tasks of one name, a `--task-end` naming none of the tasks or not in
|
||||
`NAME=CHECKPOINT` form, and a checkpoint the capture did not take once at or
|
||||
after the seed are refused. The result's `tasks` lists each task: its name,
|
||||
its number of checks, its end checkpoint, those that fail on what the seed
|
||||
shows, and its near miss's checkpoint, or null, with those that fail there.
|
||||
`describe-desktop-application-task` drafts a later task from the end before
|
||||
it: `--seed-checkpoint end --end-checkpoint end-add-bread`.
|
||||
|
||||
A task's readers run at the seed as the bundle ships it, at its own end and
|
||||
at its near miss. A screen-judged task ships the seed's screenshot and its own
|
||||
end's as the judge's examples, and is refused when a file shows its change
|
||||
since the end before it, or the seed for the first; its near miss is left to
|
||||
the witness, so its `near_checkpoint` is null.
|
||||
|
||||
## Solve rate
|
||||
|
||||
REA does not run agents. To measure how often an agent solves the task, run it
|
||||
REA's CLI runs no agent. On a build host, the MCP tool `measure_build_task`
|
||||
measures a bundle's task for a model, records it in the bundle's
|
||||
`difficulty.json`, and keeps the task in the band when it passes 1 to 3 of 5;
|
||||
see [measuring a task](desktop-build-witness.md#measure). By hand, run the task
|
||||
k times, three by default, and count the runs whose summary line says
|
||||
`passed`:
|
||||
|
||||
@@ -336,7 +958,10 @@ done
|
||||
```
|
||||
|
||||
The agent defaults to Claude Code and the model is the caller's choice (Claude
|
||||
Sonnet 5.5 is a reasonable reference). A run reported as `scored below 1`
|
||||
Sonnet 5.5 is a reasonable reference). The agent calls a model endpoint, and
|
||||
agentenv refuses the run without one: set `LITELLM_BASE_URL` and
|
||||
`LITELLM_API_KEY` where the runs start, or `base_url` and `api_key` under
|
||||
`[model]` in `.agentenv/config.toml`. A run reported as `scored below 1`
|
||||
failed the checks and counts as a miss; one reported as `failed` did not finish,
|
||||
as when the computer or the app could not be reached, and is run again rather
|
||||
than counted. Each run's instance keeps the verifier's results per check under
|
||||
|
||||
+189
-27
@@ -14,15 +14,70 @@ agent's computer-use actions to it through three MCP tools:
|
||||
The tools are MCP-only, like the debugger sessions: the computer lives as long
|
||||
as the REA MCP server that started it, or until it is closed. Capturing the
|
||||
app, exporting its bundle and witnessing the task on such a computer are in
|
||||
[Building and witnessing an environment](desktop-build-witness.md).
|
||||
[Building and witnessing an environment](desktop-build-witness.md), and REA's
|
||||
skill for doing all of it from a request is
|
||||
[desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md).
|
||||
|
||||
## Set up a build host
|
||||
|
||||
On a Linux host with Node.js and Docker, from a REA
|
||||
[checkout](getting-started/installation.md#run-rea-from-a-checkout):
|
||||
|
||||
```bash
|
||||
rea setup --client claude_code --desktop-build --install-agentenv
|
||||
rea doctor --provider desktop_build --client claude_code
|
||||
```
|
||||
|
||||
Setup shows its plan and asks before it changes anything; `--dry-run` only
|
||||
prints it and `--yes` approves it. It packs REA's runtime archive into
|
||||
`~/.cache/rea/build-computer/rea-<version>.tar.xz` (under `$XDG_CACHE_HOME`
|
||||
when set, `~/Library/Caches` on macOS), and packs again once REA's version
|
||||
changes; an archive `REA_BUILD_COMPUTER_REA_ARCHIVE` already names is
|
||||
recorded instead. It records `REA_AGENT_ENV_COMMAND`, `REA_DOCKER_COMMAND` and
|
||||
`REA_BUILD_COMPUTER_REA_ARCHIVE` as absolute paths in that client's REA
|
||||
registration, keeping a value the registration already has other than an
|
||||
archive setup packed for an earlier REA version, and refreshes the skill.
|
||||
`--install-agentenv` runs `uv tool install agentenv-framework==0.9.1485`
|
||||
when `uv` is on `PATH`, and otherwise prints the command. Setup never
|
||||
installs Docker or Node.js and never runs `sudo`. Restart the client
|
||||
afterwards.
|
||||
|
||||
Doctor is read-only and checks, in order: a Linux amd64 or arm64 host,
|
||||
Node.js, procps `ps`, the Docker CLI, a Docker daemon reachable without
|
||||
`sudo`, 20 GB free where Docker keeps images, agentenv-framework 0.9.1485,
|
||||
the runtime archive of this REA version, the build settings in the
|
||||
registration or the environment, and the skill. Each failure names its fix.
|
||||
It also reports, without counting them against `healthy`, whether a model
|
||||
endpoint for solver runs is configured and which agentenv containers
|
||||
earlier builds left. Pass `--json` for the full report.
|
||||
|
||||
When a setting is unset where the server starts, REA uses `agent-env` and
|
||||
`docker` on `PATH` and the archive setup packed for this REA version. The
|
||||
rest of this page is the same setup by hand.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Docker, and an agentenv-framework install whose `agent-env` program is named
|
||||
by the absolute `REA_AGENT_ENV_COMMAND`. Bundles target agentenv
|
||||
0.9.1431. REA runs its helper, `bridge/agentenv/computer.py`, with that
|
||||
runner's Python: the interpreter on the program's `#!` line, or the
|
||||
`python3` beside it in its virtual environment.
|
||||
- A Linux host on amd64 or arm64 with about 20 GB of free disk for the
|
||||
images agentenv builds and pulls. amd64 and arm64 hosts both build
|
||||
natively; a host builds only its own architecture.
|
||||
- Node.js at the version in `.nvmrc`, installed with
|
||||
[nvm](https://github.com/nvm-sh/nvm) or from
|
||||
[nodejs.org](https://nodejs.org/en/download), and REA
|
||||
[run from a checkout](getting-started/installation.md#run-rea-from-a-checkout).
|
||||
- [Docker Engine](https://docs.docker.com/engine/install/) and its
|
||||
[post-install steps](https://docs.docker.com/engine/install/linux-postinstall/):
|
||||
add yourself to the `docker` group, then log in again.
|
||||
- agentenv-framework 0.9.1485, the version bundles target. Install
|
||||
[uv](https://docs.astral.sh/uv/getting-started/installation/) first, then
|
||||
run `uv tool install agentenv-framework==0.9.1485`. Set
|
||||
`REA_AGENT_ENV_COMMAND` to the absolute path of the resulting `agent-env`
|
||||
program, normally `$HOME/.local/bin/agent-env`; `command -v agent-env`
|
||||
shows the path when its install directory is on your `PATH`. Installing it
|
||||
with pip into a virtual environment instead needs `python3.12-venv` on
|
||||
stock Ubuntu. REA runs its
|
||||
helper, `bridge/agentenv/computer.py`, with that runner's Python: the
|
||||
interpreter on the program's `#!` line, or the `python3` beside it in its
|
||||
virtual environment.
|
||||
- `REA_DOCKER_COMMAND`, the absolute path of the Docker CLI, lets REA remove a
|
||||
computer whose agentenv teardown was not confirmed. Without it, REA names the
|
||||
command to run instead.
|
||||
@@ -32,20 +87,83 @@ app, exporting its bundle and witnessing the task on such a computer are in
|
||||
checkout, `npm run pack:build-computer-runtime -- /path/to/rea-prod.tar.xz`
|
||||
writes one: `npm pack`, its package folder as
|
||||
`rea-prod` with the lockfile, `npm ci --omit=dev --omit=optional`, then
|
||||
`rea-prod` as `.tar.xz`.
|
||||
`rea-prod` as `.tar.xz`. The archive is a copy of the checkout: pack it
|
||||
again after `git pull` and a rebuild.
|
||||
|
||||
Set these where the MCP server starts; a shell export does not reach a server
|
||||
that is already running.
|
||||
|
||||
## Claude Code
|
||||
|
||||
Without `--desktop-build`, `rea setup --client claude_code` registers REA
|
||||
without these variables. Either
|
||||
export them in the shell that starts `claude`, which starts REA's server with
|
||||
its environment:
|
||||
|
||||
```bash
|
||||
export REA_AGENT_ENV_COMMAND="$HOME/.local/bin/agent-env"
|
||||
export REA_DOCKER_COMMAND="$(command -v docker)"
|
||||
export REA_BUILD_COMPUTER_REA_ARCHIVE="$HOME/rea-runtime/rea-prod.tar.xz"
|
||||
claude
|
||||
```
|
||||
|
||||
or set them, as absolute paths, in the `env` of the `mcpServers.rea` entry
|
||||
setup wrote, such as in `~/.claude.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "/absolute/path/to/node",
|
||||
"args": ["/absolute/path/to/rea/scripts/rea.mjs", "mcp"],
|
||||
"env": {
|
||||
"REA_AGENT_ENV_COMMAND": "/home/ada/.local/bin/agent-env",
|
||||
"REA_DOCKER_COMMAND": "/usr/bin/docker",
|
||||
"REA_BUILD_COMPUTER_REA_ARCHIVE": "/home/ada/rea-runtime/rea-prod.tar.xz"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`measure_build_task` runs solves through agentenv, which refuses them without
|
||||
a model endpoint: export `LITELLM_BASE_URL` and `LITELLM_API_KEY` in the shell
|
||||
that starts `claude`, whose REA server inherits them, or set `base_url` and
|
||||
`api_key` under `[model]` in `.agentenv/config.toml`. REA passes them on to
|
||||
agentenv and never reads the key.
|
||||
|
||||
Claude Code asks before each MCP tool call its permission settings do not
|
||||
allow, and a build makes dozens of `act_on_build_computer` calls. You can
|
||||
allow REA's tools in those settings' `permissions.allow` list:
|
||||
`mcp__rea__act_on_build_computer` allows that tool, and `mcp__rea` allows all
|
||||
of REA's.
|
||||
|
||||
A headless run, `claude -p`, ends when the skill asks its upfront questions,
|
||||
and refuses the tool calls its permission settings do not allow. Put the
|
||||
answers in the request: the install source when the app is not in the Ubuntu
|
||||
archive, a synthetic seed or the file to start from, whether to review the
|
||||
task before it is measured, and the model and budget to measure it with, or
|
||||
none:
|
||||
|
||||
```bash
|
||||
claude -p 'Build me a Gnote environment where the agent adds olive oil to the
|
||||
shopping list note. Make a synthetic note. Then measure it with <model>,
|
||||
at most $4 in all, without a review first.'
|
||||
```
|
||||
|
||||
## Start a computer
|
||||
|
||||
From a request, REA renders a bundle in a private temporary folder: a computer
|
||||
app on the `ubuntu-24.04` base with the `reference` driver that installs the
|
||||
app, opens it on demand with `gtk-launch <desktop entry>` from the computer
|
||||
user's home, so its open and save dialogs start there rather than in the
|
||||
framework's folder for the app, and keeps no seed and no data hooks. The app is not open when the computer starts: agentenv lists it
|
||||
in the desktop's Applications menu under Other by the environment's name, such
|
||||
as `mousepad`, and that entry launches it through the app's server.
|
||||
framework's folder for the app, and keeps no seed and no data hooks. The app is not open when the computer starts:
|
||||
`start_build_capture` opens it under REA's capture. When no capture is under
|
||||
way, open it from the desktop's Applications menu, where agentenv lists it
|
||||
under Other by the environment's name, such as `mousepad`; that entry launches
|
||||
it through the app's server. The name is the desktop entry's last part when it
|
||||
is reverse-DNS (`com.github.xournalpp.xournalpp` gives `xournalpp`), or else
|
||||
the whole entry without a trailing version (`glabels-3.0` gives `glabels`).
|
||||
|
||||
When the app is Electron, set `electron` to true. The install rewrites that
|
||||
desktop entry's command with the same policy an export uses: `--no-sandbox`
|
||||
@@ -69,16 +187,51 @@ Applications menu entry can open the app.
|
||||
|
||||
| `kind` | Installs |
|
||||
| ---------------- | ---------------------------------------------------------------------------------------------------------- |
|
||||
| `archive` | `package` from the Ubuntu 24.04 archive, on amd64 and arm64 |
|
||||
| `archive` | `package` from the Ubuntu 24.04 archive, on amd64 and arm64, with its optional `extra_packages` beside it |
|
||||
| `apt_repository` | `package` from the repository at `uri` with `suite` and `components`, trusting the key file `key_file` |
|
||||
| `package_file` | The `.deb` at `url`, checked against `sha256`, for one `architecture`, after its apt `dependencies` are in |
|
||||
| `appimage` | The AppImage at `url`, checked against `sha256`, for one `architecture`, extracted with its own entry |
|
||||
| `tarball` | The archive at `url`, checked against `sha256`, for one `architecture`, with an entry running `program` |
|
||||
|
||||
Build steps run with the network off, so a `.deb`'s dependencies must be
|
||||
listed. The bundle adds REA in a build-only layer: Node.js
|
||||
listed.
|
||||
|
||||
An AppImage or a tarball is downloaded by agentenv when the bundle is put,
|
||||
refused unless its SHA-256 is the one named, and unpacked by a root build
|
||||
step, `install-download.sh`, under `/opt/rea-apps/<desktop_entry>`; the
|
||||
export writes the same step. An AppImage is extracted with its own
|
||||
`--appimage-extract`, so no FUSE is needed, and the step installs the image's
|
||||
own `<desktop_entry>.desktop` under `/usr/local/share/applications` with
|
||||
each `Exec` program replaced by the image's `AppRun`, after any `env` prefix
|
||||
and its assignments, without `TryExec`, and its icon; an image without that entry fails the build, naming the entries it
|
||||
has. A tarball is read with Python's `tarfile`, since the computer has no
|
||||
`xz` program, so gzip, bzip2 and xz archives work. Its `data` filter refuses
|
||||
links that point outside the folder, absolute ones included, and special
|
||||
files, so such an archive fails the build. Its `program` is a path
|
||||
inside the archive, and REA writes the entry: `name` in the menu, or the
|
||||
entry's last part, and `program` with the files it opens (`%F`). An
|
||||
Electron app's command is rewritten after this step, on the entry it
|
||||
installed. An image or tarball holds no Debian package, so the capture
|
||||
records its package as `unknown`; the export checks instead that the
|
||||
captured program lies under the folder the download unpacks to. The
|
||||
downloaded file stays in the image's layer under the app's folder beside
|
||||
what it unpacks, so a 721 MB AppImage that unpacks to 3.0 GB adds both. `extra_packages` names up to 16 more archive packages, by the same
|
||||
name rules as `package`, for a runtime piece the app needs that its package
|
||||
does not install: apt here skips Recommends, so MyPaint 2.0.1 exits at launch
|
||||
without `librsvg2-common`, GTK's SVG icon loader, and Freeplane 1.7.10 needs
|
||||
`openjdk-17-jre`, since its launcher refuses noble's default Java 21. The
|
||||
export installs them beside the app too, as it does a `.deb`'s
|
||||
`dependencies` when apt offers the `.deb`'s package; see
|
||||
[build and witness](desktop-build-witness.md).
|
||||
|
||||
The bundle adds REA in a build-only layer: Node.js
|
||||
24.22.1, downloaded per architecture and checked against its release checksum,
|
||||
and REA from `REA_BUILD_COMPUTER_REA_ARCHIVE`, both unpacked into `/opt/rea` by
|
||||
a root build step, with `node` and `rea` on `PATH`. An exported bundle carries
|
||||
none of it.
|
||||
a root build step, with `node` and `rea` on `PATH`. It also installs
|
||||
`libgtk-3-bin`, for `gtk-launch`, and `dconf-cli`, so the capture reads the
|
||||
app's settings in dconf, such as GnuCash's "Show tips at startup", and the
|
||||
seed keeps them. An exported bundle carries none of it but `dconf-cli`, which
|
||||
it installs when it loads the app's dconf keys.
|
||||
|
||||
From a bundle folder, such as one `export-desktop-application` wrote, REA
|
||||
starts that bundle's `tasks/smoke.json` as it is:
|
||||
@@ -94,6 +247,10 @@ in the folder and its files stay as they were shared.
|
||||
REA runs the deploy with `run_bundle(keep=True, emulate=False)` on the local
|
||||
sandbox provider, so only the host's own architecture is built. The first start
|
||||
of a bundle builds its image and can take minutes; later starts reuse it.
|
||||
agentenv makes a start wait while another agentenv run on this host writes its
|
||||
bundle's ids, so starts at the same time wait for each other, several minutes
|
||||
when a few run together; the progress line then says it is waiting for another
|
||||
agent-env run to finish writing this bundle's ids.
|
||||
While it waits, REA sends an MCP progress notification every five seconds with
|
||||
agentenv's latest line, so a client that resets its request timeout on
|
||||
progress keeps waiting. REA then waits until the computer's readiness checks
|
||||
@@ -103,9 +260,12 @@ while it is one colour throughout. The retries share 10 s, each given only
|
||||
what is left of it, and when that runs out REA returns the latest frame;
|
||||
progress notifications continue meanwhile. Given `screenshot` with a
|
||||
`region`, `format` or `quality`, REA then takes one with them, a `region`
|
||||
cropping that frame. The result holds
|
||||
cropping that frame rather than capturing the screen again. The result holds
|
||||
`computer_id`, agentenv's `sandbox_id`, the environment name or bundle folder,
|
||||
and the screenshot's frame as structured content beside its image.
|
||||
`extra_packages`, what the request named beside the app for the export to
|
||||
install too (an archive install's `extra_packages` or a `.deb`'s
|
||||
`dependencies`, empty otherwise), and the screenshot's frame as structured
|
||||
content beside its image.
|
||||
|
||||
A failed start returns the last 16 KiB of the helper's stderr, where a failing
|
||||
build prints its error, and its exit status; a cancelled one stops the helper,
|
||||
@@ -119,16 +279,16 @@ progress and waits for them before closing every computer.
|
||||
`act_on_build_computer` takes `computer_id`, an `action` and the driver's own
|
||||
arguments for `computer_<action>`, which REA passes through:
|
||||
|
||||
| `action` | Arguments |
|
||||
| ------------ | --------------------------------------------------------------- |
|
||||
| `screenshot` | `region`, `format` (`png` or `jpeg`), `quality` |
|
||||
| `click` | `x`, `y`, `button`, `count` (2 double-clicks), `modifiers` |
|
||||
| `drag` | `path` of `[x, y]` points, `button`, `duration_ms`, `modifiers` |
|
||||
| `scroll` | `direction`, `x`, `y`, `amount`, `modifiers` |
|
||||
| `type` | `text`, where `\n` presses Enter and `\t` Tab |
|
||||
| `key` | `keys`, one chord pressed in order, and `repeat` |
|
||||
| `wait` | `ms` |
|
||||
| `reset` | `screenshot` |
|
||||
| `action` | Arguments |
|
||||
| ------------ | ----------------------------------------------------------------------- |
|
||||
| `screenshot` | `region`, `format` (`png` or `jpeg`), `quality` |
|
||||
| `click` | `x`, `y`, `button`, `count` (2 double-clicks), `modifiers` |
|
||||
| `drag` | `path` of 2 to 50 `[x, y]` points, `button`, `duration_ms`, `modifiers` |
|
||||
| `scroll` | `direction`, `x`, `y`, `amount`, `modifiers` |
|
||||
| `type` | `text`, where `\n` presses Enter and `\t` Tab |
|
||||
| `key` | `keys`, one chord pressed in order, and `repeat` |
|
||||
| `wait` | `ms` |
|
||||
| `reset` | `screenshot` |
|
||||
|
||||
`reset` is not a driver tool: REA calls the app's `data/reset`, which restores
|
||||
the bundle's seed and relaunches the app, then waits until the computer's
|
||||
@@ -152,7 +312,9 @@ The result carries the driver's content unchanged, its text and images, after
|
||||
REA's own JSON: `action_index` and the driver's structured result. A refusal,
|
||||
such as a stale frame, is an error carrying the computer's message. A response
|
||||
larger than the MCP budget is refused with how to ask for a smaller image: a
|
||||
`region`, or `jpeg` with a lower `quality`. That error keeps REA's own result,
|
||||
`region`, or `jpeg` with a lower `quality`. A screenshot with `region` does not
|
||||
capture the screen again; it crops the latest frame, so take a full
|
||||
screenshot, or a `wait` with `screenshot: true`, first. That error keeps REA's own result,
|
||||
`computer_id` included, as `details.result`, since the action or start has
|
||||
happened.
|
||||
|
||||
|
||||
+236
-41
@@ -7,21 +7,31 @@ it writes, and then witnesses the task in a fresh computer of that bundle.
|
||||
REA runs its own capture, task draft and export inside the build computer and
|
||||
grades the witness with the same checks the exported verifier runs.
|
||||
|
||||
| Tool | What it does |
|
||||
| -------------------------- | ------------------------------------------------------------------------------------------- |
|
||||
| `start_build_capture` | Downloads the package lists as the computer's user and launches the app under REA's capture |
|
||||
| `mark_build_checkpoint` | Takes a named checkpoint and returns the capture's acknowledgement with its counts |
|
||||
| `stop_build_capture` | Stops the capture and returns its checkpoints and the task draft between `seed` and `end` |
|
||||
| `export_build_environment` | Exports the bundle with the agent's task inside the computer and copies it to a folder here |
|
||||
| `check_build_task` | Grades a task against the app's live state; with a `stage`, files the check in the witness |
|
||||
| Tool | What it does |
|
||||
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `start_build_capture` | Downloads the package lists as the computer's user and launches the app under REA's capture |
|
||||
| `mark_build_checkpoint` | Takes a named checkpoint and returns the capture's counts and the files that changed |
|
||||
| `stop_build_capture` | Stops the capture and returns its checkpoints and the task drafts, from `seed` to `end` and on to each `end-<task name>` |
|
||||
| `export_build_environment` | Exports the bundle with the agent's tasks inside the computer and copies it to a folder here |
|
||||
| `check_build_task` | Grades a task against the app's live state; with a `stage`, files the check in the witness |
|
||||
| `measure_build_task` | Runs a bundle's task in fresh computers for a model and records how hard it is in the bundle |
|
||||
|
||||
Like the computer tools, these are MCP-only and act on a `computer_id` from
|
||||
`start_build_computer`.
|
||||
|
||||
REA's skill walks an agent from a request such as "build me an Inkscape
|
||||
environment where the agent changes a poster's headline" to a witnessed
|
||||
bundle with these tools: the questions it asks first, when it stops for the
|
||||
person, the install, the seed, the task, the witness and its final report.
|
||||
See [desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md).
|
||||
|
||||
## Build
|
||||
|
||||
1. `start_build_computer` from a request. REA is installed in the computer.
|
||||
2. `start_build_capture`. REA writes an apt configuration under
|
||||
2. `start_build_capture`. REA first checks the installed runtime's export
|
||||
schema for synthetic-seed support. If the archive is too old, repack a
|
||||
current REA, update `REA_BUILD_COMPUTER_REA_ARCHIVE`, restart the REA MCP
|
||||
server and start a new build computer. REA then writes an apt configuration under
|
||||
`~/.rea-build/apt` and runs `apt-get update` as the computer's user, so the
|
||||
capture finds the package's sources (`package_lists` counts the lists,
|
||||
and `package_lists_error` gives apt's exit status and last lines when the
|
||||
@@ -32,6 +42,34 @@ Like the computer tools, these are MCP-only and act on a `computer_id` from
|
||||
from a `.deb` file, the build layer keeps that file at
|
||||
`/opt/rea/package.deb` and the capture records it with `--package-file`. Do not open the app from
|
||||
the menu first: the capture refuses an app whose window is already open.
|
||||
When the capture ends before the window opens, REA reads its output. An app
|
||||
that exits at launch, such as one missing a runtime, is reported with its
|
||||
program, exit code, run time and the last lines of its stderr, at most 8
|
||||
lines and 800 characters of the last 8 KiB the capture keeps, and
|
||||
`details.diagnostics` keeps the capture's record of the session. A capture
|
||||
that failed, such as one refusing to start, gives its own message and
|
||||
remediation, with the whole failure in `details.diagnostics.command_failure`
|
||||
and, when its cleanup is incomplete, the run tokens and paths it left in
|
||||
`details.resources`. A checkpoint or stop that finds the capture ended
|
||||
reads its output the same way.
|
||||
|
||||
`state_paths` names files or folders in the computer's home that hold the
|
||||
app's state, relative to the home, after `~/` or absolute under it, at
|
||||
most 16; each becomes a `--state-path` of the capture, so it is retained
|
||||
from the start and its files count as the app's even before they exist.
|
||||
REA counts a file as a document only when an app window's title shows its
|
||||
name or `~/` path, so name the file the app will save when its title shows
|
||||
something else, as MuseScore's shows a score's title: a file outside
|
||||
hidden folders, other than a lock such as LibreOffice's `.~lock.<name>#`,
|
||||
becomes a document, which the bundle owns, seeds and opens
|
||||
at launch, and one under a hidden folder such as `~/.config/MuseScore` the
|
||||
app's own state, also when that folder is a link elsewhere in the home. A
|
||||
path with `..`, or naming the home itself, is refused before the computer
|
||||
is asked; the computer refuses an absolute path outside its home, one in
|
||||
`~/.rea-build`, and one a link leads out of the home or into
|
||||
`~/.rea-build`, resolving a path that does not exist yet through its
|
||||
nearest existing folder, and starts nothing.
|
||||
|
||||
3. Make the seed through the app with `act_on_build_computer`, saving it
|
||||
where the user keeps such files, such as `~/Documents`, then
|
||||
`mark_build_checkpoint` `seed`.
|
||||
@@ -39,33 +77,103 @@ Like the computer tools, these are MCP-only and act on a `computer_id` from
|
||||
`mark_build_checkpoint` `end`.
|
||||
5. `stop_build_capture`. The app is closed and the capture finishes. The
|
||||
result lists the checkpoints with their counts, the package and the
|
||||
sources the capture found, its limitations and the draft: each file that
|
||||
changed between `seed` and `end` with its net change, and suggested
|
||||
checks with a placeholder prompt.
|
||||
6. Write the task: its name, the prompt in the user's own words, and checks
|
||||
that hold after the change and not on the seed.
|
||||
7. `export_build_environment` with the task and a new or empty `directory`.
|
||||
sources the capture found, its limitations and the draft: each file under
|
||||
the paths the export owns that changed between `seed` and `end`, with its
|
||||
net change and, for XML files, archives, JSON and key files, its `values`,
|
||||
the values the change added, changed and removed with what each was and
|
||||
became, at the pointer a check names it by; suggested checks with a
|
||||
placeholder prompt; and `guards`, optional checks that the rest kept its
|
||||
seed values: like siblings' fields, each line kept beside new text, and
|
||||
the count of a list whose members were only edited. The draft's
|
||||
[task section](desktop-application-tasks.md#describe-the-change) describes
|
||||
each. `notes`, each with a `code` and a `message`, say why a draft has no
|
||||
files: `no_change` when no file under the home changed between them and
|
||||
nor did the app's dconf keys; `unowned_changes` when files changed but
|
||||
none under the owned paths, listing up to five in `files` with their
|
||||
roles, desktop files last, and counting the rest in `more`;
|
||||
`uncheckable_changes` when the app's keys in dconf were read and changed,
|
||||
listing the database in `files`, since no check reads dconf (a version
|
||||
the capture kept no copy of, as it keeps none of the database at the
|
||||
start, is `unreadable_changes`, not an empty key list);
|
||||
`unreadable_changes` when a dconf database changed but REA could not read
|
||||
it, such as without `dconf-cli` or when a present version's bytes were
|
||||
not retained, with the read `error`, so whose keys changed is unknown;
|
||||
and `no_document` when no file has the document role.
|
||||
`unowned_changes` and `no_document` point at `state_paths`. They also say,
|
||||
for a drafted `file`, why its suggestions are thin: `by_position`,
|
||||
`view_state_only`, `long_text`, `cut_view`, `unreadable_view`,
|
||||
`volatile_lines`, `generated_reference` or `read_as_text`.
|
||||
6. Write the task: its name, the prompt in the user's own words naming the
|
||||
app (a solving agent's own shell cannot see the computer's files, and the
|
||||
export's limitations say when a prompt names neither the app's title,
|
||||
environment nor package), and checks
|
||||
that hold after the change and not on the seed, from the suggestions,
|
||||
`values` and any `guards` worth keeping.
|
||||
7. `export_build_environment` with the tasks and a new or empty `directory`.
|
||||
Each task is `{"task", "end_checkpoint"}`, its end `end` by default; a
|
||||
later task's change made on top of the last, and marked `end-<task name>`,
|
||||
is checked there, and `stop_build_capture`'s `later_drafts` drafts it from
|
||||
the end before it. A checkpoint `near-<task name>` taken after a near miss
|
||||
of that task's change must fail one of its checks. An end the capture did
|
||||
not take is refused before anything runs.
|
||||
The export runs inside the computer with `--seed-checkpoint seed
|
||||
--end-checkpoint end`, so it seeds what was saved at `seed`, scrubs the
|
||||
--end-checkpoint end` and each task's `--task-end`, so it seeds what was
|
||||
saved at `seed`, scrubs the
|
||||
computer's host name and user from it, and refuses a task whose checks
|
||||
all pass on the seed. A task larger than 96 KiB as REA passes it to the
|
||||
computer is refused before anything runs. A file moved to the desktop Trash is not seeded,
|
||||
and the export's limitations name what the Trash held at `seed`. For a
|
||||
all pass on the seed. It also passes `--synthetic-seed`, since the agent
|
||||
made the seed for the task, so the README does not call it the user's real
|
||||
data. When the seed holds exactly one document and the app's desktop entry
|
||||
takes files, the bundle's app opens that document at launch. A file moved
|
||||
to the desktop Trash is not seeded, and the export's limitations name what
|
||||
the Trash held at `seed`. The limitations and the README also name the
|
||||
folders where files changed during the capture that the bundle neither
|
||||
owns nor seeds, such as settings that keep a first-run wizard away, up to
|
||||
eight collapsed to their top folder, such as `~/.config/MuseScore`, and a
|
||||
count of the rest; capture again with them in `state_paths` to own them.
|
||||
Lock files the app holds while it runs, such as LibreOffice's profile
|
||||
`.lock` and its document lock `.~lock.<name>#`, GnuCash's
|
||||
`<book>.LCK` or Grisbi's `.<file>.lock` beside its file, are left out of
|
||||
the seed, since a copied one can keep the app from starting or saving. The
|
||||
limitations and README list them after "Left out of the seed". A project's
|
||||
`Cargo.lock` or `yarn.lock`, a document or a path in `state_paths`, and app
|
||||
logs that are not locks are seeded. The limitations name seeded
|
||||
WebKit or Chromium storage and caches too, which the app keeps for its
|
||||
own running. When the application held files open in a folder whose
|
||||
starting content the capture did not retain, a limitation names the
|
||||
folder and the seed files in it. For a
|
||||
`.deb` app that no apt source the capture found offers, the export takes
|
||||
the request's URL with `--package-url`; one apt offers installs from apt. REA lists the bundle's files with their sizes
|
||||
and SHA-256 in the computer, into a file beside the bundle that it reads
|
||||
through `data/get`, since a large bundle's list outgrows a command's
|
||||
output; it then brings the files back through the driver's
|
||||
`data/get` in answers of at most 8 MiB, and checks and writes each answer
|
||||
into the folder before asking for the next, so one answer at a time is
|
||||
held; a failure leaves no partial folder.
|
||||
the request's URL with `--package-url`; one apt offers installs from apt.
|
||||
Either way the request's `dependencies` reach the export as
|
||||
`--extra-package`, as an archive request's `extra_packages` do, so a
|
||||
runtime piece the agent named, such as MyPaint's `librsvg2-common`, is
|
||||
installed beside the app; the limitations and the README list them,
|
||||
except the app itself and dependencies the bundle already names for a
|
||||
`.deb` it installs from the file. REA lists the bundle's files
|
||||
with their sizes and SHA-256 in the computer, into a file beside the
|
||||
bundle that it reads through `data/get`, since a large bundle's list
|
||||
outgrows a command's output; it then brings the files back through the
|
||||
driver's `data/get` in answers of at most 8 MiB, and checks and writes
|
||||
each answer into the folder before asking for the next, so one answer at
|
||||
a time is held; a failure leaves no partial folder.
|
||||
8. `close_build_computer`.
|
||||
|
||||
A checkpoint name is lowercase letters, digits, dot, dash and underscore; the
|
||||
capture's own `baseline`, `stop`, `time-limit` and `exit` are refused, and so
|
||||
is a name already taken. The acknowledgement comes from the capture after it
|
||||
has walked the home and retained what changed: `entries` walked, `changes`
|
||||
since the checkpoint before, `retained_files` and `retained_bytes`.
|
||||
since the checkpoint before, `retained_files` and `retained_bytes`, and up to
|
||||
ten `changed_files`, home-relative, with `more_changed_files` counting the
|
||||
rest. Folders are left out. Files an app window's title has named so far
|
||||
come first, as a document saved and no longer held open is in Gnumeric, LyX
|
||||
or Tiled; then files named themselves in `state_paths`; then those the app
|
||||
holds open, those under a selected folder and those in its state
|
||||
directories; then the rest; and caches and WebKit, Chromium or QtWebEngine
|
||||
storage last, even under a selected folder such as `.config/libreoffice`
|
||||
with its icon cache, each by path. So a save's document leads the list.
|
||||
When the computer's REA is older than these lists,
|
||||
as when `REA_BUILD_COMPUTER_REA_ARCHIVE` still names an earlier build, the
|
||||
checkpoint is still taken and both are `null`: which files changed is
|
||||
unknown. Pack the current REA into the archive to get them back.
|
||||
|
||||
Each step can be asked again after its answer was lost:
|
||||
|
||||
@@ -89,7 +197,9 @@ computer's control token. REA sends each command through agentenv's
|
||||
`computer-exec/v1` route with that token, which stays in REA, and reads larger
|
||||
outputs, such as the draft, the export's Evidence and the bundle's file
|
||||
list, from files under `~/.rea-build` through `data/get`, since a command's
|
||||
output is capped at 1 MiB.
|
||||
output is capped at 1 MiB. In a command's Evidence or failure read this way,
|
||||
as in its own output, REA replaces the token in every string, after checking
|
||||
the Evidence's digest; the bundle's files arrive as they are.
|
||||
|
||||
Calls on one computer run one at a time, in the order REA receives them:
|
||||
an action, a capture step, a check or a close waits until the call before
|
||||
@@ -102,7 +212,12 @@ cancels the call in progress on it and refuses the calls still waiting.
|
||||
## Witness
|
||||
|
||||
Start a fresh computer from the exported folder with `start_build_computer`
|
||||
and `{"kind": "bundle", "path": ...}`; agentenv boots the app with the seed.
|
||||
and `{"kind": "bundle", "path": ...}`; agentenv boots the app with the seed,
|
||||
on the seed's document when the export opens one. When the computer never
|
||||
gets ready and the bundle's seed holds lock files, the start's error names
|
||||
them: delete them from the bundle, or export again, which leaves them out.
|
||||
A bundle records no state directories, so the error names only files by a
|
||||
profile lock's name and LibreOffice's document locks.
|
||||
Then, with the task exactly as exported:
|
||||
|
||||
| `stage` | Before it | Its checks must |
|
||||
@@ -115,13 +230,29 @@ Then, with the task exactly as exported:
|
||||
`check_build_task` reads the app's `data/get`, takes the views it returns
|
||||
beside the files and evaluates each check with REA's evaluator, the one the
|
||||
export validates tasks with and the exported verifier mirrors, one result per
|
||||
check. Without `stage` it only grades. The `seed` stage begins a recording
|
||||
check. Once the files settle it runs the task's
|
||||
[readers](specs/desktop-app-environments/file-readers/spec.md) in the computer through
|
||||
`computer-exec/v1`, as the verifier does, and a check through a reader reads
|
||||
that reader's view. A task of [screen judges](specs/desktop-app-environments/screen-judge/spec.md)
|
||||
reads no files: REA takes a screenshot and runs the bundle's own judge on it
|
||||
with the agentenv runner's Python and model endpoint. Like the verifier, it grades only once the app's files settle, since an
|
||||
app such as Gnote saves a few seconds after the last key. It reads `data/get`
|
||||
every 0.5 s and grades once every file's bytes have held for 3 s, counted
|
||||
from the read that returned them. 15 s after the first read began, it grades
|
||||
the last completed read: no read starts then, and one still running is cut.
|
||||
It sends progress notifications while it waits. The result's `settled` says
|
||||
whether the files settled, and `read_cut` whether the bound cut a read; each
|
||||
stage in the record keeps both. Every check therefore costs at least 3 s,
|
||||
and a save that lands more than 3 s after the first read returns is still
|
||||
missed; see
|
||||
[grading once the app has saved](desktop-application-export.md#grading-once-the-app-has-saved).
|
||||
Without `stage` it only grades. The `seed` stage begins a recording
|
||||
of the display under REA's own request id, one per computer; a seed check
|
||||
that fails after it, such as on a failed read of the app's state or a lost
|
||||
reply to the recording's start, can be asked again, and asks for the
|
||||
recording under the same id, which the computer answers with the recording
|
||||
it already began. The stages come in order. The edit and the near miss each
|
||||
need an applied click, drag, scroll, type or key since the stage before, with
|
||||
need an applied click, drag, scroll, type, typed secret or key since the stage before, with
|
||||
no reset after the last of them, since a reset puts the seed back rather than
|
||||
changing the app; `after_reset` needs the reset as the last change, and a
|
||||
reset that leaves any check unlike on the seed fails the witness, its
|
||||
@@ -131,20 +262,28 @@ in the bundle differs from what the export writes for it. A reset
|
||||
relaunches the app, and REA waits until the computer is ready again before
|
||||
answering, sending progress notifications while it waits.
|
||||
|
||||
Re-export bundles whose generated verifier uses an older evaluator before
|
||||
witnessing them with current REA. The verifier must match the current export
|
||||
exactly; re-exporting uses the existing capture Evidence and task inputs,
|
||||
which remain unchanged.
|
||||
|
||||
Only a check that fails on what the app's view shows counts as a failure. A
|
||||
check that fails because its file or table could not be read, such as an
|
||||
SQLite file locked past the reader's wait, or because what it looks for may
|
||||
lie in what its view cut, is `undecided` in its result: it shows nothing
|
||||
about the task. That covers a line `lines_present` misses in a text view cut
|
||||
at 5,000 lines, a row `row_present` misses in a table whose view keeps only
|
||||
its first rows, a `value_equals` pointer that names nothing in a truncated
|
||||
view or past a cut zip listing, and the absence checks and counts on a cut
|
||||
view. A check that passes on what the view kept stays decided, and so does a
|
||||
value found there that differs from the one expected. The result's `passed`
|
||||
and description are the exported verifier's, which has no undecided results.
|
||||
A stage that only undecided checks would settle, such as a near miss whose
|
||||
one failing check could not read its file, files nothing and is refused;
|
||||
check it again once the files read whole.
|
||||
its first rows, a `value_equals` or text check pointer that names nothing
|
||||
in a truncated view or past a cut zip listing, the absence checks and counts
|
||||
on a cut view, and `text_equals` or `text_contains` of an element on one,
|
||||
even where the element's kept text holds the value, since the cut may have
|
||||
dropped text from between its runs. Any other check that passes on what the
|
||||
view kept stays decided, and so does a value found there that differs from
|
||||
the one expected.
|
||||
The result's `passed` and description are the exported verifier's, which has
|
||||
no undecided results. A stage that only undecided checks would settle, such
|
||||
as a near miss whose one failing check could not read its file, files nothing
|
||||
and is refused; check it again once the files read whole.
|
||||
|
||||
A stage whose checks do not show what it must ends the witness, and so does
|
||||
`after_reset`. REA then ends the recording, fetches its segments through
|
||||
@@ -154,13 +293,14 @@ A stage whose checks do not show what it must ends the witness, and so does
|
||||
- `recording/NNNN.mp4`, the recording's segments;
|
||||
- `witness.json`, REA Evidence (`rea.desktop-build-witness`) holding the
|
||||
verdict, the stage that failed if one did, the task's name and SHA-256,
|
||||
each stage's results with how many logged actions came before it, the
|
||||
each stage's results with how many logged actions came before it and
|
||||
whether the app's files had settled (`settled`), the
|
||||
computer's [action log](desktop-build-computer.md#act) as it keeps it, each
|
||||
action with its arguments, timing and outcome, with `actions_dropped`
|
||||
counting the earliest actions it no longer keeps, and the recording's
|
||||
segments with their sizes and SHA-256. The record's limitations say when
|
||||
the log dropped actions or cut arguments, which each entry names in
|
||||
`truncated_arguments`.
|
||||
`truncated_arguments`, and which stages graded files that had not settled.
|
||||
|
||||
The stages' rules read the log's count of actions and the last applied
|
||||
action of each kind, which it keeps for the whole session, so they hold
|
||||
@@ -169,7 +309,62 @@ folder something else wrote into, the witness keeps its stages: check the
|
||||
stage that ended it again, with the same task, and REA writes the record
|
||||
without checking again. Once it is saved, the same check returns the saved
|
||||
record again, so a record saved as its call was cancelled is not lost. A
|
||||
computer is witnessed once; witness again from a fresh computer.
|
||||
computer is witnessed once; witness again from a fresh computer. A bundle of
|
||||
several tasks is witnessed one task per fresh computer: the seed stage
|
||||
refuses a task whose steps and verifier differ from the bundle's for that
|
||||
task's name, so each task's record shows that task alone from the seed.
|
||||
|
||||
## Measure
|
||||
|
||||
`measure_build_task` takes a bundle folder, a task's name, a `model`, an
|
||||
`agent` (`claude-code` by default) and `budget_usd`. It runs the task through
|
||||
REA's helper, `bridge/agentenv/solve.py`, with the agentenv runner's Python on
|
||||
this host, one solve at a time, each in a fresh computer the run tears down:
|
||||
the bundle's own steps, with the agent deployed as the solver on the model.
|
||||
The model endpoint is the one agentenv reads where REA starts,
|
||||
`LITELLM_BASE_URL` and `LITELLM_API_KEY` or `[model]` in
|
||||
`.agentenv/config.toml`; REA holds no model client and never reads the key.
|
||||
|
||||
Each solve uses a private agentenv sandbox root. If the helper stops before
|
||||
reporting cleanup, REA terminates each sandbox agentenv lists under that root
|
||||
(`local_sandbox_ids`), including one killed while it recorded its ports, and
|
||||
keeps the root when cleanup fails. agentenv keeps a sandbox's folder until its
|
||||
teardown is confirmed, so once that root is empty the solve's cleanup is
|
||||
verified, report or not. On macOS, REA cannot verify host processes left by a
|
||||
recovered sandbox.
|
||||
|
||||
| Solves finished | Passes | Band | Next |
|
||||
| --------------- | --------- | ---------- | ------------- |
|
||||
| 3 | 0 | `too_hard` | stop |
|
||||
| 3 | 3 | `too_easy` | stop |
|
||||
| 3 | 1 or 2 | | 2 more solves |
|
||||
| 5 | 1, 2 or 3 | `in_band` | stop |
|
||||
| 5 | 4 | `too_easy` | stop |
|
||||
|
||||
A solve agentenv reports `failed`, `unscored` or `cancelled` did not finish:
|
||||
it is run again, at most twice, and not counted, though its cost is. Before
|
||||
each solve the measure stops, `stopped: "budget"`, when the largest cost so
|
||||
far would take the total past `budget_usd`; a solve's cost is known only when
|
||||
it ends, from Claude Code's last trajectory record, so the first solve always
|
||||
runs and a solve that reports no cost stops the measure. A solve whose helper
|
||||
is stopped before it reports, by the solve's deadline or killed from outside,
|
||||
counts as `failed` with no known cost, so the measure records it and stops. A
|
||||
solve that could not run at all, such as for want of a model endpoint, ends the measure with
|
||||
its error and records nothing.
|
||||
|
||||
The result and `difficulty.json`, at the bundle's root, keep per task: the
|
||||
model, agent and time; the band and why it stopped; `k`, the finished solves,
|
||||
the passes and unfinished solves; the cost in all and the budget; each
|
||||
finished solve's turns; `failures_per_check`, how many finished solves failed
|
||||
each check; each solve's outcome, turns, cost, seconds, failed checks,
|
||||
instance id and the error agentenv reported for it; `task_sha256`, the
|
||||
SHA-256 of the task's steps file followed by its verifier, so a record whose
|
||||
task changed since shows it is stale; and
|
||||
REA's and agentenv's versions. Measuring a task again replaces its record and
|
||||
keeps the others. A `difficulty.json` or README REA cannot read or parse is refused
|
||||
before any solve and left as it is, and measures of one bundle write one at a
|
||||
time, so neither loses the other's record. The README gets a Measured difficulty section with one
|
||||
line per task, which agentenv ignores like the file.
|
||||
|
||||
See the [specification](specs/desktop-app-environments/build-witness/spec.md)
|
||||
for scope and verification.
|
||||
@@ -19,7 +19,7 @@ initialization, discovery and target-free request flow; it does not verify a
|
||||
client's configured launcher or its inherited environment. Test the generated
|
||||
registration through the actual client, including a reduced `PATH` when checking
|
||||
desktop or service startup. Record the client version and launch environment
|
||||
separately from terminal doctor results. See [launcher troubleshooting](client-configuration.md#launchers-and-path).
|
||||
separately from terminal doctor results. See [launcher troubleshooting](../getting-started/client-configuration.md#launchers-and-path).
|
||||
|
||||
## Published Pi code-mode renderer limit
|
||||
|
||||
@@ -33,7 +33,7 @@ shared form would leave later fields unknown instead. The existing
|
||||
its character limit by advertising reduced nested validation structure, while
|
||||
REA continues to validate calls against the complete canonical input schema.
|
||||
Compact is the default for the server's advertised inputs; an explicit `full`
|
||||
choice is preserved. See the [compact input schema profile](mcp-contracts.md#compact-input-schema-profile)
|
||||
choice is preserved. See the [compact input schema profile](../reference/mcp-contracts.md#compact-input-schema-profile)
|
||||
for details of its reduced forms.
|
||||
|
||||
Custom renderer hosts can raise `inputMaxChars` through `renderToolSignature` or
|
||||
@@ -47,7 +47,7 @@ and [renderer usage notes](https://github.com/earendil-works/pi/blob/b45e0ebcc5a
|
||||
|
||||
## llama.cpp
|
||||
|
||||
The [manual stdio connection](client-configuration.md#llamacpp) was checked on
|
||||
The [manual stdio connection](../getting-started/client-configuration.md#llamacpp) was checked on
|
||||
macOS arm64 with Node.js 24.18.0, llama-server b11715 (`9d1878a43`), and the
|
||||
published `[email protected]` package. `GET /tools` exposed 207 REA tools without
|
||||
enabling llama.cpp's built-in tools. With the source checkout at `9e372c6`, all
|
||||
@@ -486,7 +486,7 @@ existing static JavaScript analysis through CLI and stdio MCP, including
|
||||
manifest readback, competing query variants, and resolved relative imports.
|
||||
It uses an installed browser and accepts an optional installed REA entrypoint.
|
||||
The complete `verify:browser` lane also exercises passive script export through
|
||||
both public adapters. See [website script export](website-script-export.md).
|
||||
both public adapters. See [website script export](../guides/web/website-script-export.md).
|
||||
|
||||
`verify:browser:modules` compares CLI and stdio MCP traces against an independent
|
||||
real Chromium module-loading fixture: import-map scopes, package prefixes,
|
||||
@@ -494,7 +494,7 @@ null/backtracking rejection, query/fragment identities, repeated module instance
|
||||
lazy and computed unknowns, exact source/map identities and no implicit refetch.
|
||||
Set `REA_BROWSER_EXECUTABLE`; the optional script argument selects an installed
|
||||
package entrypoint. The complete `verify:browser` lane and existing conditional
|
||||
Chrome CI include this verifier. See [module relationships](website-module-trace.md).
|
||||
Chrome CI include this verifier. See [module relationships](../guides/web/website-module-trace.md).
|
||||
|
||||
## Agent evaluation and conformance records
|
||||
|
||||
@@ -643,3 +643,65 @@ npm run evidence:check
|
||||
|
||||
The records preserve unsupported and unverified coverage as explicit unknowns.
|
||||
Run the matching real-tool prerequisites described in this guide.
|
||||
|
||||
### Focused tool-description trials
|
||||
|
||||
For a wording or discoverability question, begin with one representative case
|
||||
per affected workflow boundary and its nearest valid alternative. Repeat a
|
||||
concrete failure or ambiguous result after repair. Add repetitions when the
|
||||
requested confidence or comparison needs them; a large matrix is not the default.
|
||||
Use the user's selected model and budget, and verify the actual client and Node
|
||||
executables used by the trial rather than assuming the development shell's
|
||||
versions apply.
|
||||
|
||||
Reuse the existing optional lane and source-owned scenarios where they fit:
|
||||
|
||||
```bash
|
||||
REA_AGENT_EVAL_SCENARIOS=javascript-module-view npm run verify:agent
|
||||
```
|
||||
|
||||
Select the account-supported model with `REA_AGENT_EVAL_MODEL`. Read the selected
|
||||
prompt and rubric before interpreting the gate: closed factual scenarios request
|
||||
complete producer Evidence, while `javascript-module-view` exercises retained
|
||||
views with heuristic scoring. Neither gate alone proves correct selection or
|
||||
fully supported prose for a different task. Extend the narrow scenario or grader
|
||||
in the existing evaluation tooling when necessary; reusable runners and fixtures
|
||||
belong there, not in skills or copied temporary harnesses.
|
||||
|
||||
Capture the actual `initialize` and `tools/list` surface. Compact presentation may
|
||||
remove parameter descriptions or nested selector shapes, so a concise exact call
|
||||
example can remain necessary. Verify the producer-to-consumer handoff before
|
||||
removing its visible syntax. For a wording comparison, hold prompts, schemas,
|
||||
tool order, fixtures, permissions and host settings fixed; change one surface at
|
||||
a time. Use fresh agent context without the audit's answers. Record later source,
|
||||
fixture or runtime changes separately rather than claiming a controlled delta.
|
||||
|
||||
Score these outcomes separately:
|
||||
|
||||
- Tool selection, accepting other valid routes.
|
||||
- Canonical argument validity and the exact requested paths, IDs, selectors,
|
||||
fields, units and delivery. Structurally valid arguments can select the wrong
|
||||
input; fixtures should refuse that selection instead of returning the expected
|
||||
answer anyway.
|
||||
- Recovery and cleanup. A valid unresolved lookup is not a malformed request;
|
||||
normal status reads and cleanup-only retries are not automatically mistakes.
|
||||
Do not infer that work stopped from a host-only timeout.
|
||||
- Final claims against returned facts and coverage. Correct text does not erase
|
||||
malformed-call loops, and valid calls do not prove every claim. A failed name
|
||||
lookup is not a negative reference search; an exhausted partial page does not
|
||||
establish absence in unexamined source. Generic responses without state facts
|
||||
cannot establish that a session is closed.
|
||||
|
||||
Retain prompts, condition/catalog hashes, actual calls/results, failures, final
|
||||
answers, model/client/runtime versions and cleanup outcomes. Report whole-task
|
||||
input, cached input, output and elapsed time when available, and label unmeasured
|
||||
CPU or process-family memory. Fewer description bytes do not establish fewer
|
||||
task tokens or faster execution. Remove copied authentication after owned
|
||||
processes settle and review retained artifacts before sharing them.
|
||||
|
||||
Controlled errors can test agent recovery without target effects. Label them as
|
||||
simulated and verify applicable workflows separately through the production
|
||||
adapter and real provider boundaries. State tool-subset versus full/progressive
|
||||
catalog scope and remaining provider/host coverage. Small one-shot trials are
|
||||
smoke checks, not statistical success rates or evidence that newer models need
|
||||
less guidance in general.
|
||||
@@ -0,0 +1,82 @@
|
||||
# Documentation ownership and authoring
|
||||
|
||||
Use the [documentation directory](../README.md) to find existing material before
|
||||
adding another guide. Preserve one canonical owner for each contract or workflow;
|
||||
link to that owner instead of copying its claims into several guides.
|
||||
|
||||
## Choose a destination
|
||||
|
||||
| Document | Destination |
|
||||
| -------------------------------------------------------- | -------------------------------------- |
|
||||
| Installation, client setup and first CLI/MCP steps | `docs/getting-started/` |
|
||||
| Analyst workflows and evidence interpretation | `docs/guides/<domain>/` |
|
||||
| Analysis-engine configuration and format prerequisites | `docs/providers/` |
|
||||
| Shared runtime contracts and evidence semantics | `docs/reference/` |
|
||||
| Development, testing, release and authoring instructions | `docs/development/` |
|
||||
| Composition diagrams | `docs/architecture/` |
|
||||
| Compatibility changes for a released version | `docs/migrations/` |
|
||||
| One feature specification, plan and tasks | `docs/specs/<domain>/<feature-slug>/` |
|
||||
| One architectural decision | `docs/adr/<domain>/<decision-slug>.md` |
|
||||
|
||||
Use meaningful, lowercase domain and feature slugs. Global numeric prefixes are
|
||||
unnecessary: different domains can add features independently, and branches do
|
||||
not compete for the next sequence number. Check the existing domain directory
|
||||
and linked issues before creating a feature with the same slug.
|
||||
|
||||
Guides describe existing behavior and limitations. Specs define one proposed
|
||||
feature or domain change; an existing guide does not become a specification by
|
||||
moving it into `specs/`.
|
||||
|
||||
## Feature lifecycle
|
||||
|
||||
Create the specification before implementation. Each feature directory contains:
|
||||
|
||||
- `spec.md`: problem, desired behavior, requirements, acceptance and non-goals.
|
||||
- `plan.md`: the single implementation plan, contracts and architectural choices.
|
||||
- `tasks.md`: incremental, testable steps linked to requirements and verification.
|
||||
|
||||
Split independent domains or separately reviewable features before writing their
|
||||
specs. A broad `ROADMAP.md`, when useful, only lists child specifications, order
|
||||
and dependencies; requirements and implementation tasks belong to each child.
|
||||
Cross-repository contracts have one producer-owned specification. Consumers link
|
||||
to that contract instead of maintaining competing copies.
|
||||
|
||||
Record an isolated architectural decision separately in
|
||||
`docs/adr/<domain>/<decision-slug>.md`, with **Status**, **Context**,
|
||||
**Decision** and **Consequences**. Link the ADR from affected specifications.
|
||||
Do not create decision records for routine file moves or invent past decisions.
|
||||
|
||||
Specs do not replace the contribution-scope decision required for a new provider
|
||||
or expanded execution authority. Follow
|
||||
[contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
|
||||
|
||||
## Links and public routes
|
||||
|
||||
Use relative links to physical Markdown files so GitHub readers can follow them.
|
||||
Link repository files outside `docs/` with their full GitHub URLs. Preserve
|
||||
existing heading anchors when moving a guide.
|
||||
|
||||
[`documentation.ts`](https://github.com/morluto/rea/blob/main/docs/.vitepress/documentation.ts)
|
||||
owns categorized site navigation and the existing guides' stable public routes.
|
||||
Update it when adding or moving a guide. VitePress rewrites categorized sources
|
||||
to the existing public paths; its Markdown hook resolves repository links from
|
||||
the physical source and renders them against those public routes. Keep basenames
|
||||
and route assignments unique. Edit links use the physical source file.
|
||||
|
||||
For a move, search for both the basename and its old full path. Update README
|
||||
translations, authored skills, verification instructions, documentation-fact
|
||||
readers and release configuration when they consume that path. Preserve
|
||||
historical release records and references bound to earlier commits or tags.
|
||||
|
||||
## Generated ownership and checks
|
||||
|
||||
The [documentation directory](../README.md#machine-owned-documentation) identifies
|
||||
the machine-owned outputs and their producers. Their existing paths are retained
|
||||
because scripts, package consumers and release history depend on them. Change
|
||||
canonical sources and regenerate; do not fix a generated projection by hand.
|
||||
|
||||
Run `npm run docs:check` for documentation facts and generated consistency, and
|
||||
`npm run verify:docs:routes` to build the complete production site and check local
|
||||
links, anchors and physical edit targets.
|
||||
When a physical path moves, also run the tests that read it. Follow
|
||||
[development checks](testing.md#developer-commands) before requesting review.
|
||||
File renamed without changes.
@@ -181,7 +181,8 @@ When choosing the manual path, leave the automatic main release PR unmerged.
|
||||
Select the application commit from main, create the frozen branch, and prepare
|
||||
its own release PR using the steps below. Merging the main release PR starts
|
||||
automatic publication instead. Close the superseded main PR after the manual
|
||||
release has published and its metadata has been synchronized back to main.
|
||||
release has published, before merging the metadata synchronization PR back
|
||||
to main.
|
||||
|
||||
`release/VERSION` also sets the expected version for the workflow. Use an exact
|
||||
SemVer such as `release/6.1.0` or `release/6.1.0-rc.1`, without build metadata.
|
||||
@@ -192,7 +193,7 @@ registry metadata or changelog disagrees with that version before creating a tag
|
||||
|
||||
Before publishing a candidate, review its changelog's breaking changes and the
|
||||
migration notes for the release being prepared. The
|
||||
[6.1 to 6.2 migration guide](migration-6.2.md) records that release's changes;
|
||||
[6.1 to 6.2 migration guide](../migrations/migration-6.2.md) records that release's changes;
|
||||
it is a historical reference, not the compatibility checklist for later releases.
|
||||
|
||||
REA uses Release Please's `always-bump-minor` strategy: every release increments
|
||||
@@ -349,8 +350,10 @@ tag remains in main's ancestry. Keep the released tag unchanged.
|
||||
|
||||
The synchronization must include `.release-please-manifest.json`,
|
||||
`package.json`, both root versions in `package-lock.json`, `CHANGELOG.md`,
|
||||
`server.json`, both plugin manifests (`.claude-plugin/plugin.json` and
|
||||
`.codex-plugin/plugin.json`), the Codex server declaration (`.codex-plugin/mcp.json`), both
|
||||
`server.json`, the Claude Code and Codex plugin manifests
|
||||
(`.claude-plugin/plugin.json` and `.codex-plugin/plugin.json`), the Codex
|
||||
server declaration (`.codex-plugin/mcp.json`), the Cursor Marketplace manifest
|
||||
(`.cursor-plugin/plugin.json`) and server pin (`mcp.json`), both
|
||||
marketplaces (`.claude-plugin/marketplace.json` and
|
||||
`.agents/plugins/marketplace.json`), and the versioned documentation examples. Run
|
||||
`npm run docs:generate`, `npm run docs:check`, and the release configuration
|
||||
@@ -5,8 +5,8 @@ The [platform tracker](https://github.com/morluto/rea/issues/32),
|
||||
[browser tracker](https://github.com/morluto/rea/issues/39) track capability and
|
||||
verification work. Available tools and setup are documented in the
|
||||
[investigation guides](https://rea.tools/guides/),
|
||||
[installation guide](installation.md), and
|
||||
[generated catalog](mcp-contracts.md#generated-catalog).
|
||||
[installation guide](../getting-started/installation.md), and
|
||||
[generated catalog](../reference/mcp-contracts.md#generated-catalog).
|
||||
|
||||
## Priorities
|
||||
|
||||
@@ -27,7 +27,7 @@ fixture servers; custom orchestration needs a demonstrated requirement.
|
||||
Provider and platform claims require their corresponding
|
||||
[real verification lanes](testing.md#real-toolchain-verification-lanes).
|
||||
See [tool design](tool-design.md) for choosing a tool boundary and
|
||||
[Ghidra semantics](ghidra-provider.md) for the existing deep-provider contract.
|
||||
[Ghidra semantics](../providers/ghidra-provider.md) for the existing deep-provider contract.
|
||||
|
||||
Setup remains additive, reuses existing tools, and requires approval for disclosed
|
||||
configuration and Hopper installation changes. Additional installer choices need
|
||||
@@ -8,7 +8,7 @@ installed-client and agent runs.
|
||||
Historical verification records may refer to `experiments/analysis-efficiency`.
|
||||
Those experiment scripts are available in the [archived source tree](https://github.com/morluto/rea/tree/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency);
|
||||
they are not included in current checkouts. The retained [test support](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/README.md)
|
||||
contains deterministic checkers and fixtures. The [allocation report](verification/analysis-efficiency/mcp-integer-allocation.md)
|
||||
contains deterministic checkers and fixtures. The [allocation report](../verification/analysis-efficiency/mcp-integer-allocation.md)
|
||||
and its original receipt JSON remain documentation, checked by the report producer.
|
||||
|
||||
## Developer commands
|
||||
@@ -382,7 +382,7 @@ instrumentation remains equivalent.
|
||||
|
||||
## CodeQL production scope
|
||||
|
||||
[The production configuration](../.github/codeql/production.yml) excludes
|
||||
[The production configuration](https://github.com/morluto/rea/blob/main/.github/codeql/production.yml) excludes
|
||||
`tests/`, `scripts/fixtures/`, `*.test.*`, `*.fixture.*`, `test_*.py`, and `*_test.py` inputs
|
||||
during extraction. It retains the default queries and the existing `local`
|
||||
threat model. Production source, bridges, runtime scripts, installation and
|
||||
@@ -398,7 +398,7 @@ These exclusions do not change which behavioral tests CI executes. CodeQL's
|
||||
JavaScript/TypeScript, Python, and Actions lanes do not add shell, Java, or Swift
|
||||
analysis; their existing checks remain responsible for those languages.
|
||||
|
||||
[The production workflow](../.github/workflows/codeql.yml) is inactive until a
|
||||
[The production workflow](https://github.com/morluto/rea/blob/main/.github/workflows/codeql.yml) is inactive until a
|
||||
repository administrator disables CodeQL **default setup** and sets the Actions
|
||||
repository variable `REA_CODEQL_ADVANCED` to `true`. Disable default setup first:
|
||||
it rejects advanced CodeQL uploads. Then enable the variable and manually run
|
||||
@@ -602,6 +602,12 @@ For partial Evidence delivery, verify that same-session readback preserves
|
||||
identity and provenance without rerunning the provider. Include a no-writer
|
||||
control that keeps completed Evidence inline and preserves the original failure.
|
||||
|
||||
For a seekable file with a known extent, distinguish a provably impossible wire
|
||||
extent from an independent representation limit. Pair the malformed declaration
|
||||
with a structurally possible over-limit control. A count-only truncated fixture
|
||||
does not establish capacity behavior. This does not require decoding an opaque
|
||||
compressed payload merely to apply its declared expansion limit.
|
||||
|
||||
When producer order is unspecified, synchronize fixtures on the observed request
|
||||
or completion event. A filename, generated key, or fixed delay does not prove
|
||||
which item completed before cancellation. Preserve real producer ordering and
|
||||
@@ -702,16 +708,16 @@ the affected owners and applicable scenarios below. Include temporary snapshots
|
||||
and one-shot commands. Use public CLI/MCP workflows where practical and a
|
||||
controlled production seam for interleavings that cannot be reproduced reliably.
|
||||
|
||||
| Scenario | Assertion and control |
|
||||
| --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Target admitted; startup still pending | Status preserves the readiness distinction and observed startup activity; readiness requires the provider's documented proof. |
|
||||
| First handshake is late, absent, invalid, or recovered during close | Initial and recovery paths retain the same authenticated identity facts. Foreign run/token controls cannot authorize cleanup; authenticated idle/native work survives the unauthenticated listener deadline. |
|
||||
| Launch returns an error with a valid process handle | Cover already-exited and still-live owned processes. Verify observed exit separately from creation; failed termination or unavailable identity remains unresolved. |
|
||||
| Cancellation arrives after workload exit, during result reads or temporary-root removal | Return typed cancellation with available partial observations and actual cleanup status. The non-aborted control succeeds; existing primary/cleanup failures keep their precedence. |
|
||||
| Children have mixed health or cleanup outcomes | Missing peer health does not erase an observed failure or activity. Idle plus unknown stays unknown. Retry a failed child without closing a successful peer again, including after a close-option change. |
|
||||
| Cleanup fails after success or primary failure | Public output retains the completed observation or primary diagnostic, remaining resources, and owner recovery action. Restore the denied resource and verify the owning cleanup retry. |
|
||||
| Cleanup owners recover after an error was returned | Retain the original child and aggregate errors and project them after each owner releases. A successful peer stays released; the aggregate remains incomplete until all owners release. Preserve the first cause even when rejection is `undefined`. |
|
||||
| Target exits without an acknowledgement, or transport disappears while it stays alive | Positive owned exit proof resolves target lifetime; other resources still need cleanup. Lost transport alone does not prove exit. Verify unrelated/attached targets survive and same-connection recovery works when supported. |
|
||||
| Scenario | Assertion and control |
|
||||
| --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Target admitted; startup still pending | Status preserves the readiness distinction and observed startup activity; readiness requires the provider's documented proof. |
|
||||
| First handshake is late, absent, invalid, or recovered during close | Initial and recovery paths retain the same authenticated identity facts. Foreign run/token controls cannot authorize cleanup; authenticated idle/native work survives the unauthenticated listener deadline. |
|
||||
| Launch returns an error with a valid process handle | Cover already-exited and still-live owned processes. Verify observed exit separately from creation; failed termination or unavailable identity remains unresolved. |
|
||||
| Cancellation arrives after workload exit, during result reads or temporary-root removal | Before the documented commit cutoff, return typed cancellation with available partial observations and actual cleanup status. After commit, assert the owning publication outcome and receipt. The non-aborted control succeeds; existing primary/cleanup failures keep their precedence. |
|
||||
| Children have mixed health or cleanup outcomes | Missing peer health does not erase an observed failure or activity. Idle plus unknown stays unknown. Retry a failed child without closing a successful peer again, including after a close-option change. |
|
||||
| Cleanup fails after success or primary failure | Public output retains the completed observation or primary diagnostic, remaining resources, and owner recovery action. Restore the denied resource and verify the owning cleanup retry. |
|
||||
| Cleanup owners recover after an error was returned | Retain the original child and aggregate errors and project them after each owner releases. A successful peer stays released; the aggregate remains incomplete until all owners release. Preserve the first cause even when rejection is `undefined`. |
|
||||
| Target exits without an acknowledgement, or transport disappears while it stays alive | Positive owned exit proof resolves target lifetime; other resources still need cleanup. Lost transport alone does not prove exit. Verify unrelated/attached targets survive and same-connection recovery works when supported. |
|
||||
|
||||
Shared process-supervision controls separate leader exit from output closure.
|
||||
Cover output during held or failed cleanup, bounded retention and drain timeout,
|
||||
@@ -909,7 +915,7 @@ covers deterministic evaluator logic.
|
||||
reconstruction, native-boundary relationships and application graphs without
|
||||
executing fixture PE files. Operator-local manifests and actual ILSpy oracles
|
||||
remain optional, separately reported checks; the real Ghidra NativeAOT lane has
|
||||
its own toolchain prerequisites. See [the managed guide](managed-code-analysis.md)
|
||||
its own toolchain prerequisites. See [the managed guide](../guides/managed/managed-code-analysis.md)
|
||||
for those configurations. Generated completion-ledger checks use the same owning
|
||||
entrypoint and include its verifier/fixture files in their cache inputs.
|
||||
|
||||
@@ -1165,104 +1171,104 @@ public registry, and checks both the primary failure and live descendant absence
|
||||
|
||||
## Real-toolchain verification lanes
|
||||
|
||||
See [Real-toolchain verification lanes](testing-lanes.md#real-toolchain-verification-lanes).
|
||||
See [Real-toolchain verification lanes](../testing-lanes.md#real-toolchain-verification-lanes).
|
||||
|
||||
## Android APK analysis
|
||||
|
||||
See [Android APK analysis](testing-lanes.md#android-apk-analysis).
|
||||
See [Android APK analysis](../testing-lanes.md#android-apk-analysis).
|
||||
|
||||
## Apktool resource decoding
|
||||
|
||||
See [Apktool resource decoding](testing-lanes.md#apktool-resource-decoding).
|
||||
See [Apktool resource decoding](../testing-lanes.md#apktool-resource-decoding).
|
||||
|
||||
## Flutter build identification
|
||||
|
||||
See [Flutter build identification](testing-lanes.md#flutter-build-identification).
|
||||
See [Flutter build identification](../testing-lanes.md#flutter-build-identification).
|
||||
|
||||
## ADB device acquisition
|
||||
|
||||
See [ADB device acquisition](testing-lanes.md#adb-device-acquisition).
|
||||
See [ADB device acquisition](../testing-lanes.md#adb-device-acquisition).
|
||||
|
||||
## Optional NativeAOT Ghidra analysis
|
||||
|
||||
See [Optional NativeAOT Ghidra analysis](testing-lanes.md#optional-nativeaot-ghidra-analysis).
|
||||
See [Optional NativeAOT Ghidra analysis](../testing-lanes.md#optional-nativeaot-ghidra-analysis).
|
||||
|
||||
## IDA MCP adapter
|
||||
|
||||
See [IDA MCP adapter](testing-lanes.md#ida-mcp-adapter).
|
||||
See [IDA MCP adapter](../testing-lanes.md#ida-mcp-adapter).
|
||||
|
||||
## Kuna lane
|
||||
|
||||
See [Kuna lane](testing-lanes.md#kuna-lane).
|
||||
See [Kuna lane](../testing-lanes.md#kuna-lane).
|
||||
|
||||
## DOS Ghidra analysis
|
||||
|
||||
See [DOS Ghidra analysis](testing-lanes.md#dos-ghidra-analysis).
|
||||
See [DOS Ghidra analysis](../testing-lanes.md#dos-ghidra-analysis).
|
||||
|
||||
## Apple Interface Builder archives
|
||||
|
||||
See [Apple Interface Builder archives](testing-lanes.md#apple-interface-builder-archives).
|
||||
See [Apple Interface Builder archives](../testing-lanes.md#apple-interface-builder-archives).
|
||||
|
||||
## Native platform baseline in CI
|
||||
|
||||
See [Native platform baseline in CI](testing-lanes.md#native-platform-baseline-in-ci).
|
||||
See [Native platform baseline in CI](../testing-lanes.md#native-platform-baseline-in-ci).
|
||||
|
||||
### Termux Android browser smoke test
|
||||
|
||||
See [Termux Android browser smoke test](testing-lanes.md#termux-android-browser-smoke-test).
|
||||
See [Termux Android browser smoke test](../testing-lanes.md#termux-android-browser-smoke-test).
|
||||
|
||||
## Packet capture analysis
|
||||
|
||||
See [Packet capture analysis](testing-lanes.md#packet-capture-analysis).
|
||||
See [Packet capture analysis](../testing-lanes.md#packet-capture-analysis).
|
||||
|
||||
## Apple native metadata and UI
|
||||
|
||||
See [Apple native metadata and UI](testing-lanes.md#apple-native-metadata-and-ui).
|
||||
See [Apple native metadata and UI](../testing-lanes.md#apple-native-metadata-and-ui).
|
||||
|
||||
### Firmware adapters
|
||||
|
||||
See [Firmware adapters](testing-lanes.md#firmware-adapters).
|
||||
See [Firmware adapters](../testing-lanes.md#firmware-adapters).
|
||||
|
||||
### JavaScript source recovery
|
||||
|
||||
See [JavaScript source recovery](testing-lanes.md#javascript-source-recovery).
|
||||
See [JavaScript source recovery](../testing-lanes.md#javascript-source-recovery).
|
||||
|
||||
### Captured website source-map lane
|
||||
|
||||
See [Captured website source-map lane](testing-lanes.md#captured-website-source-map-lane).
|
||||
See [Captured website source-map lane](../testing-lanes.md#captured-website-source-map-lane).
|
||||
|
||||
### JavaScript large-output lane
|
||||
|
||||
See [JavaScript large-output lane](testing-lanes.md#javascript-large-output-lane).
|
||||
See [JavaScript large-output lane](../testing-lanes.md#javascript-large-output-lane).
|
||||
|
||||
### Website runtime attribution lane
|
||||
|
||||
See [Website runtime attribution lane](testing-lanes.md#website-runtime-attribution-lane).
|
||||
See [Website runtime attribution lane](../testing-lanes.md#website-runtime-attribution-lane).
|
||||
|
||||
### Offline binary layout
|
||||
|
||||
See [Offline binary layout](testing-lanes.md#offline-binary-layout).
|
||||
See [Offline binary layout](../testing-lanes.md#offline-binary-layout).
|
||||
|
||||
### Offline EVM interface
|
||||
|
||||
See [Offline EVM interface](testing-lanes.md#offline-evm-interface).
|
||||
See [Offline EVM interface](../testing-lanes.md#offline-evm-interface).
|
||||
|
||||
### Recorded crash evidence
|
||||
|
||||
See [Recorded crash evidence](testing-lanes.md#recorded-crash-evidence).
|
||||
See [Recorded crash evidence](../testing-lanes.md#recorded-crash-evidence).
|
||||
|
||||
### Exported environment bundles
|
||||
|
||||
See [Exported environment bundles](testing-lanes.md#exported-environment-bundles).
|
||||
See [Exported environment bundles](../testing-lanes.md#exported-environment-bundles).
|
||||
|
||||
## Agent evaluation and conformance records
|
||||
|
||||
See [Agent evaluation and conformance records](testing-lanes.md#agent-evaluation-and-conformance-records).
|
||||
See [Agent evaluation and conformance records](../testing-lanes.md#agent-evaluation-and-conformance-records).
|
||||
|
||||
## Frida instrumentation
|
||||
|
||||
See [Frida instrumentation](testing-lanes.md#frida-instrumentation).
|
||||
See [Frida instrumentation](../testing-lanes.md#frida-instrumentation).
|
||||
|
||||
### Offline WASM artifacts
|
||||
|
||||
See [Offline WASM artifacts](testing-lanes.md#offline-wasm-artifacts).
|
||||
See [Offline WASM artifacts](../testing-lanes.md#offline-wasm-artifacts).
|
||||
@@ -60,23 +60,11 @@ or ordinary script path when available. Identify the specific gap and what a
|
||||
new or extended contract would add; upstream capability alone does not require
|
||||
a new tool or equal support from every provider.
|
||||
|
||||
Exercise the task through REA as a whole and compare relevant provider behavior
|
||||
with an independent artifact or format observation where possible. Use the
|
||||
caller's selected target and authority. Record the provider version, target and
|
||||
observed coverage, and distinguish an unsupported operation from an empty
|
||||
result or an untested path.
|
||||
|
||||
Include the boundary cases that could change the answer: provider labels versus
|
||||
source facts, addresses versus file locations, partial results, and a useful
|
||||
recovery after failure or session close when those apply to the task. Check that
|
||||
composed results keep each observation's identity, limitations and source
|
||||
association. Existing real-provider lanes and focused tests establish their
|
||||
selected scenarios; they do not by themselves establish a different workflow or
|
||||
provider equivalence. If a provider cannot be exercised, report that limitation
|
||||
and continue with the available evidence. Measure agent tool selection only with
|
||||
an actual agent evaluation; a scripted tool sequence does not establish it. Use
|
||||
[testing guidance](testing.md#provider-backed-workflow-verification) to select
|
||||
the cases that establish the proposed contract.
|
||||
Use the caller's selected target and authority. Follow
|
||||
[provider-backed workflow verification](testing.md#provider-backed-workflow-verification)
|
||||
for independent observations, boundary controls and reporting unavailable
|
||||
provider coverage. A scripted tool sequence does not establish agent selection;
|
||||
that claim requires an actual agent evaluation.
|
||||
|
||||
## Audit existing tools
|
||||
|
||||
@@ -94,14 +82,10 @@ identity, session lifetime and available alternatives. Matching text between
|
||||
two renderers does not establish that their shared advice is correct or that an
|
||||
agent selected the workflow.
|
||||
|
||||
Exercise the selected public workflows, including the steps before and after a
|
||||
tool call. Check task selection, input choices, returned facts and their source,
|
||||
partial or unsupported outcomes, and applicable session and recovery behavior.
|
||||
Compare native provider output and an independent artifact observation where
|
||||
available. Inspect existing tests and real-provider lanes for the scenario they
|
||||
actually cover; a passing test does not establish a different task or host.
|
||||
Use [workflow verification](testing.md#provider-backed-workflow-verification)
|
||||
for counterexamples and lifecycle controls relevant to the audit.
|
||||
Exercise each selected workflow before and after the tool call. Use
|
||||
[workflow verification](testing.md#provider-backed-workflow-verification) to
|
||||
check returned facts, partial outcomes, recovery and the scope of existing
|
||||
provider evidence.
|
||||
|
||||
Report working routes as well as findings. Separate a reproduced contract bug,
|
||||
an intentional limitation, a design opportunity and an unverified hypothesis;
|
||||
@@ -109,6 +93,38 @@ include the smallest useful reproduction, impact and coverage limits. Do not
|
||||
claim autonomous-agent selection from a scripted trace. Check related reports
|
||||
before proposing another issue, and avoid treating lack of API parity as a bug.
|
||||
|
||||
### Write descriptions for selection and a valid first call
|
||||
|
||||
Start with the analyst task, selected source and useful result. Put material
|
||||
effects and the deciding boundary between neighboring tools early. Progressive
|
||||
discovery previews the first 240 characters, so those characters should identify
|
||||
the operation without requiring the rest of its reference documentation.
|
||||
|
||||
Keep exact call syntax where the advertised schema cannot convey it. The compact
|
||||
input profile strips most annotation prose and can reduce nested controls to
|
||||
`{type: "object"}`. Moving selectors into parameter descriptions alone does not
|
||||
make them available to those clients. For example, retained pseudocode needs
|
||||
both `source.kind: "retained-evidence"` and `view: {kind: "native", facet:
|
||||
"pseudocode"}`; a short exact example can be more useful than many paragraphs.
|
||||
Use the producer's handoff for other supported views rather than maintaining
|
||||
another supposedly exhaustive list in the shared consumer description.
|
||||
|
||||
Keep configuration tutorials and detailed matching algorithms with their owning
|
||||
parameter contracts, product guides and actionable failures. Retain applicability,
|
||||
source identity, uncertainty, authority and cleanup facts needed to choose or
|
||||
safely continue the operation. An unresolved symbol lookup is not a completed
|
||||
reference search; an exhausted page does not establish absence in unexamined
|
||||
source. Shorter copy must preserve these distinctions.
|
||||
|
||||
For wording changes, use a small set of representative agent tasks at the affected
|
||||
boundaries. Hold schemas, tool order, fixtures and host configuration fixed when
|
||||
comparing copy; add repetitions only for failures or ambiguous outcomes. Score
|
||||
actual arguments, corrective calls and supported final claims separately. A
|
||||
correct eventual answer can conceal malformed-call loops, while a valid call
|
||||
can still produce an overstated answer. Report whole-task usage, including
|
||||
retries and cached input, rather than treating fewer description bytes as proof
|
||||
of lower token cost.
|
||||
|
||||
## Design recovery around the failure
|
||||
|
||||
Distinguish a request that violates its input contract from a valid selection
|
||||
@@ -178,14 +194,18 @@ does not establish another instance.
|
||||
provider-observed direct relationship set. `complete` means that set was
|
||||
returned without a known read limit; it does not claim every possible runtime
|
||||
edge. Provider and execution caveats stay in `limitations` and do not imply
|
||||
incomplete relationship coverage. Missing coverage remains `unknown`.
|
||||
incomplete relationship coverage. Directional `discovery` separately describes
|
||||
whether applicable static relationships could have been missed by discovery;
|
||||
missing discovery is unknown. Preserve paths supported by observed edges,
|
||||
and qualify negative and shortest-path claims by discovery scope. Missing
|
||||
relationship coverage remains `unknown`. Computed candidates and unestablished incoming targets qualify directional discovery; they do not by themselves mean that recorded edges were omitted.
|
||||
- Declare process, filesystem, network, UI, and mutation effects truthfully;
|
||||
enforce the requested target and lifecycle.
|
||||
- Add limits only for real format, protocol, authority, or measured resource
|
||||
constraints. Report pagination, truncation, cancellation, and partial failure.
|
||||
An assumed agent budget is insufficient justification for a limit. Name the
|
||||
ceiling and its recovery as described in
|
||||
[admission limits and recovery](design/limits-and-recovery.md).
|
||||
[admission limits and recovery](../design/limits-and-recovery.md).
|
||||
|
||||
### Compose declared Evidence inputs
|
||||
|
||||
@@ -229,8 +249,8 @@ MCP supplies the current ledger lookup. CLI commands that advertise
|
||||
`--evidence-bundle` use one explicitly selected, authenticated bundle for the
|
||||
command's lifetime. Keep cleanup with that owner. A retained reference does not
|
||||
grant filesystem access: file selectors remain separate, explicit contracts
|
||||
where supported. See [retained Evidence inputs](mcp-contracts.md#retained-application-evidence-inputs)
|
||||
and [CLI reuse](cli.md#import-export-and-compare-evidence).
|
||||
where supported. See [retained Evidence inputs](../reference/mcp-contracts.md#retained-application-evidence-inputs)
|
||||
and [CLI reuse](../getting-started/cli.md#import-export-and-compare-evidence).
|
||||
|
||||
### Interpret boundary facts once
|
||||
|
||||
@@ -344,13 +364,15 @@ stages that never ran.
|
||||
and resource release separate. A valid process handle, successful launcher,
|
||||
or selected target establishes only its own fact. Publish available identity
|
||||
and exit observations even when launch fails; retain unknown ownership.
|
||||
- Process exit does not establish output closure. The shared process supervisor
|
||||
retains at most 64 KiB of additional diagnostics from owned stop entry,
|
||||
including pending cleanup and retries, and drains for up to one second after
|
||||
verified stop. It marks output incomplete when closure cannot be
|
||||
observed and releases owned event-loop references without changing the proven
|
||||
process-cleanup outcome. Failed process cleanup retains capture for retry;
|
||||
borrowed processes and streams retain their own lifetime and reference state.
|
||||
- Process exit does not establish output closure. The
|
||||
[shared supervisor](https://github.com/morluto/rea/blob/main/src/process/ProviderProcess.ts)
|
||||
owns diagnostic retention and bounded draining through cleanup and retries.
|
||||
Report incomplete output separately from process cleanup. After verified stop
|
||||
and bounded draining, release owned event-loop references even if output closure
|
||||
stays unconfirmed. Keep the output capture owned for retry while process cleanup
|
||||
is unresolved. Borrowed processes and streams retain their own lifetime and
|
||||
reference state; select the matching
|
||||
[supervision controls](testing.md#lifecycle-regressions) when changing this code.
|
||||
- Authenticate and capture identity through the same boundary on initial and
|
||||
recovery handshakes. Bound abandoned unauthenticated listeners where the
|
||||
protocol permits it; that deadline must not expire authenticated idle sessions
|
||||
@@ -369,8 +391,9 @@ stages that never ran.
|
||||
analysis snapshot is part of `open_binary`; cancellation keeps that operation
|
||||
and the caller's reason through the shared JSON reader.
|
||||
- For cancellable observations, carry cancellation through result consumption
|
||||
and awaited cleanup. Before returning success, check the signal after the
|
||||
last awaited finalization step.
|
||||
and awaited cleanup. Check the signal after the last awaited finalization step
|
||||
unless the operation has passed its documented commit cutoff. Preserve the
|
||||
owning operation's committed outcome and receipt after that cutoff.
|
||||
Retain collected observations under typed cancellation. Preserve an existing
|
||||
primary failure and let incomplete cleanup retain its own classification.
|
||||
Preserve the caller's abort reason as the typed cancellation error's cause,
|
||||
@@ -543,5 +566,5 @@ metadata to either adapter.
|
||||
When discoverability changes, evaluate representative broad, direct, and negative
|
||||
tasks on the intended host. Record host/model/catalog, selected tools, arguments,
|
||||
errors, retries, and task outcomes. Schema validity and keyword heuristics do not
|
||||
establish successful tool selection. See [mcp-contracts.md](mcp-contracts.md) for
|
||||
establish successful tool selection. See [mcp-contracts.md](../reference/mcp-contracts.md) for
|
||||
shipped runtime behavior.
|
||||
+14
-1
@@ -50,7 +50,8 @@ explicitly with the managed tools to establish PE/CLI facts.
|
||||
|
||||
## Admission and containment
|
||||
|
||||
Metadata is range-read with a 64 KiB cache. Policy budgets are 16 MiB of
|
||||
Metadata is range-read with a cache of at most 64 KiB, bounded by the remaining
|
||||
selected metadata allowance. Default policy budgets are 16 MiB of
|
||||
manifest metadata, 100,000 files, 64 KiB per UTF-8 string, 512 MiB of stored or
|
||||
expanded bytes per file, 256 path parts per member, 250,000 distinct in-memory
|
||||
path-tree nodes, and 2 GiB of aggregate expanded bytes. The node budget counts
|
||||
@@ -60,6 +61,18 @@ streams 64 KiB chunks and reuses completed probe facts during extraction;
|
||||
ordinary native files without a marker incur one extra
|
||||
read pass before host hashing.
|
||||
|
||||
Both artifact tools accept `artifact_limits.dotnet_bundle` to select
|
||||
`metadata_bytes`, `entry_count`, `string_bytes`, `member_bytes`,
|
||||
`expanded_bytes`, `path_parts`, and `path_nodes` for the next operation. The
|
||||
string allowance applies to every manifest string, including the bundle ID.
|
||||
For example, retry a complete bundle whose manifest exceeds the default with
|
||||
`{"artifact_limits":{"dotnet_bundle":{"metadata_bytes":33554432}}}`.
|
||||
The CLI accepts the same selection through `--artifact-limits` JSON. Omitted
|
||||
fields preserve defaults; errors name the exact selected control, effective
|
||||
value and required minimum when known. No REA restart is needed. Summary
|
||||
delivery and paging do not change producer admission, and larger reader
|
||||
budgets do not raise MCP response or Node representation limits.
|
||||
|
||||
Extraction hashes the whole host again through the same descriptor used to read
|
||||
members and compares it with the inventory's root digest before creating an
|
||||
output directory. This adds a full-host read pass to extraction, including when
|
||||
|
||||
+17
-1
@@ -44,6 +44,22 @@ file memory regions and paths (`NT_FILE`), fault signal and code (`NT_SIGINFO`),
|
||||
auxiliary vectors (`NT_AUXV`) across 32-bit and 64-bit images. Other owners or unparsed
|
||||
notes keep `core` null.
|
||||
|
||||
Linux `CORE` decoding requires System V/unspecified (0) or GNU/Linux (3)
|
||||
OSABI with ABI version zero. An unspecified OSABI alone does not establish
|
||||
Linux origin. An explicitly different OSABI or ABI version leaves `core` null
|
||||
and its Linux-specific `type_name` unknown, with a limitation explaining the
|
||||
profile. Owner/type numbers, descriptor ranges and hashes, GNU-owned notes,
|
||||
and complete table-examination coverage remain available.
|
||||
|
||||
For Android `NT_ANDROID_TYPE_IDENT` (`.note.android.ident`, owner `Android`,
|
||||
type `1`) notes, `android.android_api` is decoded from both legacy 4-byte
|
||||
descriptors (where `ndk_version` and `ndk_build_number` remain null) and modern
|
||||
132-byte or extended NDK descriptors, which also populate `android.ndk_version`
|
||||
and `ndk_build_number` when their 64-byte slots hold NUL-padded
|
||||
printable ASCII. Truncated or non-ASCII Android descriptors degrade `android`
|
||||
to null while preserving the raw descriptor digest and preview, and other
|
||||
owners keep `android` null.
|
||||
|
||||
`PT_NOTE` and `SHT_NOTE` are examined independently. The same file range may
|
||||
appear twice when both tables declare it.
|
||||
|
||||
@@ -54,7 +70,7 @@ the image never carried notes in another container or overlay.
|
||||
|
||||
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
|
||||
32 note entries are examined by default across every table. Retained note
|
||||
names, section names, GNU descriptor text, and core dump string representations share a scan budget based on
|
||||
names, section names, GNU descriptor text, core dump string representations, and Android ident strings share a scan budget based on
|
||||
the source bytes plus any decoded section-name table, capped at
|
||||
`max_file_bytes`. The raw source and decoded table must also fit together
|
||||
within that ceiling. If admission fails, the error reports the required
|
||||
|
||||
+1
-1
@@ -23,7 +23,7 @@ and consider setting both `REA_MCP_INPUT_SCHEMA_PROFILE` and
|
||||
advertised output schema to 4,096 bytes by relaxing nested client validation;
|
||||
REA still validates complete results against the canonical contracts. Pair
|
||||
this with retained summaries and selected inventory views to limit the context
|
||||
needed for one question. See [MCP contracts](mcp-contracts.md) for the profiles
|
||||
needed for one question. See [MCP contracts](reference/mcp-contracts.md) for the profiles
|
||||
and their client compatibility tradeoffs.
|
||||
|
||||
The layout facts follow ESP-IDF v6.1 (`esp_app_format.h`, `esp_app_desc.h`,
|
||||
|
||||
+7
-5
@@ -22,9 +22,10 @@ use the current projection identity; there is no pre-coverage FIT summary varian
|
||||
A FIT is a flattened devicetree (FDT v16 or v17) with a top-level `/images`
|
||||
node. Recognition walks the structure block within the first 4 KiB of the file
|
||||
only; the file name does not matter. A devicetree whose `/images` node begins
|
||||
after 4 KiB, a devicetree without `/images`, and any other FDT version stay on
|
||||
the plain devicetree route: `open_binary` refuses them and
|
||||
[`inspect_devicetree`](devicetree.md) still parses them. A U-Boot control DTB
|
||||
after 4 KiB or a devicetree without `/images` stays on the plain devicetree
|
||||
route: `open_binary` refuses it and [`inspect_devicetree`](devicetree.md)
|
||||
can parse it when its FDT version is 16 or 17. Other FDT versions are not
|
||||
recognized as FIT and are also unsupported by `inspect_devicetree`. A U-Boot control DTB
|
||||
that contains `/images` is treated as a FIT. The container format is `fit`.
|
||||
|
||||
Recognition is a probe. The complete devicetree parse decides, so a file whose
|
||||
@@ -124,8 +125,9 @@ Failures apply to the whole file and return no partial inventory.
|
||||
|
||||
The devicetree, including any embedded image data, must be at most 16 MiB; a
|
||||
larger `totalsize` fails as a memory limit. Store large images externally
|
||||
(`mkimage -E`): external data of any size is streamed from the file and never
|
||||
loaded into memory. A FIT may declare at most 1,024 images and 1,024
|
||||
(`mkimage -E`): external image bytes are streamed from the file within their declared
|
||||
32-bit sizes and the resource bounds below, without loading the whole image
|
||||
into memory. A FIT may declare at most 1,024 images and 1,024
|
||||
configurations, and an image at most 8 hash nodes; the devicetree parser's
|
||||
node, property and depth bounds also apply. The sum of declared image sizes
|
||||
is bounded by the shared archive expansion budget. These are fixed resource
|
||||
|
||||
@@ -1,31 +0,0 @@
|
||||
# Frida instrumentation
|
||||
|
||||
REA exposes Frida device and process discovery, attach or spawn sessions, and caller-authored JavaScript instrumentation through the CLI and MCP. The Node bindings are an optional package dependency and load only when a Frida operation runs. REA does not install Frida server on a target.
|
||||
|
||||
For remote devices, pass a host with an optional port (`host[:port]`; bracket IPv6 addresses) and any token, certificate, origin, or keepalive options on that operation. The keepalive interval is measured in seconds, matching Frida Core 17.23.2. Authentication values are held only for the active call or session and are excluded from Evidence. Do not embed credentials in the endpoint address.
|
||||
|
||||
MCP exposes `list_frida_devices`, `list_frida_processes`, `start_frida_session`, `load_frida_script`, `resume_frida_session`, `unload_frida_script`, `frida_session_status`, and `close_frida_session`. A spawned process starts paused so scripts can load first. Closing unloads session scripts and detaches. If a spawn remains paused, REA resumes it only after script cleanup or session detach confirms the instrumentation is released; if both cleanup paths fail, the target stays paused and the cleanup error is retained. REA never kills the target. MCP sessions end with the server process. The one-shot CLI command `instrument-with-frida` accepts one inline script or script file, observes messages for a bounded duration, and returns Evidence with the source hash and captured messages.
|
||||
|
||||
`instrument-with-frida --json` returns the Evidence record directly, matching MCP. Consumers of the earlier `{evidence, cleanup_error}` CLI wrapper should read the record at the JSON root and the session cleanup observation at `normalized_result.cleanup_error`. If final provider cleanup fails or cancellation arrives during it, the typed error retains completed Evidence in `details.partial_observation`. Discovery commands retain their device or process report in the same field.
|
||||
|
||||
For state and capture-coverage checks, pass `{"session_id":"SESSION_ID","detail":"summary"}` to `frida_session_status`. Each call reads the observed session state and retains the complete observation. The reply is an `inspect_analysis_view` Evidence summary with the session identity, state, scripts, `messages_retained`, and `messages_truncated`; message payloads remain in its `parent_evidence_id`. The count covers retained queued messages, not every target event. Omitted `detail` or `detail=complete` preserves the existing complete reply.
|
||||
|
||||
A summary's selected fields look like this (excerpt):
|
||||
|
||||
```json
|
||||
{
|
||||
"summary": {
|
||||
"state": "running",
|
||||
"messages_retained": 50,
|
||||
"messages_truncated": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Use `get_evidence_bundle` with `detail="summary"` and `filters={"evidence_id":"PARENT_ID"}` to discover parent metadata without message payloads. Filters apply only to summary delivery. Use `export_evidence_bundle` for the complete canonical bundle, then select the record whose `evidence_id` matches `normalized_result.parent_evidence_id` from the status summary. To use the existing CLI `inspect-analysis-view`, extract that record into a standalone Evidence JSON file and select it as an Evidence-file source with `view={"kind":"summary"}`. Reading retained Evidence does not refresh a live session or verify a process after exit or PID reuse.
|
||||
|
||||
Failed startup cleanup retains acquired Frida handles, including a paused spawn that has not attached, for later cleanup retries. Its remote connection stays leased until session detach and target resume complete. CLI commands await provider cleanup before returning; MCP retains these resources for cleanup when its server closes. Server shutdown stops admitting new Frida operations and waits for admitted startup, discovery, and queued work before releasing their resources. A failed cleanup remains reported and can be retried by closing the server again, rather than treating an unregistered session as fully released.
|
||||
|
||||
Frida scripts execute inside the selected target with that process's memory, filesystem, and network authority. Local device lookup waits 1000 ms. This release has no runtime override for that lookup; another REA timeout does not raise it. Remote endpoints do not use that lookup. Message capture is bounded to 16 MiB and reports truncation. Metadata, type, and text field caps, and the JSON node cap, are also fixed. Evidence names a cap when that observation actually dropped data. This release has no runtime override for those capture caps. Script source is not stored in Evidence; its selected path and SHA-256 digest are. For a file, `source_sha256` identifies the original selected bytes. Invalid UTF-8 continues to use replacement decoding; Evidence then records that limitation and `raw_result.source_representation` with the encoding, decoding mode, and `loaded_source_sha256` of the UTF-8 string passed to Frida. Valid UTF-8 files and inline scripts do not need this additional representation fact. Evidence retains non-authentication target selection such as device or endpoint, mode, program or PID, and spawn arguments. Authentication values remain excluded. Results report the observed session and messages, while unsupported platform metadata remains unknown. A request cancelled before a Frida operation starts has no provider effects. If cancellation arrives during a native Frida call, REA waits for that call to return, then closes any session it created and returns the observed session as a partial observation. A cancelled script load on an existing session releases only its new script before queued operations can run; the session and previously loaded scripts remain available. If cancellation is observed when the native call returns, partial Evidence retains the source identity and captured messages even if that call rejects, including messages emitted during awaited cleanup. The caller's cancellation reason remains primary and a native rejection is retained as a sanitized diagnostic. A failure already observed before cleanup stays primary if cancellation arrives during cleanup. The cancellation cutoff is REA's observation of native completion, rather than a claim about unobservable native event order. `script_id` is REA's load-attempt identity. When creation rejects without returning a handle, `script_cleanup: "no_handle"` means REA acquired no handle to unload; it does not prove that the engine performed no work. Once the provider commits a successful load, later request cancellation does not unload that script; use `unload_frida_script` to release it. If script cleanup fails, the script remains owned and a paused target cannot resume until the pending instrumentation is released. Queued script loads, resumes, and unloads check cancellation before dispatching native effects. Cancellation Evidence separately reports whether a target resume was observed, whether instrumentation cleanup completed, and that final target liveness remains unknown. REA does not kill the target. Remote discovery retains its observations and reports a separate cleanup error if the temporary connection cannot be removed.
|
||||
|
||||
The [Frida Core API](https://frida.re/docs/frida-core-api/) documents device management, sessions, and script loading; the [JavaScript API](https://frida.re/docs/javascript-api/) documents target-side instrumentation. [Luma](https://luma.frida.re/) is Frida's standalone GUI; REA does not automate or control it. `npm run verify:frida` requires a Linux host with the optional Frida binding and uses a short-lived local Node.js fixture created by the verifier. Add `-- --address HOST:PORT --pid PID` to run the separate remote attach lane against a caller-selected target. Mock tests do not establish real device/server compatibility.
|
||||
@@ -4,7 +4,7 @@ Use the CLI for a direct inspection or a script. It uses the same application
|
||||
workflows and evidence contracts as REA's MCP server. Each CLI invocation is a
|
||||
separate process; save results when you want to use them in a later command.
|
||||
For a pinned, noninteractive workflow with retained reports, see
|
||||
[static analysis in GitHub Actions](ci-analysis.md).
|
||||
[static analysis in GitHub Actions](../ci-analysis.md).
|
||||
|
||||
## Run a command
|
||||
|
||||
@@ -19,7 +19,7 @@ Use an extracted application directory or an ASAR as the target. Static
|
||||
JavaScript analysis returns the application graph and Evidence directly.
|
||||
Generic `rea analyze PATH` selects this workflow for directories and `.asar`
|
||||
files when neither `--provider` nor `--snapshot` is supplied. See
|
||||
[JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
|
||||
[JavaScript artifact reconstruction](../guides/javascript/javascript-artifact-reconstruction.md)
|
||||
for results, integrity checks and coverage.
|
||||
|
||||
The complete application Evidence can be hundreds of megabytes for a real
|
||||
@@ -59,7 +59,7 @@ For example, a 334 MB Obsidian application Evidence file yields a summary view
|
||||
of about 10 KB. Replace the view with
|
||||
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
|
||||
module page. See
|
||||
[JavaScript application workflows](javascript-application-workflows.md) for
|
||||
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md) for
|
||||
views, feature traces and comparisons that take the same saved Evidence.
|
||||
|
||||
Add `--timings` to a command to write one JSON timing record to stderr. For
|
||||
@@ -226,7 +226,7 @@ saving the complete original Evidence in a new bundle. Summary requires that
|
||||
explicit destination; existing files are preserved. Later
|
||||
`inspect-analysis-view` calls can page the exact parent from the saved bundle
|
||||
without reopening the assembly. See
|
||||
[CLI summary and saved member rows](managed-code-analysis.md#cli-summary-and-saved-member-rows)
|
||||
[CLI summary and saved member rows](../guides/managed/managed-code-analysis.md#cli-summary-and-saved-member-rows)
|
||||
for the complete command sequence. Omitted or `--detail complete` retains the
|
||||
full inline response.
|
||||
|
||||
@@ -234,8 +234,8 @@ Choose the workflow from the artifact's contents rather than its `.exe` or
|
||||
`.dll` extension. The managed parser requires PE/CLI metadata; it does not
|
||||
unpack .NET single-file hosts or decode IL2CPP. For the image, type and method
|
||||
names in a Unity `global-metadata.dat`, use
|
||||
[IL2CPP metadata](il2cpp-metadata.md). NativeAOT method bodies require native
|
||||
analysis. See the [managed-code guide](managed-code-analysis.md)
|
||||
[IL2CPP metadata](../il2cpp-metadata.md). NativeAOT method bodies require native
|
||||
analysis. See the [managed-code guide](../guides/managed/managed-code-analysis.md)
|
||||
for supported deployment forms, selection semantics, and coverage limits.
|
||||
|
||||
## SQLite snapshots
|
||||
@@ -243,7 +243,7 @@ for supported deployment forms, selection semantics, and coverage limits.
|
||||
For a local database snapshot, use `rea inspect-sqlite-database PATH --json`.
|
||||
Add `--table NAME --row-limit 20` to read a bounded sample from an ordinary
|
||||
table. This command does not require a native analysis provider. See
|
||||
[SQLite database snapshots](sqlite-databases.md) for database/WAL identity,
|
||||
[SQLite database snapshots](../guides/data/sqlite-databases.md) for database/WAL identity,
|
||||
source-file handling and row-value semantics.
|
||||
|
||||
## Native analysis
|
||||
@@ -256,7 +256,7 @@ rea inspect-go-binary /absolute/path/to/program --json
|
||||
```
|
||||
|
||||
The shared CLI/MCP workflow reports compiler version, modules, replacements,
|
||||
build settings, original bytes and SHA-256. See [Go binaries](go-binaries.md)
|
||||
build settings, original bytes and SHA-256. See [Go binaries](../guides/native/go-binaries.md)
|
||||
for supported containers and absent or malformed metadata.
|
||||
|
||||
Inspect a Godot PCK without running the game:
|
||||
@@ -267,7 +267,7 @@ rea extract-artifact game.pck --json
|
||||
```
|
||||
|
||||
Formats 2, 3 and 4 retain header, directory and integrity facts. Compact delivery
|
||||
keeps complete Evidence in the requested bundle. See [Godot PCK](godot-pck.md)
|
||||
keeps complete Evidence in the requested bundle. See [Godot PCK](../godot-pck.md)
|
||||
for embedded packs, batch queries and unknown encrypted coverage.
|
||||
|
||||
Inspect a UnityFS AssetBundle without running Unity:
|
||||
@@ -278,7 +278,7 @@ rea extract-artifact content.bundle --json
|
||||
```
|
||||
|
||||
Format versions 6 through 8 retain header, block and directory facts with
|
||||
none/LZ4/LZ4HC block decompression. See [UnityFS](unityfs.md) for encryption
|
||||
none/LZ4/LZ4HC block decompression. See [UnityFS](../unityfs.md) for encryption
|
||||
and LZMA coverage boundaries.
|
||||
|
||||
Inspect a WeChat mini-program package without WeChat or an unpacker:
|
||||
@@ -290,7 +290,7 @@ rea extract-artifact app.wxapkg --json
|
||||
|
||||
Plaintext packages retain header, table and member facts with verbatim member
|
||||
bytes. `V1MMWX` packages are detected as encrypted coverage and refused on
|
||||
extraction. See [wxapkg](wxapkg.md) for layout and coverage boundaries.
|
||||
extraction. See [wxapkg](../wxapkg.md) for layout and coverage boundaries.
|
||||
|
||||
Inspect an ESP32-family app or bootloader image without esptool:
|
||||
|
||||
@@ -301,7 +301,7 @@ rea extract-artifact firmware.bin --json
|
||||
|
||||
The inventory lists each segment with its ESP-IDF load classification, the
|
||||
checksum and SHA-256 results, the descriptor and any Secure Boot v2 signature
|
||||
blocks. See [ESP images](esp-image.md) for recognition, coverage and refusals.
|
||||
blocks. See [ESP images](../esp-image.md) for recognition, coverage and refusals.
|
||||
|
||||
Inspect an ESP flash dump, or one read from a flash address other than 0:
|
||||
|
||||
@@ -313,7 +313,7 @@ rea extract-artifact flash.bin --json
|
||||
|
||||
The inventory lists the bootloader, the partition table and each partition,
|
||||
with ESP image facts for app partitions and the OTA records. See
|
||||
[ESP flash images](esp-flash-image.md) for table location, inputs and limits.
|
||||
[ESP flash images](../esp-flash-image.md) for table location, inputs and limits.
|
||||
|
||||
Inspect a Node.js SEA carrier without running it:
|
||||
|
||||
@@ -322,7 +322,7 @@ rea inspect-artifact ./application --json
|
||||
rea extract-artifact ./application --json
|
||||
```
|
||||
|
||||
See [Node.js SEA](node-sea.md) for supported layouts, script and asset paths,
|
||||
See [Node.js SEA](../node-sea.md) for supported layouts, script and asset paths,
|
||||
fuse observations, and the real-fixture verification lane.
|
||||
|
||||
Inventory and extract Intel HEX or S-record regions:
|
||||
@@ -332,7 +332,7 @@ rea inspect-artifact ./firmware.hex --json
|
||||
rea extract-artifact ./firmware.hex --json
|
||||
```
|
||||
|
||||
See [Intel HEX and S-record images](hex-records.md) for recognition, region
|
||||
See [Intel HEX and S-record images](../hex-records.md) for recognition, region
|
||||
names, metadata and limits.
|
||||
|
||||
Inspect and extract stored U-Boot legacy uImage payloads and components:
|
||||
@@ -342,7 +342,7 @@ rea inspect-artifact ./firmware.bin --detail summary --evidence_output firmware.
|
||||
rea extract-artifact ./firmware.bin --json
|
||||
```
|
||||
|
||||
See [U-Boot uImage](uimage.md) for verified CRCs, physical component ranges,
|
||||
See [U-Boot uImage](../uimage.md) for verified CRCs, physical component ranges,
|
||||
unknown names, resource limits and retained views. Compressed bytes stay stored.
|
||||
|
||||
`inspect-artifact` and `extract-artifact` also read ZIP members compressed with
|
||||
@@ -361,7 +361,7 @@ rea inspect-artifact ./firmware.uf2 --json
|
||||
rea extract-artifact ./firmware.dfu --json
|
||||
```
|
||||
|
||||
See [UF2 and DfuSe images](uf2-dfuse.md) for recognition, families, numbering,
|
||||
See [UF2 and DfuSe images](../uf2-dfuse.md) for recognition, families, numbering,
|
||||
the DfuSe suffix CRC and limits.
|
||||
|
||||
Inventory and extract U-Boot FIT images:
|
||||
@@ -371,7 +371,7 @@ rea inspect-artifact ./boot.itb --json
|
||||
rea extract-artifact ./boot.itb --json
|
||||
```
|
||||
|
||||
See [U-Boot FIT images](fit-images.md) for recognition, data sources, hashes
|
||||
See [U-Boot FIT images](../fit-images.md) for recognition, data sources, hashes
|
||||
and limits.
|
||||
|
||||
Inspect local model declarations without loading weights:
|
||||
@@ -385,7 +385,7 @@ Default hashing reads only headers and leaves full-file identity unknown.
|
||||
`--hash full` streams the complete file for SHA-256. Both return typed metadata,
|
||||
original tensor declarations and derived totals. MCP summaries retain complete
|
||||
Evidence for filtered metadata, tensor and type-total pages; CLI compact delivery
|
||||
uses `--detail summary --evidence_output FILE`. See [model files](model-artifacts.md).
|
||||
uses `--detail summary --evidence_output FILE`. See [model files](../model-artifacts.md).
|
||||
|
||||
Inspect Unity IL2CPP metadata names without Unity or a disassembler:
|
||||
|
||||
@@ -402,7 +402,7 @@ method instances and `--ghidra-seeds PATH` then writes Ghidra function seeds. It
|
||||
with retained Evidence; query indexed rows with filters, batch indices and
|
||||
selected fields. The CLI defaults to complete Evidence;
|
||||
`--detail summary --evidence_output FILE` exports the full result to a new file
|
||||
and returns a compact overview. See [IL2CPP metadata](il2cpp-metadata.md).
|
||||
and returns a compact overview. See [IL2CPP metadata](../il2cpp-metadata.md).
|
||||
|
||||
Inspect an embedded cargo-auditable Rust dependency list without installing
|
||||
Rust or configuring a disassembler:
|
||||
@@ -414,7 +414,7 @@ rea inspect-rust-binary /absolute/path/to/program --json
|
||||
The same workflow reports package names, versions, sources, the root package,
|
||||
dependency edges, and the section's file range. A missing `.dep-v0` section is
|
||||
null metadata, not proof that the binary has no dependencies. See
|
||||
[Rust binaries](rust-binaries.md).
|
||||
[Rust binaries](../rust-binaries.md).
|
||||
|
||||
Inspect a PE Authenticode certificate table without executing the image or
|
||||
verifying signatures:
|
||||
@@ -426,7 +426,7 @@ rea inspect-pe-certificates /absolute/path/to/application.exe --json
|
||||
The workflow reports `WIN_CERTIFICATE` entries and, for PKCS#7 signedData,
|
||||
embedded X.509 serial, issuer, subject, and validity fields. A missing security
|
||||
directory is observed absence of that table. See
|
||||
[PE Authenticode certificate tables](pe-certificates.md).
|
||||
[PE Authenticode certificate tables](../pe-certificates.md).
|
||||
|
||||
Inspect PE import and delay-load tables without executing the image:
|
||||
|
||||
@@ -437,7 +437,20 @@ rea inspect-pe-imports /absolute/path/to/application.exe --json
|
||||
The workflow reports DLL names, named and ordinal functions, delay-load
|
||||
descriptors, and a pefile-compatible import hash of the standard import table.
|
||||
A missing import directory is observed absence of that table. See
|
||||
[PE import and delay-load tables](pe-imports.md).
|
||||
[PE import and delay-load tables](../pe-imports.md).
|
||||
|
||||
Inspect a Windows Shell Link (`.lnk`) without resolving or executing the
|
||||
target:
|
||||
|
||||
```bash
|
||||
rea inspect-windows-shortcut /absolute/path/to/shortcut.lnk --json
|
||||
```
|
||||
|
||||
The workflow reports header flags and FILETIMEs, IDList items, LinkInfo volume
|
||||
and network names, StringData arguments and icon location, ExtraData blocks
|
||||
including Tracker droid facts labelled derived, and derived observations such
|
||||
as argument length and a minimized ShowCommand. See
|
||||
[Windows Shell Link shortcuts](../guides/native/windows-shortcut.md).
|
||||
|
||||
Inspect ELF PT_NOTE and SHT_NOTE tables without executing the image:
|
||||
|
||||
@@ -447,7 +460,33 @@ rea inspect-elf-notes /absolute/path/to/program --json
|
||||
|
||||
The workflow reports note owners, types, descriptor hashes, and GNU build-id
|
||||
or ABI-tag fields when those descriptors match. A missing note table is
|
||||
observed absence of those declarations. See [ELF note tables](elf-notes.md).
|
||||
observed absence of those declarations. See [ELF note tables](../elf-notes.md).
|
||||
|
||||
List ELF and Mach-O initializer and finalizer functions without executing the
|
||||
image:
|
||||
|
||||
```bash
|
||||
rea inspect-native-initializers /absolute/path/to/program --json
|
||||
```
|
||||
|
||||
The workflow reports `DT_INIT`, init/fini arrays, legacy `.ctors`/`.dtors`, and
|
||||
Mach-O `__mod_init_func`, `__init_offsets` and `__mod_term_func` entries with
|
||||
their resolved targets. Universal Mach-O files need `--architecture`. See
|
||||
[ELF and Mach-O initializer functions](../guides/native/native-initializers.md).
|
||||
|
||||
Inspect ELF unwind function ranges without executing the image:
|
||||
|
||||
```bash
|
||||
rea inspect-elf-unwind-ranges /absolute/path/to/program --json
|
||||
```
|
||||
|
||||
The workflow reports FDE start, end, and size, the containing section, an exact
|
||||
symbol name when one exists, CIE identity, LSDA, and personality. Coverage
|
||||
counts ranges that match a symbol, ranges without a symbol, and whether the
|
||||
`.eh_frame_hdr` search table agrees with the decoded FDEs. `.debug_frame` is
|
||||
reported present, absent, or incomplete and is not decoded. A missing unwind table is
|
||||
observed absence of those declarations. See
|
||||
[ELF unwind function ranges](../guides/native/elf-unwind-ranges.md).
|
||||
|
||||
Inspect an Apple `.ips` crash report without executing a target:
|
||||
|
||||
@@ -458,7 +497,7 @@ rea inspect-apple-ips /absolute/path/to/report.ips --json
|
||||
The workflow reports first-line metadata, process/exception/termination facts,
|
||||
threads, usedImages, lossless integer registers, and documented crash classes
|
||||
when `bug_type` is `"309"`. Other DiagnosticReports types return no crash
|
||||
result. See [Apple IPS crash reports](apple-ips.md).
|
||||
result. See [Apple IPS crash reports](../apple-ips.md).
|
||||
Inspect Linux kernel module metadata without executing the module or requiring
|
||||
`modinfo`:
|
||||
|
||||
@@ -474,7 +513,23 @@ not decoded and the signature is not validated. Other ELF types are not
|
||||
applicable. gzip, xz, and zstd containers are unsupported and name
|
||||
`external_decompression` as the next probe: decompress with an external
|
||||
utility, then inspect the decompressed file. See
|
||||
[Linux kernel modules](kernel-modules.md).
|
||||
[Linux kernel modules](../kernel-modules.md).
|
||||
|
||||
Read a Linux kernel image's version banner and embedded `CONFIG_IKCONFIG`
|
||||
build config without executing it:
|
||||
|
||||
```bash
|
||||
rea inspect-kernel-build-config /absolute/path/to/vmlinux --json
|
||||
rea inspect-kernel-build-config /absolute/path/to/bzImage --name CONFIG_MODULE_SIG --json
|
||||
```
|
||||
|
||||
The workflow reports the banner version, builder, and compiler, the config
|
||||
option count, and paged `CONFIG_*` entries (`y`, `m`, strings, numbers, and
|
||||
`not_set`). An `IKCFG_ST` string counts as a config only when a valid gzip
|
||||
member follows it and `IKCFG_ED` follows that member. A gzip bzImage payload
|
||||
is inflated and searched; other payload codecs are unsupported and name
|
||||
`external_decompression` as the next probe. See
|
||||
[Linux kernel build config](../guides/native/kernel-build-config.md).
|
||||
|
||||
Inspect a CPython 3.7–3.13 `.pyc` header and nested marshal code objects (3.14 and
|
||||
3.15 headers are qualified but their marshal layout is not pinned)
|
||||
@@ -486,7 +541,7 @@ rea inspect-python-pyc /absolute/path/to/module.pyc --json
|
||||
|
||||
The workflow reports the pinned CPython version for the magic, PEP 552 hash or
|
||||
mtime/size fields, nested function names, and bytecode SHA-256. Bytecode is
|
||||
not decompiled. See [CPython pyc inspection](python-pyc.md).
|
||||
not decompiled. See [CPython pyc inspection](../python-pyc.md).
|
||||
|
||||
Inspect an Erlang BEAM file without executing bytecode:
|
||||
|
||||
@@ -496,7 +551,7 @@ rea inspect-erlang-beam /absolute/path/to/app.beam --json
|
||||
|
||||
The workflow reports IFF chunk names, atom tables, exports, imports, lambdas,
|
||||
and the Code chunk header. Missing optional tables stay null. See
|
||||
[Erlang BEAM files](erlang-beam.md).
|
||||
[Erlang BEAM files](../erlang-beam.md).
|
||||
|
||||
Complete `rea decompile --json` Evidence can be saved and inspected through
|
||||
`inspect-analysis-view` with an inline source and
|
||||
@@ -505,10 +560,10 @@ The projection reads that saved observation without starting a provider;
|
||||
offsets/limits count UTF-16 code units. A summary view preserves the original
|
||||
selector and distinguishes null from empty text. The MCP tool additionally
|
||||
supports `detail: "summary"` on its first call to retain the complete source
|
||||
without returning the body inline; see [native investigation](native-investigation.md).
|
||||
without returning the body inline; see [native investigation](../guides/native/native-investigation.md).
|
||||
|
||||
Configure [Hopper or Ghidra](installation.md#hopper), the
|
||||
[IDA adapter](ida-provider.md), or [Kuna](kuna-provider.md) before analyzing a
|
||||
[IDA adapter](../providers/ida-provider.md), or [Kuna](../kuna-provider.md) before analyzing a
|
||||
native target. Substitute
|
||||
your target, search text and function name or address in these examples:
|
||||
|
||||
@@ -536,11 +591,11 @@ MCP `analyze_function` defaults to the same summary and retains the complete
|
||||
dossier in the session for `inspect_analysis_view`.
|
||||
|
||||
Run `rea --help` or a command's `--help` for arguments and output options.
|
||||
The [generated catalog](mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
|
||||
The [generated catalog](../reference/mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
|
||||
tools. Provider-specific target support is described in the
|
||||
[native guide](native-investigation.md), [DOS guide](ghidra-dos.md),
|
||||
[raw binary guide](raw-binary.md), [GameCube DOL guide](ghidra-dol.md), and
|
||||
[Windows Ghidra guide](windows-ghidra-p0.md).
|
||||
[native guide](../guides/native/native-investigation.md), [DOS guide](../providers/ghidra-dos.md),
|
||||
[raw binary guide](../providers/raw-binary.md), [GameCube DOL guide](../ghidra-dol.md), and
|
||||
[Windows Ghidra guide](../providers/windows-ghidra-p0.md).
|
||||
|
||||
## Choose a provider
|
||||
|
||||
@@ -587,19 +642,27 @@ For Hopper, Ghidra, IDA, JADX, and Kuna, run
|
||||
such as `pwntools-elf`, report their configuration requirements directly and are
|
||||
not accepted by `doctor --provider`. See
|
||||
[task readiness](installation.md#check-readiness-for-your-task) and
|
||||
[provider selection](mcp-contracts.md#identity-and-discovery).
|
||||
[provider selection](../reference/mcp-contracts.md#identity-and-discovery).
|
||||
|
||||
The session reports work still in progress through `analysis_activity`.
|
||||
A client timeout can end its wait while the provider continues analyzing.
|
||||
`cleanup_incomplete` identifies resources whose shutdown or removal could not
|
||||
be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and
|
||||
[Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
|
||||
be verified. See [MCP contracts](../reference/mcp-contracts.md) for session lifecycle and
|
||||
[Ghidra first-query deadlines](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
|
||||
for import, client timeouts and recovery.
|
||||
|
||||
A one-shot command releases its provider session and REA runtime before it
|
||||
prints a failure, so it omits the same-session reuse and
|
||||
`update_analysis_settings` guidance an MCP connection receives. Its
|
||||
`details.limit` reports `scope: "rea-invocation"` and
|
||||
One-shot binary analysis validates the selected operation's input before creating
|
||||
a provider session, opening the target, or replaying a snapshot. Invalid selectors
|
||||
and option combinations report their input errors before target or provider
|
||||
availability errors. An already-cancelled request keeps its cancellation reason.
|
||||
Caller-selected paths and request parameters remain separate from parsed execution
|
||||
arguments in Evidence.
|
||||
|
||||
A one-shot command attempts provider-session and runtime cleanup before reporting
|
||||
a failure. If cleanup remains incomplete, inspect the reported resources and
|
||||
follow the error's remediation. [Same-session cleanup retry](../reference/mcp-contracts.md#retained-cleanup-retry),
|
||||
reuse and `update_analysis_settings` apply to a live MCP connection.
|
||||
For one-shot commands, `details.limit` reports `scope: "rea-invocation"` and
|
||||
`applies_at: "next_invocation"`; set the named `environment` (or the per-call
|
||||
`parameter`) for a new invocation and rerun. For example, a Ghidra decompile
|
||||
deadline names `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS`.
|
||||
@@ -628,12 +691,10 @@ an exact result when the target bytes, operation, parameters, provider and
|
||||
settings match. Mutations and cursor-dependent calls are excluded from the
|
||||
cache. Snapshot files are local and use owner-only permissions.
|
||||
|
||||
One-shot binary CLI Evidence reports the exact path argument in `caller_input_path`,
|
||||
including relative spelling and symlink aliases. Its subject identifies the
|
||||
inspected artifact; `caller_input_path` does not change the Evidence ID or
|
||||
snapshot query. A replay reports the current invocation's path while leaving
|
||||
the saved Evidence unchanged. Provider inspections continue to include the
|
||||
established `parameters.input_path` field.
|
||||
A binary CLI replay reports the current invocation's selected path while leaving
|
||||
the saved Evidence and snapshot query unchanged. See
|
||||
[caller-selected paths](../reference/mcp-contracts.md#caller-selected-paths)
|
||||
for primary and auxiliary path fields and their relation to Evidence identity.
|
||||
|
||||
Importing an Evidence bundle refreshes retained request and subject display metadata.
|
||||
It preserves cached analysis results, including observations for other targets.
|
||||
@@ -742,6 +803,12 @@ rea compare left.json right.json --record_pairs '[{"left_evidence_id":"ev_<left-
|
||||
Migration: replace `rea compare-bundles` with `rea compare`; the former command
|
||||
has been removed. The MCP tool remains `compare_bundles`.
|
||||
|
||||
`rea evidence-import` performs process-local bundle authentication and
|
||||
validation, reporting record and Unknown-revision counts. `unknowns_added`
|
||||
counts revisions, not distinct Unknown IDs. It does not retain records in a
|
||||
separate or background MCP server process. To retain records across calls in an
|
||||
active MCP session, use the MCP tool `import_evidence_bundle` in that session.
|
||||
|
||||
`rea compare` compares records by Evidence identity and explicit pairs. It does
|
||||
not read artifact contents: the inventories of two builds are unresolved
|
||||
records, or one opaque change when you pair them. Use `compare-artifacts`,
|
||||
@@ -763,8 +830,8 @@ protection still applies. A genuine pathname refusal uses `resource_constraint`
|
||||
with `details.reason: path-limit` and `boundary: filesystem-path`.
|
||||
|
||||
For application traces and version comparisons, see
|
||||
[JavaScript application workflows](javascript-application-workflows.md).
|
||||
MCP [retained Evidence references](mcp-contracts.md#retained-application-evidence-inputs)
|
||||
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md).
|
||||
MCP [retained Evidence references](../reference/mcp-contracts.md#retained-application-evidence-inputs)
|
||||
belong to one connection. Separate CLI calls can consume full saved records or
|
||||
resolve exact retained IDs from an explicitly selected local bundle:
|
||||
|
||||
@@ -972,23 +1039,23 @@ for percent signs, query marks and fragments.
|
||||
Historical-source import requires safe no-follow file opens on Linux or
|
||||
macOS. Native Windows returns `unsupported_host`; use Linux REA inside WSL
|
||||
or another supported host. See
|
||||
[source-to-bundle comparison](javascript-application-workflows.md#historical-source-to-bundle-comparison)
|
||||
[source-to-bundle comparison](../guides/javascript/javascript-application-workflows.md#historical-source-to-bundle-comparison)
|
||||
for mapping identities, inferences and unknowns.
|
||||
|
||||
## Capture runtime behavior
|
||||
|
||||
Choose the guide for the target and intended interaction:
|
||||
|
||||
- [Browser observation](browser-observation.md): inspect a selected page through CDP.
|
||||
- [Browser scenarios](browser-scenario-contract.md): run declared interactions and capture their results.
|
||||
- [Electron observation](electron-observation.md): inspect an Electron renderer or capture an application scenario.
|
||||
- [Electron fuses](electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
|
||||
- [Node/Electron Inspector](javascript-runtime-observation.md): record script locations and execution contexts.
|
||||
- [Process capture](process-capture.md): run an executable and compare terminal, exit and filesystem observations.
|
||||
- [Desktop application capture](desktop-application-capture.md): launch a Linux desktop application and record its windows and home-directory state at checkpoints while it is used.
|
||||
- [Desktop application state](desktop-application-state.md): read a capture folder into the application's state files, typed views of each version and kinds of change.
|
||||
- [Desktop application export](desktop-application-export.md): write a capture folder as an agentenv computer app bundle with its class, skill and seed.
|
||||
- [Desktop application tasks](desktop-application-tasks.md): describe how a capture's files changed, then validate an agent-written task against the capture and export it with its verifier.
|
||||
- [Browser observation](../guides/web/browser-observation.md): inspect a selected page through CDP.
|
||||
- [Browser scenarios](../guides/web/browser-scenario-contract.md): run declared interactions and capture their results.
|
||||
- [Electron observation](../guides/javascript/electron-observation.md): inspect an Electron renderer or capture an application scenario.
|
||||
- [Electron fuses](../electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
|
||||
- [Node/Electron Inspector](../guides/javascript/javascript-runtime-observation.md): record script locations and execution contexts.
|
||||
- [Process capture](../guides/runtime/process-capture.md): run an executable and compare terminal, exit and filesystem observations.
|
||||
- [Desktop application capture](../desktop-application-capture.md): launch a Linux desktop application and record its windows and home-directory state at checkpoints while it is used.
|
||||
- [Desktop application state](../desktop-application-state.md): read a capture folder into the application's state files, typed views of each version and kinds of change.
|
||||
- [Desktop application export](../desktop-application-export.md): write a capture folder as an agentenv computer app bundle with its class, skill and seed.
|
||||
- [Desktop application tasks](../desktop-application-tasks.md): describe how a capture's files changed, then validate an agent-written task against the capture and export it with its verifier.
|
||||
|
||||
Runtime requests name the target and actions. Launched targets run with your
|
||||
user permissions; consult the chosen guide for host requirements and effects.
|
||||
@@ -1027,7 +1094,7 @@ one client at a time: a session ends with the client's `DELETE`, and a client
|
||||
that connects while another session is open takes it over, so a client that
|
||||
exited without ending its session needs no restart. The listener runs until
|
||||
SIGINT or SIGTERM and does not stop when stdin closes. See
|
||||
[transports](mcp-contracts.md#transports) for the session, `Host` header and
|
||||
[transports](../reference/mcp-contracts.md#transports) for the session, `Host` header and
|
||||
shutdown contract. Client registrations written by setup and the MCP registry
|
||||
entry keep stdio.
|
||||
|
||||
@@ -1052,7 +1119,7 @@ answers every method with JSON-RPC `-32601`. The stopped document's `state`
|
||||
member is `{"served": true, "evidence_id": …}`,
|
||||
`{"served": false, "reason": "replay-only"}` or
|
||||
`{"served": false, "reason": "unavailable", "code": …, "message": …}`; see
|
||||
[serve from the state](agent-environments.md#serve-from-the-state).
|
||||
[serve from the state](../agent-environments.md#serve-from-the-state).
|
||||
|
||||
```sh
|
||||
rea environment-serve manifest.json --evidence capture.json --port 8765
|
||||
@@ -1072,7 +1139,7 @@ MCP_ALLOWED_HOSTS=127.0.0.1,rea.lan rea environment-serve manifest.json --eviden
|
||||
the served environment; set but blank, it is refused as `configuration_invalid`
|
||||
like a blank `MCP_HOST`. The admitted client URL is written to stderr as one JSON log line
|
||||
(`"msg":"MCP transport listening"`). See
|
||||
[agent environments](agent-environments.md) for what the state and replay
|
||||
[agent environments](../agent-environments.md) for what the state and replay
|
||||
return and do not establish.
|
||||
|
||||
## Build an environment bundle
|
||||
@@ -1097,7 +1164,7 @@ command. `--dry-run` writes nothing and may omit the directory.
|
||||
rea environment-build grafana.har ./grafana --json
|
||||
```
|
||||
|
||||
See [agent environments](agent-environments.md#build-a-bundle-in-one-step)
|
||||
See [agent environments](../agent-environments.md#build-a-bundle-in-one-step)
|
||||
for the report, the hints and what the bundle ships.
|
||||
|
||||
## Export an environment bundle
|
||||
@@ -1130,9 +1197,26 @@ name Windows reserves (`con`, `nul`, `com1` and the like) is refused. The
|
||||
command prints one JSON document listing the written files with their sizes,
|
||||
the pinned package source, the served `tool_count`, and `state` in the shape
|
||||
of the serve command's stopped document. See
|
||||
[agent environments](agent-environments.md#export-a-bundle) for the folder
|
||||
[agent environments](../agent-environments.md#export-a-bundle) for the folder
|
||||
layout and what the smoke task checks.
|
||||
|
||||
`rea environment-export` prints the written files and exits `0` on success.
|
||||
Input-pair refusals, an occupied destination, a missing package tarball, and
|
||||
invalid labels fail before writing output. Host permission failures identify
|
||||
whether reading or writing was denied and retain the affected path.
|
||||
|
||||
A failure during writing can leave a partial folder. Inspect it before removing
|
||||
it and retrying; the next attempt refuses a non-empty destination. When one
|
||||
file caused the failure, the diagnostic names that file. A folder-sync failure
|
||||
can report only its cause.
|
||||
|
||||
If `details.partial_observation.kind` is `environment-bundle-publication`, the
|
||||
bundle was already published. Its `directory` and `files` record the committed
|
||||
output, even when a later identity check or cleanup failed. Inspect that bundle
|
||||
and resolve the reported failure before using or replacing it; the receipt is
|
||||
not an instruction to delete published output. Cleanup resources remain a
|
||||
separate obligation.
|
||||
|
||||
## Compare an environment with its application
|
||||
|
||||
`rea environment-compare <manifest.json> --evidence <capture.json> --probe <probe.json> [--probe <probe.json> ...]`
|
||||
@@ -1143,7 +1227,7 @@ answer is compared with the application's after ids are mapped and clocks and ge
|
||||
makes no network call. It prints the report and exits `0`, or prints a structured `invalid_request` error and exits `1`
|
||||
for the same pair refusals as `environment-serve`, a probe that is not capture Evidence, or a browser probe captured
|
||||
without response bodies, or without the request bodies its writes need. See
|
||||
[agent environments](agent-environments.md#measure-fidelity-against-the-application) for the report.
|
||||
[agent environments](../agent-environments.md#measure-fidelity-against-the-application) for the report.
|
||||
|
||||
```sh
|
||||
rea environment-compare manifest.json --evidence capture.json --probe probe.json --json
|
||||
@@ -1324,62 +1408,39 @@ retains failed stages for retry; the CLI exposes no separate retry command.
|
||||
or prints the refusing stage's structured error and exits `1`, writing nothing
|
||||
when the refusal comes before the bundle is written. SIGINT cancels it with a
|
||||
`cancelled` error; a cancellation before the bundle stage writes nothing.
|
||||
`rea environment-export` is one-shot: it prints the written files and exits
|
||||
`0`, or prints a structured `invalid_request` error and exits `1` for the same
|
||||
pair refusals, a non-empty or file-occupied directory, a missing package
|
||||
tarball or a name that is not a kebab-case label or is a Windows device name,
|
||||
writing nothing in each case. A file occupying the destination is reported as
|
||||
requiring a directory; identity-change diagnostics are reserved for admitted
|
||||
paths that fail later identity checks. Host permission failures identify whether
|
||||
input reading or bundle writing was denied and retain the reported path.
|
||||
A write that fails after the first file (a permission, disk-space or
|
||||
pathname-length failure on a later path) propagates as the host's failure,
|
||||
naming its cause and, when it failed on one bundle file or the package
|
||||
tarball, that file. A failure while syncing the finished folder names only its
|
||||
cause. Either way the partial folder stays in place for inspection; remove it
|
||||
before retrying, since the next attempt refuses the non-empty directory.
|
||||
A `cancelled` result does not by itself determine the exit status. Commands
|
||||
that handle SIGINT or SIGTERM assign `130` or `143` after their operation and
|
||||
cleanup settle; their structured result can be printed before that status is
|
||||
assigned. Read the process status and typed result separately.
|
||||
|
||||
A `cancelled` result does not by itself determine the exit status; it depends on
|
||||
the command's signal handling. Commands routed through the shared
|
||||
`withCommandCancellation` handler in the CLI set status `130` for SIGINT or
|
||||
`143` for SIGTERM after their operation settles. Current source registrations
|
||||
use that handler for process capture and comparison (`capture-process`,
|
||||
`compare-process-captures`), desktop application capture and state
|
||||
(`capture-desktop-application`, `inspect-desktop-application-state`,
|
||||
`export-desktop-application`, `describe-desktop-application-task`), JavaScript application analysis (`analyze` when it
|
||||
selects the JavaScript lane, and `analyze-javascript-application`), selected
|
||||
browser, web and Electron commands (including `analyze-web-bundle`,
|
||||
`observe-web-execution`, `capture-browser-scenario` and
|
||||
`capture-electron-scenario`), and the platform and provider command families
|
||||
for Android/ADB, Apktool/JEB/Flutter, firmware, PCAP and signal analysis,
|
||||
WASM/EVM, Go/Rust and binary diagnostics, Frida instrumentation, the Cutter
|
||||
command, objdump, Rizin and GDB, and the agent environment server
|
||||
(`environment-serve`). The handler also wraps `doctor`,
|
||||
`inspect-sqlite-database`, `recover-javascript-sources`,
|
||||
`inspect-il2cpp-metadata`, and `inspect-desktop-application`. In these paths, a returned structured
|
||||
`cancelled` result can be printed before the wrapper assigns the
|
||||
signal-derived status. This describes the current source routing; it is not a
|
||||
claim that every command or provider runtime has been independently exercised.
|
||||
Commands with `--evidence_output` save the complete Evidence bundle before
|
||||
delivering the selected view. For committed-file receipts, retained partial
|
||||
Evidence and missing source records, see
|
||||
[Evidence publication and source recovery](../reference/mcp-contracts.md#evidence-publication-and-source-recovery).
|
||||
Other outputs have their own receipt shapes; for example,
|
||||
[environment export](#export-an-environment-bundle) reports a published directory.
|
||||
|
||||
Commands with `--evidence_output` select their delivered view, then publish the
|
||||
complete Evidence bundle atomically. Cancellation before publication leaves no
|
||||
new file. A publication that has committed can win a later cancellation; the
|
||||
`cancelled` result then keeps the written path and byte count under
|
||||
`details.diagnostics.evidence_bundle`. If staging cleanup stays incomplete
|
||||
after commit, the `cleanup_incomplete` error reports the same file as
|
||||
`details.published_path` and `details.bytes`.
|
||||
|
||||
When a failure retains a completed observation and no bundle committed,
|
||||
`details.source_closure` states what that observation cites. `publication` is
|
||||
`not_published`. `records` holds authentic source Evidence this invocation
|
||||
captured, within 64 KiB. `omitted` lists up to 64 other identities and
|
||||
`omitted_count` the total: `not_captured` means the invocation holds no such
|
||||
record, so nothing is shown for it or its ancestors; `not_inlined` means it was
|
||||
captured but is not in this response. `complete` is true only when every cited
|
||||
record is in `records` or embedded in the observation. If examining the closure
|
||||
itself fails, `collection_failure` names the error type and the closure state is
|
||||
unknown; the original failure is still returned. Nothing is retained after exit,
|
||||
and running the command again is a new observation.
|
||||
A failure that carries partial Evidence in `details.partial_observation`, such
|
||||
as `provider_timeout` or `resource_constraint`, publishes that partial Evidence
|
||||
to `--evidence_output` under the same new-file and atomic rules, then keeps its
|
||||
error code, message and remediation when cleanup is confirmed. With `--detail summary`,
|
||||
`details.partial_observation` holds the summary view of the partial Evidence
|
||||
instead of the complete record, and `details.diagnostics.evidence_bundle`
|
||||
reports the written path and byte count. Linked source records must all be
|
||||
available before publication; missing sources leave the destination unchanged
|
||||
and are reported in `details.source_closure`. A provider cleanup failure can
|
||||
still publish its captured Evidence without clearing the cleanup error or its
|
||||
resource ownership. If publication is refused before commit, for example because
|
||||
the file already exists, the complete partial Evidence stays inline and `details.diagnostics.evidence_bundle_error` gives the
|
||||
refusal's code, message and a limitation. A refusal after commit keeps the
|
||||
receipt in `details.diagnostics.evidence_bundle` and the delivered view: when
|
||||
staging cleanup stays incomplete, the code becomes `cleanup_incomplete`, the
|
||||
original failure is retained in `details.execution_failure`, and `details` also carries `published`,
|
||||
`published_path`, `bytes`, `cleanup: "incomplete"` and `resources`, as for a
|
||||
successful run, and a later cancellation reports `cancelled` in
|
||||
`evidence_bundle_error` with the signal's exit status. Without
|
||||
`--evidence_output`, and for cancellation before publication or raw provider
|
||||
observations, the failure output is unchanged.
|
||||
|
||||
When the reader of piped output closes early, as `rea ... | head` does, the
|
||||
CLI stops writing, prints no error and exits `141` (`128 + 13`, the status of a
|
||||
@@ -1423,7 +1484,7 @@ the total token count and have no next offset.
|
||||
|
||||
Inspect an explicit local FDT v16/v17 blob without execution or an external tool.
|
||||
Returns header, reservations, node/property tree, derived standard values and
|
||||
selected phandle references. See [devicetree inspection](devicetree.md) for
|
||||
selected phandle references. See [devicetree inspection](../devicetree.md) for
|
||||
limits and retained-view pagination.
|
||||
|
||||
```bash
|
||||
@@ -1439,7 +1500,7 @@ rea inspect-pe-load-config /absolute/path/to/application.exe --json
|
||||
|
||||
Reports TLS callbacks and size-covered load-config fields without execution.
|
||||
Use `--max-callbacks` and `--max-file-bytes` to set read budgets. See
|
||||
[PE TLS and load configuration](pe-load-config.md).
|
||||
[PE TLS and load configuration](../pe-load-config.md).
|
||||
|
||||
### Binary hardening
|
||||
|
||||
@@ -1449,9 +1510,16 @@ rea inspect-binary-hardening /absolute/path/to/program --json
|
||||
|
||||
Reports ELF, PE, and Mach-O hardening rows without execution. A missing import
|
||||
is `unknown`, not `disabled`. Use `--max-file-bytes` to set the read budget. See
|
||||
[binary hardening](binary-hardening.md).
|
||||
[binary hardening](../binary-hardening.md).
|
||||
|
||||
Bun standalone executables use the existing `inspect-artifact` and
|
||||
`extract-artifact` commands to inventory and materialize embedded source,
|
||||
sourcemaps and bytecode without running the executable. See
|
||||
[Bun standalone executables](bun-standalone.md) for layout coverage and limits.
|
||||
[Bun standalone executables](../bun-standalone.md) for layout coverage and limits.
|
||||
|
||||
Deno compile executables use the same commands to inventory and materialize the
|
||||
embedded virtual filesystem, remote modules, and recorded options. Inventory retains
|
||||
embedded environment-file values and certificate bytes in local Evidence with their
|
||||
counts and byte lengths, and extraction does not write them as files. The npm snapshot
|
||||
length is reported and not decoded, and the executable is not run. See
|
||||
[Deno compile executables](../deno-standalone.md).
|
||||
@@ -39,11 +39,17 @@ path and supported version of the runtime you intend to use, then update only
|
||||
the runtime command, preserving the REA script arguments and other settings.
|
||||
Rerun setup to review the registration and restart the client.
|
||||
|
||||
Doctor JSON records each registration's `command_identity` as `owned`, `foreign`,
|
||||
or `unknown`, separately from its readiness `state`. An invalid timeout or client
|
||||
policy can leave ownership established. Setup uses that recorded observation for
|
||||
its default client selection, then inspects selected configuration again before
|
||||
planning writes; a missing or unreadable launcher remains unknown.
|
||||
|
||||
`rea mcp doctor --json` tests the production MCP adapter using the runtime
|
||||
executable and environment of the terminal running doctor. It does not launch
|
||||
the client's configured command or reproduce the client's environment. If doctor
|
||||
passes but the client disconnects, inspect the client's startup error and registered
|
||||
launcher; see [verification scope](client-verification.md#launch-environment-verification).
|
||||
launcher; see [verification scope](../development/client-verification.md#launch-environment-verification).
|
||||
|
||||
`rea doctor` has the same limit for provider checks, and reports a provider setting
|
||||
the registration's file does not carry as a
|
||||
@@ -68,7 +74,7 @@ not update an already running MCP server or desktop client.
|
||||
To select this profile for an agent, set `REA_MCP_INPUT_SCHEMA_PROFILE` in that
|
||||
client's existing REA registration `env` object (`environment` for OpenCode) and
|
||||
restart the client, preserving its other settings. A successful terminal doctor run does not verify the client's
|
||||
launch environment. See [launch-environment verification](client-verification.md#launch-environment-verification)
|
||||
launch environment. See [launch-environment verification](../development/client-verification.md#launch-environment-verification)
|
||||
and the client-specific instructions below. REA setup forwards supported selections,
|
||||
including `REA_MCP_INPUT_SCHEMA_PROFILE`, from its environment into the registration;
|
||||
it does not copy unrelated environment values. Restoring the shell's previous
|
||||
@@ -83,7 +89,7 @@ raising a REA startup or operation deadline, separately ensure the client allows
|
||||
enough time for the full request. Use `inspect_analysis_settings` for the running
|
||||
server's defaults; `rea doctor` checks launch configuration and cannot observe
|
||||
another process's runtime overrides. See the
|
||||
[settings design](design/limits-and-recovery.md) for ownership and lifetime.
|
||||
[settings design](../design/limits-and-recovery.md) for ownership and lifetime.
|
||||
|
||||
## Client tool names and discovery
|
||||
|
||||
@@ -102,13 +108,15 @@ does not repair a client's naming behavior or infer permission drift from an
|
||||
unfamiliar prefix. The unavailable fork reported in
|
||||
[#5512](https://github.com/morluto/rea/issues/5512) has not been reverified.
|
||||
|
||||
<a id="hotcold-catalogs"></a>
|
||||
|
||||
## Hot/cold catalogs
|
||||
|
||||
For a client that supports MCP catalog refresh, set
|
||||
`REA_MCP_TOOL_CATALOG_PROFILE=hot-cold` in REA's server environment and restart
|
||||
the connection. REA then advertises a small initial catalog and loads other
|
||||
named tools through `discover_tools`. Setup preserves this explicit setting.
|
||||
See the [discovery contract](mcp-contracts.md#hotcold-catalog-discovery) for
|
||||
See the [discovery contract](../reference/mcp-contracts.md#hotcold-catalog-discovery) for
|
||||
pagination, identity and family-scope semantics.
|
||||
|
||||
Analysis tools, including `inspect_analysis_view`, load on demand. For example,
|
||||
@@ -155,7 +163,7 @@ REA_MCP_TOOL_CATALOG_PROFILE=full
|
||||
|
||||
This advertises all tools without requiring catalog refresh. Compact output
|
||||
schemas widen client-side validation; REA still validates complete results
|
||||
against the canonical contracts. See [output schemas](mcp-contracts.md#output-schemas).
|
||||
against the canonical contracts. See [output schemas](../reference/mcp-contracts.md#output-schemas).
|
||||
|
||||
On main `3e0599a94`, a real FastMCP 4.1.6 / Python MCP 2.3.0 HTTP client with
|
||||
the default 1 MiB SSE event limit rejected the 4,108,975-byte full catalog.
|
||||
@@ -179,7 +187,7 @@ receive limits and full Evidence access still apply.
|
||||
|
||||
Connect Fiddler Everywhere's MCP server separately for HTTP traffic debugging,
|
||||
preserving your REA client entry. See the
|
||||
[setup instructions and public skills](web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
|
||||
[setup instructions and public skills](../guides/web/web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
|
||||
|
||||
## VS Code
|
||||
|
||||
@@ -217,7 +225,7 @@ tool inventory on Linux using a loopback model fixture. After setup, add
|
||||
settings, then restart the client. Compact advertisements can reduce nested
|
||||
schema constraints; REA still validates complete canonical inputs. Full-profile
|
||||
`gpt-5.4` metadata is also verified locally. Live model API acceptance and
|
||||
Windows remain unverified; see [native client verification](testing.md).
|
||||
Windows remain unverified; see [native client verification](../development/testing.md).
|
||||
|
||||
## Claude Desktop
|
||||
|
||||
@@ -329,7 +337,7 @@ and Claude Code reports that autocompact is thrashing. To avoid this:
|
||||
override.
|
||||
- To remove entire tool families from the upfront catalog, run
|
||||
`rea setup --client claude_code --tool-families <ids>` with the families the
|
||||
workflow needs (see [the advertising scope contract](mcp-contracts.md#tool-family-advertising-scope)).
|
||||
workflow needs (see [the advertising scope contract](../reference/mcp-contracts.md#tool-family-advertising-scope)).
|
||||
Out-of-scope tools are not advertised and their calls fail with the SDK's
|
||||
disabled-tool error, so widen the selection when a task needs another
|
||||
family. Setup writes the scope into the entry's `env` as
|
||||
@@ -346,7 +354,7 @@ Both keep canonical result validation and complete structured results; `compact`
|
||||
keeps a widened root shape and `off` advertises nothing. On catalogs large
|
||||
enough that full input schemas alone approach the frame limit, also keep the
|
||||
input profile at its `compact` default. See [MCP output schema
|
||||
profiles](mcp-contracts.md#output-schemas).
|
||||
profiles](../reference/mcp-contracts.md#output-schemas).
|
||||
|
||||
## Kimi Code
|
||||
|
||||
@@ -391,7 +399,7 @@ doctor reports its readiness separately. Explicit selections override discovered
|
||||
defaults, including when setup runs again with a changed selection. Unrelated
|
||||
environment variables, process ownership tokens and one-shot target paths are not
|
||||
copied. For an explicitly configured startup target, see the
|
||||
[initial-target settings](mcp-contracts.md#initial-target-at-startup). Existing
|
||||
[initial-target settings](../reference/mcp-contracts.md#initial-target-at-startup). Existing
|
||||
client environment overrides remain intact unless setup supplies the same key.
|
||||
You can edit `startupTimeoutMs` and `toolTimeoutMs` there; doctor keeps positive
|
||||
caller-selected values aligned and reports a non-blocking advisory when a tool
|
||||
@@ -502,7 +510,7 @@ standalone CLI check.
|
||||
A model API's `400 INVALID_ARGUMENT` alone does not identify a rejected schema,
|
||||
tool-count limit or skill defect. Preserve the response's parameter path and
|
||||
diagnostic, the actual running REA version, Gemini CLI version and selected
|
||||
model. For schema rejection, review the [input presentation profiles](mcp-contracts.md#compact-input-schema-profile)
|
||||
model. For schema rejection, review the [input presentation profiles](../reference/mcp-contracts.md#compact-input-schema-profile)
|
||||
and reconnect after changing REA's registration environment. Compact is the
|
||||
current default; an explicitly stored `full` choice is preserved. The generic
|
||||
error in [#4620](https://github.com/morluto/rea/issues/4620) does not establish
|
||||
@@ -583,7 +591,7 @@ Responses-compatible Kimi gateway as “infinite recursion”; expanding those
|
||||
references succeeded. A separate full-profile input failed the provider's
|
||||
schema-size limit. These are distinct reported failures, not proof that the
|
||||
canonical schema is recursive. Current REA inputs are already self-contained;
|
||||
see [schema advertisement](mcp-contracts.md#schema-advertisement).
|
||||
see [schema advertisement](../reference/mcp-contracts.md#schema-advertisement).
|
||||
|
||||
For this failure, inspect the active MCP launch command and refresh the REA
|
||||
installation that command actually runs. A configured package selector or the
|
||||
@@ -832,7 +840,7 @@ in `~/.agents/skills` (`$DSH_AGENTS_HOME/skills` when configured); Harness
|
||||
discovers that directory, and its native `skill` tool can load
|
||||
`reverse-engineer-anything`.
|
||||
|
||||
REA's optional [native client verification](testing.md#end-to-end-integration-and-golden-evidence)
|
||||
REA's optional [native client verification](../development/testing.md#end-to-end-integration-and-golden-evidence)
|
||||
checks headless chat, skill loading, all tool input schemas and actual analysis
|
||||
using the installed Harness and a local model fixture. This establishes the
|
||||
Harness integration, with live DeepSeek API compatibility remaining unverified.
|
||||
@@ -898,7 +906,7 @@ servers if you are adding REA to an existing configuration. `timeout_ms` is
|
||||
llama.cpp's per-tool-call timeout, including cold engine startup. The example
|
||||
allows REA's default 330-second Ghidra startup budget plus 120 seconds for the
|
||||
operation. Increase it if you configure a longer provider allowance or need more
|
||||
operation time; see [provider deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
operation time; see [provider deadlines](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
|
||||
Start the server with your own tool-capable GGUF model:
|
||||
|
||||
@@ -933,7 +941,7 @@ keep the server local, and select targets and operations you are authorized to u
|
||||
|
||||
See llama.cpp's [MCP server configuration](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md#mcp-servers)
|
||||
and [function-calling guidance](https://github.com/ggml-org/llama.cpp/blob/master/docs/function-calling.md).
|
||||
The [verification record](client-verification.md#llamacpp) distinguishes connection
|
||||
The [verification record](../development/client-verification.md#llamacpp) distinguishes connection
|
||||
checks from model behavior and lists the tested versions.
|
||||
|
||||
## Other client settings
|
||||
@@ -3,14 +3,16 @@
|
||||
Run REA, connect your selected agents, and check the tools needed for your task.
|
||||
Setup previews configuration changes and asks for approval before applying them.
|
||||
|
||||
| Task | Section |
|
||||
| ---------------------------------- | --------------------------------------------------------------------------------------------------- |
|
||||
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
|
||||
| Install a Claude Code/Codex plugin | [Agent plugins and marketplaces](#agent-plugins-and-marketplaces) |
|
||||
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
|
||||
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), the [IDA guide](ida-provider.md), or [Kuna](kuna-provider.md) |
|
||||
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
|
||||
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
|
||||
| Task | Section |
|
||||
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
|
||||
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
|
||||
| Install a Claude Code/Codex plugin | [Agent plugins and marketplaces](#agent-plugins-and-marketplaces) |
|
||||
| Install the Cursor Marketplace plugin | [Cursor Marketplace](#cursor-marketplace) |
|
||||
| Run unreleased code | [Run REA from a checkout](#run-rea-from-a-checkout) |
|
||||
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
|
||||
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), the [IDA guide](../providers/ida-provider.md), or [Kuna](../kuna-provider.md) |
|
||||
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
|
||||
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
|
||||
|
||||
## Start setup
|
||||
|
||||
@@ -374,7 +376,7 @@ choosing an operation; it works before a target is opened, and
|
||||
`result.tool_availability` reports per-operation availability, reasons and
|
||||
client requirements for the current session state. It does not guarantee that
|
||||
a particular caller-supplied input will be valid. See
|
||||
[MCP discovery](mcp-contracts.md#identity-and-discovery).
|
||||
[MCP discovery](../reference/mcp-contracts.md#identity-and-discovery).
|
||||
|
||||
Select the part you want to check:
|
||||
|
||||
@@ -384,14 +386,22 @@ Select the part you want to check:
|
||||
| One analysis engine | `rea doctor --provider ghidra --json` (or `hopper`, `ida`, `jadx`, `kuna`) |
|
||||
| One agent registration | `rea doctor --client codex --json` (see [client IDs](#supported-agents)) |
|
||||
| Installed workflow instructions | `rea doctor --skill --json` |
|
||||
| Desktop app environment builds | `rea doctor --provider desktop_build --client claude_code --json` |
|
||||
|
||||
A scoped report has `scope.mode: "explicit"`. Its `scope_checks` determine
|
||||
`healthy` and the exit status. Other checks appear in `informational_checks`;
|
||||
`environment_healthy` summarizes the full audit. `--target` adds a target check,
|
||||
but the report remains audit-wide unless a scope option is also supplied.
|
||||
|
||||
Provider checks read the terminal running doctor. A client-started server may
|
||||
receive a different environment, and doctor cannot observe it. When an available
|
||||
With one selected client, Ghidra, IDA, JADX and Kuna provider checks use the
|
||||
client's recorded environment over the terminal it inherits. If that registration
|
||||
cannot be read or a setting a provider needs cannot be resolved, its available
|
||||
value is `null` and its check is `unknown`; its configured value is also `null`
|
||||
when the configuration setting itself is unknown. Doctor does not infer
|
||||
readiness from the terminal. A missing registration still uses terminal settings
|
||||
for setup. Other checks, including Hopper, use the terminal running doctor. A
|
||||
client-started server may receive a different environment, and doctor cannot
|
||||
observe it. When an available
|
||||
provider relies on a setting that a registration's file does not record, or
|
||||
records with other text, doctor adds a non-blocking
|
||||
`registration_environment:<client>` check with `classification: "unknown"`. The
|
||||
@@ -472,6 +482,12 @@ updated stops the remaining removals. Uninstall reports each candidate
|
||||
their individual `actions` alongside its aggregate status and detail,
|
||||
including earlier removals when a later file fails. The paths name
|
||||
the discovered configuration candidates, not resolved physical file identities.
|
||||
If resource cleanup remains incomplete, uninstall reports a failed
|
||||
`uninstall_cleanup` item alongside the completed removals, with a structured
|
||||
`cleanup` observation naming the outstanding resources. An embedded host retains
|
||||
the exact cleanup owners: retrying that host's `close()` updates previously
|
||||
returned cleanup observations when release is confirmed. A new CLI invocation
|
||||
does not inherit those owners; inspect the reported resources before proceeding.
|
||||
For JSON clients, uninstall removes every repeated `rea` member in the effective
|
||||
server table only when each registration is REA-owned. An unowned earlier
|
||||
member, or an unowned OpenCode V1 entry that removing V2 would expose, stops
|
||||
@@ -595,6 +611,36 @@ feature selection. The same package version string in a development checkout
|
||||
does not establish that its bytes match the npm tarball. Update a registration
|
||||
through a reviewed, scoped setup plan, then restart/reconnect the agent.
|
||||
|
||||
### Run REA from a checkout
|
||||
|
||||
To run code that is not released yet, build a checkout with the Node.js
|
||||
version in `.nvmrc`:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/morluto/rea.git
|
||||
cd rea
|
||||
npm ci
|
||||
npm run build:cached
|
||||
```
|
||||
|
||||
Then run `npm link` once in the checkout to put its `rea` on your `PATH`, or,
|
||||
wherever these docs say `rea …`, run
|
||||
`node /absolute/path/to/rea/scripts/rea.mjs …`. Setup run this way registers
|
||||
the checkout's `scripts/rea.mjs` with your agents.
|
||||
|
||||
After `git pull`, run `npm ci` and `npm run build:cached` again. If the
|
||||
checkout's skill changed, refresh each installed copy before restarting its
|
||||
agent. For example, preview and review the Claude Code skill changes:
|
||||
|
||||
```bash
|
||||
rea setup --skill-client claude_code --dry-run --json
|
||||
```
|
||||
|
||||
Then apply the reviewed changes with `rea setup --skill-client claude_code --yes --json`.
|
||||
Substitute the relevant skill client; use the checkout's
|
||||
`node /absolute/path/to/rea/scripts/rea.mjs` form if `rea` is not linked. For desktop builds, also pack the [build computer's runtime
|
||||
archive](../desktop-build-computer.md#requirements) again.
|
||||
|
||||
## Skill-only installation
|
||||
|
||||
```bash
|
||||
@@ -698,7 +744,7 @@ Linux architectures; configure [Ghidra](#ghidra) there instead. REA itself runs
|
||||
Linux distribution with a recognized apt, dnf, or pacman package family, where
|
||||
an existing Hopper launcher is selected through `HOPPER_LAUNCHER_PATH`. Ghidra
|
||||
and IDA have their own provider-specific host requirements; Windows Ghidra uses
|
||||
the [experimental P0 boundary](windows-ghidra-p0.md).
|
||||
the [experimental P0 boundary](../providers/windows-ghidra-p0.md).
|
||||
|
||||
Bazzite uses RPM packages, but REA does not install Hopper automatically on it.
|
||||
Setup skips the Hopper offer because Bazzite is outside the supported host list,
|
||||
@@ -1025,7 +1071,7 @@ output-stream closure; `output_closed: false` keeps that uncertainty explicit.
|
||||
Later requests retain the known exit instead of reducing it to socket loss.
|
||||
An external GUI's helper exit does not establish that the GUI itself exited.
|
||||
For cursor navigation, string decoding, symbol-name limits, and regex matching,
|
||||
see [Hopper analysis behavior](native-investigation.md#hopper-analysis-behavior).
|
||||
see [Hopper analysis behavior](../guides/native/native-investigation.md#hopper-analysis-behavior).
|
||||
|
||||
Closing or switching a target closes its bound Hopper document, shuts down REA's
|
||||
bridge and removes its temporary socket directory. The macOS Hopper application
|
||||
@@ -1150,13 +1196,13 @@ session metadata edits leave executable bytes unchanged and are discarded on clo
|
||||
require Hopper; Windows P0 remains read-only.
|
||||
|
||||
Windows P0 admits native x86, x86-64, ARMNT (Thumb-2) and ARM64 PE applications and DLLs,
|
||||
[DOS MZ executables](ghidra-dos.md), plus explicitly
|
||||
selected [raw NMOS 6502 images](raw-binary.md#nmos-6502) with a caller-supplied
|
||||
[DOS MZ executables](../providers/ghidra-dos.md), plus explicitly
|
||||
selected [raw NMOS 6502 images](../providers/raw-binary.md#nmos-6502) with a caller-supplied
|
||||
entry at file offset 0, on fixed local NTFS volumes. Other raw CPU families remain
|
||||
unsupported on Windows.
|
||||
The npm package bundles native Job Object ownership, protected private runtime
|
||||
DACLs, and handle-based path admission; no separate addon installation is needed.
|
||||
See the [Windows Ghidra P0 guide](windows-ghidra-p0.md) for verified scope and
|
||||
See the [Windows Ghidra P0 guide](../providers/windows-ghidra-p0.md) for verified scope and
|
||||
limitations.
|
||||
|
||||
Extract Ghidra and install the JDK outside REA, then export absolute paths:
|
||||
@@ -1199,7 +1245,7 @@ REA skill after approval. Managed registrations use the absolute selected Node o
|
||||
Bun runtime and REA entry script, including when setup ran through a package runner. Hopper
|
||||
installation remains unavailable on Windows. Setup never installs Ghidra,
|
||||
Java, or Python. It preserves valid detected Ghidra/JDK settings in agent
|
||||
registrations. See [Windows Ghidra P0](windows-ghidra-p0.md) for provider diagnostics.
|
||||
registrations. See [Windows Ghidra P0](../providers/windows-ghidra-p0.md) for provider diagnostics.
|
||||
|
||||
The separate `rea mcp add` builtin delegates most clients to Incur's package-runner
|
||||
child launcher. Run that acquisition from a trusted directory; prefer managed
|
||||
@@ -1317,7 +1363,7 @@ passes the deadline to each provider client; a startup failure is returned by th
|
||||
without exposing partial analysis. This deadline is separate from MCP transport
|
||||
initialization and the client's deadline for that individual tool call. A client
|
||||
can time out earlier even when Ghidra would complete within its startup deadline.
|
||||
See [Ghidra first-query deadlines and recovery](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
|
||||
See [Ghidra first-query deadlines and recovery](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
|
||||
for client options and the close/reopen recovery flow.
|
||||
|
||||
The 330-second default can be changed on a running MCP server by setting
|
||||
@@ -1370,7 +1416,7 @@ invocation instead. In MCP, `ghidra.decompile_timeout_seconds` can override the
|
||||
value for newly accepted operations on the existing Program; explicit `null`
|
||||
removes the deadline. A transient timeout remains uncached so the same function
|
||||
can be retried after an update. See the
|
||||
[settings design](design/limits-and-recovery.md#runtime-settings-through-mcp).
|
||||
[settings design](../design/limits-and-recovery.md#runtime-settings-through-mcp).
|
||||
Codex/Kimi registrations derived by `rea setup` add the selected launch value
|
||||
to the tool deadline as a single-decompile estimate; a runtime update does not
|
||||
change that registration. A bulk call can need that limit for
|
||||
@@ -1558,6 +1604,11 @@ For supported header-selected MIPS ELF targets, the profile also binds the
|
||||
selected target's ELF metadata; it does not claim a fresh-import MIPS lane was
|
||||
used for the saved project.
|
||||
A matching recorded import digest does not claim that the database is unedited.
|
||||
Caller and callee coverage therefore remains unknown. A single-direction query
|
||||
with known omissions remains partial and also reports the unknown project history.
|
||||
Observed call edges are still available: a path to an observed caller or callee
|
||||
can be reached, but a missing path is unknown when the project's prior analysis
|
||||
cannot establish completeness.
|
||||
Saved REA seed/analyzer reports remain historical facts in retained source Evidence;
|
||||
they are not interpreted as overrides applied by this session.
|
||||
Project content participates in snapshot identity, so different project states
|
||||
@@ -1735,7 +1786,7 @@ contains the launch arguments and `/proc/<jvm-pid>/status` reports
|
||||
session and retain the resource flags needed for the measurement rather than
|
||||
copying full command lines or environments into reports. Heap and affinity
|
||||
settings affect cost and latency; they are not part of REA's semantic analysis
|
||||
profile identity. See [real-provider testing](testing.md) for bounded
|
||||
profile identity. See [real-provider testing](../development/testing.md) for bounded
|
||||
verification workflows.
|
||||
|
||||
### Ghidra diagnostics and verification
|
||||
@@ -1799,7 +1850,10 @@ The repository's single-plugin marketplaces are declared in
|
||||
`.claude-plugin/marketplace.json` and `.agents/plugins/marketplace.json`.
|
||||
Both select an exact `rea-agents` npm version instead of mutable source-checkout
|
||||
contents. Release Please updates their selectors alongside the plugin manifests.
|
||||
No plugin-only publication or generated skill files committed to Git are needed.
|
||||
Claude Code and Codex do not need plugin-only publication or generated skill
|
||||
files committed to Git. The Cursor Marketplace plugin is the exception in
|
||||
[Cursor Marketplace](#cursor-marketplace): its skill copy is committed because
|
||||
that install is a Git checkout.
|
||||
The qualified release archive contains a runtime-only manifest and
|
||||
`npm-shrinkwrap.json` plus bundled runtime dependencies, including PTY packages
|
||||
for each supported OS and CPU; development dependencies and development-only overrides
|
||||
@@ -1840,12 +1894,13 @@ only the registrations and skill locations that `rea setup` manages, so for a
|
||||
plugin-only install `rea doctor --client <id>` and `rea doctor --skill` report
|
||||
`missing` and recommend `rea setup`. Do not follow that remediation. Check the
|
||||
plugin with the client instead (`claude plugin list` and `/mcp` in Claude
|
||||
Code, `codex plugin list` in Codex).
|
||||
Code, `codex plugin list` in Codex, Cursor's plugin settings for a Marketplace
|
||||
install).
|
||||
|
||||
For analysis engines, ask the connected server: have the agent call
|
||||
`binary_session` and read `result.tool_availability`, which reports what the
|
||||
server the plugin started can use, with reasons and remediation. See
|
||||
[MCP contracts](mcp-contracts.md).
|
||||
[MCP contracts](../reference/mcp-contracts.md).
|
||||
|
||||
A plugin does not put `rea` on your `PATH`. To check an engine from a
|
||||
terminal, use the package runner at the version the plugin pins:
|
||||
@@ -1863,6 +1918,66 @@ That report describes the terminal's environment, not the plugin server's.
|
||||
The server inherits the client's environment, so a setting such as
|
||||
`GHIDRA_INSTALL_DIR` that exists only in your terminal does not reach it.
|
||||
|
||||
### Cursor Marketplace
|
||||
|
||||
Cursor's Marketplace installs this public repository as one plugin. The
|
||||
install includes both the `reverse-engineer-anything` skill and the `rea` MCP
|
||||
server. REA does not publish an MCP-only "Add to Cursor" install link, because
|
||||
that link would register the server without the skill.
|
||||
|
||||
The manifest is `.cursor-plugin/plugin.json`. Cursor discovers `mcp.json` at
|
||||
the repository root and the skill directory named by `skills`. The committed
|
||||
skill is `.cursor-plugin/skills/reverse-engineer-anything`, a copy of the
|
||||
authored skill at `.agents/skills/reverse-engineer-anything`.
|
||||
`node scripts/generate-cursor-plugin-skill.mjs --check` fails when that copy
|
||||
drifts, including in `npm run docs:check` and package prepack. Marketplace
|
||||
installs are a Git checkout and do not contain `dist/`. The server entry
|
||||
therefore runs the published package instead of a local build:
|
||||
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@6.8.0", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<!-- x-release-please-end -->
|
||||
|
||||
Release Please updates that pin with `.cursor-plugin/plugin.json` and
|
||||
`package.json`. The skill text can describe capabilities newer than the pinned
|
||||
server until the next release; follow the connected server's tool list. The
|
||||
same pin is what `rea doctor --provider <id>` should use when you inspect an
|
||||
engine from a terminal, as shown above.
|
||||
|
||||
A Marketplace install gives you:
|
||||
|
||||
- the `reverse-engineer-anything` skill and its references
|
||||
- an `rea` MCP server launched with `npx -y rea-agents@<pinned version> mcp`
|
||||
(Cursor shows it as `plugin-rea-rea`)
|
||||
- the shared plugin logo at `assets/rea-plugin.png`
|
||||
|
||||
It does not install analysis engines, put `rea` on `PATH`, or write
|
||||
`~/.cursor/mcp.json`. `rea setup --client cursor` remains the direct
|
||||
registration path and still writes that file plus the setup-owned skill. Use
|
||||
the Marketplace plugin or that registration, not both, so Cursor does not
|
||||
start two `rea` servers.
|
||||
|
||||
Node.js and npm must be available to Cursor. The plugin does not install them.
|
||||
Restart Cursor after the plugin is listed. If tools stay missing, read the
|
||||
plugin server's launch error. `rea doctor --client cursor` does not see
|
||||
Marketplace state and can report `missing`; do not run setup to repair a
|
||||
plugin-only install.
|
||||
|
||||
Publishing the listing is a separate maintainer step at
|
||||
[cursor.com/marketplace/publish](https://cursor.com/marketplace/publish).
|
||||
This repository change does not submit the plugin.
|
||||
|
||||
## MCP Registry
|
||||
|
||||
REA is published in the official MCP Registry as `io.github.morluto/rea`. Registry
|
||||
File renamed without changes.
+2
-2
@@ -1,7 +1,7 @@
|
||||
# GameCube DOL analysis through Ghidra
|
||||
|
||||
This lane imports 32-bit big-endian PowerPC GameCube/Wii DOL images through
|
||||
Ghidra. It does not execute the image. DOL is a headerless format with no magic
|
||||
Ghidra. It does not execute the image. DOL has a section and entry-point header but no magic
|
||||
number, so REA never detects it: the caller must select it explicitly.
|
||||
|
||||
Only the cached MEM1 window `0x80000000..0x81800000` is admitted. Wii MEM2
|
||||
@@ -29,7 +29,7 @@ MCP, with the existing `open_binary` tool:
|
||||
}
|
||||
```
|
||||
|
||||
The architecture comes from the DOL header (`powerpc`). Do not pass
|
||||
Selecting the DOL format supplies the architecture (`powerpc`). Do not pass
|
||||
`architecture` with `gamecube-dol`; it is rejected. `entry_point` is optional,
|
||||
and if given it must equal the header entry.
|
||||
|
||||
|
||||
+30
-3
@@ -49,8 +49,10 @@ as the 4.3 and 4.4.1 exporters do, so the declared MD5 does not depend on the
|
||||
encryption IV. Encrypted members whose declared MD5s differ therefore also
|
||||
compare `unknown` in `metadata` and `availability`: the MD5s declare
|
||||
different scripts, but REA cannot verify them, and equal MD5s add nothing for
|
||||
the same reason. An encrypted member with an MD5 on only one side stays
|
||||
`unknown`.
|
||||
the same reason. An encrypted member with an MD5 on only one side, including
|
||||
an all-zero MD5 that records no claim, stays `unknown` in `metadata` and
|
||||
`availability`: the claim's presence differs, and one claim does not prove
|
||||
the script changed.
|
||||
|
||||
## CLI
|
||||
|
||||
@@ -144,9 +146,34 @@ keys, decode Godot resources, decompile GDScript bytecode, analyze signals,
|
||||
apply patch/removal entries or execute targets.
|
||||
|
||||
Readable logical member bytes, including repeated physical ranges, share the ZIP
|
||||
archive expansion policy: at most the larger of 256 MiB or 500 times the PCK's
|
||||
archive expansion policy by default: at most the larger of 256 MiB or 500 times the PCK's
|
||||
encoded byte span. Embedded carrier bytes do not enlarge that budget. The full
|
||||
directory is checked before any member is yielded, hashed or extracted. Encrypted
|
||||
members are not read and do not consume this plaintext-work allowance. Small
|
||||
bounded aliases remain valid. This is a processing/output resource bound, separate
|
||||
from the directory metadata retention budget and format validity.
|
||||
|
||||
Directory retention defaults to 64 MiB of encoded metadata. Select
|
||||
`artifact_limits.godot.directory_bytes` to change it; path/source facts reserve
|
||||
worst-case JSON expansion before declared path bytes are read, and the reader
|
||||
then charges actual retained metadata. A preallocation `required_value` is the
|
||||
allowance needed by that admission check, not a measurement of final output size.
|
||||
Select `artifact_limits.expanded_bytes` independently to change aggregate
|
||||
plaintext work, including repeated readable logical occurrences. Both controls
|
||||
apply to inspection and extraction, before any member is yielded.
|
||||
|
||||
For example:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./game.pck --artifact-limits '{"godot":{"directory_bytes":134217728},"expanded_bytes":1073741824}' --json
|
||||
```
|
||||
|
||||
MCP uses the same object under `artifact_limits` on `inspect_artifact` or
|
||||
`extract_artifact`. A refusal reports the exact `reader_limit` parameter,
|
||||
effective value, unit, supported range and required value when known. Raise the
|
||||
named admission and retry the same complete PCK on the next call; no REA restart
|
||||
is needed. Summary or page size does not change producer admission. Omitted
|
||||
policy selections stay omitted in Evidence parameters; a selected object records
|
||||
the accepted selection and `effective_limits`, conditional on which readers are
|
||||
encountered. Raising these policies does not admit malformed offsets, unsupported
|
||||
flags, unsafe extraction paths or unavailable encrypted contents.
|
||||
@@ -113,8 +113,8 @@ existing analysis families directly, for example
|
||||
| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); checks for existing targets and refuses them without explicit overwrite (the check is not atomic); compares digests when device `sha256sum` is available |
|
||||
| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves its Success/Failure verdict, records `replace_requested`, and leaves actual replacement `null` because prior package state is not observed |
|
||||
| `uninstall_adb_package` | **mutates device** | Requests package removal for user 0 via `adb shell pm uninstall --user 0`, without data retention |
|
||||
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
|
||||
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
|
||||
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -W -n`) |
|
||||
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -W -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
|
||||
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
|
||||
|
||||
Screen capture validates the PNG signature, IHDR framing, dimensions, and IHDR
|
||||
@@ -188,7 +188,7 @@ and during shutdown.
|
||||
property value remains available in `properties`.
|
||||
- **Device paths are not host paths.** `base_apk_device_path` values live on
|
||||
the device filesystem. Only pulls and captures produce host-local paths,
|
||||
each paired with byte size and SHA-256 digest. A pulled file's host name is
|
||||
with byte size and SHA-256 digest on success. A pulled file's host name is
|
||||
the device path's final POSIX segment. REA preserves the original device
|
||||
path and filename; a missing final segment or `.`/`..` is refused before
|
||||
output creation. Windows hosts follow [Windows filename rules](https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file),
|
||||
@@ -249,6 +249,10 @@ and during shutdown.
|
||||
- Pull and capture outputs are additive: the destination must not already
|
||||
exist. Re-running a completed pull refuses the existing directory instead
|
||||
of overwriting evidence.
|
||||
- If a single-file pull fails and REA observes a regular file at the destination,
|
||||
the error includes its `local_path` with `transfer_status: "unconfirmed"`
|
||||
and `digest_status: "unknown"`. The file remains in place; the receipt does
|
||||
not establish that the transfer completed.
|
||||
- Cancellation during a package pull preserves caller-selected output. The
|
||||
error includes partial observations identifying completed artifacts and the
|
||||
interrupted local path; remaining APKs were not attempted.
|
||||
@@ -288,7 +292,7 @@ Record: verified against adb 34.0.5 on Linux against an Android 14
|
||||
(API 34) x86_64 emulator — all operations above, a 224-package inventory,
|
||||
287 services, 92 features, a real two-APK split set pulled with byte-exact
|
||||
digests, a verified push/pull roundtrip, a 1.3 MB screen capture, and the
|
||||
complete package lifecycle; see `docs/testing.md` for the lane summary.
|
||||
complete package lifecycle; see `docs/development/testing.md` for the lane summary.
|
||||
|
||||
Intent integer extras use signed decimal text. The ADB argument encoder removes
|
||||
redundant leading zeros for Android's radix-sensitive `--ei` parser while retained
|
||||
@@ -6,12 +6,12 @@ kept as an unmodified, commit-pinned Git submodule; see
|
||||
application workflow and return artifact-bound Evidence inline.
|
||||
|
||||
This family is present on repository main and in npm 4.1.0. Check the
|
||||
[released package boundary](installation.md#released-package-and-main) and the
|
||||
[released package boundary](../../getting-started/installation.md#released-package-and-main) and the
|
||||
connected server's tool list before selecting it.
|
||||
|
||||
On Linux, JADX launches an REA-owned process and requires a procps-compatible
|
||||
`ps` on REA's `PATH`; see the shared
|
||||
[Linux process prerequisite](installation.md#linux-owned-process-prerequisite).
|
||||
[Linux process prerequisite](../../getting-started/installation.md#linux-owned-process-prerequisite).
|
||||
|
||||
## Optional package delivery
|
||||
|
||||
@@ -545,7 +545,8 @@ and each upstream request and reply collected before the failure. Failed replies
|
||||
remain available even when their payload cannot be interpreted. Cancellation
|
||||
also stops target hashing before provider admission.
|
||||
|
||||
These operations do not execute the APK or provide split APK/AAB handling,
|
||||
The JADX package, class, method and reference operations do not execute the APK
|
||||
or provide split APK/AAB handling,
|
||||
signature validation, full resource-table semantics, native-library analysis or
|
||||
Android runtime capture. An `.aab` suffix is a ZIP archive, not an APK, so
|
||||
Android tools and the application graph do not treat it as a standalone APK.
|
||||
@@ -687,14 +688,14 @@ bounds each upstream call. Timeout recovery reports the effective milliseconds
|
||||
and range, with `details.limit` identifying the runtime setting. An invalid
|
||||
provider-local launch value can also be repaired by a runtime override; reset
|
||||
exposes that invalid launch value again. See the
|
||||
[settings design](design/limits-and-recovery.md#invalid-launch-configuration).
|
||||
[settings design](../../design/limits-and-recovery.md#invalid-launch-configuration).
|
||||
A passing JADX configuration check in `rea doctor` reports its launch-derived
|
||||
`operation_timeout_ms`; it does not inspect another running server's overrides. Explicit
|
||||
cancellation still ends the operation immediately, and partial observations
|
||||
and owned-process cleanup are unchanged. Raising the deadline does not extend
|
||||
the bridge's 30-second startup handshake or the upstream **90-second**
|
||||
decompilation limit. MCP clients enforce their own tool-call timeouts, and
|
||||
setup's derived [client tool timeout](client-configuration.md) does not include
|
||||
setup's derived [client tool timeout](../../getting-started/client-configuration.md) does not include
|
||||
this setting, so also raise the client timeout when selecting a longer
|
||||
deadline.
|
||||
|
||||
+36
-8
@@ -66,6 +66,8 @@ rea decode-android-resources /targets/Example.apk --locales de --locales pt-BR
|
||||
rea decode-android-resources /targets/Example.apk --locales de --locales pt-BR --string-tables-only
|
||||
rea decode-android-resources /targets/Example.apk --no-strings
|
||||
rea decode-android-resources /targets/Example.apk --decode-timeout-ms 900000
|
||||
rea decode-android-resources /targets/Example.apk --string-offset 2000 --string-limit 100 --string-tables-only
|
||||
rea decode-android-resources /targets/Example.apk --max-string-table-bytes 16777216 --max-total-string-table-bytes 67108864
|
||||
```
|
||||
|
||||
Equivalent MCP methods are `inspect_apktool_client` and
|
||||
@@ -87,6 +89,31 @@ responses and charges the aggregate output budget for one copy per table.
|
||||
Omitting the option preserves existing responses. `include_strings: false`
|
||||
still suppresses all string projection, including in this mode.
|
||||
|
||||
Use `string_offset` and `string_limit` (CLI `--string-offset` and
|
||||
`--string-limit`) to select a page of valid parsed entries from each requested
|
||||
table, in source order. Defaults are offset 0 and limit 2000. Explicit page or
|
||||
budget controls include `string_tables` even for a single primary table; its
|
||||
`page` gives the requested offset and limit, returned count, and `next_offset`.
|
||||
Retry that offset with the same locale and confirm `target.sha256` matches.
|
||||
Each call decodes a fresh owned snapshot; it does not retain a workspace or
|
||||
promise identity across changed APKs. A null continuation means the page reaches
|
||||
the end, while table coverage remains partial if earlier entries were omitted.
|
||||
An offset at or beyond the end returns an empty, present table with known counts.
|
||||
If the continuation equals the requested offset, no output entry fit: increase
|
||||
`max_total_strings` or use `string_tables_only` before retrying.
|
||||
|
||||
`max_total_strings` (CLI `--max-total-strings`, default 10000) bounds returned
|
||||
string copies across all tables. Raise it, request fewer locales, or use
|
||||
`string_tables_only` when aggregate output is exhausted. The legacy
|
||||
multi-table shape charges both copies of primary strings. `max_string_table_bytes`
|
||||
(CLI `--max-string-table-bytes`, default 8388608) bounds one XML source read;
|
||||
`max_total_string_table_bytes` (CLI `--max-total-string-table-bytes`, default 33554432) bounds XML reads across distinct physical tables. Locale aliases
|
||||
share the physical read and parsed page. Raise the relevant read budget to
|
||||
retry an oversized table; a larger string limit or timeout does not raise it.
|
||||
These controls take nonnegative safe-integer offsets and positive safe-integer
|
||||
budgets, apply on the next decode, and require no REA restart. Process memory
|
||||
and response transport capacity still apply.
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- **An owned APK snapshot is digested and decoded.** `target.sha256` and
|
||||
@@ -117,8 +144,8 @@ still suppresses all string projection, including in this mode.
|
||||
Malformed XML and a wrong document root are whole-document projection
|
||||
failures: the result is partial, the failure reason is disclosed, and entry
|
||||
counts remain unknown. They are not counted as one unsupported string.
|
||||
The default table is bounded to 2000 entries with partial coverage when more
|
||||
exist. `locales` lists observed regular `strings.xml` tables whose exact
|
||||
The default page returns up to 2000 entries with partial coverage when more
|
||||
exist; its limitation names the next `string_offset`. `locales` lists observed regular `strings.xml` tables whose exact
|
||||
`values-<locale>` directory is selectable by the request's locale grammar.
|
||||
Locale suffixes allow up to 23 ASCII characters, derived from Apktool's
|
||||
language, script, region, and variant fields; for example,
|
||||
@@ -133,9 +160,10 @@ still suppresses all string projection, including in this mode.
|
||||
When multiple tables are requested with `locales`, `string_tables` records
|
||||
each projected table with its locale, status (`present`, `missing`,
|
||||
`oversized`, `malformed`, or `skipped`), entry counts, and coverage,
|
||||
bounded at an aggregate budget of 10,000 strings and 32 MiB XML. When `locales`
|
||||
is omitted and `string_tables_only` is not true, legacy single-table payloads are preserved
|
||||
without adding `string_tables`.
|
||||
with default aggregate budgets of 10,000 strings and 32 MiB XML. When `locales`
|
||||
is omitted, `string_tables_only` is not true and no string page or budget
|
||||
control is selected, legacy single-table payloads are preserved without adding
|
||||
`string_tables`.
|
||||
- **Smali is skipped by design** (`apktool d --no-src`): Java-source
|
||||
recovery belongs to the JADX family. Resource smali edits and rebuild are
|
||||
a follow-up lane with mutation authority.
|
||||
@@ -156,8 +184,8 @@ still suppresses all string projection, including in this mode.
|
||||
- The version probe waits 20000 ms and retains 65536 bytes. This release has
|
||||
no runtime override for that probe. `decode_timeout_ms` does not raise it.
|
||||
- Manifest reads are bounded to 512 KiB before decoding UTF-8. Strings XML
|
||||
reads are bounded to 8 MiB; larger documents are skipped with partial
|
||||
coverage and an explicit limit, rather than reported as missing.
|
||||
reads default to 8 MiB per table; larger documents are skipped with partial
|
||||
coverage and an explicit read-budget recovery, rather than reported as missing.
|
||||
- `decoded_file_count` is capped at 100,000 files; the cap is reported in
|
||||
the limitations when hit.
|
||||
- An ordinary nonzero decode exit surfaces as `unsupported_target` naming
|
||||
@@ -174,7 +202,7 @@ OpenJDK 25 against a signed aapt2-built probe APK carrying two resource
|
||||
locales — launcher version, on-disk digest agreement, apktool.yml metadata
|
||||
(1.2.3, SDK 24–34), manifest package agreement, default and `de` string
|
||||
tables projected. Parser goldens in the test suite come from the same
|
||||
decode. See `docs/testing.md` for the lane summary.
|
||||
decode. See `docs/development/testing.md` for the lane summary.
|
||||
|
||||
Multi-locale and table-only projection were also verified through real CLI and
|
||||
stdio MCP workflows on Linux with Apktool 3.0.3/JDK 21 against NewPipe 0.29.1,
|
||||
@@ -217,3 +217,47 @@ The Gallery workflow and all 147 focused Flutter tests also pass on macOS Sonoma
|
||||
including the conflicting-load and note-range-limit CLI/MCP regressions. Windows
|
||||
verification runs under a non-administrator account. Verifiers use a shared
|
||||
temporary parent so each CLI/MCP process can own and release its own snapshots.
|
||||
|
||||
## Selecting and recovering string-pool observations
|
||||
|
||||
`inspect_dart_aot` accepts `pool_policy`; the CLI uses `--pool-policy` JSON:
|
||||
|
||||
```sh
|
||||
rea inspect-dart-aot app.apk --pool-policy '{"contains":"package:my_app/"}'
|
||||
rea inspect-dart-aot app.apk --pool-policy '{"uri_limit":2000,"token_limit":4000,"run_bytes":4096,"token_bytes":256}'
|
||||
```
|
||||
|
||||
A literal, case-sensitive `contains` filter runs before distinct collection,
|
||||
so an omitted URI or identifier can be selected without returning every other
|
||||
string. URI/token counts describe matching entries; `printable_run_count` counts
|
||||
complete printable runs before filtering. Unset budgets preserve the defaults:
|
||||
500 returned entries per URI category, 2000 identifier-like tokens, 1024 bytes
|
||||
per printable run, and 128 bytes per identifier. Budget values must be positive
|
||||
safe integers. Collection retains at most twice the selected category limit;
|
||||
counts remain lower bounds when that capacity is reached. A run exceeding
|
||||
`run_bytes` is omitted whole, rather than reported as a fabricated prefix.
|
||||
The result names the budget reached, and `string_pool.policy` records the
|
||||
selected filter and effective values. A larger budget or a narrower selection
|
||||
applies to the next call without a REA restart; explicit choices stay in Evidence.
|
||||
|
||||
For a larger package-URI inventory, select the category and its relevant budget:
|
||||
|
||||
```sh
|
||||
rea inspect-dart-aot app.apk --pool-policy '{"contains":"package:","uri_limit":2048}'
|
||||
```
|
||||
|
||||
The equivalent MCP input is `pool_policy: {contains: "package:", uri_limit: 2048}`.
|
||||
This recovers all bounded package-URI observations in the measured Gallery and
|
||||
LocalSend APKs. Check the result's limits and coverage for other targets; long
|
||||
runs and unsupported snapshot layouts remain separate limitations. The
|
||||
[calibration report](../../verification/flutter-budget-calibration.md)
|
||||
keeps the default at 500 because increasing every complete overview also delivers
|
||||
more strings when they are unnecessary. Use `detail: "summary"` for the MCP
|
||||
identity/count task described above.
|
||||
|
||||
These controls do not raise APK, ELF, library or response budgets, deserialize
|
||||
Dart objects, or establish typed class/function semantics. Retained
|
||||
`inspect_analysis_view` pages cover the returned selection; exhausting those
|
||||
pages does not recover facts that the producing pool scan omitted. Re-run the
|
||||
producer with the reported selection or budget change to obtain a new
|
||||
observation, checking its APK and library digests before comparing it.
|
||||
File renamed without changes.
@@ -117,8 +117,8 @@ are never passed to TShark.
|
||||
Installed libraries, global configuration and plugins remain trusted, are not
|
||||
fully fingerprinted, and can influence dissection. This is not an OS sandbox.
|
||||
|
||||
Input is limited to 32 MiB before reading, each command to 30 seconds, and
|
||||
combined stdout/stderr to 4 MiB during collection. Output overflow fails instead
|
||||
Default limits are 32 MiB input before reading, 30 seconds per command, and
|
||||
4 MiB combined stdout/stderr during collection. Output overflow fails instead
|
||||
of returning truncated success. Before execution, `prlimit` enforces **1 GiB
|
||||
address space per process**, **64 MiB per file**, **30 CPU seconds** and no core
|
||||
dumps. Stricter inherited soft/hard limits are preserved; Evidence records the effective
|
||||
@@ -128,6 +128,23 @@ These are not aggregate process-memory or filesystem quotas. The selected
|
||||
limiter is trusted; its path, digest and arguments are retained in Evidence.
|
||||
No arbitrary process flags are accepted.
|
||||
|
||||
Each capture/recording operation accepts `producer_limits` with positive safe
|
||||
integer overrides: `input_bytes`, `output_bytes`, `command_timeout_ms`,
|
||||
`address_space_bytes`, `file_bytes`, and `cpu_seconds`. The wall-clock timeout
|
||||
is at most 2,147,483,647 milliseconds, matching Node timers. Omitted fields keep
|
||||
the defaults above. Discovery accepts the same controls except `input_bytes`,
|
||||
since it does not read a capture or recording. Packet field discovery defaults
|
||||
to 32 MiB output rather than 4 MiB.
|
||||
|
||||
CLI supplies this object as JSON with `--producer-limits`, for example
|
||||
`--producer-limits '{"input_bytes":67108864,"output_bytes":8388608,"cpu_seconds":60}'`.
|
||||
Values apply to the next call without restarting REA. Effective native ceilings
|
||||
still preserve any tighter inherited soft and hard limit. Errors identify the
|
||||
exhausted producer control and retain partial output and cleanup status; follow
|
||||
that repair after cleanup settles. Raising producer budgets does not raise
|
||||
separate parser representations, transport buffers or external client deadlines.
|
||||
Never truncate the selected artifact to fit an input budget.
|
||||
|
||||
## Verification
|
||||
|
||||
```sh
|
||||
@@ -182,8 +199,9 @@ proof that the field appears in a selected recording.
|
||||
|
||||
The adapter uses the documented `tshark -G fields` grammar under its existing
|
||||
private empty profile and Linux process limits. TShark 4.6.9's full glossary was
|
||||
measured at 22,303,197 bytes; discovery admits at most **32 MiB** combined command
|
||||
output (capture inspection remains at **4 MiB**). Parsing walks lines once and
|
||||
measured at 22,303,197 bytes; discovery defaults to **32 MiB** combined command
|
||||
output (capture inspection defaults to **4 MiB**); `producer_limits.output_bytes`
|
||||
can change either request budget. Parsing walks lines once and
|
||||
constructs objects only for the selected page. Returned strings are copied so
|
||||
retained pages do not retain the complete glossary string. Before constructing a selected
|
||||
record it charges six times the source UTF-8 bytes plus 512 bytes against an
|
||||
Loaded 100 of 3206 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user