mirror of
https://github.com/morluto/rea.git
synced 2026-11-07 05:56:19 -10:00
Compare commits
57
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1a4db31eea | ||
|
|
2e1e99283e | ||
|
|
2e8d463400 | ||
|
|
e62484931c | ||
|
|
89d5bb7056 | ||
|
|
75c7187290 | ||
|
|
d7df9df339 | ||
|
|
5a11bf09a5 | ||
|
|
f8f9095d2d | ||
|
|
e75d1c741c | ||
|
|
1fc86c45aa | ||
|
|
de9d468211 | ||
|
|
c3647ade1e | ||
|
|
6be6596aac | ||
|
|
f3e5b33538 | ||
|
|
26b033d51f | ||
|
|
9a68ce57a0 | ||
|
|
276d99e787 | ||
|
|
19082acddb | ||
|
|
e8cb71740b | ||
|
|
c28d47bf13 | ||
|
|
ec05b9de27 | ||
|
|
8c857ce56c | ||
|
|
28bffe92c8 | ||
|
|
4f7c4cc218 | ||
|
|
4b961e3715 | ||
|
|
8db538a7ca | ||
|
|
e01fcfb4b2 | ||
|
|
f5cfe09c07 | ||
|
|
f59d884f35 | ||
|
|
4f4e2552cc | ||
|
|
b3e22620a5 | ||
|
|
53fca98e3a | ||
|
|
5ed283d168 | ||
|
|
f251009e0a | ||
|
|
d892e7861a | ||
|
|
9ebbe67e05 | ||
|
|
b912ec184c | ||
|
|
193e01abb1 | ||
|
|
d496d803a4 | ||
|
|
6259ac41a4 | ||
|
|
79766ca96a | ||
|
|
22c493ad5c | ||
|
|
b7911b2d30 | ||
|
|
6442ea02cb | ||
|
|
eef8f44003 | ||
|
|
510e83eadc | ||
|
|
2d11fd92d2 | ||
|
|
9bea563e30 | ||
|
|
e749160ab0 | ||
|
|
920d634e83 | ||
|
|
55078b9f57 | ||
|
|
75c3cab540 | ||
|
|
3c05b4d3fe | ||
|
|
2a0415cd5b | ||
|
|
673d971381 | ||
|
|
4809afeef5 |
No files matched your search
@@ -106,7 +106,8 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
|
||||
|
||||
## Author readiness
|
||||
|
||||
<!-- The human owner completes these declarations before requesting review.
|
||||
<!-- The human owner completes these declarations before requesting review,
|
||||
unless the PR author is exempt as listed in CONTRIBUTING.md.
|
||||
The PR author is the owner by default. A different owner must explicitly accept
|
||||
ownership on this PR.
|
||||
AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA.
|
||||
|
||||
@@ -55,12 +55,12 @@ jobs:
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Install ELF32 cross toolchains for source-owned fixtures
|
||||
- name: Install AArch64 and ELF32 cross toolchains for source-owned fixtures
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --yes --no-install-recommends \
|
||||
gcc-i686-linux-gnu gcc-arm-linux-gnueabi gcc-mips-linux-gnu gcc-mipsel-linux-gnu \
|
||||
libc6-dev-i386-cross libc6-dev-armel-cross libc6-dev-mips-cross libc6-dev-mipsel-cross
|
||||
gcc-aarch64-linux-gnu gcc-i686-linux-gnu gcc-arm-linux-gnueabi gcc-mips-linux-gnu gcc-mipsel-linux-gnu \
|
||||
libc6-dev-arm64-cross libc6-dev-i386-cross libc6-dev-armel-cross libc6-dev-mips-cross libc6-dev-mipsel-cross
|
||||
- name: Acquire exact upstream libraries in an isolated runner environment
|
||||
shell: bash
|
||||
run: |
|
||||
|
||||
@@ -21,6 +21,10 @@ jobs:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Set up Node.js for the website Worker checks
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
|
||||
@@ -54,6 +54,10 @@ jobs:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Set up Node.js for the website Worker checks
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
@@ -75,10 +79,21 @@ jobs:
|
||||
PY
|
||||
- name: Configure Pages
|
||||
uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
|
||||
# upload-pages-artifact excludes dot directories, including .well-known.
|
||||
- name: Package the website including agent discovery
|
||||
run: |
|
||||
tar --dereference --hard-dereference --directory website/_site \
|
||||
--exclude=.git --exclude=.github -cf "$RUNNER_TEMP/artifact.tar" .
|
||||
tar -tf "$RUNNER_TEMP/artifact.tar" \
|
||||
./.well-known/ard.json ./.well-known/ai-catalog.json \
|
||||
./.well-known/agent-skills/index.json > /dev/null
|
||||
- name: Upload the website
|
||||
uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
path: website/_site
|
||||
name: github-pages
|
||||
path: ${{ runner.temp }}/artifact.tar
|
||||
retention-days: 1
|
||||
if-no-files-found: error
|
||||
|
||||
deploy:
|
||||
needs: prepare
|
||||
@@ -130,14 +145,24 @@ jobs:
|
||||
PUBLISH_TARGET: ${{ inputs.target }}
|
||||
run: |
|
||||
python3 -u - <<'PY'
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from urllib.error import URLError
|
||||
from urllib.parse import urljoin
|
||||
from urllib.parse import urljoin, urlsplit
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
def open_public(url, accept=None):
|
||||
headers = {
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
}
|
||||
if accept:
|
||||
headers["Accept"] = accept
|
||||
return urlopen(Request(url, headers=headers), timeout=10)
|
||||
|
||||
expected = os.environ["GITHUB_SHA"]
|
||||
pages_url = os.environ["PAGES_URL"]
|
||||
if not pages_url:
|
||||
@@ -153,11 +178,7 @@ jobs:
|
||||
for name, base in list(pending.items()):
|
||||
url = urljoin(base, "deployment.json") + "?commit=" + expected
|
||||
try:
|
||||
request = Request(url, headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
})
|
||||
with urlopen(request, timeout=10) as response:
|
||||
with open_public(url) as response:
|
||||
actual = json.load(response).get("commit")
|
||||
if actual == expected:
|
||||
print(f"{name} serves {expected}: {base}")
|
||||
@@ -172,6 +193,27 @@ jobs:
|
||||
time.sleep(5)
|
||||
if pending:
|
||||
sys.exit("Could not verify published revision on: " + ", ".join(pending))
|
||||
for name, base in hosts.items():
|
||||
with open_public(urljoin(base, ".well-known/agent-skills/index.json")) as response:
|
||||
skill, = json.load(response)["skills"]
|
||||
# Fetch from each selected host, including the /rea/ mirror.
|
||||
artifact_url = urljoin(base, urlsplit(skill["url"]).path.lstrip("/"))
|
||||
with open_public(artifact_url) as response:
|
||||
digest = "sha256:" + hashlib.sha256(response.read()).hexdigest()
|
||||
if digest != skill["digest"]:
|
||||
sys.exit(name + " serves a skill archive with the wrong digest")
|
||||
for manifest in ("ard.json", "ai-catalog.json"):
|
||||
with open_public(urljoin(base, ".well-known/" + manifest)) as response:
|
||||
if not json.load(response).get("entries"):
|
||||
sys.exit(name + " serves an empty discovery manifest")
|
||||
if name == "Cloudflare":
|
||||
with open_public(base, accept="text/markdown") as response:
|
||||
vary = {value.strip().lower() for value in response.headers.get("Vary", "").split(",")}
|
||||
if response.headers.get_content_type() != "text/markdown" or "accept" not in vary:
|
||||
sys.exit("Cloudflare did not negotiate Markdown with Vary: Accept")
|
||||
if 'rel="ard"' not in response.headers.get("Link", ""):
|
||||
sys.exit("Cloudflare is missing its discovery Link header")
|
||||
print(name + " serves the discovery manifests and verified skill archive")
|
||||
with open(os.environ["GITHUB_STEP_SUMMARY"], "a", encoding="utf-8") as summary:
|
||||
links = " and ".join(f"[{name}]({url})" for name, url in hosts.items())
|
||||
summary.write(f"Published commit `{expected}` to {links}.\n")
|
||||
|
||||
@@ -128,8 +128,8 @@ owner to complete. Never attest to human review on their behalf. Draft status
|
||||
is optional; preserve the PR's current draft status unless the user asks to change
|
||||
it. Later pushes do not invalidate the readiness declaration or require another
|
||||
human sign-off. Keep the PR body and affected validation current as work changes.
|
||||
The readiness check validates
|
||||
the filled declaration; CI and automated review results do not establish that
|
||||
The readiness check validates the filled declaration or the author exemption
|
||||
listed in CONTRIBUTING.md; CI and automated review results do not establish that
|
||||
human review occurred, maintainer scope approval, or merge permission.
|
||||
|
||||
<!-- BEGIN:turborepo-agent-rules -->
|
||||
|
||||
+8
-4
@@ -93,7 +93,8 @@ Before requesting maintainer review, the human owner must:
|
||||
passing. Follow the [verification requirements](#development-feedback-and-pr-verification)
|
||||
for required checks and real-provider coverage.
|
||||
- Complete the **Author readiness** declaration in the PR template, explicitly
|
||||
stating that the PR is ready for initial review.
|
||||
stating that the PR is ready for initial review, unless the PR author is
|
||||
exempt from the declaration as listed below.
|
||||
|
||||
Keep the PR description concise and current. Explain the problem, observable
|
||||
outcome, approach, and useful evidence; link long logs or put them in optional
|
||||
@@ -114,9 +115,12 @@ validation current as work changes. AI agents must leave human declarations to
|
||||
the human owner and must not attest to a human review on their behalf.
|
||||
|
||||
The `PR readiness` check reports pending for drafts and requires a named human
|
||||
owner and all three template declarations for non-drafts. It checks that the
|
||||
declaration is filled, not whether the code or evidence is correct. Existing PRs
|
||||
can copy the **Author readiness** section from the current template.
|
||||
owner and all three template declarations for non-drafts. PRs authored by
|
||||
@morluto, @kaoru0822-kitauji, @N0zoM1z0, or @gensokyo-dev are exempt from
|
||||
the declaration; their human ownership, review, and evidence responsibilities
|
||||
above still apply. The check identifies authors by GitHub account ID. It checks
|
||||
that the declaration is filled, not whether the code or evidence is correct.
|
||||
Existing PRs can copy the **Author readiness** section from the current template.
|
||||
Open PRs must have distinct head commits because GitHub shares commit statuses.
|
||||
|
||||
Maintainers enable merge enforcement after the workflow is merged: run
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Website](https://rea.tools/) · [Guides](https://rea.tools/guides/) · [Showcases](https://rea.tools/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Join the REA community on Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [Build RL environments](#build-rl-environments) · [FAQ](#faq) · [Documentation](#documentation)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -60,7 +60,7 @@ pin inspection sources; they are not included in the npm package.
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.988/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.988/NOTICE) | Bring your own `kuna`; REA verified 1.988 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
|
||||
@@ -228,6 +228,22 @@ func processRunTokenObservation(
|
||||
parse: Result<ParsedRunToken, RunTokenReadError>,
|
||||
codeSigningFlags: () -> UInt32?
|
||||
) -> RunTokenObservation {
|
||||
// Native inspection restrictions apply even when argv is incomplete or
|
||||
// overwritten, before parsing reaches the environment. Keep this fact at
|
||||
// the native boundary; cleanup still decides whether the process is related.
|
||||
func withheldEnvironment() -> RunTokenObservation? {
|
||||
guard let flags = codeSigningFlags() else { return nil }
|
||||
if (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func unavailableEnvironment(_ reason: String) -> RunTokenObservation {
|
||||
withheldEnvironment() ?? unavailable(pid: pid, reason: reason)
|
||||
}
|
||||
switch parse {
|
||||
case .success(let parsed):
|
||||
if let runId = parsed.runId {
|
||||
@@ -238,40 +254,25 @@ func processRunTokenObservation(
|
||||
if parsed.environmentEntryCount > 0 {
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
}
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
if let withheld = withheldEnvironment() { return withheld }
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
case .failure(let failure):
|
||||
switch failure {
|
||||
case .environmentUnavailable:
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "environment_unavailable")
|
||||
return unavailableEnvironment("environment_unavailable")
|
||||
case .ambiguousEnvironmentBoundary:
|
||||
if let flags = codeSigningFlags(), (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "ambiguous_environment_boundary")
|
||||
return unavailableEnvironment("ambiguous_environment_boundary")
|
||||
// These failures already establish that token bytes were available.
|
||||
case .duplicateToken:
|
||||
return unavailable(pid: pid, reason: "duplicate_run_token")
|
||||
case .invalidTokenEncoding:
|
||||
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
|
||||
case .appleVectorUnavailable:
|
||||
return unavailable(pid: pid, reason: "apple_vector_unavailable")
|
||||
return unavailableEnvironment("apple_vector_unavailable")
|
||||
case .invalidBuffer:
|
||||
return unavailable(pid: pid, reason: "malformed_procargs")
|
||||
return unavailableEnvironment("malformed_procargs")
|
||||
case .invalidBufferAt(let stage):
|
||||
return unavailable(pid: pid, reason: "malformed_procargs_\(stage.rawValue)")
|
||||
return unavailableEnvironment("malformed_procargs_\(stage.rawValue)")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -16,10 +16,10 @@ PROFILE = "[email protected];[email protected];[email protected]"
|
||||
PACKAGES = {"pwntools": "4.15.0", "pyelftools": "0.33", "unicorn": "2.1.2"}
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
# (e_machine, ELF class bits, byte order) -> architecture pwntools must also report.
|
||||
PROFILES = {(62, 64, "little"): "amd64", (3, 32, "little"): "i386", (40, 32, "little"): "arm",
|
||||
(8, 32, "little"): "mips", (8, 32, "big"): "mips"}
|
||||
PROFILE_SCOPE = ("Layout profile requires ELF64 x86-64, ELF32 i386, ELF32 ARM little-endian "
|
||||
"or ELF32 MIPS little- or big-endian objects.")
|
||||
PROFILES = {(62, 64, "little"): "amd64", (183, 64, "little"): "aarch64", (3, 32, "little"): "i386",
|
||||
(40, 32, "little"): "arm", (8, 32, "little"): "mips", (8, 32, "big"): "mips"}
|
||||
PROFILE_SCOPE = ("Layout profile requires ELF64 x86-64, ELF64 AArch64 little-endian, ELF32 i386, "
|
||||
"ELF32 ARM little-endian or ELF32 MIPS little- or big-endian objects.")
|
||||
|
||||
|
||||
class LayoutFailure(Exception):
|
||||
|
||||
@@ -1149,7 +1149,12 @@ global conflict involving an equal pair is not used to normalize either pair; th
|
||||
|
||||
A step agrees when the statuses and the normalised bodies are equal. The report gives agreement overall, per probe and
|
||||
per tool, `agreement_ignoring_order` for steps that differ only in item order, and each step's differences, at most 20,
|
||||
with a JSON pointer and both values cut short:
|
||||
with a JSON pointer and both values cut short. Each step reports `differences_total` before that cap,
|
||||
`differences_omitted`, and `differences_truncated` (true when entries were omitted). Agreement and categories use the
|
||||
complete comparison. Omitted entries are not retained, so `--full-output` cannot recover them; inspect the original
|
||||
capture and probe Evidence to investigate beyond the returned pointers.
|
||||
|
||||
The difference categories are:
|
||||
|
||||
| Category | Meaning |
|
||||
| --------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
|
||||
@@ -9,9 +9,11 @@ ordinary binary inspection use the same modular capability.
|
||||
## Bring your own engine
|
||||
|
||||
Real verification covers **Linux x64** hosts and **EXEC/DYN/REL** objects in
|
||||
these profiles: ELF64 x86-64, ELF32 i386, ELF32 little-endian ARM and ELF32
|
||||
little- or big-endian MIPS. Other machines, classes and byte orders, including
|
||||
AArch64 and big-endian ARM, return `unsupported_target`. Provide an absolute
|
||||
these profiles: ELF64 x86-64, ELF64 little-endian AArch64, ELF32 i386, ELF32
|
||||
little-endian ARM and ELF32 little- or big-endian MIPS. Other machines, classes
|
||||
and byte orders, including big-endian ARM, return `unsupported_target`. Other
|
||||
hosts, including Linux arm64, return `capability_unavailable` naming the
|
||||
detected host before any target is read. Provide an absolute
|
||||
Python executable whose environment already contains unchanged pwntools 4.15.0,
|
||||
pyelftools 0.33 and Unicorn 2.1.2:
|
||||
|
||||
|
||||
@@ -500,7 +500,7 @@ REA retains local observation data and removes only URL userinfo credentials, ex
|
||||
connection/frame inventories; matching direction/opcode sequences alone do
|
||||
not prove alignment after drops. Redirect resource types use only shared known metadata:
|
||||
an omitted type leaves equality unknown rather than proving a modification.
|
||||
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
|
||||
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. Source locations are CDP's zero-based lines and UTF-16 columns. A negative column, which Chrome reports on the first line of some inline scripts, is kept as reported and does not locate a column in the source URL; an invalid line or a non-integer column is `null` (WebMCP `registration_source` is `null`) and is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
|
||||
- WebSocket observations retain direction, opcode, and payload byte length. A missing or malformed CDP opcode is returned as `null`, with opcode-dependent payload byte length and shape also `null`, and marks `websocket_frames` incomplete. Missing or wrong-typed payload data is not treated as empty; its byte length is `null`, and a text payload shape is unknown. Numeric opcode `0` and an observed empty payload remain distinct from missing data. Opcode is nullable in the public inspection schema, so clients must handle unknown protocol values. When shape capture is selected, text messages are classified as text or complete value-free JSON shape; binary bytes, hashes, prefixes, and raw payloads are never retained.
|
||||
- Storage observations always redact values. Key names, IndexedDB names, and cache names require `include_storage_keys`; stable content fingerprints require the additional `include_storage_fingerprints` selection. Cookie fingerprints cover cookies applicable to the current main-frame URL, including its path, rather than every cookie path on that origin. Other storage remains scoped to the origin. Cache fingerprints hash all decoded response bytes, including bodies above 64 KiB, subject to the [64 MiB per-message CDP limit](#request-boundary). Missing or undecodable cache bodies and partial IndexedDB remote objects make fingerprint coverage incomplete, so identical observations remain `unknown`.
|
||||
- Unavailable or malformed storage inventories mark `storage_keys` unavailable, even without fingerprints. Comparisons suppress inferred key additions and removals while preserving modifications proven by complete fingerprints on both sides. Previously exported captures are not retroactively corrected; recapture them to obtain these coverage markers.
|
||||
|
||||
@@ -612,6 +612,9 @@ Saved snapshots include only cache entries whose subject identifies the selected
|
||||
target. Unbound observations and observations of child artifacts remain available
|
||||
in the open session and its Evidence bundle; they are excluded from the target's
|
||||
saved replay entries without changing their subjects.
|
||||
The snapshot also retains available Evidence linked by selected records, including
|
||||
transitive sources with an unbound or different subject. Those sources are not
|
||||
replay entries for the selected target. Unrelated records remain excluded.
|
||||
|
||||
Snapshots retain eligible target-scoped question histories in full, including
|
||||
mutation Evidence recorded while another target was active. Observations and
|
||||
|
||||
+7
-1
@@ -38,6 +38,12 @@ were examined; an unavailable section name does not erase those note facts.
|
||||
For GNU notes, `gnu.build_id_hex`, `gnu.abi_tag`, or `gnu.gold_version` are
|
||||
filled when the descriptor matches that type. Other owners keep `gnu` null.
|
||||
|
||||
For Linux `CORE` notes, `core.prpsinfo`, `core.file`, `core.siginfo`, or
|
||||
`core.auxv` are decoded for process arguments and status (`NT_PRPSINFO`), mapped
|
||||
file memory regions and paths (`NT_FILE`), fault signal and code (`NT_SIGINFO`), and
|
||||
auxiliary vectors (`NT_AUXV`) across 32-bit and 64-bit images. Other owners or unparsed
|
||||
notes keep `core` null.
|
||||
|
||||
`PT_NOTE` and `SHT_NOTE` are examined independently. The same file range may
|
||||
appear twice when both tables declare it.
|
||||
|
||||
@@ -48,7 +54,7 @@ the image never carried notes in another container or overlay.
|
||||
|
||||
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
|
||||
32 note entries are examined by default across every table. Retained note
|
||||
names, section names, and GNU descriptor text share a scan budget based on
|
||||
names, section names, GNU descriptor text, and core dump string representations share a scan budget based on
|
||||
the source bytes plus any decoded section-name table, capped at
|
||||
`max_file_bytes`. The raw source and decoded table must also fit together
|
||||
within that ceiling. If admission fails, the error reports the required
|
||||
|
||||
@@ -28,6 +28,15 @@ The host must already provide util-linux `prlimit`, normally `/usr/bin/prlimit`;
|
||||
no system tools and changes no user configuration. [Upstream provenance and
|
||||
license](https://github.com/morluto/rea/blob/main/third_party/evmole/README.md) are packaged.
|
||||
|
||||
Limiter discovery validates the selected path without launching it. A relative
|
||||
path or directory is reported as `not_configured`, a missing file as
|
||||
`executable_missing`, and denied access or another filesystem-probe failure as
|
||||
`runtime_unavailable`. Recovery names the selected path and
|
||||
`REA_EVM_PRLIMIT_COMMAND`; denied access asks for execute permission and
|
||||
filesystem access. The original admission or system code stays in diagnostics.
|
||||
An available filesystem probe does not verify the selected tool's version or
|
||||
util-linux compatibility, and REA does not substitute another command.
|
||||
|
||||
## Evidence semantics
|
||||
|
||||
- `encoding` is required: raw bytes or UTF-8 hex, with optional `0x` prefix and
|
||||
|
||||
@@ -26,6 +26,9 @@ REA does not install these tools, their dependencies or system packages. It
|
||||
checks the tool's `--version` banner when executing a request. Binwalk must
|
||||
print `binwalk <version>` and Unblob must print the version alone. Inspection and
|
||||
extraction have independent prerequisites; one does not require the other.
|
||||
Capability discovery recommends `REA_BINWALK_COMMAND` for missing inspection
|
||||
and `REA_UNBLOB_COMMAND` for missing extraction. Configuring the other engine
|
||||
does not enable the unavailable operation.
|
||||
Both require util-linux `prlimit`, normally `/usr/bin/prlimit`; override its
|
||||
absolute path with `REA_FIRMWARE_PRLIMIT_COMMAND` if needed. `rea doctor` does
|
||||
not currently inspect firmware readiness. Engine selection/version checks run
|
||||
|
||||
@@ -109,9 +109,13 @@ server; older servers keep the complete request after schema discovery.
|
||||
header check does not establish valid table contents or exported symbols.
|
||||
- **`libflutter` facts** are per-ABI observations: byte size, SHA-256
|
||||
digest, GNU build IDs from ELF-declared `PT_NOTE` / `SHT_NOTE` ranges,
|
||||
and any _labeled_ strings the build
|
||||
carries. Release engine builds often strip the `Dart VM version` string —
|
||||
absence is reported as `null`, not guessed. Android clang/LLVM toolchain
|
||||
and Dart/toolchain strings the build carries. `dart_version` accepts the
|
||||
labeled `Dart VM version:` form and the unlabeled Dart
|
||||
`Version::String()` form (`<version> (<channel>) (<commit time>)` with an
|
||||
optional ` on "<os>_<arch>"` suffix) that release engines embed. An
|
||||
unlabeled match is a candidate based on its shape; it is not bound to a
|
||||
Dart symbol and another string can match. `null` means no supported pattern
|
||||
was found, not proof that no version exists. Android clang/LLVM toolchain
|
||||
lines are reported verbatim as build provenance.
|
||||
`build_id_candidates` preserves distinct IDs. `build_id` is populated only
|
||||
when all declared note ranges were examined and one ID was found;
|
||||
@@ -122,10 +126,13 @@ server; older servers keep the complete request after schema discovery.
|
||||
Compressed `SHT_NOTE` sections are skipped with an unsupported-compression
|
||||
limitation and partial coverage; IDs from other note ranges remain candidates.
|
||||
Encoded note storage is never interpreted as note headers or inflated.
|
||||
- Labeled strings must fit their complete grammar. `dart_version_capped`
|
||||
and `toolchain_capped` independently report labels omitted by grammar or
|
||||
projection bounds; either makes identification coverage partial. Values
|
||||
split across read chunks are retained without publishing truncated prefixes.
|
||||
- Version and toolchain strings must fit their complete grammar.
|
||||
`dart_version_capped` and `toolchain_capped` independently report labels
|
||||
omitted by grammar or projection bounds; either makes identification
|
||||
coverage partial. Values split across read chunks are retained without
|
||||
publishing truncated prefixes. Unlabeled runs that miss the
|
||||
`Version::String()` grammar are ignored and do not set
|
||||
`dart_version_capped`.
|
||||
- **`coverage: "partial"`** also appears when an ABI carries `libapp.so`
|
||||
without `libflutter.so` (or vice versa).
|
||||
- The APK itself is digested before parsing, so `target.sha256` binds the
|
||||
|
||||
@@ -51,7 +51,6 @@ For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
|
||||
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
|
||||
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [PE imports](pe-imports.md), [PE TLS/load config](pe-load-config.md), [ELF notes](elf-notes.md), [Linux kernel modules](kernel-modules.md), [Apple IPS](apple-ips.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
|
||||
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [PE imports](pe-imports.md), [PE TLS/load config](pe-load-config.md), [ELF notes](elf-notes.md), [Apple IPS](apple-ips.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
|
||||
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [GameCube DOL](ghidra-dol.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
|
||||
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
|
||||
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
|
||||
|
||||
@@ -93,6 +93,10 @@ coverage. It does not print captured source text.
|
||||
Runtime targets, frames, scripts, and workers become capture-scoped graph
|
||||
nodes. REA evaluates compatible static candidates conservatively:
|
||||
|
||||
V8 Inspector targets whose protocol type is `node` use JavaScript asset/module
|
||||
candidates for their verified entry location. A target-only correspondence does
|
||||
not mark that script loaded; load-state evidence requires a script observation.
|
||||
|
||||
1. A captured-source SHA-256 plus one matching mapped location yields
|
||||
a content-and-location match.
|
||||
2. When no static candidate exists at a mapped location, one unique exact
|
||||
|
||||
+23
-1
@@ -1,9 +1,27 @@
|
||||
# Kuna provider
|
||||
|
||||
[Kuna](https://github.com/Noelo-Lab/kuna) is an Apache-2.0 decompiler distributed as one `kuna` executable. For a new installation, use Kuna 1.988, REA's recommended verified release. REA uses a `kuna` you supply; it does not download, install, update or bundle Kuna. REA accepts other releases, which it reports as unverified in `rea doctor` and in each analysis profile. The verified-release flag compares the reported version; it does not authenticate the executable. The 1.988 real-provider checks ran on macOS arm64; Linux and other hosts have not been verified with that release.
|
||||
[Kuna](https://github.com/Noelo-Lab/kuna) is an Apache-2.0 decompiler distributed as one `kuna` executable. For a new installation, use Kuna 1.990, REA's recommended verified release. REA uses a `kuna` you supply; it does not download, install, update or bundle Kuna. REA accepts other releases; `rea doctor` and analysis Evidence report them as unverified. Verification compares the reported version; it does not authenticate the executable. Analysis profiles retain the observed version and executable digest, independently of REA's current verified-release policy. The 1.990 real-provider checks ran on macOS arm64; Linux and other hosts have not been verified with that release.
|
||||
|
||||
Kuna is static: it reads the selected image and never executes it.
|
||||
|
||||
This profile change makes earlier Kuna analysis snapshots and resumable directory
|
||||
exports incompatible, even with the same Kuna executable. Keep the old files:
|
||||
their recorded Evidence remains historical. To continue analysis, open the
|
||||
binary without the old snapshot and save a new one, or choose a new export
|
||||
directory. Existing snapshot Evidence can be extracted from its
|
||||
`evidence_bundle` JSON member and imported as a separate Evidence bundle; that
|
||||
does not restore cached workflow entries or resume the old directory export.
|
||||
For example, using `jq` and a new output path:
|
||||
|
||||
```bash
|
||||
jq -e '.evidence_bundle | select(type == "object")' old-snapshot.json > history.json
|
||||
rea evidence-import history.json --json
|
||||
```
|
||||
|
||||
The CLI command validates the bundle in a one-shot session. To use the records
|
||||
in an MCP session, call `import_evidence_bundle` with the extracted file path.
|
||||
This validates the embedded bundle, not the rest of the snapshot container.
|
||||
|
||||
## Configure
|
||||
|
||||
```bash
|
||||
@@ -30,6 +48,10 @@ completes. Cleanup retries keep both the observations and the exact file owner.
|
||||
| `REA_KUNA_TIMEOUT_MS` | 120 s | Deadline for one single-function decompile, and for the inventory run REA makes when the export ends without one. |
|
||||
| `REA_KUNA_JOBS` | 1 | `decompile-project --jobs`. 1 is Kuna's reference answer; more jobs finish sooner but lose callee-first prototypes, and change about 40–60% of function bodies on the measured binaries. |
|
||||
|
||||
Pseudocode read from a parallel project export includes a prototype-context
|
||||
limitation in its Evidence, including in function dossiers and composed
|
||||
workflows. Early single-function results have their own source limitation.
|
||||
|
||||
A single-function decompile, inventory, string, graph or xref run that exceeds its deadline fails as `provider_timeout`, naming the setting that owns the deadline and that raising it takes a restart. A project export that exceeds `REA_KUNA_PROJECT_TIMEOUT_MS` is an export failure instead: requests that need the export report `capability_unavailable` with the export's own retry guidance.
|
||||
|
||||
Windows is not supported yet. Supported targets are ELF and thin Mach-O executables for x86-64, AArch64 and ARM; PE and universal Mach-O are rejected with a typed reason.
|
||||
|
||||
@@ -218,6 +218,12 @@ Malformed provider replies return `unreadable_output` for
|
||||
`resolve_containing_procedure` before successful-result projection;
|
||||
cancellation and incomplete cleanup remain errors.
|
||||
|
||||
For a procedure match, `procedure_references` with `direction: "incoming"` can
|
||||
provide the selected provider's reference kinds when supported. Its scope covers
|
||||
the procedure body, so compare target addresses when selecting references to the
|
||||
exact entry. It is a separate observation and does not establish runtime use or
|
||||
complete reference discovery.
|
||||
|
||||
REA prepares complete MCP results within the pinned stdio client's 10 MiB
|
||||
receive-buffer budget, including both text and structured representations and
|
||||
room for the JSON-RPC envelope. Tool errors omit `structuredContent` from the
|
||||
@@ -292,6 +298,12 @@ limitation and reason IDs index that page's `items.messages` dictionary. Expand
|
||||
those references to recover exact text, repeated messages, and absent versus
|
||||
empty optional fields. Failed nodes retain their original error.
|
||||
|
||||
Retained `xrefs` Evidence also supports `native-observation` address pages and
|
||||
`native-context` source caveats. `xrefs` still returns complete Evidence by
|
||||
default and does not accept `detail`; when its MCP response exceeds the budget,
|
||||
follow the returned continuation to page the retained result. These addresses
|
||||
remain untyped; paging does not add reference kinds or establish completeness.
|
||||
|
||||
These views reference complete source context through `source_context.digest`
|
||||
and caveat counts. Read it using the same parent with
|
||||
`view: {"kind":"native-context","facet":"overview"}`. The other facets are
|
||||
@@ -371,6 +383,12 @@ is null. `total` counts all matching heads; each page preserves complete records
|
||||
in stable unknown ID order. Filters apply before paging. Pages reflect current
|
||||
session state, so keep the unknown ledger unchanged while traversing it.
|
||||
|
||||
`list_unknowns` only lists registered heads. Import restores the Unknown records
|
||||
present in a bundle; it does not turn residual question text embedded in Evidence
|
||||
into registry entries. An empty registry therefore does not establish that
|
||||
retained Evidence has no open questions. Inspect the relevant Evidence or its
|
||||
analysis view for its residual questions.
|
||||
|
||||
Automatically retained workflow questions include their immutable result
|
||||
Evidence ID. Distinct results retain independent questions and source links;
|
||||
repeating the same result does not create another unknown. Workflow Evidence
|
||||
|
||||
@@ -300,6 +300,20 @@ relationships but reports their completeness as unknown. In particular, a resolv
|
||||
tail call observed in one direction does not prove that the reverse inventory is
|
||||
complete. An empty Hopper inventory cannot establish that no such call exists.
|
||||
No whole-binary reverse scan is performed.
|
||||
|
||||
For large native traces, `trace_feature` accepts `detail: "summary"`. It retains
|
||||
the complete trace and returns counts and its parent Evidence ID. Use
|
||||
`inspect_analysis_view` with that ID and `view: {kind: "native-feature-trace",
|
||||
collection: "references", offset: 0, limit: 64}` to page the original rows;
|
||||
`matches` and `unknowns` are separate collections. Matches accept an exact
|
||||
`address`; references accept exact `source_address` and `target_address`
|
||||
filters. Row indices refer to the retained collection, and page coverage does not establish provider discovery
|
||||
completeness. `list_unknowns` pages the independently managed question heads.
|
||||
In the one-shot CLI, `rea trace ... --detail summary --evidence_output PATH`
|
||||
stores the trace and its linked source Evidence for later
|
||||
`inspect-analysis-view` calls. This bundle does not include the CLI session's
|
||||
Unknown registry; its original question texts remain available in the trace's
|
||||
`unknowns` pages.
|
||||
Regex searches use ECMAScript Unicode syntax in a cancellable worker with a
|
||||
matching deadline and a worker old-generation heap. Deadline or cancellation
|
||||
stops matching while leaving the Hopper API available. The deadline is
|
||||
@@ -388,6 +402,11 @@ sites remain distinct. A static route does not establish runtime reachability.
|
||||
Swift/closure dispatch requires resolved static references; unavailable ABI
|
||||
forms remain unknown.
|
||||
|
||||
Encoded Objective-C method records retain their own locations in dispatch
|
||||
Evidence. A handler node uses the implementation address with an unknown file
|
||||
offset unless that function already has an independently located graph node;
|
||||
the method-record offset does not locate the handler's code.
|
||||
|
||||
Incomplete reference kinds do not discard known typed calls or partial
|
||||
provider call references. A `provider_call` with `observed` resolution retains
|
||||
the provider-reported call site and endpoint without claiming generic direct,
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[الموقع](https://rea.tools/ar/) · [الأدلة](https://rea.tools/ar/guides/) · [دراسات الحالة](https://rea.tools/ar/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="انضم إلى مجتمع REA على Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[البدء السريع](#البدء-السريع) · [كيف يعمل REA](#كيف-يعمل-rea) · [ما الذي يمكنك تحليله](#ما-الذي-يمكنك-تحليله) · [دراسات الحالة](#دراسات-الحالة) · [إنشاء بيئات التعلم المعزز](#build-rl-environments) · [الأسئلة الشائعة](#الأسئلة-الشائعة) · [أدوات GNU objdump وRizin](../reverse-engineering-tools.md) · [التوثيق](#التوثيق)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Website](https://rea.tools/de/) · [Anleitungen](https://rea.tools/de/guides/) · [Fallstudien](https://rea.tools/de/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Der REA-Community auf Discord beitreten" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Schnellstart](#schnellstart) · [So funktioniert REA](#so-funktioniert-rea) · [Was du analysieren kannst](#was-du-analysieren-kannst) · [Fallstudien](#fallstudien) · [RL-Umgebungen erstellen](#build-rl-environments) · [Häufige Fragen](#häufige-fragen) · [Dokumentation](#dokumentation)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -19,6 +19,8 @@
|
||||
|
||||
**[Ιστότοπος](https://rea.tools/) · [Οδηγοί](https://rea.tools/guides/) · [Μελέτες περίπτωσης](https://rea.tools/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Έλα στην κοινότητα REA στο Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Γρήγορη εκκίνηση](#γρήγορη-εκκίνηση) · [Πώς λειτουργεί το REA](#πώς-λειτουργεί-το-rea) · [Τι μπορείς να αναλύσεις](#τι-μπορείς-να-αναλύσεις) · [Μελέτες περίπτωσης](#μελέτες-περίπτωσης) · [Δημιουργία περιβαλλόντων RL](#build-rl-environments) · [Συχνές ερωτήσεις](#συχνές-ερωτήσεις) · [Τεκμηρίωση](#τεκμηρίωση)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Sitio web](https://rea.tools/es/) · [Guías](https://rea.tools/es/guides/) · [Casos prácticos](https://rea.tools/es/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Únete a la comunidad de REA en Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Inicio rápido](#inicio-rápido) · [Cómo funciona REA](#cómo-funciona-rea) · [Qué puedes analizar](#qué-puedes-analizar) · [Casos prácticos](#casos-prácticos) · [Crear entornos de RL](#build-rl-environments) · [Preguntas frecuentes](#preguntas-frecuentes) · [Documentación](#documentación)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[وبسایت](https://rea.tools/fa/) · [راهنماها](https://rea.tools/fa/guides/) · [نمونههای عملی](https://rea.tools/fa/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="به انجمن REA در دیسکورد بپیوندید" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[شروع سریع](#شروع-سریع) · [REA چگونه کار میکند؟](#rea-چگونه-کار-میکند) · [چه چیزهایی را میتوان تحلیل کرد؟](#چه-چیزهایی-را-میتوان-تحلیل-کرد) · [نمونههای عملی](#نمونههای-عملی) · [ساخت محیطهای RL](#build-rl-environments) · [پرسشهای متداول](#پرسشهای-متداول) · [مستندات](#مستندات)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Site web](https://rea.tools/fr/) · [Guides](https://rea.tools/fr/guides/) · [Études de cas](https://rea.tools/fr/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Rejoignez la communauté REA sur Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Démarrage rapide](#démarrage-rapide) · [Comment fonctionne REA](#comment-fonctionne-rea) · [Ce que vous pouvez analyser](#ce-que-vous-pouvez-analyser) · [Études de cas](#études-de-cas) · [Créer des environnements RL](#build-rl-environments) · [Questions fréquentes](#questions-fréquentes) · [Documentation](#documentation)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Situs Web](https://rea.tools/id/) · [Panduan](https://rea.tools/id/guides/) · [Contoh Penggunaan](https://rea.tools/id/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Bergabunglah dengan komunitas REA di Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Mulai Cepat](#mulai-cepat) · [Cara Kerja REA](#cara-kerja-rea) · [Hal yang Dapat Anda Analisis](#hal-yang-dapat-anda-analisis) · [Contoh Penggunaan](#contoh-penggunaan) · [Membangun lingkungan RL](#build-rl-environments) · [FAQ](#faq) · [Dokumentasi](#dokumentasi)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Sito web](https://rea.tools/it/) · [Guide](https://rea.tools/it/guides/) · [Showcase](https://rea.tools/it/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Unisciti alla community REA su Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Avvio rapido](#avvio-rapido) · [Come funziona REA](#come-funziona-rea) · [Cosa puoi analizzare](#cosa-puoi-analizzare) · [Showcase](#showcase) · [Creare ambienti RL](#build-rl-environments) · [FAQ](#faq) · [Documentazione](#documentazione)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[ウェブサイト](https://rea.tools/ja/) · [ガイド](https://rea.tools/ja/guides/) · [事例](https://rea.tools/ja/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="DiscordのREAコミュニティに参加" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[クイックスタート](#クイックスタート) · [REA の仕組み](#rea-の仕組み) · [分析できる対象](#分析できる対象) · [事例](#事例) · [RL 環境を構築する](#build-rl-environments) · [よくある質問](#よくある質問) · [ドキュメント](#ドキュメント)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[웹사이트](https://rea.tools/ko/) · [가이드](https://rea.tools/ko/guides/) · [사례](https://rea.tools/ko/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Discord에서 REA 커뮤니티에 참여하세요" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[빠른 시작](#빠른-시작) · [REA의 작동 방식](#rea의-작동-방식) · [분석할 수 있는 대상](#분석할-수-있는-대상) · [사례](#사례) · [RL 환경 만들기](#build-rl-environments) · [자주 묻는 질문](#자주-묻는-질문) · [문서](#문서)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[വെബ്സൈറ്റ്](https://rea.tools/) · [ഗൈഡുകൾ](https://rea.tools/guides/) · [ഉദാഹരണങ്ങൾ](https://rea.tools/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Discord-ൽ REA കമ്മ്യൂണിറ്റിയിൽ ചേരുക" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[ദ്രുതാരംഭം](#ദ്രുതാരംഭം) · [REA എങ്ങനെ പ്രവർത്തിക്കുന്നു](#rea-എങ്ങനെ-പ്രവർത്തിക്കുന്നു) · [എന്തൊക്കെ വിശകലനം ചെയ്യാം](#എന്തൊക്കെ-വിശകലനം-ചെയ്യാം) · [ഉദാഹരണങ്ങൾ](#ഉദാഹരണങ്ങൾ) · [RL പരിതസ്ഥിതികൾ നിർമ്മിക്കുക](#build-rl-environments) · [പതിവുചോദ്യങ്ങൾ](#പതിവുചോദ്യങ്ങൾ) · [ഡോക്യുമെന്റേഷൻ](#ഡോക്യുമെന്റേഷൻ)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Strona internetowa](https://rea.tools/pl/) · [Przewodniki](https://rea.tools/pl/guides/) · [Przykłady zastosowań](https://rea.tools/pl/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Dołącz do społeczności REA na Discordzie" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Szybki start](#szybki-start) · [Jak działa REA](#jak-działa-rea) · [Co możesz analizować](#co-możesz-analizować) · [Przykłady zastosowań](#przykłady-zastosowań) · [Tworzenie środowisk RL](#build-rl-environments) · [FAQ](#faq) · [Dokumentacja](#dokumentacja)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Site](https://rea.tools/pt-BR/) · [Guias](https://rea.tools/pt-BR/guides/) · [Casos de uso](https://rea.tools/pt-BR/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Participe da comunidade REA no Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Início rápido](#início-rápido) · [Como o REA funciona](#como-o-rea-funciona) · [O que você pode analisar](#o-que-você-pode-analisar) · [Casos de uso](#casos-de-uso) · [Criar ambientes de RL](#build-rl-environments) · [Perguntas frequentes](#perguntas-frequentes) · [Documentação](#documentação)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Site web](https://rea.tools/) · [Ghiduri](https://rea.tools/guides/) · [Studii de caz](https://rea.tools/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Alătură-te comunității REA pe Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Pornire rapidă](#pornire-rapidă) · [Cum funcționează REA](#cum-funcționează-rea) · [Ce poți analiza](#ce-poți-analiza) · [Studii de caz](#studii-de-caz) · [Construiește medii RL](#build-rl-environments) · [Întrebări frecvente](#întrebări-frecvente) · [Documentație](#documentație)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Сайт](https://rea.tools/ru/) · [Руководства](https://rea.tools/ru/guides/) · [Примеры](https://rea.tools/ru/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Присоединяйтесь к сообществу REA в Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Быстрый старт](#быстрый-старт) · [Как работает REA](#как-работает-rea) · [Что можно анализировать](#что-можно-анализировать) · [Примеры](#примеры) · [Создание RL-окружений](#build-rl-environments) · [Частые вопросы](#частые-вопросы) · [Документация](#документация)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[เว็บไซต์](https://rea.tools/th/) · [คู่มือ](https://rea.tools/th/guides/) · [ตัวอย่างการใช้งาน](https://rea.tools/th/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="เข้าร่วมชุมชน REA บน Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[เริ่มต้นอย่างรวดเร็ว](#เริ่มต้นอย่างรวดเร็ว) · [REA ทำงานอย่างไร](#rea-ทำงานอย่างไร) · [สิ่งที่วิเคราะห์ได้](#สิ่งที่วิเคราะห์ได้) · [ตัวอย่างการใช้งาน](#ตัวอย่างการใช้งาน) · [สร้างสภาพแวดล้อม RL](#build-rl-environments) · [คำถามที่พบบ่อย](#คำถามที่พบบ่อย) · [เอกสาร](#เอกสาร)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Web sitesi](https://rea.tools/tr/) · [Rehberler](https://rea.tools/tr/guides/) · [Örnek çalışmalar](https://rea.tools/tr/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Discord'da REA topluluğuna katılın" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Hızlı başlangıç](#hızlı-başlangıç) · [REA nasıl çalışır](#rea-nasıl-çalışır) · [Neleri analiz edebilirsiniz](#neleri-analiz-edebilirsiniz) · [Örnek çalışmalar](#örnek-çalışmalar) · [RL ortamları oluşturun](#build-rl-environments) · [Sık sorulan sorular](#sık-sorulan-sorular) · [GNU objdump ve Rizin araçları](../reverse-engineering-tools.md) · [Belgeler](#belgeler)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Сайт](https://rea.tools/uk/) · [Посібники](https://rea.tools/uk/guides/) · [Приклади](https://rea.tools/uk/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Приєднуйтеся до спільноти REA в Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Швидкий старт](#швидкий-старт) · [Як працює REA](#як-працює-rea) · [Що можна аналізувати](#що-можна-аналізувати) · [Приклади](#приклади) · [Створення RL-середовищ](#build-rl-environments) · [Поширені запитання](#поширені-запитання) · [Документація](#документація)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Website](https://rea.tools/) · [Guides](https://rea.tools/guides/) · [Showcases](https://rea.tools/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Discord par REA community mein shamil hon" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Jaldi shuru karein](#jaldi-shuru-karein) · [REA kaise kaam karta hai](#rea-kaise-kaam-karta-hai) · [Kya kuch analyze kar sakte hain](#kya-kuch-analyze-kar-sakte-hain) · [Amli misaalein](#amli-misaalein) · [RL environments banayein](#build-rl-environments) · [FAQ](#faq) · [Documentation](#documentation)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[Trang web](https://rea.tools/vi/) · [Hướng dẫn](https://rea.tools/vi/guides/) · [Các nghiên cứu điển hình](https://rea.tools/vi/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="Tham gia cộng đồng REA trên Discord" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[Bắt đầu nhanh](#bắt-đầu-nhanh) · [Cách REA hoạt động](#cách-rea-hoạt-động) · [Những gì bạn có thể phân tích](#những-gì-bạn-có-thể-phân-tích) · [Các nghiên cứu điển hình](#các-nghiên-cứu-điển-hình) · [Xây dựng môi trường RL](#build-rl-environments) · [Câu hỏi thường gặp](#câu-hỏi-thường-gặp) · [Tài liệu](#tài-liệu)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[網站](https://rea.tools/zh-TW/) · [指南](https://rea.tools/zh-TW/guides/) · [案例](https://rea.tools/zh-TW/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="加入 REA 的 Discord 社群" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[快速開始](#快速開始) · [REA 的工作原理](#rea-的工作原理) · [可以分析什麼](#可以分析什麼) · [案例](#案例) · [建立 RL 環境](#build-rl-environments) · [常見問題](#常見問題) · [文件](#文件)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
|
||||
**[网站](https://rea.tools/zh-CN/) · [指南](https://rea.tools/zh-CN/guides/) · [案例](https://rea.tools/zh-CN/showcase/)**
|
||||
|
||||
<a href="https://discord.com/invite/GkcryMnJDM"><img alt="加入 REA 的 Discord 社区" src="https://img.shields.io/badge/Join%20us%20on%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" height="40"></a>
|
||||
|
||||
[快速开始](#快速开始) · [REA 的工作原理](#rea-的工作原理) · [可以分析什么](#可以分析什么) · [案例](#案例) · [构建 RL 环境](#build-rl-environments) · [常见问题](#常见问题) · [GNU objdump 和 Rizin 工具](../reverse-engineering-tools.md) · [文档](#文档)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -14,7 +14,13 @@ can run.
|
||||
|
||||
## GNU objdump
|
||||
|
||||
`inspect_with_objdump` and `rea inspect-with-objdump` capture stdout and stderr as inline Evidence. Artifact inspection Evidence records SHA-256 digests before and after the provider call and whether those observed digests differ; this cannot prove the exact bytes consumed if the artifact changes during execution. Choose one artifact operation: `file_headers`, `section_headers`, `symbols`, `relocations`, `dwarf`, `disassemble`, or `disassemble_all`. `disassemble` maps to `-d`; `disassemble_all` maps to `-D` and has different symbol-boundary semantics. The pathless `inspect_with_objdump {"operation":"architectures"}` MCP call or `rea list-objdump-architectures` CLI command maps to `-i` and produces subjectless provider-capability Evidence describing BFD targets compiled into the installed binary, not a guarantee that a target is supported.
|
||||
REA sends GNU-style arguments to the selected command but does not verify its
|
||||
implementation or version. Evidence identifies the command profile as
|
||||
`objdump.command` and retains the selected executable; it does not claim that
|
||||
the executable is GNU objdump. Other implementations may accept some commands
|
||||
while differing in supported options and output.
|
||||
|
||||
`inspect_with_objdump` and `rea inspect-with-objdump` capture stdout and stderr as inline Evidence. Artifact inspection Evidence records SHA-256 digests before and after the provider call and whether those observed digests differ; this cannot prove the exact bytes consumed if the artifact changes during execution. Choose one artifact operation: `file_headers`, `section_headers`, `symbols`, `relocations`, `dwarf`, `disassemble`, or `disassemble_all`. `disassemble` maps to `-d`; `disassemble_all` maps to `-D` and has different symbol-boundary semantics. The pathless `inspect_with_objdump {"operation":"architectures"}` MCP call or `rea list-objdump-architectures` CLI command maps to `-i` and produces subjectless Evidence containing the selected command’s output, not a guarantee that any target is supported. GNU objdump uses this option to list its compiled-in BFD targets.
|
||||
|
||||
Failure to launch the selected executable reports provider unavailability, the
|
||||
configured command and the host error under `boundary: "provider-launch"`.
|
||||
@@ -25,7 +31,7 @@ their rejected fields in input issues before artifact reads or provider launch.
|
||||
|
||||
For DWARF, REA does not follow local debug links by default. Set `follow_debug_links` to `true` to pass `-WK`; external debug-file identities are not bound to Evidence when following links is enabled. The CLI flag is `--follow-debug-links`; `--no-follow-debug-links` or `--follow-debug-links=false` keeps the default. A separate `true` or `false` word after the flag is not its value. REA clears `DEBUGINFOD_URLS` in the objdump child environment to disable debuginfod network access on GNU binutils versions that do not support `-WE`. The complete raw output is authoritative; REA does not normalize it into a version-independent disassembly schema.
|
||||
|
||||
For a known native symbol, use `operation: "disassemble", symbol: "my_function"` in MCP or `--symbol my_function` in CLI. REA passes `--disassemble=my_function` to objdump before collecting output, rather than producing the whole disassembly and filtering it afterward. Omit `symbol` to keep full code-section disassembly. Other operations reject this selector. Use a nonempty, NUL-free name with well-formed Unicode; unsupported values are rejected before reading the artifact or starting objdump. Symbol names are passed unchanged; matching, versioned symbols, aliases and the end of the disassembly follow the installed GNU objdump's semantics. Evidence records the selector and actual arguments. An empty match does not establish artifact-wide absence, and the selected observation does not contain the omitted disassembly; request the complete operation when needed.
|
||||
For a known native symbol, use `operation: "disassemble", symbol: "my_function"` in MCP or `--symbol my_function` in CLI. REA passes `--disassemble=my_function` to objdump before collecting output, rather than producing the whole disassembly and filtering it afterward. Omit `symbol` to keep full code-section disassembly. Other operations reject this selector. Use a nonempty, NUL-free name with well-formed Unicode; unsupported values are rejected before reading the artifact or starting objdump. Symbol names are passed unchanged; matching, versioned symbols, aliases and the end of the disassembly follow the selected executable's semantics. Evidence records the selector and actual arguments. An empty match does not establish artifact-wide absence, and the selected observation does not contain the omitted disassembly; request the complete operation when needed.
|
||||
|
||||
```sh
|
||||
rea inspect-with-objdump ./program disassemble
|
||||
|
||||
@@ -86,7 +86,10 @@ credentials and resets it between sequences.
|
||||
environment refused the call), `not-served`, `no-tool` and `unmapped-id`, each with a JSON pointer and short values.
|
||||
- The report gives agreement overall, per probe and per tool, agreement ignoring item order beside it, the counts per
|
||||
category, the ids mapped and the values normalised, the records left uncompared, and each step's differences (at most
|
||||
20 per step).
|
||||
20 per step). Each step gives `differences_total` before that cap, `differences_omitted`, and
|
||||
`differences_truncated`, true when entries were omitted. Equality and categories use all differences.
|
||||
Omitted entries are not retained and `--full-output` cannot recover them; further investigation uses the original
|
||||
capture and probe Evidence.
|
||||
|
||||
## Non-goals
|
||||
|
||||
|
||||
+20
-7
@@ -269,8 +269,8 @@ APK for the negative check. No engine or external tool is involved. Record:
|
||||
Flutter Gallery 2.9.2 (112 MB, arm64-v8a / armeabi-v7a / x86_64) — every
|
||||
ABI reports the same agreeing snapshot hash
|
||||
(`65817c30a78bb44c3dc3771876b6010a`, two sections each) read 20 bytes after
|
||||
the Dart snapshot magic, per-ABI engine GNU build-ids, no labeled Dart
|
||||
version string (expected for a release engine build), and the Android
|
||||
the Dart snapshot magic, per-ABI engine GNU build-ids, an unlabeled
|
||||
Dart-shaped version candidate, and the Android
|
||||
clang/LLVM toolchain line; the AOT inspection locates all four snapshot
|
||||
symbols through the dynamic symbol table on both ELF64 and ELF32 images,
|
||||
validates the data-section headers, agrees with the identification hash,
|
||||
@@ -284,7 +284,7 @@ prefixes, with no private local-file dependency. The verifier now compares
|
||||
compiled CLI and stdio MCP Evidence on every ABI, including data-header kind
|
||||
and declared lengths. Add
|
||||
`--expectations scripts/fixtures/flutter-gallery-expectations.json` for the
|
||||
documented Gallery APK to assert its pinned digest, hash, exact engine labels
|
||||
documented Gallery APK to assert its pinned digest, hash, exact engine strings
|
||||
and per-ABI URI counts. The verifier requires complete structural snapshot
|
||||
coverage, reports bounded pool coverage as partial/unverified, and reports
|
||||
exact labels/counts on other APKs unverified without an expectations file.
|
||||
@@ -421,7 +421,10 @@ the first cold pseudocode answer comes from a single-function run or an export
|
||||
that already finished, assembly waits for and serves from that export, and
|
||||
pseudocode and both dossier facets match their recorded file spans and digests,
|
||||
a fresh server reuses the export without rebuilding it, staging is empty after
|
||||
a completed export, and no owned process survives. Pass
|
||||
a completed export, and no owned process survives. A separate forced 1 ms
|
||||
project deadline checks that a timed-out inventory recommends raising the
|
||||
project deadline without changing jobs; a fresh server retries with 60 seconds
|
||||
and the same two-worker setting. Pass
|
||||
`--targets DIR --functions a,b` to add caller-built binaries laid out as
|
||||
`DIR/<flavor>/<name>` beside `<name>.stripped`; addresses come from the
|
||||
symbol-bearing build. `--compare-ghidra` with `GHIDRA_INSTALL_DIR` runs those
|
||||
@@ -451,6 +454,16 @@ Mach-O fixture and caller-built symbol-bearing and stripped x86-64 ELF fixtures.
|
||||
Linux and other hosts have not been verified with 1.988; this is not a repeat
|
||||
of the larger 1.833 matrix.
|
||||
|
||||
Kuna 1.989 passed the real-provider lane on macOS arm64 with the generated
|
||||
symbol-bearing and stripped Mach-O fixtures, including cold and warm reads,
|
||||
references, and cleanup. Linux, other hosts, and the larger 1.833 target matrix
|
||||
were not verified with 1.989.
|
||||
|
||||
Kuna 1.990 passed the real-provider lane on macOS arm64 with the generated
|
||||
symbol-bearing and stripped Mach-O fixtures, including cold and warm reads,
|
||||
references, timeout recovery, and cleanup. Linux, other hosts, and the larger
|
||||
1.833 target matrix were not verified with 1.990.
|
||||
|
||||
## DOS Ghidra analysis
|
||||
|
||||
`npm run verify:ghidra:dos` requires the supported Ghidra and JDK installation
|
||||
@@ -1103,13 +1116,13 @@ coverage limits.
|
||||
### Offline binary layout
|
||||
|
||||
`npm run verify:binary:layout` requires Linux x64, GCC/binutils, the
|
||||
`i686-linux-gnu`, `arm-linux-gnueabi`, `mips-linux-gnu` and `mipsel-linux-gnu`
|
||||
cross `gcc`/`readelf` toolchains, absolute
|
||||
`aarch64-linux-gnu`, `i686-linux-gnu`, `arm-linux-gnueabi`, `mips-linux-gnu` and
|
||||
`mipsel-linux-gnu` cross `gcc`/`readelf` toolchains, absolute
|
||||
`REA_PWNTOOLS_PYTHON` with pwntools 4.15.0/pyelftools 0.33/Unicorn 2.1.2 and
|
||||
absolute `REA_VERIFY_STRACE_COMMAND`. It compiles ephemeral source-owned ELF
|
||||
fixtures and checks public CLI/MCP, lossless addresses/names, file ranges,
|
||||
mitigation inferences, malformed/unsupported input, original file hashes and
|
||||
released process ownership. Cross-compiled ELF32 executables and relocatable
|
||||
released process ownership. Cross-compiled ELF64 AArch64 and ELF32 executables and relocatable
|
||||
objects check the reported class, machine and byte order, and every section
|
||||
name, address, offset and size against GNU `readelf`. Exec syscall tracing must identify only the declared
|
||||
Node/Python launchers; no target binary is executed. Core/debugger claims need
|
||||
|
||||
@@ -97,9 +97,13 @@ it does not invent an empty string or body bytes.
|
||||
The observed Playwright 1.58.2 profile can report a bracketed IPv6
|
||||
`serverIPAddress`, forwarded from Chromium's `remoteIPAddress` through
|
||||
[Playwright's HAR tracer](https://github.com/microsoft/playwright/blob/v1.58.2/packages/playwright-core/src/server/har/harTracer.ts).
|
||||
Chromium DevTools exports (declared as `WebInspector` `537.36`, the frozen
|
||||
AppleWebKit version in Chromium's user agent) report the same bracketed form:
|
||||
[its HAR writer](https://github.com/ChromeDevTools/devtools-frontend/blob/63739e8ec45ca01b9e4be9b8b05946acdadd8f1f/front_end/models/har/Log.ts)
|
||||
strips the port but not the brackets.
|
||||
REA preserves the original spelling in reported Evidence and removes brackets
|
||||
only from its validation copy, only for this declared producer and a valid IPv6
|
||||
literal. Invalid addresses, bracketed IPv4, ports and zone identifiers remain
|
||||
only from its validation copy, only for these declared producers and a valid
|
||||
IPv6 literal. Invalid addresses, bracketed IPv4, ports and zone identifiers remain
|
||||
validation failures. Producer declarations do not authenticate origin.
|
||||
|
||||
Native mitmproxy decoding requires a caller-supplied **mitmdump 12.2.3 on
|
||||
|
||||
+1
-1
@@ -22,7 +22,7 @@ Results retain:
|
||||
- Main-document script inventory, exact session script IDs, resource start coordinates, sourceURL/source-map declarations, producer hash and independently computed UTF-8 text digest.
|
||||
- Armed-window requests from the selected main frame and selected origins, complete reported initiator objects (including stack hierarchy/descriptions), derived callsites, exact script-ID associations and unresolved async parent IDs. Source association does not establish UI causality. Callsite and listener `line_number`/`column_number` values are CDP's zero-based lines and UTF-16 columns in the resource named by `url`. With a `script_id`, that is the script's enclosing resource: the script file for an external script, or the HTML document for an inline script. `resource_start` is where the script text starts in that resource. A callsite with an empty `script_id` comes from the request's reported initiator rather than a script call frame, such as markup that the parser read in a document or a module import in a script file. It still locates a position in `url`, but has no session-script association, so it cannot be joined to captured source or `resource_start`. `trace_web_source_location` takes one-based, script-relative lines and zero-based UTF-16 columns.
|
||||
|
||||
For an external script, where `url` is the script file, use `line = line_number + 1` and keep `column = column_number`; this also applies to an empty-`script_id` initiator position in a script file. For an inline script, which requires a `script_id` and its `resource_start`, use `line = line_number - resource_start.line_number + 1`. Subtract `resource_start.column_number` from `column_number` only when `line_number === resource_start.line_number`; keep the reported column on later lines. Columns remain zero-based. This conversion requires known source-start and column coordinates; preserve unavailable coordinates as unknown.
|
||||
For an external script, where `url` is the script file, use `line = line_number + 1` and keep `column = column_number`; this also applies to an empty-`script_id` initiator position in a script file. For an inline script, which requires a `script_id` and its `resource_start`, use `line = line_number - resource_start.line_number + 1`. Subtract `resource_start.column_number` from `column_number` only when `line_number === resource_start.line_number`; keep the reported column on later lines. Columns remain zero-based. This conversion requires known source-start and column coordinates; preserve unavailable coordinates as unknown. Chrome reports a negative start column for some inline scripts, such as one in a helper frame, and positions on that script's first line include it. REA keeps those columns as reported rather than failing the observation; a negative column does not locate a column in `url`, but the subtraction above still gives the script-relative column.
|
||||
|
||||
The coverage counter interval starts at backend acceptance before the armed receipt and ends at the resetting sample after the local timer. Counts may include execution during these command intervals; request events are restricted to the locally armed window. The result retains both backend timestamps and the distinct local clock.
|
||||
|
||||
|
||||
@@ -111,7 +111,7 @@
|
||||
+ existingSha = data.sha;
|
||||
+ }
|
||||
+ catch (error) {
|
||||
+ if (!(error instanceof request_error_1.RequestError && error.status === 404)) {
|
||||
+ if (!(error && error.status === 404)) {
|
||||
+ throw error;
|
||||
+ }
|
||||
+ }
|
||||
|
||||
@@ -3,6 +3,7 @@ export interface ReadinessPullRequest {
|
||||
readonly state: "open" | "closed";
|
||||
readonly draft: boolean;
|
||||
readonly body: string | null;
|
||||
readonly user: { readonly id: number } | null;
|
||||
readonly head: { readonly sha: string };
|
||||
readonly base: { readonly ref: string };
|
||||
}
|
||||
@@ -53,7 +54,7 @@ export interface ReadinessGitHub {
|
||||
>;
|
||||
}
|
||||
export function evaluateReadiness(
|
||||
pull: Pick<ReadinessPullRequest, "body" | "draft">,
|
||||
pull: Pick<ReadinessPullRequest, "body" | "draft" | "user">,
|
||||
template: string,
|
||||
): ReadinessResult;
|
||||
export function runReadinessCheck(input: {
|
||||
|
||||
@@ -2,6 +2,14 @@ import { readFile } from "node:fs/promises";
|
||||
|
||||
const result = (state, description) => ({ state, description });
|
||||
|
||||
// GitHub account IDs remain stable when a login changes.
|
||||
const EXEMPT_AUTHOR_IDS = new Set([
|
||||
76467478, // morluto
|
||||
208281322, // kaoru0822-kitauji
|
||||
161784452, // N0zoM1z0
|
||||
342580529, // gensokyo-dev
|
||||
]);
|
||||
|
||||
const OPEN_HEADS_QUERY = `query ReadinessOpenHeads($owner: String!, $name: String!, $after: String) {
|
||||
repository(owner: $owner, name: $name) {
|
||||
pullRequests(first: 100, after: $after, states: OPEN, orderBy: {field: CREATED_AT, direction: ASC}) {
|
||||
@@ -105,6 +113,11 @@ export function evaluateReadiness(pull, template) {
|
||||
"pending",
|
||||
"Draft PR: complete Author readiness before requesting review.",
|
||||
);
|
||||
if (EXEMPT_AUTHOR_IDS.has(pull.user?.id))
|
||||
return result(
|
||||
"success",
|
||||
"PR author is exempt from the readiness declaration.",
|
||||
);
|
||||
const expected = readinessLines(template).flatMap((line) => {
|
||||
const match = /^- \[ \] (.+)$/u.exec(line);
|
||||
return match ? [match[1]] : [];
|
||||
@@ -210,6 +223,7 @@ export async function runReadinessCheck({ github, context, core }) {
|
||||
latest.head.sha !== pull.head.sha ||
|
||||
latest.body !== pull.body ||
|
||||
latest.draft !== pull.draft ||
|
||||
latest.user?.id !== pull.user?.id ||
|
||||
latest.base.ref !== pull.base.ref
|
||||
)
|
||||
verdict = result(
|
||||
|
||||
@@ -704,7 +704,9 @@ export const CI_LANES = {
|
||||
agentenv: sameJobs(PATH_SCOPES.agentenv, ["agentenv"]),
|
||||
nativeaot: sameJobs(PATH_SCOPES.nativeaot, ["nativeaot"]),
|
||||
ghidra_bridge: sameJobs(PATH_SCOPES.ghidra_bridge, ["ghidra_bridge"]),
|
||||
website: sameJobs(PATH_SCOPES.website, ["website"]),
|
||||
website: sameJobs(PATH_SCOPES.website, ["website"], {
|
||||
earlyInputs: [".agents/skills/reverse-engineer-anything/**"],
|
||||
}),
|
||||
};
|
||||
// An input owner omitted from the registry must never silently lose a lane.
|
||||
for (const [name, paths] of Object.entries(PATH_SCOPES))
|
||||
|
||||
@@ -3,19 +3,19 @@
|
||||
"snapshot_hash": "65817c30a78bb44c3dc3771876b6010a",
|
||||
"engines": {
|
||||
"x86_64": {
|
||||
"dart_version": null,
|
||||
"dart_version": "2.14.0-151.0.dev (dev) (Tue May 25 10:58:39 2021 -0700)",
|
||||
"toolchain_lines": [
|
||||
"Android (5900059 based on r365631c) clang version 9.0.8 (https://android.googlesource.com/toolchain/llvm-project 207d7abc1a2abf3ef8d4301736d6a7ebc224a290) (based on LLVM 9.0.8svn)"
|
||||
]
|
||||
},
|
||||
"armeabi-v7a": {
|
||||
"dart_version": null,
|
||||
"dart_version": "2.14.0-151.0.dev (dev) (Tue May 25 10:58:39 2021 -0700)",
|
||||
"toolchain_lines": [
|
||||
"Android (5900059 based on r365631c) clang version 9.0.8 (https://android.googlesource.com/toolchain/llvm-project 207d7abc1a2abf3ef8d4301736d6a7ebc224a290) (based on LLVM 9.0.8svn)"
|
||||
]
|
||||
},
|
||||
"arm64-v8a": {
|
||||
"dart_version": null,
|
||||
"dart_version": "2.14.0-151.0.dev (dev) (Tue May 25 10:58:39 2021 -0700)",
|
||||
"toolchain_lines": [
|
||||
"Android (5900059 based on r365631c) clang version 9.0.8 (https://android.googlesource.com/toolchain/llvm-project 207d7abc1a2abf3ef8d4301736d6a7ebc224a290) (based on LLVM 9.0.8svn)"
|
||||
]
|
||||
|
||||
@@ -114,6 +114,46 @@ export const verifyKunaReferences = async ({
|
||||
);
|
||||
}
|
||||
assert.equal(trace.value.truncated, false);
|
||||
const summary = await call("trace_feature", {
|
||||
query: name,
|
||||
detail: "summary",
|
||||
});
|
||||
assert.equal(
|
||||
summary.value.parent_evidence_id,
|
||||
trace.structured.evidence_id,
|
||||
);
|
||||
assert.equal(
|
||||
summary.value.summary.references,
|
||||
trace.value.references.length,
|
||||
);
|
||||
assert.equal(
|
||||
summary.value.summary.unknowns,
|
||||
trace.value.residual_unknowns.length,
|
||||
);
|
||||
for (const [collection, values] of [
|
||||
["matches", trace.value.matches],
|
||||
["references", trace.value.references],
|
||||
["unknowns", trace.value.residual_unknowns],
|
||||
]) {
|
||||
const view = await call("inspect_analysis_view", {
|
||||
source: {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: trace.structured.evidence_id,
|
||||
},
|
||||
view: {
|
||||
kind: "native-feature-trace",
|
||||
collection,
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
},
|
||||
});
|
||||
assert.equal(view.value.coverage.total, values.length);
|
||||
assert.deepEqual(
|
||||
view.value.items.rows.map(({ value }) => value),
|
||||
values.slice(0, 1),
|
||||
);
|
||||
assert.equal(view.value.items.collection, collection);
|
||||
}
|
||||
}
|
||||
composed = {
|
||||
graph_ms: graph.ms,
|
||||
|
||||
@@ -253,20 +253,9 @@ async function verifyNativeValueAdapters(
|
||||
functionEvidence.analysis_profile,
|
||||
);
|
||||
assert.deepEqual(functionProfile, upstreamProfile);
|
||||
const comparison = await client.callTool(
|
||||
{
|
||||
name: "compare_functions",
|
||||
arguments: {
|
||||
left: analyzed.structuredContent,
|
||||
right: analyzed.structuredContent,
|
||||
},
|
||||
},
|
||||
{ timeout: 180000 },
|
||||
);
|
||||
assert.notEqual(comparison.isError, true, JSON.stringify(comparison));
|
||||
assert.notEqual(comparison.structuredContent, undefined);
|
||||
const comparisonEvidence = parseEvidence(
|
||||
requireMcpOutput(comparison, "compare_functions"),
|
||||
const comparisonEvidence = await compareRetainedFunctionEvidence(
|
||||
client,
|
||||
functionEvidence,
|
||||
);
|
||||
assert.equal(
|
||||
comparisonEvidence.normalized_result.status,
|
||||
@@ -395,21 +384,27 @@ async function verifyUnresolvedCallEvidence(client) {
|
||||
assert.equal(trace.goal_status, "unknown");
|
||||
assert.equal(trace.truncated, true);
|
||||
assert.ok(trace.residual_unknowns.length > 0);
|
||||
const compared = await compareRetainedFunctionEvidence(client, evidence);
|
||||
assert.equal(compared.normalized_result.status, "unknown");
|
||||
}
|
||||
|
||||
/** Compare an already admitted function record without retransmitting its payload. */
|
||||
export async function compareRetainedFunctionEvidence(client, evidence) {
|
||||
const reference = {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: evidence.evidence_id,
|
||||
};
|
||||
const compared = await client.callTool(
|
||||
{
|
||||
name: "compare_functions",
|
||||
arguments: {
|
||||
left: analyzed.structuredContent,
|
||||
right: analyzed.structuredContent,
|
||||
left: reference,
|
||||
right: reference,
|
||||
},
|
||||
},
|
||||
{ timeout: 180000 },
|
||||
);
|
||||
assert.notEqual(compared.isError, true, JSON.stringify(compared));
|
||||
assert.notEqual(compared.structuredContent, undefined);
|
||||
assert.equal(
|
||||
parseEvidence(requireMcpOutput(compared, "compare_functions"))
|
||||
.normalized_result.status,
|
||||
"unknown",
|
||||
);
|
||||
return parseEvidence(requireMcpOutput(compared, "compare_functions"));
|
||||
}
|
||||
@@ -7,6 +7,7 @@ import xml.etree.ElementTree as ET
|
||||
from zipfile import ZIP_DEFLATED, ZipFile, ZipInfo
|
||||
|
||||
from website import PageMetadata, page_url
|
||||
from website_agents import prepare_discovery, prepare_markdown
|
||||
|
||||
|
||||
EXAMPLE_FILES = (
|
||||
@@ -81,6 +82,8 @@ def prepare_assets(site):
|
||||
prepare_sharing_image(site)
|
||||
prepare_sitemap(site)
|
||||
prepare_notes_example(site)
|
||||
prepare_discovery(Path(__file__).resolve().parent.parent, site)
|
||||
prepare_markdown(site)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
"""Check the public build artifacts and Markdown content contract."""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
from zipfile import ZipFile
|
||||
|
||||
from bs4 import BeautifulSoup
|
||||
import yaml
|
||||
|
||||
from website_agents import prepare_discovery, prepare_markdown
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
class AgentPublishingChecks(unittest.TestCase):
|
||||
def test_published_skill_digest_covers_every_reference_and_catalogs_resolve(self):
|
||||
site = ROOT / "website/_site"
|
||||
index = json.loads((site / ".well-known/agent-skills/index.json").read_text())
|
||||
self.assertEqual(index["$schema"], "https://schemas.agentskills.io/discovery/0.2.0/schema.json")
|
||||
skill, = index["skills"]
|
||||
self.assertEqual(skill["name"], "reverse-engineer-anything")
|
||||
self.assertEqual(skill["type"], "archive")
|
||||
artifact = site / skill["url"].removeprefix("https://rea.tools/")
|
||||
self.assertEqual(skill["digest"], "sha256:" + hashlib.sha256(artifact.read_bytes()).hexdigest())
|
||||
source = ROOT / ".agents/skills/reverse-engineer-anything"
|
||||
with ZipFile(artifact) as archive:
|
||||
expected = {f"{source.name}/{p.relative_to(source).as_posix()}": p.read_bytes()
|
||||
for p in source.rglob("*.md")}
|
||||
self.assertEqual(set(archive.namelist()), set(expected))
|
||||
for name, data in expected.items():
|
||||
self.assertEqual(archive.read(name), data)
|
||||
manifest = archive.read(f"{source.name}/SKILL.md").decode()
|
||||
metadata = yaml.safe_load(manifest[4:].split("\n---\n", 1)[0])
|
||||
self.assertEqual(skill["description"], metadata["description"])
|
||||
catalog = json.loads((site / ".well-known/ai-catalog.json").read_text())
|
||||
self.assertEqual(catalog["specVersion"], "1.0")
|
||||
entry, = catalog["entries"]
|
||||
self.assertEqual(entry["identifier"], "urn:air:rea.tools:skill:reverse-engineer-anything")
|
||||
self.assertEqual(entry["url"], skill["url"])
|
||||
self.assertNotIn("data", entry)
|
||||
ard = json.loads((site / ".well-known/ard.json").read_text())
|
||||
resource, = ard["entries"]
|
||||
self.assertEqual(resource["url"], skill["url"])
|
||||
self.assertTrue(2 <= len(resource["representativeQueries"]) <= 5)
|
||||
|
||||
def test_reference_changes_invalidate_digest_and_yaml_scalars_are_parsed(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
source = root / ".agents/skills/reverse-engineer-anything"
|
||||
source.mkdir(parents=True)
|
||||
(source / "SKILL.md").write_text(
|
||||
'---\nname: reverse-engineer-anything\ndescription: >-\n Inspect programs:\n preserve evidence.\n---\n# REA\n'
|
||||
)
|
||||
(source / "references").mkdir()
|
||||
reference = source / "references/evidence.md"
|
||||
reference.write_text("First revision")
|
||||
site = root / "site"
|
||||
def build():
|
||||
prepare_discovery(root, site)
|
||||
return json.loads((site / ".well-known/agent-skills/index.json").read_text())["skills"][0]
|
||||
before = build()
|
||||
self.assertEqual(before["description"], "Inspect programs: preserve evidence.")
|
||||
self.assertEqual(build(), before)
|
||||
reference.write_text("Second revision")
|
||||
self.assertNotEqual(build()["digest"], before["digest"])
|
||||
|
||||
def test_markdown_preserves_content_and_resources_without_publishing_excluded_pages(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
site = Path(directory)
|
||||
page = site / "zh-CN/guides/test/index.html"
|
||||
page.parent.mkdir(parents=True)
|
||||
page.write_text('''<html><head><title>REA: test</title></head><body>
|
||||
<nav>Outside navigation</nav><main><aside>Shortened display paths.</aside>
|
||||
<h1>Inspect a program</h1><nav>Contents navigation</nav>
|
||||
<ol><li><span class="flow-label">Read</span><span class="flow-node"><code>input.exe</code></span><span class="flow-note">Keep evidence.</span></li></ol>
|
||||
<p><a href="../other/?q=a%2Fb#code">Another guide</a></p>
|
||||
<pre><code>const value = a < b ? "x_y" : "&";\n keepIndent();</code></pre>
|
||||
<details><summary>Evidence</summary><p>Hidden evidence remains available.</p></details>
|
||||
<img src="../../../assets/diagram.svg" alt="Investigation diagram">
|
||||
<button>Copy code</button><script>doNotPublish()</script></main></body></html>''')
|
||||
fixture = site / "examples/notes-web/index.html"
|
||||
fixture.parent.mkdir(parents=True)
|
||||
fixture.write_text("<main>Target application</main>")
|
||||
fallback = site / "zh-CN/fallback/index.html"
|
||||
fallback.parent.mkdir(parents=True)
|
||||
fallback.write_text('<head><meta name="robots" content="noindex, follow"></head><main>English fallback notice.</main>')
|
||||
prepare_markdown(site)
|
||||
markdown = page.with_suffix(".md").read_text()
|
||||
self.assertIn('# Inspect a program', markdown)
|
||||
self.assertIn('const value = a < b ? "x_y" : "&";\n keepIndent();', markdown)
|
||||
self.assertIn("Hidden evidence remains available.", markdown)
|
||||
self.assertIn("Shortened display paths.", markdown)
|
||||
self.assertNotIn("Read`input.exe`", markdown)
|
||||
self.assertIn("https://rea.tools/zh-CN/guides/other/?q=a%2Fb#code", markdown)
|
||||
self.assertIn("https://rea.tools/assets/diagram.svg", markdown)
|
||||
for text in ("Outside navigation", "Contents navigation", "Copy code", "doNotPublish"):
|
||||
self.assertNotIn(text, markdown)
|
||||
self.assertFalse(fixture.with_suffix(".md").exists())
|
||||
self.assertFalse(fallback.with_suffix(".md").exists())
|
||||
html = BeautifulSoup(page.read_text(), "html.parser")
|
||||
self.assertEqual(html.find("link", rel="alternate")["href"], "https://rea.tools/zh-CN/guides/test/index.md")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -38,15 +38,16 @@ if (
|
||||
!isAbsolute(strace ?? "")
|
||||
)
|
||||
throw new Error(
|
||||
"verify:binary:layout requires Linux x64, absolute REA_PWNTOOLS_PYTHON (pwntools 4.15.0/pyelftools 0.33/Unicorn 2.1.2) and REA_VERIFY_STRACE_COMMAND. gcc, ld, strip and the i686/ARM/MIPS/MIPSel linux-gnu cross gcc and readelf must be available. No target is executed.",
|
||||
"verify:binary:layout requires Linux x64, absolute REA_PWNTOOLS_PYTHON (pwntools 4.15.0/pyelftools 0.33/Unicorn 2.1.2) and REA_VERIFY_STRACE_COMMAND. gcc, ld, strip and the AArch64/i686/ARM/MIPS/MIPSel linux-gnu cross gcc and readelf must be available. No target is executed.",
|
||||
);
|
||||
const run = createVerifierRun();
|
||||
const execute = promisify(execFile);
|
||||
const ELF32_TOOLCHAINS = [
|
||||
["i686-linux-gnu", "EM_386", "little"],
|
||||
["arm-linux-gnueabi", "EM_ARM", "little"],
|
||||
["mips-linux-gnu", "EM_MIPS", "big"],
|
||||
["mipsel-linux-gnu", "EM_MIPS", "little"],
|
||||
const CROSS_TOOLCHAINS = [
|
||||
["aarch64-linux-gnu", "EM_AARCH64", 64, "little"],
|
||||
["i686-linux-gnu", "EM_386", 32, "little"],
|
||||
["arm-linux-gnueabi", "EM_ARM", 32, "little"],
|
||||
["mips-linux-gnu", "EM_MIPS", 32, "big"],
|
||||
["mipsel-linux-gnu", "EM_MIPS", 32, "little"],
|
||||
];
|
||||
for (const command of [
|
||||
"gcc",
|
||||
@@ -54,7 +55,7 @@ for (const command of [
|
||||
"strip",
|
||||
"/usr/bin/prlimit",
|
||||
strace,
|
||||
...ELF32_TOOLCHAINS.flatMap(([prefix]) => [
|
||||
...CROSS_TOOLCHAINS.flatMap(([prefix]) => [
|
||||
`${prefix}-gcc`,
|
||||
`${prefix}-readelf`,
|
||||
]),
|
||||
@@ -347,14 +348,15 @@ try {
|
||||
cases++;
|
||||
}
|
||||
}
|
||||
// ELF32 profiles: cross-compiled, never executed, checked against GNU readelf.
|
||||
// Cross profiles: cross-compiled, never executed, checked against GNU readelf.
|
||||
for (const [
|
||||
prefix,
|
||||
machine,
|
||||
bits,
|
||||
byteOrder,
|
||||
kind,
|
||||
flags,
|
||||
] of ELF32_TOOLCHAINS.flatMap((toolchain) => [
|
||||
] of CROSS_TOOLCHAINS.flatMap((toolchain) => [
|
||||
[...toolchain, "executable", ["-fPIE", "-pie"]],
|
||||
[...toolchain, "relocatable", ["-c", "-fPIC"]],
|
||||
])) {
|
||||
@@ -423,16 +425,18 @@ try {
|
||||
`readelf relocation count: ${name}`,
|
||||
);
|
||||
for (const [entryIndex, [, offset, info]] of rows.entries()) {
|
||||
// ELF32 r_info uses the low byte for type and the upper 24 bits for
|
||||
// the symbol index. readelf has already decoded the file's byte order.
|
||||
// ELF32 r_info packs an 8-bit type under a 24-bit symbol index;
|
||||
// ELF64 packs a 32-bit type under a 32-bit index. readelf has already
|
||||
// decoded the file's byte order.
|
||||
const decodedInfo = BigInt(`0x${info}`);
|
||||
const typeBits = bits === 32 ? 8n : 32n;
|
||||
expectedRelocations.push({
|
||||
section_index: section.index,
|
||||
entry_index: entryIndex,
|
||||
reported_offset: BigInt(`0x${offset}`),
|
||||
type: Number(decodedInfo & 0xffn),
|
||||
type: Number(decodedInfo & ((1n << typeBits) - 1n)),
|
||||
symbol_table_index: section.link,
|
||||
symbol_index: Number(decodedInfo >> 8n),
|
||||
symbol_index: Number(decodedInfo >> typeBits),
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -445,7 +449,7 @@ try {
|
||||
const value = await inspect(mode, path);
|
||||
assert.deepEqual(value.architecture, {
|
||||
machine,
|
||||
bits: 32,
|
||||
bits,
|
||||
byte_order: byteOrder,
|
||||
});
|
||||
assert.equal(
|
||||
@@ -1312,7 +1316,8 @@ try {
|
||||
}
|
||||
}
|
||||
}
|
||||
unsupported.writeUInt16LE(183, 18);
|
||||
// EM_RISCV: AArch64 (183) is now an admitted profile.
|
||||
unsupported.writeUInt16LE(243, 18);
|
||||
const core = Buffer.from(object);
|
||||
core.writeUInt16LE(4, 16);
|
||||
for (const [name, bytes, category] of [
|
||||
@@ -1337,7 +1342,7 @@ try {
|
||||
"invalid_input",
|
||||
],
|
||||
["undersized-program-header", undersizedProgramHeader, "invalid_input"],
|
||||
["arm64", unsupported, "unsupported_target"],
|
||||
["riscv64", unsupported, "unsupported_target"],
|
||||
["core", core, "unsupported_target"],
|
||||
]) {
|
||||
const path = join(root.path, name);
|
||||
|
||||
@@ -28,7 +28,10 @@ import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { inspectKunaInstallation } from "../dist/kuna/KunaInstallation.js";
|
||||
import { ArtifactResourceScope } from "../dist/artifacts/ArtifactResourceScope.js";
|
||||
import { AnalysisCleanupLease } from "../dist/artifacts/AnalysisCleanupLease.js";
|
||||
import { requireMcpOperationResult } from "./lib/mcp-verifier-results.mjs";
|
||||
import {
|
||||
requireMcpOperationResult,
|
||||
requireMcpToolError,
|
||||
} from "./lib/mcp-verifier-results.mjs";
|
||||
import { verifyKunaReferences } from "./lib/kuna-reference-verifier.mjs";
|
||||
import {
|
||||
nativeKunaStrings,
|
||||
@@ -172,7 +175,7 @@ const sampleTree = async (root) => {
|
||||
};
|
||||
|
||||
/** One MCP server with a selected provider, sampled while it runs. */
|
||||
const openServer = async (provider) => {
|
||||
const openServer = async (provider, kunaSettings = {}) => {
|
||||
const owner = Symbol(provider.id);
|
||||
pendingServers.add(owner);
|
||||
const transport = new StdioClientTransport({
|
||||
@@ -185,6 +188,7 @@ const openServer = async (provider) => {
|
||||
REA_PROCESS_RUN_ID: run.run_id,
|
||||
REA_KUNA_COMMAND: kuna,
|
||||
REA_KUNA_CACHE_DIR: cacheRoot,
|
||||
...kunaSettings,
|
||||
...(process.env.GHIDRA_INSTALL_DIR === undefined
|
||||
? {}
|
||||
: { GHIDRA_INSTALL_DIR: process.env.GHIDRA_INSTALL_DIR }),
|
||||
@@ -229,6 +233,13 @@ const openServer = async (provider) => {
|
||||
structured: result.structuredContent,
|
||||
};
|
||||
};
|
||||
const callError = async (name, arguments_ = {}) => {
|
||||
const result = await client.callTool(
|
||||
{ name, arguments: arguments_ },
|
||||
{ timeout: 3_600_000 },
|
||||
);
|
||||
return requireMcpToolError(result);
|
||||
};
|
||||
const close = async (priorFailure) => {
|
||||
clearInterval(timer);
|
||||
await sample();
|
||||
@@ -270,7 +281,59 @@ const openServer = async (provider) => {
|
||||
// A rejected connection/target still owns its transport and workspace.
|
||||
await close(cause);
|
||||
}
|
||||
return { call, close };
|
||||
return { call, callError, close };
|
||||
};
|
||||
|
||||
/** Verify a forced inventory timeout can be retried without changing jobs. */
|
||||
const verifyProjectTimeoutRecovery = async (path, address) => {
|
||||
const before = createHash("sha256")
|
||||
.update(await readFile(path))
|
||||
.digest("hex");
|
||||
const settings = {
|
||||
REA_KUNA_JOBS: "2",
|
||||
REA_KUNA_TIMEOUT_MS: "60000",
|
||||
REA_KUNA_PROJECT_TIMEOUT_MS: "1",
|
||||
};
|
||||
const timed = await openServer({ path, id: "kuna" }, settings);
|
||||
let timedFailure;
|
||||
try {
|
||||
const failure = await timed.callError("procedure_assembly", {
|
||||
procedure: address,
|
||||
});
|
||||
assert.equal(failure.code, "capability_unavailable");
|
||||
assert.match(failure.message, /kuna functions/u);
|
||||
assert.match(failure.remediation.action, /REA_KUNA_PROJECT_TIMEOUT_MS/u);
|
||||
assert.doesNotMatch(failure.remediation.action, /REA_KUNA_JOBS/u);
|
||||
} catch (cause) {
|
||||
timedFailure = cause;
|
||||
} finally {
|
||||
await timed.close(timedFailure);
|
||||
}
|
||||
const retry = await openServer(
|
||||
{ path, id: "kuna" },
|
||||
{ ...settings, REA_KUNA_PROJECT_TIMEOUT_MS: "60000" },
|
||||
);
|
||||
let retryFailure;
|
||||
try {
|
||||
const assembly = await retry.call("procedure_assembly", {
|
||||
procedure: address,
|
||||
});
|
||||
assert.equal(typeof assembly.value, "string");
|
||||
assert.ok(assembly.value.length > 0);
|
||||
assert.equal(assembly.structured.analysis_profile.parameters.jobs, 2);
|
||||
} catch (cause) {
|
||||
retryFailure = cause;
|
||||
} finally {
|
||||
await retry.close(retryFailure);
|
||||
}
|
||||
assert.equal(
|
||||
createHash("sha256")
|
||||
.update(await readFile(path))
|
||||
.digest("hex"),
|
||||
before,
|
||||
"Kuna timeout recovery changed the target",
|
||||
);
|
||||
return { forced_deadline_ms: 1, retry_deadline_ms: 60000, jobs: 2 };
|
||||
};
|
||||
|
||||
/** Count structuring and diagnostic markers in provider pseudocode. */
|
||||
@@ -535,6 +598,7 @@ const verifyVariant = async ({ flavor, directory, symbols }, variant) => {
|
||||
};
|
||||
|
||||
const results = [];
|
||||
let timeoutRecovery;
|
||||
const failures = [];
|
||||
let inspectionResourcesClean = true;
|
||||
try {
|
||||
@@ -596,6 +660,11 @@ try {
|
||||
}),
|
||||
});
|
||||
|
||||
timeoutRecovery = await verifyProjectTimeoutRecovery(
|
||||
built,
|
||||
fixtureFunctions.get("rea_checksum"),
|
||||
);
|
||||
|
||||
// Caller-built matrix.
|
||||
for (const pair of callerPairs)
|
||||
for (const variant of [pair.file, `${pair.file}.stripped`])
|
||||
@@ -650,6 +719,9 @@ const report = {
|
||||
platform: `${process.platform}/${process.arch}`,
|
||||
verifier_run: lifecycle,
|
||||
results,
|
||||
...(timeoutRecovery === undefined
|
||||
? {}
|
||||
: { timeout_recovery: timeoutRecovery }),
|
||||
limitations: [
|
||||
"Targets are analyzed statically; they are never executed.",
|
||||
"Memory is the sampled peak resident set of the MCP server process tree; CPU is its sampled cumulative time. A short-lived peak between 200 ms samples is not observed.",
|
||||
|
||||
+2
-1
@@ -327,8 +327,9 @@ def main():
|
||||
refresh_contributors(root)
|
||||
elif args.command == "check":
|
||||
sync_pages(root, check=True)
|
||||
for name in ("prepare-website.py", "verify-website.py", "test-website.py"):
|
||||
for name in ("prepare-website.py", "verify-website.py", "test-website.py", "test-website-agents.py"):
|
||||
run_script(root, name)
|
||||
subprocess.run(["node", "--test", "website/worker.test.mjs"], cwd=root, check=True)
|
||||
elif args.command == "links":
|
||||
run_script(root, "audit-website-links.py", ["--report", args.report] if args.report else [])
|
||||
else:
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
"""Build public agent discovery and Markdown from their maintained sources."""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from urllib.parse import urljoin
|
||||
from zipfile import ZIP_DEFLATED, ZipFile, ZipInfo
|
||||
|
||||
from bs4 import BeautifulSoup
|
||||
from markdownify import MarkdownConverter
|
||||
import yaml
|
||||
|
||||
from website import NOTES_FIXTURES, PageMetadata, SITE_ORIGIN, page_url
|
||||
|
||||
|
||||
def write_json(path, value):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(value, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def prepare_discovery(root, site):
|
||||
"""Publish the complete skill, including its relative reference documents."""
|
||||
source = root / ".agents/skills/reverse-engineer-anything"
|
||||
skill = (source / "SKILL.md").read_text(encoding="utf-8")
|
||||
if not skill.startswith("---\n") or "\n---\n" not in skill[4:]:
|
||||
raise ValueError("Public REA skill needs YAML frontmatter")
|
||||
metadata = yaml.safe_load(skill[4:].split("\n---\n", 1)[0])
|
||||
if not isinstance(metadata, dict):
|
||||
raise ValueError("Public REA skill needs YAML metadata")
|
||||
name, description = metadata["name"], metadata["description"]
|
||||
if name != source.name or not isinstance(description, str) or not description.strip():
|
||||
raise ValueError("Invalid public REA skill metadata")
|
||||
discovery = site / ".well-known/agent-skills"
|
||||
discovery.mkdir(parents=True, exist_ok=True)
|
||||
artifact = discovery / f"{name}.zip"
|
||||
with ZipFile(artifact, "w") as archive:
|
||||
for path in [source / "SKILL.md", *sorted(source.glob("references/**/*.md"))]:
|
||||
relative = path.relative_to(source)
|
||||
entry = ZipInfo(f"{name}/{relative.as_posix()}", (1980, 1, 1, 0, 0, 0))
|
||||
entry.create_system = 3
|
||||
entry.external_attr = 0o100644 << 16
|
||||
archive.writestr(entry, path.read_bytes(), compress_type=ZIP_DEFLATED)
|
||||
artifact_url = f"{SITE_ORIGIN}/.well-known/agent-skills/{artifact.name}"
|
||||
write_json(discovery / "index.json", {
|
||||
"$schema": "https://schemas.agentskills.io/discovery/0.2.0/schema.json",
|
||||
"skills": [{
|
||||
"name": name, "type": "archive", "description": description,
|
||||
"url": artifact_url,
|
||||
"digest": "sha256:" + hashlib.sha256(artifact.read_bytes()).hexdigest(),
|
||||
}],
|
||||
})
|
||||
entry = {
|
||||
"identifier": f"urn:air:rea.tools:skill:{name}",
|
||||
"displayName": "REA reverse engineering skill",
|
||||
"description": description,
|
||||
"type": "application/zip",
|
||||
"url": artifact_url,
|
||||
}
|
||||
write_json(site / ".well-known/ai-catalog.json", {
|
||||
"specVersion": "1.0",
|
||||
"host": {"displayName": "REA", "documentationUrl": f"{SITE_ORIGIN}/get-started/"},
|
||||
"entries": [entry],
|
||||
})
|
||||
write_json(site / ".well-known/ard.json", {"entries": [{
|
||||
**entry,
|
||||
"representativeQueries": [
|
||||
"Explain how a feature works in a shipped executable using REA.",
|
||||
"Trace an Electron application's behavior back to its packaged JavaScript.",
|
||||
"Build an AgentEnv bundle from a web application capture.",
|
||||
],
|
||||
}]})
|
||||
print("Prepared agent skill archive, discovery index, AI Catalog and ARD manifest.")
|
||||
|
||||
|
||||
def prepare_markdown(site):
|
||||
"""Convert rendered reading pages, keeping code and resolving resource URLs."""
|
||||
converter = MarkdownConverter(heading_style="ATX", bullets="-", escape_underscores=False)
|
||||
count = 0
|
||||
for path in sorted(site.rglob("*.html")):
|
||||
if path.relative_to(site).as_posix() in NOTES_FIXTURES:
|
||||
continue
|
||||
source = path.read_text(encoding="utf-8")
|
||||
metadata = PageMetadata()
|
||||
metadata.feed(source)
|
||||
# Drafts and English fallbacks keep their HTML indexing policy intact.
|
||||
if metadata.noindex:
|
||||
continue
|
||||
soup = BeautifulSoup(source, "html.parser")
|
||||
main = soup.find("main")
|
||||
if main is None:
|
||||
continue
|
||||
# Preserve fallback notices and closed disclosures; remove interface chrome.
|
||||
for node in main.select('script, style, nav, button, svg, .home-page-nav, [aria-hidden="true"]'):
|
||||
if node.parent is not None:
|
||||
node.decompose()
|
||||
# These spans are separate lines in the shared flow component's CSS.
|
||||
for node in main.select("span.flow-label, span.flow-node, span.flow-note"):
|
||||
node.name = "div"
|
||||
canonical = page_url(site, path)
|
||||
for node in main.select("[href], [src]"):
|
||||
for attribute in ("href", "src"):
|
||||
if node.has_attr(attribute):
|
||||
node[attribute] = urljoin(canonical, node[attribute])
|
||||
title = metadata.titles[0] if metadata.titles else "REA"
|
||||
header = yaml.safe_dump({"title": title, "url": canonical}, allow_unicode=True, sort_keys=False)
|
||||
markdown = converter.convert_soup(main).strip()
|
||||
path.with_suffix(".md").write_text(f"---\n{header}---\n\n{markdown}\n", encoding="utf-8")
|
||||
alternate = page_url(site, path.with_suffix(".md"))
|
||||
links = (
|
||||
f'<link rel="alternate" type="text/markdown" href="{alternate}" />\n'
|
||||
f'<link rel="ard" type="application/json" href="{SITE_ORIGIN}/.well-known/ard.json" />\n'
|
||||
f'<link rel="ai-catalog" type="application/ai-catalog+json" href="{SITE_ORIGIN}/.well-known/ai-catalog.json" />\n'
|
||||
f'<link rel="related" type="application/json" title="Agent skills index" href="{SITE_ORIGIN}/.well-known/agent-skills/index.json" />\n'
|
||||
)
|
||||
path.write_text(source.replace("</head>", links + "</head>", 1), encoding="utf-8")
|
||||
count += 1
|
||||
print(f"Prepared Markdown for {count} reading pages.")
|
||||
@@ -880,6 +880,62 @@ describe("trace_feature requirements", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("firmware operation recovery", () => {
|
||||
it.each([
|
||||
[false, false],
|
||||
[true, false],
|
||||
[false, true],
|
||||
[true, true],
|
||||
])(
|
||||
"keeps inspection=%s and extraction=%s independent",
|
||||
(inspection, extraction) => {
|
||||
const inventory = buildCapabilityInventory(status(), {
|
||||
processCaptureEnabled: false,
|
||||
firmwareInspectionEnabled: inspection,
|
||||
firmwareExtractionEnabled: extraction,
|
||||
});
|
||||
for (const [name, enabled, command, otherCommand] of [
|
||||
[
|
||||
"inspect_firmware_regions",
|
||||
inspection,
|
||||
"REA_BINWALK_COMMAND",
|
||||
"REA_UNBLOB_COMMAND",
|
||||
],
|
||||
[
|
||||
"extract_firmware",
|
||||
extraction,
|
||||
"REA_UNBLOB_COMMAND",
|
||||
"REA_BINWALK_COMMAND",
|
||||
],
|
||||
] as const) {
|
||||
const tool = inventory.find((candidate) => candidate.name === name);
|
||||
expect(tool).toMatchObject({
|
||||
available: enabled,
|
||||
reason: enabled ? "available" : "provider_missing",
|
||||
});
|
||||
if (enabled) {
|
||||
expect(tool?.remediation).toBeNull();
|
||||
} else {
|
||||
expect(tool?.remediation).toContain(command);
|
||||
expect(tool?.remediation).not.toContain(otherCommand);
|
||||
expect(tool?.remediation).toContain("prlimit");
|
||||
expect(tool?.remediation).toContain("procps ps");
|
||||
expect(tool?.remediation?.includes("external extractor")).toBe(
|
||||
name === "extract_firmware",
|
||||
);
|
||||
}
|
||||
}
|
||||
const byName = (left: ToolAvailability, right: ToolAvailability) =>
|
||||
left.name.localeCompare(right.name);
|
||||
expect(
|
||||
expandToolAvailability(compactToolAvailability(inventory)).toSorted(
|
||||
byName,
|
||||
),
|
||||
).toEqual(inventory.toSorted(byName));
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
const HOST_RESTRICTED_TOOLS = [
|
||||
["inspect_packet_capture", "packetCapture", "packetCaptureEnabled"],
|
||||
["inspect_packet_fields", "packetCapture", "packetCaptureEnabled"],
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
import { PCAP_CAPTURE_UNSUPPORTED_HOST_REASON } from "../domain/pcap/pcapAnalysis.js";
|
||||
import { DEFAULT_CALL_RELATIONSHIP_DIRECTION } from "../contracts/enhancedInputs.js";
|
||||
import { FIRMWARE_TOOL_REQUIREMENTS } from "../contracts/firmware/firmwareToolRequirements.js";
|
||||
import {
|
||||
FIRMWARE_OPERATION_ENGINES,
|
||||
FIRMWARE_TOOL_REQUIREMENTS,
|
||||
} from "../contracts/firmware/firmwareToolRequirements.js";
|
||||
import { CALL_RELATIONSHIP_TRAVERSALS } from "./CallRelationshipTraversal.js";
|
||||
import { NATIVE_DISPATCH_METADATA_SOURCES } from "./native/NativeDispatchMetadataInspection.js";
|
||||
import { NATIVE_UI_ACTION_OPERATIONS } from "./native/NativeUiActionTrace.js";
|
||||
@@ -595,15 +598,19 @@ const workflowAvailabilityFor = ({
|
||||
"On Linux x64, set absolute REA_PWNTOOLS_PYTHON to caller-supplied Python with pwntools 4.15.0. No active binary target is required.",
|
||||
);
|
||||
if (kind === "firmware-provider") {
|
||||
const { binwalk, unblob } = FIRMWARE_TOOL_REQUIREMENTS;
|
||||
const inspection = name === "inspect_firmware_regions";
|
||||
const operation = inspection
|
||||
? "inspect_firmware_regions"
|
||||
: "extract_firmware";
|
||||
const requirement =
|
||||
FIRMWARE_TOOL_REQUIREMENTS[FIRMWARE_OPERATION_ENGINES[operation]];
|
||||
return hostRestrictedAvailability(
|
||||
policy,
|
||||
inspection ? "firmwareInspection" : "firmwareExtraction",
|
||||
inspection
|
||||
? policy.firmwareInspectionEnabled
|
||||
: policy.firmwareExtractionEnabled,
|
||||
`On Linux, provide an absolute ${binwalk.commandEnvironment} (${binwalk.name} ${binwalk.line}, verified with ${binwalk.version}) or ${unblob.commandEnvironment} (${unblob.name} ${unblob.line}, verified with ${unblob.version}) path and util-linux prlimit. Extraction also requires the selected format’s external extractor.`,
|
||||
`On Linux, provide an absolute ${requirement.commandEnvironment} (${requirement.requirement}) path, util-linux prlimit and procps ps.${inspection ? "" : " Extraction also requires the selected format’s external extractor."}`,
|
||||
);
|
||||
}
|
||||
if (kind === "android-provider")
|
||||
|
||||
@@ -18,7 +18,14 @@ import { silentLogger } from "../logger.js";
|
||||
import type { Logger } from "pino";
|
||||
import { withEvidenceCallerInputPath } from "../domain/evidence.js";
|
||||
import { createImmutableEvidenceBundle } from "../domain/evidenceBundle.js";
|
||||
import { retainAnalysisResultSources } from "./investigation/recordEvidenceSources.js";
|
||||
import {
|
||||
retainAnalysisResultSources,
|
||||
type AnalysisResultWithSources,
|
||||
} from "./investigation/recordEvidenceSources.js";
|
||||
import {
|
||||
createInvestigationEvidenceClosureChecker,
|
||||
resolveInvestigationEvidenceClosure,
|
||||
} from "./investigation/InvestigationEvidence.js";
|
||||
import type { Evidence } from "../domain/evidence.js";
|
||||
import type { NativeToolName } from "../contracts/native/nativeToolContracts.js";
|
||||
import type { EnhancedToolName } from "../contracts/enhancedInputs.js";
|
||||
@@ -139,13 +146,17 @@ export const runDirectAnalysis = async (
|
||||
readonly architecture?: BinaryArchitecture;
|
||||
readonly entryPoint?: string;
|
||||
readonly espFlash?: EspFlashSelection;
|
||||
readonly includeEvidenceSources?: boolean;
|
||||
} = {},
|
||||
): Promise<Result<Evidence, JsonValue>> =>
|
||||
): Promise<AnalysisResultWithSources<Evidence, JsonValue>> =>
|
||||
withProcessCancellation(options.signal, (signal) =>
|
||||
runAnalysis(dependencies, path, tool, arguments_, {
|
||||
logger: options.logger ?? silentLogger,
|
||||
snapshotPath: options.snapshotPath,
|
||||
signal,
|
||||
...(options.includeEvidenceSources
|
||||
? { includeEvidenceSources: true }
|
||||
: {}),
|
||||
...targetSelection(options),
|
||||
...(options.providerId === undefined
|
||||
? {}
|
||||
@@ -365,13 +376,14 @@ const runAnalysis = async (
|
||||
readonly signal: AbortSignal;
|
||||
readonly evidenceParameters?: Readonly<Record<string, JsonValue>>;
|
||||
readonly createSession?: () => BinarySession | Promise<BinarySession>;
|
||||
readonly includeEvidenceSources?: boolean;
|
||||
readonly providerId?: AnalysisProviderSelector;
|
||||
readonly formatHint?: ExecutableFormatHint;
|
||||
readonly architecture?: BinaryArchitecture;
|
||||
readonly entryPoint?: string;
|
||||
readonly espFlash?: EspFlashSelection;
|
||||
},
|
||||
): Promise<Result<Evidence, JsonValue>> => {
|
||||
): Promise<AnalysisResultWithSources<Evidence, JsonValue>> => {
|
||||
const { logger, signal, snapshotPath } = options;
|
||||
const evidenceParameters = options.evidenceParameters ?? arguments_;
|
||||
let session: BinarySession;
|
||||
@@ -387,6 +399,29 @@ const runAnalysis = async (
|
||||
)(config.value, logger);
|
||||
}
|
||||
let publishedSnapshot: PublishedEvidenceFile | undefined;
|
||||
let sourceEvidence: readonly Evidence[] | undefined;
|
||||
const captureSources = (
|
||||
evidence: Evidence,
|
||||
supplied: readonly Evidence[] = [],
|
||||
): Result<null, AnalysisError> => {
|
||||
if (!options.includeEvidenceSources) return ok(null);
|
||||
const records = new Map(
|
||||
supplied.map((record) => [record.evidence_id, record]),
|
||||
);
|
||||
const captured = new Map<string, Evidence>();
|
||||
const closure = resolveInvestigationEvidenceClosure(
|
||||
{
|
||||
evidenceForAnalysis: (id) => {
|
||||
const record = records.get(id) ?? session.evidenceForAnalysis(id);
|
||||
if (record !== undefined) captured.set(id, record);
|
||||
return record;
|
||||
},
|
||||
},
|
||||
evidence.evidence_links,
|
||||
);
|
||||
sourceEvidence = [...captured.values()];
|
||||
return closure.ok ? ok(null) : closure;
|
||||
};
|
||||
const execute = async (): Promise<Result<Evidence, AnalysisError>> => {
|
||||
let isDirectory = false;
|
||||
if (BUNDLE_RESOURCE_OPERATIONS.has(tool)) {
|
||||
@@ -458,8 +493,29 @@ const runAnalysis = async (
|
||||
: routeProviderIdentity(route, tool),
|
||||
evidenceProfile,
|
||||
});
|
||||
if (cached !== undefined)
|
||||
return ok(withEvidenceCallerInputPath(cached, path));
|
||||
if (cached !== undefined) {
|
||||
if (cached.evidence_links.length === 0)
|
||||
return ok(withEvidenceCallerInputPath(cached, path));
|
||||
const records = new Map(
|
||||
snapshot.evidence_bundle.records.map((record) => [
|
||||
record.evidence_id,
|
||||
record,
|
||||
]),
|
||||
);
|
||||
const checkClosure = createInvestigationEvidenceClosureChecker({
|
||||
evidenceForAnalysis: (id) => records.get(id),
|
||||
});
|
||||
if (checkClosure(cached.evidence_links)) {
|
||||
const evidence = withEvidenceCallerInputPath(cached, path);
|
||||
const captured = captureSources(
|
||||
evidence,
|
||||
snapshot.evidence_bundle.records,
|
||||
);
|
||||
if (!captured.ok)
|
||||
return err(captured.error.retainPartialObservation(evidence));
|
||||
return ok(evidence);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
const openOptions = {
|
||||
@@ -496,6 +552,9 @@ const runAnalysis = async (
|
||||
);
|
||||
if (!recorded.ok)
|
||||
return err(recorded.error.retainPartialObservation(evidence));
|
||||
const captured = captureSources(evidence);
|
||||
if (!captured.ok)
|
||||
return err(captured.error.retainPartialObservation(evidence));
|
||||
if (isWorkflowEvidenceTool(tool)) {
|
||||
const unknowns = recordWorkflowUnknowns({
|
||||
name: tool,
|
||||
@@ -575,11 +634,15 @@ const runAnalysis = async (
|
||||
outcome.cleanupError.retainDiagnostics({
|
||||
analysis_snapshot: { ...publishedSnapshot },
|
||||
});
|
||||
return err(
|
||||
directAnalysisCleanupFailure(result, outcome.cleanupError, path),
|
||||
);
|
||||
return {
|
||||
...err(directAnalysisCleanupFailure(result, outcome.cleanupError, path)),
|
||||
...(sourceEvidence === undefined ? {} : { sourceEvidence }),
|
||||
};
|
||||
}
|
||||
return result.ok ? ok(result.value) : err(cliError(result.error, path));
|
||||
return {
|
||||
...(result.ok ? ok(result.value) : err(cliError(result.error, path))),
|
||||
...(sourceEvidence === undefined ? {} : { sourceEvidence }),
|
||||
};
|
||||
};
|
||||
|
||||
const executeAnalysisTool = async (input: {
|
||||
|
||||
@@ -3,7 +3,7 @@ import type { z } from "zod";
|
||||
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import type { TraceContainingProcedure } from "../domain/native/nativeObservation.js";
|
||||
import type { LiteralFeatureReference } from "../domain/native/literalFeatureTrace.js";
|
||||
import { err, type Result } from "../domain/result.js";
|
||||
import {
|
||||
enhancedInputSchemas,
|
||||
@@ -27,11 +27,7 @@ export const failedWithPartialResult = (
|
||||
error: AnalysisError,
|
||||
partialResult: JsonValue,
|
||||
): PartialEnhancedFailure => ({ ok: false, error, partialResult });
|
||||
export type TraceReference = {
|
||||
readonly target_address: string;
|
||||
readonly source_address: string;
|
||||
readonly containing_procedure: TraceContainingProcedure;
|
||||
};
|
||||
export type TraceReference = LiteralFeatureReference;
|
||||
|
||||
/** One enhanced call whose input was parsed at its owning adapter boundary. */
|
||||
export type ValidatedEnhancedCall = {
|
||||
|
||||
@@ -125,6 +125,8 @@ export interface ReferenceSourceImportOptions {
|
||||
export const DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS = [
|
||||
".git/",
|
||||
".git/hooks/",
|
||||
// Match pointer files after directory rules to preserve their provenance.
|
||||
".git",
|
||||
"node_modules/",
|
||||
"dist/",
|
||||
"build/",
|
||||
|
||||
@@ -301,10 +301,13 @@ describe("setup client selection", () => {
|
||||
configPath: "/codex.toml",
|
||||
format: "toml" as const,
|
||||
};
|
||||
host.clients = [codex];
|
||||
host.clients = [
|
||||
codex,
|
||||
{ name: "grok_bot", configPath: "/grokbot", format: "unsupported" },
|
||||
];
|
||||
host.availableClients = [
|
||||
{ name: "cursor", configPath: "/cursor.json" },
|
||||
codex,
|
||||
...host.clients,
|
||||
];
|
||||
|
||||
const result = await runSetup(
|
||||
@@ -331,6 +334,31 @@ describe("setup client selection", () => {
|
||||
expect(host.skillInstalls).toBe(0);
|
||||
});
|
||||
|
||||
it.each([undefined, false])(
|
||||
"omits manual clients from automatic readiness while preserving detection (skill: %s)",
|
||||
async (installSkill) => {
|
||||
const host = new FakeSetupHost();
|
||||
host.clients = [
|
||||
{ name: "grok_bot", configPath: "/grokbot", format: "unsupported" },
|
||||
];
|
||||
const result = await runSetup(
|
||||
{
|
||||
...options(true),
|
||||
allDetectedClients: true,
|
||||
...(installSkill === undefined ? {} : { installSkill }),
|
||||
},
|
||||
host,
|
||||
);
|
||||
expect(result.status).toBe("ready");
|
||||
expect(result.doctor.scope.clients).toEqual([]);
|
||||
expect(result.clientStates).toMatchObject([
|
||||
{ client: { name: "grok_bot" }, detected: true },
|
||||
]);
|
||||
expect(result.clients).toEqual({});
|
||||
expect(host.configurations).toBe(0);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([true, false])(
|
||||
"handles repeated clients once in first-selected order (dry run: %s)",
|
||||
async (dryRun) => {
|
||||
|
||||
@@ -139,7 +139,9 @@ export const discoverSetupState = async (input: {
|
||||
installHopper,
|
||||
hopperInstall,
|
||||
clients,
|
||||
detectedClientIds: [...detectedIds],
|
||||
detectedClientIds: detectedClients
|
||||
.filter(({ format }) => format !== "unsupported")
|
||||
.map(({ name }) => name),
|
||||
defaultClientIds,
|
||||
clientStates,
|
||||
...(linuxDistribution?.packageFamily === undefined
|
||||
|
||||
@@ -121,53 +121,56 @@ const graph = (): NativeCallGraph =>
|
||||
limitations: ["Graph caveat", "Graph caveat"],
|
||||
});
|
||||
|
||||
it("returns useful caller/callee facts with exact source context identity and partial coverage", async () => {
|
||||
const source = parent("procedure_callees", [
|
||||
"ram:0x0010",
|
||||
"ram:0x0010",
|
||||
"ram:0x0020",
|
||||
]);
|
||||
const before = JSON.stringify(source);
|
||||
const summary = await project(source, { kind: "summary" });
|
||||
expect(summary.value).toMatchObject({
|
||||
kind: "summary",
|
||||
parent_evidence_id: source.evidence_id,
|
||||
source_provider: source.provider,
|
||||
source_parameters: source.parameters,
|
||||
source_analysis_profile_digest: source.analysis_profile?.digest,
|
||||
source_context: {
|
||||
evidence_limitation_count: 1,
|
||||
result_limitation_count: null,
|
||||
},
|
||||
summary: { observations: 3 },
|
||||
items: {
|
||||
type: "addresses",
|
||||
rows: [
|
||||
{ source_index: 0, address: "ram:0x0010" },
|
||||
{ source_index: 1, address: "ram:0x0010" },
|
||||
{ source_index: 2, address: "ram:0x0020" },
|
||||
],
|
||||
},
|
||||
});
|
||||
expect(summary.evidence.relationship_coverage).toEqual(
|
||||
source.relationship_coverage,
|
||||
);
|
||||
const page = await project(source, {
|
||||
kind: "native-observation",
|
||||
offset: 1,
|
||||
limit: 1,
|
||||
});
|
||||
expect(page.value).toMatchObject({
|
||||
items: { rows: [{ source_index: 1, address: "ram:0x0010" }] },
|
||||
coverage: { total: 3, next_offset: 2, exhausted: false },
|
||||
});
|
||||
expect(page.value).toHaveProperty(
|
||||
"source_context.digest",
|
||||
contextDigest(summary.value),
|
||||
);
|
||||
expect(JSON.stringify(summary.value)).not.toContain('"analysis_profile":');
|
||||
expect(JSON.stringify(source)).toBe(before);
|
||||
});
|
||||
it.each(["procedure_callees", "xrefs"])(
|
||||
"pages %s addresses with exact source context and duplicate identity",
|
||||
async (operation) => {
|
||||
const source = parent(operation, [
|
||||
"ram:0x0010",
|
||||
"ram:0x0010",
|
||||
"ram:0x0020",
|
||||
]);
|
||||
const before = JSON.stringify(source);
|
||||
const summary = await project(source, { kind: "summary" });
|
||||
expect(summary.value).toMatchObject({
|
||||
kind: "summary",
|
||||
parent_evidence_id: source.evidence_id,
|
||||
source_provider: source.provider,
|
||||
source_parameters: source.parameters,
|
||||
source_analysis_profile_digest: source.analysis_profile?.digest,
|
||||
source_context: {
|
||||
evidence_limitation_count: 1,
|
||||
result_limitation_count: null,
|
||||
},
|
||||
summary: { observations: 3 },
|
||||
items: {
|
||||
type: "addresses",
|
||||
rows: [
|
||||
{ source_index: 0, address: "ram:0x0010" },
|
||||
{ source_index: 1, address: "ram:0x0010" },
|
||||
{ source_index: 2, address: "ram:0x0020" },
|
||||
],
|
||||
},
|
||||
});
|
||||
expect(summary.evidence.relationship_coverage).toEqual(
|
||||
source.relationship_coverage,
|
||||
);
|
||||
const page = await project(source, {
|
||||
kind: "native-observation",
|
||||
offset: 1,
|
||||
limit: 1,
|
||||
});
|
||||
expect(page.value).toMatchObject({
|
||||
items: { rows: [{ source_index: 1, address: "ram:0x0010" }] },
|
||||
coverage: { total: 3, next_offset: 2, exhausted: false },
|
||||
});
|
||||
expect(page.value).toHaveProperty(
|
||||
"source_context.digest",
|
||||
contextDigest(summary.value),
|
||||
);
|
||||
expect(JSON.stringify(summary.value)).not.toContain('"analysis_profile":');
|
||||
expect(JSON.stringify(source)).toBe(before);
|
||||
},
|
||||
);
|
||||
|
||||
it("pages complete context fields and changes its digest when source profile or caveats change", async () => {
|
||||
const source = parent("procedure_callers", []);
|
||||
|
||||
@@ -7,7 +7,8 @@ import {
|
||||
ANALYSIS_SNAPSHOT_PROVIDER,
|
||||
ANALYSIS_SNAPSHOT_TARGET,
|
||||
} from "../../domain/analysisSnapshot.fixture.js";
|
||||
import { createEvidence } from "../../domain/evidence.js";
|
||||
import { createEvidence, immutableEvidence } from "../../domain/evidence.js";
|
||||
import { createAnalysisExecutionEvidence } from "../../domain/analysisObservation.js";
|
||||
import {
|
||||
createEvidenceBundle,
|
||||
evidenceBundleForTarget,
|
||||
@@ -1208,6 +1209,154 @@ it("declines source-bearing snapshot entries while replaying omitted and empty s
|
||||
expect(cache.entries()).toHaveLength(2);
|
||||
});
|
||||
|
||||
it("replays source-bearing entries only when their complete retained closure exists", () => {
|
||||
const target = ANALYSIS_SNAPSHOT_TARGET;
|
||||
const profile = ANALYSIS_SNAPSHOT_PROFILE;
|
||||
const leaf = immutableEvidence(
|
||||
createEvidence(target, ANALYSIS_SNAPSHOT_PROVIDER, {
|
||||
operation: "leaf",
|
||||
parameters: {},
|
||||
result: "leaf observation",
|
||||
}),
|
||||
);
|
||||
const source = immutableEvidence(
|
||||
createEvidence(target, ANALYSIS_SNAPSHOT_PROVIDER, {
|
||||
operation: "source",
|
||||
parameters: {},
|
||||
result: "source observation",
|
||||
evidenceLinks: [leaf.evidence_id],
|
||||
}),
|
||||
);
|
||||
const parameters = { address: "0x1000", document: "fixture" };
|
||||
const execution = createAnalysisExecution(
|
||||
"parent observation",
|
||||
ANALYSIS_SNAPSHOT_PROVIDER,
|
||||
{
|
||||
subject: target,
|
||||
analysisProfile: profile,
|
||||
sourceEvidence: [source],
|
||||
},
|
||||
);
|
||||
const parent = immutableEvidence(
|
||||
createAnalysisExecutionEvidence(
|
||||
"procedure_info",
|
||||
parameters,
|
||||
execution,
|
||||
"observed",
|
||||
),
|
||||
);
|
||||
const complete = new Map(
|
||||
[parent, source, leaf].map((record) => [record.evidence_id, record]),
|
||||
);
|
||||
const input = {
|
||||
target,
|
||||
profile,
|
||||
operation: "procedure_info" as const,
|
||||
parameters,
|
||||
execution: { ...execution, evidenceId: parent.evidence_id },
|
||||
};
|
||||
const missingLeaf = new AnalysisSnapshotCache();
|
||||
missingLeaf.record(input, (id) =>
|
||||
id === leaf.evidence_id ? undefined : complete.get(id),
|
||||
);
|
||||
expect(
|
||||
missingLeaf.lookup(target, profile, "procedure_info", parameters),
|
||||
).toBeUndefined();
|
||||
const cache = new AnalysisSnapshotCache();
|
||||
cache.record(input, (id) => complete.get(id));
|
||||
expect(
|
||||
cache
|
||||
.lookup(target, profile, "procedure_info", parameters)
|
||||
?.sourceEvidence?.map((record) => record.evidence_id),
|
||||
).toEqual([source.evidence_id]);
|
||||
if (source.subject === null) throw new Error("Expected source subject");
|
||||
const ledger = new EvidenceLedger();
|
||||
for (const record of [parent, source, leaf])
|
||||
expect(ledger.record(record).ok).toBe(true);
|
||||
const movedSource = {
|
||||
...source,
|
||||
subject: {
|
||||
...source.subject,
|
||||
local_path: "/relocated/source",
|
||||
name: "source",
|
||||
},
|
||||
};
|
||||
const replacement = ledger.import(createEvidenceBundle([movedSource]));
|
||||
if (!replacement.ok) throw replacement.error;
|
||||
cache.refreshEvidenceMetadata(replacement.value.replacements);
|
||||
expect(
|
||||
cache.lookup(target, profile, "procedure_info", parameters)
|
||||
?.sourceEvidence?.[0]?.subject?.local_path,
|
||||
).toBe("/relocated/source");
|
||||
const exported = cache.export(target, profile, ledger.export());
|
||||
if (!exported.ok) throw exported.error;
|
||||
const imported = new AnalysisSnapshotCache();
|
||||
imported.stage(exported.value);
|
||||
expect(
|
||||
imported
|
||||
.lookup(target, profile, "procedure_info", parameters)
|
||||
?.sourceEvidence?.map((record) => record.evidence_id),
|
||||
).toEqual([source.evidence_id]);
|
||||
expect(
|
||||
imported.lookup(target, profile, "procedure_info", parameters)
|
||||
?.sourceEvidence?.[0]?.subject?.local_path,
|
||||
).toBe("/relocated/source");
|
||||
const pruned = parseAnalysisSnapshot({
|
||||
...exported.value,
|
||||
evidence_bundle: createEvidenceBundle([parent, movedSource]),
|
||||
});
|
||||
const incomplete = new AnalysisSnapshotCache();
|
||||
incomplete.stage(pruned);
|
||||
const partial = new Map(
|
||||
[parent, movedSource].map((record) => [record.evidence_id, record]),
|
||||
);
|
||||
expect(
|
||||
incomplete.lookup(target, profile, "procedure_info", parameters, (id) =>
|
||||
partial.get(id),
|
||||
),
|
||||
).toBeUndefined();
|
||||
partial.set(leaf.evidence_id, leaf);
|
||||
expect(
|
||||
incomplete
|
||||
.lookup(target, profile, "procedure_info", parameters, (id) =>
|
||||
partial.get(id),
|
||||
)
|
||||
?.sourceEvidence?.map((record) => record.evidence_id),
|
||||
).toEqual([source.evidence_id]);
|
||||
|
||||
const otherSource = immutableEvidence(
|
||||
createEvidence(target, ANALYSIS_SNAPSHOT_PROVIDER, {
|
||||
operation: "other_source",
|
||||
parameters: {},
|
||||
result: "different source",
|
||||
}),
|
||||
);
|
||||
const otherParent = immutableEvidence(
|
||||
createAnalysisExecutionEvidence(
|
||||
"procedure_info",
|
||||
parameters,
|
||||
{ ...execution, sourceEvidence: [otherSource] },
|
||||
"observed",
|
||||
),
|
||||
);
|
||||
const records = [parent, movedSource, leaf, otherParent, otherSource];
|
||||
const ambiguous = new AnalysisSnapshotCache();
|
||||
ambiguous.stage(
|
||||
parseAnalysisSnapshot({
|
||||
...exported.value,
|
||||
evidence_bundle: createEvidenceBundle(records),
|
||||
}),
|
||||
);
|
||||
const allRecords = new Map(
|
||||
records.map((record) => [record.evidence_id, record]),
|
||||
);
|
||||
expect(
|
||||
ambiguous.lookup(target, profile, "procedure_info", parameters, (id) =>
|
||||
allRecords.get(id),
|
||||
),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
describe("executions that report truncated references", () => {
|
||||
const record = (referencesTruncated: boolean | undefined) => {
|
||||
const cache = new AnalysisSnapshotCache();
|
||||
|
||||
@@ -45,6 +45,7 @@ import {
|
||||
snapshotReplayBinding,
|
||||
type SnapshotReplayBinding,
|
||||
} from "./AnalysisSnapshotReplayBinding.js";
|
||||
import { createInvestigationEvidenceClosureChecker } from "../investigation/InvestigationEvidence.js";
|
||||
|
||||
const RELATIONSHIP_OPERATIONS: ReadonlySet<AnalysisOperation> = new Set([
|
||||
"procedure_callers",
|
||||
@@ -147,9 +148,12 @@ export const SNAPSHOT_REPLAY_LIMITATION =
|
||||
/** Bounded in-memory cache for one immutable binary identity. */
|
||||
export class AnalysisSnapshotCache {
|
||||
readonly #entries = new Map<string, AnalysisSnapshotEntry>();
|
||||
// Null means imported provenance cannot be faithfully replayed. It is distinct
|
||||
// from an entry recorded before an Evidence ID was assigned.
|
||||
readonly #replayBindings = new Map<string, SnapshotReplayBinding | null>();
|
||||
// Null is ambiguous provenance; a pending ID is an exact parent whose source
|
||||
// closure was missing and can be checked again against the owning ledger.
|
||||
readonly #replayBindings = new Map<
|
||||
string,
|
||||
SnapshotReplayBinding | { readonly pendingEvidenceId: string } | null
|
||||
>();
|
||||
readonly #workflowEntries = new Map<string, AnalysisSnapshotWorkflowEntry>();
|
||||
#target: AnalysisSnapshotTarget | undefined;
|
||||
#binding: AnalysisSnapshotBinding | undefined;
|
||||
@@ -218,6 +222,10 @@ export class AnalysisSnapshotCache {
|
||||
record,
|
||||
]),
|
||||
);
|
||||
const lookup = (id: string) => records.get(id);
|
||||
const checkClosure = createInvestigationEvidenceClosureChecker({
|
||||
evidenceForAnalysis: lookup,
|
||||
});
|
||||
const ambiguous = new Set<string>();
|
||||
for (const evidence of snapshot.evidence_bundle.records) {
|
||||
const queryId = analysisQueryId(
|
||||
@@ -242,11 +250,15 @@ export class AnalysisSnapshotCache {
|
||||
else evidenceByQuery.set(queryId, evidence);
|
||||
}
|
||||
for (const [queryId, evidence] of evidenceByQuery) {
|
||||
if (ambiguous.has(queryId)) {
|
||||
this.#replayBindings.set(queryId, null);
|
||||
continue;
|
||||
}
|
||||
this.#replayBindings.set(
|
||||
queryId,
|
||||
ambiguous.has(queryId)
|
||||
? null
|
||||
: snapshotReplayBinding(evidence, records),
|
||||
snapshotReplayBinding(evidence, lookup, checkClosure) ?? {
|
||||
pendingEvidenceId: evidence.evidence_id,
|
||||
},
|
||||
);
|
||||
}
|
||||
for (const entry of snapshot.workflow_entries) {
|
||||
@@ -272,7 +284,12 @@ export class AnalysisSnapshotCache {
|
||||
replacements.map(({ current }) => [current.evidence_id, current]),
|
||||
);
|
||||
for (const [queryId, binding] of this.#replayBindings) {
|
||||
if (binding === null || binding.sourceEvidence === undefined) continue;
|
||||
if (
|
||||
binding === null ||
|
||||
"pendingEvidenceId" in binding ||
|
||||
binding.sourceEvidence === undefined
|
||||
)
|
||||
continue;
|
||||
const sources = binding.sourceEvidence;
|
||||
if (!sources.some((source) => currentRecords.has(source.evidence_id)))
|
||||
continue;
|
||||
@@ -475,6 +492,7 @@ export class AnalysisSnapshotCache {
|
||||
profile: AnalysisProfileCommitment,
|
||||
operation: AnalysisOperation,
|
||||
parameters: Readonly<Record<string, JsonValue>>,
|
||||
lookup?: (id: string) => Evidence | undefined,
|
||||
): AnalysisExecution | undefined {
|
||||
const queryId = analysisQueryId(
|
||||
snapshotTarget(target),
|
||||
@@ -484,10 +502,18 @@ export class AnalysisSnapshotCache {
|
||||
);
|
||||
const cached = this.#entries.get(queryId);
|
||||
if (cached === undefined) return undefined;
|
||||
const binding = this.#replayBindings.get(queryId);
|
||||
let binding = this.#replayBindings.get(queryId);
|
||||
// Do not turn a refused provenance selection into a fresh, unlinked record.
|
||||
// The provider must produce a new observation before this query can replay.
|
||||
if (binding === null) return undefined;
|
||||
if (binding !== undefined && "pendingEvidenceId" in binding) {
|
||||
const parent = lookup?.(binding.pendingEvidenceId);
|
||||
if (parent === undefined || lookup === undefined) return undefined;
|
||||
const restored = snapshotReplayBinding(parent, lookup);
|
||||
if (restored === null) return undefined;
|
||||
this.#replayBindings.set(queryId, restored);
|
||||
binding = restored;
|
||||
}
|
||||
const cachedSubject = cached.execution.subject;
|
||||
const subject =
|
||||
cachedSubject == null
|
||||
@@ -535,21 +561,30 @@ export class AnalysisSnapshotCache {
|
||||
}
|
||||
|
||||
/** Record one successful immutable call unless the cache is full. */
|
||||
record(input: {
|
||||
readonly target: BinaryTarget;
|
||||
readonly profile: AnalysisProfileCommitment;
|
||||
readonly operation: AnalysisOperation;
|
||||
readonly parameters: Readonly<Record<string, JsonValue>>;
|
||||
readonly execution: AnalysisExecution;
|
||||
}): void {
|
||||
record(
|
||||
input: {
|
||||
readonly target: BinaryTarget;
|
||||
readonly profile: AnalysisProfileCommitment;
|
||||
readonly operation: AnalysisOperation;
|
||||
readonly parameters: Readonly<Record<string, JsonValue>>;
|
||||
readonly execution: AnalysisExecution;
|
||||
},
|
||||
lookup?: (id: string) => Evidence | undefined,
|
||||
): void {
|
||||
const { target, profile, operation, parameters, execution } = input;
|
||||
this.select(target, profile);
|
||||
// The snapshot format does not carry reference truncation; a replay would
|
||||
// present a capped read as complete, so it is not recorded.
|
||||
if (execution.referencesTruncated === true) return;
|
||||
// Provider snapshot entries do not carry execution source records. Replay
|
||||
// must not discard the source closure that supplies the parent's Evidence links.
|
||||
if ((execution.sourceEvidence?.length ?? 0) > 0) return;
|
||||
let sourceBinding: SnapshotReplayBinding | undefined;
|
||||
if ((execution.sourceEvidence?.length ?? 0) > 0) {
|
||||
if (execution.evidenceId === undefined || lookup === undefined) return;
|
||||
const parent = lookup(execution.evidenceId);
|
||||
if (parent === undefined) return;
|
||||
const closed = snapshotReplayBinding(parent, lookup);
|
||||
if (closed === null) return;
|
||||
sourceBinding = closed;
|
||||
}
|
||||
const queryId = analysisQueryId(
|
||||
snapshotTarget(target),
|
||||
snapshotBinding(profile),
|
||||
@@ -581,9 +616,10 @@ export class AnalysisSnapshotCache {
|
||||
if (execution.evidenceId === undefined)
|
||||
this.#replayBindings.delete(entry.query_id);
|
||||
else
|
||||
this.#replayBindings.set(entry.query_id, {
|
||||
evidenceId: execution.evidenceId,
|
||||
});
|
||||
this.#replayBindings.set(
|
||||
entry.query_id,
|
||||
sourceBinding ?? { evidenceId: execution.evidenceId },
|
||||
);
|
||||
}
|
||||
|
||||
/** Forget all target-bound entries. */
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { Evidence } from "../../domain/evidence.js";
|
||||
import { createInvestigationEvidenceClosureChecker } from "../investigation/InvestigationEvidence.js";
|
||||
|
||||
/** Authenticated observation identity and its immediate source records. */
|
||||
export interface SnapshotReplayBinding {
|
||||
@@ -9,13 +10,17 @@ export interface SnapshotReplayBinding {
|
||||
/** Preserve direct links using the snapshot's authenticated immutable records. */
|
||||
export const snapshotReplayBinding = (
|
||||
evidence: Evidence,
|
||||
records: ReadonlyMap<string, Evidence>,
|
||||
lookup: (id: string) => Evidence | undefined,
|
||||
checkClosure = createInvestigationEvidenceClosureChecker({
|
||||
evidenceForAnalysis: lookup,
|
||||
}),
|
||||
): SnapshotReplayBinding | null => {
|
||||
if (evidence.evidence_links.length === 0)
|
||||
return { evidenceId: evidence.evidence_id };
|
||||
if (!checkClosure(evidence.evidence_links)) return null;
|
||||
const sources: Evidence[] = [];
|
||||
for (const id of evidence.evidence_links) {
|
||||
const source = records.get(id);
|
||||
const source = lookup(id);
|
||||
// An external link without its source cannot be represented by sourceEvidence.
|
||||
if (source === undefined) return null;
|
||||
sources.push(source);
|
||||
|
||||
@@ -323,14 +323,16 @@ export abstract class BinarySessionRecords {
|
||||
>,
|
||||
): AnalysisExecution | undefined {
|
||||
if (this.#snapshotInvalidated) return undefined;
|
||||
return this.#snapshot.lookup(target, profile, operation, parameters);
|
||||
return this.#snapshot.lookup(target, profile, operation, parameters, (id) =>
|
||||
this.evidenceForAnalysis(id),
|
||||
);
|
||||
}
|
||||
|
||||
protected recordSnapshot(
|
||||
input: Parameters<AnalysisSnapshotCache["record"]>[0],
|
||||
): void {
|
||||
if (this.#snapshotInvalidated) return;
|
||||
this.#snapshot.record(input);
|
||||
this.#snapshot.record(input, (id) => this.evidenceForAnalysis(id));
|
||||
this.#emitSnapshotChanged();
|
||||
}
|
||||
|
||||
|
||||
@@ -58,6 +58,7 @@ it("records the decoded layout once in its Evidence", async () => {
|
||||
|
||||
it.each([
|
||||
[{ machine: "EM_X86_64", bits: 64, byte_order: "little" }, "x86_64"],
|
||||
[{ machine: "EM_AARCH64", bits: 64, byte_order: "little" }, "arm64"],
|
||||
[{ machine: "EM_386", bits: 32, byte_order: "little" }, "x86"],
|
||||
[{ machine: "EM_ARM", bits: 32, byte_order: "little" }, "arm"],
|
||||
[{ machine: "EM_MIPS", bits: 32, byte_order: "big" }, "mips"],
|
||||
|
||||
@@ -14,6 +14,7 @@ import { parseAnalysisInput } from "../AnalysisInput.js";
|
||||
const OPERATION = "inspect_binary_layout";
|
||||
const EVIDENCE_ARCHITECTURES = {
|
||||
EM_X86_64: "x86_64",
|
||||
EM_AARCH64: "arm64",
|
||||
EM_386: "x86",
|
||||
EM_ARM: "arm",
|
||||
EM_MIPS: "mips",
|
||||
|
||||
@@ -64,6 +64,56 @@ const compare = (...probes: (readonly TimedCaptureRow[])[]) => {
|
||||
return report.value;
|
||||
};
|
||||
|
||||
describe("step difference coverage", () => {
|
||||
it("keeps categories from differences beyond the returned prefix", () => {
|
||||
const fields = Object.fromEntries(
|
||||
Array.from({ length: 20 }, (_, index) => [`field${String(index)}`, 0]),
|
||||
);
|
||||
const capture = timedCaptureEvidence([["GET", "/api/config", fields]]);
|
||||
const served = environmentStateServeFixture(capture);
|
||||
const report = compareEnvironment({
|
||||
source: environmentServeSource(served),
|
||||
captureValue: capture,
|
||||
probes: [
|
||||
{
|
||||
label: "changed",
|
||||
value: timedCaptureEvidence([
|
||||
[
|
||||
"GET",
|
||||
"/api/config",
|
||||
{
|
||||
...Object.fromEntries(
|
||||
Object.keys(fields).map((key) => [key, 1]),
|
||||
),
|
||||
z_missing: true,
|
||||
},
|
||||
],
|
||||
]),
|
||||
},
|
||||
],
|
||||
operation: "environment-compare",
|
||||
});
|
||||
if (!report.ok) throw report.error;
|
||||
const step = report.value.probes[0]!.results[0]!;
|
||||
expect(step).toMatchObject({
|
||||
equal: false,
|
||||
equal_ignoring_order: false,
|
||||
categories: ["value", "missing-field"],
|
||||
differences_total: 21,
|
||||
differences_omitted: 1,
|
||||
differences_truncated: true,
|
||||
});
|
||||
expect(step.differences).toHaveLength(20);
|
||||
expect(step.differences.every(({ category }) => category === "value")).toBe(
|
||||
true,
|
||||
);
|
||||
expect(report.value.summary.categories).toMatchObject({
|
||||
value: 1,
|
||||
"missing-field": 1,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("matchProbeRequest", () => {
|
||||
it("compares a repeated-query probe against the matching recorded response", () => {
|
||||
const recorded = timedCaptureEvidence([
|
||||
|
||||
@@ -81,6 +81,10 @@ export interface CompareStep {
|
||||
readonly equal_ignoring_order: boolean;
|
||||
readonly categories: DifferenceCategory[];
|
||||
readonly differences: Difference[];
|
||||
/** All differences computed for this step, before the returned list is capped. */
|
||||
readonly differences_total: number;
|
||||
readonly differences_omitted: number;
|
||||
readonly differences_truncated: boolean;
|
||||
readonly ignored_query: string[];
|
||||
}
|
||||
|
||||
@@ -287,6 +291,8 @@ const stepOf = (
|
||||
outcome: { env_status: number | null; differences: Difference[] },
|
||||
): CompareStep => {
|
||||
const categories = [...new Set(outcome.differences.map((d) => d.category))];
|
||||
const differences = outcome.differences.slice(0, STEP_DIFFERENCE_LIMIT);
|
||||
const omitted = outcome.differences.length - differences.length;
|
||||
return {
|
||||
...base,
|
||||
tool: call?.tool.name ?? null,
|
||||
@@ -296,7 +302,10 @@ const stepOf = (
|
||||
(difference) => difference.category === "order",
|
||||
),
|
||||
categories,
|
||||
differences: outcome.differences.slice(0, STEP_DIFFERENCE_LIMIT),
|
||||
differences,
|
||||
differences_total: outcome.differences.length,
|
||||
differences_omitted: omitted,
|
||||
differences_truncated: omitted > 0,
|
||||
ignored_query: [...(call?.ignoredQuery ?? [])],
|
||||
};
|
||||
};
|
||||
|
||||
@@ -5,7 +5,10 @@ import { createEvidenceBundle } from "../../domain/evidenceBundle.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { analysisErrorProjectionSchema } from "../../contracts/errorSchemas.js";
|
||||
import { InvestigationRecords } from "./InvestigationRecords.js";
|
||||
import { resolveInvestigationEvidenceClosure } from "./InvestigationEvidence.js";
|
||||
import {
|
||||
createInvestigationEvidenceClosureChecker,
|
||||
resolveInvestigationEvidenceClosure,
|
||||
} from "./InvestigationEvidence.js";
|
||||
|
||||
const observation = (path: string, links: readonly string[] = []): Evidence =>
|
||||
createEvidence(
|
||||
@@ -108,3 +111,50 @@ it("rejects a lookup that returns another semantic Evidence identity", () => {
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("resolves an authenticated source with a wide repeated-link array", () => {
|
||||
const leaf = observation("/wide-leaf");
|
||||
const source = observation(
|
||||
"/wide-source",
|
||||
Array(500_000).fill(leaf.evidence_id),
|
||||
);
|
||||
expect(source.evidence_links).toHaveLength(500_000);
|
||||
const records = new InvestigationRecords();
|
||||
for (const record of [leaf, source]) {
|
||||
const added = records.recordEvidence(record);
|
||||
if (!added.ok) throw added.error;
|
||||
}
|
||||
expect(
|
||||
records.evidenceForAnalysis(source.evidence_id)?.evidence_links,
|
||||
).toHaveLength(500_000);
|
||||
expect(
|
||||
resolveInvestigationEvidenceClosure(records, [source.evidence_id]),
|
||||
).toEqual({
|
||||
ok: true,
|
||||
value: [source.evidence_id, leaf.evidence_id].sort(),
|
||||
});
|
||||
});
|
||||
|
||||
it("reuses a complete shared source graph without weakening missing-source checks", () => {
|
||||
const leaf = observation("/shared-leaf");
|
||||
const source = observation("/shared-source", [leaf.evidence_id]);
|
||||
const records = new Map(
|
||||
[leaf, source].map((record) => [record.evidence_id, record]),
|
||||
);
|
||||
let lookups = 0;
|
||||
const check = createInvestigationEvidenceClosureChecker({
|
||||
evidenceForAnalysis: (id) => {
|
||||
lookups++;
|
||||
return records.get(id);
|
||||
},
|
||||
});
|
||||
expect(check([source.evidence_id])).toBe(true);
|
||||
expect(lookups).toBe(2);
|
||||
expect(check([source.evidence_id, leaf.evidence_id])).toBe(true);
|
||||
expect(lookups).toBe(2);
|
||||
const missing = `ev_${"0".repeat(64)}`;
|
||||
expect(check([missing])).toBe(false);
|
||||
expect(check([missing])).toBe(false);
|
||||
expect(lookups).toBe(3);
|
||||
expect(check([source.evidence_id, missing])).toBe(false);
|
||||
});
|
||||
@@ -5,6 +5,37 @@ import type { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
|
||||
import type { Evidence } from "../../domain/evidence.js";
|
||||
import { ok, type Result } from "../../domain/result.js";
|
||||
|
||||
type Lookup = (id: string) => Evidence | undefined;
|
||||
|
||||
/** Traverse without expanding a wide link array into function arguments. */
|
||||
const visitClosure = (
|
||||
lookup: Lookup,
|
||||
seedIds: readonly string[],
|
||||
knownComplete?: ReadonlySet<string>,
|
||||
): Result<Set<string>, EvidenceIntegrityError> => {
|
||||
const visited = new Set<string>();
|
||||
const pending = [...seedIds];
|
||||
while (pending.length > 0) {
|
||||
const evidenceId = pending.pop();
|
||||
if (
|
||||
evidenceId === undefined ||
|
||||
visited.has(evidenceId) ||
|
||||
knownComplete?.has(evidenceId)
|
||||
)
|
||||
continue;
|
||||
const evidence = resolveEvidenceInput(
|
||||
{ kind: "retained-evidence", evidence_id: evidenceId },
|
||||
lookup,
|
||||
"retained-closure",
|
||||
);
|
||||
if (!evidence.ok) return evidence;
|
||||
visited.add(evidenceId);
|
||||
for (const linkedId of evidence.value.evidence_links)
|
||||
pending.push(linkedId);
|
||||
}
|
||||
return ok(visited);
|
||||
};
|
||||
|
||||
/** Follow exact session references without exporting unrelated records or Unknowns. */
|
||||
export const resolveInvestigationEvidenceClosure = (
|
||||
reader: Pick<EvidenceReader, "evidenceForAnalysis">,
|
||||
@@ -16,21 +47,32 @@ export const resolveInvestigationEvidenceClosure = (
|
||||
const supplied = new Map(
|
||||
sources.map((source) => [source.evidence_id, source]),
|
||||
);
|
||||
const lookup = (id: string) =>
|
||||
supplied.get(id) ?? reader.evidenceForAnalysis(id);
|
||||
const visited = new Set<string>();
|
||||
const pending = [...seedIds];
|
||||
while (pending.length > 0) {
|
||||
const evidenceId = pending.pop();
|
||||
if (evidenceId === undefined || visited.has(evidenceId)) continue;
|
||||
const evidence = resolveEvidenceInput(
|
||||
{ kind: "retained-evidence", evidence_id: evidenceId },
|
||||
lookup,
|
||||
"retained-closure",
|
||||
const visited = visitClosure(
|
||||
(id) => supplied.get(id) ?? reader.evidenceForAnalysis(id),
|
||||
seedIds,
|
||||
);
|
||||
return visited.ok ? ok(uniqueSorted([...visited.value])) : visited;
|
||||
};
|
||||
|
||||
/** Reuse closure decisions while the caller's lookup remains immutable. */
|
||||
export const createInvestigationEvidenceClosureChecker = (
|
||||
reader: Pick<EvidenceReader, "evidenceForAnalysis">,
|
||||
): ((seedIds: readonly string[]) => boolean) => {
|
||||
const complete = new Set<string>();
|
||||
const incomplete = new Set<string>();
|
||||
return (seedIds) => {
|
||||
const single = seedIds.length === 1 ? seedIds[0] : undefined;
|
||||
if (single !== undefined && incomplete.has(single)) return false;
|
||||
const visited = visitClosure(
|
||||
(id) => reader.evidenceForAnalysis(id),
|
||||
seedIds,
|
||||
complete,
|
||||
);
|
||||
if (!evidence.ok) return evidence;
|
||||
visited.add(evidenceId);
|
||||
pending.push(...evidence.value.evidence_links);
|
||||
}
|
||||
return ok(uniqueSorted([...visited]));
|
||||
if (!visited.ok) {
|
||||
if (single !== undefined) incomplete.add(single);
|
||||
return false;
|
||||
}
|
||||
for (const id of visited.value) complete.add(id);
|
||||
return true;
|
||||
};
|
||||
};
|
||||
@@ -5,7 +5,10 @@ import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js";
|
||||
|
||||
/** Source facts remain available even when a completed observation is cancelled. */
|
||||
export type AnalysisResultWithSources<Value> = Result<Value, AnalysisError> & {
|
||||
export type AnalysisResultWithSources<Value, Failure = AnalysisError> = Result<
|
||||
Value,
|
||||
Failure
|
||||
> & {
|
||||
readonly sourceEvidence?: readonly Evidence[];
|
||||
};
|
||||
|
||||
|
||||
@@ -549,7 +549,7 @@ export class ReverseEngineeringService {
|
||||
? "REA_OBJDUMP_COMMAND"
|
||||
: "REA_RIZIN_COMMAND";
|
||||
earlyFailure = new AnalysisCapabilityUnavailableError(
|
||||
provider,
|
||||
profile.identity.id,
|
||||
operation,
|
||||
`Unable to run configured command (${String(code ?? "unknown error")})`,
|
||||
{
|
||||
@@ -708,10 +708,10 @@ export class ReverseEngineeringService {
|
||||
const limitations = [
|
||||
...(absolutePath === undefined
|
||||
? [
|
||||
"The objdump version is not reported; the output lists BFD targets advertised by the installed tool and does not establish support for the selected artifact.",
|
||||
"The objdump implementation and version are unknown. The captured output is the selected command's response and does not establish support for an artifact.",
|
||||
]
|
||||
: [
|
||||
"The executable version and supported formats are reported as unknown because no reliable version probe was performed.",
|
||||
"The executable implementation, version and supported formats are unknown because no reliable identity probe was performed.",
|
||||
]),
|
||||
"A non-zero process exit is preserved as an observed command result.",
|
||||
...(operation === "inspect_with_objdump" &&
|
||||
@@ -759,9 +759,7 @@ export class ReverseEngineeringService {
|
||||
`Command completion is unknown because the operation failed with ${failure._tag}: ${failure.message}`,
|
||||
]),
|
||||
...(absolutePath === undefined
|
||||
? [
|
||||
"This output lists BFD targets advertised by the installed objdump and is independent of the selected artifact.",
|
||||
]
|
||||
? []
|
||||
: [
|
||||
"When both are available, pre-run and post-run digests detect artifact changes across the provider call but cannot prove the exact bytes consumed during the call.",
|
||||
]),
|
||||
|
||||
@@ -36,7 +36,8 @@ export const DMG_EXISTING_ATTACHMENT_RECOVERY =
|
||||
"hdiutil refused a private read-only DMG attachment after listing an existing attachment of this image path. If it is still attached, ask its owner to release it when safe, then retry. REA does not borrow or detach existing attachments.";
|
||||
const DMG_CHECKSUM_ABSENT_LIMITATION =
|
||||
"hdiutil verify found no stored checksum in this disk image, so its integrity was not verified; the root SHA-256 identifies the inventoried bytes.";
|
||||
const VERIFY_SETTLE_DELAYS_MS = [250, 500] as const;
|
||||
const VERIFY_TRANSIENT_DELAY_MS = 250;
|
||||
const VERIFY_UNKNOWN_SETTLE_DELAYS_MS = [250, 500] as const;
|
||||
// APFS ejects and busy reporting can lag a detach attempt.
|
||||
const DETACH_SETTLE_DELAYS_MS = [250, 500, 1000] as const;
|
||||
const delay = (milliseconds: number): Promise<void> =>
|
||||
@@ -545,7 +546,13 @@ export class NativeDmgArtifactReader implements ArtifactReader {
|
||||
},
|
||||
);
|
||||
}
|
||||
const delayMs = VERIFY_SETTLE_DELAYS_MS[attempt];
|
||||
// An explicit resource refusal can outlast the initial settle window
|
||||
// after image creation. Its retries share the existing command deadline;
|
||||
// an ambiguous localized refusal keeps only the short diagnostic probe.
|
||||
const delayMs =
|
||||
cause.verifyDiagnostic === "Resource temporarily unavailable"
|
||||
? VERIFY_TRANSIENT_DELAY_MS
|
||||
: VERIFY_UNKNOWN_SETTLE_DELAYS_MS[attempt];
|
||||
// Retry only the producer's explicit transient diagnostic, or a localized
|
||||
// refusal of a readable target that may be the same, before any mount.
|
||||
if (
|
||||
|
||||
@@ -14,6 +14,10 @@ import {
|
||||
resolveOccurrenceParents,
|
||||
} from "./ArtifactGraphConstruction.js";
|
||||
import { createArtifactClassificationProbe } from "../ArtifactClassificationProbe.js";
|
||||
import { createAsarHeaderSignatureProbe } from "../AsarHeader.js";
|
||||
import { ArtifactReaderFailure } from "../ArtifactReader.js";
|
||||
import { translateArtifactFailure } from "../ArtifactOperationFailure.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { artifactIdForContent } from "../../domain/artifactIdentity.js";
|
||||
import { pe } from "../../domain/binaryTarget.fixture.js";
|
||||
import type { ArtifactEntry } from "../ArtifactReader.js";
|
||||
@@ -26,6 +30,7 @@ import {
|
||||
} from "../../domain/firmware/uf2.fixture.js";
|
||||
|
||||
import { classifyArtifactBytesAtPath } from "../../../tests/fixtures/artifactClassification.js";
|
||||
import { asarHeaderBytes } from "../../../tests/fixtures/asarHeader.js";
|
||||
import {
|
||||
ESP32C3_APP_SEGMENTS,
|
||||
esp8266Image,
|
||||
@@ -73,6 +78,42 @@ it("classifies original DOS header padding as an executable", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("projects the parsed ASAR header size when admission rejects an archive", () => {
|
||||
const bytes = asarHeaderBytes("{}", { padding: Buffer.alloc(8, 0x20) });
|
||||
const headerBytes = bytes.readUInt32LE(4);
|
||||
const probe = createAsarHeaderSignatureProbe({
|
||||
maximumBytes: headerBytes - 1,
|
||||
source: "setting",
|
||||
});
|
||||
probe.write(bytes);
|
||||
const asarHeader = probe.finish(bytes.length);
|
||||
expect(asarHeader).toMatchObject({
|
||||
state: "limit",
|
||||
extents: { headerBytes },
|
||||
});
|
||||
|
||||
let failure: unknown;
|
||||
try {
|
||||
classifyArtifactContent(
|
||||
"app.asar",
|
||||
{ prefix: bytes, asarHeader },
|
||||
bytes.length,
|
||||
);
|
||||
} catch (cause) {
|
||||
failure = cause;
|
||||
}
|
||||
expect(failure).toBeInstanceOf(ArtifactReaderFailure);
|
||||
if (!(failure instanceof ArtifactReaderFailure))
|
||||
throw new Error("ASAR admission failure was not classified");
|
||||
|
||||
const projected = projectAnalysisError(
|
||||
translateArtifactFailure("inspect_artifact", failure),
|
||||
);
|
||||
expect(projected.details?.limit).toMatchObject({
|
||||
required_value: headerBytes,
|
||||
});
|
||||
});
|
||||
|
||||
const entry = (path: string, kind: ArtifactEntry["kind"]): ArtifactEntry => ({
|
||||
path,
|
||||
kind,
|
||||
|
||||
@@ -557,7 +557,10 @@ export const classifyArtifactContent = (
|
||||
asarHeader.state !== "rejected"
|
||||
) {
|
||||
if (asarHeader.state === "limit")
|
||||
throw asarHeaderAdmissionFailure(asarHeader.admission);
|
||||
throw asarHeaderAdmissionFailure(
|
||||
asarHeader.admission,
|
||||
asarHeader.extents?.headerBytes,
|
||||
);
|
||||
if (asarHeader.state === "matched")
|
||||
role = { kind: "container", format: "asar" };
|
||||
else role = artifactRoleForFormat(path, contentFormat);
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { mediaTypeEssence } from "../domain/analysisView/recordedMediaType.js";
|
||||
import { cdpResourceColumnSchema } from "../domain/cdpObservationSchemas.js";
|
||||
import {
|
||||
admitRecords,
|
||||
isHttpUrl,
|
||||
@@ -57,18 +58,22 @@ export const exclusionReasonForUrl = (
|
||||
? "disallowed_origin"
|
||||
: "unsupported_url";
|
||||
|
||||
/** Preserve omitted and invalid protocol offsets as unknown, never invented coordinates. */
|
||||
/**
|
||||
* Preserve omitted and invalid protocol offsets as unknown, never invented
|
||||
* coordinates. Columns follow the shared CDP resource-column policy, which
|
||||
* keeps a negative column Chrome reports.
|
||||
*/
|
||||
export const cdpSourceCoordinates = (frame: UnknownRecord | undefined) => {
|
||||
const valid = (value: unknown): value is number =>
|
||||
const validLine = (value: unknown): value is number =>
|
||||
typeof value === "number" && Number.isSafeInteger(value) && value >= 0;
|
||||
const line = frame?.lineNumber;
|
||||
const column = frame?.columnNumber;
|
||||
const column = cdpResourceColumnSchema.safeParse(frame?.columnNumber);
|
||||
return {
|
||||
line: valid(line) ? line : null,
|
||||
column: valid(column) ? column : null,
|
||||
line: validLine(line) ? line : null,
|
||||
column: column.success ? column.data : null,
|
||||
invalid:
|
||||
(line !== undefined && !valid(line)) ||
|
||||
(column !== undefined && !valid(column)),
|
||||
(line !== undefined && !validLine(line)) ||
|
||||
(frame?.columnNumber !== undefined && !column.success),
|
||||
};
|
||||
};
|
||||
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
recordValue,
|
||||
type UnknownRecord,
|
||||
} from "./CdpCaptureValues.js";
|
||||
import { cdpResourceColumnSchema } from "../domain/cdpObservationSchemas.js";
|
||||
|
||||
const integer = z.int().min(0);
|
||||
/** Active runtime admission remains stricter than passive request observations. */
|
||||
@@ -25,7 +26,7 @@ export const runtimeRequestSchema = z.object({
|
||||
stack: z.unknown().optional(),
|
||||
url: z.string().optional(),
|
||||
lineNumber: integer.optional(),
|
||||
columnNumber: integer.optional(),
|
||||
columnNumber: cdpResourceColumnSchema.optional(),
|
||||
})
|
||||
.passthrough(),
|
||||
});
|
||||
|
||||
@@ -4,6 +4,7 @@ import {
|
||||
numberValue,
|
||||
recordValue,
|
||||
} from "./CdpCaptureValues.js";
|
||||
import { cdpResourceColumnSchema } from "../domain/cdpObservationSchemas.js";
|
||||
|
||||
const integer = z.int().min(0);
|
||||
const contextAuxDataSchema = z
|
||||
@@ -31,9 +32,9 @@ export const runtimeScriptParsedSchema = scriptDeclaration.extend({
|
||||
scriptId: z.string().min(1),
|
||||
executionContextId: integer,
|
||||
startLine: integer,
|
||||
startColumn: integer,
|
||||
startColumn: cdpResourceColumnSchema,
|
||||
endLine: integer,
|
||||
endColumn: integer,
|
||||
endColumn: cdpResourceColumnSchema,
|
||||
hash: z.string().optional(),
|
||||
sourceMapURL: z.string().optional(),
|
||||
hasSourceURL: z.boolean().optional(),
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { z } from "zod";
|
||||
import { cdpResourceColumnSchema } from "../../domain/cdpObservationSchemas.js";
|
||||
|
||||
const integer = z.int().min(0);
|
||||
const range = z.object({
|
||||
@@ -30,7 +31,7 @@ export const runtimeStackSchema = z.object({
|
||||
scriptId: z.string(),
|
||||
url: z.string(),
|
||||
lineNumber: integer,
|
||||
columnNumber: integer,
|
||||
columnNumber: cdpResourceColumnSchema,
|
||||
functionName: z.string(),
|
||||
}),
|
||||
),
|
||||
@@ -49,7 +50,7 @@ export const runtimeListenersSchema = z.object({
|
||||
once: z.boolean(),
|
||||
scriptId: z.string(),
|
||||
lineNumber: integer,
|
||||
columnNumber: integer,
|
||||
columnNumber: cdpResourceColumnSchema,
|
||||
}),
|
||||
),
|
||||
});
|
||||
|
||||
@@ -805,12 +805,46 @@ it.each(["[2001:db8::1]", "[::1]", "[::ffff:192.0.2.1]"])(
|
||||
},
|
||||
);
|
||||
|
||||
it.each(["[2620:0:860:ed1a::1]", "[::1]"])(
|
||||
"retains Chromium DevTools bracketed IPv6 spelling in reported Evidence: %s",
|
||||
(address) => {
|
||||
const fixture = historicalHar();
|
||||
const first = fixture.log.entries[0];
|
||||
if (first === undefined) throw new Error("fixture missing");
|
||||
const input = {
|
||||
log: {
|
||||
...fixture.log,
|
||||
creator: { name: "WebInspector", version: "537.36" },
|
||||
entries: [
|
||||
{ ...first, serverIPAddress: address },
|
||||
structuredClone(first),
|
||||
],
|
||||
},
|
||||
};
|
||||
const decoded = decodeHarCapture(JSON.stringify(input), []);
|
||||
expect(decoded.records[0]).toMatchObject({
|
||||
reported: { serverIPAddress: address },
|
||||
});
|
||||
expect(decoded.records[0]?.limitations).toContain(
|
||||
"This capture declares Chromium DevTools (WebInspector 537.36) and reports bracketed IPv6 serverIPAddress. Original spellings are retained; brackets were removed only from valid IPv6 values in the validation copy. Producer declarations are not authenticated.",
|
||||
);
|
||||
expect(
|
||||
decoded.records[1]?.limitations.some((limitation) =>
|
||||
limitation.includes("bracketed IPv6 serverIPAddress"),
|
||||
),
|
||||
).toBe(false);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
{ name: "other", version: "1.58.2", address: "[2001:db8::1]" },
|
||||
{ name: "Playwright", version: "1.58.1", address: "[2001:db8::1]" },
|
||||
{ name: "Playwright", version: "1.58.2", address: "[192.0.2.1]" },
|
||||
{ name: "Playwright", version: "1.58.2", address: "[2001:db8::1]:443" },
|
||||
{ name: "Playwright", version: "1.58.2", address: "[fe80::1%eth0]" },
|
||||
{ name: "WebInspector", version: "605.1.15", address: "[2001:db8::1]" },
|
||||
{ name: "WebInspector", version: "537.36", address: "[192.0.2.1]" },
|
||||
{ name: "WebInspector", version: "537.36", address: "[2001:db8::1]:443" },
|
||||
])(
|
||||
"rejects unrecognized bracketed HAR address input: $name $version $address",
|
||||
({ name, version, address }) => {
|
||||
|
||||
@@ -33,9 +33,14 @@ export const decodeHarCapture = (
|
||||
): DecodedCapture => {
|
||||
const raw = parseCaptureJson(text, "HAR");
|
||||
const validation = projectHar(raw, [], "validation");
|
||||
const adaptedPlaywrightEntries = new Set(
|
||||
createHarValidator()(validation.reported),
|
||||
);
|
||||
const ipv6Adaptation = createHarValidator()(validation.reported);
|
||||
const adaptedIpv6Entries = new Set(ipv6Adaptation?.entries);
|
||||
const ipv6Limitations =
|
||||
ipv6Adaptation === null
|
||||
? []
|
||||
: [
|
||||
`This capture declares ${ipv6Adaptation.producer} and reports bracketed IPv6 serverIPAddress. Original spellings are retained; brackets were removed only from valid IPv6 values in the validation copy. Producer declarations are not authenticated.`,
|
||||
];
|
||||
if (
|
||||
!captureObject(raw) ||
|
||||
!captureObject(raw.log) ||
|
||||
@@ -76,11 +81,7 @@ export const decodeHarCapture = (
|
||||
...projection,
|
||||
limitations: [
|
||||
"HAR text without an encoding declaration is producer-decoded Unicode. Unsupported declared body encodings remain opaque; original body bytes remain unknown unless content explicitly uses valid base64. Declared sizes are preserved independently of retained bytes. mitmproxy 12.2.3 SaveHar may report null optional postData.text; that null is retained and omitted only from the upstream validation copy.",
|
||||
...(adaptedPlaywrightEntries.has(ordinal)
|
||||
? [
|
||||
"This capture declares Playwright 1.58.2 and reports bracketed IPv6 serverIPAddress. Original spellings are retained; brackets were removed only from valid IPv6 values in the validation copy. Producer declarations are not authenticated.",
|
||||
]
|
||||
: []),
|
||||
...(adaptedIpv6Entries.has(ordinal) ? ipv6Limitations : []),
|
||||
...(recordsExcluded
|
||||
? [
|
||||
"This record's payload and sidecars are excluded because its structural ancestor property was explicitly marked sensitive.",
|
||||
|
||||
@@ -30,10 +30,28 @@ const timingsSchema = {
|
||||
},
|
||||
};
|
||||
|
||||
// Observed producers that copy Chromium's bracketed IPv6 remote address into
|
||||
// serverIPAddress. Chromium DevTools declares WebInspector with the frozen
|
||||
// AppleWebKit version from its user agent.
|
||||
const BRACKETED_IPV6_PRODUCERS = [
|
||||
{ name: "Playwright", version: "1.58.2", label: "Playwright 1.58.2" },
|
||||
{
|
||||
name: "WebInspector",
|
||||
version: "537.36",
|
||||
label: "Chromium DevTools (WebInspector 537.36)",
|
||||
},
|
||||
] as const;
|
||||
|
||||
/** Entries whose bracketed IPv6 address was adapted, and the declared producer. */
|
||||
interface BracketedIpv6Adaptation {
|
||||
readonly producer: string;
|
||||
readonly entries: readonly number[];
|
||||
}
|
||||
|
||||
/** Validate with pinned HAR 1.2 draft-06 schemas and the owned schema corrections. */
|
||||
export const createHarValidator = (): ((
|
||||
value: unknown,
|
||||
) => readonly number[]) => {
|
||||
) => BracketedIpv6Adaptation | null) => {
|
||||
const require = createRequire(import.meta.url);
|
||||
const schemas = z
|
||||
.record(z.string(), z.record(z.string(), z.unknown()))
|
||||
@@ -78,8 +96,7 @@ export const createHarValidator = (): ((
|
||||
if (validate === undefined)
|
||||
throw new Error("Pinned HAR root schema is unavailable");
|
||||
return (value) => {
|
||||
const adaptedPlaywrightEntries =
|
||||
allowObservedProducerRepresentations(value);
|
||||
const adaptations = allowObservedProducerRepresentations(value);
|
||||
if (!validate(value)) {
|
||||
const error = validate.errors?.[0];
|
||||
// AJV messages name the violated constraint, such as the missing
|
||||
@@ -92,36 +109,43 @@ export const createHarValidator = (): ((
|
||||
error?.instancePath ?? "",
|
||||
);
|
||||
}
|
||||
return adaptedPlaywrightEntries;
|
||||
return adaptations;
|
||||
};
|
||||
};
|
||||
|
||||
/** Name the declared producer when it is observed to bracket IPv6 addresses. */
|
||||
const bracketedIpv6ProducerOf = (
|
||||
creator: Record<string, unknown>,
|
||||
): string | null =>
|
||||
BRACKETED_IPV6_PRODUCERS.find(
|
||||
({ name, version }) => creator.name === name && creator.version === version,
|
||||
)?.label ?? null;
|
||||
|
||||
/** Adapt only observed producer representations in the disposable validation copy. */
|
||||
const allowObservedProducerRepresentations = (
|
||||
value: unknown,
|
||||
): readonly number[] => {
|
||||
if (!captureObject(value) || !captureObject(value.log)) return [];
|
||||
): BracketedIpv6Adaptation | null => {
|
||||
if (!captureObject(value) || !captureObject(value.log)) return null;
|
||||
const log = value.log;
|
||||
if (!captureObject(log.creator) || !Array.isArray(log.entries)) return [];
|
||||
if (!captureObject(log.creator) || !Array.isArray(log.entries)) return null;
|
||||
const mitmproxy =
|
||||
log.creator.name === "mitmproxy" && log.creator.version === "12.2.3";
|
||||
const playwright =
|
||||
log.creator.name === "Playwright" && log.creator.version === "1.58.2";
|
||||
if (!mitmproxy && !playwright) return [];
|
||||
const adaptedPlaywrightEntries: number[] = [];
|
||||
const bracketedIpv6Producer = bracketedIpv6ProducerOf(log.creator);
|
||||
if (!mitmproxy && bracketedIpv6Producer === null) return null;
|
||||
const adaptedEntries: number[] = [];
|
||||
for (const [ordinal, entry] of log.entries.entries()) {
|
||||
const candidate: unknown = entry;
|
||||
if (!captureObject(candidate)) continue;
|
||||
const address = candidate.serverIPAddress;
|
||||
if (
|
||||
playwright &&
|
||||
bracketedIpv6Producer !== null &&
|
||||
typeof address === "string" &&
|
||||
address.startsWith("[") &&
|
||||
address.endsWith("]") &&
|
||||
isIP(address.slice(1, -1)) === 6
|
||||
) {
|
||||
candidate.serverIPAddress = address.slice(1, -1);
|
||||
adaptedPlaywrightEntries.push(ordinal);
|
||||
adaptedEntries.push(ordinal);
|
||||
}
|
||||
if (!mitmproxy) continue;
|
||||
const request = candidate.request;
|
||||
@@ -129,7 +153,9 @@ const allowObservedProducerRepresentations = (
|
||||
const body = request.postData;
|
||||
if (captureObject(body) && body.text === null) delete body.text;
|
||||
}
|
||||
return adaptedPlaywrightEntries;
|
||||
return bracketedIpv6Producer === null
|
||||
? null
|
||||
: { producer: bracketedIpv6Producer, entries: adaptedEntries };
|
||||
};
|
||||
|
||||
const HAR_REQUEST_URL_FORMAT = "absolute-url";
|
||||
|
||||
+38
-5
@@ -252,7 +252,7 @@ export const createCli = (
|
||||
);
|
||||
cli.serve = (arguments_ = process.argv.slice(2), options) => {
|
||||
if (complete(arguments_, options ?? {})) return Promise.resolve();
|
||||
const normalized = normalizeCliArguments(arguments_);
|
||||
let normalized = normalizeCliArguments(arguments_);
|
||||
const { command, flags } = splitIncurBuiltinArguments(normalized);
|
||||
const builtin = command[0] === PRODUCT_IDENTITY.cliBinary ? 1 : 0;
|
||||
if (
|
||||
@@ -320,10 +320,34 @@ export const createCli = (
|
||||
// prevents a save. Guard that path: reject a settings file REA cannot
|
||||
// update, back up the file, preserve the caller's environment, and verify
|
||||
// the registration landed.
|
||||
// SyncMcp.register also writes Amp when no agents are selected. Mode
|
||||
// switches return before registration and must not touch its settings.
|
||||
// Use the same normalized arguments for selection and dispatch, including
|
||||
// non-Amp equals forms. Incur consumes command values before agent flags.
|
||||
normalized = serveArguments;
|
||||
const registration = splitIncurBuiltinArguments(serveArguments);
|
||||
const rest = registration.command.slice(builtin + 2);
|
||||
const agents: string[] = [];
|
||||
for (let index = 0; index < rest.length; index += 1) {
|
||||
if (
|
||||
(rest[index] === "--command" || rest[index] === "-c") &&
|
||||
rest[index + 1]
|
||||
)
|
||||
index += 1;
|
||||
else if (rest[index] === "--agent" && rest[index + 1])
|
||||
agents.push(rest[++index]!);
|
||||
}
|
||||
const ampRequested =
|
||||
command.some(
|
||||
(token, index) => token === "--agent" && command[index + 1] === "amp",
|
||||
) || command.includes("--agent=amp");
|
||||
![
|
||||
"--help",
|
||||
"-h",
|
||||
"--llms",
|
||||
"--llms-full",
|
||||
"--mcp",
|
||||
"--update",
|
||||
"--incur-update-check",
|
||||
].some((flag) => registration.flags.has(flag)) &&
|
||||
(agents.length === 0 || agents.includes("amp"));
|
||||
if (ampRequested) {
|
||||
const settingsPath = join(homedir(), ".config", "amp", "settings.json");
|
||||
let priorEnv: unknown;
|
||||
@@ -409,9 +433,17 @@ export const createCli = (
|
||||
}
|
||||
|
||||
const runRegistration = (backupPath?: string): Promise<void> => {
|
||||
let failed = false;
|
||||
const result = serveStrict(
|
||||
serveArguments,
|
||||
options ?? {},
|
||||
{
|
||||
...options,
|
||||
exit: (code) => {
|
||||
failed ||= code !== 0;
|
||||
if (options?.exit !== undefined) options.exit(code);
|
||||
else process.exitCode = code;
|
||||
},
|
||||
},
|
||||
(tokens, selected) =>
|
||||
serveDeclared(tokens, selected, (declared, declaredOptions) =>
|
||||
serve(
|
||||
@@ -421,6 +453,7 @@ export const createCli = (
|
||||
),
|
||||
);
|
||||
return result.then(() => {
|
||||
if (failed) return;
|
||||
let failure: string | undefined;
|
||||
try {
|
||||
if (!existsSync(settingsPath))
|
||||
|
||||
@@ -32,6 +32,7 @@ import {
|
||||
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
||||
import type { ZodSafeParseResult } from "zod";
|
||||
import { bindCliEvidenceCallerPath } from "./cliEvidenceCallerPath.js";
|
||||
import type { AnalysisResultWithSources } from "../application/investigation/recordEvidenceSources.js";
|
||||
|
||||
export const analysisEvidenceDeliveryOptions = {
|
||||
detail: z
|
||||
@@ -203,9 +204,16 @@ const withUndeliveredSourceClosure = (
|
||||
|
||||
/** Publish already finalized CLI analysis without opening a second provider/session. */
|
||||
export const publishCliAnalysisEvidence = async (
|
||||
result: Result<Evidence, JsonValue>,
|
||||
result: AnalysisResultWithSources<Evidence, JsonValue>,
|
||||
input: CliEvidenceDelivery,
|
||||
): Promise<JsonValue> => {
|
||||
input = {
|
||||
...input,
|
||||
sourceEvidence: [
|
||||
...(input.sourceEvidence ?? []),
|
||||
...(result.sourceEvidence ?? []),
|
||||
],
|
||||
};
|
||||
const { operation, signal, detail, evidenceOutput } = input;
|
||||
const projectFailure = (error: AnalysisError) =>
|
||||
projectAnalysisError(
|
||||
|
||||
@@ -357,24 +357,41 @@ const registerTraceCommand = (
|
||||
.default(false)
|
||||
.describe("Match the query with exact letter case"),
|
||||
snapshot: snapshotSelectionOption,
|
||||
...analysisEvidenceDeliveryOptions,
|
||||
...nativeOpenOptionFields,
|
||||
}),
|
||||
run: ({ args, options }) =>
|
||||
logCliCommand(logger, "trace", () =>
|
||||
runDirectAnalysis(
|
||||
args.path,
|
||||
"trace_feature",
|
||||
{
|
||||
query: resolveCliAnalysisSelector(
|
||||
args.query,
|
||||
options.query,
|
||||
"query",
|
||||
"trace_feature",
|
||||
),
|
||||
case_sensitive: options.caseSensitive,
|
||||
},
|
||||
directAnalysisOptionsFromCli(logger, options),
|
||||
).then(projectCliAnalysisResult),
|
||||
withCommandCancellation((signal) =>
|
||||
logCliCommand(logger, "trace", async () => {
|
||||
const delivery = {
|
||||
providerId: "auto",
|
||||
operation: "trace_feature",
|
||||
signal,
|
||||
detail: options.detail,
|
||||
evidenceOutput: options.evidence_output,
|
||||
};
|
||||
const inputError = cliEvidenceDeliveryInputError(delivery);
|
||||
if (inputError !== undefined) return projectAnalysisError(inputError);
|
||||
const complete = await runDirectAnalysis(
|
||||
args.path,
|
||||
"trace_feature",
|
||||
{
|
||||
query: resolveCliAnalysisSelector(
|
||||
args.query,
|
||||
options.query,
|
||||
"query",
|
||||
"trace_feature",
|
||||
),
|
||||
case_sensitive: options.caseSensitive,
|
||||
},
|
||||
{
|
||||
...directAnalysisOptionsFromCli(logger, options),
|
||||
signal,
|
||||
includeEvidenceSources: options.evidence_output !== undefined,
|
||||
},
|
||||
);
|
||||
return publishCliAnalysisEvidence(complete, delivery);
|
||||
}),
|
||||
),
|
||||
});
|
||||
};
|
||||
@@ -430,7 +447,11 @@ const registerFunctionCommand = (
|
||||
"analyze_function",
|
||||
),
|
||||
},
|
||||
{ ...directAnalysisOptionsFromCli(logger, options), signal },
|
||||
{
|
||||
...directAnalysisOptionsFromCli(logger, options),
|
||||
signal,
|
||||
includeEvidenceSources: options.evidence_output !== undefined,
|
||||
},
|
||||
);
|
||||
return publishCliAnalysisEvidence(complete, delivery);
|
||||
}),
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { FIRMWARE_OPERATION_ENGINES } from "../contracts/firmware/firmwareToolRequirements.js";
|
||||
import { projectCliAnalysisResult } from "./projectCliAnalysisResult.js";
|
||||
import { z } from "incur";
|
||||
import { FirmwareAnalysisService } from "../application/firmware/FirmwareAnalysisService.js";
|
||||
@@ -36,7 +37,7 @@ export const registerFirmwareCommands = (
|
||||
callerPath,
|
||||
() => service.execute(operation, input, { signal }),
|
||||
service,
|
||||
operation === "inspect_firmware_regions" ? "binwalk" : "unblob",
|
||||
FIRMWARE_OPERATION_ENGINES[operation],
|
||||
operation,
|
||||
signal,
|
||||
).then(projectCliAnalysisResult),
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { z } from "incur";
|
||||
import type { Logger } from "pino";
|
||||
import { OBJDUMP_PROVIDER_IDENTITY } from "../objdump/ObjdumpProviderIdentity.js";
|
||||
|
||||
import { createReverseEngineeringService } from "../composition/reverseEngineering.js";
|
||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
@@ -192,7 +193,7 @@ export const registerReverseEngineeringCommands = (
|
||||
const service = createService(environment);
|
||||
cli.command(CLI_COMMANDS.inspectWithObjdump, {
|
||||
description:
|
||||
"Inspect a local artifact with GNU objdump and retain its raw output as Evidence",
|
||||
"Inspect a local artifact using GNU-style objdump arguments and retain the selected command output as Evidence",
|
||||
args: z.object({
|
||||
path: z
|
||||
.string()
|
||||
@@ -200,7 +201,7 @@ export const registerReverseEngineeringCommands = (
|
||||
.describe("Path to the local artifact to inspect."),
|
||||
operation: z
|
||||
.enum(OBJDUMP_ARTIFACT_OPERATIONS)
|
||||
.describe("GNU objdump inspection operation to run."),
|
||||
.describe("GNU-style objdump operation to run."),
|
||||
}),
|
||||
options: z.object({
|
||||
...analysisEvidenceDeliveryOptions,
|
||||
@@ -208,7 +209,7 @@ export const registerReverseEngineeringCommands = (
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
"Native symbol to select with the disassemble operation; GNU objdump controls matching and boundaries.",
|
||||
"Native symbol to select with the disassemble operation; the selected executable controls matching and boundaries.",
|
||||
),
|
||||
followDebugLinks: z
|
||||
.boolean()
|
||||
@@ -222,7 +223,7 @@ export const registerReverseEngineeringCommands = (
|
||||
withCommandCancellation((signal) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.inspectWithObjdump, async () => {
|
||||
const delivery = {
|
||||
providerId: "gnu.objdump",
|
||||
providerId: OBJDUMP_PROVIDER_IDENTITY.id,
|
||||
operation: "inspect_with_objdump",
|
||||
signal,
|
||||
detail: options.detail,
|
||||
@@ -255,7 +256,7 @@ export const registerReverseEngineeringCommands = (
|
||||
),
|
||||
});
|
||||
cli.command(CLI_COMMANDS.listObjdumpArchitectures, {
|
||||
description: "List BFD targets advertised by the installed GNU objdump",
|
||||
description: "Capture the selected objdump command response to -i",
|
||||
args: z.object({}),
|
||||
options: z.object({
|
||||
...analysisEvidenceDeliveryOptions,
|
||||
@@ -268,7 +269,7 @@ export const registerReverseEngineeringCommands = (
|
||||
CLI_COMMANDS.listObjdumpArchitectures,
|
||||
async () => {
|
||||
const delivery = {
|
||||
providerId: "gnu.objdump",
|
||||
providerId: OBJDUMP_PROVIDER_IDENTITY.id,
|
||||
operation: "inspect_with_objdump",
|
||||
signal,
|
||||
detail: options.detail,
|
||||
|
||||
@@ -46,8 +46,8 @@ export interface IncurBuiltinArgumentSplit {
|
||||
|
||||
/**
|
||||
* Mirror Incur's builtin pass over normalized arguments without stopping at a
|
||||
* rejected value, so callers can still see every later token. REA's boolean
|
||||
* timing global follows the same valueless pass; no config flag is registered.
|
||||
* rejected value, so callers can still see every later token. Then mirror the
|
||||
* boolean timing global pass over the remaining tokens; no config is registered.
|
||||
*/
|
||||
export const splitIncurBuiltinArguments = (
|
||||
arguments_: readonly string[],
|
||||
@@ -59,7 +59,7 @@ export const splitIncurBuiltinArguments = (
|
||||
const argument = arguments_[index]!;
|
||||
const next = arguments_[index + 1];
|
||||
if (
|
||||
INCUR_BUILTIN_FLAGS.has(argument) ||
|
||||
(INCUR_BUILTIN_FLAGS.has(argument) && argument !== "--timings") ||
|
||||
// A following value belongs to a command-local `--version` option.
|
||||
(argument === "--version" && (next === undefined || next.startsWith("-")))
|
||||
) {
|
||||
@@ -73,7 +73,24 @@ export const splitIncurBuiltinArguments = (
|
||||
}
|
||||
command.push(argument);
|
||||
}
|
||||
return { command, flags, outputValues };
|
||||
// Incur extracts builtin output values first, then parses globals until `--`.
|
||||
// A timing-shaped output value is data; a command option's value is still
|
||||
// subject to this global pass before the registration builtin consumes it.
|
||||
let globalsEnded = false;
|
||||
const remaining = command.filter((token) => {
|
||||
if (token === "--") globalsEnded = true;
|
||||
if (
|
||||
!globalsEnded &&
|
||||
(token === "--timings" ||
|
||||
token === "--no-timings" ||
|
||||
token.startsWith("--timings="))
|
||||
) {
|
||||
flags.add(token);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
});
|
||||
return { command: remaining, flags, outputValues };
|
||||
};
|
||||
|
||||
/**
|
||||
|
||||
@@ -17,7 +17,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
|
||||
...toolContractMetadata("inspect_analysis_view"),
|
||||
kind: "application",
|
||||
description:
|
||||
'Use javascript-routes collections to query route patterns, source-linked occurrences, source status, unknowns or summary counts from retained JavaScript application or web-bundle Evidence. Use web-bundle collections endpoints or inferences to page endpoint observations and inferences from retained analyze_web_bundle Evidence; summary returns target URL, origin, and counts. Inspect selected views of supported completed Evidence without repeating analysis. Choose a view compatible with the parent operation described below. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Native export tables use kind native-exports, collection exports/issues, exact names or original indices, selected fields and matching-row pagination; parent Evidence plus source_index identifies each row. Source coverage stays separate from page coverage. BEAM views use kind beam, collections atoms/exports/imports/lambdas/chunks, exact names/indices/fields and matching-row pages; import modules intersects function names. source_index identifies original parent-and-collection rows, preserving every arity and raw atom index. beam.counts preserves null missing tables versus decoded empty tables; runtime absence remains unknown. PYC views use kind python-pyc, collection code_objects, names/indices/fields and paging; source_index scopes every same-name alternative to the parent, nested_code keeps code-constant references, and pyc.coverage preserves source coverage. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent also supports summary and browser-scenario steps/events/transactions/text pages, preserving step_indices, event_kinds, transaction_ids and selected artifacts. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the fLine truncated
|
||||
'Use javascript-routes collections to query route patterns, source-linked occurrences, source status, unknowns or summary counts from retained JavaScript application or web-bundle Evidence. Use web-bundle collections endpoints or inferences to page endpoint observations and inferences from retained analyze_web_bundle Evidence; summary returns target URL, origin, and counts. For retained trace_feature Evidence, kind native-feature-trace pages matches, references or unknowns; matches filter exact address, references filter exact source_address and target_address, source_index identifies each original row, and list_unknowns pages durable question heads. Inspect selected views of supported completed Evidence without repeating analysis. Choose a view compatible with the parent operation described below. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Native export tables use kind native-exports, collection exports/issues, exact names or original indices, selected fields and matching-row pagination; parent Evidence plus source_index identifies each row. Source coverage stays separate from page coverage. BEAM views use kind beam, collections atoms/exports/imports/lambdas/chunks, exact names/indices/fields and matching-row pages; import modules intersects function names. source_index identifies original parent-and-collection rows, preserving every arity and raw atom index. beam.counts preserves null missing tables versus decoded empty tables; runtime absence remains unknown. PYC views use kind python-pyc, collection code_objects, names/indices/fields and paging; source_index scopes every same-name alternative to the parent, nested_code keeps code-constant references, and pyc.coverage preserves source coverage. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent also supports summary and browser-scenario steps/events/transactions/text pages, preserving step_indices, event_kinds, transaction_ids and selected artifacts. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redactLine truncated
|
||||
inputSchema: inspectAnalysisViewInputSchema,
|
||||
outputSchema: evidenceResultOf(analysisViewResultSchema),
|
||||
examples: [
|
||||
|
||||
Loaded 100 of 224 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user