feat(firmware): integrate Binwalk and Unblob analysis

* feat(firmware): integrate Binwalk and Unblob analysis

* fix(firmware): verify published files and namespace source references

* test(firmware): isolate extractor dependency failure from process cleanup

* fix(firmware): retain actionable report and cleanup diagnostics

* test: assert retained batch output failure diagnostics
This commit is contained in:
N0zoM1z0
2026-10-06 14:16:04 +00:00
committed by GitHub
parent 8d7071da4c
commit 5a7c769692
58 changed files with 3230 additions and 16 deletions
+6
View File
@@ -1,3 +1,9 @@
[submodule "third_party/jadx-headless-mcp"]
path = third_party/jadx-headless-mcp
url = https://github.com/1013503897/jadx-headless-mcp.git
[submodule "third_party/binwalk"]
path = third_party/binwalk
url = https://github.com/ReFirmLabs/binwalk.git
[submodule "third_party/unblob"]
path = third_party/unblob
url = https://github.com/onekey-sec/unblob.git
+1 -1
View File
@@ -9,7 +9,7 @@
".codex/**",
".devin/**",
"tests/fixtures/**",
"third_party/jadx-headless-mcp/**",
"third_party/**",
"tests/conformance/**/*.c",
"tests/conformance/**/*.m",
"tests/conformance/**/*.swift",
+1 -1
View File
@@ -11,6 +11,6 @@
"CHANGELOG.md",
"package-lock.json",
"_reference/**",
"third_party/jadx-headless-mcp/**"
"third_party/**"
]
}
+1 -1
View File
@@ -140,6 +140,6 @@
".codex",
".devin",
".factory",
"third_party/jadx-headless-mcp"
"third_party"
]
}
+5
View File
@@ -151,6 +151,8 @@ Update either installation with `rea update`.
Native binary analysis requires [Hopper](https://www.hopperapp.com/) or [Ghidra](#ghidra-analysis-provider). Hopper is separate software with its own license; its demo supports analysis with vendor-defined limits. REA can use Ghidra that you have already installed.
Firmware region inspection and explicit extraction use caller-supplied Binwalk and Unblob on Linux. See [Firmware analysis](docs/firmware-analysis.md) for setup, provenance, resource limits and native handoff.
Static APK analysis uses a separately supplied headless JADX JAR and Java, with no emulator or APK execution. See [Android analysis](docs/android-analysis.md) for setup, CLI/MCP operations, coverage and public test fixtures.
Windows Ghidra support is experimental for native x86-64 PE applications on local NTFS. The package bundles native Job Object, private-DACL, and path-admission controls. See [Windows Ghidra P0](docs/windows-ghidra-p0.md) for prerequisites and verified scope.
@@ -316,6 +318,7 @@ See [native investigation](docs/native-investigation.md) for keyed archives, ins
| Native macOS utilities | 7 | Mach-O metadata, code signatures, plists, architectures, and Swift demangling without launching Hopper |
| Artifact graph | 5 | directory and package inventories, compiled Interface Builder files, Apple asset catalogs, and extraction |
| Managed PE/CLI | 7 | .NET identity, metadata, CIL instructions, native dependencies, reconstruction imports, and build comparisons |
| Firmware | 2 | Linux firmware region inspection and explicit extraction |
| Android APK | 5 | package and manifest declarations, class search, member inventories, method decompilation, and incoming static references |
| Browser observation | 9 | page structure, network metadata, scripts, source maps, WebMCP discovery, screenshots, and capture comparisons |
| Electron analysis | 5 | renderer observation, static app mapping, and static/runtime reconciliation |
@@ -468,6 +471,8 @@ flowchart LR
REA --> Browser["Browser CDP provider"]
REA --> Android["Android static provider<br/>headless JADX adapter"]
Android --> Runtime
REA --> Firmware["Firmware providers<br/>Binwalk / Unblob adapters"]
Firmware --> Runtime
REA --> Process["Process capture provider"]
Runtime --> Target["Target software"]
Process --> Target
+1
View File
@@ -209,6 +209,7 @@ rea setup
| أدوات macOS الأصلية | 7 | بيانات Mach-O والتوقيعات وplist والمعماريات واستعادة أسماء Swift |
| الملفات والحزم | 5 | فحص الدلائل والحزم وInterface Builder وموارد Apple والاستخراج |
| .NET PE/CLI | 7 | هوية التجميع والبيانات الوصفية وCIL والاستدعاءات الأصلية والمقارنة واستيراد إعادة البناء |
| البرامج الثابتة | 2 | فحص مناطق البرامج الثابتة واستخراجها الصريح على Linux |
| Android APK | 5 | بيانات الحزمة وmanifest والبحث عن الأصناف والأعضاء وفك ترجمة الأساليب والمراجع الثابتة |
| مراقبة المتصفح | 9 | الصفحات والبرامج النصية وخرائط المصدر وWebMCP والصور ومقارنة الالتقاط |
| تحليل Electron | 5 | مراقبة الصفحات وبنية التطبيق وربط النتائج الثابتة بنتائج التشغيل |
+1
View File
@@ -198,6 +198,7 @@ REA は手順 1〜5 のバイナリ解析を処理し、手順 6 はエージェ
| macOS ネイティブ | 7 | Mach-O メタデータ、署名、plist、アーキテクチャ、Swift 名の復元 |
| ファイルとパッケージ | 5 | ディレクトリとパッケージ、Interface Builder、Apple リソース、抽出 |
| .NET PE/CLI | 7 | アセンブリ識別、メタデータ、CIL、ネイティブ呼び出し、比較、再構築の取り込み |
| ファームウェア | 2 | Linux の領域検査と明示的な展開 |
| Android APK | 5 | パッケージと manifest、クラス検索、メンバー一覧、メソッド逆コンパイル、静的参照 |
| ブラウザー観察 | 9 | ページ、スクリプト、ソースマップ、WebMCP、スクリーンショット、比較 |
| Electron 解析 | 5 | ページ観察、アプリ構造、静的・実行時結果の対応付け |
+1
View File
@@ -199,6 +199,7 @@ REA는 1–5단계의 바이너리 분석을 처리합니다. 6단계는 에이
| macOS 네이티브 | 7 | Mach-O 메타데이터, 서명, plist, 아키텍처, Swift 이름 복원 |
| 파일과 패키지 | 5 | 디렉터리와 패키지 검사, Interface Builder, Apple 리소스, 추출 |
| .NET PE/CLI | 7 | 어셈블리 식별, 메타데이터, CIL, 네이티브 호출, 비교, 재구성 가져오기 |
| 펌웨어 | 2 | Linux 펌웨어 영역 검사 및 명시적 추출 |
| Android APK | 5 | 패키지와 manifest, 클래스 검색, 멤버 목록, 메서드 디컴파일, 정적 참조 |
| 브라우저 관찰 | 9 | 페이지, 스크립트, 소스 맵, WebMCP, 스크린샷, 캡처 비교 |
| Electron 분석 | 5 | 페이지 관찰, 앱 구조, 정적·런타임 결과 연결 |
+1
View File
@@ -198,6 +198,7 @@ REA 负责第 1–5 步中的二进制分析。第 6 步由智能体使用其常
| macOS 原生工具 | 7 | Mach-O 元数据、代码签名、plist、架构与 Swift 符号还原 |
| 制品检查 | 5 | 目录与软件包检查、Interface Builder、Apple 资源目录与提取 |
| .NET PE/CLI | 7 | 程序集身份、元数据、CIL、原生调用声明、成员比较与重建导入 |
| 韧体 | 2 | Linux 韧体区域识别与显式解包 |
| Android APK | 5 | 包与 manifest 声明、class 搜索、成员清单、方法反编译与静态引用 |
| 浏览器观察 | 9 | 页面、脚本、来源映射、WebMCP 工具、截图与捕获比较 |
| Electron 分析 | 5 | 页面观察、应用结构与静态/运行时结果关联 |
+4
View File
@@ -25,6 +25,7 @@ graph TD
AUX["src/artifacts/ + src/native/<br/>disjoint artifact and native capabilities"]
BROWSER["src/browser/<br/>Loopback CDP transport<br/>exact-origin passive observation"]
DOTNET["src/dotnet/<br/>Managed static metadata provider<br/>PE/CLI metadata inventory, members, IL"]
FIRMWARE["src/firmware/<br/>Binwalk / Unblob adapters<br/>owned processes, report normalization, private extraction"]
ANDROID["src/android/<br/>Static APK adapter<br/>owned SDK stdio, pinned headless JADX, normalization"]
REFERENCE["src/reference/<br/>Filesystem reference-source reader<br/>bounded traversal, validation, errors"]
end
@@ -62,6 +63,9 @@ graph TD
MAIN --> SERVER
MAIN --> APP
CLI --> APP
APP --> FIRMWARE
FIRMWARE --> PROCESS
FIRMWARE --> CONTRACTS
APP --> ANDROID
ANDROID --> PROCESS
ANDROID --> CONTRACTS
+173
View File
@@ -0,0 +1,173 @@
# Firmware analysis
REA integrates **Binwalk 3.1.0** for region inspection and **Unblob 26.6.4** for
explicit extraction. This first firmware milestone is verified on Linux x64.
No device, root privilege, mount, emulation or execution of extracted programs
is needed. Arbitrary firmware bytes do not pass native executable admission;
select a returned ELF separately for supported Ghidra analysis.
## Caller-supplied tools
Provide existing tools using absolute executable paths, scoped to your shell:
```sh
export REA_BINWALK_COMMAND=/absolute/path/to/binwalk
export REA_UNBLOB_COMMAND=/absolute/path/to/unblob
```
REA does not install these tools, their dependencies or system packages. It
checks the actual tool version when executing a request. Inspection and
extraction have independent prerequisites; one does not require the other.
Both require util-linux `prlimit`, normally `/usr/bin/prlimit`; override its
absolute path with `REA_FIRMWARE_PRLIMIT_COMMAND` if needed.
Unblob needs the external extractor for the selected format on its inherited
`PATH`: for example, `debugfs` for ext2/ext3/ext4. Its default Landlock sandbox
and skip rules stay enabled. Sandbox startup failures remain actionable errors;
REA does not retry with `--no-sandbox`. Missing extractors reported in a valid
analysis report retain partial results and dependency names inline.
Upstream sources are preserved unmodified as optional pinned submodules:
```sh
git submodule update --init third_party/binwalk third_party/unblob
```
Normal REA builds and npm packages do not include or build these sources or
engine binaries. See [upstream provenance](../third_party/README.md).
## CLI and MCP
```sh
rea inspect-firmware-regions /path/to/firmware.bin
rea extract-firmware /path/to/firmware.bin /path/to/new-output
rea extract-firmware /path/to/firmware.bin /path/to/selected-output \
--offset 64 --length 4096 --max-depth 3
```
The corresponding MCP tools accept the same intent:
```json
{
"name": "inspect_firmware_regions",
"arguments": { "path": "/path/to/firmware.bin" }
}
```
```json
{
"name": "extract_firmware",
"arguments": {
"path": "/path/to/firmware.bin",
"output_directory": "/path/to/new-output",
"range": { "offset": 64, "length": 4096 },
"max_depth": 3
}
}
```
Both return Evidence with the original path and SHA-256, raw producer reports,
actual executable path/digest, version and limitations. Binwalk signature UUIDs
and Unblob blob IDs are producer-local identities; compare content hashes and
source ranges across runs. A matching version does not attest an installed
build's source revision, so that field remains unknown.
### Interpret the results
- Region offsets refer to bytes in the original firmware. `reported_size`
preserves Binwalk's output; `size_basis=provider_reported_validation_unknown`
reflects that Binwalk can replace unknown lengths with the next signature or
EOF before serialization. Neither a signature nor its confidence byte proves
a valid partition, encryption or runtime behavior.
- Extraction returns regular file paths, independently verified SHA-256 values,
producer hashes when available, chunks, diagnostics and task derivations.
Unblob's task report is unordered; REA resolves relationships by paths and
blob IDs. Logical references use `$input`, `$output`, or `$output/<relative-path>`
so a real file named `$input` stays distinct from the original input.
Chunk offsets belong to the task's input file. Only chunks in the
selected input have a known `root_file_range`, adjusted by the selected offset.
- Decompressed children have unknown original byte ranges and runtime addresses.
Task lineage does not establish a byte-for-byte mapping through decompression.
- `partial` marks unknown/encrypted chunks, reported extraction problems,
unpublished links/special files or reached depth limits. `complete` means the
engine finished its configured extraction without those reported conditions;
default skip rules and format coverage still apply. It does not establish
complete firmware reconstruction.
- The destination must be absent, absolute, with an existing parent. Publication
uses exclusive, symlink-resistant writes and durable digest readback. Existing
destinations are never overwritten. This is not an atomic directory rename:
a new tree can be visible while files are written. Failed uncommitted output
is rolled back. Symlinks and special files are recorded without publication;
empty directories, ownership and executable permission bits are not reproduced.
Select a verified native child for the existing tools:
```json
{
"name": "open_binary",
"arguments": {
"path": "/path/from/extraction/usr/bin/program",
"provider_id": "ghidra"
}
}
```
Then use `analyze_function`, `inspect_native_api` or the other native primitives.
Native architecture and executable validation still apply. This milestone does
not add MIPS/PPC/RISC-V decompilation, raw flash load profiles, bootloader/device
tree semantics, vulnerability scanning, emulation or hardware runtime capture.
## Resource and lifecycle policy
- Nonempty regular firmware inputs: at most **128 MiB**, copied and hashed into
a private workspace before analysis. No in-place source modification.
- One queued heavy operation per provider instance; Binwalk one thread and
Unblob one worker. No automatic parallel extraction engines.
- **120-second** deadline; inherited **1 GiB per-process address-space** limit,
120 CPU seconds and zero core-dump allowance through `prlimit`.
- JSON reports: at most **8 MiB**; oversized reports fail without a truncated
success. Diagnostic output is bounded with exact captured byte counts.
- Default staging budget: **256 MiB** and **10,000 entries**, configurable using
`max_output_bytes` / `max_output_files` (CLI `--max-output-bytes` /
`--max-output-files`). Depth defaults to 3, allowed range 1–10.
Staging budgets are polled and rechecked after exit; short-lived overshoot is
possible. The inherited file-size limit bounds each file, not total storage;
address-space limits are per process, not an aggregate memory quota. Worker
count is not CPU affinity for every external extractor. These limits do not
claim cgroup/container containment. Cancellation cleans the owned process group
before removing its workspace; uncertain cleanup retains resources and blocks
further operations in that provider instance.
## Verification
Generate small public, source-derived fixtures into ignored `_reference/`:
```sh
npm run fixtures:firmware
npm run verify:firmware
```
Fixture generation requires existing Python 3 and a host C compiler (`cc`);
override with `REA_FIRMWARE_PYTHON` / `REA_FIRMWARE_CC`. The independent oracle
records gzip offset/length and extracted file hashes. Real verification checks
CLI/MCP, full-file and selected-range extraction, unknown chunks, depth limits,
content parity and unchanged source bytes. No binary fixture is committed.
Additional lanes are explicit, so the base lane stays small:
```sh
# Existing mke2fs and debugfs required; put debugfs on the scoped PATH.
REA_FIRMWARE_VERIFY_EXT4=1 npm run fixtures:firmware
REA_FIRMWARE_VERIFY_EXT4=1 npm run verify:firmware
# Existing Ghidra 12.1.4 and Java required.
REA_FIRMWARE_VERIFY_GHIDRA=1 npm run verify:firmware
```
`REA_FIRMWARE_MKE2FS_COMMAND` selects the optional ext4 fixture producer.
`REA_FIRMWARE_FIXTURE_ROOT` selects generated data storage.
`REA_FIRMWARE_TEST_ENTRYPOINT` can select an isolated installed npm package's
`scripts/rea.mjs` to verify the packaged CLI/MCP workflow. Real coverage here is
generated gzip/USTAR/ext4 with a Linux x64 ELF child; SquashFS, UBIFS, proprietary
firmware, other hosts and architectures remain unverified.
+27 -7
View File
@@ -10,7 +10,7 @@
"skill_version": "24"
},
"tools": {
"total": 123,
"total": 125,
"families": [
{
"id": "direct",
@@ -121,6 +121,12 @@
"verify_managed_native_boundaries"
]
},
{
"id": "firmware",
"surface": "firmware-provider",
"count": 2,
"tools": ["extract_firmware", "inspect_firmware_regions"]
},
{
"id": "android",
"surface": "android-provider",
@@ -215,6 +221,12 @@
]
},
"providers": [
{
"id": "binwalk",
"name": "Binwalk",
"version": "3.1.0",
"capabilities": ["inspect_firmware_regions"]
},
{
"id": "ghidra",
"name": "Ghidra",
@@ -418,6 +430,12 @@
"list_javascript_runtime_targets",
"observe_javascript_runtime"
]
},
{
"id": "unblob",
"name": "Unblob",
"version": "26.6.4",
"capabilities": ["extract_firmware"]
}
],
"setup_clients": [
@@ -495,7 +513,7 @@
}
],
"cli": {
"primary_count": 78,
"primary_count": 80,
"commands": [
"address-to-file-offset",
"analyze",
@@ -526,6 +544,7 @@
"evidence-export",
"evidence-import",
"extract-artifact",
"extract-firmware",
"function",
"import-managed-reconstruction",
"import-reference-source",
@@ -536,6 +555,7 @@
"inspect-artifact",
"inspect-asset-catalog",
"inspect-electron-page",
"inspect-firmware-regions",
"inspect-keyed-archive",
"inspect-macho",
"inspect-managed-artifact",
@@ -585,15 +605,15 @@
},
"runtime_catalog": {
"counts": {
"cli_commands": 78,
"mcp_tools": 123,
"cli_commands": 80,
"mcp_tools": 125,
"mcp_prompts": 6
},
"digests": {
"tools_sha256": "6e60913fad4c67f3a12aa8bf5113971ce6a57e9492ce402c8cd9e539d6fde01f",
"tools_sha256": "45a8aea00dba2a8ad8563b4943cd177d07b14889bc9b823bc39b89b6ddb26d2c",
"prompts_sha256": "ef2de8814390b340623c32001610aa2709d90bed51537b6e562ab3abf144ab4d",
"combined_sha256": "d5fbda3a283916ab3489a62fe88aeb89da8b0a7b19c248e3742c970b1b51600e",
"providers_sha256": "e60c6e949f499fe8a015a20ff36752516ce57edcb1ce355f0308361c8d322dd6"
"combined_sha256": "297aced29e7ce5e845c505d5f2e9e2dee42b7ed3e628f088f424fe8088c9e8dc",
"providers_sha256": "dfba0ee59f12c82cdfb5320e7e0d70a114e533b2cb88afd15db9b9a4a175c9b0"
}
}
}
+12
View File
@@ -311,3 +311,15 @@ allows a host-permission-boundary-only result and explicitly reports
Both commands reject a changed executable digest and clean up the fixture
process and helper. These lanes require an interactive macOS desktop. See [native investigation](native-investigation.md)
for the exact ABI, authority, graph and observation boundaries.
### Firmware adapters
`npm run fixtures:firmware` uses existing Python 3 and a host C compiler to make
an ignored gzip/USTAR firmware fixture and independent offset/hash oracle.
`npm run verify:firmware` requires caller-supplied Binwalk 3.1.0, Unblob 26.6.4
and util-linux prlimit on Linux. It verifies CLI/MCP parity, selected ranges,
unknown chunks, depth limits and extracted child digests. The optional
`REA_FIRMWARE_VERIFY_EXT4=1` lane requires existing mke2fs/debugfs; the separate
`REA_FIRMWARE_VERIFY_GHIDRA=1` lane checks a selected host ELF through real Ghidra.
Neither optional toolchain is a base-lane prerequisite. See
[firmware analysis](firmware-analysis.md) for limits and unverified formats.
@@ -1,5 +1,5 @@
{
"manifest_id": "ecm_42bc9f06a00e9c510cb57605d7f1cde613e2762ea0c59bf3915e032147a7c493",
"manifest_id": "ecm_18ee96f3c7c08df6a29b2075b8dadf6e7053e88c52a65cecb35aea374112cbc6",
"verifier": {
"id": "rea-managed-conformance",
"version": "1"
@@ -13,7 +13,7 @@
"skill_digests": [
{
"skill_id": "reverse-engineer-anything",
"sha256": "ace0a167b9161d9d96c9f5cd65ff0bb39e483c3426e6cc86aac5c7c9893c31cb"
"sha256": "f72e03be44a47ba176e6aac47c85e930fe1fe5723b650dacbd5873f28a3b3f8f"
}
],
"claims": [
+2
View File
@@ -136,6 +136,8 @@
"verify:javascript:digests": "npm run build:cached && node scripts/verify-large-javascript-digests.mjs",
"verify:application-workflows": "npm run build:cached && node scripts/verify-local-application-workflows.mjs",
"verify:managed": "npm run build:cached && node scripts/verify-managed-conformance.mjs",
"verify:firmware": "npm run build:cached && node scripts/verify-real-firmware.mjs",
"fixtures:firmware": "node scripts/generate-firmware-fixtures.mjs",
"verify:android": "npm run build:cached && node scripts/verify-real-android.mjs",
"fixtures:android": "node scripts/fetch-android-fixtures.mjs",
"verify:agent": "npm run build:cached && node scripts/verify-agent-experience.mjs",
+42
View File
@@ -0,0 +1,42 @@
import { execFile } from "node:child_process";
import { resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
const exec = promisify(execFile);
const root = resolve(
process.env.REA_FIRMWARE_FIXTURE_ROOT ??
"_reference/firmware-integration/generated",
);
for (const [command, args, purpose] of [
[
process.env.REA_FIRMWARE_PYTHON ?? "python3",
["--version"],
"Python 3 standard-library fixture generator",
],
[
process.env.REA_FIRMWARE_CC ?? "cc",
["--version"],
"host ELF fixture compiler",
],
]) {
try {
await exec(command, args);
} catch (cause) {
throw new Error(`fixtures:firmware requires ${purpose}: ${command}`, {
cause,
});
}
}
const ext4 =
process.env.REA_FIRMWARE_VERIFY_EXT4 === "1"
? [process.env.REA_FIRMWARE_MKE2FS_COMMAND ?? "mke2fs"]
: [];
for (const command of ext4) await exec(command, ["-V"]);
const generated = await exec(process.env.REA_FIRMWARE_PYTHON ?? "python3", [
fileURLToPath(new URL("./lib/generate-firmware-fixture.py", import.meta.url)),
root,
process.env.REA_FIRMWARE_CC ?? "cc",
...ext4,
]);
console.log(generated.stdout.trim());
+17
View File
@@ -31,6 +31,11 @@ export const toolFamilyCatalog = (sources) => {
...sources.managedWorkflowContracts.MANAGED_WORKFLOW_TOOL_CONTRACTS,
],
},
{
id: "firmware",
surface: "firmware-provider",
contracts: sources.firmwareContracts.FIRMWARE_TOOL_CONTRACTS,
},
{
id: "android",
surface: "android-provider",
@@ -118,6 +123,18 @@ export const providerCatalog = (sources) => {
contracts:
sources.managedWorkflowContracts.MANAGED_WORKFLOW_TOOL_CONTRACTS,
},
{
identity: sources.firmwareProvider.BINWALK_PROVIDER_IDENTITY,
contracts: sources.firmwareContracts.FIRMWARE_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_firmware_regions",
),
},
{
identity: sources.firmwareProvider.UNBLOB_PROVIDER_IDENTITY,
contracts: sources.firmwareContracts.FIRMWARE_TOOL_CONTRACTS.filter(
({ name }) => name === "extract_firmware",
),
},
{
identity: sources.androidProvider.JADX_PROVIDER_IDENTITY,
contracts: sources.androidContracts.ANDROID_TOOL_CONTRACTS,
+2
View File
@@ -43,6 +43,8 @@ const SOURCE_PATHS = {
nativeContracts: "dist/contracts/nativeToolContracts.js",
artifactContracts: "dist/contracts/artifactToolContracts.js",
managedContracts: "dist/contracts/managedToolContracts.js",
firmwareContracts: "dist/contracts/firmwareToolContracts.js",
firmwareProvider: "dist/firmware/FirmwareRelease.js",
androidContracts: "dist/contracts/androidToolContracts.js",
androidProvider: "dist/android/JadxRelease.js",
managedWorkflowContracts: "dist/contracts/managedWorkflowToolContracts.js",
+43
View File
@@ -0,0 +1,43 @@
"""Generate firmware bytes using standard format producers, with an independent oracle."""
import gzip
import hashlib
import io
import json
from pathlib import Path
import subprocess
import sys
import tarfile
root = Path(sys.argv[1]).resolve()
root.mkdir(parents=True, exist_ok=True)
source = root / "probe.c"
source.write_text("int firmware_probe(int x){return x*7+3;} int main(void){return firmware_probe(4);}\n")
subprocess.run([sys.argv[2], "-O0", "-g", "-fno-inline", str(source), "-o", str(root / "probe")], check=True)
payloads = {"etc/rea.conf": b"rea_fixture=true\n", "usr/bin/probe": (root / "probe").read_bytes()}
buffer = io.BytesIO()
with tarfile.open(fileobj=buffer, mode="w", format=tarfile.USTAR_FORMAT) as archive:
for name, data in payloads.items():
info = tarfile.TarInfo(name)
info.size = len(data)
info.mode = 0o644
info.mtime = 0
archive.addfile(info, io.BytesIO(data))
compressed = gzip.compress(buffer.getvalue(), mtime=0)
image = b"REA firmware fixture\n".ljust(64, b"\xff") + compressed + b"\x00" * 128
(root / "firmware.bin").write_bytes(image)
(root / "oracle.json").write_text(json.dumps({
"generator": "Python tarfile USTAR + gzip; host cc; no target execution",
"offset": 64, "length": len(compressed), "size": len(image),
"sha256": hashlib.sha256(image).hexdigest(),
"files": {name: {"sha256": hashlib.sha256(data).hexdigest(), "size": len(data)} for name, data in payloads.items()},
}, indent=2) + "\n")
print(root)
if len(sys.argv) > 3:
tree = root / "ext4-root"
tree.mkdir(exist_ok=True)
for name, data in payloads.items():
target = tree / name
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(data)
subprocess.run([sys.argv[3], "-q", "-t", "ext4", "-F", "-d", str(tree), str(root / "ext4.bin"), "8192"], check=True)
+254
View File
@@ -0,0 +1,254 @@
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import {
access,
mkdir,
mkdtemp,
readFile,
rm,
symlink,
} from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { firmwareResultSchemas } from "../dist/domain/firmwareAnalysis.js";
import { parseEvidence } from "../dist/domain/evidence.js";
import { hashFirmwareFile } from "../dist/firmware/FirmwareFiles.js";
const exec = promisify(execFile);
const repository = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const entrypoint = resolve(
process.env.REA_FIRMWARE_TEST_ENTRYPOINT ??
join(repository, "scripts/rea.mjs"),
);
if (process.platform !== "linux")
throw new Error(
"verify:firmware requires Linux; other hosts are not verified",
);
for (const variable of ["REA_BINWALK_COMMAND", "REA_UNBLOB_COMMAND"]) {
const path = process.env[variable];
if (path === undefined)
throw new Error(
`verify:firmware requires ${variable}; see docs/firmware-analysis.md`,
);
await access(path);
}
const fixtureRoot = resolve(
process.env.REA_FIRMWARE_FIXTURE_ROOT ??
join(repository, "_reference/firmware-integration/generated"),
);
await access(join(fixtureRoot, "oracle.json")).catch((cause) => {
throw new Error(
"verify:firmware requires generated fixtures; run npm run fixtures:firmware",
{ cause },
);
});
const oracle = JSON.parse(
await readFile(join(fixtureRoot, "oracle.json"), "utf8"),
);
const path = join(fixtureRoot, "firmware.bin");
assert.equal(await hashFirmwareFile(path), oracle.sha256);
const root = await mkdtemp(join(tmpdir(), "rea-real-firmware-"));
const environment = Object.fromEntries(
Object.entries(process.env).filter(([, value]) => value !== undefined),
);
const client = new Client({ name: "rea-real-firmware-verifier", version: "1" });
const cli = async (args, env = environment) => {
const response = await exec(
process.execPath,
[entrypoint, ...args, "--format", "json"],
{
env,
cwd: repository,
timeout: 150_000,
maxBuffer: 16 * 1024 * 1024,
},
);
return parseEvidence(JSON.parse(response.stdout));
};
const call = async (name, args) => {
const response = await client.callTool(
{ name, arguments: args },
{ timeout: 150_000 },
);
assert.notEqual(response.isError, true, JSON.stringify(response));
assert.ok(response.structuredContent !== undefined);
return response.structuredContent;
};
const verifyRegions = (evidence) => {
assert.equal(evidence.subject.digest.sha256, oracle.sha256);
const result = firmwareResultSchemas.inspect_firmware_regions.parse(
evidence.normalized_result,
);
const gzip = result.regions.find(
(region) => region.signature === "gzip" && region.offset === oracle.offset,
);
assert.ok(
gzip !== undefined,
"Binwalk must find the independently generated gzip offset",
);
assert.equal(gzip.reported_size, oracle.length);
assert.equal(gzip.size_basis, "provider_reported_validation_unknown");
return result.regions.map(({ provider_id, ...region }) => region);
};
const verifyFiles = async (evidence) => {
const result = firmwareResultSchemas.extract_firmware.parse(
evidence.normalized_result,
);
assert.ok(result.chunks.some((chunk) => chunk.handler === "gzip"));
for (const [suffix, expected] of Object.entries(oracle.files)) {
const file = result.files.find((candidate) =>
candidate.relative_path.endsWith(`/${suffix}`),
);
assert.ok(file !== undefined, `Missing extracted ${suffix}`);
assert.equal(file.sha256, expected.sha256);
assert.equal(file.size, expected.size);
assert.equal(await hashFirmwareFile(file.path), expected.sha256);
assert.equal(file.original_file_range, null);
assert.equal(file.runtime_address, null);
}
return result;
};
try {
const inspected = await cli(["inspect-firmware-regions", path]);
const cliRegions = verifyRegions(inspected);
const extracted = await cli([
"extract-firmware",
path,
join(root, "cli-output"),
]);
const cliFiles = await verifyFiles(extracted);
assert.equal(cliFiles.coverage, "partial");
assert.ok(
cliFiles.chunks.some(
(chunk) =>
chunk.handler === null &&
chunk.range.offset === 0 &&
chunk.range.length === 64,
),
);
await client.connect(
new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
cwd: repository,
env: environment,
stderr: "pipe",
}),
{ timeout: 30_000 },
);
const inspectedMcp = await call("inspect_firmware_regions", { path });
const mcpEvidence = parseEvidence(inspectedMcp.evidence);
assert.deepEqual(verifyRegions(mcpEvidence), cliRegions);
const extractedMcp = await call("extract_firmware", {
path,
output_directory: join(root, "mcp-output"),
});
const mcpFiles = await verifyFiles(parseEvidence(extractedMcp.evidence));
const contentIdentity = (result) =>
result.files.map(({ relative_path, sha256, size }) => ({
relative_path,
sha256,
size,
}));
assert.deepEqual(contentIdentity(mcpFiles), contentIdentity(cliFiles));
const selected = await call("extract_firmware", {
path,
output_directory: join(root, "selected-output"),
range: { offset: oracle.offset, length: oracle.length },
});
const selectedFiles = await verifyFiles(parseEvidence(selected.evidence));
assert.equal(selectedFiles.selection.offset, oracle.offset);
assert.equal(selectedFiles.selection.length, oracle.length);
const depth = await call("extract_firmware", {
path,
output_directory: join(root, "depth-output"),
max_depth: 1,
});
const depthResult = firmwareResultSchemas.extract_firmware.parse(
depth.result,
);
assert.equal(depthResult.coverage, "partial");
assert.ok(depthResult.depth_limited_paths.length > 0);
console.log(
"PASS real Binwalk/Unblob: CLI, MCP, independent offsets/digests, range extraction, unknown chunks and depth limits",
);
if (process.env.REA_FIRMWARE_VERIFY_EXT4 === "1") {
const ext4 = join(fixtureRoot, "ext4.bin");
await access(ext4);
const response = await call("extract_firmware", {
path: ext4,
output_directory: join(root, "ext4-output"),
});
const result = firmwareResultSchemas.extract_firmware.parse(
response.result,
);
assert.ok(result.chunks.some((chunk) => chunk.handler === "extfs"));
assert.equal(result.coverage, "complete");
for (const [suffix, expected] of Object.entries(oracle.files))
assert.equal(
result.files.find((file) => file.relative_path.endsWith(`/${suffix}`))
?.sha256,
expected.sha256,
);
console.log("PASS real ext4 extraction with debugfs; no mounts");
// Preserve the process-ownership utility while withholding all extractors.
const withoutExtractors = join(root, "without-extractors");
await access("/bin/ps");
await mkdir(withoutExtractors);
await symlink("/bin/ps", join(withoutExtractors, "ps"));
const missing = await cli(
["extract-firmware", ext4, join(root, "missing-extractor-output")],
{ ...environment, PATH: withoutExtractors },
);
const partial = firmwareResultSchemas.extract_firmware.parse(
missing.normalized_result,
);
assert.equal(partial.coverage, "partial");
assert.ok(JSON.stringify(partial.diagnostics).includes("debugfs"));
console.log(
"PASS real missing-extractor diagnostics retained despite Unblob exit 1",
);
}
if (process.env.REA_FIRMWARE_VERIFY_GHIDRA === "1") {
if (process.env.GHIDRA_INSTALL_DIR === undefined)
throw new Error(
"Firmware Ghidra handoff requires GHIDRA_INSTALL_DIR and Java; see docs/ghidra.md",
);
const child = mcpFiles.files.find((file) =>
file.relative_path.endsWith("/usr/bin/probe"),
);
assert.ok(child !== undefined);
await call("open_binary", { path: child.path, provider_id: "ghidra" });
const dossier = await call("analyze_function", {
procedure: "firmware_probe",
});
const evidence = parseEvidence(dossier.evidence);
assert.equal(
evidence.subject.digest.sha256,
oracle.files["usr/bin/probe"].sha256,
);
const pseudocode = await call("procedure_pseudo_code", {
procedure: "firmware_probe",
});
assert.equal(parseEvidence(pseudocode.evidence).provider.id, "ghidra");
assert.ok(
JSON.stringify(evidence.normalized_result).includes("firmware_probe"),
);
await call("close_binary", {});
console.log(
"PASS real Ghidra handoff of the selected extracted host ELF, bound to its digest",
);
}
assert.equal(
await hashFirmwareFile(path),
oracle.sha256,
"Original firmware bytes must remain unchanged",
);
} finally {
await client.close();
await rm(root, { recursive: true, force: true });
}
+2 -2
View File
@@ -3,8 +3,8 @@ name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, and browser applications with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "24"
tool_count: 123
catalog_digest: "d5fbda3a283916ab3489a62fe88aeb89da8b0a7b19c248e3742c970b1b51600e"
tool_count: 125
catalog_digest: "297aced29e7ce5e845c505d5f2e9e2dee42b7ed3e628f088f424fe8088c9e8dc"
---
# REA
@@ -55,6 +55,27 @@ const entry = (
};
describe("capability inventory: provider status", () => {
it("reports firmware inspection and extraction prerequisites independently", () => {
const policy = {
processCaptureEnabled: true,
firmwareInspectionEnabled: true,
};
expect(entry("inspect_firmware_regions", status(), policy)).toMatchObject({
available: true,
surface: "firmware-provider",
});
expect(entry("extract_firmware", status(), policy)).toMatchObject({
available: false,
reason: "provider_missing",
remediation: expect.stringContaining("REA_UNBLOB_COMMAND"),
});
expect(
entry("extract_firmware", status(), {
processCaptureEnabled: true,
firmwareExtractionEnabled: true,
}),
).toMatchObject({ available: true });
});
it("keeps Android APK operations target-free and reports their explicit engine prerequisite", () => {
expect(
entry("inspect_android_package", status(), {
+15
View File
@@ -28,6 +28,8 @@ type ToolAvailabilityReason = "available" | ToolUnavailabilityReason;
type ProviderDescriptor = ProviderCapability;
export type AvailabilityPolicy = {
readonly processCaptureEnabled: boolean;
readonly firmwareInspectionEnabled?: boolean;
readonly firmwareExtractionEnabled?: boolean;
readonly androidAnalysisEnabled?: boolean;
readonly browserObservationEnabled?: boolean;
readonly browserScenarioEnabled?: boolean;
@@ -206,6 +208,19 @@ const workflowAvailabilityFor = ({
kind,
policy,
}: AvailabilityContext): Availability | null => {
if (kind === "firmware-provider") {
const enabled =
name === "inspect_firmware_regions"
? policy.firmwareInspectionEnabled
: policy.firmwareExtractionEnabled;
return enabled === true
? { reason: "available", remediation: null }
: {
reason: "provider_missing",
remediation:
"On Linux, provide an absolute REA_BINWALK_COMMAND (Binwalk 3.1.0) or REA_UNBLOB_COMMAND (Unblob 26.6.4) path and util-linux prlimit. Extraction also requires the selected format’s external extractor.",
};
}
if (kind === "android-provider")
return policy.androidAnalysisEnabled === true
? { reason: "available", remediation: null }
+15
View File
@@ -0,0 +1,15 @@
import type {
AnalysisExecution,
ExecutionOptions,
} from "./AnalysisProvider.js";
import type { FirmwareRequest } from "../domain/firmwareAnalysis.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { Result } from "../domain/result.js";
/** File-oriented firmware engines; executable admission remains a separate operation. */
export interface FirmwareAnalysisPort {
execute(
request: FirmwareRequest,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>;
}
@@ -0,0 +1,54 @@
import type { ExecutionOptions } from "./AnalysisProvider.js";
import type { FirmwareAnalysisPort } from "./FirmwareAnalysisPort.js";
import {
firmwareRequestSchema,
type FirmwareOperation,
} from "../domain/firmwareAnalysis.js";
import {
AnalysisCancelledError,
AnalysisInputError,
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { createEvidence, type Evidence } from "../domain/evidence.js";
import { jsonObjectSchema } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
/** Shared CLI/MCP validation and Evidence composition for arbitrary firmware files. */
export class FirmwareAnalysisService {
constructor(readonly provider: FirmwareAnalysisPort) {}
/** Execute an explicit inspection or extraction and preserve its producing identity. */
async execute(
operation: FirmwareOperation,
input: unknown,
options?: ExecutionOptions,
): Promise<Result<Evidence, AnalysisError>> {
if (options?.signal?.aborted === true)
return err(new AnalysisCancelledError(operation));
const parsed = firmwareRequestSchema.safeParse({ operation, input });
if (!parsed.success)
return err(new AnalysisInputError(operation, { cause: parsed.error }));
const executed = await this.provider.execute(parsed.data, options);
if (!executed.ok) return executed;
const execution = executed.value;
if (execution.subject === null)
return err(
new AnalysisOutputError(
operation,
"Firmware execution did not bind its source artifact",
),
);
return ok(
createEvidence(execution.subject, execution.provider, {
operation,
parameters: jsonObjectSchema.parse(parsed.data.input),
result: execution.result,
rawResult: execution.rawResult,
locations: execution.locations,
limitations: execution.limitations,
confidence: "observed",
}),
);
}
}
+2
View File
@@ -7,6 +7,7 @@ import { registerCoreAnalysisCommands } from "./cli/coreAnalysisCommands.js";
import { registerUtilityCommands } from "./cli/utilityCommands.js";
import { registerArtifactCommands } from "./cli/artifactCommands.js";
import { registerManagedCommands } from "./cli/managedCommands.js";
import { registerFirmwareCommands } from "./cli/firmwareCommands.js";
import { registerAndroidCommands } from "./cli/androidCommands.js";
import { registerEvidenceCommands } from "./cliEvidenceCommands.js";
import { registerProcessCommands } from "./cli/processCommands.js";
@@ -49,6 +50,7 @@ export const createCli = (
registerArtifactCommands(cli, logger);
registerManagedCommands(cli, logger);
registerAndroidCommands(cli, logger, environment);
registerFirmwareCommands(cli, logger, environment);
registerEvidenceCommands(cli, logger);
registerProcessCommands(cli, logger, environment);
registerBrowserCommands(cli, logger);
+100
View File
@@ -0,0 +1,100 @@
import { z } from "incur";
import { FirmwareAnalysisService } from "../application/FirmwareAnalysisService.js";
import { FirmwareProvider } from "../firmware/FirmwareProvider.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { FirmwareOperation } from "../domain/firmwareAnalysis.js";
import type { Logger } from "../logger.js";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
/** One-shot firmware CLI commands backed by the same workflows as MCP. */
export const registerFirmwareCommands = (
cli: CliInstance,
logger: Logger,
environment: Readonly<Record<string, string | undefined>>,
): void => {
const service = new FirmwareAnalysisService(
new FirmwareProvider(environment),
);
const execute = (
name: string,
operation: FirmwareOperation,
input: unknown,
) =>
withCommandCancellation((signal) =>
logCliCommand(logger, name, async () => {
const result = await service.execute(operation, input, { signal });
return result.ok ? result.value : projectAnalysisError(result.error);
}),
);
const path = z.string().describe("Local firmware path; never executed");
cli.command(CLI_COMMANDS.inspectFirmwareRegions, {
description:
"Inspect firmware signature candidates without extracting them",
args: z.object({ path }),
run: ({ args }) =>
execute(
CLI_COMMANDS.inspectFirmwareRegions,
"inspect_firmware_regions",
args,
),
});
cli.command(CLI_COMMANDS.extractFirmware, {
description:
"Extract selected firmware bytes and return verified files and provenance",
args: z.object({
path,
outputDirectory: z
.string()
.describe("Absolute absent output directory; parent must exist"),
}),
options: z.object({
offset: z
.number()
.int()
.nonnegative()
.optional()
.describe("Selected byte offset; requires length"),
length: z
.number()
.int()
.positive()
.optional()
.describe("Selected byte length; requires offset"),
maxDepth: z
.number()
.int()
.optional()
.describe("Extraction depth, 1–10; default 3"),
maxOutputBytes: z
.number()
.int()
.optional()
.describe("Staging byte budget; default 256 MiB, maximum 512 MiB"),
maxOutputFiles: z
.number()
.int()
.optional()
.describe("Staging entry budget; default 10000"),
}),
run: ({ args, options }) =>
execute(CLI_COMMANDS.extractFirmware, "extract_firmware", {
path: args.path,
output_directory: args.outputDirectory,
...(options.offset === undefined && options.length === undefined
? {}
: { range: { offset: options.offset, length: options.length } }),
...(options.maxDepth === undefined
? {}
: { max_depth: options.maxDepth }),
...(options.maxOutputBytes === undefined
? {}
: { max_output_bytes: options.maxOutputBytes }),
...(options.maxOutputFiles === undefined
? {}
: { max_output_files: options.maxOutputFiles }),
}),
});
};
+2
View File
@@ -32,6 +32,8 @@ export const CLI_COMMANDS = Object.freeze({
resolveNativeCallTargets: "resolve-native-call-targets",
inspectAssetCatalog: "inspect-asset-catalog",
inspectManagedArtifact: "inspect-managed-artifact",
inspectFirmwareRegions: "inspect-firmware-regions",
extractFirmware: "extract-firmware",
inspectAndroidPackage: "inspect-android-package",
searchAndroidClasses: "search-android-classes",
inspectAndroidClass: "inspect-android-class",
+2
View File
@@ -8,6 +8,7 @@ import { OFFICIAL_TOOL_CONTRACTS } from "./officialToolContracts.js";
import { SESSION_TOOL_CONTRACTS } from "./sessionToolContracts.js";
import { NATIVE_TOOL_CONTRACTS } from "./nativeToolContracts.js";
import { ARTIFACT_TOOL_CONTRACTS } from "./artifactToolContracts.js";
import { FIRMWARE_TOOL_CONTRACTS } from "./firmwareToolContracts.js";
import { ANDROID_TOOL_CONTRACTS } from "./androidToolContracts.js";
import { MANAGED_TOOL_CONTRACTS } from "./managedToolContracts.js";
import { MANAGED_WORKFLOW_TOOL_CONTRACTS } from "./managedWorkflowToolContracts.js";
@@ -75,6 +76,7 @@ describe("tool contract surface", () => {
...NATIVE_TOOL_CONTRACTS,
...ARTIFACT_TOOL_CONTRACTS,
...ANDROID_TOOL_CONTRACTS,
...FIRMWARE_TOOL_CONTRACTS,
...MANAGED_TOOL_CONTRACTS,
...MANAGED_WORKFLOW_TOOL_CONTRACTS,
...BROWSER_TOOL_CONTRACTS,
+47
View File
@@ -0,0 +1,47 @@
import {
firmwareInputSchemas,
firmwareResultSchemas,
} from "../domain/firmwareAnalysis.js";
import type { ToolContract } from "./toolContractTypes.js";
import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js";
import { toolContractMetadata } from "./toolEffects.js";
/** Explicit local firmware observations and extraction, backed by caller-supplied tools. */
export const FIRMWARE_TOOL_CONTRACTS = [
{
name: "inspect_firmware_regions",
...toolContractMetadata("inspect_firmware_regions"),
kind: "firmware-provider",
description:
"Inspect signature candidates in a local firmware file with Binwalk 3.1.0. Returns offsets, reported lengths, descriptions and raw confidence values. Length validation and runtime addresses remain unknown. No extraction or target execution. Linux, caller-supplied tool and util-linux prlimit required; 128 MiB input limit, one worker and 120-second deadline.",
inputSchema: firmwareInputSchemas.inspect_firmware_regions,
outputSchema: evidenceResultOf(
firmwareResultSchemas.inspect_firmware_regions,
),
examples: [
{
title: "Inspect firmware regions",
input: { path: "/tmp/firmware.bin" },
},
],
},
{
name: "extract_firmware",
...toolContractMetadata("extract_firmware"),
kind: "firmware-provider",
description:
"Extract caller-selected firmware bytes with Unblob 26.6.4 into an absent output directory. Returns verified child paths/digests, task derivations, unknown chunks and extraction diagnostics inline. Preserves the default sandbox; publishes regular files only. Linux, caller-supplied tool, format-specific extractors and util-linux prlimit required. No mounts or extracted-code execution. Select a returned native file with open_binary for Ghidra analysis; decompressed offsets are not original-file addresses.",
inputSchema: firmwareInputSchemas.extract_firmware,
outputSchema: evidenceResultOf(firmwareResultSchemas.extract_firmware),
examples: [
{
title: "Extract selected firmware",
input: {
path: "/tmp/firmware.bin",
output_directory: "/tmp/firmware-output",
range: { offset: 64, length: 4096 },
},
},
],
},
] as const satisfies readonly ToolContract[];
+1
View File
@@ -25,6 +25,7 @@ export const TOOL_KINDS = [
"artifact-provider",
"managed-provider",
"android-provider",
"firmware-provider",
"browser-provider",
"electron-provider",
"runtime-provider",
+4
View File
@@ -6,6 +6,7 @@ import { SESSION_TOOL_CONTRACTS } from "./sessionToolContracts.js";
import { TOOL_CONTRACTS } from "./toolContracts.js";
import { ARTIFACT_TOOL_CONTRACTS } from "./artifactToolContracts.js";
import { MANAGED_TOOL_CONTRACTS } from "./managedToolContracts.js";
import { FIRMWARE_TOOL_CONTRACTS } from "./firmwareToolContracts.js";
import { ANDROID_TOOL_CONTRACTS } from "./androidToolContracts.js";
import { MANAGED_WORKFLOW_TOOL_CONTRACTS } from "./managedWorkflowToolContracts.js";
import { NATIVE_TOOL_CONTRACTS } from "./nativeToolContracts.js";
@@ -21,6 +22,7 @@ const GROUPS = {
artifact: ARTIFACT_TOOL_CONTRACTS,
managed: MANAGED_TOOL_CONTRACTS,
android: ANDROID_TOOL_CONTRACTS,
firmware: FIRMWARE_TOOL_CONTRACTS,
managed_workflow: MANAGED_WORKFLOW_TOOL_CONTRACTS,
browser_provider: BROWSER_PROVIDER_TOOL_CONTRACTS,
electron: ELECTRON_TOOL_CONTRACTS,
@@ -93,6 +95,8 @@ const PUBLISHED_TOOL_NAME_FLOOR = [
"inspect_electron_page",
"inspect_macho",
"inspect_managed_artifact",
"inspect_firmware_regions",
"extract_firmware",
"inspect_android_package",
"search_android_classes",
"inspect_android_class",
+2
View File
@@ -1,6 +1,7 @@
import { NATIVE_TOOL_CONTRACTS } from "./nativeToolContracts.js";
import { ARTIFACT_TOOL_CONTRACTS } from "./artifactToolContracts.js";
import { MANAGED_TOOL_CONTRACTS } from "./managedToolContracts.js";
import { FIRMWARE_TOOL_CONTRACTS } from "./firmwareToolContracts.js";
import { ANDROID_TOOL_CONTRACTS } from "./androidToolContracts.js";
import { MANAGED_WORKFLOW_TOOL_CONTRACTS } from "./managedWorkflowToolContracts.js";
import { BROWSER_PROVIDER_TOOL_CONTRACTS } from "./browserProviderToolContracts.js";
@@ -21,6 +22,7 @@ export const TOOL_CONTRACTS = [
...ARTIFACT_TOOL_CONTRACTS,
...MANAGED_TOOL_CONTRACTS,
...ANDROID_TOOL_CONTRACTS,
...FIRMWARE_TOOL_CONTRACTS,
...MANAGED_WORKFLOW_TOOL_CONTRACTS,
...BROWSER_PROVIDER_TOOL_CONTRACTS,
...ELECTRON_TOOL_CONTRACTS,
+11
View File
@@ -41,6 +41,17 @@ const sessionEvidence = effects({ mutatesSession: true });
/** Explicit effect audit for every public tool. */
export const TOOL_EFFECTS: Readonly<Record<string, ToolEffects>> = {
inspect_firmware_regions: effects({
mutatesSession: true,
launchesProcess: true,
writesFilesystem: true,
}),
extract_firmware: effects({
mutatesSession: true,
launchesProcess: true,
writesFilesystem: true,
idempotent: false,
}),
inspect_android_package: effects({
mutatesSession: true,
launchesProcess: true,
@@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest";
import {
AnalysisCapabilityUnavailableError,
AnalysisInputError,
AnalysisOutputError,
} from "./analysisErrorCore.js";
import { ArtifactOperationError } from "./artifactOperationError.js";
import { BinaryTargetError } from "./configurationErrors.js";
@@ -160,6 +161,23 @@ describe("analysis error projection: provider failures", () => {
});
describe("analysis error projection: caller contract", () => {
it("preserves the reported output constraint without exposing its internal cause", () => {
const projected = projectAnalysisError(
new AnalysisOutputError(
"extract_firmware",
"Report exceeded the declared byte budget",
{ cause: new Error("private internal cause") },
),
);
expect(projected).toMatchObject({
code: "unreadable_output",
details: {
operation: "extract_firmware",
reason: "Report exceeded the declared byte budget",
},
});
expect(JSON.stringify(projected)).not.toContain("private internal cause");
});
it("maps representative failures without exposing causes", () => {
const secretCause = new Error("secret-token");
const projected = [
+3
View File
@@ -10,6 +10,7 @@ import {
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
AnalysisInputError,
AnalysisOutputError,
AnalysisTimeoutError,
} from "./analysisErrorCore.js";
import { ArtifactOperationError } from "./artifactOperationError.js";
@@ -176,6 +177,8 @@ const errorDetails = (
const requestErrorDetails = (
error: AnalysisError,
): Readonly<Record<string, JsonValue>> | undefined => {
if (error instanceof AnalysisOutputError)
return { operation: error.operation, reason: error.reason };
if (error instanceof AnalysisInputError && error.issues.length > 0)
return {
operation: error.operation,
+145
View File
@@ -0,0 +1,145 @@
import { z } from "zod";
import { digestSchema } from "./digests.js";
import { jsonValueSchema } from "./jsonValue.js";
const bytes = z.number().int().min(0).max(Number.MAX_SAFE_INTEGER);
const range = z.strictObject({ offset: bytes, length: bytes.positive() });
const target = {
path: z.string().min(1).describe("Local firmware file; never executed"),
};
/** Explicit firmware inspection and extraction intent, independent of engines. */
export const firmwareInputSchemas = {
inspect_firmware_regions: z.strictObject(target),
extract_firmware: z.strictObject({
...target,
output_directory: z
.string()
.min(1)
.describe("Absolute, absent output directory; parent must exist"),
range: range
.optional()
.describe("Explicit file-byte interval; omit to extract the whole file"),
max_depth: z
.number()
.int()
.min(1)
.max(10)
.default(3)
.describe("Extraction depth; reaching it yields partial analysis"),
max_output_bytes: z
.number()
.int()
.min(1)
.max(512 * 1024 * 1024)
.default(256 * 1024 * 1024)
.describe(
"Staging file-byte budget, checked during extraction; transient overshoot is possible",
),
max_output_files: z
.number()
.int()
.min(1)
.max(100_000)
.default(10_000)
.describe("Staging entry budget, including directories and links"),
}),
} as const;
/** Supported firmware operations. */
export type FirmwareOperation = keyof typeof firmwareInputSchemas;
/** Correlated, validated firmware request. */
export type FirmwareRequest = {
[Name in FirmwareOperation]: {
operation: Name;
input: z.infer<(typeof firmwareInputSchemas)[Name]>;
};
}[FirmwareOperation];
/** Validate the operation with its exact input contract. */
export const firmwareRequestSchema = z.discriminatedUnion("operation", [
z.strictObject({
operation: z.literal("inspect_firmware_regions"),
input: firmwareInputSchemas.inspect_firmware_regions,
}),
z.strictObject({
operation: z.literal("extract_firmware"),
input: firmwareInputSchemas.extract_firmware,
}),
]);
const engine = z.strictObject({
name: z.string(),
version: z.string(),
executable_path: z.string(),
executable_sha256: digestSchema,
source_revision: z.null(),
worker_count: z.literal(1),
});
const selection = z.strictObject({ ...range.shape, sha256: digestSchema });
const chunk = z.strictObject({
input_path: z.string(),
provider_id: z.string(),
range,
handler: z.string().nullable(),
encrypted: z.boolean().nullable(),
root_file_range: range.nullable(),
});
/** Portable results retain reported boundaries, unknowns and extraction lineage. */
export const firmwareResultSchemas = {
inspect_firmware_regions: z.strictObject({
engine,
input_size: bytes,
regions: z.array(
z.strictObject({
provider_id: z.string(),
offset: bytes,
reported_size: bytes,
signature: z.string(),
description: z.string(),
reported_confidence: z.number().int().min(0).max(255),
size_basis: z.literal("provider_reported_validation_unknown"),
}),
),
coverage: z.literal("signature_scan"),
}),
extract_firmware: z.strictObject({
engine,
selection,
output_directory: z.string(),
coverage: z.enum(["complete", "partial"]),
files: z.array(
z.strictObject({
relative_path: z.string(),
path: z.string(),
size: bytes,
sha256: digestSchema,
reported_sha256: digestSchema.nullable(),
mime_type: z.string().nullable(),
analysis_depth: bytes.nullable(),
runtime_address: z.null(),
original_file_range: z.null(),
}),
),
unpublished_entries: z.array(
z.strictObject({
relative_path: z.string(),
kind: z.enum(["symlink", "special"]),
link_target: z.string().nullable(),
}),
),
chunks: z.array(chunk),
derivations: z.array(
z.strictObject({
parent_path: z.string(),
child_path: z.string(),
provider_blob_id: z.string(),
handler: z.string().nullable(),
parent_file_range: range.nullable(),
}),
),
depth_limited_paths: z.array(z.string()),
diagnostics: z.array(jsonValueSchema),
published_bytes: bytes,
}),
} as const;
+229
View File
@@ -0,0 +1,229 @@
import { randomUUID } from "node:crypto";
import { access, realpath } from "node:fs/promises";
import { constants } from "node:fs";
import { isAbsolute } from "node:path";
import {
AnalysisCapabilityUnavailableError,
AnalysisCancelledError,
AnalysisTimeoutError,
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import {
spawnOwnedProviderProcess,
ProviderProcessSupervisor,
type OwnedProviderProcessSpawnOptions,
type SpawnedOwnedProviderProcess,
} from "../process/ProviderProcess.js";
import { cleanupOwnedProcessGroup } from "../process/ProcessOwnership.js";
import { FIRMWARE_LIMITS } from "./FirmwareRelease.js";
import {
hashFirmwareFile,
inventoryFirmwareOutput,
readFirmwareReport,
} from "./FirmwareFiles.js";
/** Injectable owned launcher for production protocol and lifecycle tests. */
export type FirmwareLauncher = (
options: OwnedProviderProcessSpawnOptions,
) => Promise<SpawnedOwnedProviderProcess>;
/** Resolve explicitly supplied executable paths; never install or search for engines. */
export const resolveFirmwareCommand = async (
environment: Readonly<Record<string, string | undefined>>,
engine: "binwalk" | "unblob",
operation: string,
) => {
const variable =
engine === "binwalk" ? "REA_BINWALK_COMMAND" : "REA_UNBLOB_COMMAND";
const configured = environment[variable];
if (
process.platform !== "linux" ||
configured === undefined ||
!isAbsolute(configured)
)
throw new AnalysisCapabilityUnavailableError(
engine,
operation,
`This integration requires Linux and an absolute ${variable} path to the caller-supplied tool`,
);
const command = await realpath(configured).catch((cause: unknown) => {
throw new AnalysisCapabilityUnavailableError(
engine,
operation,
`${variable} is unavailable: ${cause instanceof Error ? cause.message : String(cause)}`,
);
});
await access(command, constants.X_OK);
const limiter =
environment.REA_FIRMWARE_PRLIMIT_COMMAND ?? "/usr/bin/prlimit";
if (!isAbsolute(limiter))
throw new AnalysisCapabilityUnavailableError(
engine,
operation,
"REA_FIRMWARE_PRLIMIT_COMMAND must be absolute",
);
await access(limiter, constants.X_OK).catch(() => {
throw new AnalysisCapabilityUnavailableError(
engine,
operation,
`Resource limiter unavailable: ${limiter}; provide util-linux prlimit`,
);
});
return { command, limiter, sha256: await hashFirmwareFile(command) };
};
/** Execute one bounded provider command and verify its owned descendants have stopped. */
export const runFirmwareCommand = async (context: {
engine: string;
operation: string;
command: string;
limiter: string;
args: readonly string[];
cwd: string;
environment: Readonly<Record<string, string | undefined>>;
signal: AbortSignal;
launcher?: FirmwareLauncher;
outputBudget?: { root: string; bytes: number; entries: number };
acceptReportedExtractionFailure?: boolean;
}) => {
const { engine, operation, signal } = context;
signal.throwIfAborted();
const runId = `rea-firmware-${randomUUID()}`;
const spawned = await (context.launcher ?? spawnOwnedProviderProcess)({
command: context.limiter,
arguments: [
`--as=${FIRMWARE_LIMITS.addressSpaceBytes}`,
`--fsize=${context.outputBudget?.bytes ?? FIRMWARE_LIMITS.reportBytes}`,
"--core=0",
"--cpu=120",
"--",
context.command,
...context.args,
],
expectedCommand: null,
runId,
cwd: context.cwd,
hostEnvironment: context.environment,
env: {
TMPDIR: context.cwd,
OMP_NUM_THREADS: "1",
OPENBLAS_NUM_THREADS: "1",
RAYON_NUM_THREADS: "1",
NO_COLOR: "1",
},
});
const supervisor = new ProviderProcessSupervisor({
...spawned,
ownsProcessLifetime: true,
cleanup:
spawned.cleanup ?? (() => cleanupOwnedProcessGroup(spawned.ownership)),
});
let failure: unknown;
try {
const deadline = Date.now() + FIRMWARE_LIMITS.timeoutMs;
while (!(await supervisor.waitForExit(100))) {
if (signal.aborted) throw new AnalysisCancelledError(operation);
if (Date.now() >= deadline)
throw new AnalysisTimeoutError(operation, FIRMWARE_LIMITS.timeoutMs);
if (context.outputBudget !== undefined)
await inventoryFirmwareOutput(
context.outputBudget.root,
context.outputBudget.bytes,
context.outputBudget.entries,
operation,
signal,
);
}
if (signal.aborted) throw new AnalysisCancelledError(operation);
} catch (cause: unknown) {
failure = cause;
}
const stopped = await supervisor.stop();
const snapshot = supervisor.snapshot();
supervisor.dispose();
if (stopped.status === "incomplete")
throw new ProviderCleanupError(engine, [runId, context.cwd], {
reason: stopped.reason,
leader_pid: spawned.ownership.leaderPid,
process_group_id: spawned.ownership.processGroupId,
exit_code: snapshot.exitCode ?? null,
signal: snapshot.signal ?? null,
stdout: snapshot.stdout.text,
stderr: snapshot.stderr.text,
stdout_bytes: snapshot.stdout.bytes,
stderr_bytes: snapshot.stderr.bytes,
previous_error: failure instanceof Error ? failure.message : null,
});
if (failure !== undefined) throw failure;
if (
snapshot.exitCode !== 0 &&
!(
context.acceptReportedExtractionFailure === true &&
snapshot.exitCode === 1
)
)
throw new ProviderAdapterError(engine, operation, {
diagnostics: {
reason:
"Tool failed; inspect exit status and stderr for format, dependency, sandbox or resource-limit failures",
exit_code: snapshot.exitCode ?? null,
signal: snapshot.signal ?? null,
stdout: snapshot.stdout.text,
stderr: snapshot.stderr.text,
stdout_bytes: snapshot.stdout.bytes,
stderr_bytes: snapshot.stderr.bytes,
},
});
return {
stdout: snapshot.stdout,
stderr: snapshot.stderr,
exit_code: snapshot.exitCode,
signal: snapshot.signal ?? null,
command: context.command,
args: [...context.args],
resource_limiter: context.limiter,
};
};
/** Preserve tool failure diagnostics when an accepted exit 1 has no usable report. */
export const readFirmwareCommandReport = async (context: {
path: string;
engine: string;
operation: string;
execution: Awaited<ReturnType<typeof runFirmwareCommand>>;
}): Promise<unknown> => {
try {
return await readFirmwareReport(context.path, context.operation);
} catch (cause: unknown) {
if (context.execution.exit_code === 1)
throw new ProviderAdapterError(context.engine, context.operation, {
cause,
diagnostics: {
reason:
"Tool exited 1 without a usable analysis report; inspect stderr for startup, sandbox or dependency failures",
report_failure:
cause instanceof Error ? cause.message : String(cause),
exit_code: 1,
signal: context.execution.signal,
stdout: context.execution.stdout.text,
stderr: context.execution.stderr.text,
stdout_bytes: context.execution.stdout.bytes,
stderr_bytes: context.execution.stderr.bytes,
},
});
const diagnostics = `stdout (${context.execution.stdout.bytes} bytes): ${context.execution.stdout.text}; stderr (${context.execution.stderr.bytes} bytes): ${context.execution.stderr.text}`;
if (cause instanceof AnalysisOutputError)
throw new AnalysisOutputError(
context.operation,
`${cause.reason}; ${diagnostics}`,
{ cause },
);
throw new AnalysisOutputError(
context.operation,
`Tool did not produce the expected complete report: ${context.path}; ${diagnostics}`,
{ cause },
);
}
};
+265
View File
@@ -0,0 +1,265 @@
import { createHash } from "node:crypto";
import { constants, createReadStream, createWriteStream } from "node:fs";
import {
chmod,
lstat,
open,
readdir,
readlink,
realpath,
} from "node:fs/promises";
import { isAbsolute, join, relative, resolve, sep } from "node:path";
import { Transform } from "node:stream";
import { pipeline } from "node:stream/promises";
import {
AnalysisCancelledError,
AnalysisInputError,
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import type { FirmwareRequest } from "../domain/firmwareAnalysis.js";
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
import { FIRMWARE_LIMITS } from "./FirmwareRelease.js";
/** One regular file or unpublished filesystem entry in a private extraction. */
export interface FirmwareEntry {
readonly relativePath: string;
readonly path: string;
readonly size: number;
readonly kind: "file" | "symlink" | "special";
readonly linkTarget: string | null;
}
/** Hash a regular tool or extracted file without retaining its bytes in memory. */
export const hashFirmwareFile = async (
path: string,
signal?: AbortSignal,
): Promise<string> => {
const handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW);
const hash = createHash("sha256");
try {
if (!(await handle.stat()).isFile())
throw new Error(`Not a regular file: ${path}`);
for await (const chunk of handle.createReadStream({ autoClose: false })) {
signal?.throwIfAborted();
hash.update(chunk);
}
} finally {
await handle.close();
}
return hash.digest("hex");
};
/** Copy and identify the exact bytes consumed by firmware providers. */
export const snapshotFirmware = async (
request: FirmwareRequest,
root: string,
signal: AbortSignal,
) => {
const sourcePath = resolve(request.input.path);
const source = await realpath(sourcePath);
const handle = await open(source, constants.O_RDONLY | constants.O_NOFOLLOW);
const fullPath = join(root, "firmware.bin");
const hash = createHash("sha256");
let size = 0;
try {
const stat = await handle.stat();
if (
!stat.isFile() ||
stat.size === 0 ||
stat.size > FIRMWARE_LIMITS.inputBytes
)
throw new AnalysisInputError(request.operation, undefined, [
{
path: ["path"],
reason: "out_of_range",
message: `Firmware must be a nonempty regular file no larger than ${FIRMWARE_LIMITS.inputBytes} bytes`,
},
]);
const meter = new Transform({
transform(chunk: Buffer, _encoding, callback) {
size += chunk.length;
if (size > FIRMWARE_LIMITS.inputBytes) {
callback(new Error("Firmware grew beyond the input byte budget"));
return;
}
hash.update(chunk);
callback(null, chunk);
},
});
await pipeline(
handle.createReadStream({ autoClose: false }),
meter,
createWriteStream(fullPath, { flags: "wx", mode: 0o600 }),
{ signal },
);
} finally {
await handle.close();
}
await chmod(fullPath, 0o400);
const sha256 = hash.digest("hex");
const selected =
request.operation === "extract_firmware" ? request.input.range : undefined;
const offset = selected?.offset ?? 0;
const length = selected?.length ?? size;
if (length === 0 || offset > size || length > size - offset)
throw new AnalysisInputError(request.operation, undefined, [
{
path: ["range"],
reason: "out_of_range",
message: `Selected range must be within the ${size}-byte firmware file`,
},
]);
let inputPath = fullPath;
if (selected !== undefined) {
inputPath = join(root, "selected.bin");
await pipeline(
createReadStream(fullPath, { start: offset, end: offset + length - 1 }),
createWriteStream(inputPath, { flags: "wx", mode: 0o600 }),
{ signal },
);
await chmod(inputPath, 0o400);
}
return {
subject: { path: sourcePath, sha256, format: "file" as const },
size,
inputPath,
selection: {
offset,
length,
sha256:
selected === undefined
? sha256
: await hashFirmwareFile(inputPath, signal),
},
};
};
/** Read a complete bounded report; oversized data is never reported as complete. */
export const readFirmwareReport = async (
path: string,
operation: string,
): Promise<unknown> => {
const handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW);
try {
if (!(await handle.stat()).isFile())
throw new AnalysisOutputError(operation, "Report is not a regular file");
const bytes = await readBoundedFileBytes(
handle,
FIRMWARE_LIMITS.reportBytes,
);
if (bytes === undefined)
throw new AnalysisOutputError(
operation,
`Report exceeded ${FIRMWARE_LIMITS.reportBytes} bytes`,
);
try {
const value: unknown = JSON.parse(bytes.toString("utf8"));
return value;
} catch (cause: unknown) {
throw new AnalysisOutputError(
operation,
"Tool report is not valid complete JSON",
{ cause },
);
}
} finally {
await handle.close();
}
};
/** Inventory private output without following links, enforcing aggregate staging budgets. */
export const inventoryFirmwareOutput = async (
root: string,
maxBytes: number,
maxEntries: number,
operation: string,
signal?: AbortSignal,
): Promise<FirmwareEntry[]> => {
const entries: FirmwareEntry[] = [];
const pending = [root];
let bytes = 0;
let count = 0;
while (pending.length > 0) {
if (signal?.aborted === true) throw new AnalysisCancelledError(operation);
const directory = pending.pop();
if (directory === undefined) break;
const directoryStat = await lstat(directory).catch((cause: unknown) => {
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
return undefined;
throw cause;
});
if (
directoryStat === undefined ||
!directoryStat.isDirectory() ||
directoryStat.isSymbolicLink()
)
continue;
const names = await readdir(directory).catch((cause: unknown) => {
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
return [];
throw cause;
});
for (const name of names.sort()) {
count++;
if (count > maxEntries)
throw new AnalysisOutputError(
operation,
`Staging entry budget exceeded (${maxEntries})`,
);
const path = join(directory, name);
const stat = await lstat(path).catch((cause: unknown) => {
if (
cause instanceof Error &&
"code" in cause &&
cause.code === "ENOENT"
)
return undefined;
throw cause;
});
if (stat === undefined) continue;
if (stat.isDirectory()) {
pending.push(path);
continue;
}
if (stat.isFile()) bytes += stat.size;
if (bytes > maxBytes)
throw new AnalysisOutputError(
operation,
`Staging byte budget exceeded (${maxBytes})`,
);
entries.push({
path,
relativePath: relative(root, path).split(sep).join("/"),
size: stat.size,
kind: stat.isSymbolicLink()
? "symlink"
: stat.isFile()
? "file"
: "special",
linkTarget: stat.isSymbolicLink() ? await readlink(path) : null,
});
}
}
return entries.sort((a, b) => a.relativePath.localeCompare(b.relativePath));
};
/** Bind producer paths to the selected input or private extraction tree. */
export const firmwareLogicalPath = (
path: string,
inputPath: string,
outputRoot: string,
): string => {
if (path === inputPath) return "$input";
const rel = relative(outputRoot, path);
if (
!isAbsolute(path) ||
rel === ".." ||
rel.startsWith(`..${sep}`) ||
isAbsolute(rel)
)
throw new AnalysisOutputError(
"extract_firmware",
`Provider reported a path outside its owned output: ${path}`,
);
return rel === "" ? "$output" : `$output/${rel.split(sep).join("/")}`;
};
+350
View File
@@ -0,0 +1,350 @@
import { join, isAbsolute } from "node:path";
import { z } from "zod";
import type { FirmwareAnalysisPort } from "../application/FirmwareAnalysisPort.js";
import {
createAnalysisExecution,
type ExecutionOptions,
} from "../application/AnalysisProvider.js";
import {
firmwareResultSchemas,
type FirmwareRequest,
} from "../domain/firmwareAnalysis.js";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
AnalysisInputError,
AnalysisOutputError,
AnalysisTimeoutError,
} from "../domain/analysisErrorCore.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { err, ok } from "../domain/result.js";
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import { publishFirmwareExtraction } from "./FirmwarePublication.js";
import {
FIRMWARE_LIMITS,
FIRMWARE_RELEASES,
BINWALK_PROVIDER_IDENTITY,
UNBLOB_PROVIDER_IDENTITY,
} from "./FirmwareRelease.js";
import {
resolveFirmwareCommand,
runFirmwareCommand,
readFirmwareCommandReport,
type FirmwareLauncher,
} from "./FirmwareCommand.js";
import { inventoryFirmwareOutput, snapshotFirmware } from "./FirmwareFiles.js";
import {
normalizeBinwalkReport,
normalizeUnblobReport,
} from "./FirmwareReports.js";
type Outcome = Awaited<ReturnType<FirmwareAnalysisPort["execute"]>>;
const waitForPredecessor = (
predecessor: Promise<void>,
operation: string,
signal?: AbortSignal,
): Promise<void> =>
new Promise((resolve, reject) => {
const abort = () => reject(new AnalysisCancelledError(operation));
signal?.addEventListener("abort", abort, { once: true });
if (signal?.aborted === true) abort();
void predecessor.then(() => {
signal?.removeEventListener("abort", abort);
resolve();
});
});
const limitations = [
"Signature matches and reported sizes do not establish partition validity, encryption or runtime mappings. Binwalk may infer unknown lengths before emitting its report.",
"Only Linux is supported by this integration. One worker, inherited per-process address-space/file-size/CPU limits and a deadline constrain execution; staging byte and entry budgets are polled and may transiently overshoot. Limits are not aggregate process-memory or filesystem quotas.",
"Unblob's default sandbox and skip rules remain enabled. Missing external extractors, unknown chunks, encrypted chunks and depth limits yield partial coverage. Extracted code is never executed; only regular files are published, without executable permission bits.",
"The actual launcher executable is hashed. Interpreter, shared libraries and installed Python package dependencies are not independently attested; source revision remains unknown for caller-supplied builds.",
] as const;
/** Queued bring-your-own firmware adapters with private snapshots and owned cleanup. */
export class FirmwareProvider implements FirmwareAnalysisPort {
#tail: Promise<void> = Promise.resolve();
#cleanupFailure: AnalysisError | undefined;
constructor(
readonly environment: Readonly<
Record<string, string | undefined>
> = process.env,
readonly launcher?: FirmwareLauncher,
) {}
/** Inspect regions or explicitly extract selected bytes without changing the source. */
async execute(
request: FirmwareRequest,
options?: ExecutionOptions,
): Promise<Outcome> {
const predecessor = this.#tail;
let release = () => {};
this.#tail = new Promise<void>((resolve) => {
release = resolve;
});
// Cancel the caller promptly without letting a later request bypass its predecessor.
try {
await waitForPredecessor(predecessor, request.operation, options?.signal);
} catch (cause: unknown) {
void predecessor.then(release);
return err(
cause instanceof AnalysisError
? cause
: new AnalysisCancelledError(request.operation),
);
}
try {
return await this.#execute(request, options);
} finally {
release();
}
}
async #execute(
request: FirmwareRequest,
options?: ExecutionOptions,
): Promise<Outcome> {
let root: PrivateRuntimeRoot | undefined;
const engineName =
request.operation === "inspect_firmware_regions" ? "binwalk" : "unblob";
const deadline = AbortSignal.timeout(FIRMWARE_LIMITS.timeoutMs);
const signal =
options?.signal === undefined
? deadline
: AbortSignal.any([options.signal, deadline]);
let outcome: Outcome;
try {
if (this.#cleanupFailure !== undefined) return err(this.#cleanupFailure);
if (signal.aborted) throw new AnalysisCancelledError(request.operation);
const command = await resolveFirmwareCommand(
this.environment,
engineName,
request.operation,
);
root = await PrivateRuntimeRoot.create({ prefix: "rea-firmware-" });
const target = await snapshotFirmware(request, root.path, signal);
const run = (
args: readonly string[],
outputBudget?: { root: string; bytes: number; entries: number },
) =>
runFirmwareCommand({
engine: engineName,
operation: request.operation,
...command,
args,
cwd: root?.path ?? "",
environment: this.environment,
signal,
...(this.launcher === undefined ? {} : { launcher: this.launcher }),
...(outputBudget === undefined
? {}
: { outputBudget, acceptReportedExtractionFailure: true }),
});
const versionRun = await run(["--version"]);
const version = versionRun.stdout.text.trim();
const expected = FIRMWARE_RELEASES[engineName].version;
if (
versionRun.stdout.bytes > 1024 ||
(engineName === "binwalk"
? version !== `binwalk ${expected}`
: version !== expected)
)
throw new AnalysisCapabilityUnavailableError(
engineName,
request.operation,
`Unsupported tool version ${version}; this adapter is verified against ${expected}`,
);
const engine = {
name: engineName,
version: expected,
executable_path: command.command,
executable_sha256: command.sha256,
source_revision: null,
worker_count: 1 as const,
};
const reportPath = join(root.path, "report.json");
let result: unknown;
let raw: unknown;
if (request.operation === "inspect_firmware_regions") {
const processRun = await run([
"--quiet",
"--search-all",
"--threads",
"1",
"--log",
reportPath,
target.inputPath,
]);
const report = await readFirmwareCommandReport({
path: reportPath,
engine: engineName,
operation: request.operation,
execution: processRun,
});
result = firmwareResultSchemas.inspect_firmware_regions.parse({
engine,
input_size: target.size,
regions: normalizeBinwalkReport(
report,
target.inputPath,
target.size,
),
coverage: "signature_scan",
});
raw = { report, execution: processRun, version_execution: versionRun };
} else {
if (!isAbsolute(request.input.output_directory))
throw new AnalysisInputError(request.operation, undefined, [
{
path: ["output_directory"],
reason: "invalid_format",
message: "Extraction output directory must be absolute",
},
]);
const outputRoot = join(root.path, "extracted");
const processRun = await run(
[
"--process-num",
"1",
"--depth",
String(request.input.max_depth),
"--randomness-depth",
"0",
"--report",
reportPath,
"--log",
join(root.path, "unblob.log"),
"--extract-dir",
outputRoot,
target.inputPath,
],
{
root: outputRoot,
bytes: request.input.max_output_bytes,
entries: request.input.max_output_files,
},
);
const report = await readFirmwareCommandReport({
path: reportPath,
engine: engineName,
operation: request.operation,
execution: processRun,
});
const normalized = normalizeUnblobReport(report, {
inputPath: target.inputPath,
outputRoot,
...target.selection,
maxDepth: request.input.max_depth,
});
if (processRun.exit_code === 1 && normalized.diagnostics.length === 0)
throw new AnalysisOutputError(
request.operation,
"Unblob failed without a reported extraction diagnostic",
);
const entries = await inventoryFirmwareOutput(
outputRoot,
request.input.max_output_bytes,
request.input.max_output_files,
request.operation,
signal,
);
result = await publishFirmwareExtraction({
outputDirectory: request.input.output_directory,
entries,
normalized,
selection: target.selection,
engine,
signal,
});
raw = { report, execution: processRun, version_execution: versionRun };
}
outcome = ok(
createAnalysisExecution(
result,
engineName === "binwalk"
? BINWALK_PROVIDER_IDENTITY
: UNBLOB_PROVIDER_IDENTITY,
{
subject: target.subject,
rawResult: raw,
limitations,
locations: [
{
kind: "file-offset-range",
start: target.selection.offset,
end: target.selection.offset + target.selection.length,
},
],
},
),
);
} catch (cause: unknown) {
outcome = err(
cause instanceof ProviderCleanupError
? cause
: options?.signal?.aborted === true
? new AnalysisCancelledError(request.operation)
: deadline.aborted
? new AnalysisTimeoutError(
request.operation,
FIRMWARE_LIMITS.timeoutMs,
)
: cause instanceof AnalysisError
? cause
: cause instanceof z.ZodError
? new AnalysisOutputError(
request.operation,
"Tool report violated the pinned producer protocol",
{ cause },
)
: new ProviderAdapterError(engineName, request.operation, {
cause,
diagnostics: {
reason:
cause instanceof Error
? cause.message
: String(cause),
},
}),
);
if (!outcome.ok && outcome.error.cleanupIncomplete) {
this.#cleanupFailure =
root === undefined ||
outcome.error.cleanupResources.includes(root.path)
? outcome.error
: new ProviderCleanupError(
engineName,
[...outcome.error.cleanupResources, root.path],
outcome.error instanceof ProviderAdapterError
? (outcome.error.diagnostics ?? {
reason: outcome.error.message,
})
: { reason: outcome.error.message },
{ cause: outcome.error },
);
}
}
// Never remove a workspace while process ownership remains uncertain.
if (this.#cleanupFailure !== undefined) return err(this.#cleanupFailure);
try {
await root?.close();
} catch (cause: unknown) {
this.#cleanupFailure = new ProviderCleanupError(
engineName,
[
root?.path ?? "unknown",
...(outcome.ok && request.operation === "extract_firmware"
? [request.input.output_directory]
: []),
],
{
reason: cause instanceof Error ? cause.message : String(cause),
previous_result: outcome.ok ? outcome.value.result : null,
},
);
return err(this.#cleanupFailure);
}
return outcome;
}
}
+115
View File
@@ -0,0 +1,115 @@
import { createReadStream } from "node:fs";
import { join } from "node:path";
import type { z } from "zod";
import { SafeOutputTree } from "../artifacts/SafeOutputTree.js";
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { firmwareResultSchemas } from "../domain/firmwareAnalysis.js";
import { hashFirmwareFile, type FirmwareEntry } from "./FirmwareFiles.js";
import type { normalizeUnblobReport } from "./FirmwareReports.js";
/** Publish verified regular files, preserving links as metadata and rolling back failed writes. */
export const publishFirmwareExtraction = async (context: {
outputDirectory: string;
entries: readonly FirmwareEntry[];
normalized: ReturnType<typeof normalizeUnblobReport>;
selection: { offset: number; length: number; sha256: string };
engine: z.infer<typeof firmwareResultSchemas.extract_firmware>["engine"];
signal: AbortSignal;
}) => {
let tree: SafeOutputTree | undefined;
try {
const materialized = new Set(
context.entries
.filter((entry) => entry.kind === "file")
.map((entry) => `$output/${entry.relativePath}`),
);
for (const path of context.normalized.files.keys()) {
if (path !== "$input" && !materialized.has(path))
throw new AnalysisOutputError(
"extract_firmware",
`Reported regular file is absent from materialized output: ${path}`,
);
}
const files = [];
const unpublished = [];
tree = await SafeOutputTree.create(context.outputDirectory);
for (const entry of context.entries) {
if (entry.kind !== "file") {
unpublished.push({
relative_path: entry.relativePath,
kind: entry.kind,
link_target: entry.linkTarget,
});
continue;
}
const logicalPath = `$output/${entry.relativePath}`;
const reported = context.normalized.files.get(logicalPath);
const sha256 = await hashFirmwareFile(entry.path, context.signal);
if (
(reported?.sha256 !== null &&
reported?.sha256 !== undefined &&
reported.sha256 !== sha256) ||
(context.normalized.sizes.has(logicalPath) &&
context.normalized.sizes.get(logicalPath) !== entry.size)
)
throw new AnalysisOutputError(
"extract_firmware",
`Extracted bytes disagree with provider report: ${entry.relativePath}`,
);
await tree.write(
entry.relativePath,
createReadStream(entry.path),
sha256,
context.signal,
);
files.push({
relative_path: entry.relativePath,
path: join(tree.outputRoot, entry.relativePath),
size: entry.size,
sha256,
reported_sha256: reported?.sha256 ?? null,
mime_type: reported?.mimeType ?? null,
analysis_depth: reported?.depth ?? null,
runtime_address: null,
original_file_range: null,
});
}
const result = firmwareResultSchemas.extract_firmware.parse({
engine: context.engine,
selection: context.selection,
output_directory: tree.outputRoot,
coverage:
unpublished.length > 0 ? "partial" : context.normalized.coverage,
files,
unpublished_entries: unpublished,
chunks: context.normalized.chunks,
derivations: context.normalized.derivations,
depth_limited_paths: context.normalized.depth_limited_paths,
diagnostics: context.normalized.diagnostics,
published_bytes: files.reduce((sum, file) => sum + file.size, 0),
});
context.signal.throwIfAborted();
await tree.commit();
return result;
} catch (cause: unknown) {
try {
await tree?.rollback();
} catch (cleanupCause: unknown) {
throw new ProviderCleanupError(
"unblob",
[context.outputDirectory],
{
reason:
cleanupCause instanceof Error
? cleanupCause.message
: String(cleanupCause),
previous_error:
cause instanceof Error ? cause.message : String(cause),
},
{ cause: cleanupCause },
);
}
throw cause;
}
};
+34
View File
@@ -0,0 +1,34 @@
/** Audited upstream sources; installed tool bytes are fingerprinted separately. */
export const FIRMWARE_RELEASES = {
binwalk: {
version: "3.1.0",
revision: "4fdab3d464d97b68e0af9088df3f9e2e1545b21c",
repository: "https://github.com/ReFirmLabs/binwalk",
},
unblob: {
version: "26.6.4",
revision: "1fcc7a0a584a70a96c31f5a276c20944d199a089",
repository: "https://github.com/onekey-sec/unblob",
},
} as const;
/** Catalog identities; Evidence uses the version observed from the executed tool. */
export const BINWALK_PROVIDER_IDENTITY = {
id: "binwalk",
name: "Binwalk",
version: FIRMWARE_RELEASES.binwalk.version,
} as const;
/** Catalog identity for explicit firmware extraction. */
export const UNBLOB_PROVIDER_IDENTITY = {
id: "unblob",
name: "Unblob",
version: FIRMWARE_RELEASES.unblob.version,
} as const;
/** Resource policy for this Linux integration, not a claim of aggregate containment. */
export const FIRMWARE_LIMITS = {
inputBytes: 128 * 1024 * 1024,
reportBytes: 8 * 1024 * 1024,
addressSpaceBytes: 1024 * 1024 * 1024,
timeoutMs: 120_000,
} as const;
+148
View File
@@ -0,0 +1,148 @@
import { describe, expect, it } from "vitest";
import {
normalizeBinwalkReport,
normalizeUnblobReport,
} from "./FirmwareReports.js";
const context = {
inputPath: "/private/input.bin",
outputRoot: "/private/output",
sha256: "a".repeat(64),
length: 20,
offset: 64,
maxDepth: 3,
};
const root = () => ({
task: {
path: context.inputPath,
depth: 0,
blob_id: "",
is_multi_file: false,
},
reports: [
{
__typename__: "StatReport",
path: context.inputPath,
size: 20,
is_file: true,
is_dir: false,
is_link: false,
link_target: null,
},
{ __typename__: "HashReport", sha256: context.sha256 },
{
__typename__: "UnknownChunkReport",
id: "unknown",
start_offset: 0,
end_offset: 20,
size: 20,
randomness: null,
},
],
subtasks: [],
});
describe("firmware producer boundaries", () => {
it("retains unknown chunks and their selected original-file range", () => {
const normalized = normalizeUnblobReport([root()], context);
expect(normalized.coverage).toBe("partial");
expect(normalized.chunks[0]).toMatchObject({
handler: null,
range: { offset: 0, length: 20 },
root_file_range: { offset: 64, length: 20 },
});
});
it("preserves unfamiliar report types as explicit diagnostics", () => {
const record = root();
const normalized = normalizeUnblobReport(
[
{
...record,
reports: [
...record.reports,
{ __typename__: "FutureError", reason: "new producer facet" },
],
},
],
context,
);
expect(normalized.diagnostics).toEqual([
{
input_path: "$input",
report: { __typename__: "FutureError", reason: "new producer facet" },
},
]);
expect(normalized.coverage).toBe("partial");
});
it("rejects unknown tasks outside the owned output and disconnected task graphs", () => {
const record = root();
for (const path of [
"/private/output/../escaped",
"/private/output/unconnected",
]) {
const child = { ...record, task: { ...record.task, path, depth: 1 } };
expect(() => normalizeUnblobReport([record, child], context)).toThrow();
}
});
it("rejects absent root identity, inconsistent chunk bounds and duplicate tasks", () => {
const record = root();
expect(() => normalizeUnblobReport([], context)).toThrow("exactly one");
expect(() =>
normalizeUnblobReport(
[
{
...record,
reports: record.reports.filter(
(report) => report.__typename__ !== "HashReport",
),
},
],
context,
),
).toThrow("hash");
expect(() =>
normalizeUnblobReport(
[
{
...record,
reports: [
...record.reports,
{
__typename__: "UnknownChunkReport",
id: "bad",
start_offset: 10,
end_offset: 21,
size: 11,
},
],
},
],
context,
),
).toThrow("bounds");
expect(() => normalizeUnblobReport([record, record], context)).toThrow(
"exactly one",
);
});
it("rejects empty, combined and extracting Binwalk reports", () => {
const record = {
Analysis: { file_path: context.inputPath, file_map: [], extractions: {} },
};
expect(normalizeBinwalkReport([record], context.inputPath, 20)).toEqual([]);
for (const report of [
[],
[record, record],
[
{
Analysis: {
...record.Analysis,
extractions: { hit: { status: "success" } },
},
},
],
])
expect(() =>
normalizeBinwalkReport(report, context.inputPath, 20),
).toThrow();
});
});
+327
View File
@@ -0,0 +1,327 @@
import { z } from "zod";
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
import { digestSchema } from "../domain/digests.js";
import { firmwareLogicalPath } from "./FirmwareFiles.js";
const integer = z.number().int().min(0).max(Number.MAX_SAFE_INTEGER);
const signature = z.object({
id: z.string().min(1),
offset: integer,
size: integer,
name: z.string(),
description: z.string(),
confidence: z.number().int().min(0).max(255),
always_display: z.boolean(),
extraction_declined: z.boolean(),
});
const scan = z.array(
z.object({
Analysis: z.object({
file_path: z.string(),
file_map: z.array(signature),
extractions: z.record(z.string(), jsonValueSchema),
}),
}),
);
const report = z.object({ __typename__: z.string() }).catchall(jsonValueSchema);
const task = z.object({
path: z.string(),
depth: integer,
blob_id: z.string(),
is_multi_file: z.boolean(),
});
const extraction = z.array(
z.object({ task, reports: z.array(report), subtasks: z.array(task) }),
);
const stat = z.object({
__typename__: z.literal("StatReport"),
path: z.string(),
size: integer,
is_dir: z.boolean(),
is_file: z.boolean(),
is_link: z.boolean(),
link_target: z.string().nullable(),
});
const hashes = z.object({ sha256: digestSchema });
const region = z.object({
id: z.string(),
start_offset: integer,
end_offset: integer,
size: integer,
});
const known = new Set([
"StatReport",
"HashReport",
"FileMagicReport",
"RandomnessReport",
"ChunkReport",
"UnknownChunkReport",
"CarveDirectoryReport",
"MultiFileReport",
]);
/** Interpret the pinned Binwalk report without inventing validated region lengths. */
export const normalizeBinwalkReport = (
raw: unknown,
inputPath: string,
size: number,
) => {
const records = scan.parse(raw);
const analysis = records[0]?.Analysis;
if (
records.length !== 1 ||
analysis?.file_path !== inputPath ||
Object.keys(analysis.extractions).length !== 0
)
throw new AnalysisOutputError(
"inspect_firmware_regions",
"Expected one inspection-only report for the selected immutable input",
);
return analysis.file_map.map((hit) => {
if (hit.offset >= size || hit.size > size - hit.offset)
throw new AnalysisOutputError(
"inspect_firmware_regions",
"Reported region lies outside the selected file",
);
return {
provider_id: hit.id,
offset: hit.offset,
reported_size: hit.size,
signature: hit.name,
description: hit.description,
reported_confidence: hit.confidence,
size_basis: "provider_reported_validation_unknown" as const,
};
});
};
/** Normalize Unblob's unordered task graph, retaining unknown and failed analyses. */
export const normalizeUnblobReport = (
raw: unknown,
context: {
inputPath: string;
outputRoot: string;
sha256: string;
length: number;
offset: number;
maxDepth: number;
},
) => {
const records = extraction.parse(raw);
const logical = (path: string) =>
firmwareLogicalPath(path, context.inputPath, context.outputRoot);
const roots = records.filter(
({ task: item }) => item.path === context.inputPath && item.depth === 0,
);
if (roots.length !== 1)
throw new AnalysisOutputError(
"extract_firmware",
"Expected exactly one selected input task",
);
const rootHash = roots[0]?.reports.find(
(r) => r.__typename__ === "HashReport",
);
if (
rootHash === undefined ||
hashes.parse(rootHash).sha256 !== context.sha256
)
throw new AnalysisOutputError(
"extract_firmware",
"Unblob input hash disagrees with selected snapshot",
);
const files = new Map<
string,
{ sha256: string | null; mimeType: string | null; depth: number }
>();
const sizes = new Map<string, number>();
const depths: string[] = [];
const diagnostics: JsonValue[] = [];
const chunks: {
input_path: string;
provider_id: string;
range: { offset: number; length: number };
handler: string | null;
encrypted: boolean | null;
root_file_range: { offset: number; length: number } | null;
}[] = [];
const seen = new Set<string>();
for (const record of records) {
const path = logical(record.task.path);
if (seen.has(path))
throw new AnalysisOutputError(
"extract_firmware",
`Duplicate analysis task: ${path}`,
);
seen.add(path);
if (record.task.depth >= context.maxDepth) depths.push(path);
const metadata = record.reports.find(
(r) => r.__typename__ === "StatReport",
);
if (metadata === undefined)
throw new AnalysisOutputError(
"extract_firmware",
`Missing StatReport: ${path}`,
);
const parsedStat = stat.parse(metadata);
if (parsedStat.path !== record.task.path)
throw new AnalysisOutputError(
"extract_firmware",
`StatReport is bound to another task: ${path}`,
);
if (path === "$input" && parsedStat.size !== context.length)
throw new AnalysisOutputError(
"extract_firmware",
"Unblob input size disagrees with selected snapshot",
);
if (
path === "$input" &&
(!parsedStat.is_file || parsedStat.is_link || parsedStat.is_dir)
)
throw new AnalysisOutputError(
"extract_firmware",
"Selected input was not reported as a regular file",
);
sizes.set(path, parsedStat.size);
const hash = record.reports.find((r) => r.__typename__ === "HashReport");
const magic = record.reports.find(
(r) => r.__typename__ === "FileMagicReport",
);
if (parsedStat.is_file && !parsedStat.is_link)
files.set(path, {
sha256: hash === undefined ? null : hashes.parse(hash).sha256,
mimeType:
magic === undefined
? null
: z.object({ mime_type: z.string() }).parse(magic).mime_type,
depth: record.task.depth,
});
for (const item of record.reports) {
if (!known.has(item.__typename__))
diagnostics.push(
jsonValueSchema.parse({ input_path: path, report: item }),
);
if (
item.__typename__ === "ChunkReport" ||
item.__typename__ === "UnknownChunkReport"
) {
const parsed = region.parse(item);
if (
parsed.end_offset <= parsed.start_offset ||
parsed.end_offset > parsedStat.size ||
parsed.size !== parsed.end_offset - parsed.start_offset
)
throw new AnalysisOutputError(
"extract_firmware",
`Invalid chunk bounds for ${path}`,
);
const detail =
item.__typename__ === "ChunkReport"
? z
.object({
handler_name: z.string(),
is_encrypted: z.boolean(),
extraction_reports: z.array(jsonValueSchema),
})
.parse(item)
: null;
if (detail !== null)
for (const problem of detail.extraction_reports)
diagnostics.push({ input_path: path, report: problem });
chunks.push({
input_path: path,
provider_id: parsed.id,
range: { offset: parsed.start_offset, length: parsed.size },
handler: detail?.handler_name ?? null,
encrypted: detail?.is_encrypted ?? null,
root_file_range:
path === "$input"
? {
offset: context.offset + parsed.start_offset,
length: parsed.size,
}
: null,
});
}
}
}
const recordsByPath = new Map(
records.map((record) => [record.task.path, record.task]),
);
const chunksByOrigin = new Map<string, (typeof chunks)[number]>();
for (const item of chunks) {
const key = JSON.stringify([item.input_path, item.provider_id]);
if (chunksByOrigin.has(key))
throw new AnalysisOutputError(
"extract_firmware",
`Duplicate chunk identity in ${item.input_path}`,
);
chunksByOrigin.set(key, item);
}
const derivations = records.flatMap((record) =>
record.subtasks.map((child) => {
const parentPath = logical(record.task.path);
const childPath = logical(child.path);
if (!seen.has(childPath))
throw new AnalysisOutputError(
"extract_firmware",
`Missing child task: ${childPath}`,
);
const reportedChild = recordsByPath.get(child.path);
if (
reportedChild?.depth !== child.depth ||
reportedChild.blob_id !== child.blob_id ||
reportedChild.is_multi_file !== child.is_multi_file
)
throw new AnalysisOutputError(
"extract_firmware",
`Child task identity disagrees with its analysis: ${childPath}`,
);
const origin = chunksByOrigin.get(
JSON.stringify([parentPath, child.blob_id]),
);
return {
parent_path: parentPath,
child_path: childPath,
provider_blob_id: child.blob_id,
handler: origin?.handler ?? null,
parent_file_range: origin?.range ?? null,
};
}),
);
const reachable = new Set(["$input"]);
const pending = ["$input"];
const childrenByParent = new Map<string, string[]>();
for (const edge of derivations) {
const children = childrenByParent.get(edge.parent_path) ?? [];
children.push(edge.child_path);
childrenByParent.set(edge.parent_path, children);
}
while (pending.length > 0) {
const parent = pending.pop();
for (const child of childrenByParent.get(parent ?? "") ?? []) {
if (reachable.has(child)) continue;
reachable.add(child);
pending.push(child);
}
}
if (reachable.size !== seen.size)
throw new AnalysisOutputError(
"extract_firmware",
"Report contains tasks disconnected from the selected input",
);
return {
files,
sizes,
chunks,
derivations,
depth_limited_paths: depths.sort(),
diagnostics,
coverage:
depths.length > 0 ||
diagnostics.length > 0 ||
chunks.some((c) => c.handler === null || c.encrypted === true)
? ("partial" as const)
: ("complete" as const),
};
};
File diff suppressed because one or more lines are too long
+21
View File
@@ -18,6 +18,10 @@ import { registerElectronTools } from "./registerElectronTools.js";
import { registerEnhancedTools } from "./registerEnhancedTools.js";
import { registerJavaScriptRuntimeObservationTools } from "./registerJavaScriptRuntimeObservationTools.js";
import { registerManagedTools } from "./registerManagedTools.js";
import { registerFirmwareTools } from "./registerFirmwareTools.js";
import { FirmwareAnalysisService } from "../application/FirmwareAnalysisService.js";
import type { FirmwareAnalysisPort } from "../application/FirmwareAnalysisPort.js";
import { FirmwareProvider } from "../firmware/FirmwareProvider.js";
import { registerAndroidTools } from "./registerAndroidTools.js";
import { AndroidAnalysisService } from "../application/AndroidAnalysisService.js";
import type { AndroidAnalysisPort } from "../application/AndroidAnalysisPort.js";
@@ -38,6 +42,7 @@ const ACTIVE_TARGET_INSTRUCTIONS =
export interface CreateServerOptions {
readonly logger?: Logger;
readonly firmwareAnalysis?: FirmwareAnalysisPort;
readonly androidAnalysis?: AndroidAnalysisPort;
readonly browserObservation?: BrowserObservationPort;
readonly browserScenarioCapture?: BrowserScenarioCapturePort;
@@ -55,6 +60,14 @@ const installSessionToolAvailability = (
if (session === undefined) return undefined;
const policy = sessionAvailabilityPolicy(options.availabilityPolicy, {
optionalFeatures: {
firmwareInspectionEnabled:
options.firmwareAnalysis !== undefined ||
(process.platform === "linux" &&
isAbsolute(process.env.REA_BINWALK_COMMAND ?? "")),
firmwareExtractionEnabled:
options.firmwareAnalysis !== undefined ||
(process.platform === "linux" &&
isAbsolute(process.env.REA_UNBLOB_COMMAND ?? "")),
androidAnalysisEnabled:
options.androidAnalysis !== undefined ||
((process.platform === "linux" || process.platform === "darwin") &&
@@ -127,6 +140,14 @@ export const createServer = (
toolLogger,
recordEvidence,
);
registerFirmwareTools(
server,
new FirmwareAnalysisService(
options.firmwareAnalysis ?? new FirmwareProvider(),
),
toolLogger,
recordEvidence,
);
registerObservationTools(toolContext);
registerGuidedPrompts(server, analysis, session);
if (session !== undefined) {
+45
View File
@@ -0,0 +1,45 @@
import type { McpServer, ServerContext } from "@modelcontextprotocol/server";
import type { FirmwareAnalysisService } from "../application/FirmwareAnalysisService.js";
import type { BinarySessionPort } from "../application/BinarySession.js";
import { toolContract } from "../contracts/toolContracts.js";
import type { ToolContract } from "../contracts/toolContractTypes.js";
import type { FirmwareOperation } from "../domain/firmwareAnalysis.js";
import type { Logger } from "../logger.js";
import { logToolExecution } from "./toolLogging.js";
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
import { toCallToolResult } from "./toolResult.js";
/** Bind firmware handlers to their exact named schemas. */
export const registerFirmwareTools = (
server: McpServer,
service: FirmwareAnalysisService,
logger: Logger,
recordEvidence?: BinarySessionPort["recordEvidence"],
): void => {
const handler =
(contract: ToolContract<FirmwareOperation>) =>
async (input: unknown, context: ServerContext) => {
const result = await logToolExecution(logger, contract.name, () =>
service.execute(contract.name, input, {
signal: context.mcpReq.signal,
}),
);
if (!result.ok) return toCallToolResult(result, contract);
const recorded = recordEvidence?.(result.value);
return recorded !== undefined && !recorded.ok
? toCallToolResult(recorded, contract)
: toCallToolResult(result, contract);
};
const inspect = toolContract("inspect_firmware_regions");
server.registerTool(
inspect.name,
toolRegistrationOptions(inspect),
handler(inspect),
);
const extract = toolContract("extract_firmware");
server.registerTool(
extract.name,
toolRegistrationOptions(extract),
handler(extract),
);
};
+6
View File
@@ -12,6 +12,8 @@ export interface SessionAvailabilityDefaults {
| "electronAutomationEnabled"
| "v8InspectorObservationEnabled"
| "androidAnalysisEnabled"
| "firmwareInspectionEnabled"
| "firmwareExtractionEnabled"
>;
}
@@ -23,6 +25,10 @@ export const sessionAvailabilityPolicy = (
configured ??
(() => ({
processCaptureEnabled: platform !== "win32",
firmwareInspectionEnabled:
defaults.optionalFeatures?.firmwareInspectionEnabled ?? false,
firmwareExtractionEnabled:
defaults.optionalFeatures?.firmwareExtractionEnabled ?? false,
androidAnalysisEnabled:
defaults.optionalFeatures?.androidAnalysisEnabled ?? false,
browserObservationEnabled:
@@ -22,6 +22,10 @@ import { PLAYWRIGHT_BROWSER_SCENARIO_PROVIDER_IDENTITY } from "../../../src/brow
import { V8_INSPECTOR_PROVIDER_IDENTITY } from "../../../src/browser/V8InspectorProvider.js";
import { CLI_COMMAND_NAMES } from "../../../src/cliCommandNames.js";
import { createCli } from "../../../src/cli.js";
import {
BINWALK_PROVIDER_IDENTITY,
UNBLOB_PROVIDER_IDENTITY,
} from "../../../src/firmware/FirmwareRelease.js";
import { JADX_PROVIDER_IDENTITY } from "../../../src/android/JadxRelease.js";
import { TOOL_CONTRACTS } from "../../../src/contracts/toolContracts.js";
import {
@@ -73,6 +77,8 @@ describe("canonical product catalog", () => {
[
HOPPER_PROVIDER_IDENTITY,
JADX_PROVIDER_IDENTITY,
BINWALK_PROVIDER_IDENTITY,
UNBLOB_PROVIDER_IDENTITY,
GHIDRA_PROVIDER_IDENTITY,
NATIVE_MACOS_PROVIDER_IDENTITY,
ARTIFACT_GRAPH_PROVIDER,
@@ -0,0 +1,343 @@
import { access, mkdir, readFile, writeFile } from "node:fs/promises";
import { expect, it as test } from "vitest";
import {
firmwareFixture,
assertFirmwareCleanup,
} from "../../fixtures/firmware/provider.js";
import { firmwareResultSchemas } from "../../../src/domain/firmwareAnalysis.js";
import { FirmwareAnalysisService } from "../../../src/application/FirmwareAnalysisService.js";
import { FirmwareProvider } from "../../../src/firmware/FirmwareProvider.js";
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
import { toolContract } from "../../../src/contracts/toolContracts.js";
import { toCallToolResult } from "../../../src/server/toolResult.js";
const it = test.skipIf(process.platform !== "linux");
it("binds region observations to exact bytes and preserves provider-reported uncertainty", async () => {
const fixture = await firmwareFixture();
const result = await fixture.service.execute("inspect_firmware_regions", {
path: fixture.path,
});
if (!result.ok) throw result.error;
const normalized = firmwareResultSchemas.inspect_firmware_regions.parse(
result.value.normalized_result,
);
expect(normalized.regions[0]).toMatchObject({
offset: 2,
reported_size: 4,
size_basis: "provider_reported_validation_unknown",
});
expect(result.value.subject?.local_path).toBe(fixture.path);
expect(result.value.subject?.format).toBe("file");
expect(result.value.raw_result).toMatchObject({
report: [{ Analysis: { file_map: [{ id: "hit" }] } }],
});
expect(fixture.launches[1]?.args).toEqual(
expect.arrayContaining(["--as=1073741824", "--threads", "1"]),
);
await assertFirmwareCleanup(fixture.launches);
});
it.each(["wrong-input", "bounds"])(
"rejects a mismatched or out-of-bounds inspection: %s",
async (mode) => {
const fixture = await firmwareFixture(mode);
const result = await fixture.service.execute("inspect_firmware_regions", {
path: fixture.path,
});
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisOutputError" },
});
await assertFirmwareCleanup(fixture.launches);
},
);
it("extracts a selected interval with verified file identity and parent-byte lineage", async () => {
const fixture = await firmwareFixture();
const result = await fixture.service.execute("extract_firmware", {
path: fixture.path,
output_directory: fixture.output,
range: { offset: 2, length: 4 },
});
if (!result.ok) throw result.error;
const normalized = firmwareResultSchemas.extract_firmware.parse(
result.value.normalized_result,
);
expect(normalized.selection).toMatchObject({ offset: 2, length: 4 });
expect(normalized.files[0]).toMatchObject({
relative_path: "rootfs/config",
size: 14,
original_file_range: null,
runtime_address: null,
});
expect(normalized.chunks[0]?.root_file_range).toEqual({
offset: 2,
length: 4,
});
expect(normalized.derivations[0]).toMatchObject({
parent_path: "$input",
child_path: "$output/rootfs/config",
handler: "gzip",
parent_file_range: { offset: 0, length: 4 },
});
expect(await readFile(normalized.files[0]?.path ?? "", "utf8")).toBe(
"firmware=true\n",
);
expect(fixture.launches[1]?.args).toEqual(
expect.arrayContaining(["--process-num", "1", "--randomness-depth", "0"]),
);
expect(fixture.launches[1]?.args).not.toContain("--no-sandbox");
await assertFirmwareCleanup(fixture.launches);
});
it.each(["depth", "dependency", "link"])(
"reports partial extraction for %s without losing inline results",
async (mode) => {
const fixture = await firmwareFixture(mode);
const result = await fixture.service.execute("extract_firmware", {
path: fixture.path,
output_directory: fixture.output,
});
if (!result.ok) throw result.error;
const normalized = firmwareResultSchemas.extract_firmware.parse(
result.value.normalized_result,
);
expect(normalized.coverage).toBe("partial");
expect(normalized.files).toHaveLength(1);
if (mode === "depth")
expect(normalized.depth_limited_paths).toEqual(["$output/rootfs/config"]);
if (mode === "dependency")
expect(normalized.diagnostics).toEqual([
{
input_path: "$input",
report: {
__typename__: "ExtractorDependencyNotFoundReport",
severity: "ERROR",
dependencies: ["sasquatch"],
},
},
]);
if (mode === "link") {
expect(normalized.unpublished_entries).toEqual([
{
relative_path: "escape",
kind: "symlink",
link_target: "/etc/passwd",
},
]);
await expect(access(`${fixture.output}/escape`)).rejects.toMatchObject({
code: "ENOENT",
});
}
await assertFirmwareCleanup(fixture.launches);
},
);
it("distinguishes a real output file named $input from the selected original", async () => {
const fixture = await firmwareFixture("reserved-name");
const outcome = await fixture.service.execute("extract_firmware", {
path: fixture.path,
output_directory: fixture.output,
});
if (!outcome.ok) throw outcome.error;
const result = firmwareResultSchemas.extract_firmware.parse(
outcome.value.normalized_result,
);
expect(result.files[0]?.relative_path).toBe("$input");
expect(result.derivations[0]).toMatchObject({
parent_path: "$input",
child_path: "$output/$input",
});
expect(await readFile(`${fixture.output}/$input`, "utf8")).toBe(
"firmware=true\n",
);
await assertFirmwareCleanup(fixture.launches);
});
it.each(["hash", "malformed", "budget", "missing-file"])(
"rejects invalid or over-budget extraction and rolls back output: %s",
async (mode) => {
const fixture = await firmwareFixture(mode);
const result = await fixture.service.execute("extract_firmware", {
path: fixture.path,
output_directory: fixture.output,
...(mode === "budget" ? { max_output_bytes: 1000 } : {}),
});
expect(result.ok).toBe(false);
await expect(access(fixture.output)).rejects.toMatchObject({
code: "ENOENT",
});
await assertFirmwareCleanup(fixture.launches);
},
);
it("preserves existing output and rejects invalid intervals before invoking extraction", async () => {
const fixture = await firmwareFixture();
await mkdir(fixture.output);
await writeFile(`${fixture.output}/existing`, "keep");
const existing = await fixture.service.execute("extract_firmware", {
path: fixture.path,
output_directory: fixture.output,
});
expect(existing.ok).toBe(false);
expect(await readFile(`${fixture.output}/existing`, "utf8")).toBe("keep");
const previous = fixture.launches.length;
const invalid = await fixture.service.execute("extract_firmware", {
path: fixture.path,
output_directory: fixture.output,
range: { offset: 100, length: 1 },
});
expect(invalid).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError" },
});
expect(fixture.launches).toHaveLength(previous);
await assertFirmwareCleanup(fixture.launches);
});
it("retains startup stderr when Unblob exits 1 without a report", async () => {
const fixture = await firmwareFixture("startup");
const result = await fixture.service.execute("extract_firmware", {
path: fixture.path,
output_directory: fixture.output,
});
if (result.ok) throw new Error("Expected startup failure");
const details = {
diagnostics: {
stderr: expect.stringContaining("Landlock sandbox is not available"),
exit_code: 1,
},
};
expect(projectAnalysisError(result.error)).toMatchObject({ details });
expect(
toCallToolResult(result, toolContract("extract_firmware"))
.structuredContent,
).toMatchObject({ error: { details } });
expect(result).toMatchObject({
ok: false,
error: {
_tag: "ProviderAdapterError",
diagnostics: {
exit_code: 1,
stderr: expect.stringContaining("Landlock sandbox is not available"),
report_failure: expect.stringContaining("ENOENT"),
},
},
});
await expect(access(fixture.output)).rejects.toMatchObject({
code: "ENOENT",
});
await assertFirmwareCleanup(fixture.launches);
});
it("distinguishes a successful process with missing output from startup failure", async () => {
const fixture = await firmwareFixture("missing-report");
const result = await fixture.service.execute("extract_firmware", {
path: fixture.path,
output_directory: fixture.output,
});
expect(result).toMatchObject({
ok: false,
error: {
_tag: "AnalysisOutputError",
reason: expect.stringContaining("expected complete report"),
},
});
if (result.ok) throw new Error("Expected missing-output failure");
expect(projectAnalysisError(result.error)).toMatchObject({
details: { reason: expect.stringContaining("Producer forgot its report") },
});
await assertFirmwareCleanup(fixture.launches);
});
it("cancels an owned active process and permits the next queued operation", async () => {
const fixture = await firmwareFixture("stall");
const controller = new AbortController();
const pending = fixture.service.execute(
"inspect_firmware_regions",
{ path: fixture.path },
{ signal: controller.signal },
);
await expect.poll(() => fixture.launches.length).toBe(2);
controller.abort();
expect(await pending).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
const cancelled = await fixture.service.execute(
"inspect_firmware_regions",
{ path: fixture.path },
{ signal: AbortSignal.abort() },
);
expect(cancelled).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
expect(fixture.launches).toHaveLength(2);
await assertFirmwareCleanup(fixture.launches);
});
it("cancels a waiting request promptly while keeping later launches behind the active operation", async () => {
const fixture = await firmwareFixture("stall");
const activeController = new AbortController();
const active = fixture.service.execute(
"inspect_firmware_regions",
{ path: fixture.path },
{ signal: activeController.signal },
);
await expect.poll(() => fixture.launches.length).toBe(2);
const queuedController = new AbortController();
const queued = fixture.service.execute(
"inspect_firmware_regions",
{ path: fixture.path },
{ signal: queuedController.signal },
);
queuedController.abort();
expect(await queued).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
expect(fixture.launches).toHaveLength(2);
activeController.abort();
expect((await active).ok).toBe(false);
await assertFirmwareCleanup(fixture.launches);
});
it("retains an uncertain workspace and blocks new launches after cleanup failure", async () => {
const fixture = await firmwareFixture("cleanup-failure");
const first = await fixture.service.execute("inspect_firmware_regions", {
path: fixture.path,
});
expect(first).toMatchObject({
ok: false,
error: { cleanupIncomplete: true },
});
const workspace = fixture.launches[0]?.cwd;
expect(workspace).toBeDefined();
if (workspace !== undefined) await access(workspace);
const next = await fixture.service.execute("inspect_firmware_regions", {
path: fixture.path,
});
expect(next).toMatchObject({ ok: false, error: { cleanupIncomplete: true } });
expect(fixture.launches).toHaveLength(1);
});
it("rejects absent or unsupported engines without installing anything", async () => {
const missing = await new FirmwareAnalysisService(
new FirmwareProvider({}),
).execute("inspect_firmware_regions", { path: "/tmp/unused" });
expect(missing).toMatchObject({
ok: false,
error: { _tag: "AnalysisCapabilityUnavailableError" },
});
const fixture = await firmwareFixture("version");
const result = await fixture.service.execute("inspect_firmware_regions", {
path: fixture.path,
});
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisCapabilityUnavailableError" },
});
await assertFirmwareCleanup(fixture.launches);
});
@@ -94,6 +94,7 @@ describe("enhanced MCP tools", () => {
error: {
code: "unreadable_output",
category: "execution_failure",
details: { operation: "decompile", reason: "failed" },
message:
"Analysis returned an unreadable result. Retry once; if it continues, run `rea doctor`.",
retryable: false,
+55
View File
@@ -0,0 +1,55 @@
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { expect, it as test, onTestFinished } from "vitest";
import { FIRMWARE_TOOL_CONTRACTS } from "../../../src/contracts/firmwareToolContracts.js";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { createServer } from "../../../src/server/createServer.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import {
firmwareFixture,
assertFirmwareCleanup,
} from "../../fixtures/firmware/provider.js";
const it = test.skipIf(process.platform !== "linux");
it("executes named firmware contracts with inline Evidence and no native binary admission", async () => {
const fixture = await firmwareFixture();
const session = createTestBinarySession(() => {
throw new Error("Firmware inspection must not acquire a native provider");
});
const server = createServer(session, session, {
firmwareAnalysis: fixture.provider,
});
const client = new Client({ name: "firmware-regression", version: "1" });
onTestFinished(async () => {
await client.close();
await server.close();
await session.close();
});
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
await server.connect(serverTransport);
await client.connect(clientTransport);
const advertised = await client.listTools();
for (const contract of FIRMWARE_TOOL_CONTRACTS) {
expect(
advertised.tools.find((tool) => tool.name === contract.name)?.annotations,
).toMatchObject(contract.annotations);
const response = await client.callTool({
name: contract.name,
arguments: {
path: fixture.path,
...(contract.name === "extract_firmware"
? { output_directory: fixture.output }
: {}),
},
});
expect(response.isError, JSON.stringify(response)).not.toBe(true);
const result = contract.outputSchema.parse(response.structuredContent);
const evidence = parseEvidence(result.evidence);
expect(result.result).toEqual(evidence.normalized_result);
expect(result.evidence_id).toBe(evidence.evidence_id);
expect(evidence.operation).toBe(contract.name);
expect(evidence.raw_result).not.toBeNull();
}
await assertFirmwareCleanup(fixture.launches);
});
+129
View File
@@ -0,0 +1,129 @@
import { createHash } from "node:crypto";
import { mkdir, readFile, writeFile, symlink } from "node:fs/promises";
import { join } from "node:path";
const args = process.argv.slice(2);
const mode = process.env.REA_FIRMWARE_FIXTURE_MODE;
const engine = process.env.REA_FIRMWARE_FIXTURE_ENGINE;
if (args.includes("--version")) {
console.log(
mode === "version"
? "unsupported"
: engine === "binwalk"
? "binwalk 3.1.0"
: "26.6.4",
);
} else if (mode === "stall") {
setInterval(() => {}, 1000);
} else if (mode === "startup") {
console.error("Landlock sandbox is not available on this fixture host");
process.exitCode = 1;
} else if (mode === "missing-report") {
console.error("Producer forgot its report");
} else {
const input = args.at(-1);
const bytes = await readFile(input);
const sha256 = createHash("sha256").update(bytes).digest("hex");
const reportPath =
args[args.indexOf(engine === "binwalk" ? "--log" : "--report") + 1];
if (engine === "binwalk") {
await writeFile(
reportPath,
JSON.stringify([
{
Analysis: {
file_path: mode === "wrong-input" ? "/tmp/another.bin" : input,
file_map: [
{
id: "hit",
offset: 2,
size: mode === "bounds" ? bytes.length : 4,
name: "gzip",
confidence: 250,
description: "fixture signature",
always_display: false,
extraction_declined: false,
},
],
extractions: {},
},
},
]),
);
} else {
const output = args[args.indexOf("--extract-dir") + 1];
await mkdir(output, { recursive: true });
const child =
mode === "reserved-name"
? join(output, "$input")
: join(output, "rootfs", "config");
await mkdir(join(output, "rootfs"));
const content = Buffer.from("firmware=true\n");
if (mode !== "missing-file")
await writeFile(child, mode === "budget" ? Buffer.alloc(10000) : content);
if (mode === "link") await symlink("/etc/passwd", join(output, "escape"));
const childHash = createHash("sha256").update(content).digest("hex");
const depth =
mode === "depth" ? Number(args[args.indexOf("--depth") + 1]) : 1;
const stat = (path, size) => ({
__typename__: "StatReport",
path,
size,
is_file: true,
is_dir: false,
is_link: false,
link_target: null,
});
const task = (path, level, id) => ({
path,
depth: level,
blob_id: id,
is_multi_file: false,
});
const hash = (value) => ({ __typename__: "HashReport", sha256: value });
const childTask = task(child, depth, "gzip-id");
const records = [
{
task: childTask,
reports: [
stat(child, content.length),
...(mode === "depth"
? []
: [hash(mode === "hash" ? "a".repeat(64) : childHash)]),
],
subtasks: [],
},
{
task: task(input, 0, ""),
reports: [
stat(input, bytes.length),
hash(sha256),
{
__typename__: "ChunkReport",
id: "gzip-id",
start_offset: 0,
end_offset: bytes.length,
size: bytes.length,
handler_name: "gzip",
is_encrypted: false,
extraction_reports:
mode === "dependency"
? [
{
__typename__: "ExtractorDependencyNotFoundReport",
severity: "ERROR",
dependencies: ["sasquatch"],
},
]
: [],
},
],
subtasks: [childTask],
},
];
await writeFile(
reportPath,
mode === "malformed" ? "[" : JSON.stringify(records),
);
if (mode === "dependency") process.exitCode = 1;
}
}
+94
View File
@@ -0,0 +1,94 @@
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { access, rm, writeFile } from "node:fs/promises";
import { expect, onTestFinished } from "vitest";
import { FirmwareProvider } from "../../../src/firmware/FirmwareProvider.js";
import { FirmwareAnalysisService } from "../../../src/application/FirmwareAnalysisService.js";
import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js";
import { createTestTempDirectory } from "../temporaryDirectory.js";
/** Exercise real subprocess ownership with synthetic pinned producer representations. */
export const firmwareFixture = async (mode = "normal") => {
const root = await createTestTempDirectory("rea-firmware-test-");
const path = join(root, "input.bin");
await writeFile(path, Buffer.from("fixture-input-data"));
onTestFinished(() => rm(root, { recursive: true, force: true }));
const launches: {
cwd: string | undefined;
pid: number | undefined;
args: readonly string[];
}[] = [];
const provider = new FirmwareProvider(
{
...process.env,
REA_BINWALK_COMMAND: "/usr/bin/true",
REA_UNBLOB_COMMAND: process.execPath,
},
async (options) => {
const start = options.arguments.indexOf("--") + 2;
const args = options.arguments.slice(start);
const engine =
options.arguments[start - 1]?.endsWith("/true") === true
? "binwalk"
: "unblob";
const spawned = await spawnOwnedProviderProcess({
...options,
command: process.execPath,
expectedCommand: process.execPath,
arguments: [
fileURLToPath(new URL("./producer.mjs", import.meta.url)),
...args,
],
env: {
...options.env,
REA_FIRMWARE_FIXTURE_MODE: mode,
REA_FIRMWARE_FIXTURE_ENGINE: engine,
},
});
launches.push({
cwd: options.cwd,
pid: spawned.process.pid,
args: options.arguments,
});
if (mode === "cleanup-failure") {
onTestFinished(async () => {
if (options.cwd !== undefined)
await rm(options.cwd, { recursive: true, force: true });
});
return {
...spawned,
cleanup: async () => ({
cleaned: false,
reason: "injected ownership failure",
}),
};
}
return spawned;
},
);
return {
provider,
service: new FirmwareAnalysisService(provider),
path,
output: join(root, "output"),
root,
launches,
};
};
/** Verify only the acquired process groups and private runtime directories. */
export const assertFirmwareCleanup = async (
launches: Awaited<ReturnType<typeof firmwareFixture>>["launches"],
): Promise<void> => {
for (const launch of launches) {
const pid = launch.pid;
if (pid !== undefined)
expect(() => process.kill(-pid, 0)).toThrowError(
expect.objectContaining({ code: "ESRCH" }),
);
if (launch.cwd !== undefined)
await expect(access(launch.cwd)).rejects.toMatchObject({
code: "ENOENT",
});
}
};
+18
View File
@@ -40,3 +40,21 @@ are checked against fixed SHA-256 values. Fixture provenance is recorded in
`scripts/fixtures/android-apidemos.json`; the source uses Apache-2.0. This is a
static analysis fixture: no emulator, device, SDK installation or app execution
is needed. APKs and JARs are neither tracked nor included in npm packages.
## Firmware engines
| Engine | Source | Release | Pinned commit | License |
| --- | --- | --- | --- | --- |
| Binwalk | <https://github.com/ReFirmLabs/binwalk> | v3.1.0 | `4fdab3d464d97b68e0af9088df3f9e2e1545b21c` | MIT, upstream `LICENSE` |
| Unblob | <https://github.com/onekey-sec/unblob> | 26.6.4 | `1fcc7a0a584a70a96c31f5a276c20944d199a089` | MIT, upstream `LICENSE` |
Source remains unmodified in `third_party/binwalk` and `third_party/unblob`.
Release records live in `src/firmware/FirmwareRelease.ts`; REA adapters own the
CLI/report interpretation and lifecycle in `src/firmware/`. Caller-supplied
executables retain their actual launcher digest and unknown source revision;
version output alone does not establish source identity. REA does not bundle
extractors or download/install firmware engines during analysis.
On updates: inspect actual pinned CLI and report producers, preserve licenses,
update gitlinks/release records, run report regressions and the real firmware
lane. See [firmware setup and coverage](../docs/firmware-analysis.md).
Vendored Submodule
+1
Submodule third_party/binwalk added at 4fdab3d464
Vendored Submodule
+1
Submodule third_party/unblob added at 1fcc7a0a58
+1 -1
View File
@@ -37,7 +37,7 @@ const TEST_PROJECTS = [
name: "adapters",
include: [
"src/*.test.ts",
"src/{artifacts,browser,dotnet,ghidra,hopper,native,process,reference,server,windows}/**/*.test.ts",
"src/{artifacts,browser,dotnet,firmware,ghidra,hopper,native,process,reference,server,windows}/**/*.test.ts",
],
pool: "forks" as const,
maxWorkers: MAX_TEST_WORKERS,