mirror of
https://github.com/morluto/rea.git
synced 2026-10-11 21:39:55 +00:00
feat(firmware): integrate Binwalk and Unblob analysis
* feat(firmware): integrate Binwalk and Unblob analysis * fix(firmware): verify published files and namespace source references * test(firmware): isolate extractor dependency failure from process cleanup * fix(firmware): retain actionable report and cleanup diagnostics * test: assert retained batch output failure diagnostics
This commit is contained in:
N0zoM1z0
@@ -1,3 +1,9 @@
|
||||
[submodule "third_party/jadx-headless-mcp"]
|
||||
path = third_party/jadx-headless-mcp
|
||||
url = https://github.com/1013503897/jadx-headless-mcp.git
|
||||
[submodule "third_party/binwalk"]
|
||||
path = third_party/binwalk
|
||||
url = https://github.com/ReFirmLabs/binwalk.git
|
||||
[submodule "third_party/unblob"]
|
||||
path = third_party/unblob
|
||||
url = https://github.com/onekey-sec/unblob.git
|
||||
|
||||
+1
-1
@@ -9,7 +9,7 @@
|
||||
".codex/**",
|
||||
".devin/**",
|
||||
"tests/fixtures/**",
|
||||
"third_party/jadx-headless-mcp/**",
|
||||
"third_party/**",
|
||||
"tests/conformance/**/*.c",
|
||||
"tests/conformance/**/*.m",
|
||||
"tests/conformance/**/*.swift",
|
||||
|
||||
+1
-1
@@ -11,6 +11,6 @@
|
||||
"CHANGELOG.md",
|
||||
"package-lock.json",
|
||||
"_reference/**",
|
||||
"third_party/jadx-headless-mcp/**"
|
||||
"third_party/**"
|
||||
]
|
||||
}
|
||||
|
||||
+1
-1
@@ -140,6 +140,6 @@
|
||||
".codex",
|
||||
".devin",
|
||||
".factory",
|
||||
"third_party/jadx-headless-mcp"
|
||||
"third_party"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -151,6 +151,8 @@ Update either installation with `rea update`.
|
||||
|
||||
Native binary analysis requires [Hopper](https://www.hopperapp.com/) or [Ghidra](#ghidra-analysis-provider). Hopper is separate software with its own license; its demo supports analysis with vendor-defined limits. REA can use Ghidra that you have already installed.
|
||||
|
||||
Firmware region inspection and explicit extraction use caller-supplied Binwalk and Unblob on Linux. See [Firmware analysis](docs/firmware-analysis.md) for setup, provenance, resource limits and native handoff.
|
||||
|
||||
Static APK analysis uses a separately supplied headless JADX JAR and Java, with no emulator or APK execution. See [Android analysis](docs/android-analysis.md) for setup, CLI/MCP operations, coverage and public test fixtures.
|
||||
|
||||
Windows Ghidra support is experimental for native x86-64 PE applications on local NTFS. The package bundles native Job Object, private-DACL, and path-admission controls. See [Windows Ghidra P0](docs/windows-ghidra-p0.md) for prerequisites and verified scope.
|
||||
@@ -316,6 +318,7 @@ See [native investigation](docs/native-investigation.md) for keyed archives, ins
|
||||
| Native macOS utilities | 7 | Mach-O metadata, code signatures, plists, architectures, and Swift demangling without launching Hopper |
|
||||
| Artifact graph | 5 | directory and package inventories, compiled Interface Builder files, Apple asset catalogs, and extraction |
|
||||
| Managed PE/CLI | 7 | .NET identity, metadata, CIL instructions, native dependencies, reconstruction imports, and build comparisons |
|
||||
| Firmware | 2 | Linux firmware region inspection and explicit extraction |
|
||||
| Android APK | 5 | package and manifest declarations, class search, member inventories, method decompilation, and incoming static references |
|
||||
| Browser observation | 9 | page structure, network metadata, scripts, source maps, WebMCP discovery, screenshots, and capture comparisons |
|
||||
| Electron analysis | 5 | renderer observation, static app mapping, and static/runtime reconciliation |
|
||||
@@ -468,6 +471,8 @@ flowchart LR
|
||||
REA --> Browser["Browser CDP provider"]
|
||||
REA --> Android["Android static provider<br/>headless JADX adapter"]
|
||||
Android --> Runtime
|
||||
REA --> Firmware["Firmware providers<br/>Binwalk / Unblob adapters"]
|
||||
Firmware --> Runtime
|
||||
REA --> Process["Process capture provider"]
|
||||
Runtime --> Target["Target software"]
|
||||
Process --> Target
|
||||
|
||||
@@ -209,6 +209,7 @@ rea setup
|
||||
| أدوات macOS الأصلية | 7 | بيانات Mach-O والتوقيعات وplist والمعماريات واستعادة أسماء Swift |
|
||||
| الملفات والحزم | 5 | فحص الدلائل والحزم وInterface Builder وموارد Apple والاستخراج |
|
||||
| .NET PE/CLI | 7 | هوية التجميع والبيانات الوصفية وCIL والاستدعاءات الأصلية والمقارنة واستيراد إعادة البناء |
|
||||
| البرامج الثابتة | 2 | فحص مناطق البرامج الثابتة واستخراجها الصريح على Linux |
|
||||
| Android APK | 5 | بيانات الحزمة وmanifest والبحث عن الأصناف والأعضاء وفك ترجمة الأساليب والمراجع الثابتة |
|
||||
| مراقبة المتصفح | 9 | الصفحات والبرامج النصية وخرائط المصدر وWebMCP والصور ومقارنة الالتقاط |
|
||||
| تحليل Electron | 5 | مراقبة الصفحات وبنية التطبيق وربط النتائج الثابتة بنتائج التشغيل |
|
||||
|
||||
@@ -198,6 +198,7 @@ REA は手順 1〜5 のバイナリ解析を処理し、手順 6 はエージェ
|
||||
| macOS ネイティブ | 7 | Mach-O メタデータ、署名、plist、アーキテクチャ、Swift 名の復元 |
|
||||
| ファイルとパッケージ | 5 | ディレクトリとパッケージ、Interface Builder、Apple リソース、抽出 |
|
||||
| .NET PE/CLI | 7 | アセンブリ識別、メタデータ、CIL、ネイティブ呼び出し、比較、再構築の取り込み |
|
||||
| ファームウェア | 2 | Linux の領域検査と明示的な展開 |
|
||||
| Android APK | 5 | パッケージと manifest、クラス検索、メンバー一覧、メソッド逆コンパイル、静的参照 |
|
||||
| ブラウザー観察 | 9 | ページ、スクリプト、ソースマップ、WebMCP、スクリーンショット、比較 |
|
||||
| Electron 解析 | 5 | ページ観察、アプリ構造、静的・実行時結果の対応付け |
|
||||
|
||||
@@ -199,6 +199,7 @@ REA는 1–5단계의 바이너리 분석을 처리합니다. 6단계는 에이
|
||||
| macOS 네이티브 | 7 | Mach-O 메타데이터, 서명, plist, 아키텍처, Swift 이름 복원 |
|
||||
| 파일과 패키지 | 5 | 디렉터리와 패키지 검사, Interface Builder, Apple 리소스, 추출 |
|
||||
| .NET PE/CLI | 7 | 어셈블리 식별, 메타데이터, CIL, 네이티브 호출, 비교, 재구성 가져오기 |
|
||||
| 펌웨어 | 2 | Linux 펌웨어 영역 검사 및 명시적 추출 |
|
||||
| Android APK | 5 | 패키지와 manifest, 클래스 검색, 멤버 목록, 메서드 디컴파일, 정적 참조 |
|
||||
| 브라우저 관찰 | 9 | 페이지, 스크립트, 소스 맵, WebMCP, 스크린샷, 캡처 비교 |
|
||||
| Electron 분석 | 5 | 페이지 관찰, 앱 구조, 정적·런타임 결과 연결 |
|
||||
|
||||
@@ -198,6 +198,7 @@ REA 负责第 1–5 步中的二进制分析。第 6 步由智能体使用其常
|
||||
| macOS 原生工具 | 7 | Mach-O 元数据、代码签名、plist、架构与 Swift 符号还原 |
|
||||
| 制品检查 | 5 | 目录与软件包检查、Interface Builder、Apple 资源目录与提取 |
|
||||
| .NET PE/CLI | 7 | 程序集身份、元数据、CIL、原生调用声明、成员比较与重建导入 |
|
||||
| 韧体 | 2 | Linux 韧体区域识别与显式解包 |
|
||||
| Android APK | 5 | 包与 manifest 声明、class 搜索、成员清单、方法反编译与静态引用 |
|
||||
| 浏览器观察 | 9 | 页面、脚本、来源映射、WebMCP 工具、截图与捕获比较 |
|
||||
| Electron 分析 | 5 | 页面观察、应用结构与静态/运行时结果关联 |
|
||||
|
||||
@@ -25,6 +25,7 @@ graph TD
|
||||
AUX["src/artifacts/ + src/native/<br/>disjoint artifact and native capabilities"]
|
||||
BROWSER["src/browser/<br/>Loopback CDP transport<br/>exact-origin passive observation"]
|
||||
DOTNET["src/dotnet/<br/>Managed static metadata provider<br/>PE/CLI metadata inventory, members, IL"]
|
||||
FIRMWARE["src/firmware/<br/>Binwalk / Unblob adapters<br/>owned processes, report normalization, private extraction"]
|
||||
ANDROID["src/android/<br/>Static APK adapter<br/>owned SDK stdio, pinned headless JADX, normalization"]
|
||||
REFERENCE["src/reference/<br/>Filesystem reference-source reader<br/>bounded traversal, validation, errors"]
|
||||
end
|
||||
@@ -62,6 +63,9 @@ graph TD
|
||||
MAIN --> SERVER
|
||||
MAIN --> APP
|
||||
CLI --> APP
|
||||
APP --> FIRMWARE
|
||||
FIRMWARE --> PROCESS
|
||||
FIRMWARE --> CONTRACTS
|
||||
APP --> ANDROID
|
||||
ANDROID --> PROCESS
|
||||
ANDROID --> CONTRACTS
|
||||
|
||||
@@ -0,0 +1,173 @@
|
||||
# Firmware analysis
|
||||
|
||||
REA integrates **Binwalk 3.1.0** for region inspection and **Unblob 26.6.4** for
|
||||
explicit extraction. This first firmware milestone is verified on Linux x64.
|
||||
No device, root privilege, mount, emulation or execution of extracted programs
|
||||
is needed. Arbitrary firmware bytes do not pass native executable admission;
|
||||
select a returned ELF separately for supported Ghidra analysis.
|
||||
|
||||
## Caller-supplied tools
|
||||
|
||||
Provide existing tools using absolute executable paths, scoped to your shell:
|
||||
|
||||
```sh
|
||||
export REA_BINWALK_COMMAND=/absolute/path/to/binwalk
|
||||
export REA_UNBLOB_COMMAND=/absolute/path/to/unblob
|
||||
```
|
||||
|
||||
REA does not install these tools, their dependencies or system packages. It
|
||||
checks the actual tool version when executing a request. Inspection and
|
||||
extraction have independent prerequisites; one does not require the other.
|
||||
Both require util-linux `prlimit`, normally `/usr/bin/prlimit`; override its
|
||||
absolute path with `REA_FIRMWARE_PRLIMIT_COMMAND` if needed.
|
||||
|
||||
Unblob needs the external extractor for the selected format on its inherited
|
||||
`PATH`: for example, `debugfs` for ext2/ext3/ext4. Its default Landlock sandbox
|
||||
and skip rules stay enabled. Sandbox startup failures remain actionable errors;
|
||||
REA does not retry with `--no-sandbox`. Missing extractors reported in a valid
|
||||
analysis report retain partial results and dependency names inline.
|
||||
|
||||
Upstream sources are preserved unmodified as optional pinned submodules:
|
||||
|
||||
```sh
|
||||
git submodule update --init third_party/binwalk third_party/unblob
|
||||
```
|
||||
|
||||
Normal REA builds and npm packages do not include or build these sources or
|
||||
engine binaries. See [upstream provenance](../third_party/README.md).
|
||||
|
||||
## CLI and MCP
|
||||
|
||||
```sh
|
||||
rea inspect-firmware-regions /path/to/firmware.bin
|
||||
rea extract-firmware /path/to/firmware.bin /path/to/new-output
|
||||
rea extract-firmware /path/to/firmware.bin /path/to/selected-output \
|
||||
--offset 64 --length 4096 --max-depth 3
|
||||
```
|
||||
|
||||
The corresponding MCP tools accept the same intent:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_firmware_regions",
|
||||
"arguments": { "path": "/path/to/firmware.bin" }
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "extract_firmware",
|
||||
"arguments": {
|
||||
"path": "/path/to/firmware.bin",
|
||||
"output_directory": "/path/to/new-output",
|
||||
"range": { "offset": 64, "length": 4096 },
|
||||
"max_depth": 3
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Both return Evidence with the original path and SHA-256, raw producer reports,
|
||||
actual executable path/digest, version and limitations. Binwalk signature UUIDs
|
||||
and Unblob blob IDs are producer-local identities; compare content hashes and
|
||||
source ranges across runs. A matching version does not attest an installed
|
||||
build's source revision, so that field remains unknown.
|
||||
|
||||
### Interpret the results
|
||||
|
||||
- Region offsets refer to bytes in the original firmware. `reported_size`
|
||||
preserves Binwalk's output; `size_basis=provider_reported_validation_unknown`
|
||||
reflects that Binwalk can replace unknown lengths with the next signature or
|
||||
EOF before serialization. Neither a signature nor its confidence byte proves
|
||||
a valid partition, encryption or runtime behavior.
|
||||
- Extraction returns regular file paths, independently verified SHA-256 values,
|
||||
producer hashes when available, chunks, diagnostics and task derivations.
|
||||
Unblob's task report is unordered; REA resolves relationships by paths and
|
||||
blob IDs. Logical references use `$input`, `$output`, or `$output/<relative-path>`
|
||||
so a real file named `$input` stays distinct from the original input.
|
||||
Chunk offsets belong to the task's input file. Only chunks in the
|
||||
selected input have a known `root_file_range`, adjusted by the selected offset.
|
||||
- Decompressed children have unknown original byte ranges and runtime addresses.
|
||||
Task lineage does not establish a byte-for-byte mapping through decompression.
|
||||
- `partial` marks unknown/encrypted chunks, reported extraction problems,
|
||||
unpublished links/special files or reached depth limits. `complete` means the
|
||||
engine finished its configured extraction without those reported conditions;
|
||||
default skip rules and format coverage still apply. It does not establish
|
||||
complete firmware reconstruction.
|
||||
- The destination must be absent, absolute, with an existing parent. Publication
|
||||
uses exclusive, symlink-resistant writes and durable digest readback. Existing
|
||||
destinations are never overwritten. This is not an atomic directory rename:
|
||||
a new tree can be visible while files are written. Failed uncommitted output
|
||||
is rolled back. Symlinks and special files are recorded without publication;
|
||||
empty directories, ownership and executable permission bits are not reproduced.
|
||||
|
||||
Select a verified native child for the existing tools:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "open_binary",
|
||||
"arguments": {
|
||||
"path": "/path/from/extraction/usr/bin/program",
|
||||
"provider_id": "ghidra"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then use `analyze_function`, `inspect_native_api` or the other native primitives.
|
||||
Native architecture and executable validation still apply. This milestone does
|
||||
not add MIPS/PPC/RISC-V decompilation, raw flash load profiles, bootloader/device
|
||||
tree semantics, vulnerability scanning, emulation or hardware runtime capture.
|
||||
|
||||
## Resource and lifecycle policy
|
||||
|
||||
- Nonempty regular firmware inputs: at most **128 MiB**, copied and hashed into
|
||||
a private workspace before analysis. No in-place source modification.
|
||||
- One queued heavy operation per provider instance; Binwalk one thread and
|
||||
Unblob one worker. No automatic parallel extraction engines.
|
||||
- **120-second** deadline; inherited **1 GiB per-process address-space** limit,
|
||||
120 CPU seconds and zero core-dump allowance through `prlimit`.
|
||||
- JSON reports: at most **8 MiB**; oversized reports fail without a truncated
|
||||
success. Diagnostic output is bounded with exact captured byte counts.
|
||||
- Default staging budget: **256 MiB** and **10,000 entries**, configurable using
|
||||
`max_output_bytes` / `max_output_files` (CLI `--max-output-bytes` /
|
||||
`--max-output-files`). Depth defaults to 3, allowed range 1–10.
|
||||
|
||||
Staging budgets are polled and rechecked after exit; short-lived overshoot is
|
||||
possible. The inherited file-size limit bounds each file, not total storage;
|
||||
address-space limits are per process, not an aggregate memory quota. Worker
|
||||
count is not CPU affinity for every external extractor. These limits do not
|
||||
claim cgroup/container containment. Cancellation cleans the owned process group
|
||||
before removing its workspace; uncertain cleanup retains resources and blocks
|
||||
further operations in that provider instance.
|
||||
|
||||
## Verification
|
||||
|
||||
Generate small public, source-derived fixtures into ignored `_reference/`:
|
||||
|
||||
```sh
|
||||
npm run fixtures:firmware
|
||||
npm run verify:firmware
|
||||
```
|
||||
|
||||
Fixture generation requires existing Python 3 and a host C compiler (`cc`);
|
||||
override with `REA_FIRMWARE_PYTHON` / `REA_FIRMWARE_CC`. The independent oracle
|
||||
records gzip offset/length and extracted file hashes. Real verification checks
|
||||
CLI/MCP, full-file and selected-range extraction, unknown chunks, depth limits,
|
||||
content parity and unchanged source bytes. No binary fixture is committed.
|
||||
|
||||
Additional lanes are explicit, so the base lane stays small:
|
||||
|
||||
```sh
|
||||
# Existing mke2fs and debugfs required; put debugfs on the scoped PATH.
|
||||
REA_FIRMWARE_VERIFY_EXT4=1 npm run fixtures:firmware
|
||||
REA_FIRMWARE_VERIFY_EXT4=1 npm run verify:firmware
|
||||
|
||||
# Existing Ghidra 12.1.4 and Java required.
|
||||
REA_FIRMWARE_VERIFY_GHIDRA=1 npm run verify:firmware
|
||||
```
|
||||
|
||||
`REA_FIRMWARE_MKE2FS_COMMAND` selects the optional ext4 fixture producer.
|
||||
`REA_FIRMWARE_FIXTURE_ROOT` selects generated data storage.
|
||||
`REA_FIRMWARE_TEST_ENTRYPOINT` can select an isolated installed npm package's
|
||||
`scripts/rea.mjs` to verify the packaged CLI/MCP workflow. Real coverage here is
|
||||
generated gzip/USTAR/ext4 with a Linux x64 ELF child; SquashFS, UBIFS, proprietary
|
||||
firmware, other hosts and architectures remain unverified.
|
||||
@@ -10,7 +10,7 @@
|
||||
"skill_version": "24"
|
||||
},
|
||||
"tools": {
|
||||
"total": 123,
|
||||
"total": 125,
|
||||
"families": [
|
||||
{
|
||||
"id": "direct",
|
||||
@@ -121,6 +121,12 @@
|
||||
"verify_managed_native_boundaries"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "firmware",
|
||||
"surface": "firmware-provider",
|
||||
"count": 2,
|
||||
"tools": ["extract_firmware", "inspect_firmware_regions"]
|
||||
},
|
||||
{
|
||||
"id": "android",
|
||||
"surface": "android-provider",
|
||||
@@ -215,6 +221,12 @@
|
||||
]
|
||||
},
|
||||
"providers": [
|
||||
{
|
||||
"id": "binwalk",
|
||||
"name": "Binwalk",
|
||||
"version": "3.1.0",
|
||||
"capabilities": ["inspect_firmware_regions"]
|
||||
},
|
||||
{
|
||||
"id": "ghidra",
|
||||
"name": "Ghidra",
|
||||
@@ -418,6 +430,12 @@
|
||||
"list_javascript_runtime_targets",
|
||||
"observe_javascript_runtime"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "unblob",
|
||||
"name": "Unblob",
|
||||
"version": "26.6.4",
|
||||
"capabilities": ["extract_firmware"]
|
||||
}
|
||||
],
|
||||
"setup_clients": [
|
||||
@@ -495,7 +513,7 @@
|
||||
}
|
||||
],
|
||||
"cli": {
|
||||
"primary_count": 78,
|
||||
"primary_count": 80,
|
||||
"commands": [
|
||||
"address-to-file-offset",
|
||||
"analyze",
|
||||
@@ -526,6 +544,7 @@
|
||||
"evidence-export",
|
||||
"evidence-import",
|
||||
"extract-artifact",
|
||||
"extract-firmware",
|
||||
"function",
|
||||
"import-managed-reconstruction",
|
||||
"import-reference-source",
|
||||
@@ -536,6 +555,7 @@
|
||||
"inspect-artifact",
|
||||
"inspect-asset-catalog",
|
||||
"inspect-electron-page",
|
||||
"inspect-firmware-regions",
|
||||
"inspect-keyed-archive",
|
||||
"inspect-macho",
|
||||
"inspect-managed-artifact",
|
||||
@@ -585,15 +605,15 @@
|
||||
},
|
||||
"runtime_catalog": {
|
||||
"counts": {
|
||||
"cli_commands": 78,
|
||||
"mcp_tools": 123,
|
||||
"cli_commands": 80,
|
||||
"mcp_tools": 125,
|
||||
"mcp_prompts": 6
|
||||
},
|
||||
"digests": {
|
||||
"tools_sha256": "6e60913fad4c67f3a12aa8bf5113971ce6a57e9492ce402c8cd9e539d6fde01f",
|
||||
"tools_sha256": "45a8aea00dba2a8ad8563b4943cd177d07b14889bc9b823bc39b89b6ddb26d2c",
|
||||
"prompts_sha256": "ef2de8814390b340623c32001610aa2709d90bed51537b6e562ab3abf144ab4d",
|
||||
"combined_sha256": "d5fbda3a283916ab3489a62fe88aeb89da8b0a7b19c248e3742c970b1b51600e",
|
||||
"providers_sha256": "e60c6e949f499fe8a015a20ff36752516ce57edcb1ce355f0308361c8d322dd6"
|
||||
"combined_sha256": "297aced29e7ce5e845c505d5f2e9e2dee42b7ed3e628f088f424fe8088c9e8dc",
|
||||
"providers_sha256": "dfba0ee59f12c82cdfb5320e7e0d70a114e533b2cb88afd15db9b9a4a175c9b0"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -311,3 +311,15 @@ allows a host-permission-boundary-only result and explicitly reports
|
||||
Both commands reject a changed executable digest and clean up the fixture
|
||||
process and helper. These lanes require an interactive macOS desktop. See [native investigation](native-investigation.md)
|
||||
for the exact ABI, authority, graph and observation boundaries.
|
||||
|
||||
### Firmware adapters
|
||||
|
||||
`npm run fixtures:firmware` uses existing Python 3 and a host C compiler to make
|
||||
an ignored gzip/USTAR firmware fixture and independent offset/hash oracle.
|
||||
`npm run verify:firmware` requires caller-supplied Binwalk 3.1.0, Unblob 26.6.4
|
||||
and util-linux prlimit on Linux. It verifies CLI/MCP parity, selected ranges,
|
||||
unknown chunks, depth limits and extracted child digests. The optional
|
||||
`REA_FIRMWARE_VERIFY_EXT4=1` lane requires existing mke2fs/debugfs; the separate
|
||||
`REA_FIRMWARE_VERIFY_GHIDRA=1` lane checks a selected host ELF through real Ghidra.
|
||||
Neither optional toolchain is a base-lane prerequisite. See
|
||||
[firmware analysis](firmware-analysis.md) for limits and unverified formats.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"manifest_id": "ecm_42bc9f06a00e9c510cb57605d7f1cde613e2762ea0c59bf3915e032147a7c493",
|
||||
"manifest_id": "ecm_18ee96f3c7c08df6a29b2075b8dadf6e7053e88c52a65cecb35aea374112cbc6",
|
||||
"verifier": {
|
||||
"id": "rea-managed-conformance",
|
||||
"version": "1"
|
||||
@@ -13,7 +13,7 @@
|
||||
"skill_digests": [
|
||||
{
|
||||
"skill_id": "reverse-engineer-anything",
|
||||
"sha256": "ace0a167b9161d9d96c9f5cd65ff0bb39e483c3426e6cc86aac5c7c9893c31cb"
|
||||
"sha256": "f72e03be44a47ba176e6aac47c85e930fe1fe5723b650dacbd5873f28a3b3f8f"
|
||||
}
|
||||
],
|
||||
"claims": [
|
||||
|
||||
@@ -136,6 +136,8 @@
|
||||
"verify:javascript:digests": "npm run build:cached && node scripts/verify-large-javascript-digests.mjs",
|
||||
"verify:application-workflows": "npm run build:cached && node scripts/verify-local-application-workflows.mjs",
|
||||
"verify:managed": "npm run build:cached && node scripts/verify-managed-conformance.mjs",
|
||||
"verify:firmware": "npm run build:cached && node scripts/verify-real-firmware.mjs",
|
||||
"fixtures:firmware": "node scripts/generate-firmware-fixtures.mjs",
|
||||
"verify:android": "npm run build:cached && node scripts/verify-real-android.mjs",
|
||||
"fixtures:android": "node scripts/fetch-android-fixtures.mjs",
|
||||
"verify:agent": "npm run build:cached && node scripts/verify-agent-experience.mjs",
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const root = resolve(
|
||||
process.env.REA_FIRMWARE_FIXTURE_ROOT ??
|
||||
"_reference/firmware-integration/generated",
|
||||
);
|
||||
for (const [command, args, purpose] of [
|
||||
[
|
||||
process.env.REA_FIRMWARE_PYTHON ?? "python3",
|
||||
["--version"],
|
||||
"Python 3 standard-library fixture generator",
|
||||
],
|
||||
[
|
||||
process.env.REA_FIRMWARE_CC ?? "cc",
|
||||
["--version"],
|
||||
"host ELF fixture compiler",
|
||||
],
|
||||
]) {
|
||||
try {
|
||||
await exec(command, args);
|
||||
} catch (cause) {
|
||||
throw new Error(`fixtures:firmware requires ${purpose}: ${command}`, {
|
||||
cause,
|
||||
});
|
||||
}
|
||||
}
|
||||
const ext4 =
|
||||
process.env.REA_FIRMWARE_VERIFY_EXT4 === "1"
|
||||
? [process.env.REA_FIRMWARE_MKE2FS_COMMAND ?? "mke2fs"]
|
||||
: [];
|
||||
for (const command of ext4) await exec(command, ["-V"]);
|
||||
const generated = await exec(process.env.REA_FIRMWARE_PYTHON ?? "python3", [
|
||||
fileURLToPath(new URL("./lib/generate-firmware-fixture.py", import.meta.url)),
|
||||
root,
|
||||
process.env.REA_FIRMWARE_CC ?? "cc",
|
||||
...ext4,
|
||||
]);
|
||||
console.log(generated.stdout.trim());
|
||||
@@ -31,6 +31,11 @@ export const toolFamilyCatalog = (sources) => {
|
||||
...sources.managedWorkflowContracts.MANAGED_WORKFLOW_TOOL_CONTRACTS,
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "firmware",
|
||||
surface: "firmware-provider",
|
||||
contracts: sources.firmwareContracts.FIRMWARE_TOOL_CONTRACTS,
|
||||
},
|
||||
{
|
||||
id: "android",
|
||||
surface: "android-provider",
|
||||
@@ -118,6 +123,18 @@ export const providerCatalog = (sources) => {
|
||||
contracts:
|
||||
sources.managedWorkflowContracts.MANAGED_WORKFLOW_TOOL_CONTRACTS,
|
||||
},
|
||||
{
|
||||
identity: sources.firmwareProvider.BINWALK_PROVIDER_IDENTITY,
|
||||
contracts: sources.firmwareContracts.FIRMWARE_TOOL_CONTRACTS.filter(
|
||||
({ name }) => name === "inspect_firmware_regions",
|
||||
),
|
||||
},
|
||||
{
|
||||
identity: sources.firmwareProvider.UNBLOB_PROVIDER_IDENTITY,
|
||||
contracts: sources.firmwareContracts.FIRMWARE_TOOL_CONTRACTS.filter(
|
||||
({ name }) => name === "extract_firmware",
|
||||
),
|
||||
},
|
||||
{
|
||||
identity: sources.androidProvider.JADX_PROVIDER_IDENTITY,
|
||||
contracts: sources.androidContracts.ANDROID_TOOL_CONTRACTS,
|
||||
|
||||
@@ -43,6 +43,8 @@ const SOURCE_PATHS = {
|
||||
nativeContracts: "dist/contracts/nativeToolContracts.js",
|
||||
artifactContracts: "dist/contracts/artifactToolContracts.js",
|
||||
managedContracts: "dist/contracts/managedToolContracts.js",
|
||||
firmwareContracts: "dist/contracts/firmwareToolContracts.js",
|
||||
firmwareProvider: "dist/firmware/FirmwareRelease.js",
|
||||
androidContracts: "dist/contracts/androidToolContracts.js",
|
||||
androidProvider: "dist/android/JadxRelease.js",
|
||||
managedWorkflowContracts: "dist/contracts/managedWorkflowToolContracts.js",
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
"""Generate firmware bytes using standard format producers, with an independent oracle."""
|
||||
import gzip
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tarfile
|
||||
|
||||
root = Path(sys.argv[1]).resolve()
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
source = root / "probe.c"
|
||||
source.write_text("int firmware_probe(int x){return x*7+3;} int main(void){return firmware_probe(4);}\n")
|
||||
subprocess.run([sys.argv[2], "-O0", "-g", "-fno-inline", str(source), "-o", str(root / "probe")], check=True)
|
||||
payloads = {"etc/rea.conf": b"rea_fixture=true\n", "usr/bin/probe": (root / "probe").read_bytes()}
|
||||
buffer = io.BytesIO()
|
||||
with tarfile.open(fileobj=buffer, mode="w", format=tarfile.USTAR_FORMAT) as archive:
|
||||
for name, data in payloads.items():
|
||||
info = tarfile.TarInfo(name)
|
||||
info.size = len(data)
|
||||
info.mode = 0o644
|
||||
info.mtime = 0
|
||||
archive.addfile(info, io.BytesIO(data))
|
||||
compressed = gzip.compress(buffer.getvalue(), mtime=0)
|
||||
image = b"REA firmware fixture\n".ljust(64, b"\xff") + compressed + b"\x00" * 128
|
||||
(root / "firmware.bin").write_bytes(image)
|
||||
(root / "oracle.json").write_text(json.dumps({
|
||||
"generator": "Python tarfile USTAR + gzip; host cc; no target execution",
|
||||
"offset": 64, "length": len(compressed), "size": len(image),
|
||||
"sha256": hashlib.sha256(image).hexdigest(),
|
||||
"files": {name: {"sha256": hashlib.sha256(data).hexdigest(), "size": len(data)} for name, data in payloads.items()},
|
||||
}, indent=2) + "\n")
|
||||
print(root)
|
||||
|
||||
if len(sys.argv) > 3:
|
||||
tree = root / "ext4-root"
|
||||
tree.mkdir(exist_ok=True)
|
||||
for name, data in payloads.items():
|
||||
target = tree / name
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_bytes(data)
|
||||
subprocess.run([sys.argv[3], "-q", "-t", "ext4", "-F", "-d", str(tree), str(root / "ext4.bin"), "8192"], check=True)
|
||||
@@ -0,0 +1,254 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import {
|
||||
access,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readFile,
|
||||
rm,
|
||||
symlink,
|
||||
} from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { firmwareResultSchemas } from "../dist/domain/firmwareAnalysis.js";
|
||||
import { parseEvidence } from "../dist/domain/evidence.js";
|
||||
import { hashFirmwareFile } from "../dist/firmware/FirmwareFiles.js";
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const repository = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const entrypoint = resolve(
|
||||
process.env.REA_FIRMWARE_TEST_ENTRYPOINT ??
|
||||
join(repository, "scripts/rea.mjs"),
|
||||
);
|
||||
if (process.platform !== "linux")
|
||||
throw new Error(
|
||||
"verify:firmware requires Linux; other hosts are not verified",
|
||||
);
|
||||
for (const variable of ["REA_BINWALK_COMMAND", "REA_UNBLOB_COMMAND"]) {
|
||||
const path = process.env[variable];
|
||||
if (path === undefined)
|
||||
throw new Error(
|
||||
`verify:firmware requires ${variable}; see docs/firmware-analysis.md`,
|
||||
);
|
||||
await access(path);
|
||||
}
|
||||
const fixtureRoot = resolve(
|
||||
process.env.REA_FIRMWARE_FIXTURE_ROOT ??
|
||||
join(repository, "_reference/firmware-integration/generated"),
|
||||
);
|
||||
await access(join(fixtureRoot, "oracle.json")).catch((cause) => {
|
||||
throw new Error(
|
||||
"verify:firmware requires generated fixtures; run npm run fixtures:firmware",
|
||||
{ cause },
|
||||
);
|
||||
});
|
||||
const oracle = JSON.parse(
|
||||
await readFile(join(fixtureRoot, "oracle.json"), "utf8"),
|
||||
);
|
||||
const path = join(fixtureRoot, "firmware.bin");
|
||||
assert.equal(await hashFirmwareFile(path), oracle.sha256);
|
||||
const root = await mkdtemp(join(tmpdir(), "rea-real-firmware-"));
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries(process.env).filter(([, value]) => value !== undefined),
|
||||
);
|
||||
const client = new Client({ name: "rea-real-firmware-verifier", version: "1" });
|
||||
const cli = async (args, env = environment) => {
|
||||
const response = await exec(
|
||||
process.execPath,
|
||||
[entrypoint, ...args, "--format", "json"],
|
||||
{
|
||||
env,
|
||||
cwd: repository,
|
||||
timeout: 150_000,
|
||||
maxBuffer: 16 * 1024 * 1024,
|
||||
},
|
||||
);
|
||||
return parseEvidence(JSON.parse(response.stdout));
|
||||
};
|
||||
const call = async (name, args) => {
|
||||
const response = await client.callTool(
|
||||
{ name, arguments: args },
|
||||
{ timeout: 150_000 },
|
||||
);
|
||||
assert.notEqual(response.isError, true, JSON.stringify(response));
|
||||
assert.ok(response.structuredContent !== undefined);
|
||||
return response.structuredContent;
|
||||
};
|
||||
const verifyRegions = (evidence) => {
|
||||
assert.equal(evidence.subject.digest.sha256, oracle.sha256);
|
||||
const result = firmwareResultSchemas.inspect_firmware_regions.parse(
|
||||
evidence.normalized_result,
|
||||
);
|
||||
const gzip = result.regions.find(
|
||||
(region) => region.signature === "gzip" && region.offset === oracle.offset,
|
||||
);
|
||||
assert.ok(
|
||||
gzip !== undefined,
|
||||
"Binwalk must find the independently generated gzip offset",
|
||||
);
|
||||
assert.equal(gzip.reported_size, oracle.length);
|
||||
assert.equal(gzip.size_basis, "provider_reported_validation_unknown");
|
||||
return result.regions.map(({ provider_id, ...region }) => region);
|
||||
};
|
||||
const verifyFiles = async (evidence) => {
|
||||
const result = firmwareResultSchemas.extract_firmware.parse(
|
||||
evidence.normalized_result,
|
||||
);
|
||||
assert.ok(result.chunks.some((chunk) => chunk.handler === "gzip"));
|
||||
for (const [suffix, expected] of Object.entries(oracle.files)) {
|
||||
const file = result.files.find((candidate) =>
|
||||
candidate.relative_path.endsWith(`/${suffix}`),
|
||||
);
|
||||
assert.ok(file !== undefined, `Missing extracted ${suffix}`);
|
||||
assert.equal(file.sha256, expected.sha256);
|
||||
assert.equal(file.size, expected.size);
|
||||
assert.equal(await hashFirmwareFile(file.path), expected.sha256);
|
||||
assert.equal(file.original_file_range, null);
|
||||
assert.equal(file.runtime_address, null);
|
||||
}
|
||||
return result;
|
||||
};
|
||||
try {
|
||||
const inspected = await cli(["inspect-firmware-regions", path]);
|
||||
const cliRegions = verifyRegions(inspected);
|
||||
const extracted = await cli([
|
||||
"extract-firmware",
|
||||
path,
|
||||
join(root, "cli-output"),
|
||||
]);
|
||||
const cliFiles = await verifyFiles(extracted);
|
||||
assert.equal(cliFiles.coverage, "partial");
|
||||
assert.ok(
|
||||
cliFiles.chunks.some(
|
||||
(chunk) =>
|
||||
chunk.handler === null &&
|
||||
chunk.range.offset === 0 &&
|
||||
chunk.range.length === 64,
|
||||
),
|
||||
);
|
||||
await client.connect(
|
||||
new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [entrypoint, "mcp"],
|
||||
cwd: repository,
|
||||
env: environment,
|
||||
stderr: "pipe",
|
||||
}),
|
||||
{ timeout: 30_000 },
|
||||
);
|
||||
const inspectedMcp = await call("inspect_firmware_regions", { path });
|
||||
const mcpEvidence = parseEvidence(inspectedMcp.evidence);
|
||||
assert.deepEqual(verifyRegions(mcpEvidence), cliRegions);
|
||||
const extractedMcp = await call("extract_firmware", {
|
||||
path,
|
||||
output_directory: join(root, "mcp-output"),
|
||||
});
|
||||
const mcpFiles = await verifyFiles(parseEvidence(extractedMcp.evidence));
|
||||
const contentIdentity = (result) =>
|
||||
result.files.map(({ relative_path, sha256, size }) => ({
|
||||
relative_path,
|
||||
sha256,
|
||||
size,
|
||||
}));
|
||||
assert.deepEqual(contentIdentity(mcpFiles), contentIdentity(cliFiles));
|
||||
const selected = await call("extract_firmware", {
|
||||
path,
|
||||
output_directory: join(root, "selected-output"),
|
||||
range: { offset: oracle.offset, length: oracle.length },
|
||||
});
|
||||
const selectedFiles = await verifyFiles(parseEvidence(selected.evidence));
|
||||
assert.equal(selectedFiles.selection.offset, oracle.offset);
|
||||
assert.equal(selectedFiles.selection.length, oracle.length);
|
||||
const depth = await call("extract_firmware", {
|
||||
path,
|
||||
output_directory: join(root, "depth-output"),
|
||||
max_depth: 1,
|
||||
});
|
||||
const depthResult = firmwareResultSchemas.extract_firmware.parse(
|
||||
depth.result,
|
||||
);
|
||||
assert.equal(depthResult.coverage, "partial");
|
||||
assert.ok(depthResult.depth_limited_paths.length > 0);
|
||||
console.log(
|
||||
"PASS real Binwalk/Unblob: CLI, MCP, independent offsets/digests, range extraction, unknown chunks and depth limits",
|
||||
);
|
||||
if (process.env.REA_FIRMWARE_VERIFY_EXT4 === "1") {
|
||||
const ext4 = join(fixtureRoot, "ext4.bin");
|
||||
await access(ext4);
|
||||
const response = await call("extract_firmware", {
|
||||
path: ext4,
|
||||
output_directory: join(root, "ext4-output"),
|
||||
});
|
||||
const result = firmwareResultSchemas.extract_firmware.parse(
|
||||
response.result,
|
||||
);
|
||||
assert.ok(result.chunks.some((chunk) => chunk.handler === "extfs"));
|
||||
assert.equal(result.coverage, "complete");
|
||||
for (const [suffix, expected] of Object.entries(oracle.files))
|
||||
assert.equal(
|
||||
result.files.find((file) => file.relative_path.endsWith(`/${suffix}`))
|
||||
?.sha256,
|
||||
expected.sha256,
|
||||
);
|
||||
console.log("PASS real ext4 extraction with debugfs; no mounts");
|
||||
// Preserve the process-ownership utility while withholding all extractors.
|
||||
const withoutExtractors = join(root, "without-extractors");
|
||||
await access("/bin/ps");
|
||||
await mkdir(withoutExtractors);
|
||||
await symlink("/bin/ps", join(withoutExtractors, "ps"));
|
||||
const missing = await cli(
|
||||
["extract-firmware", ext4, join(root, "missing-extractor-output")],
|
||||
{ ...environment, PATH: withoutExtractors },
|
||||
);
|
||||
const partial = firmwareResultSchemas.extract_firmware.parse(
|
||||
missing.normalized_result,
|
||||
);
|
||||
assert.equal(partial.coverage, "partial");
|
||||
assert.ok(JSON.stringify(partial.diagnostics).includes("debugfs"));
|
||||
console.log(
|
||||
"PASS real missing-extractor diagnostics retained despite Unblob exit 1",
|
||||
);
|
||||
}
|
||||
if (process.env.REA_FIRMWARE_VERIFY_GHIDRA === "1") {
|
||||
if (process.env.GHIDRA_INSTALL_DIR === undefined)
|
||||
throw new Error(
|
||||
"Firmware Ghidra handoff requires GHIDRA_INSTALL_DIR and Java; see docs/ghidra.md",
|
||||
);
|
||||
const child = mcpFiles.files.find((file) =>
|
||||
file.relative_path.endsWith("/usr/bin/probe"),
|
||||
);
|
||||
assert.ok(child !== undefined);
|
||||
await call("open_binary", { path: child.path, provider_id: "ghidra" });
|
||||
const dossier = await call("analyze_function", {
|
||||
procedure: "firmware_probe",
|
||||
});
|
||||
const evidence = parseEvidence(dossier.evidence);
|
||||
assert.equal(
|
||||
evidence.subject.digest.sha256,
|
||||
oracle.files["usr/bin/probe"].sha256,
|
||||
);
|
||||
const pseudocode = await call("procedure_pseudo_code", {
|
||||
procedure: "firmware_probe",
|
||||
});
|
||||
assert.equal(parseEvidence(pseudocode.evidence).provider.id, "ghidra");
|
||||
assert.ok(
|
||||
JSON.stringify(evidence.normalized_result).includes("firmware_probe"),
|
||||
);
|
||||
await call("close_binary", {});
|
||||
console.log(
|
||||
"PASS real Ghidra handoff of the selected extracted host ELF, bound to its digest",
|
||||
);
|
||||
}
|
||||
assert.equal(
|
||||
await hashFirmwareFile(path),
|
||||
oracle.sha256,
|
||||
"Original firmware bytes must remain unchanged",
|
||||
);
|
||||
} finally {
|
||||
await client.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
@@ -3,8 +3,8 @@ name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, and browser applications with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "24"
|
||||
tool_count: 123
|
||||
catalog_digest: "d5fbda3a283916ab3489a62fe88aeb89da8b0a7b19c248e3742c970b1b51600e"
|
||||
tool_count: 125
|
||||
catalog_digest: "297aced29e7ce5e845c505d5f2e9e2dee42b7ed3e628f088f424fe8088c9e8dc"
|
||||
---
|
||||
|
||||
# REA
|
||||
|
||||
@@ -55,6 +55,27 @@ const entry = (
|
||||
};
|
||||
|
||||
describe("capability inventory: provider status", () => {
|
||||
it("reports firmware inspection and extraction prerequisites independently", () => {
|
||||
const policy = {
|
||||
processCaptureEnabled: true,
|
||||
firmwareInspectionEnabled: true,
|
||||
};
|
||||
expect(entry("inspect_firmware_regions", status(), policy)).toMatchObject({
|
||||
available: true,
|
||||
surface: "firmware-provider",
|
||||
});
|
||||
expect(entry("extract_firmware", status(), policy)).toMatchObject({
|
||||
available: false,
|
||||
reason: "provider_missing",
|
||||
remediation: expect.stringContaining("REA_UNBLOB_COMMAND"),
|
||||
});
|
||||
expect(
|
||||
entry("extract_firmware", status(), {
|
||||
processCaptureEnabled: true,
|
||||
firmwareExtractionEnabled: true,
|
||||
}),
|
||||
).toMatchObject({ available: true });
|
||||
});
|
||||
it("keeps Android APK operations target-free and reports their explicit engine prerequisite", () => {
|
||||
expect(
|
||||
entry("inspect_android_package", status(), {
|
||||
|
||||
@@ -28,6 +28,8 @@ type ToolAvailabilityReason = "available" | ToolUnavailabilityReason;
|
||||
type ProviderDescriptor = ProviderCapability;
|
||||
export type AvailabilityPolicy = {
|
||||
readonly processCaptureEnabled: boolean;
|
||||
readonly firmwareInspectionEnabled?: boolean;
|
||||
readonly firmwareExtractionEnabled?: boolean;
|
||||
readonly androidAnalysisEnabled?: boolean;
|
||||
readonly browserObservationEnabled?: boolean;
|
||||
readonly browserScenarioEnabled?: boolean;
|
||||
@@ -206,6 +208,19 @@ const workflowAvailabilityFor = ({
|
||||
kind,
|
||||
policy,
|
||||
}: AvailabilityContext): Availability | null => {
|
||||
if (kind === "firmware-provider") {
|
||||
const enabled =
|
||||
name === "inspect_firmware_regions"
|
||||
? policy.firmwareInspectionEnabled
|
||||
: policy.firmwareExtractionEnabled;
|
||||
return enabled === true
|
||||
? { reason: "available", remediation: null }
|
||||
: {
|
||||
reason: "provider_missing",
|
||||
remediation:
|
||||
"On Linux, provide an absolute REA_BINWALK_COMMAND (Binwalk 3.1.0) or REA_UNBLOB_COMMAND (Unblob 26.6.4) path and util-linux prlimit. Extraction also requires the selected format’s external extractor.",
|
||||
};
|
||||
}
|
||||
if (kind === "android-provider")
|
||||
return policy.androidAnalysisEnabled === true
|
||||
? { reason: "available", remediation: null }
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
import type {
|
||||
AnalysisExecution,
|
||||
ExecutionOptions,
|
||||
} from "./AnalysisProvider.js";
|
||||
import type { FirmwareRequest } from "../domain/firmwareAnalysis.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import type { Result } from "../domain/result.js";
|
||||
|
||||
/** File-oriented firmware engines; executable admission remains a separate operation. */
|
||||
export interface FirmwareAnalysisPort {
|
||||
execute(
|
||||
request: FirmwareRequest,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>>;
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
import type { ExecutionOptions } from "./AnalysisProvider.js";
|
||||
import type { FirmwareAnalysisPort } from "./FirmwareAnalysisPort.js";
|
||||
import {
|
||||
firmwareRequestSchema,
|
||||
type FirmwareOperation,
|
||||
} from "../domain/firmwareAnalysis.js";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisInputError,
|
||||
AnalysisOutputError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { createEvidence, type Evidence } from "../domain/evidence.js";
|
||||
import { jsonObjectSchema } from "../domain/jsonValue.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
/** Shared CLI/MCP validation and Evidence composition for arbitrary firmware files. */
|
||||
export class FirmwareAnalysisService {
|
||||
constructor(readonly provider: FirmwareAnalysisPort) {}
|
||||
|
||||
/** Execute an explicit inspection or extraction and preserve its producing identity. */
|
||||
async execute(
|
||||
operation: FirmwareOperation,
|
||||
input: unknown,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<Evidence, AnalysisError>> {
|
||||
if (options?.signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
const parsed = firmwareRequestSchema.safeParse({ operation, input });
|
||||
if (!parsed.success)
|
||||
return err(new AnalysisInputError(operation, { cause: parsed.error }));
|
||||
const executed = await this.provider.execute(parsed.data, options);
|
||||
if (!executed.ok) return executed;
|
||||
const execution = executed.value;
|
||||
if (execution.subject === null)
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
"Firmware execution did not bind its source artifact",
|
||||
),
|
||||
);
|
||||
return ok(
|
||||
createEvidence(execution.subject, execution.provider, {
|
||||
operation,
|
||||
parameters: jsonObjectSchema.parse(parsed.data.input),
|
||||
result: execution.result,
|
||||
rawResult: execution.rawResult,
|
||||
locations: execution.locations,
|
||||
limitations: execution.limitations,
|
||||
confidence: "observed",
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -7,6 +7,7 @@ import { registerCoreAnalysisCommands } from "./cli/coreAnalysisCommands.js";
|
||||
import { registerUtilityCommands } from "./cli/utilityCommands.js";
|
||||
import { registerArtifactCommands } from "./cli/artifactCommands.js";
|
||||
import { registerManagedCommands } from "./cli/managedCommands.js";
|
||||
import { registerFirmwareCommands } from "./cli/firmwareCommands.js";
|
||||
import { registerAndroidCommands } from "./cli/androidCommands.js";
|
||||
import { registerEvidenceCommands } from "./cliEvidenceCommands.js";
|
||||
import { registerProcessCommands } from "./cli/processCommands.js";
|
||||
@@ -49,6 +50,7 @@ export const createCli = (
|
||||
registerArtifactCommands(cli, logger);
|
||||
registerManagedCommands(cli, logger);
|
||||
registerAndroidCommands(cli, logger, environment);
|
||||
registerFirmwareCommands(cli, logger, environment);
|
||||
registerEvidenceCommands(cli, logger);
|
||||
registerProcessCommands(cli, logger, environment);
|
||||
registerBrowserCommands(cli, logger);
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
import { z } from "incur";
|
||||
import { FirmwareAnalysisService } from "../application/FirmwareAnalysisService.js";
|
||||
import { FirmwareProvider } from "../firmware/FirmwareProvider.js";
|
||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||
import { logCliCommand } from "../cliLogging.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import type { FirmwareOperation } from "../domain/firmwareAnalysis.js";
|
||||
import type { Logger } from "../logger.js";
|
||||
import type { CliInstance } from "./types.js";
|
||||
import { withCommandCancellation } from "./commandCancellation.js";
|
||||
|
||||
/** One-shot firmware CLI commands backed by the same workflows as MCP. */
|
||||
export const registerFirmwareCommands = (
|
||||
cli: CliInstance,
|
||||
logger: Logger,
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
): void => {
|
||||
const service = new FirmwareAnalysisService(
|
||||
new FirmwareProvider(environment),
|
||||
);
|
||||
const execute = (
|
||||
name: string,
|
||||
operation: FirmwareOperation,
|
||||
input: unknown,
|
||||
) =>
|
||||
withCommandCancellation((signal) =>
|
||||
logCliCommand(logger, name, async () => {
|
||||
const result = await service.execute(operation, input, { signal });
|
||||
return result.ok ? result.value : projectAnalysisError(result.error);
|
||||
}),
|
||||
);
|
||||
const path = z.string().describe("Local firmware path; never executed");
|
||||
cli.command(CLI_COMMANDS.inspectFirmwareRegions, {
|
||||
description:
|
||||
"Inspect firmware signature candidates without extracting them",
|
||||
args: z.object({ path }),
|
||||
run: ({ args }) =>
|
||||
execute(
|
||||
CLI_COMMANDS.inspectFirmwareRegions,
|
||||
"inspect_firmware_regions",
|
||||
args,
|
||||
),
|
||||
});
|
||||
cli.command(CLI_COMMANDS.extractFirmware, {
|
||||
description:
|
||||
"Extract selected firmware bytes and return verified files and provenance",
|
||||
args: z.object({
|
||||
path,
|
||||
outputDirectory: z
|
||||
.string()
|
||||
.describe("Absolute absent output directory; parent must exist"),
|
||||
}),
|
||||
options: z.object({
|
||||
offset: z
|
||||
.number()
|
||||
.int()
|
||||
.nonnegative()
|
||||
.optional()
|
||||
.describe("Selected byte offset; requires length"),
|
||||
length: z
|
||||
.number()
|
||||
.int()
|
||||
.positive()
|
||||
.optional()
|
||||
.describe("Selected byte length; requires offset"),
|
||||
maxDepth: z
|
||||
.number()
|
||||
.int()
|
||||
.optional()
|
||||
.describe("Extraction depth, 1–10; default 3"),
|
||||
maxOutputBytes: z
|
||||
.number()
|
||||
.int()
|
||||
.optional()
|
||||
.describe("Staging byte budget; default 256 MiB, maximum 512 MiB"),
|
||||
maxOutputFiles: z
|
||||
.number()
|
||||
.int()
|
||||
.optional()
|
||||
.describe("Staging entry budget; default 10000"),
|
||||
}),
|
||||
run: ({ args, options }) =>
|
||||
execute(CLI_COMMANDS.extractFirmware, "extract_firmware", {
|
||||
path: args.path,
|
||||
output_directory: args.outputDirectory,
|
||||
...(options.offset === undefined && options.length === undefined
|
||||
? {}
|
||||
: { range: { offset: options.offset, length: options.length } }),
|
||||
...(options.maxDepth === undefined
|
||||
? {}
|
||||
: { max_depth: options.maxDepth }),
|
||||
...(options.maxOutputBytes === undefined
|
||||
? {}
|
||||
: { max_output_bytes: options.maxOutputBytes }),
|
||||
...(options.maxOutputFiles === undefined
|
||||
? {}
|
||||
: { max_output_files: options.maxOutputFiles }),
|
||||
}),
|
||||
});
|
||||
};
|
||||
@@ -32,6 +32,8 @@ export const CLI_COMMANDS = Object.freeze({
|
||||
resolveNativeCallTargets: "resolve-native-call-targets",
|
||||
inspectAssetCatalog: "inspect-asset-catalog",
|
||||
inspectManagedArtifact: "inspect-managed-artifact",
|
||||
inspectFirmwareRegions: "inspect-firmware-regions",
|
||||
extractFirmware: "extract-firmware",
|
||||
inspectAndroidPackage: "inspect-android-package",
|
||||
searchAndroidClasses: "search-android-classes",
|
||||
inspectAndroidClass: "inspect-android-class",
|
||||
|
||||
@@ -8,6 +8,7 @@ import { OFFICIAL_TOOL_CONTRACTS } from "./officialToolContracts.js";
|
||||
import { SESSION_TOOL_CONTRACTS } from "./sessionToolContracts.js";
|
||||
import { NATIVE_TOOL_CONTRACTS } from "./nativeToolContracts.js";
|
||||
import { ARTIFACT_TOOL_CONTRACTS } from "./artifactToolContracts.js";
|
||||
import { FIRMWARE_TOOL_CONTRACTS } from "./firmwareToolContracts.js";
|
||||
import { ANDROID_TOOL_CONTRACTS } from "./androidToolContracts.js";
|
||||
import { MANAGED_TOOL_CONTRACTS } from "./managedToolContracts.js";
|
||||
import { MANAGED_WORKFLOW_TOOL_CONTRACTS } from "./managedWorkflowToolContracts.js";
|
||||
@@ -75,6 +76,7 @@ describe("tool contract surface", () => {
|
||||
...NATIVE_TOOL_CONTRACTS,
|
||||
...ARTIFACT_TOOL_CONTRACTS,
|
||||
...ANDROID_TOOL_CONTRACTS,
|
||||
...FIRMWARE_TOOL_CONTRACTS,
|
||||
...MANAGED_TOOL_CONTRACTS,
|
||||
...MANAGED_WORKFLOW_TOOL_CONTRACTS,
|
||||
...BROWSER_TOOL_CONTRACTS,
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
import {
|
||||
firmwareInputSchemas,
|
||||
firmwareResultSchemas,
|
||||
} from "../domain/firmwareAnalysis.js";
|
||||
import type { ToolContract } from "./toolContractTypes.js";
|
||||
import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js";
|
||||
import { toolContractMetadata } from "./toolEffects.js";
|
||||
|
||||
/** Explicit local firmware observations and extraction, backed by caller-supplied tools. */
|
||||
export const FIRMWARE_TOOL_CONTRACTS = [
|
||||
{
|
||||
name: "inspect_firmware_regions",
|
||||
...toolContractMetadata("inspect_firmware_regions"),
|
||||
kind: "firmware-provider",
|
||||
description:
|
||||
"Inspect signature candidates in a local firmware file with Binwalk 3.1.0. Returns offsets, reported lengths, descriptions and raw confidence values. Length validation and runtime addresses remain unknown. No extraction or target execution. Linux, caller-supplied tool and util-linux prlimit required; 128 MiB input limit, one worker and 120-second deadline.",
|
||||
inputSchema: firmwareInputSchemas.inspect_firmware_regions,
|
||||
outputSchema: evidenceResultOf(
|
||||
firmwareResultSchemas.inspect_firmware_regions,
|
||||
),
|
||||
examples: [
|
||||
{
|
||||
title: "Inspect firmware regions",
|
||||
input: { path: "/tmp/firmware.bin" },
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "extract_firmware",
|
||||
...toolContractMetadata("extract_firmware"),
|
||||
kind: "firmware-provider",
|
||||
description:
|
||||
"Extract caller-selected firmware bytes with Unblob 26.6.4 into an absent output directory. Returns verified child paths/digests, task derivations, unknown chunks and extraction diagnostics inline. Preserves the default sandbox; publishes regular files only. Linux, caller-supplied tool, format-specific extractors and util-linux prlimit required. No mounts or extracted-code execution. Select a returned native file with open_binary for Ghidra analysis; decompressed offsets are not original-file addresses.",
|
||||
inputSchema: firmwareInputSchemas.extract_firmware,
|
||||
outputSchema: evidenceResultOf(firmwareResultSchemas.extract_firmware),
|
||||
examples: [
|
||||
{
|
||||
title: "Extract selected firmware",
|
||||
input: {
|
||||
path: "/tmp/firmware.bin",
|
||||
output_directory: "/tmp/firmware-output",
|
||||
range: { offset: 64, length: 4096 },
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
] as const satisfies readonly ToolContract[];
|
||||
@@ -25,6 +25,7 @@ export const TOOL_KINDS = [
|
||||
"artifact-provider",
|
||||
"managed-provider",
|
||||
"android-provider",
|
||||
"firmware-provider",
|
||||
"browser-provider",
|
||||
"electron-provider",
|
||||
"runtime-provider",
|
||||
|
||||
@@ -6,6 +6,7 @@ import { SESSION_TOOL_CONTRACTS } from "./sessionToolContracts.js";
|
||||
import { TOOL_CONTRACTS } from "./toolContracts.js";
|
||||
import { ARTIFACT_TOOL_CONTRACTS } from "./artifactToolContracts.js";
|
||||
import { MANAGED_TOOL_CONTRACTS } from "./managedToolContracts.js";
|
||||
import { FIRMWARE_TOOL_CONTRACTS } from "./firmwareToolContracts.js";
|
||||
import { ANDROID_TOOL_CONTRACTS } from "./androidToolContracts.js";
|
||||
import { MANAGED_WORKFLOW_TOOL_CONTRACTS } from "./managedWorkflowToolContracts.js";
|
||||
import { NATIVE_TOOL_CONTRACTS } from "./nativeToolContracts.js";
|
||||
@@ -21,6 +22,7 @@ const GROUPS = {
|
||||
artifact: ARTIFACT_TOOL_CONTRACTS,
|
||||
managed: MANAGED_TOOL_CONTRACTS,
|
||||
android: ANDROID_TOOL_CONTRACTS,
|
||||
firmware: FIRMWARE_TOOL_CONTRACTS,
|
||||
managed_workflow: MANAGED_WORKFLOW_TOOL_CONTRACTS,
|
||||
browser_provider: BROWSER_PROVIDER_TOOL_CONTRACTS,
|
||||
electron: ELECTRON_TOOL_CONTRACTS,
|
||||
@@ -93,6 +95,8 @@ const PUBLISHED_TOOL_NAME_FLOOR = [
|
||||
"inspect_electron_page",
|
||||
"inspect_macho",
|
||||
"inspect_managed_artifact",
|
||||
"inspect_firmware_regions",
|
||||
"extract_firmware",
|
||||
"inspect_android_package",
|
||||
"search_android_classes",
|
||||
"inspect_android_class",
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { NATIVE_TOOL_CONTRACTS } from "./nativeToolContracts.js";
|
||||
import { ARTIFACT_TOOL_CONTRACTS } from "./artifactToolContracts.js";
|
||||
import { MANAGED_TOOL_CONTRACTS } from "./managedToolContracts.js";
|
||||
import { FIRMWARE_TOOL_CONTRACTS } from "./firmwareToolContracts.js";
|
||||
import { ANDROID_TOOL_CONTRACTS } from "./androidToolContracts.js";
|
||||
import { MANAGED_WORKFLOW_TOOL_CONTRACTS } from "./managedWorkflowToolContracts.js";
|
||||
import { BROWSER_PROVIDER_TOOL_CONTRACTS } from "./browserProviderToolContracts.js";
|
||||
@@ -21,6 +22,7 @@ export const TOOL_CONTRACTS = [
|
||||
...ARTIFACT_TOOL_CONTRACTS,
|
||||
...MANAGED_TOOL_CONTRACTS,
|
||||
...ANDROID_TOOL_CONTRACTS,
|
||||
...FIRMWARE_TOOL_CONTRACTS,
|
||||
...MANAGED_WORKFLOW_TOOL_CONTRACTS,
|
||||
...BROWSER_PROVIDER_TOOL_CONTRACTS,
|
||||
...ELECTRON_TOOL_CONTRACTS,
|
||||
|
||||
@@ -41,6 +41,17 @@ const sessionEvidence = effects({ mutatesSession: true });
|
||||
|
||||
/** Explicit effect audit for every public tool. */
|
||||
export const TOOL_EFFECTS: Readonly<Record<string, ToolEffects>> = {
|
||||
inspect_firmware_regions: effects({
|
||||
mutatesSession: true,
|
||||
launchesProcess: true,
|
||||
writesFilesystem: true,
|
||||
}),
|
||||
extract_firmware: effects({
|
||||
mutatesSession: true,
|
||||
launchesProcess: true,
|
||||
writesFilesystem: true,
|
||||
idempotent: false,
|
||||
}),
|
||||
inspect_android_package: effects({
|
||||
mutatesSession: true,
|
||||
launchesProcess: true,
|
||||
|
||||
@@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisInputError,
|
||||
AnalysisOutputError,
|
||||
} from "./analysisErrorCore.js";
|
||||
import { ArtifactOperationError } from "./artifactOperationError.js";
|
||||
import { BinaryTargetError } from "./configurationErrors.js";
|
||||
@@ -160,6 +161,23 @@ describe("analysis error projection: provider failures", () => {
|
||||
});
|
||||
|
||||
describe("analysis error projection: caller contract", () => {
|
||||
it("preserves the reported output constraint without exposing its internal cause", () => {
|
||||
const projected = projectAnalysisError(
|
||||
new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
"Report exceeded the declared byte budget",
|
||||
{ cause: new Error("private internal cause") },
|
||||
),
|
||||
);
|
||||
expect(projected).toMatchObject({
|
||||
code: "unreadable_output",
|
||||
details: {
|
||||
operation: "extract_firmware",
|
||||
reason: "Report exceeded the declared byte budget",
|
||||
},
|
||||
});
|
||||
expect(JSON.stringify(projected)).not.toContain("private internal cause");
|
||||
});
|
||||
it("maps representative failures without exposing causes", () => {
|
||||
const secretCause = new Error("secret-token");
|
||||
const projected = [
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisInputError,
|
||||
AnalysisOutputError,
|
||||
AnalysisTimeoutError,
|
||||
} from "./analysisErrorCore.js";
|
||||
import { ArtifactOperationError } from "./artifactOperationError.js";
|
||||
@@ -176,6 +177,8 @@ const errorDetails = (
|
||||
const requestErrorDetails = (
|
||||
error: AnalysisError,
|
||||
): Readonly<Record<string, JsonValue>> | undefined => {
|
||||
if (error instanceof AnalysisOutputError)
|
||||
return { operation: error.operation, reason: error.reason };
|
||||
if (error instanceof AnalysisInputError && error.issues.length > 0)
|
||||
return {
|
||||
operation: error.operation,
|
||||
|
||||
@@ -0,0 +1,145 @@
|
||||
import { z } from "zod";
|
||||
import { digestSchema } from "./digests.js";
|
||||
import { jsonValueSchema } from "./jsonValue.js";
|
||||
|
||||
const bytes = z.number().int().min(0).max(Number.MAX_SAFE_INTEGER);
|
||||
const range = z.strictObject({ offset: bytes, length: bytes.positive() });
|
||||
const target = {
|
||||
path: z.string().min(1).describe("Local firmware file; never executed"),
|
||||
};
|
||||
|
||||
/** Explicit firmware inspection and extraction intent, independent of engines. */
|
||||
export const firmwareInputSchemas = {
|
||||
inspect_firmware_regions: z.strictObject(target),
|
||||
extract_firmware: z.strictObject({
|
||||
...target,
|
||||
output_directory: z
|
||||
.string()
|
||||
.min(1)
|
||||
.describe("Absolute, absent output directory; parent must exist"),
|
||||
range: range
|
||||
.optional()
|
||||
.describe("Explicit file-byte interval; omit to extract the whole file"),
|
||||
max_depth: z
|
||||
.number()
|
||||
.int()
|
||||
.min(1)
|
||||
.max(10)
|
||||
.default(3)
|
||||
.describe("Extraction depth; reaching it yields partial analysis"),
|
||||
max_output_bytes: z
|
||||
.number()
|
||||
.int()
|
||||
.min(1)
|
||||
.max(512 * 1024 * 1024)
|
||||
.default(256 * 1024 * 1024)
|
||||
.describe(
|
||||
"Staging file-byte budget, checked during extraction; transient overshoot is possible",
|
||||
),
|
||||
max_output_files: z
|
||||
.number()
|
||||
.int()
|
||||
.min(1)
|
||||
.max(100_000)
|
||||
.default(10_000)
|
||||
.describe("Staging entry budget, including directories and links"),
|
||||
}),
|
||||
} as const;
|
||||
|
||||
/** Supported firmware operations. */
|
||||
export type FirmwareOperation = keyof typeof firmwareInputSchemas;
|
||||
/** Correlated, validated firmware request. */
|
||||
export type FirmwareRequest = {
|
||||
[Name in FirmwareOperation]: {
|
||||
operation: Name;
|
||||
input: z.infer<(typeof firmwareInputSchemas)[Name]>;
|
||||
};
|
||||
}[FirmwareOperation];
|
||||
/** Validate the operation with its exact input contract. */
|
||||
export const firmwareRequestSchema = z.discriminatedUnion("operation", [
|
||||
z.strictObject({
|
||||
operation: z.literal("inspect_firmware_regions"),
|
||||
input: firmwareInputSchemas.inspect_firmware_regions,
|
||||
}),
|
||||
z.strictObject({
|
||||
operation: z.literal("extract_firmware"),
|
||||
input: firmwareInputSchemas.extract_firmware,
|
||||
}),
|
||||
]);
|
||||
|
||||
const engine = z.strictObject({
|
||||
name: z.string(),
|
||||
version: z.string(),
|
||||
executable_path: z.string(),
|
||||
executable_sha256: digestSchema,
|
||||
source_revision: z.null(),
|
||||
worker_count: z.literal(1),
|
||||
});
|
||||
const selection = z.strictObject({ ...range.shape, sha256: digestSchema });
|
||||
const chunk = z.strictObject({
|
||||
input_path: z.string(),
|
||||
provider_id: z.string(),
|
||||
range,
|
||||
handler: z.string().nullable(),
|
||||
encrypted: z.boolean().nullable(),
|
||||
root_file_range: range.nullable(),
|
||||
});
|
||||
|
||||
/** Portable results retain reported boundaries, unknowns and extraction lineage. */
|
||||
export const firmwareResultSchemas = {
|
||||
inspect_firmware_regions: z.strictObject({
|
||||
engine,
|
||||
input_size: bytes,
|
||||
regions: z.array(
|
||||
z.strictObject({
|
||||
provider_id: z.string(),
|
||||
offset: bytes,
|
||||
reported_size: bytes,
|
||||
signature: z.string(),
|
||||
description: z.string(),
|
||||
reported_confidence: z.number().int().min(0).max(255),
|
||||
size_basis: z.literal("provider_reported_validation_unknown"),
|
||||
}),
|
||||
),
|
||||
coverage: z.literal("signature_scan"),
|
||||
}),
|
||||
extract_firmware: z.strictObject({
|
||||
engine,
|
||||
selection,
|
||||
output_directory: z.string(),
|
||||
coverage: z.enum(["complete", "partial"]),
|
||||
files: z.array(
|
||||
z.strictObject({
|
||||
relative_path: z.string(),
|
||||
path: z.string(),
|
||||
size: bytes,
|
||||
sha256: digestSchema,
|
||||
reported_sha256: digestSchema.nullable(),
|
||||
mime_type: z.string().nullable(),
|
||||
analysis_depth: bytes.nullable(),
|
||||
runtime_address: z.null(),
|
||||
original_file_range: z.null(),
|
||||
}),
|
||||
),
|
||||
unpublished_entries: z.array(
|
||||
z.strictObject({
|
||||
relative_path: z.string(),
|
||||
kind: z.enum(["symlink", "special"]),
|
||||
link_target: z.string().nullable(),
|
||||
}),
|
||||
),
|
||||
chunks: z.array(chunk),
|
||||
derivations: z.array(
|
||||
z.strictObject({
|
||||
parent_path: z.string(),
|
||||
child_path: z.string(),
|
||||
provider_blob_id: z.string(),
|
||||
handler: z.string().nullable(),
|
||||
parent_file_range: range.nullable(),
|
||||
}),
|
||||
),
|
||||
depth_limited_paths: z.array(z.string()),
|
||||
diagnostics: z.array(jsonValueSchema),
|
||||
published_bytes: bytes,
|
||||
}),
|
||||
} as const;
|
||||
@@ -0,0 +1,229 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { access, realpath } from "node:fs/promises";
|
||||
import { constants } from "node:fs";
|
||||
import { isAbsolute } from "node:path";
|
||||
import {
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisCancelledError,
|
||||
AnalysisTimeoutError,
|
||||
AnalysisOutputError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import {
|
||||
spawnOwnedProviderProcess,
|
||||
ProviderProcessSupervisor,
|
||||
type OwnedProviderProcessSpawnOptions,
|
||||
type SpawnedOwnedProviderProcess,
|
||||
} from "../process/ProviderProcess.js";
|
||||
import { cleanupOwnedProcessGroup } from "../process/ProcessOwnership.js";
|
||||
import { FIRMWARE_LIMITS } from "./FirmwareRelease.js";
|
||||
import {
|
||||
hashFirmwareFile,
|
||||
inventoryFirmwareOutput,
|
||||
readFirmwareReport,
|
||||
} from "./FirmwareFiles.js";
|
||||
|
||||
/** Injectable owned launcher for production protocol and lifecycle tests. */
|
||||
export type FirmwareLauncher = (
|
||||
options: OwnedProviderProcessSpawnOptions,
|
||||
) => Promise<SpawnedOwnedProviderProcess>;
|
||||
|
||||
/** Resolve explicitly supplied executable paths; never install or search for engines. */
|
||||
export const resolveFirmwareCommand = async (
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
engine: "binwalk" | "unblob",
|
||||
operation: string,
|
||||
) => {
|
||||
const variable =
|
||||
engine === "binwalk" ? "REA_BINWALK_COMMAND" : "REA_UNBLOB_COMMAND";
|
||||
const configured = environment[variable];
|
||||
if (
|
||||
process.platform !== "linux" ||
|
||||
configured === undefined ||
|
||||
!isAbsolute(configured)
|
||||
)
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
engine,
|
||||
operation,
|
||||
`This integration requires Linux and an absolute ${variable} path to the caller-supplied tool`,
|
||||
);
|
||||
const command = await realpath(configured).catch((cause: unknown) => {
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
engine,
|
||||
operation,
|
||||
`${variable} is unavailable: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
);
|
||||
});
|
||||
await access(command, constants.X_OK);
|
||||
const limiter =
|
||||
environment.REA_FIRMWARE_PRLIMIT_COMMAND ?? "/usr/bin/prlimit";
|
||||
if (!isAbsolute(limiter))
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
engine,
|
||||
operation,
|
||||
"REA_FIRMWARE_PRLIMIT_COMMAND must be absolute",
|
||||
);
|
||||
await access(limiter, constants.X_OK).catch(() => {
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
engine,
|
||||
operation,
|
||||
`Resource limiter unavailable: ${limiter}; provide util-linux prlimit`,
|
||||
);
|
||||
});
|
||||
return { command, limiter, sha256: await hashFirmwareFile(command) };
|
||||
};
|
||||
|
||||
/** Execute one bounded provider command and verify its owned descendants have stopped. */
|
||||
export const runFirmwareCommand = async (context: {
|
||||
engine: string;
|
||||
operation: string;
|
||||
command: string;
|
||||
limiter: string;
|
||||
args: readonly string[];
|
||||
cwd: string;
|
||||
environment: Readonly<Record<string, string | undefined>>;
|
||||
signal: AbortSignal;
|
||||
launcher?: FirmwareLauncher;
|
||||
outputBudget?: { root: string; bytes: number; entries: number };
|
||||
acceptReportedExtractionFailure?: boolean;
|
||||
}) => {
|
||||
const { engine, operation, signal } = context;
|
||||
signal.throwIfAborted();
|
||||
const runId = `rea-firmware-${randomUUID()}`;
|
||||
const spawned = await (context.launcher ?? spawnOwnedProviderProcess)({
|
||||
command: context.limiter,
|
||||
arguments: [
|
||||
`--as=${FIRMWARE_LIMITS.addressSpaceBytes}`,
|
||||
`--fsize=${context.outputBudget?.bytes ?? FIRMWARE_LIMITS.reportBytes}`,
|
||||
"--core=0",
|
||||
"--cpu=120",
|
||||
"--",
|
||||
context.command,
|
||||
...context.args,
|
||||
],
|
||||
expectedCommand: null,
|
||||
runId,
|
||||
cwd: context.cwd,
|
||||
hostEnvironment: context.environment,
|
||||
env: {
|
||||
TMPDIR: context.cwd,
|
||||
OMP_NUM_THREADS: "1",
|
||||
OPENBLAS_NUM_THREADS: "1",
|
||||
RAYON_NUM_THREADS: "1",
|
||||
NO_COLOR: "1",
|
||||
},
|
||||
});
|
||||
const supervisor = new ProviderProcessSupervisor({
|
||||
...spawned,
|
||||
ownsProcessLifetime: true,
|
||||
cleanup:
|
||||
spawned.cleanup ?? (() => cleanupOwnedProcessGroup(spawned.ownership)),
|
||||
});
|
||||
let failure: unknown;
|
||||
try {
|
||||
const deadline = Date.now() + FIRMWARE_LIMITS.timeoutMs;
|
||||
while (!(await supervisor.waitForExit(100))) {
|
||||
if (signal.aborted) throw new AnalysisCancelledError(operation);
|
||||
if (Date.now() >= deadline)
|
||||
throw new AnalysisTimeoutError(operation, FIRMWARE_LIMITS.timeoutMs);
|
||||
if (context.outputBudget !== undefined)
|
||||
await inventoryFirmwareOutput(
|
||||
context.outputBudget.root,
|
||||
context.outputBudget.bytes,
|
||||
context.outputBudget.entries,
|
||||
operation,
|
||||
signal,
|
||||
);
|
||||
}
|
||||
if (signal.aborted) throw new AnalysisCancelledError(operation);
|
||||
} catch (cause: unknown) {
|
||||
failure = cause;
|
||||
}
|
||||
const stopped = await supervisor.stop();
|
||||
const snapshot = supervisor.snapshot();
|
||||
supervisor.dispose();
|
||||
if (stopped.status === "incomplete")
|
||||
throw new ProviderCleanupError(engine, [runId, context.cwd], {
|
||||
reason: stopped.reason,
|
||||
leader_pid: spawned.ownership.leaderPid,
|
||||
process_group_id: spawned.ownership.processGroupId,
|
||||
exit_code: snapshot.exitCode ?? null,
|
||||
signal: snapshot.signal ?? null,
|
||||
stdout: snapshot.stdout.text,
|
||||
stderr: snapshot.stderr.text,
|
||||
stdout_bytes: snapshot.stdout.bytes,
|
||||
stderr_bytes: snapshot.stderr.bytes,
|
||||
previous_error: failure instanceof Error ? failure.message : null,
|
||||
});
|
||||
if (failure !== undefined) throw failure;
|
||||
if (
|
||||
snapshot.exitCode !== 0 &&
|
||||
!(
|
||||
context.acceptReportedExtractionFailure === true &&
|
||||
snapshot.exitCode === 1
|
||||
)
|
||||
)
|
||||
throw new ProviderAdapterError(engine, operation, {
|
||||
diagnostics: {
|
||||
reason:
|
||||
"Tool failed; inspect exit status and stderr for format, dependency, sandbox or resource-limit failures",
|
||||
exit_code: snapshot.exitCode ?? null,
|
||||
signal: snapshot.signal ?? null,
|
||||
stdout: snapshot.stdout.text,
|
||||
stderr: snapshot.stderr.text,
|
||||
stdout_bytes: snapshot.stdout.bytes,
|
||||
stderr_bytes: snapshot.stderr.bytes,
|
||||
},
|
||||
});
|
||||
return {
|
||||
stdout: snapshot.stdout,
|
||||
stderr: snapshot.stderr,
|
||||
exit_code: snapshot.exitCode,
|
||||
signal: snapshot.signal ?? null,
|
||||
command: context.command,
|
||||
args: [...context.args],
|
||||
resource_limiter: context.limiter,
|
||||
};
|
||||
};
|
||||
|
||||
/** Preserve tool failure diagnostics when an accepted exit 1 has no usable report. */
|
||||
export const readFirmwareCommandReport = async (context: {
|
||||
path: string;
|
||||
engine: string;
|
||||
operation: string;
|
||||
execution: Awaited<ReturnType<typeof runFirmwareCommand>>;
|
||||
}): Promise<unknown> => {
|
||||
try {
|
||||
return await readFirmwareReport(context.path, context.operation);
|
||||
} catch (cause: unknown) {
|
||||
if (context.execution.exit_code === 1)
|
||||
throw new ProviderAdapterError(context.engine, context.operation, {
|
||||
cause,
|
||||
diagnostics: {
|
||||
reason:
|
||||
"Tool exited 1 without a usable analysis report; inspect stderr for startup, sandbox or dependency failures",
|
||||
report_failure:
|
||||
cause instanceof Error ? cause.message : String(cause),
|
||||
exit_code: 1,
|
||||
signal: context.execution.signal,
|
||||
stdout: context.execution.stdout.text,
|
||||
stderr: context.execution.stderr.text,
|
||||
stdout_bytes: context.execution.stdout.bytes,
|
||||
stderr_bytes: context.execution.stderr.bytes,
|
||||
},
|
||||
});
|
||||
const diagnostics = `stdout (${context.execution.stdout.bytes} bytes): ${context.execution.stdout.text}; stderr (${context.execution.stderr.bytes} bytes): ${context.execution.stderr.text}`;
|
||||
if (cause instanceof AnalysisOutputError)
|
||||
throw new AnalysisOutputError(
|
||||
context.operation,
|
||||
`${cause.reason}; ${diagnostics}`,
|
||||
{ cause },
|
||||
);
|
||||
throw new AnalysisOutputError(
|
||||
context.operation,
|
||||
`Tool did not produce the expected complete report: ${context.path}; ${diagnostics}`,
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,265 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { constants, createReadStream, createWriteStream } from "node:fs";
|
||||
import {
|
||||
chmod,
|
||||
lstat,
|
||||
open,
|
||||
readdir,
|
||||
readlink,
|
||||
realpath,
|
||||
} from "node:fs/promises";
|
||||
import { isAbsolute, join, relative, resolve, sep } from "node:path";
|
||||
import { Transform } from "node:stream";
|
||||
import { pipeline } from "node:stream/promises";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisInputError,
|
||||
AnalysisOutputError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { FirmwareRequest } from "../domain/firmwareAnalysis.js";
|
||||
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
|
||||
import { FIRMWARE_LIMITS } from "./FirmwareRelease.js";
|
||||
|
||||
/** One regular file or unpublished filesystem entry in a private extraction. */
|
||||
export interface FirmwareEntry {
|
||||
readonly relativePath: string;
|
||||
readonly path: string;
|
||||
readonly size: number;
|
||||
readonly kind: "file" | "symlink" | "special";
|
||||
readonly linkTarget: string | null;
|
||||
}
|
||||
|
||||
/** Hash a regular tool or extracted file without retaining its bytes in memory. */
|
||||
export const hashFirmwareFile = async (
|
||||
path: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string> => {
|
||||
const handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW);
|
||||
const hash = createHash("sha256");
|
||||
try {
|
||||
if (!(await handle.stat()).isFile())
|
||||
throw new Error(`Not a regular file: ${path}`);
|
||||
for await (const chunk of handle.createReadStream({ autoClose: false })) {
|
||||
signal?.throwIfAborted();
|
||||
hash.update(chunk);
|
||||
}
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
return hash.digest("hex");
|
||||
};
|
||||
|
||||
/** Copy and identify the exact bytes consumed by firmware providers. */
|
||||
export const snapshotFirmware = async (
|
||||
request: FirmwareRequest,
|
||||
root: string,
|
||||
signal: AbortSignal,
|
||||
) => {
|
||||
const sourcePath = resolve(request.input.path);
|
||||
const source = await realpath(sourcePath);
|
||||
const handle = await open(source, constants.O_RDONLY | constants.O_NOFOLLOW);
|
||||
const fullPath = join(root, "firmware.bin");
|
||||
const hash = createHash("sha256");
|
||||
let size = 0;
|
||||
try {
|
||||
const stat = await handle.stat();
|
||||
if (
|
||||
!stat.isFile() ||
|
||||
stat.size === 0 ||
|
||||
stat.size > FIRMWARE_LIMITS.inputBytes
|
||||
)
|
||||
throw new AnalysisInputError(request.operation, undefined, [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "out_of_range",
|
||||
message: `Firmware must be a nonempty regular file no larger than ${FIRMWARE_LIMITS.inputBytes} bytes`,
|
||||
},
|
||||
]);
|
||||
const meter = new Transform({
|
||||
transform(chunk: Buffer, _encoding, callback) {
|
||||
size += chunk.length;
|
||||
if (size > FIRMWARE_LIMITS.inputBytes) {
|
||||
callback(new Error("Firmware grew beyond the input byte budget"));
|
||||
return;
|
||||
}
|
||||
hash.update(chunk);
|
||||
callback(null, chunk);
|
||||
},
|
||||
});
|
||||
await pipeline(
|
||||
handle.createReadStream({ autoClose: false }),
|
||||
meter,
|
||||
createWriteStream(fullPath, { flags: "wx", mode: 0o600 }),
|
||||
{ signal },
|
||||
);
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
await chmod(fullPath, 0o400);
|
||||
const sha256 = hash.digest("hex");
|
||||
const selected =
|
||||
request.operation === "extract_firmware" ? request.input.range : undefined;
|
||||
const offset = selected?.offset ?? 0;
|
||||
const length = selected?.length ?? size;
|
||||
if (length === 0 || offset > size || length > size - offset)
|
||||
throw new AnalysisInputError(request.operation, undefined, [
|
||||
{
|
||||
path: ["range"],
|
||||
reason: "out_of_range",
|
||||
message: `Selected range must be within the ${size}-byte firmware file`,
|
||||
},
|
||||
]);
|
||||
let inputPath = fullPath;
|
||||
if (selected !== undefined) {
|
||||
inputPath = join(root, "selected.bin");
|
||||
await pipeline(
|
||||
createReadStream(fullPath, { start: offset, end: offset + length - 1 }),
|
||||
createWriteStream(inputPath, { flags: "wx", mode: 0o600 }),
|
||||
{ signal },
|
||||
);
|
||||
await chmod(inputPath, 0o400);
|
||||
}
|
||||
return {
|
||||
subject: { path: sourcePath, sha256, format: "file" as const },
|
||||
size,
|
||||
inputPath,
|
||||
selection: {
|
||||
offset,
|
||||
length,
|
||||
sha256:
|
||||
selected === undefined
|
||||
? sha256
|
||||
: await hashFirmwareFile(inputPath, signal),
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
/** Read a complete bounded report; oversized data is never reported as complete. */
|
||||
export const readFirmwareReport = async (
|
||||
path: string,
|
||||
operation: string,
|
||||
): Promise<unknown> => {
|
||||
const handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW);
|
||||
try {
|
||||
if (!(await handle.stat()).isFile())
|
||||
throw new AnalysisOutputError(operation, "Report is not a regular file");
|
||||
const bytes = await readBoundedFileBytes(
|
||||
handle,
|
||||
FIRMWARE_LIMITS.reportBytes,
|
||||
);
|
||||
if (bytes === undefined)
|
||||
throw new AnalysisOutputError(
|
||||
operation,
|
||||
`Report exceeded ${FIRMWARE_LIMITS.reportBytes} bytes`,
|
||||
);
|
||||
try {
|
||||
const value: unknown = JSON.parse(bytes.toString("utf8"));
|
||||
return value;
|
||||
} catch (cause: unknown) {
|
||||
throw new AnalysisOutputError(
|
||||
operation,
|
||||
"Tool report is not valid complete JSON",
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
};
|
||||
|
||||
/** Inventory private output without following links, enforcing aggregate staging budgets. */
|
||||
export const inventoryFirmwareOutput = async (
|
||||
root: string,
|
||||
maxBytes: number,
|
||||
maxEntries: number,
|
||||
operation: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<FirmwareEntry[]> => {
|
||||
const entries: FirmwareEntry[] = [];
|
||||
const pending = [root];
|
||||
let bytes = 0;
|
||||
let count = 0;
|
||||
while (pending.length > 0) {
|
||||
if (signal?.aborted === true) throw new AnalysisCancelledError(operation);
|
||||
const directory = pending.pop();
|
||||
if (directory === undefined) break;
|
||||
const directoryStat = await lstat(directory).catch((cause: unknown) => {
|
||||
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
|
||||
return undefined;
|
||||
throw cause;
|
||||
});
|
||||
if (
|
||||
directoryStat === undefined ||
|
||||
!directoryStat.isDirectory() ||
|
||||
directoryStat.isSymbolicLink()
|
||||
)
|
||||
continue;
|
||||
const names = await readdir(directory).catch((cause: unknown) => {
|
||||
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
|
||||
return [];
|
||||
throw cause;
|
||||
});
|
||||
for (const name of names.sort()) {
|
||||
count++;
|
||||
if (count > maxEntries)
|
||||
throw new AnalysisOutputError(
|
||||
operation,
|
||||
`Staging entry budget exceeded (${maxEntries})`,
|
||||
);
|
||||
const path = join(directory, name);
|
||||
const stat = await lstat(path).catch((cause: unknown) => {
|
||||
if (
|
||||
cause instanceof Error &&
|
||||
"code" in cause &&
|
||||
cause.code === "ENOENT"
|
||||
)
|
||||
return undefined;
|
||||
throw cause;
|
||||
});
|
||||
if (stat === undefined) continue;
|
||||
if (stat.isDirectory()) {
|
||||
pending.push(path);
|
||||
continue;
|
||||
}
|
||||
if (stat.isFile()) bytes += stat.size;
|
||||
if (bytes > maxBytes)
|
||||
throw new AnalysisOutputError(
|
||||
operation,
|
||||
`Staging byte budget exceeded (${maxBytes})`,
|
||||
);
|
||||
entries.push({
|
||||
path,
|
||||
relativePath: relative(root, path).split(sep).join("/"),
|
||||
size: stat.size,
|
||||
kind: stat.isSymbolicLink()
|
||||
? "symlink"
|
||||
: stat.isFile()
|
||||
? "file"
|
||||
: "special",
|
||||
linkTarget: stat.isSymbolicLink() ? await readlink(path) : null,
|
||||
});
|
||||
}
|
||||
}
|
||||
return entries.sort((a, b) => a.relativePath.localeCompare(b.relativePath));
|
||||
};
|
||||
|
||||
/** Bind producer paths to the selected input or private extraction tree. */
|
||||
export const firmwareLogicalPath = (
|
||||
path: string,
|
||||
inputPath: string,
|
||||
outputRoot: string,
|
||||
): string => {
|
||||
if (path === inputPath) return "$input";
|
||||
const rel = relative(outputRoot, path);
|
||||
if (
|
||||
!isAbsolute(path) ||
|
||||
rel === ".." ||
|
||||
rel.startsWith(`..${sep}`) ||
|
||||
isAbsolute(rel)
|
||||
)
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
`Provider reported a path outside its owned output: ${path}`,
|
||||
);
|
||||
return rel === "" ? "$output" : `$output/${rel.split(sep).join("/")}`;
|
||||
};
|
||||
@@ -0,0 +1,350 @@
|
||||
import { join, isAbsolute } from "node:path";
|
||||
import { z } from "zod";
|
||||
import type { FirmwareAnalysisPort } from "../application/FirmwareAnalysisPort.js";
|
||||
import {
|
||||
createAnalysisExecution,
|
||||
type ExecutionOptions,
|
||||
} from "../application/AnalysisProvider.js";
|
||||
import {
|
||||
firmwareResultSchemas,
|
||||
type FirmwareRequest,
|
||||
} from "../domain/firmwareAnalysis.js";
|
||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisInputError,
|
||||
AnalysisOutputError,
|
||||
AnalysisTimeoutError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import { err, ok } from "../domain/result.js";
|
||||
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
|
||||
import { publishFirmwareExtraction } from "./FirmwarePublication.js";
|
||||
import {
|
||||
FIRMWARE_LIMITS,
|
||||
FIRMWARE_RELEASES,
|
||||
BINWALK_PROVIDER_IDENTITY,
|
||||
UNBLOB_PROVIDER_IDENTITY,
|
||||
} from "./FirmwareRelease.js";
|
||||
import {
|
||||
resolveFirmwareCommand,
|
||||
runFirmwareCommand,
|
||||
readFirmwareCommandReport,
|
||||
type FirmwareLauncher,
|
||||
} from "./FirmwareCommand.js";
|
||||
import { inventoryFirmwareOutput, snapshotFirmware } from "./FirmwareFiles.js";
|
||||
import {
|
||||
normalizeBinwalkReport,
|
||||
normalizeUnblobReport,
|
||||
} from "./FirmwareReports.js";
|
||||
|
||||
type Outcome = Awaited<ReturnType<FirmwareAnalysisPort["execute"]>>;
|
||||
const waitForPredecessor = (
|
||||
predecessor: Promise<void>,
|
||||
operation: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> =>
|
||||
new Promise((resolve, reject) => {
|
||||
const abort = () => reject(new AnalysisCancelledError(operation));
|
||||
signal?.addEventListener("abort", abort, { once: true });
|
||||
if (signal?.aborted === true) abort();
|
||||
void predecessor.then(() => {
|
||||
signal?.removeEventListener("abort", abort);
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
const limitations = [
|
||||
"Signature matches and reported sizes do not establish partition validity, encryption or runtime mappings. Binwalk may infer unknown lengths before emitting its report.",
|
||||
"Only Linux is supported by this integration. One worker, inherited per-process address-space/file-size/CPU limits and a deadline constrain execution; staging byte and entry budgets are polled and may transiently overshoot. Limits are not aggregate process-memory or filesystem quotas.",
|
||||
"Unblob's default sandbox and skip rules remain enabled. Missing external extractors, unknown chunks, encrypted chunks and depth limits yield partial coverage. Extracted code is never executed; only regular files are published, without executable permission bits.",
|
||||
"The actual launcher executable is hashed. Interpreter, shared libraries and installed Python package dependencies are not independently attested; source revision remains unknown for caller-supplied builds.",
|
||||
] as const;
|
||||
|
||||
/** Queued bring-your-own firmware adapters with private snapshots and owned cleanup. */
|
||||
export class FirmwareProvider implements FirmwareAnalysisPort {
|
||||
#tail: Promise<void> = Promise.resolve();
|
||||
#cleanupFailure: AnalysisError | undefined;
|
||||
constructor(
|
||||
readonly environment: Readonly<
|
||||
Record<string, string | undefined>
|
||||
> = process.env,
|
||||
readonly launcher?: FirmwareLauncher,
|
||||
) {}
|
||||
|
||||
/** Inspect regions or explicitly extract selected bytes without changing the source. */
|
||||
async execute(
|
||||
request: FirmwareRequest,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Outcome> {
|
||||
const predecessor = this.#tail;
|
||||
let release = () => {};
|
||||
this.#tail = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
// Cancel the caller promptly without letting a later request bypass its predecessor.
|
||||
try {
|
||||
await waitForPredecessor(predecessor, request.operation, options?.signal);
|
||||
} catch (cause: unknown) {
|
||||
void predecessor.then(release);
|
||||
return err(
|
||||
cause instanceof AnalysisError
|
||||
? cause
|
||||
: new AnalysisCancelledError(request.operation),
|
||||
);
|
||||
}
|
||||
try {
|
||||
return await this.#execute(request, options);
|
||||
} finally {
|
||||
release();
|
||||
}
|
||||
}
|
||||
|
||||
async #execute(
|
||||
request: FirmwareRequest,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Outcome> {
|
||||
let root: PrivateRuntimeRoot | undefined;
|
||||
const engineName =
|
||||
request.operation === "inspect_firmware_regions" ? "binwalk" : "unblob";
|
||||
const deadline = AbortSignal.timeout(FIRMWARE_LIMITS.timeoutMs);
|
||||
const signal =
|
||||
options?.signal === undefined
|
||||
? deadline
|
||||
: AbortSignal.any([options.signal, deadline]);
|
||||
let outcome: Outcome;
|
||||
try {
|
||||
if (this.#cleanupFailure !== undefined) return err(this.#cleanupFailure);
|
||||
if (signal.aborted) throw new AnalysisCancelledError(request.operation);
|
||||
const command = await resolveFirmwareCommand(
|
||||
this.environment,
|
||||
engineName,
|
||||
request.operation,
|
||||
);
|
||||
root = await PrivateRuntimeRoot.create({ prefix: "rea-firmware-" });
|
||||
const target = await snapshotFirmware(request, root.path, signal);
|
||||
const run = (
|
||||
args: readonly string[],
|
||||
outputBudget?: { root: string; bytes: number; entries: number },
|
||||
) =>
|
||||
runFirmwareCommand({
|
||||
engine: engineName,
|
||||
operation: request.operation,
|
||||
...command,
|
||||
args,
|
||||
cwd: root?.path ?? "",
|
||||
environment: this.environment,
|
||||
signal,
|
||||
...(this.launcher === undefined ? {} : { launcher: this.launcher }),
|
||||
...(outputBudget === undefined
|
||||
? {}
|
||||
: { outputBudget, acceptReportedExtractionFailure: true }),
|
||||
});
|
||||
const versionRun = await run(["--version"]);
|
||||
const version = versionRun.stdout.text.trim();
|
||||
const expected = FIRMWARE_RELEASES[engineName].version;
|
||||
if (
|
||||
versionRun.stdout.bytes > 1024 ||
|
||||
(engineName === "binwalk"
|
||||
? version !== `binwalk ${expected}`
|
||||
: version !== expected)
|
||||
)
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
engineName,
|
||||
request.operation,
|
||||
`Unsupported tool version ${version}; this adapter is verified against ${expected}`,
|
||||
);
|
||||
const engine = {
|
||||
name: engineName,
|
||||
version: expected,
|
||||
executable_path: command.command,
|
||||
executable_sha256: command.sha256,
|
||||
source_revision: null,
|
||||
worker_count: 1 as const,
|
||||
};
|
||||
const reportPath = join(root.path, "report.json");
|
||||
let result: unknown;
|
||||
let raw: unknown;
|
||||
if (request.operation === "inspect_firmware_regions") {
|
||||
const processRun = await run([
|
||||
"--quiet",
|
||||
"--search-all",
|
||||
"--threads",
|
||||
"1",
|
||||
"--log",
|
||||
reportPath,
|
||||
target.inputPath,
|
||||
]);
|
||||
const report = await readFirmwareCommandReport({
|
||||
path: reportPath,
|
||||
engine: engineName,
|
||||
operation: request.operation,
|
||||
execution: processRun,
|
||||
});
|
||||
result = firmwareResultSchemas.inspect_firmware_regions.parse({
|
||||
engine,
|
||||
input_size: target.size,
|
||||
regions: normalizeBinwalkReport(
|
||||
report,
|
||||
target.inputPath,
|
||||
target.size,
|
||||
),
|
||||
coverage: "signature_scan",
|
||||
});
|
||||
raw = { report, execution: processRun, version_execution: versionRun };
|
||||
} else {
|
||||
if (!isAbsolute(request.input.output_directory))
|
||||
throw new AnalysisInputError(request.operation, undefined, [
|
||||
{
|
||||
path: ["output_directory"],
|
||||
reason: "invalid_format",
|
||||
message: "Extraction output directory must be absolute",
|
||||
},
|
||||
]);
|
||||
const outputRoot = join(root.path, "extracted");
|
||||
const processRun = await run(
|
||||
[
|
||||
"--process-num",
|
||||
"1",
|
||||
"--depth",
|
||||
String(request.input.max_depth),
|
||||
"--randomness-depth",
|
||||
"0",
|
||||
"--report",
|
||||
reportPath,
|
||||
"--log",
|
||||
join(root.path, "unblob.log"),
|
||||
"--extract-dir",
|
||||
outputRoot,
|
||||
target.inputPath,
|
||||
],
|
||||
{
|
||||
root: outputRoot,
|
||||
bytes: request.input.max_output_bytes,
|
||||
entries: request.input.max_output_files,
|
||||
},
|
||||
);
|
||||
const report = await readFirmwareCommandReport({
|
||||
path: reportPath,
|
||||
engine: engineName,
|
||||
operation: request.operation,
|
||||
execution: processRun,
|
||||
});
|
||||
const normalized = normalizeUnblobReport(report, {
|
||||
inputPath: target.inputPath,
|
||||
outputRoot,
|
||||
...target.selection,
|
||||
maxDepth: request.input.max_depth,
|
||||
});
|
||||
if (processRun.exit_code === 1 && normalized.diagnostics.length === 0)
|
||||
throw new AnalysisOutputError(
|
||||
request.operation,
|
||||
"Unblob failed without a reported extraction diagnostic",
|
||||
);
|
||||
const entries = await inventoryFirmwareOutput(
|
||||
outputRoot,
|
||||
request.input.max_output_bytes,
|
||||
request.input.max_output_files,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
result = await publishFirmwareExtraction({
|
||||
outputDirectory: request.input.output_directory,
|
||||
entries,
|
||||
normalized,
|
||||
selection: target.selection,
|
||||
engine,
|
||||
signal,
|
||||
});
|
||||
raw = { report, execution: processRun, version_execution: versionRun };
|
||||
}
|
||||
outcome = ok(
|
||||
createAnalysisExecution(
|
||||
result,
|
||||
engineName === "binwalk"
|
||||
? BINWALK_PROVIDER_IDENTITY
|
||||
: UNBLOB_PROVIDER_IDENTITY,
|
||||
{
|
||||
subject: target.subject,
|
||||
rawResult: raw,
|
||||
limitations,
|
||||
locations: [
|
||||
{
|
||||
kind: "file-offset-range",
|
||||
start: target.selection.offset,
|
||||
end: target.selection.offset + target.selection.length,
|
||||
},
|
||||
],
|
||||
},
|
||||
),
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
outcome = err(
|
||||
cause instanceof ProviderCleanupError
|
||||
? cause
|
||||
: options?.signal?.aborted === true
|
||||
? new AnalysisCancelledError(request.operation)
|
||||
: deadline.aborted
|
||||
? new AnalysisTimeoutError(
|
||||
request.operation,
|
||||
FIRMWARE_LIMITS.timeoutMs,
|
||||
)
|
||||
: cause instanceof AnalysisError
|
||||
? cause
|
||||
: cause instanceof z.ZodError
|
||||
? new AnalysisOutputError(
|
||||
request.operation,
|
||||
"Tool report violated the pinned producer protocol",
|
||||
{ cause },
|
||||
)
|
||||
: new ProviderAdapterError(engineName, request.operation, {
|
||||
cause,
|
||||
diagnostics: {
|
||||
reason:
|
||||
cause instanceof Error
|
||||
? cause.message
|
||||
: String(cause),
|
||||
},
|
||||
}),
|
||||
);
|
||||
if (!outcome.ok && outcome.error.cleanupIncomplete) {
|
||||
this.#cleanupFailure =
|
||||
root === undefined ||
|
||||
outcome.error.cleanupResources.includes(root.path)
|
||||
? outcome.error
|
||||
: new ProviderCleanupError(
|
||||
engineName,
|
||||
[...outcome.error.cleanupResources, root.path],
|
||||
outcome.error instanceof ProviderAdapterError
|
||||
? (outcome.error.diagnostics ?? {
|
||||
reason: outcome.error.message,
|
||||
})
|
||||
: { reason: outcome.error.message },
|
||||
{ cause: outcome.error },
|
||||
);
|
||||
}
|
||||
}
|
||||
// Never remove a workspace while process ownership remains uncertain.
|
||||
if (this.#cleanupFailure !== undefined) return err(this.#cleanupFailure);
|
||||
try {
|
||||
await root?.close();
|
||||
} catch (cause: unknown) {
|
||||
this.#cleanupFailure = new ProviderCleanupError(
|
||||
engineName,
|
||||
[
|
||||
root?.path ?? "unknown",
|
||||
...(outcome.ok && request.operation === "extract_firmware"
|
||||
? [request.input.output_directory]
|
||||
: []),
|
||||
],
|
||||
{
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
previous_result: outcome.ok ? outcome.value.result : null,
|
||||
},
|
||||
);
|
||||
return err(this.#cleanupFailure);
|
||||
}
|
||||
return outcome;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
import { createReadStream } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import type { z } from "zod";
|
||||
import { SafeOutputTree } from "../artifacts/SafeOutputTree.js";
|
||||
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import { firmwareResultSchemas } from "../domain/firmwareAnalysis.js";
|
||||
import { hashFirmwareFile, type FirmwareEntry } from "./FirmwareFiles.js";
|
||||
import type { normalizeUnblobReport } from "./FirmwareReports.js";
|
||||
|
||||
/** Publish verified regular files, preserving links as metadata and rolling back failed writes. */
|
||||
export const publishFirmwareExtraction = async (context: {
|
||||
outputDirectory: string;
|
||||
entries: readonly FirmwareEntry[];
|
||||
normalized: ReturnType<typeof normalizeUnblobReport>;
|
||||
selection: { offset: number; length: number; sha256: string };
|
||||
engine: z.infer<typeof firmwareResultSchemas.extract_firmware>["engine"];
|
||||
signal: AbortSignal;
|
||||
}) => {
|
||||
let tree: SafeOutputTree | undefined;
|
||||
try {
|
||||
const materialized = new Set(
|
||||
context.entries
|
||||
.filter((entry) => entry.kind === "file")
|
||||
.map((entry) => `$output/${entry.relativePath}`),
|
||||
);
|
||||
for (const path of context.normalized.files.keys()) {
|
||||
if (path !== "$input" && !materialized.has(path))
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
`Reported regular file is absent from materialized output: ${path}`,
|
||||
);
|
||||
}
|
||||
const files = [];
|
||||
const unpublished = [];
|
||||
tree = await SafeOutputTree.create(context.outputDirectory);
|
||||
for (const entry of context.entries) {
|
||||
if (entry.kind !== "file") {
|
||||
unpublished.push({
|
||||
relative_path: entry.relativePath,
|
||||
kind: entry.kind,
|
||||
link_target: entry.linkTarget,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
const logicalPath = `$output/${entry.relativePath}`;
|
||||
const reported = context.normalized.files.get(logicalPath);
|
||||
const sha256 = await hashFirmwareFile(entry.path, context.signal);
|
||||
if (
|
||||
(reported?.sha256 !== null &&
|
||||
reported?.sha256 !== undefined &&
|
||||
reported.sha256 !== sha256) ||
|
||||
(context.normalized.sizes.has(logicalPath) &&
|
||||
context.normalized.sizes.get(logicalPath) !== entry.size)
|
||||
)
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
`Extracted bytes disagree with provider report: ${entry.relativePath}`,
|
||||
);
|
||||
await tree.write(
|
||||
entry.relativePath,
|
||||
createReadStream(entry.path),
|
||||
sha256,
|
||||
context.signal,
|
||||
);
|
||||
files.push({
|
||||
relative_path: entry.relativePath,
|
||||
path: join(tree.outputRoot, entry.relativePath),
|
||||
size: entry.size,
|
||||
sha256,
|
||||
reported_sha256: reported?.sha256 ?? null,
|
||||
mime_type: reported?.mimeType ?? null,
|
||||
analysis_depth: reported?.depth ?? null,
|
||||
runtime_address: null,
|
||||
original_file_range: null,
|
||||
});
|
||||
}
|
||||
const result = firmwareResultSchemas.extract_firmware.parse({
|
||||
engine: context.engine,
|
||||
selection: context.selection,
|
||||
output_directory: tree.outputRoot,
|
||||
coverage:
|
||||
unpublished.length > 0 ? "partial" : context.normalized.coverage,
|
||||
files,
|
||||
unpublished_entries: unpublished,
|
||||
chunks: context.normalized.chunks,
|
||||
derivations: context.normalized.derivations,
|
||||
depth_limited_paths: context.normalized.depth_limited_paths,
|
||||
diagnostics: context.normalized.diagnostics,
|
||||
published_bytes: files.reduce((sum, file) => sum + file.size, 0),
|
||||
});
|
||||
context.signal.throwIfAborted();
|
||||
await tree.commit();
|
||||
return result;
|
||||
} catch (cause: unknown) {
|
||||
try {
|
||||
await tree?.rollback();
|
||||
} catch (cleanupCause: unknown) {
|
||||
throw new ProviderCleanupError(
|
||||
"unblob",
|
||||
[context.outputDirectory],
|
||||
{
|
||||
reason:
|
||||
cleanupCause instanceof Error
|
||||
? cleanupCause.message
|
||||
: String(cleanupCause),
|
||||
previous_error:
|
||||
cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ cause: cleanupCause },
|
||||
);
|
||||
}
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,34 @@
|
||||
/** Audited upstream sources; installed tool bytes are fingerprinted separately. */
|
||||
export const FIRMWARE_RELEASES = {
|
||||
binwalk: {
|
||||
version: "3.1.0",
|
||||
revision: "4fdab3d464d97b68e0af9088df3f9e2e1545b21c",
|
||||
repository: "https://github.com/ReFirmLabs/binwalk",
|
||||
},
|
||||
unblob: {
|
||||
version: "26.6.4",
|
||||
revision: "1fcc7a0a584a70a96c31f5a276c20944d199a089",
|
||||
repository: "https://github.com/onekey-sec/unblob",
|
||||
},
|
||||
} as const;
|
||||
|
||||
/** Catalog identities; Evidence uses the version observed from the executed tool. */
|
||||
export const BINWALK_PROVIDER_IDENTITY = {
|
||||
id: "binwalk",
|
||||
name: "Binwalk",
|
||||
version: FIRMWARE_RELEASES.binwalk.version,
|
||||
} as const;
|
||||
/** Catalog identity for explicit firmware extraction. */
|
||||
export const UNBLOB_PROVIDER_IDENTITY = {
|
||||
id: "unblob",
|
||||
name: "Unblob",
|
||||
version: FIRMWARE_RELEASES.unblob.version,
|
||||
} as const;
|
||||
|
||||
/** Resource policy for this Linux integration, not a claim of aggregate containment. */
|
||||
export const FIRMWARE_LIMITS = {
|
||||
inputBytes: 128 * 1024 * 1024,
|
||||
reportBytes: 8 * 1024 * 1024,
|
||||
addressSpaceBytes: 1024 * 1024 * 1024,
|
||||
timeoutMs: 120_000,
|
||||
} as const;
|
||||
@@ -0,0 +1,148 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
normalizeBinwalkReport,
|
||||
normalizeUnblobReport,
|
||||
} from "./FirmwareReports.js";
|
||||
|
||||
const context = {
|
||||
inputPath: "/private/input.bin",
|
||||
outputRoot: "/private/output",
|
||||
sha256: "a".repeat(64),
|
||||
length: 20,
|
||||
offset: 64,
|
||||
maxDepth: 3,
|
||||
};
|
||||
const root = () => ({
|
||||
task: {
|
||||
path: context.inputPath,
|
||||
depth: 0,
|
||||
blob_id: "",
|
||||
is_multi_file: false,
|
||||
},
|
||||
reports: [
|
||||
{
|
||||
__typename__: "StatReport",
|
||||
path: context.inputPath,
|
||||
size: 20,
|
||||
is_file: true,
|
||||
is_dir: false,
|
||||
is_link: false,
|
||||
link_target: null,
|
||||
},
|
||||
{ __typename__: "HashReport", sha256: context.sha256 },
|
||||
{
|
||||
__typename__: "UnknownChunkReport",
|
||||
id: "unknown",
|
||||
start_offset: 0,
|
||||
end_offset: 20,
|
||||
size: 20,
|
||||
randomness: null,
|
||||
},
|
||||
],
|
||||
subtasks: [],
|
||||
});
|
||||
|
||||
describe("firmware producer boundaries", () => {
|
||||
it("retains unknown chunks and their selected original-file range", () => {
|
||||
const normalized = normalizeUnblobReport([root()], context);
|
||||
expect(normalized.coverage).toBe("partial");
|
||||
expect(normalized.chunks[0]).toMatchObject({
|
||||
handler: null,
|
||||
range: { offset: 0, length: 20 },
|
||||
root_file_range: { offset: 64, length: 20 },
|
||||
});
|
||||
});
|
||||
it("preserves unfamiliar report types as explicit diagnostics", () => {
|
||||
const record = root();
|
||||
const normalized = normalizeUnblobReport(
|
||||
[
|
||||
{
|
||||
...record,
|
||||
reports: [
|
||||
...record.reports,
|
||||
{ __typename__: "FutureError", reason: "new producer facet" },
|
||||
],
|
||||
},
|
||||
],
|
||||
context,
|
||||
);
|
||||
expect(normalized.diagnostics).toEqual([
|
||||
{
|
||||
input_path: "$input",
|
||||
report: { __typename__: "FutureError", reason: "new producer facet" },
|
||||
},
|
||||
]);
|
||||
expect(normalized.coverage).toBe("partial");
|
||||
});
|
||||
it("rejects unknown tasks outside the owned output and disconnected task graphs", () => {
|
||||
const record = root();
|
||||
for (const path of [
|
||||
"/private/output/../escaped",
|
||||
"/private/output/unconnected",
|
||||
]) {
|
||||
const child = { ...record, task: { ...record.task, path, depth: 1 } };
|
||||
expect(() => normalizeUnblobReport([record, child], context)).toThrow();
|
||||
}
|
||||
});
|
||||
it("rejects absent root identity, inconsistent chunk bounds and duplicate tasks", () => {
|
||||
const record = root();
|
||||
expect(() => normalizeUnblobReport([], context)).toThrow("exactly one");
|
||||
expect(() =>
|
||||
normalizeUnblobReport(
|
||||
[
|
||||
{
|
||||
...record,
|
||||
reports: record.reports.filter(
|
||||
(report) => report.__typename__ !== "HashReport",
|
||||
),
|
||||
},
|
||||
],
|
||||
context,
|
||||
),
|
||||
).toThrow("hash");
|
||||
expect(() =>
|
||||
normalizeUnblobReport(
|
||||
[
|
||||
{
|
||||
...record,
|
||||
reports: [
|
||||
...record.reports,
|
||||
{
|
||||
__typename__: "UnknownChunkReport",
|
||||
id: "bad",
|
||||
start_offset: 10,
|
||||
end_offset: 21,
|
||||
size: 11,
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
context,
|
||||
),
|
||||
).toThrow("bounds");
|
||||
expect(() => normalizeUnblobReport([record, record], context)).toThrow(
|
||||
"exactly one",
|
||||
);
|
||||
});
|
||||
it("rejects empty, combined and extracting Binwalk reports", () => {
|
||||
const record = {
|
||||
Analysis: { file_path: context.inputPath, file_map: [], extractions: {} },
|
||||
};
|
||||
expect(normalizeBinwalkReport([record], context.inputPath, 20)).toEqual([]);
|
||||
for (const report of [
|
||||
[],
|
||||
[record, record],
|
||||
[
|
||||
{
|
||||
Analysis: {
|
||||
...record.Analysis,
|
||||
extractions: { hit: { status: "success" } },
|
||||
},
|
||||
},
|
||||
],
|
||||
])
|
||||
expect(() =>
|
||||
normalizeBinwalkReport(report, context.inputPath, 20),
|
||||
).toThrow();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,327 @@
|
||||
import { z } from "zod";
|
||||
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
|
||||
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
|
||||
import { digestSchema } from "../domain/digests.js";
|
||||
import { firmwareLogicalPath } from "./FirmwareFiles.js";
|
||||
|
||||
const integer = z.number().int().min(0).max(Number.MAX_SAFE_INTEGER);
|
||||
const signature = z.object({
|
||||
id: z.string().min(1),
|
||||
offset: integer,
|
||||
size: integer,
|
||||
name: z.string(),
|
||||
description: z.string(),
|
||||
confidence: z.number().int().min(0).max(255),
|
||||
always_display: z.boolean(),
|
||||
extraction_declined: z.boolean(),
|
||||
});
|
||||
const scan = z.array(
|
||||
z.object({
|
||||
Analysis: z.object({
|
||||
file_path: z.string(),
|
||||
file_map: z.array(signature),
|
||||
extractions: z.record(z.string(), jsonValueSchema),
|
||||
}),
|
||||
}),
|
||||
);
|
||||
const report = z.object({ __typename__: z.string() }).catchall(jsonValueSchema);
|
||||
const task = z.object({
|
||||
path: z.string(),
|
||||
depth: integer,
|
||||
blob_id: z.string(),
|
||||
is_multi_file: z.boolean(),
|
||||
});
|
||||
const extraction = z.array(
|
||||
z.object({ task, reports: z.array(report), subtasks: z.array(task) }),
|
||||
);
|
||||
const stat = z.object({
|
||||
__typename__: z.literal("StatReport"),
|
||||
path: z.string(),
|
||||
size: integer,
|
||||
is_dir: z.boolean(),
|
||||
is_file: z.boolean(),
|
||||
is_link: z.boolean(),
|
||||
link_target: z.string().nullable(),
|
||||
});
|
||||
const hashes = z.object({ sha256: digestSchema });
|
||||
const region = z.object({
|
||||
id: z.string(),
|
||||
start_offset: integer,
|
||||
end_offset: integer,
|
||||
size: integer,
|
||||
});
|
||||
const known = new Set([
|
||||
"StatReport",
|
||||
"HashReport",
|
||||
"FileMagicReport",
|
||||
"RandomnessReport",
|
||||
"ChunkReport",
|
||||
"UnknownChunkReport",
|
||||
"CarveDirectoryReport",
|
||||
"MultiFileReport",
|
||||
]);
|
||||
|
||||
/** Interpret the pinned Binwalk report without inventing validated region lengths. */
|
||||
export const normalizeBinwalkReport = (
|
||||
raw: unknown,
|
||||
inputPath: string,
|
||||
size: number,
|
||||
) => {
|
||||
const records = scan.parse(raw);
|
||||
const analysis = records[0]?.Analysis;
|
||||
if (
|
||||
records.length !== 1 ||
|
||||
analysis?.file_path !== inputPath ||
|
||||
Object.keys(analysis.extractions).length !== 0
|
||||
)
|
||||
throw new AnalysisOutputError(
|
||||
"inspect_firmware_regions",
|
||||
"Expected one inspection-only report for the selected immutable input",
|
||||
);
|
||||
return analysis.file_map.map((hit) => {
|
||||
if (hit.offset >= size || hit.size > size - hit.offset)
|
||||
throw new AnalysisOutputError(
|
||||
"inspect_firmware_regions",
|
||||
"Reported region lies outside the selected file",
|
||||
);
|
||||
return {
|
||||
provider_id: hit.id,
|
||||
offset: hit.offset,
|
||||
reported_size: hit.size,
|
||||
signature: hit.name,
|
||||
description: hit.description,
|
||||
reported_confidence: hit.confidence,
|
||||
size_basis: "provider_reported_validation_unknown" as const,
|
||||
};
|
||||
});
|
||||
};
|
||||
|
||||
/** Normalize Unblob's unordered task graph, retaining unknown and failed analyses. */
|
||||
export const normalizeUnblobReport = (
|
||||
raw: unknown,
|
||||
context: {
|
||||
inputPath: string;
|
||||
outputRoot: string;
|
||||
sha256: string;
|
||||
length: number;
|
||||
offset: number;
|
||||
maxDepth: number;
|
||||
},
|
||||
) => {
|
||||
const records = extraction.parse(raw);
|
||||
const logical = (path: string) =>
|
||||
firmwareLogicalPath(path, context.inputPath, context.outputRoot);
|
||||
const roots = records.filter(
|
||||
({ task: item }) => item.path === context.inputPath && item.depth === 0,
|
||||
);
|
||||
if (roots.length !== 1)
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
"Expected exactly one selected input task",
|
||||
);
|
||||
const rootHash = roots[0]?.reports.find(
|
||||
(r) => r.__typename__ === "HashReport",
|
||||
);
|
||||
if (
|
||||
rootHash === undefined ||
|
||||
hashes.parse(rootHash).sha256 !== context.sha256
|
||||
)
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
"Unblob input hash disagrees with selected snapshot",
|
||||
);
|
||||
const files = new Map<
|
||||
string,
|
||||
{ sha256: string | null; mimeType: string | null; depth: number }
|
||||
>();
|
||||
const sizes = new Map<string, number>();
|
||||
const depths: string[] = [];
|
||||
const diagnostics: JsonValue[] = [];
|
||||
const chunks: {
|
||||
input_path: string;
|
||||
provider_id: string;
|
||||
range: { offset: number; length: number };
|
||||
handler: string | null;
|
||||
encrypted: boolean | null;
|
||||
root_file_range: { offset: number; length: number } | null;
|
||||
}[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (const record of records) {
|
||||
const path = logical(record.task.path);
|
||||
if (seen.has(path))
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
`Duplicate analysis task: ${path}`,
|
||||
);
|
||||
seen.add(path);
|
||||
if (record.task.depth >= context.maxDepth) depths.push(path);
|
||||
const metadata = record.reports.find(
|
||||
(r) => r.__typename__ === "StatReport",
|
||||
);
|
||||
if (metadata === undefined)
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
`Missing StatReport: ${path}`,
|
||||
);
|
||||
const parsedStat = stat.parse(metadata);
|
||||
if (parsedStat.path !== record.task.path)
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
`StatReport is bound to another task: ${path}`,
|
||||
);
|
||||
if (path === "$input" && parsedStat.size !== context.length)
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
"Unblob input size disagrees with selected snapshot",
|
||||
);
|
||||
if (
|
||||
path === "$input" &&
|
||||
(!parsedStat.is_file || parsedStat.is_link || parsedStat.is_dir)
|
||||
)
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
"Selected input was not reported as a regular file",
|
||||
);
|
||||
sizes.set(path, parsedStat.size);
|
||||
const hash = record.reports.find((r) => r.__typename__ === "HashReport");
|
||||
const magic = record.reports.find(
|
||||
(r) => r.__typename__ === "FileMagicReport",
|
||||
);
|
||||
if (parsedStat.is_file && !parsedStat.is_link)
|
||||
files.set(path, {
|
||||
sha256: hash === undefined ? null : hashes.parse(hash).sha256,
|
||||
mimeType:
|
||||
magic === undefined
|
||||
? null
|
||||
: z.object({ mime_type: z.string() }).parse(magic).mime_type,
|
||||
depth: record.task.depth,
|
||||
});
|
||||
for (const item of record.reports) {
|
||||
if (!known.has(item.__typename__))
|
||||
diagnostics.push(
|
||||
jsonValueSchema.parse({ input_path: path, report: item }),
|
||||
);
|
||||
if (
|
||||
item.__typename__ === "ChunkReport" ||
|
||||
item.__typename__ === "UnknownChunkReport"
|
||||
) {
|
||||
const parsed = region.parse(item);
|
||||
if (
|
||||
parsed.end_offset <= parsed.start_offset ||
|
||||
parsed.end_offset > parsedStat.size ||
|
||||
parsed.size !== parsed.end_offset - parsed.start_offset
|
||||
)
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
`Invalid chunk bounds for ${path}`,
|
||||
);
|
||||
const detail =
|
||||
item.__typename__ === "ChunkReport"
|
||||
? z
|
||||
.object({
|
||||
handler_name: z.string(),
|
||||
is_encrypted: z.boolean(),
|
||||
extraction_reports: z.array(jsonValueSchema),
|
||||
})
|
||||
.parse(item)
|
||||
: null;
|
||||
if (detail !== null)
|
||||
for (const problem of detail.extraction_reports)
|
||||
diagnostics.push({ input_path: path, report: problem });
|
||||
chunks.push({
|
||||
input_path: path,
|
||||
provider_id: parsed.id,
|
||||
range: { offset: parsed.start_offset, length: parsed.size },
|
||||
handler: detail?.handler_name ?? null,
|
||||
encrypted: detail?.is_encrypted ?? null,
|
||||
root_file_range:
|
||||
path === "$input"
|
||||
? {
|
||||
offset: context.offset + parsed.start_offset,
|
||||
length: parsed.size,
|
||||
}
|
||||
: null,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
const recordsByPath = new Map(
|
||||
records.map((record) => [record.task.path, record.task]),
|
||||
);
|
||||
const chunksByOrigin = new Map<string, (typeof chunks)[number]>();
|
||||
for (const item of chunks) {
|
||||
const key = JSON.stringify([item.input_path, item.provider_id]);
|
||||
if (chunksByOrigin.has(key))
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
`Duplicate chunk identity in ${item.input_path}`,
|
||||
);
|
||||
chunksByOrigin.set(key, item);
|
||||
}
|
||||
const derivations = records.flatMap((record) =>
|
||||
record.subtasks.map((child) => {
|
||||
const parentPath = logical(record.task.path);
|
||||
const childPath = logical(child.path);
|
||||
if (!seen.has(childPath))
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
`Missing child task: ${childPath}`,
|
||||
);
|
||||
const reportedChild = recordsByPath.get(child.path);
|
||||
if (
|
||||
reportedChild?.depth !== child.depth ||
|
||||
reportedChild.blob_id !== child.blob_id ||
|
||||
reportedChild.is_multi_file !== child.is_multi_file
|
||||
)
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
`Child task identity disagrees with its analysis: ${childPath}`,
|
||||
);
|
||||
const origin = chunksByOrigin.get(
|
||||
JSON.stringify([parentPath, child.blob_id]),
|
||||
);
|
||||
return {
|
||||
parent_path: parentPath,
|
||||
child_path: childPath,
|
||||
provider_blob_id: child.blob_id,
|
||||
handler: origin?.handler ?? null,
|
||||
parent_file_range: origin?.range ?? null,
|
||||
};
|
||||
}),
|
||||
);
|
||||
const reachable = new Set(["$input"]);
|
||||
const pending = ["$input"];
|
||||
const childrenByParent = new Map<string, string[]>();
|
||||
for (const edge of derivations) {
|
||||
const children = childrenByParent.get(edge.parent_path) ?? [];
|
||||
children.push(edge.child_path);
|
||||
childrenByParent.set(edge.parent_path, children);
|
||||
}
|
||||
while (pending.length > 0) {
|
||||
const parent = pending.pop();
|
||||
for (const child of childrenByParent.get(parent ?? "") ?? []) {
|
||||
if (reachable.has(child)) continue;
|
||||
reachable.add(child);
|
||||
pending.push(child);
|
||||
}
|
||||
}
|
||||
if (reachable.size !== seen.size)
|
||||
throw new AnalysisOutputError(
|
||||
"extract_firmware",
|
||||
"Report contains tasks disconnected from the selected input",
|
||||
);
|
||||
return {
|
||||
files,
|
||||
sizes,
|
||||
chunks,
|
||||
derivations,
|
||||
depth_limited_paths: depths.sort(),
|
||||
diagnostics,
|
||||
coverage:
|
||||
depths.length > 0 ||
|
||||
diagnostics.length > 0 ||
|
||||
chunks.some((c) => c.handler === null || c.encrypted === true)
|
||||
? ("partial" as const)
|
||||
: ("complete" as const),
|
||||
};
|
||||
};
|
||||
File diff suppressed because one or more lines are too long
@@ -18,6 +18,10 @@ import { registerElectronTools } from "./registerElectronTools.js";
|
||||
import { registerEnhancedTools } from "./registerEnhancedTools.js";
|
||||
import { registerJavaScriptRuntimeObservationTools } from "./registerJavaScriptRuntimeObservationTools.js";
|
||||
import { registerManagedTools } from "./registerManagedTools.js";
|
||||
import { registerFirmwareTools } from "./registerFirmwareTools.js";
|
||||
import { FirmwareAnalysisService } from "../application/FirmwareAnalysisService.js";
|
||||
import type { FirmwareAnalysisPort } from "../application/FirmwareAnalysisPort.js";
|
||||
import { FirmwareProvider } from "../firmware/FirmwareProvider.js";
|
||||
import { registerAndroidTools } from "./registerAndroidTools.js";
|
||||
import { AndroidAnalysisService } from "../application/AndroidAnalysisService.js";
|
||||
import type { AndroidAnalysisPort } from "../application/AndroidAnalysisPort.js";
|
||||
@@ -38,6 +42,7 @@ const ACTIVE_TARGET_INSTRUCTIONS =
|
||||
|
||||
export interface CreateServerOptions {
|
||||
readonly logger?: Logger;
|
||||
readonly firmwareAnalysis?: FirmwareAnalysisPort;
|
||||
readonly androidAnalysis?: AndroidAnalysisPort;
|
||||
readonly browserObservation?: BrowserObservationPort;
|
||||
readonly browserScenarioCapture?: BrowserScenarioCapturePort;
|
||||
@@ -55,6 +60,14 @@ const installSessionToolAvailability = (
|
||||
if (session === undefined) return undefined;
|
||||
const policy = sessionAvailabilityPolicy(options.availabilityPolicy, {
|
||||
optionalFeatures: {
|
||||
firmwareInspectionEnabled:
|
||||
options.firmwareAnalysis !== undefined ||
|
||||
(process.platform === "linux" &&
|
||||
isAbsolute(process.env.REA_BINWALK_COMMAND ?? "")),
|
||||
firmwareExtractionEnabled:
|
||||
options.firmwareAnalysis !== undefined ||
|
||||
(process.platform === "linux" &&
|
||||
isAbsolute(process.env.REA_UNBLOB_COMMAND ?? "")),
|
||||
androidAnalysisEnabled:
|
||||
options.androidAnalysis !== undefined ||
|
||||
((process.platform === "linux" || process.platform === "darwin") &&
|
||||
@@ -127,6 +140,14 @@ export const createServer = (
|
||||
toolLogger,
|
||||
recordEvidence,
|
||||
);
|
||||
registerFirmwareTools(
|
||||
server,
|
||||
new FirmwareAnalysisService(
|
||||
options.firmwareAnalysis ?? new FirmwareProvider(),
|
||||
),
|
||||
toolLogger,
|
||||
recordEvidence,
|
||||
);
|
||||
registerObservationTools(toolContext);
|
||||
registerGuidedPrompts(server, analysis, session);
|
||||
if (session !== undefined) {
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
import type { McpServer, ServerContext } from "@modelcontextprotocol/server";
|
||||
import type { FirmwareAnalysisService } from "../application/FirmwareAnalysisService.js";
|
||||
import type { BinarySessionPort } from "../application/BinarySession.js";
|
||||
import { toolContract } from "../contracts/toolContracts.js";
|
||||
import type { ToolContract } from "../contracts/toolContractTypes.js";
|
||||
import type { FirmwareOperation } from "../domain/firmwareAnalysis.js";
|
||||
import type { Logger } from "../logger.js";
|
||||
import { logToolExecution } from "./toolLogging.js";
|
||||
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
|
||||
import { toCallToolResult } from "./toolResult.js";
|
||||
|
||||
/** Bind firmware handlers to their exact named schemas. */
|
||||
export const registerFirmwareTools = (
|
||||
server: McpServer,
|
||||
service: FirmwareAnalysisService,
|
||||
logger: Logger,
|
||||
recordEvidence?: BinarySessionPort["recordEvidence"],
|
||||
): void => {
|
||||
const handler =
|
||||
(contract: ToolContract<FirmwareOperation>) =>
|
||||
async (input: unknown, context: ServerContext) => {
|
||||
const result = await logToolExecution(logger, contract.name, () =>
|
||||
service.execute(contract.name, input, {
|
||||
signal: context.mcpReq.signal,
|
||||
}),
|
||||
);
|
||||
if (!result.ok) return toCallToolResult(result, contract);
|
||||
const recorded = recordEvidence?.(result.value);
|
||||
return recorded !== undefined && !recorded.ok
|
||||
? toCallToolResult(recorded, contract)
|
||||
: toCallToolResult(result, contract);
|
||||
};
|
||||
const inspect = toolContract("inspect_firmware_regions");
|
||||
server.registerTool(
|
||||
inspect.name,
|
||||
toolRegistrationOptions(inspect),
|
||||
handler(inspect),
|
||||
);
|
||||
const extract = toolContract("extract_firmware");
|
||||
server.registerTool(
|
||||
extract.name,
|
||||
toolRegistrationOptions(extract),
|
||||
handler(extract),
|
||||
);
|
||||
};
|
||||
@@ -12,6 +12,8 @@ export interface SessionAvailabilityDefaults {
|
||||
| "electronAutomationEnabled"
|
||||
| "v8InspectorObservationEnabled"
|
||||
| "androidAnalysisEnabled"
|
||||
| "firmwareInspectionEnabled"
|
||||
| "firmwareExtractionEnabled"
|
||||
>;
|
||||
}
|
||||
|
||||
@@ -23,6 +25,10 @@ export const sessionAvailabilityPolicy = (
|
||||
configured ??
|
||||
(() => ({
|
||||
processCaptureEnabled: platform !== "win32",
|
||||
firmwareInspectionEnabled:
|
||||
defaults.optionalFeatures?.firmwareInspectionEnabled ?? false,
|
||||
firmwareExtractionEnabled:
|
||||
defaults.optionalFeatures?.firmwareExtractionEnabled ?? false,
|
||||
androidAnalysisEnabled:
|
||||
defaults.optionalFeatures?.androidAnalysisEnabled ?? false,
|
||||
browserObservationEnabled:
|
||||
|
||||
@@ -22,6 +22,10 @@ import { PLAYWRIGHT_BROWSER_SCENARIO_PROVIDER_IDENTITY } from "../../../src/brow
|
||||
import { V8_INSPECTOR_PROVIDER_IDENTITY } from "../../../src/browser/V8InspectorProvider.js";
|
||||
import { CLI_COMMAND_NAMES } from "../../../src/cliCommandNames.js";
|
||||
import { createCli } from "../../../src/cli.js";
|
||||
import {
|
||||
BINWALK_PROVIDER_IDENTITY,
|
||||
UNBLOB_PROVIDER_IDENTITY,
|
||||
} from "../../../src/firmware/FirmwareRelease.js";
|
||||
import { JADX_PROVIDER_IDENTITY } from "../../../src/android/JadxRelease.js";
|
||||
import { TOOL_CONTRACTS } from "../../../src/contracts/toolContracts.js";
|
||||
import {
|
||||
@@ -73,6 +77,8 @@ describe("canonical product catalog", () => {
|
||||
[
|
||||
HOPPER_PROVIDER_IDENTITY,
|
||||
JADX_PROVIDER_IDENTITY,
|
||||
BINWALK_PROVIDER_IDENTITY,
|
||||
UNBLOB_PROVIDER_IDENTITY,
|
||||
GHIDRA_PROVIDER_IDENTITY,
|
||||
NATIVE_MACOS_PROVIDER_IDENTITY,
|
||||
ARTIFACT_GRAPH_PROVIDER,
|
||||
|
||||
@@ -0,0 +1,343 @@
|
||||
import { access, mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { expect, it as test } from "vitest";
|
||||
import {
|
||||
firmwareFixture,
|
||||
assertFirmwareCleanup,
|
||||
} from "../../fixtures/firmware/provider.js";
|
||||
import { firmwareResultSchemas } from "../../../src/domain/firmwareAnalysis.js";
|
||||
import { FirmwareAnalysisService } from "../../../src/application/FirmwareAnalysisService.js";
|
||||
import { FirmwareProvider } from "../../../src/firmware/FirmwareProvider.js";
|
||||
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
|
||||
import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
import { toCallToolResult } from "../../../src/server/toolResult.js";
|
||||
|
||||
const it = test.skipIf(process.platform !== "linux");
|
||||
|
||||
it("binds region observations to exact bytes and preserves provider-reported uncertainty", async () => {
|
||||
const fixture = await firmwareFixture();
|
||||
const result = await fixture.service.execute("inspect_firmware_regions", {
|
||||
path: fixture.path,
|
||||
});
|
||||
if (!result.ok) throw result.error;
|
||||
const normalized = firmwareResultSchemas.inspect_firmware_regions.parse(
|
||||
result.value.normalized_result,
|
||||
);
|
||||
expect(normalized.regions[0]).toMatchObject({
|
||||
offset: 2,
|
||||
reported_size: 4,
|
||||
size_basis: "provider_reported_validation_unknown",
|
||||
});
|
||||
expect(result.value.subject?.local_path).toBe(fixture.path);
|
||||
expect(result.value.subject?.format).toBe("file");
|
||||
expect(result.value.raw_result).toMatchObject({
|
||||
report: [{ Analysis: { file_map: [{ id: "hit" }] } }],
|
||||
});
|
||||
expect(fixture.launches[1]?.args).toEqual(
|
||||
expect.arrayContaining(["--as=1073741824", "--threads", "1"]),
|
||||
);
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it.each(["wrong-input", "bounds"])(
|
||||
"rejects a mismatched or out-of-bounds inspection: %s",
|
||||
async (mode) => {
|
||||
const fixture = await firmwareFixture(mode);
|
||||
const result = await fixture.service.execute("inspect_firmware_regions", {
|
||||
path: fixture.path,
|
||||
});
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisOutputError" },
|
||||
});
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
},
|
||||
);
|
||||
|
||||
it("extracts a selected interval with verified file identity and parent-byte lineage", async () => {
|
||||
const fixture = await firmwareFixture();
|
||||
const result = await fixture.service.execute("extract_firmware", {
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
range: { offset: 2, length: 4 },
|
||||
});
|
||||
if (!result.ok) throw result.error;
|
||||
const normalized = firmwareResultSchemas.extract_firmware.parse(
|
||||
result.value.normalized_result,
|
||||
);
|
||||
expect(normalized.selection).toMatchObject({ offset: 2, length: 4 });
|
||||
expect(normalized.files[0]).toMatchObject({
|
||||
relative_path: "rootfs/config",
|
||||
size: 14,
|
||||
original_file_range: null,
|
||||
runtime_address: null,
|
||||
});
|
||||
expect(normalized.chunks[0]?.root_file_range).toEqual({
|
||||
offset: 2,
|
||||
length: 4,
|
||||
});
|
||||
expect(normalized.derivations[0]).toMatchObject({
|
||||
parent_path: "$input",
|
||||
child_path: "$output/rootfs/config",
|
||||
handler: "gzip",
|
||||
parent_file_range: { offset: 0, length: 4 },
|
||||
});
|
||||
expect(await readFile(normalized.files[0]?.path ?? "", "utf8")).toBe(
|
||||
"firmware=true\n",
|
||||
);
|
||||
expect(fixture.launches[1]?.args).toEqual(
|
||||
expect.arrayContaining(["--process-num", "1", "--randomness-depth", "0"]),
|
||||
);
|
||||
expect(fixture.launches[1]?.args).not.toContain("--no-sandbox");
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it.each(["depth", "dependency", "link"])(
|
||||
"reports partial extraction for %s without losing inline results",
|
||||
async (mode) => {
|
||||
const fixture = await firmwareFixture(mode);
|
||||
const result = await fixture.service.execute("extract_firmware", {
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (!result.ok) throw result.error;
|
||||
const normalized = firmwareResultSchemas.extract_firmware.parse(
|
||||
result.value.normalized_result,
|
||||
);
|
||||
expect(normalized.coverage).toBe("partial");
|
||||
expect(normalized.files).toHaveLength(1);
|
||||
if (mode === "depth")
|
||||
expect(normalized.depth_limited_paths).toEqual(["$output/rootfs/config"]);
|
||||
if (mode === "dependency")
|
||||
expect(normalized.diagnostics).toEqual([
|
||||
{
|
||||
input_path: "$input",
|
||||
report: {
|
||||
__typename__: "ExtractorDependencyNotFoundReport",
|
||||
severity: "ERROR",
|
||||
dependencies: ["sasquatch"],
|
||||
},
|
||||
},
|
||||
]);
|
||||
if (mode === "link") {
|
||||
expect(normalized.unpublished_entries).toEqual([
|
||||
{
|
||||
relative_path: "escape",
|
||||
kind: "symlink",
|
||||
link_target: "/etc/passwd",
|
||||
},
|
||||
]);
|
||||
await expect(access(`${fixture.output}/escape`)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
}
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
},
|
||||
);
|
||||
|
||||
it("distinguishes a real output file named $input from the selected original", async () => {
|
||||
const fixture = await firmwareFixture("reserved-name");
|
||||
const outcome = await fixture.service.execute("extract_firmware", {
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (!outcome.ok) throw outcome.error;
|
||||
const result = firmwareResultSchemas.extract_firmware.parse(
|
||||
outcome.value.normalized_result,
|
||||
);
|
||||
expect(result.files[0]?.relative_path).toBe("$input");
|
||||
expect(result.derivations[0]).toMatchObject({
|
||||
parent_path: "$input",
|
||||
child_path: "$output/$input",
|
||||
});
|
||||
expect(await readFile(`${fixture.output}/$input`, "utf8")).toBe(
|
||||
"firmware=true\n",
|
||||
);
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it.each(["hash", "malformed", "budget", "missing-file"])(
|
||||
"rejects invalid or over-budget extraction and rolls back output: %s",
|
||||
async (mode) => {
|
||||
const fixture = await firmwareFixture(mode);
|
||||
const result = await fixture.service.execute("extract_firmware", {
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
...(mode === "budget" ? { max_output_bytes: 1000 } : {}),
|
||||
});
|
||||
expect(result.ok).toBe(false);
|
||||
await expect(access(fixture.output)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
},
|
||||
);
|
||||
|
||||
it("preserves existing output and rejects invalid intervals before invoking extraction", async () => {
|
||||
const fixture = await firmwareFixture();
|
||||
await mkdir(fixture.output);
|
||||
await writeFile(`${fixture.output}/existing`, "keep");
|
||||
const existing = await fixture.service.execute("extract_firmware", {
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
expect(existing.ok).toBe(false);
|
||||
expect(await readFile(`${fixture.output}/existing`, "utf8")).toBe("keep");
|
||||
const previous = fixture.launches.length;
|
||||
const invalid = await fixture.service.execute("extract_firmware", {
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
range: { offset: 100, length: 1 },
|
||||
});
|
||||
expect(invalid).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisInputError" },
|
||||
});
|
||||
expect(fixture.launches).toHaveLength(previous);
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it("retains startup stderr when Unblob exits 1 without a report", async () => {
|
||||
const fixture = await firmwareFixture("startup");
|
||||
const result = await fixture.service.execute("extract_firmware", {
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (result.ok) throw new Error("Expected startup failure");
|
||||
const details = {
|
||||
diagnostics: {
|
||||
stderr: expect.stringContaining("Landlock sandbox is not available"),
|
||||
exit_code: 1,
|
||||
},
|
||||
};
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({ details });
|
||||
expect(
|
||||
toCallToolResult(result, toolContract("extract_firmware"))
|
||||
.structuredContent,
|
||||
).toMatchObject({ error: { details } });
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
_tag: "ProviderAdapterError",
|
||||
diagnostics: {
|
||||
exit_code: 1,
|
||||
stderr: expect.stringContaining("Landlock sandbox is not available"),
|
||||
report_failure: expect.stringContaining("ENOENT"),
|
||||
},
|
||||
},
|
||||
});
|
||||
await expect(access(fixture.output)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it("distinguishes a successful process with missing output from startup failure", async () => {
|
||||
const fixture = await firmwareFixture("missing-report");
|
||||
const result = await fixture.service.execute("extract_firmware", {
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
_tag: "AnalysisOutputError",
|
||||
reason: expect.stringContaining("expected complete report"),
|
||||
},
|
||||
});
|
||||
if (result.ok) throw new Error("Expected missing-output failure");
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
details: { reason: expect.stringContaining("Producer forgot its report") },
|
||||
});
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it("cancels an owned active process and permits the next queued operation", async () => {
|
||||
const fixture = await firmwareFixture("stall");
|
||||
const controller = new AbortController();
|
||||
const pending = fixture.service.execute(
|
||||
"inspect_firmware_regions",
|
||||
{ path: fixture.path },
|
||||
{ signal: controller.signal },
|
||||
);
|
||||
await expect.poll(() => fixture.launches.length).toBe(2);
|
||||
controller.abort();
|
||||
expect(await pending).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisCancelledError" },
|
||||
});
|
||||
const cancelled = await fixture.service.execute(
|
||||
"inspect_firmware_regions",
|
||||
{ path: fixture.path },
|
||||
{ signal: AbortSignal.abort() },
|
||||
);
|
||||
expect(cancelled).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisCancelledError" },
|
||||
});
|
||||
expect(fixture.launches).toHaveLength(2);
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it("cancels a waiting request promptly while keeping later launches behind the active operation", async () => {
|
||||
const fixture = await firmwareFixture("stall");
|
||||
const activeController = new AbortController();
|
||||
const active = fixture.service.execute(
|
||||
"inspect_firmware_regions",
|
||||
{ path: fixture.path },
|
||||
{ signal: activeController.signal },
|
||||
);
|
||||
await expect.poll(() => fixture.launches.length).toBe(2);
|
||||
const queuedController = new AbortController();
|
||||
const queued = fixture.service.execute(
|
||||
"inspect_firmware_regions",
|
||||
{ path: fixture.path },
|
||||
{ signal: queuedController.signal },
|
||||
);
|
||||
queuedController.abort();
|
||||
expect(await queued).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisCancelledError" },
|
||||
});
|
||||
expect(fixture.launches).toHaveLength(2);
|
||||
activeController.abort();
|
||||
expect((await active).ok).toBe(false);
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it("retains an uncertain workspace and blocks new launches after cleanup failure", async () => {
|
||||
const fixture = await firmwareFixture("cleanup-failure");
|
||||
const first = await fixture.service.execute("inspect_firmware_regions", {
|
||||
path: fixture.path,
|
||||
});
|
||||
expect(first).toMatchObject({
|
||||
ok: false,
|
||||
error: { cleanupIncomplete: true },
|
||||
});
|
||||
const workspace = fixture.launches[0]?.cwd;
|
||||
expect(workspace).toBeDefined();
|
||||
if (workspace !== undefined) await access(workspace);
|
||||
const next = await fixture.service.execute("inspect_firmware_regions", {
|
||||
path: fixture.path,
|
||||
});
|
||||
expect(next).toMatchObject({ ok: false, error: { cleanupIncomplete: true } });
|
||||
expect(fixture.launches).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("rejects absent or unsupported engines without installing anything", async () => {
|
||||
const missing = await new FirmwareAnalysisService(
|
||||
new FirmwareProvider({}),
|
||||
).execute("inspect_firmware_regions", { path: "/tmp/unused" });
|
||||
expect(missing).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisCapabilityUnavailableError" },
|
||||
});
|
||||
const fixture = await firmwareFixture("version");
|
||||
const result = await fixture.service.execute("inspect_firmware_regions", {
|
||||
path: fixture.path,
|
||||
});
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisCapabilityUnavailableError" },
|
||||
});
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
});
|
||||
@@ -94,6 +94,7 @@ describe("enhanced MCP tools", () => {
|
||||
error: {
|
||||
code: "unreadable_output",
|
||||
category: "execution_failure",
|
||||
details: { operation: "decompile", reason: "failed" },
|
||||
message:
|
||||
"Analysis returned an unreadable result. Retry once; if it continues, run `rea doctor`.",
|
||||
retryable: false,
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
import { expect, it as test, onTestFinished } from "vitest";
|
||||
import { FIRMWARE_TOOL_CONTRACTS } from "../../../src/contracts/firmwareToolContracts.js";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
import { createTestBinarySession } from "../../fixtures/binarySession.js";
|
||||
import {
|
||||
firmwareFixture,
|
||||
assertFirmwareCleanup,
|
||||
} from "../../fixtures/firmware/provider.js";
|
||||
|
||||
const it = test.skipIf(process.platform !== "linux");
|
||||
|
||||
it("executes named firmware contracts with inline Evidence and no native binary admission", async () => {
|
||||
const fixture = await firmwareFixture();
|
||||
const session = createTestBinarySession(() => {
|
||||
throw new Error("Firmware inspection must not acquire a native provider");
|
||||
});
|
||||
const server = createServer(session, session, {
|
||||
firmwareAnalysis: fixture.provider,
|
||||
});
|
||||
const client = new Client({ name: "firmware-regression", version: "1" });
|
||||
onTestFinished(async () => {
|
||||
await client.close();
|
||||
await server.close();
|
||||
await session.close();
|
||||
});
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
await server.connect(serverTransport);
|
||||
await client.connect(clientTransport);
|
||||
const advertised = await client.listTools();
|
||||
for (const contract of FIRMWARE_TOOL_CONTRACTS) {
|
||||
expect(
|
||||
advertised.tools.find((tool) => tool.name === contract.name)?.annotations,
|
||||
).toMatchObject(contract.annotations);
|
||||
const response = await client.callTool({
|
||||
name: contract.name,
|
||||
arguments: {
|
||||
path: fixture.path,
|
||||
...(contract.name === "extract_firmware"
|
||||
? { output_directory: fixture.output }
|
||||
: {}),
|
||||
},
|
||||
});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
const result = contract.outputSchema.parse(response.structuredContent);
|
||||
const evidence = parseEvidence(result.evidence);
|
||||
expect(result.result).toEqual(evidence.normalized_result);
|
||||
expect(result.evidence_id).toBe(evidence.evidence_id);
|
||||
expect(evidence.operation).toBe(contract.name);
|
||||
expect(evidence.raw_result).not.toBeNull();
|
||||
}
|
||||
await assertFirmwareCleanup(fixture.launches);
|
||||
});
|
||||
Vendored
+129
@@ -0,0 +1,129 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, readFile, writeFile, symlink } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
const args = process.argv.slice(2);
|
||||
const mode = process.env.REA_FIRMWARE_FIXTURE_MODE;
|
||||
const engine = process.env.REA_FIRMWARE_FIXTURE_ENGINE;
|
||||
if (args.includes("--version")) {
|
||||
console.log(
|
||||
mode === "version"
|
||||
? "unsupported"
|
||||
: engine === "binwalk"
|
||||
? "binwalk 3.1.0"
|
||||
: "26.6.4",
|
||||
);
|
||||
} else if (mode === "stall") {
|
||||
setInterval(() => {}, 1000);
|
||||
} else if (mode === "startup") {
|
||||
console.error("Landlock sandbox is not available on this fixture host");
|
||||
process.exitCode = 1;
|
||||
} else if (mode === "missing-report") {
|
||||
console.error("Producer forgot its report");
|
||||
} else {
|
||||
const input = args.at(-1);
|
||||
const bytes = await readFile(input);
|
||||
const sha256 = createHash("sha256").update(bytes).digest("hex");
|
||||
const reportPath =
|
||||
args[args.indexOf(engine === "binwalk" ? "--log" : "--report") + 1];
|
||||
if (engine === "binwalk") {
|
||||
await writeFile(
|
||||
reportPath,
|
||||
JSON.stringify([
|
||||
{
|
||||
Analysis: {
|
||||
file_path: mode === "wrong-input" ? "/tmp/another.bin" : input,
|
||||
file_map: [
|
||||
{
|
||||
id: "hit",
|
||||
offset: 2,
|
||||
size: mode === "bounds" ? bytes.length : 4,
|
||||
name: "gzip",
|
||||
confidence: 250,
|
||||
description: "fixture signature",
|
||||
always_display: false,
|
||||
extraction_declined: false,
|
||||
},
|
||||
],
|
||||
extractions: {},
|
||||
},
|
||||
},
|
||||
]),
|
||||
);
|
||||
} else {
|
||||
const output = args[args.indexOf("--extract-dir") + 1];
|
||||
await mkdir(output, { recursive: true });
|
||||
const child =
|
||||
mode === "reserved-name"
|
||||
? join(output, "$input")
|
||||
: join(output, "rootfs", "config");
|
||||
await mkdir(join(output, "rootfs"));
|
||||
const content = Buffer.from("firmware=true\n");
|
||||
if (mode !== "missing-file")
|
||||
await writeFile(child, mode === "budget" ? Buffer.alloc(10000) : content);
|
||||
if (mode === "link") await symlink("/etc/passwd", join(output, "escape"));
|
||||
const childHash = createHash("sha256").update(content).digest("hex");
|
||||
const depth =
|
||||
mode === "depth" ? Number(args[args.indexOf("--depth") + 1]) : 1;
|
||||
const stat = (path, size) => ({
|
||||
__typename__: "StatReport",
|
||||
path,
|
||||
size,
|
||||
is_file: true,
|
||||
is_dir: false,
|
||||
is_link: false,
|
||||
link_target: null,
|
||||
});
|
||||
const task = (path, level, id) => ({
|
||||
path,
|
||||
depth: level,
|
||||
blob_id: id,
|
||||
is_multi_file: false,
|
||||
});
|
||||
const hash = (value) => ({ __typename__: "HashReport", sha256: value });
|
||||
const childTask = task(child, depth, "gzip-id");
|
||||
const records = [
|
||||
{
|
||||
task: childTask,
|
||||
reports: [
|
||||
stat(child, content.length),
|
||||
...(mode === "depth"
|
||||
? []
|
||||
: [hash(mode === "hash" ? "a".repeat(64) : childHash)]),
|
||||
],
|
||||
subtasks: [],
|
||||
},
|
||||
{
|
||||
task: task(input, 0, ""),
|
||||
reports: [
|
||||
stat(input, bytes.length),
|
||||
hash(sha256),
|
||||
{
|
||||
__typename__: "ChunkReport",
|
||||
id: "gzip-id",
|
||||
start_offset: 0,
|
||||
end_offset: bytes.length,
|
||||
size: bytes.length,
|
||||
handler_name: "gzip",
|
||||
is_encrypted: false,
|
||||
extraction_reports:
|
||||
mode === "dependency"
|
||||
? [
|
||||
{
|
||||
__typename__: "ExtractorDependencyNotFoundReport",
|
||||
severity: "ERROR",
|
||||
dependencies: ["sasquatch"],
|
||||
},
|
||||
]
|
||||
: [],
|
||||
},
|
||||
],
|
||||
subtasks: [childTask],
|
||||
},
|
||||
];
|
||||
await writeFile(
|
||||
reportPath,
|
||||
mode === "malformed" ? "[" : JSON.stringify(records),
|
||||
);
|
||||
if (mode === "dependency") process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
Vendored
+94
@@ -0,0 +1,94 @@
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { access, rm, writeFile } from "node:fs/promises";
|
||||
import { expect, onTestFinished } from "vitest";
|
||||
import { FirmwareProvider } from "../../../src/firmware/FirmwareProvider.js";
|
||||
import { FirmwareAnalysisService } from "../../../src/application/FirmwareAnalysisService.js";
|
||||
import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js";
|
||||
import { createTestTempDirectory } from "../temporaryDirectory.js";
|
||||
|
||||
/** Exercise real subprocess ownership with synthetic pinned producer representations. */
|
||||
export const firmwareFixture = async (mode = "normal") => {
|
||||
const root = await createTestTempDirectory("rea-firmware-test-");
|
||||
const path = join(root, "input.bin");
|
||||
await writeFile(path, Buffer.from("fixture-input-data"));
|
||||
onTestFinished(() => rm(root, { recursive: true, force: true }));
|
||||
const launches: {
|
||||
cwd: string | undefined;
|
||||
pid: number | undefined;
|
||||
args: readonly string[];
|
||||
}[] = [];
|
||||
const provider = new FirmwareProvider(
|
||||
{
|
||||
...process.env,
|
||||
REA_BINWALK_COMMAND: "/usr/bin/true",
|
||||
REA_UNBLOB_COMMAND: process.execPath,
|
||||
},
|
||||
async (options) => {
|
||||
const start = options.arguments.indexOf("--") + 2;
|
||||
const args = options.arguments.slice(start);
|
||||
const engine =
|
||||
options.arguments[start - 1]?.endsWith("/true") === true
|
||||
? "binwalk"
|
||||
: "unblob";
|
||||
const spawned = await spawnOwnedProviderProcess({
|
||||
...options,
|
||||
command: process.execPath,
|
||||
expectedCommand: process.execPath,
|
||||
arguments: [
|
||||
fileURLToPath(new URL("./producer.mjs", import.meta.url)),
|
||||
...args,
|
||||
],
|
||||
env: {
|
||||
...options.env,
|
||||
REA_FIRMWARE_FIXTURE_MODE: mode,
|
||||
REA_FIRMWARE_FIXTURE_ENGINE: engine,
|
||||
},
|
||||
});
|
||||
launches.push({
|
||||
cwd: options.cwd,
|
||||
pid: spawned.process.pid,
|
||||
args: options.arguments,
|
||||
});
|
||||
if (mode === "cleanup-failure") {
|
||||
onTestFinished(async () => {
|
||||
if (options.cwd !== undefined)
|
||||
await rm(options.cwd, { recursive: true, force: true });
|
||||
});
|
||||
return {
|
||||
...spawned,
|
||||
cleanup: async () => ({
|
||||
cleaned: false,
|
||||
reason: "injected ownership failure",
|
||||
}),
|
||||
};
|
||||
}
|
||||
return spawned;
|
||||
},
|
||||
);
|
||||
return {
|
||||
provider,
|
||||
service: new FirmwareAnalysisService(provider),
|
||||
path,
|
||||
output: join(root, "output"),
|
||||
root,
|
||||
launches,
|
||||
};
|
||||
};
|
||||
|
||||
/** Verify only the acquired process groups and private runtime directories. */
|
||||
export const assertFirmwareCleanup = async (
|
||||
launches: Awaited<ReturnType<typeof firmwareFixture>>["launches"],
|
||||
): Promise<void> => {
|
||||
for (const launch of launches) {
|
||||
const pid = launch.pid;
|
||||
if (pid !== undefined)
|
||||
expect(() => process.kill(-pid, 0)).toThrowError(
|
||||
expect.objectContaining({ code: "ESRCH" }),
|
||||
);
|
||||
if (launch.cwd !== undefined)
|
||||
await expect(access(launch.cwd)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
}
|
||||
};
|
||||
Vendored
+18
@@ -40,3 +40,21 @@ are checked against fixed SHA-256 values. Fixture provenance is recorded in
|
||||
`scripts/fixtures/android-apidemos.json`; the source uses Apache-2.0. This is a
|
||||
static analysis fixture: no emulator, device, SDK installation or app execution
|
||||
is needed. APKs and JARs are neither tracked nor included in npm packages.
|
||||
|
||||
## Firmware engines
|
||||
|
||||
| Engine | Source | Release | Pinned commit | License |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Binwalk | <https://github.com/ReFirmLabs/binwalk> | v3.1.0 | `4fdab3d464d97b68e0af9088df3f9e2e1545b21c` | MIT, upstream `LICENSE` |
|
||||
| Unblob | <https://github.com/onekey-sec/unblob> | 26.6.4 | `1fcc7a0a584a70a96c31f5a276c20944d199a089` | MIT, upstream `LICENSE` |
|
||||
|
||||
Source remains unmodified in `third_party/binwalk` and `third_party/unblob`.
|
||||
Release records live in `src/firmware/FirmwareRelease.ts`; REA adapters own the
|
||||
CLI/report interpretation and lifecycle in `src/firmware/`. Caller-supplied
|
||||
executables retain their actual launcher digest and unknown source revision;
|
||||
version output alone does not establish source identity. REA does not bundle
|
||||
extractors or download/install firmware engines during analysis.
|
||||
|
||||
On updates: inspect actual pinned CLI and report producers, preserve licenses,
|
||||
update gitlinks/release records, run report regressions and the real firmware
|
||||
lane. See [firmware setup and coverage](../docs/firmware-analysis.md).
|
||||
|
||||
+1
Submodule third_party/binwalk added at 4fdab3d464
+1
Submodule third_party/unblob added at 1fcc7a0a58
+1
-1
@@ -37,7 +37,7 @@ const TEST_PROJECTS = [
|
||||
name: "adapters",
|
||||
include: [
|
||||
"src/*.test.ts",
|
||||
"src/{artifacts,browser,dotnet,ghidra,hopper,native,process,reference,server,windows}/**/*.test.ts",
|
||||
"src/{artifacts,browser,dotnet,firmware,ghidra,hopper,native,process,reference,server,windows}/**/*.test.ts",
|
||||
],
|
||||
pool: "forks" as const,
|
||||
maxWorkers: MAX_TEST_WORKERS,
|
||||
|
||||
Reference in New Issue
Block a user