feat(artifacts): add U-Boot legacy uImage reader (part of #7129) (#7234)

* feat(artifacts): add U-Boot legacy uImage reader (part of #7129)

Recognize legacy uImages by their 0x27051956 magic and inventory them as containers: 64-byte header facts (arch/os/type/compression raw and named, load/entry addresses, name, timestamp) with header and data CRC32 verified, stored payload extractable as payload.bin, and multi-file images split from the stored size table into image-<n>.bin. Compressed payloads stay as stored; nothing is executed or decompressed. Wires the format through classification, the artifact reader dispatch, extraction, and the evidence/snapshot format vocabularies. Related to #7129 (uImage part only); HEX/SREC and FIT are out of scope.

* fix(uimage): preserve aligned component ranges and exact header facts

* fix(uimage): retain buffered members through consumption and cancellation

* fix(uimage): preserve active read and validation failure precedence

---------

Co-authored-by: gensokyo-dev <[email protected]>
This commit is contained in:
Cipherandgensokyo-dev authored and GitHub committed 2026-10-28 19:11:41 +08:00
1 parent 06d1247836
commit 18a8e71c9f
25 files changed
+1744 -17

No files matched your search

@@ -662,3 +662,10 @@ Virtual member offsets are not physical carrier locations. Format 6–8 readers
cache one decoded block and support retained filtered/paged views. No
decryption or SerializedFile object decoding is provided. See
[UnityFS](https://github.com/morluto/rea/blob/main/docs/unityfs.md).
U-Boot legacy uImages use the same inspection/extraction workflows. Prefer a
retained summary and selected inventory views for header facts or component
IDs. Header addresses/CPU/OS are declarations; payloads remain as stored.
Multi-file components retain their aligned physical ranges; the full payload
is also materialized. Unknown name text retains its original 32 bytes. See
[U-Boot uImage](https://github.com/morluto/rea/blob/main/docs/uimage.md).
+10
View File
@@ -281,6 +281,16 @@ rea extract-artifact ./firmware.hex --json
See [Intel HEX and S-record images](hex-records.md) for recognition, region
names, metadata and limits.
Inspect and extract stored U-Boot legacy uImage payloads and components:
```sh
rea inspect-artifact ./firmware.bin --detail summary --evidence_output firmware.json --json
rea extract-artifact ./firmware.bin --json
```
See [U-Boot uImage](uimage.md) for verified CRCs, physical component ranges,
unknown names, resource limits and retained views. Compressed bytes stay stored.
Inspect local model declarations without loading weights:
```bash
+14
View File
@@ -300,3 +300,17 @@ file for its address regions, declared start address and record facts, then
`extract-artifact` / `extract_artifact` for one raw file per region. Decoding is
portable and requires no external tool. See
[Intel HEX and S-record images](hex-records.md).
## Inspect U-Boot legacy uImages
`inspect-artifact` / `extract-artifact` recognize a legacy uImage by its
`0x27051956` magic and inventory it without running or decompressing it.
The root facts record the 64-byte header (architecture, OS, type, and
compression as raw values and names, load and entry addresses, image name,
and timestamp) with the header and data CRC32 verified; the stored payload
is extractable as `payload.bin`, and multi-file images additionally expose
each sub-image as `image-<n>.bin` from the stored size table. Declared
addresses are facts from the header, not proven entry points, and compressed
payloads stay as stored. Intel HEX/S-record and FIT images are separate
readers and are not covered here. See [U-Boot legacy uImage](uimage.md) for
component padding, preserved names, resource policies and verification scope.
+14 -14
View File
@@ -47,20 +47,20 @@ For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
## Choose a guide
| Task | Guides |
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [PE imports](pe-imports.md), [PE TLS/load config](pe-load-config.md), [ELF notes](elf-notes.md), [Linux kernel modules](kernel-modules.md), [Apple IPS](apple-ips.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [PE imports](pe-imports.md), [PE TLS/load config](pe-load-config.md), [ELF notes](elf-notes.md), [Apple IPS](apple-ips.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [GameCube DOL](ghidra-dol.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
| Inspect assemblies, packages, or firmware | [Godot PCK](godot-pck.md), [UnityFS](unityfs.md), [.NET](managed-code-analysis.md), [Python packages](python-packaged-applications.md), [Android](android-analysis.md), [APK signatures](apk-signature.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
| Serve recorded behavior as an agent environment | [Agent environments](agent-environments.md) |
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
| Capture and compare a command | [Process capture](process-capture.md) |
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
| Task | Guides |
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [PE imports](pe-imports.md), [PE TLS/load config](pe-load-config.md), [ELF notes](elf-notes.md), [Linux kernel modules](kernel-modules.md), [Apple IPS](apple-ips.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [PE imports](pe-imports.md), [PE TLS/load config](pe-load-config.md), [ELF notes](elf-notes.md), [Apple IPS](apple-ips.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [GameCube DOL](ghidra-dol.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
| Inspect assemblies, packages, or firmware | [Godot PCK](godot-pck.md), [UnityFS](unityfs.md), [U-Boot uImage](uimage.md), [.NET](managed-code-analysis.md), [Python packages](python-packaged-applications.md), [Android](android-analysis.md), [APK signatures](apk-signature.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
| Serve recorded behavior as an agent environment | [Agent environments](agent-environments.md) |
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
| Capture and compare a command | [Process capture](process-capture.md) |
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
For development, use [testing](testing.md), [client verification](client-verification.md),
[tool design](tool-design.md), [admission limits and recovery](design/limits-and-recovery.md),
+72
View File
@@ -0,0 +1,72 @@
# U-Boot legacy uImage containers
`inspect-artifact` and `extract-artifact` recognize the big-endian
`0x27051956` legacy uImage signature regardless of filename. They read local
stored bytes without booting, executing, decompressing or modifying the image.
FIT is a different format; its analysis is outside this reader.
The same domain header/table facts and artifact reader serve CLI and MCP.
The root is an archive with unknown executable architecture. The header's CPU,
OS, type, compression, load address and entry address are declarations, not proof of
compatible or executable payloads. Raw numeric identifiers remain available
alongside names; unrecognized identifiers have the name `unknown`.
The header's CRC32 and the declared payload's CRC32 must verify before a
successful inventory. `payload.bin` contains the complete stored payload.
Multi-file images also expose `image-0.bin`, `image-1.bin`, and so on. These
component names are generated from table order. Components follow the zero-terminated big-endian size table, with four-byte
padding after every nonfinal component. Their physical source ranges and
content hashes retain precise provenance. Extraction writes the complete
payload and every component, so multi-file output can approach twice the input
size. Compressed payloads remain compressed.
The fixed 32-byte name field is retained as `name_bytes_hex`. `name` contains
the UTF-8 bytes before the first NUL, or null when those bytes cannot be decoded
exactly. For example, a producer can truncate a multibyte character at the
32-byte boundary; replacement characters do not establish a source name.
## Bounded work and retained facts
The reader admits at most 4,096 components, a resource policy rather than a
format maximum. It reads at most 16,388 bytes for the size table and refuses an
excess count as a capacity limit before its remaining payload CRC scan.
Shared artifact identity hashing has separate cost. The
same prefix participates in the native incremental CRC32 calculation; no
custom bytewise CRC table or full-payload buffer is retained. Remaining CRC
reads and member streams use 64 KiB chunks. Parsed facts are cached for one reader. Carrier state is checked during
directory preparation, at member open and after stream completion; metadata
getters return the retained facts. Close joins active member
pipelines and final validation before releasing the descriptor; failed cleanup
remains owned for retry.
Prefer a retained summary when only a few facts are needed:
```sh
rea inspect-artifact firmware.bin --detail summary \
--evidence_output firmware.evidence.json --json
rea extract-artifact firmware.bin --json
```
With MCP, `inspect_artifact` with `detail: "summary"` retains the complete
Evidence. Use `inspect_analysis_view` and its `parent_evidence_id` for exact
logical paths, batches of occurrence/content IDs, selected fields and pages.
Coverage and limitations remain attached. Retained views are historical
observations tied to the saved Evidence. Complete Evidence remains available;
these views do not narrow the examined inventory.
## Verification scope
The public OpenWrt 24.10.0 GL-AR150 initramfs kernel is checked against its
[published digest](https://downloads.openwrt.org/releases/24.10.0/targets/ath79/generic/sha256sums):
`027c60a7061462d8145b11efa23ab02b334acb9a23a215f9095860b632ace4ff`.
Independent `mkimage`/`dumpimage` 2023.01 provide producer and extraction
oracles. Generated containers use real host ELF files, system metadata and
stored gzip bytes; their declared ARM64, VxWorks or RISC-V labels do not prove
boot/runtime compatibility. U-Boot's identifier and alignment definitions are
pinned to commit `ece349ade2973e220f524ce59e59711cc919263f`.
Source and compiled CLI/full/compact MCP tests cover aligned component ranges,
unknown identifiers/text, malformed/truncated inputs, CRC failures, capacity,
source changes, cancellation, pending reads, final validation and cleanup
retry. Real producer results, complete-task cost measurements and limitations
are reported in the PR. No target runtime or decompressor behavior is verified.
+12 -3
View File
@@ -75,6 +75,7 @@ import {
} from "../domain/apple/appleArchiveFormat.js";
import { hasDiskImageSignature } from "../domain/apple/diskImageFormat.js";
import { hasGodotPckHeader } from "../domain/godotPckFormat.js";
import { hasUImageMagic } from "../domain/uimageFormat.js";
import { hasUnityFsHeader } from "../domain/unityfsFormat.js";
import { detectHexRecordEncoding } from "../domain/firmware/hexRecords.js";
import {
@@ -668,6 +669,7 @@ const detectArtifactFormat = async (
if (hasZipSignature(magic)) return zipRootPackageFormat(lower);
if (hasGodotPckHeader(magic)) return "godot-pck";
if (hasUnityFsHeader(magic)) return "unityfs";
if (hasUImageMagic(magic)) return "uimage";
const hexRecords = detectHexRecordEncoding(magic);
if (hexRecords !== undefined) return hexRecords;
// Binary plists, including keyed archives, often use other extensions.
@@ -737,9 +739,16 @@ const isArchiveFormat = (
>,
): format is Extract<BinaryTarget, { kind: "archive" }>["format"] =>
isZipPackageFormat(format) ||
["asar", "godot-pck", "unityfs", "dmg", "pkg", "intel-hex", "srec"].includes(
format,
);
[
"asar",
"godot-pck",
"unityfs",
"uimage",
"dmg",
"pkg",
"intel-hex",
"srec",
].includes(format);
/** Whether the leading bytes begin any executable header family, valid or not. */
const claimsLeadingExecutableHeader = async (
+1
View File
@@ -178,6 +178,7 @@ export interface ArtifactReader {
| "asar"
| "godot-pck"
| "unityfs"
| "uimage"
| "pyinstaller"
| "node-sea"
| "intel-hex"
@@ -0,0 +1,363 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { buffer } from "node:stream/consumers";
import { finished } from "node:stream/promises";
import { setImmediate } from "node:timers/promises";
import { expect, it, vi } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { createDeferred } from "../../tests/fixtures/binarySession.js";
import { awaitAcknowledgement } from "../../tests/fixtures/lifecycleAcknowledgement.js";
import { openRegularFile } from "../filesystem/RegularFile.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { UImageArtifactReader } from "./UImageArtifactReader.js";
import { buildUImage, buildMultiPayload } from "../../tests/fixtures/uimage.js";
import {
parseUImageHeaderFields,
parseMultiImageTable,
} from "../domain/uimageFormat.js";
const fixture = async (bytes: Buffer): Promise<string> => {
const path = join(
await createTestTempDirectory("rea-uimage-boundary-"),
"image",
);
await writeFile(path, bytes);
return path;
};
const entries = async (reader: UImageArtifactReader) => {
const result = [];
for await (const entry of reader.entries()) result.push(entry);
return result;
};
it.each([
[13, "nios", 13, "lynxos"],
[14, "microblaze", 14, "vxworks"],
[21, "openrisc", 16, "qnx"],
[22, "arm64", 17, "u-boot"],
[26, "riscv", 29, "elf"],
] as const)(
"retains official architecture %s and OS identifiers",
(arch, archName, os, osName) => {
const result = parseUImageHeaderFields(buildUImage({ arch, os }));
expect(result).toMatchObject({
ok: true,
value: { arch, archName, os, osName },
});
},
);
it("retains raw name bytes when mkimage's fixed field truncates UTF-8", () => {
const bytes = buildUImage({ name: "测".repeat(11) });
const result = parseUImageHeaderFields(bytes);
expect(result).toMatchObject({
ok: true,
value: { name: null, nameBytesHex: bytes.subarray(32, 64).toString("hex") },
});
});
it("aligns large component extents without signed 32-bit coercion or allocation", () => {
const table = Buffer.alloc(12);
table.writeUInt32BE(0x8000_0001, 0);
table.writeUInt32BE(1, 4);
expect(parseMultiImageTable(table, 12 + 0x8000_0004 + 1)).toEqual({
ok: true,
value: [
{ index: 0, byteOffset: 76, declaredSize: 0x8000_0001 },
{ index: 1, byteOffset: 76 + 0x8000_0004, declaredSize: 1 },
],
});
});
it("preserves cancellation during payload CRC admission", async () => {
const path = await fixture(
buildUImage({ payload: Buffer.alloc(128 * 1024, 0x61) }),
);
const admitted = await openRegularFile(path, { symlinks: "reject" });
const reader = new UImageArtifactReader(path, admitted);
const controller = new AbortController(),
reason = new Error("cancel CRC verification");
const nativeRead = admitted.handle.read.bind(admitted.handle);
const reads = vi.spyOn(admitted.handle, "read").mockImplementation((async (
...args: unknown[]
) => {
const result = await Reflect.apply(nativeRead, admitted.handle, args);
if (args[3] === 64) controller.abort(reason);
return result;
}) as typeof admitted.handle.read);
try {
await expect(
(async () => {
for await (const _entry of reader.entries(controller.signal)) {
/* Exhaust the admitted operation. */
}
})(),
).rejects.toMatchObject({ reason: "cancelled", cause: reason });
} finally {
await reader.close();
reads.mockRestore();
await admitted.handle.close();
}
});
it("admits a multi-image count before scanning the remaining payload", async () => {
const payload = Buffer.concat([
buildMultiPayload(Array.from({ length: 4097 }, () => Buffer.from("x"))),
Buffer.alloc(1024 * 1024),
]);
const path = await fixture(buildUImage({ type: 4, payload }));
const admitted = await openRegularFile(path, { symlinks: "reject" });
const reader = new UImageArtifactReader(path, admitted);
const reads = vi.spyOn(admitted.handle, "read");
try {
await expect(entries(reader)).rejects.toMatchObject({
reason: "limit",
boundary: "capacity",
});
expect(reads).toHaveBeenCalledTimes(2); // header and bounded table, no payload scan
} finally {
await reader.close();
reads.mockRestore();
await admitted.handle.close();
}
});
it("does not complete paused inventory after its reader closes", async () => {
const reader = new UImageArtifactReader(
await fixture(
buildUImage({
type: 4,
payload: buildMultiPayload([Buffer.from("a"), Buffer.from("b")]),
}),
),
);
const iterator = reader.entries()[Symbol.asyncIterator]();
try {
expect((await iterator.next()).done).toBe(false);
await reader.close();
await expect(iterator.next()).rejects.toMatchObject({
reason: "unavailable",
});
} finally {
await reader.close();
}
});
it.each(["cancel", "close"] as const)(
"retires fully buffered, unconsumed member bytes on %s",
async (action) => {
const reader = new UImageArtifactReader(await fixture(buildUImage()));
const controller = new AbortController();
const reason = new Error("cancel buffered member");
try {
const [entry] = await entries(reader);
const stream = await reader.open(entry!, controller.signal);
await finished(stream, { readable: false, cleanup: true });
await setImmediate();
expect(stream.readableLength).toBeGreaterThan(0);
if (action === "cancel") controller.abort(reason);
else await reader.close();
await expect(buffer(stream)).rejects.toMatchObject(
action === "cancel"
? { reason: "cancelled", cause: reason }
: { reason: "unavailable" },
);
expect(stream.destroyed).toBe(true);
} finally {
await reader.close();
}
},
);
it("joins a member's outstanding positional read before descriptor closure", async () => {
const path = await fixture(buildUImage());
const admitted = await openRegularFile(path, { symlinks: "reject" });
const reader = new UImageArtifactReader(path, admitted);
const reached = createDeferred<void>(),
release = createDeferred<void>();
const nativeRead = admitted.handle.read.bind(admitted.handle);
const reads = vi.spyOn(admitted.handle, "read");
let outcome: Promise<unknown> | undefined;
let closing: Promise<void> | undefined;
try {
const [entry] = await entries(reader);
reads.mockImplementation((async (...args: unknown[]) => {
reached.resolve();
await release.promise;
return Reflect.apply(nativeRead, admitted.handle, args);
}) as typeof admitted.handle.read);
const stream = await reader.open(entry!);
outcome = buffer(stream).then(
() => ({ ok: true }),
(cause: unknown) => ({ ok: false, cause }),
);
await awaitAcknowledgement(
reached.promise,
outcome,
"member positional read",
);
let closed = false;
closing = reader.close().then(() => {
closed = true;
});
await setImmediate();
expect(closed).toBe(false);
release.resolve();
await closing;
expect(stream.closed).toBe(true);
} finally {
release.resolve();
await outcome;
await closing;
await reader.close();
reads.mockRestore();
await admitted.handle.close();
}
});
it("retains failed descriptor cleanup for retry", async () => {
const reader = new UImageArtifactReader(await fixture(buildUImage()));
const [entry] = await entries(reader);
const stream = await reader.open(entry!);
const failure = new Error("uImage close denied");
const close = vi
.spyOn(OwnedFileHandle.prototype, "close")
.mockRejectedValueOnce(failure);
try {
await expect(reader.close()).rejects.toBe(failure);
await expect(reader.open(entry!)).rejects.toMatchObject({
reason: "unavailable",
});
expect(stream.closed).toBe(true);
await reader.close();
expect(close).toHaveBeenCalledTimes(2);
} finally {
close.mockRestore();
await reader.close();
}
});
it.each(["cancel", "cancel-with-validation-error", "close"] as const)(
"joins final freshness validation during %s",
async (action) => {
const path = await fixture(buildUImage());
const admitted = await openRegularFile(path, { symlinks: "reject" });
const reader = new UImageArtifactReader(path, admitted);
const reached = createDeferred<void>(),
release = createDeferred<void>();
const nativeStat = admitted.handle.stat.bind(admitted.handle);
const nativeRead = admitted.handle.read.bind(admitted.handle);
const reads = vi.spyOn(admitted.handle, "read");
const stats = vi.spyOn(admitted.handle, "stat");
const controller = new AbortController(),
reason = new Error("cancel final validation");
const validationFailure = new Error("final descriptor stat failed");
let outcome: Promise<unknown> | undefined;
let closing: Promise<void> | undefined;
try {
const [entry] = await entries(reader);
const memberEnd = BigInt(entry!.byteOffset! + entry!.declaredSize!);
let memberReadComplete = false;
reads.mockImplementation((async (...args: unknown[]) => {
const result = (await Reflect.apply(
nativeRead,
admitted.handle,
args,
)) as {
bytesRead: number;
buffer: Buffer;
};
if (typeof args[3] === "bigint")
memberReadComplete = args[3] + BigInt(result.bytesRead) === memberEnd;
return result;
}) as typeof admitted.handle.read);
stats.mockImplementation((async (...args: unknown[]) => {
const result = await Reflect.apply(nativeStat, admitted.handle, args);
if (memberReadComplete) {
memberReadComplete = false;
reached.resolve();
await release.promise;
if (action === "cancel-with-validation-error")
throw validationFailure;
}
return result;
}) as typeof admitted.handle.stat);
const stream = await reader.open(entry!, controller.signal);
outcome = buffer(stream).then(
() => ({ ok: true }) as const,
(cause: unknown) => ({ ok: false, cause }) as const,
);
await awaitAcknowledgement(
reached.promise,
outcome,
"post-read freshness validation",
);
let closed = false;
if (action !== "close") controller.abort(reason);
else
closing = reader.close().then(() => {
closed = true;
});
await setImmediate();
expect(closed).toBe(false);
release.resolve();
expect(await outcome).toMatchObject({
ok: false,
cause:
action === "cancel-with-validation-error"
? validationFailure
: action === "cancel"
? { reason: "cancelled", cause: reason }
: { reason: "unavailable" },
});
await closing;
} finally {
release.resolve();
await outcome;
await closing;
await reader.close();
stats.mockRestore();
reads.mockRestore();
await admitted.handle.close();
}
},
);
it("rejects a source changed after its header and CRC were prepared", async () => {
const bytes = buildUImage();
const path = await fixture(bytes),
reader = new UImageArtifactReader(path);
try {
const [entry] = await entries(reader);
await writeFile(path, Buffer.concat([bytes, Buffer.from([0])]));
await expect(reader.open(entry!)).rejects.toMatchObject({
reason: "integrity",
});
} finally {
await reader.close();
}
});
it("checks a multi-chunk stored payload incrementally and preserves read cancellation", async () => {
const bytes = buildUImage({ payload: Buffer.alloc(2 * 1024 * 1024, 0x61) });
const path = await fixture(bytes);
const reader = new UImageArtifactReader(path);
try {
const [entry] = await entries(reader);
const controller = new AbortController(),
reason = new Error("cancel member bytes");
const stream = await reader.open(entry!, controller.signal);
const iterator = stream[Symbol.asyncIterator]();
await iterator.next();
controller.abort(reason);
await expect(iterator.next()).rejects.toMatchObject({
reason: "cancelled",
cause: reason,
});
expect(
(await buffer(await reader.open(entry!))).equals(bytes.subarray(64)),
).toBe(true);
} finally {
await reader.close();
}
});
+313
View File
@@ -0,0 +1,313 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import type { Readable } from "node:stream";
import { expect, it } from "vitest";
import { ArtifactReaderFailure } from "./ArtifactReader.js";
import { UImageArtifactReader } from "./UImageArtifactReader.js";
import { buildMultiPayload, buildUImage } from "../../tests/fixtures/uimage.js";
import {
parseMultiImageTable,
parseUImageHeaderFields,
UIMAGE_HEADER_SIZE,
UIMAGE_MAGIC,
} from "../domain/uimageFormat.js";
import { classifyArtifactBytesAtPath } from "../../tests/fixtures/artifactClassification.js";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { scanCanonicalArtifactInventory } from "../../tests/fixtures/artifactInventory.js";
const collect = async (stream: Readable): Promise<Buffer> => {
const chunks: Buffer[] = [];
for await (const chunk of stream) chunks.push(chunk as Buffer);
return Buffer.concat(chunks);
};
const writeCarrier = async (name: string, bytes: Buffer): Promise<string> => {
const directory = await createTestTempDirectory("rea-uimage-");
const path = join(directory, name);
await writeFile(path, bytes);
return path;
};
it("parses header facts with raw and named fields", () => {
const bytes = buildUImage({
arch: 2,
os: 5,
type: 2,
compression: 0,
loadAddress: 0x8000,
entryPoint: 0x8000,
timestamp: 1700000000,
name: "test-kernel",
payload: Buffer.from("kernel-bytes"),
});
const parsed = parseUImageHeaderFields(bytes.subarray(0, UIMAGE_HEADER_SIZE));
expect(parsed.ok).toBe(true);
if (!parsed.ok) return;
expect(parsed.value).toMatchObject({
magic: UIMAGE_MAGIC,
arch: 2,
archName: "arm",
os: 5,
osName: "linux",
type: 2,
typeName: "kernel",
compression: 0,
compressionName: "none",
loadAddress: 0x8000,
entryPoint: 0x8000,
timestamp: 1700000000,
timestampIso: "2023-11-14T22:13:20.000Z",
payloadSize: "kernel-bytes".length,
name: "test-kernel",
});
});
it("names unknown enum values as unknown instead of failing", () => {
const bytes = buildUImage({
arch: 250,
os: 250,
type: 250,
compression: 250,
});
const parsed = parseUImageHeaderFields(bytes.subarray(0, UIMAGE_HEADER_SIZE));
expect(parsed.ok).toBe(true);
if (!parsed.ok) return;
expect(parsed.value).toMatchObject({
archName: "unknown",
osName: "unknown",
typeName: "unknown",
compressionName: "unknown",
});
});
it("rejects a bad magic and a bad header CRC", () => {
const good = buildUImage();
const badMagic = Buffer.from(good);
badMagic.writeUInt32BE(0xdeadbeef, 0);
expect(
parseUImageHeaderFields(badMagic.subarray(0, UIMAGE_HEADER_SIZE)).ok,
).toBe(false);
const badCrc = buildUImage({ corruptHeaderCrc: true });
const parsed = parseUImageHeaderFields(
badCrc.subarray(0, UIMAGE_HEADER_SIZE),
);
expect(parsed.ok).toBe(false);
if (!parsed.ok)
expect(parsed.error.message).toContain("header CRC32 mismatch");
});
it("splits a multi-image size table and rejects malformed tables", () => {
const parts = [Buffer.from("aaa"), Buffer.from("bbbbb")];
const payload = buildMultiPayload(parts);
const parsed = parseMultiImageTable(payload, payload.length);
expect(parsed.ok).toBe(true);
if (!parsed.ok) return;
expect(parsed.value).toEqual([
{
index: 0,
byteOffset: UIMAGE_HEADER_SIZE + 12,
declaredSize: 3,
},
{
index: 1,
byteOffset: UIMAGE_HEADER_SIZE + 16,
declaredSize: 5,
},
]);
const noTerminator = Buffer.from([0, 0, 0, 3, 0x61, 0x62, 0x63]);
expect(parseMultiImageTable(noTerminator, noTerminator.length).ok).toBe(
false,
);
const overrun = Buffer.concat([
Buffer.from([0, 0, 0, 99, 0, 0, 0, 0]),
Buffer.from("short"),
]);
expect(parseMultiImageTable(overrun, overrun.length).ok).toBe(false);
});
it("classifies uImage bytes as a container by content, not by suffix", () => {
const bytes = buildUImage();
expect(classifyArtifactBytesAtPath("firmware.bin", bytes)).toEqual({
kind: "container",
format: "uimage",
});
expect(
classifyArtifactBytesAtPath("firmware.bin", Buffer.from("nope")),
).toEqual({
kind: "resource",
format: "file",
});
});
it("inventories a kernel image with its stored payload", async () => {
const payload = Buffer.from("kernel-bytes");
const path = await writeCarrier(
"uimage-kernel",
buildUImage({
arch: 22,
os: 5,
type: 2,
compression: 0,
loadAddress: 0x40080000,
entryPoint: 0x40080000,
name: "arm64-kernel",
payload,
}),
);
const snapshot = await scanCanonicalArtifactInventory(path);
expect(snapshot.manifest.root_format).toBe("uimage");
expect(snapshot.coverage.status).toBe("complete");
expect(snapshot.limitations.join(" ")).toContain("not decompressed");
const root = snapshot.occurrences.find((row) => row.logical_path === ".");
expect(root).toMatchObject({ artifact_kind: "container" });
expect(root?.format_metadata).toMatchObject({
kind: "uimage",
magic: UIMAGE_MAGIC,
arch: 22,
arch_name: "arm64",
os_name: "linux",
type_name: "kernel",
compression_name: "none",
load_address: 0x40080000,
entry_point: 0x40080000,
name: "arm64-kernel",
crc_verified: true,
sub_image_count: 0,
});
const entry = snapshot.occurrences.find(
(row) => row.logical_path === "payload.bin",
);
expect(entry).toMatchObject({
declared_size: payload.length,
artifact_format: "file",
});
const reader = new UImageArtifactReader(path);
try {
const entries = [];
for await (const item of reader.entries()) entries.push(item);
expect(entries).toHaveLength(1);
expect(
(await collect(await reader.open(entries[0]!))).equals(payload),
).toBe(true);
} finally {
await reader.close();
}
});
it("splits multi images and keeps compressed payloads as stored", async () => {
const partA = Buffer.from("first-part");
const partB = Buffer.from("second-part-longer");
const multi = await writeCarrier(
"uimage-multi",
buildUImage({
type: 4,
name: "multi",
payload: buildMultiPayload([partA, partB]),
}),
);
const snapshot = await scanCanonicalArtifactInventory(multi);
expect(snapshot.manifest.root_format).toBe("uimage");
const root = snapshot.occurrences.find((row) => row.logical_path === ".");
expect(root?.format_metadata).toMatchObject({
type_name: "multi",
sub_image_count: 2,
});
expect(snapshot.limitations.join(" ")).toContain("size table");
const reader = new UImageArtifactReader(multi);
try {
const entries = [];
for await (const item of reader.entries()) entries.push(item);
expect(entries.map((item) => item.path).sort()).toEqual([
"image-0.bin",
"image-1.bin",
"payload.bin",
]);
const byPath = new Map(entries.map((item) => [item.path, item]));
expect(
(await collect(await reader.open(byPath.get("image-0.bin")!))).equals(
partA,
),
).toBe(true);
expect(
(await collect(await reader.open(byPath.get("image-1.bin")!))).equals(
partB,
),
).toBe(true);
expect(
(await collect(await reader.open(byPath.get("payload.bin")!))).equals(
buildMultiPayload([partA, partB]),
),
).toBe(true);
} finally {
await reader.close();
}
const gzipPayload = Buffer.from([0x1f, 0x8b, 0x08, 0, 0, 0, 0, 0, 0, 3]);
const gzip = await writeCarrier(
"uimage-gzip",
buildUImage({ type: 2, compression: 1, payload: gzipPayload }),
);
const gzipSnapshot = await scanCanonicalArtifactInventory(gzip);
expect(
gzipSnapshot.occurrences.find((row) => row.logical_path === ".")
?.format_metadata,
).toMatchObject({ compression_name: "gzip" });
const gzipReader = new UImageArtifactReader(gzip);
try {
const entries = [];
for await (const item of gzipReader.entries()) entries.push(item);
expect(
(await collect(await gzipReader.open(entries[0]!))).equals(gzipPayload),
).toBe(true);
} finally {
await gzipReader.close();
}
});
it("fails closed on corrupt CRCs, truncation, and bad multi tables", async () => {
const cases: [string, Buffer, RegExp][] = [
["data CRC", buildUImage({ corruptDataCrc: true }), /data CRC32 mismatch/u],
[
"header CRC",
buildUImage({ corruptHeaderCrc: true }),
/header CRC32 mismatch/u,
],
[
"truncated payload",
buildUImage({ payload: Buffer.from("twelve-bytes") }).subarray(0, 70),
/truncated/u,
],
[
"multi table without terminator",
buildUImage({
type: 4,
payload: Buffer.from([0, 0, 0, 3, 0x61, 0x62, 0x63]),
}),
/no zero terminator/u,
],
];
for (const [label, bytes, pattern] of cases) {
const path = await writeCarrier(
`uimage-bad-${label.replaceAll(" ", "-")}`,
bytes,
);
await expect(scanCanonicalArtifactInventory(path)).rejects.toThrow(pattern);
}
});
it("rejects an entry that does not match its owning inventory", async () => {
const path = await writeCarrier("uimage-mismatch", buildUImage());
const reader = new UImageArtifactReader(path);
try {
const entries = [];
for await (const item of reader.entries()) entries.push(item);
const forged = { ...entries[0]!, declaredSize: 999 };
await expect(reader.open(forged)).rejects.toBeInstanceOf(
ArtifactReaderFailure,
);
} finally {
await reader.close();
}
});
+358
View File
@@ -0,0 +1,358 @@
import { Readable, Transform, pipeline } from "node:stream";
import { crc32 } from "node:zlib";
import { readArtifactFileBytes } from "./ArtifactFileRange.js";
import { setImmediate } from "node:timers/promises";
import type {
UImageEntryMetadata,
UImageMetadata,
} from "../domain/artifactFormatMetadata.js";
import { ArtifactFileSource } from "./ArtifactFileSource.js";
import { assertArtifactReadActive } from "./ArtifactFileOperation.js";
import {
ArtifactReaderFailure,
copyArtifactEntry,
sameArtifactEntry,
type ArtifactEntry,
type ArtifactReader,
type ArtifactTraversalOptions,
} from "./ArtifactReader.js";
import { ArtifactResourceScope } from "./ArtifactResourceScope.js";
import { readFileHandleChunks } from "../filesystem/readFileHandleChunks.js";
import type { StableRegularFileDescriptor } from "../filesystem/RegularFile.js";
import {
parseMultiImageTable,
parseUImageHeaderFields,
UIMAGE_HEADER_SIZE,
UIMAGE_MAGIC,
UIMAGE_MULTI_TABLE_WINDOW,
type UImageHeaderFields,
type UImageParseError,
type UImageSubImage,
} from "../domain/uimageFormat.js";
const PAYLOAD_PATH = "payload.bin";
const READ_CHUNK_BYTES = 64 * 1024;
type UImageEntry = ArtifactEntry & {
readonly byteOffset: number;
readonly declaredSize: number;
readonly formatMetadata: UImageEntryMetadata;
};
/** Read a U-Boot legacy uImage without running, decompressing, or altering it. */
export class UImageArtifactReader implements ArtifactReader {
readonly format = "uimage" as const;
readonly #scope = new ArtifactResourceScope();
readonly #entries = new Map<string, UImageEntry>();
readonly #streams = new Set<Readable>();
readonly #source: ArtifactFileSource;
#prepared: Promise<void> | undefined;
#header: UImageMetadata | undefined;
#closing = false;
constructor(path: string, admitted?: StableRegularFileDescriptor) {
this.#source = new ArtifactFileSource(path, "U-Boot uImage", admitted);
}
formatMetadata(): UImageMetadata | undefined {
return this.#header === undefined ? undefined : { ...this.#header };
}
async prepareContainer(
expectedSha256: string,
signal?: AbortSignal,
): Promise<void> {
await this.#scope.run(async () => {
this.#assertOpen(signal);
await this.#source.bindSha256(expectedSha256, signal);
});
}
async *entries(
signal?: AbortSignal,
options?: ArtifactTraversalOptions,
): AsyncIterable<ArtifactEntry> {
await this.#scope.run(async () => {
this.#assertOpen(signal);
this.#prepared ??= this.#prepare(signal);
await this.#prepared;
await this.#source.read(signal);
});
this.#assertOpen(signal);
options?.onCoverage?.({
status: "complete",
limitations:
this.#header?.type_name === "multi"
? [
"Multi-image sub-images are split from the stored size table; the payload entry retains the full stored bytes.",
]
: [],
});
let index = 0;
for (const entry of this.#entries.values()) {
if (index % 256 === 0) await setImmediate();
index += 1;
this.#assertOpen(signal);
yield copyArtifactEntry(entry);
}
this.#assertOpen(signal);
}
async open(entry: ArtifactEntry, signal?: AbortSignal): Promise<Readable> {
return this.#scope.run(async () => {
this.#assertOpen(signal);
const stored = this.#entries.get(entry.adapterKey);
if (stored === undefined || !sameArtifactEntry(entry, stored))
throw new ArtifactReaderFailure(
"integrity",
"uImage entry does not match its owning inventory",
);
const { handle } = await this.#source.read(signal);
this.#assertOpen(signal);
const source =
stored.declaredSize === 0
? Readable.from([])
: readFileHandleChunks(handle, {
start: stored.byteOffset,
end: stored.byteOffset + stored.declaredSize - 1,
signal,
cancellationFailure: (reason) =>
new ArtifactReaderFailure(
"cancelled",
"uImage read cancelled",
{
cause: reason,
},
),
});
const output = this.#memberStream(stored, signal);
output.on("error", () => undefined);
this.#streams.add(output);
let producerComplete = false;
const onAbort = (): void => {
// Active reads/validation own failure precedence. After they complete,
// cancellation must still retire any unconsumed readable bytes.
if (!producerComplete || output.destroyed) return;
output.destroy(
new ArtifactReaderFailure("cancelled", "uImage read cancelled", {
cause: signal?.reason,
}),
);
};
const retired = new Promise<void>((resolve) => {
output.once("close", () => {
this.#streams.delete(output);
signal?.removeEventListener("abort", onAbort);
resolve();
});
});
signal?.addEventListener("abort", onAbort, { once: true });
if (signal?.aborted) onAbort();
void this.#scope
.run(async () => {
// Writable completion can leave readable bytes buffered. Retain both
// producer I/O and the caller's stream until consumption or closure.
await Promise.all([
new Promise<void>((resolve) =>
pipeline(source, output, () => {
producerComplete = true;
if (signal?.aborted) onAbort();
resolve();
}),
),
retired,
]);
})
.catch((cause: unknown) =>
output.destroy(
cause instanceof Error
? cause
: new Error("uImage stream ownership failed", { cause }),
),
);
return output;
});
}
async close(): Promise<void> {
this.#closing = true;
for (const stream of this.#streams)
stream.destroy(
new ArtifactReaderFailure("unavailable", "uImage reader closed"),
);
await this.#scope.close();
await this.#source.close();
this.#entries.clear();
}
provenance(): readonly [] {
return [];
}
#assertOpen(signal?: AbortSignal): void {
assertArtifactReadActive(signal);
if (this.#closing)
throw new ArtifactReaderFailure("unavailable", "uImage reader closed");
}
async #prepare(signal?: AbortSignal): Promise<void> {
const { handle, initial } = await this.#source.read(signal);
const headerBytes = await readArtifactFileBytes(
handle,
0,
Math.min(initial.size, UIMAGE_HEADER_SIZE),
initial.size,
signal,
);
const fields = parseUImageHeaderFields(headerBytes);
if (!fields.ok) throw parseFailure(fields.error);
const header = fields.value;
if (initial.size < UIMAGE_HEADER_SIZE + header.payloadSize)
throw formatFailure(
`uImage payload is truncated: header declares ${header.payloadSize} payload bytes but only ${initial.size - UIMAGE_HEADER_SIZE} follow the header`,
);
// Admit the bounded size table first, then include those same bytes in the
// native CRC stream. No payload byte needs to be read twice for verification.
let dataCrc32 = 0;
let read = 0;
let subImages: readonly UImageSubImage[] = [];
if (header.type === 4) {
const tablePrefix = await readArtifactFileBytes(
handle,
UIMAGE_HEADER_SIZE,
Math.min(header.payloadSize, UIMAGE_MULTI_TABLE_WINDOW),
initial.size,
signal,
);
const parsed = parseMultiImageTable(tablePrefix, header.payloadSize);
if (!parsed.ok) throw parseFailure(parsed.error);
subImages = parsed.value;
dataCrc32 = crc32(tablePrefix);
read = tablePrefix.length;
}
while (read < header.payloadSize) {
this.#assertOpen(signal);
const chunk = await readArtifactFileBytes(
handle,
UIMAGE_HEADER_SIZE + read,
Math.min(READ_CHUNK_BYTES, header.payloadSize - read),
initial.size,
signal,
);
dataCrc32 = crc32(chunk, dataCrc32);
read += chunk.length;
}
if (dataCrc32 !== header.dataCrc32)
throw formatFailure(
`uImage data CRC32 mismatch: declared 0x${header.dataCrc32.toString(16)}, computed 0x${dataCrc32.toString(16)}`,
);
this.#header = toMetadata(header, subImages.length);
const payload = entryFor("payload", 0, {
byteOffset: UIMAGE_HEADER_SIZE,
declaredSize: header.payloadSize,
});
this.#entries.set(payload.adapterKey, payload);
for (const sub of subImages) {
const entry = entryFor("sub-image", sub.index, sub);
this.#entries.set(entry.adapterKey, entry);
}
}
#memberStream(entry: UImageEntry, signal?: AbortSignal): Transform {
let bytes = 0;
const check = async (): Promise<void> => {
if (bytes !== entry.declaredSize)
throw formatFailure(`member ended early: ${entry.path}`);
await this.#source.read(signal);
};
return new Transform({
transform(chunk: Buffer, _encoding, callback) {
bytes += chunk.length;
callback(null, chunk);
},
flush(callback) {
void check().then(
() => callback(),
(cause: unknown) =>
callback(
cause instanceof Error
? cause
: new Error("uImage read failed", { cause }),
),
);
},
});
}
}
const formatFailure = (
detail: string,
options?: ErrorOptions,
): ArtifactReaderFailure =>
new ArtifactReaderFailure(
"format",
`uImage is malformed: ${detail}`,
options,
);
const parseFailure = (failure: UImageParseError): ArtifactReaderFailure =>
failure._tag === "UImageCapacityFailure"
? new ArtifactReaderFailure("limit", failure.message, {
boundary: "capacity",
cause: failure,
})
: formatFailure(failure.message, { cause: failure });
const entryFor = (
role: "payload" | "sub-image",
index: number,
range: Pick<UImageSubImage, "byteOffset" | "declaredSize">,
): UImageEntry => {
const payload = role === "payload";
const path = payload ? PAYLOAD_PATH : `image-${String(index)}.bin`;
const adapterKey = payload
? "uimage:payload"
: `uimage:image-${String(index)}`;
return {
path,
kind: "file",
declaredSize: range.declaredSize,
compressedSize: null,
executable: false,
encrypted: false,
byteOffset: range.byteOffset,
declaredIntegrity: null,
unpacked: false,
limitations: [],
adapterKey,
formatMetadata: { kind: "uimage-entry", role, index },
sourcePath: path,
};
};
const toMetadata = (
facts: UImageHeaderFields,
subImageCount: number,
): UImageMetadata => ({
kind: "uimage",
magic: UIMAGE_MAGIC,
timestamp: facts.timestamp,
timestamp_iso: facts.timestampIso,
payload_size: facts.payloadSize,
load_address: facts.loadAddress,
entry_point: facts.entryPoint,
header_crc32: facts.headerCrc32,
data_crc32: facts.dataCrc32,
crc_verified: true,
os: facts.os,
os_name: facts.osName,
arch: facts.arch,
arch_name: facts.archName,
type: facts.type,
type_name: facts.typeName,
compression: facts.compression,
compression_name: facts.compressionName,
name: facts.name,
name_bytes_hex: facts.nameBytesHex,
sub_image_count: subImageCount,
});
+4
View File
@@ -11,6 +11,7 @@ import { MachOSliceArtifactReader } from "./MachOSliceArtifactReader.js";
import { NativeDmgArtifactReader } from "./NativeDmgArtifactReader.js";
import { NodeSeaArtifactReader } from "./NodeSeaArtifactReader.js";
import { PyInstallerArtifactReader } from "./PyInstallerArtifactReader.js";
import { UImageArtifactReader } from "./UImageArtifactReader.js";
import { UnityFsArtifactReader } from "./UnityFsArtifactReader.js";
import { ZipArtifactReader } from "./ZipArtifactReader.js";
import type { ArtifactOccurrence } from "../domain/artifactGraph.js";
@@ -28,6 +29,7 @@ export type ArtifactReaderFormat =
| "node-sea"
| "godot-pck"
| "unityfs"
| "uimage"
| "intel-hex"
| "srec"
| "mach-o-universal"
@@ -110,6 +112,8 @@ export function createArtifactReader(
? admission
: rootSource,
);
case "uimage":
return new UImageArtifactReader(path, rootSource);
case "unityfs":
return new UnityFsArtifactReader(path, rootSource);
case "intel-hex":
@@ -587,6 +587,7 @@ const requireExtractableFormat = async (
format !== "asar" &&
format !== "godot-pck" &&
format !== "unityfs" &&
format !== "uimage" &&
format !== "intel-hex" &&
format !== "srec" &&
format !== "mach-o-universal" &&
@@ -1,4 +1,5 @@
import { hasGodotPckHeader } from "../../domain/godotPckFormat.js";
import { hasUImageMagic } from "../../domain/uimageFormat.js";
import { hasUnityFsHeader } from "../../domain/unityfsFormat.js";
import { detectHexRecordEncoding } from "../../domain/firmware/hexRecords.js";
import type { ArtifactClassificationEvidence } from "../ArtifactClassificationProbe.js";
@@ -281,6 +282,7 @@ export const rootOccurrenceFor = (
"asar",
"godot-pck",
"unityfs",
"uimage",
"dmg",
"pkg",
"pyinstaller",
@@ -581,6 +583,7 @@ const artifactRoleForFormat = (
if (
format === "godot-pck" ||
format === "unityfs" ||
format === "uimage" ||
format === "intel-hex" ||
format === "srec"
)
@@ -602,6 +605,7 @@ export const classifyArtifactBytes = (
if (hasZipSignature(prefix)) return "zip";
if (hasGodotPckHeader(prefix)) return "godot-pck";
if (hasUnityFsHeader(prefix)) return "unityfs";
if (hasUImageMagic(prefix)) return "uimage";
const hexRecords = detectHexRecordEncoding(prefix);
if (hexRecords !== undefined) return hexRecords;
if (hasBinaryPlistSignature(prefix)) return "plist";
+1
View File
@@ -166,6 +166,7 @@ export const classifyAndHashRootForInventory = async (
format === "mach-o-universal" ||
format === "godot-pck" ||
format === "unityfs" ||
format === "uimage" ||
format === "pyinstaller" ||
format === "node-sea" ||
format === "intel-hex" ||
+6
View File
@@ -79,6 +79,11 @@ export const inventoryCoverage = (
"Intel HEX/S-record inventory decodes data records into address-contiguous regions. It does not establish CPU, byte order, memory map, or that a declared start address is executable; gaps are unwritten, not zero.",
]
: []),
...(reader.format === "uimage"
? [
"uImage inventory establishes the 64-byte legacy header facts and stored payload bytes. Compressed payloads are not decompressed, and the image is not executed.",
]
: []),
...traversal.limitations,
],
};
@@ -103,6 +108,7 @@ export const inventoryCoverage = (
format === "asar" ||
format === "godot-pck" ||
format === "unityfs" ||
format === "uimage" ||
format === "node-sea" ||
format === "intel-hex" ||
format === "srec"
+1
View File
@@ -82,6 +82,7 @@ const formatSchema = z.enum([
"plist",
"javascript",
"source-map",
"uimage",
]);
const kindSchema = z.enum(["executable", "database", "archive", "artifact"]);
const subjectBaseSchema = z.object({
+39
View File
@@ -215,6 +215,43 @@ export const pyInstallerEntryDeclaration = (
type_code: metadata.type_code,
});
/**
* U-Boot legacy uImage header facts. Declared addresses are facts from the
* header, not proven entry points; CRCs are verified before inventory.
*/
export const uImageMetadataSchema = z.strictObject({
kind: z.literal("uimage"),
magic: z.literal(0x27051956),
timestamp: uint32,
timestamp_iso: z.string(),
payload_size: uint32,
load_address: uint32,
entry_point: uint32,
header_crc32: uint32,
data_crc32: uint32,
crc_verified: z.literal(true),
os: z.number().int().min(0).max(255),
os_name: z.string(),
arch: z.number().int().min(0).max(255),
arch_name: z.string(),
type: z.number().int().min(0).max(255),
type_name: z.string(),
compression: z.number().int().min(0).max(255),
compression_name: z.string(),
name: z.string().nullable(),
name_bytes_hex: z.string().regex(/^[0-9a-f]{64}$/u),
sub_image_count: uint32,
});
export type UImageMetadata = z.infer<typeof uImageMetadataSchema>;
/** One stored payload, or one multi-image sub-image, as stored bytes. */
export const uImageEntryMetadataSchema = z.strictObject({
kind: z.literal("uimage-entry"),
role: z.enum(["payload", "sub-image"]),
index: uint32,
});
export type UImageEntryMetadata = z.infer<typeof uImageEntryMetadataSchema>;
export const artifactFormatMetadataSchema = z.discriminatedUnion("kind", [
godotPckMetadataSchema,
godotPckEntryMetadataSchema,
@@ -226,6 +263,8 @@ export const artifactFormatMetadataSchema = z.discriminatedUnion("kind", [
nodeSeaEntryMetadataSchema,
hexRecordsMetadataSchema,
hexRecordRegionMetadataSchema,
uImageMetadataSchema,
uImageEntryMetadataSchema,
]);
export type ArtifactFormatMetadata = z.infer<
typeof artifactFormatMetadataSchema
+2
View File
@@ -62,6 +62,7 @@ const artifactFormatSchema = z.enum([
"asar",
"godot-pck",
"unityfs",
"uimage",
"javascript-bundle",
"source-map",
"mach-o-universal",
@@ -97,6 +98,7 @@ const artifactContentFormatSchema = artifactFormatSchema.extract([
"zip",
"godot-pck",
"unityfs",
"uimage",
"intel-hex",
"srec",
"mach-o-universal",
+1
View File
@@ -259,6 +259,7 @@ export type BinaryTarget =
| "asar"
| "godot-pck"
| "unityfs"
| "uimage"
| "dmg"
| "pkg"
| "intel-hex"
+1
View File
@@ -26,6 +26,7 @@ export const evidenceSubjectFormatSchema = z.enum([
"asar",
"godot-pck",
"unityfs",
"uimage",
"dmg",
"pkg",
"plist",
+1
View File
@@ -72,6 +72,7 @@ export const liveBinaryTargetSummaryVariants = [
"asar",
"godot-pck",
"unityfs",
"uimage",
"dmg",
"pkg",
"intel-hex",
+269
View File
@@ -0,0 +1,269 @@
import { crc32 } from "node:zlib";
import { err, ok, type Result } from "./result.js";
/**
* U-Boot legacy uImage header (U-Boot `include/image.h`, all fields
* big-endian). Only the 64-byte header is interpreted here; payload bytes
* are never decompressed, executed, or otherwise transformed.
*/
export const UIMAGE_MAGIC = 0x27051956;
export const UIMAGE_HEADER_SIZE = 64;
export const UIMAGE_NAME_LENGTH = 32;
/** Cap on multi-image sub-images: one inventory entry per sub-image. */
export const UIMAGE_MAX_SUB_IMAGES = 4096;
/** Bytes of payload prefix that can hold a maximal multi-image size table. */
export const UIMAGE_MULTI_TABLE_WINDOW = (UIMAGE_MAX_SUB_IMAGES + 1) * 4;
export interface UImageSubImage {
readonly index: number;
/** Byte offset of the sub-image payload relative to the carrier start. */
readonly byteOffset: number;
readonly declaredSize: number;
}
export interface UImageHeaderFields {
readonly magic: number;
readonly timestamp: number;
readonly timestampIso: string;
readonly payloadSize: number;
readonly loadAddress: number;
readonly entryPoint: number;
readonly headerCrc32: number;
readonly dataCrc32: number;
readonly os: number;
readonly osName: string;
readonly arch: number;
readonly archName: string;
readonly type: number;
readonly typeName: string;
readonly compression: number;
readonly compressionName: string;
readonly name: string | null;
readonly nameBytesHex: string;
}
// IDs are append-only in U-Boot include/image.h, pinned ece349ade2973e220f524ce59e59711cc919263f.
const ARCH_NAMES: Readonly<Record<number, string>> = {
0: "invalid",
1: "alpha",
2: "arm",
3: "x86",
4: "ia64",
5: "mips",
6: "mips64",
7: "ppc",
8: "s390",
9: "sh",
10: "sparc",
11: "sparc64",
12: "m68k",
13: "nios",
14: "microblaze",
15: "nios2",
16: "blackfin",
17: "avr32",
18: "st200",
19: "sandbox",
20: "nds32",
21: "openrisc",
22: "arm64",
23: "arc",
24: "x86_64",
25: "xtensa",
26: "riscv",
};
const OS_NAMES: Readonly<Record<number, string>> = {
0: "invalid",
1: "openbsd",
2: "netbsd",
3: "freebsd",
4: "4_4bsd",
5: "linux",
6: "svr4",
7: "esix",
8: "solaris",
9: "irix",
10: "sco",
11: "dell",
12: "ncr",
13: "lynxos",
14: "vxworks",
15: "psos",
16: "qnx",
17: "u-boot",
18: "rtems",
19: "artos",
20: "unity",
21: "integrity",
22: "ose",
23: "plan9",
24: "openrtos",
25: "arm-trusted-firmware",
26: "tee",
27: "opensbi",
28: "efi",
29: "elf",
};
const TYPE_NAMES: Readonly<Record<number, string>> = {
0: "invalid",
1: "standalone",
2: "kernel",
3: "ramdisk",
4: "multi",
5: "firmware",
6: "script",
7: "filesystem",
8: "flat_dt",
};
const COMPRESSION_NAMES: Readonly<Record<number, string>> = {
0: "none",
1: "gzip",
2: "bzip2",
3: "lzma",
4: "lzo",
5: "lz4",
6: "zstd",
};
const named = (
table: Readonly<Record<number, string>>,
value: number,
): string => table[value] ?? "unknown";
export type UImageParseError =
| { readonly _tag: "UImageFormatFailure"; readonly message: string }
| { readonly _tag: "UImageCapacityFailure"; readonly message: string };
const malformed = (message: string): UImageParseError => ({
_tag: "UImageFormatFailure",
message,
});
const UTF8 = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
/** True when the first four bytes are the U-Boot legacy magic. */
export const hasUImageMagic = (prefix: Buffer): boolean =>
prefix.length >= 4 && prefix.readUInt32BE(0) === UIMAGE_MAGIC;
/**
* Parse the 64-byte header and verify its CRC32 (computed over the header
* with the ih_hcrc field zeroed, per `image_check_hcrc`). Fail-closed with
* a named reason; the payload is not touched here.
*/
export const parseUImageHeaderFields = (
header: Buffer,
): Result<UImageHeaderFields, UImageParseError> => {
if (header.length < UIMAGE_HEADER_SIZE)
return err(
malformed(
`uImage header is truncated: ${header.length} bytes, expected ${UIMAGE_HEADER_SIZE}`,
),
);
if (header.readUInt32BE(0) !== UIMAGE_MAGIC)
return err(
malformed(
`uImage magic mismatch: expected 0x${UIMAGE_MAGIC.toString(16)}, found 0x${header.readUInt32BE(0).toString(16)}`,
),
);
const declaredHeaderCrc = header.readUInt32BE(4);
const zeroed = Buffer.from(header.subarray(0, UIMAGE_HEADER_SIZE));
zeroed.writeUInt32BE(0, 4);
const computedHeaderCrc = crc32(zeroed);
if (computedHeaderCrc !== declaredHeaderCrc)
return err(
malformed(
`uImage header CRC32 mismatch: declared 0x${declaredHeaderCrc.toString(16)}, computed 0x${computedHeaderCrc.toString(16)}`,
),
);
const timestamp = header.readUInt32BE(8);
const nameBytes = header.subarray(32, 32 + UIMAGE_NAME_LENGTH);
const nul = nameBytes.indexOf(0);
let name: string | null;
try {
name = UTF8.decode(
nameBytes.subarray(0, nul === -1 ? UIMAGE_NAME_LENGTH : nul),
);
} catch {
name = null;
}
const os = header[28] ?? 0;
const arch = header[29] ?? 0;
const type = header[30] ?? 0;
const compression = header[31] ?? 0;
return ok({
magic: UIMAGE_MAGIC,
timestamp,
timestampIso: new Date(timestamp * 1000).toISOString(),
payloadSize: header.readUInt32BE(12),
loadAddress: header.readUInt32BE(16),
entryPoint: header.readUInt32BE(20),
headerCrc32: declaredHeaderCrc,
dataCrc32: header.readUInt32BE(24),
os,
osName: named(OS_NAMES, os),
arch,
archName: named(ARCH_NAMES, arch),
type,
typeName: named(TYPE_NAMES, type),
compression,
compressionName: named(COMPRESSION_NAMES, compression),
name,
nameBytesHex: nameBytes.toString("hex"),
});
};
/**
* Multi-file payload layout (U-Boot `image_multi_getimg`): the payload
* begins with a table of big-endian sub-image sizes terminated by a zero
* entry; the sub-image bytes follow the table in order, with nonfinal components
* padded to a four-byte boundary. `prefix` must hold
* the table's opening bytes (up to UIMAGE_MULTI_TABLE_WINDOW); the table
* must terminate inside the payload and the sub-images must fit within
* `payloadSize`. Trailing slack after the last sub-image is tolerated
* because it stays inside the stored payload entry.
*/
export const parseMultiImageTable = (
prefix: Buffer,
payloadSize: number,
): Result<readonly UImageSubImage[], UImageParseError> => {
if (!Number.isSafeInteger(payloadSize) || payloadSize < 0)
return err(
malformed("uImage payload size is not an exact nonnegative extent"),
);
const sizes: number[] = [];
let cursor = 0;
for (;;) {
if (cursor + 4 > prefix.length || cursor + 4 > payloadSize)
return err(
malformed(
"uImage multi-image size table has no zero terminator inside the payload",
),
);
const size = prefix.readUInt32BE(cursor);
cursor += 4;
if (size === 0) break;
sizes.push(size);
if (sizes.length > UIMAGE_MAX_SUB_IMAGES)
return err({
_tag: "UImageCapacityFailure",
message: `uImage multi-image table declares more than ${UIMAGE_MAX_SUB_IMAGES} sub-images`,
});
}
const subImages: UImageSubImage[] = [];
let offset = UIMAGE_HEADER_SIZE + cursor;
for (const [index, size] of sizes.entries()) {
if (offset + size > UIMAGE_HEADER_SIZE + payloadSize)
return err(
malformed(
`uImage multi-image sub-image ${index} of ${size} bytes overruns the ${payloadSize}-byte payload`,
),
);
subImages.push({ index, byteOffset: offset, declaredSize: size });
offset += index + 1 < sizes.length ? Math.ceil(size / 4) * 4 : size;
}
return ok(subImages);
};
+84
View File
@@ -0,0 +1,84 @@
import { readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect } from "vitest";
import { cliTest } from "../support/cli/cliFixture.js";
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
import { buildMultiPayload, buildUImage } from "../fixtures/uimage.js";
import { parseEvidence } from "../../src/domain/evidence.js";
import { artifactInspectionResultSchema } from "../../src/domain/artifactInspection.js";
import {
artifactExtractionResultSchema,
artifactInventoryResultSchema,
} from "../../src/domain/artifactGraph.js";
cliTest(
"inspects and extracts a uImage through the compiled CLI",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-uimage-cli-");
const partA = Buffer.from("first-part");
const partB = Buffer.from("second-part");
const source = join(root, "firmware.bin");
await writeFile(
source,
buildUImage({
arch: 22,
os: 14,
type: 4,
compression: 0,
loadAddress: 0x8000,
entryPoint: 0x8000,
name: "multi-test",
payload: buildMultiPayload([partA, partB]),
}),
);
const inspected = await cli.run({
arguments: ["inspect-artifact", source, "--json"],
});
expect(inspected.exitCode, inspected.stdout + inspected.stderr).toBe(0);
const inspection = artifactInspectionResultSchema.parse(
parseEvidence(inspected.json).normalized_result,
);
const graph = artifactInventoryResultSchema.parse(
inspection.substeps[0]?.evidence.normalized_result,
);
expect(graph.manifest.root_format).toBe("uimage");
expect(graph.coverage.status).toBe("complete");
expect(
graph.occurrences.find((row) => row.logical_path === ".")
?.format_metadata,
).toMatchObject({
kind: "uimage",
arch: 22,
arch_name: "arm64",
os: 14,
os_name: "vxworks",
type_name: "multi",
compression_name: "none",
name: "multi-test",
crc_verified: true,
sub_image_count: 2,
});
const extracted = await cli.run({
arguments: ["extract-artifact", source, "--json"],
environment: { TMPDIR: root, TMP: root, TEMP: root },
});
expect(extracted.exitCode, extracted.stdout + extracted.stderr).toBe(0);
const result = artifactExtractionResultSchema.parse(
parseEvidence(extracted.json).normalized_result,
);
expect(result.containment_verified).toBe(true);
expect(
(await readFile(join(result.output_root, "payload.bin"))).equals(
buildMultiPayload([partA, partB]),
),
).toBe(true);
expect(
(await readFile(join(result.output_root, "image-0.bin"))).equals(partA),
).toBe(true);
expect(
(await readFile(join(result.output_root, "image-1.bin"))).equals(partB),
).toBe(true);
},
);
+95
View File
@@ -0,0 +1,95 @@
import { readFile, rm, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { createHash } from "node:crypto";
import { expect, it } from "vitest";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { artifactInspectionResultSchema } from "../../../src/domain/artifactInspection.js";
import { artifactExtractionResultSchema } from "../../../src/domain/artifactGraph.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js";
import { buildUImage, buildMultiPayload } from "../../fixtures/uimage.js";
it.each(["full", "compact"] as const)(
"retains uImage physical facts and unknown text through compiled MCP (%s)",
async (profile) => {
const root = await createTestTempDirectory("rea-uimage-mcp-");
const path = join(root, "firmware");
const parts = [Buffer.from("abc"), Buffer.from("second part")];
const bytes = buildUImage({
arch: 22,
os: 14,
type: 4,
name: "测".repeat(11),
payload: buildMultiPayload(parts),
});
await writeFile(path, bytes);
const { call } = await connectLocalToolsMcp({
REA_MCP_INPUT_SCHEMA_PROFILE: profile,
});
let outputRoot: string | undefined;
try {
const opened = await call("open_binary", { path });
expect(opened.isError, JSON.stringify(opened)).not.toBe(true);
expect(opened.structuredContent).toMatchObject({
result: { kind: "archive", format: "uimage", architecture: null },
});
const reply = await call("inspect_artifact", {});
expect(reply.isError, JSON.stringify(reply)).not.toBe(true);
const inspection = artifactInspectionResultSchema.parse(
parseEvidence(reply.structuredContent).normalized_result,
);
const inventory = inspection.substeps[0]!.evidence.normalized_result;
expect(inventory.coverage.status).toBe("complete");
expect(inventory.occurrences).toContainEqual(
expect.objectContaining({
logical_path: ".",
format_metadata: expect.objectContaining({
kind: "uimage",
arch: 22,
arch_name: "arm64",
os: 14,
os_name: "vxworks",
name: null,
name_bytes_hex: bytes.subarray(32, 64).toString("hex"),
crc_verified: true,
}),
}),
);
expect(inventory.occurrences).toContainEqual(
expect.objectContaining({
logical_path: "image-1.bin",
source_location: { offset: 80, length: parts[1]!.length },
}),
);
for (const part of parts)
expect(inventory.nodes).toContainEqual(
expect.objectContaining({
sha256: createHash("sha256").update(part).digest("hex"),
}),
);
const extracted = await call("extract_artifact", {});
expect(extracted.isError, JSON.stringify(extracted)).not.toBe(true);
const result = artifactExtractionResultSchema.parse(
parseEvidence(extracted.structuredContent).normalized_result,
);
outputRoot = result.output_root;
expect(result.containment_verified).toBe(true);
expect(result.cleanup).toMatchObject({
verified: true,
residual_paths: [],
});
for (const [index, part] of parts.entries())
expect(
(
await readFile(join(outputRoot, `image-${String(index)}.bin`))
).equals(part),
).toBe(true);
expect((await readFile(path)).equals(bytes)).toBe(true);
await call("close_binary", {});
} finally {
if (outputRoot !== undefined)
await rm(outputRoot, { recursive: true, force: true });
}
},
60_000,
);
+71
View File
@@ -0,0 +1,71 @@
import { crc32 } from "node:zlib";
import {
UIMAGE_HEADER_SIZE,
UIMAGE_MAGIC,
UIMAGE_NAME_LENGTH,
} from "../../src/domain/uimageFormat.js";
export interface UImageFixtureOptions {
readonly arch?: number;
readonly os?: number;
readonly type?: number;
readonly compression?: number;
readonly loadAddress?: number;
readonly entryPoint?: number;
readonly timestamp?: number;
readonly name?: string;
/** Raw stored payload bytes; for multi images pass the size table + parts. */
readonly payload?: Buffer;
/** Corrupt the header CRC after computing it (keeps a valid magic). */
readonly corruptHeaderCrc?: boolean;
/** Corrupt the data CRC after computing it. */
readonly corruptDataCrc?: boolean;
}
/** Build a synthetic legacy uImage with correctly computed CRCs. */
export const buildUImage = (options: UImageFixtureOptions = {}): Buffer => {
const payload = options.payload ?? Buffer.from("uimage-payload");
const header = Buffer.alloc(UIMAGE_HEADER_SIZE);
header.writeUInt32BE(UIMAGE_MAGIC, 0);
header.writeUInt32BE(options.timestamp ?? 1700000000, 8);
header.writeUInt32BE(payload.length, 12);
header.writeUInt32BE(options.loadAddress ?? 0x8000, 16);
header.writeUInt32BE(options.entryPoint ?? 0x8000, 20);
// Data CRC first: corrupting it must keep the header CRC valid, so the
// header CRC is computed over the final header bytes below.
const dataCrc =
(crc32(payload) >>> 0) ^ (options.corruptDataCrc === true ? 0xff : 0);
header.writeUInt32BE(dataCrc >>> 0, 24);
header[28] = options.os ?? 5;
header[29] = options.arch ?? 2;
header[30] = options.type ?? 2;
header[31] = options.compression ?? 0;
const name = Buffer.from(options.name ?? "test-kernel", "utf8");
name.copy(header, 32, 0, Math.min(name.length, UIMAGE_NAME_LENGTH));
const headerCrc = crc32(header) >>> 0;
header.writeUInt32BE(
options.corruptHeaderCrc === true ? (headerCrc ^ 0xff) >>> 0 : headerCrc,
4,
);
return Buffer.concat([header, payload]);
};
/** Size table + parts layout for a multi-file image payload. */
export const buildMultiPayload = (parts: readonly Buffer[]): Buffer => {
const table = Buffer.alloc((parts.length + 1) * 4);
parts.forEach((part, index) => {
table.writeUInt32BE(part.length, index * 4);
});
const stored: Buffer[] = [table];
parts.forEach((part, index) => {
if (part.length === 0)
throw new Error(
"A zero size terminates the uImage table; it cannot describe a component",
);
stored.push(part);
if (index + 1 < parts.length)
stored.push(Buffer.alloc((4 - (part.length % 4)) % 4));
});
return Buffer.concat(stored);
};