feat(artifacts): read WeChat mini-program wxapkg containers (#8148)

* feat(artifacts): read WeChat mini-program wxapkg containers

Add a read-only reader for plaintext wxapkg packages: the 0xBE/0xED
18-byte big-endian header, the bounded name/offset/size member table,
and verbatim member bytes at absolute carrier offsets.

- Members retain declared names, absolute offsets and sizes, so
  source_location reports real carrier positions.
- V1MMWX PC-client packages are detected as encrypted coverage with
  null member facts; extraction refuses with a typed error and no
  decryption is attempted.
- Bounded table and directory budgets are admitted before allocation;
  malformed marks, truncated tables, invalid UTF-8 names, member
  extents inside the header or beyond the file, and counts that cannot
  fit their table are typed format or limit failures.
- Member reads stream in bounded 64 KiB chunks with carrier freshness
  checks; the reader joins outstanding streams before close and bounds
  active streams and repeated member I/O.

Refs #8121.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(artifacts): bound wxapkg members to the declared data region

Normalize the two indexLength conventions before the table bounds check
and the table read. Member ranges must then lie in that declared data
region, which has to fit in the file.

Record wxapkg format metadata as not compared: header facts repeat the
package bytes, and entry source paths keep a producer root prefix.

* fix(artifacts): choose the wxapkg index after member ranges fit

Four trailing bytes made the rows-only layout end on the file and dropped
the count-inclusive layout. Keep every admitted layout until the parsed
rows and member ranges select one.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <[email protected]>
This commit is contained in:
authored and GitHub committed 2026-11-05 00:52:20 +08:00
1 parent 63b66fa498
commit 5990ff0324
25 files changed
+1889 -14

No files matched your search

@@ -708,6 +708,13 @@ cache one decoded block and support retained filtered/paged views. No
decryption or SerializedFile object decoding is provided. See
[UnityFS](https://github.com/morluto/rea/blob/main/docs/unityfs.md).
WeChat mini-program packages (`.wxapkg` files with `0xBE`/`0xED` marks) list
members with declared names and absolute carrier offsets; member bytes are
stored verbatim. `V1MMWX` PC-client packages report `encrypted` directory
status with unknown coverage and are refused on extraction; no decryption is
attempted. See
[wxapkg](https://github.com/morluto/rea/blob/main/docs/wxapkg.md).
U-Boot legacy uImages use the same inspection/extraction workflows. Prefer a
retained summary and selected inventory views for header facts or component
IDs. Header addresses/CPU/OS are declarations; payloads remain as stored.
+11
View File
@@ -281,6 +281,17 @@ Format versions 6 through 8 retain header, block and directory facts with
none/LZ4/LZ4HC block decompression. See [UnityFS](unityfs.md) for encryption
and LZMA coverage boundaries.
Inspect a WeChat mini-program package without WeChat or an unpacker:
```sh
rea inspect-artifact app.wxapkg --detail summary --evidence_output evidence.json --json
rea extract-artifact app.wxapkg --json
```
Plaintext packages retain header, table and member facts with verbatim member
bytes. `V1MMWX` packages are detected as encrypted coverage and refused on
extraction. See [wxapkg](wxapkg.md) for layout and coverage boundaries.
Inspect an ESP32-family app or bootloader image without esptool:
```sh
+13 -13
View File
@@ -47,19 +47,19 @@ For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
## Choose a guide
| Task | Guides |
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [PE imports](pe-imports.md), [PE TLS/load config](pe-load-config.md), [ELF notes](elf-notes.md), [Linux kernel modules](kernel-modules.md), [Apple IPS](apple-ips.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [GameCube DOL](ghidra-dol.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
| Inspect assemblies, packages, or firmware | [Godot PCK](godot-pck.md), [UnityFS](unityfs.md), [U-Boot uImage](uimage.md), [.NET](managed-code-analysis.md), [Python packages](python-packaged-applications.md), [Android](android-analysis.md), [APK signatures](apk-signature.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [ESP images](esp-image.md), [ESP flash images](esp-flash-image.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
| Serve recorded behavior as an agent environment | [Agent environments](agent-environments.md) |
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
| Capture and compare a command | [Process capture](process-capture.md) |
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
| Task | Guides |
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [PE imports](pe-imports.md), [PE TLS/load config](pe-load-config.md), [ELF notes](elf-notes.md), [Linux kernel modules](kernel-modules.md), [Apple IPS](apple-ips.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [GameCube DOL](ghidra-dol.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
| Inspect assemblies, packages, or firmware | [Godot PCK](godot-pck.md), [UnityFS](unityfs.md), [wxapkg](wxapkg.md), [U-Boot uImage](uimage.md), [.NET](managed-code-analysis.md), [Python packages](python-packaged-applications.md), [Android](android-analysis.md), [APK signatures](apk-signature.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [ESP images](esp-image.md), [ESP flash images](esp-flash-image.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
| Serve recorded behavior as an agent environment | [Agent environments](agent-environments.md) |
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
| Capture and compare a command | [Process capture](process-capture.md) |
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
For development, use [testing](testing.md), [client verification](client-verification.md),
[tool design](tool-design.md), [admission limits and recovery](design/limits-and-recovery.md),
+84
View File
@@ -0,0 +1,84 @@
# WeChat mini-program wxapkg archives
`inspect-artifact` and `extract-artifact` read plaintext WeChat mini-program
package (`.wxapkg`) containers without running WeChat or an unpacker. The
plaintext form starts with a `0xBE` mark and an 18-byte big-endian header:
an unknown `info` field producers leave zero, the member-table byte length,
the member-data byte length, a `0xED` mark and the member count. Producers
either count only the name/offset/size rows in that table length or include
the four-byte member count as well; both declarations are accepted. The table
region then carries one name/offset/size row per member; member bytes are
stored verbatim at absolute file offsets.
Inspection retains the header facts: `info`, index and data lengths, member
count and directory status. Members retain declared source paths, absolute
carrier offsets and declared sizes, so `source_location` reports real file
positions. Member reads only examine their declared range in bounded 64 KiB
chunks. Carrier freshness is checked when a member is opened and after its
stream completes. Reader close cancels and joins outstanding streams before
releasing its descriptor.
Encrypted PC-client packages start with the `V1MMWX` magic instead of the
plain header. They keep their format identity but report `encrypted`
directory status with unknown coverage: member identities and bytes remain
unknown and no decryption is attempted, so extraction refuses with a typed
error rather than producing ciphertext. Declared tables or data regions
that exceed the file, member extents outside the declared data region,
truncated rows, invalid UTF-8 names, and counts that cannot fit their table
are malformed-input errors.
## Resource bounds
These are resource policies, not format maxima. The encoded member table is
limited to 16 MiB and its buffer, parsed rows, retained entries and escaped
path projections share a conservative 64 MiB admission budget, reserved
before allocation; a declared member name is limited to 64 KiB. A reader
admits four active member streams. The shared archive plaintext-work ceiling
is the larger of 256 MiB and 500 times carrier size: every declared member
extent is charged, including aliases that overlap one stored range. A
separate member-I/O budget allows four times the larger of 1,024 and the sum
of the member count and declared 64 KiB plaintext chunks, bounding repeated
tiny-member reads independently of output bytes.
## CLI
```sh
rea inspect-artifact app.wxapkg --detail summary \
--evidence_output evidence/package.json --json
rea extract-artifact app.wxapkg --json
```
Detection is byte-based on the `0xBE`/`0xED` marks or the `V1MMWX` magic, so
`.wxapkg`, `__APP__.wxapkg` and extensionless cache files all classify as
`wxapkg` containers. Compact delivery keeps complete Evidence in the
requested bundle.
## Verification scope
Synthetic fixtures cover plaintext headers, multiple members, empty members,
chunked reads, unicode and dotted names, malformed marks, truncated tables,
invalid UTF-8, table and directory-budget admission, member extents inside
the header, the index, or beyond the declared data region, a data region
declared past the file, a one-byte member whose table length includes the
member count, an overstated first row consuming its successor, V1MMWX
packages, cancellation inside member reads, paused iteration, close during
I/O, cleanup retry, stream capacity and carrier freshness. Compiled
full/compact MCP tests cover inventory and extraction.
Real producer verification used plaintext `0xBE` packages from a local macOS
WeChat mini-program cache (`__APP__.wxapkg` main packages and
`__PLUGINCODE__.wxapkg` plugin packages across three distinct app
directories): every member extracted byte-identical to its declared carrier
range, including `__plugin__/`-namespaced members and the producer's
root-prefixed (`/name`) member convention. A cached `V1MMWX` PC package
reports the encrypted directory state with unknown member facts and refuses
extraction.
`wxml`/`wxss` source recovery, compiled bundle semantics, subpackage
relationships and runtime behavior are outside this container reader's
verification scope.
Artifact comparison pairs members by their package-relative paths and
compares their bytes. It does not compare `wxapkg` or `wxapkg-entry`
metadata: header fields repeat the package bytes, `index` is layout, and
`source_path` keeps a producer root prefix that the occurrence path strips.
+6
View File
@@ -85,6 +85,10 @@ import { hasDiskImageSignature } from "../domain/apple/diskImageFormat.js";
import { hasGodotPckHeader } from "../domain/godotPckFormat.js";
import { hasUImageMagic } from "../domain/uimageFormat.js";
import { hasUnityFsHeader } from "../domain/unityfsFormat.js";
import {
hasWxapkgEncryptedMagic,
hasWxapkgHeader,
} from "../domain/wxapkgFormat.js";
import { detectHexRecordEncoding } from "../domain/firmware/hexRecords.js";
import { hasUf2Block } from "../domain/firmware/uf2.js";
import {
@@ -777,6 +781,7 @@ const detectArtifactFormat = async (
if (hasZipSignature(magic)) return zipRootPackageFormat(lower);
if (hasGodotPckHeader(magic)) return "godot-pck";
if (hasUnityFsHeader(magic)) return "unityfs";
if (hasWxapkgEncryptedMagic(magic) || hasWxapkgHeader(magic)) return "wxapkg";
if (hasUImageMagic(magic)) return "uimage";
const hexRecords = detectHexRecordEncoding(magic);
if (hexRecords !== undefined) return hexRecords;
@@ -883,6 +888,7 @@ const isArchiveFormat = (
"ines",
"godot-pck",
"unityfs",
"wxapkg",
"uimage",
"dmg",
"pkg",
@@ -120,6 +120,7 @@ const TARGETS: TargetFixtures = {
ines: { ...archive, format: "ines" },
"godot-pck": { ...archive, format: "godot-pck" },
unityfs: { ...archive, format: "unityfs" },
wxapkg: { ...archive, format: "wxapkg" },
uimage: { ...archive, format: "uimage" },
uf2: { ...archive, format: "uf2" },
dfuse: { ...archive, format: "dfuse" },
+1
View File
@@ -199,6 +199,7 @@ export interface ArtifactReader {
| "ines"
| "godot-pck"
| "unityfs"
| "wxapkg"
| "uimage"
| "pyinstaller"
| "dotnet-bundle"
@@ -0,0 +1,310 @@
import { once } from "node:events";
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { buffer, text } from "node:stream/consumers";
import { setImmediate } from "node:timers/promises";
import { describe, expect, it, vi } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { createDeferred } from "../../tests/fixtures/binarySession.js";
import { awaitAcknowledgement } from "../../tests/fixtures/lifecycleAcknowledgement.js";
import { buildWxapkg } from "../../tests/fixtures/wxapkg.js";
import { openRegularFile } from "../filesystem/RegularFile.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { WxapkgArtifactReader } from "./WxapkgArtifactReader.js";
const fixture = async (bytes: Buffer): Promise<string> => {
const path = join(
await createTestTempDirectory("rea-wxapkg-boundary-"),
"package.wxapkg",
);
await writeFile(path, bytes);
return path;
};
const entries = async (reader: WxapkgArtifactReader) => {
const result = [];
for await (const entry of reader.entries()) result.push(entry);
return result;
};
describe("wxapkg allocation and malformed-input boundaries", () => {
it("admits the encoded table budget before reading it", async () => {
const bytes = buildWxapkg([{ path: "a", data: "x" }]);
bytes.writeUInt32BE(17 * 1024 * 1024, 5);
const file = Buffer.alloc(18 + 17 * 1024 * 1024);
bytes.copy(file);
const path = await fixture(file);
const admitted = await openRegularFile(path, { symlinks: "reject" });
const reader = new WxapkgArtifactReader(path, admitted);
const reads = vi.spyOn(admitted.handle, "read");
try {
await expect(entries(reader)).rejects.toMatchObject({
reason: "limit",
boundary: "capacity",
});
expect(reads).toHaveBeenCalledTimes(1); // bounded header only
} finally {
await reader.close();
reads.mockRestore();
await admitted.handle.close();
}
});
it("reserves row storage before constructing a large member array", async () => {
// 140k minimal rows fit the encoded table budget but exceed the retained
// directory representation budget mid-parse.
const fileCount = 140_000;
const tableBytes = fileCount * 13;
const file = Buffer.alloc(18 + tableBytes);
file[0] = 0xbe;
file.writeUInt32BE(0, 1);
file.writeUInt32BE(tableBytes, 5);
file.writeUInt32BE(0, 9);
file[13] = 0xed;
file.writeUInt32BE(fileCount, 14);
for (let index = 0; index < fileCount; index++) {
const row = 18 + index * 13;
file.writeUInt32BE(1, row);
file[row + 4] = 0x61;
// Zero-size members at the data start stay admissible so parsing
// reaches the representation budget.
file.writeUInt32BE(18 + tableBytes, row + 5);
file.writeUInt32BE(0, row + 9);
}
const reader = new WxapkgArtifactReader(await fixture(file));
try {
await expect(entries(reader)).rejects.toMatchObject({
reason: "limit",
boundary: "capacity",
});
} finally {
await reader.close();
}
});
it("charges overlapping readable members, not only their unique range", async () => {
const size = 1024 * 1024;
const members = Array.from({ length: 600 }, (_, index) => ({
path: `alias-${String(index)}`,
data: index === 0 ? Buffer.alloc(size, 0x61) : Buffer.alloc(0),
size,
}));
const reader = new WxapkgArtifactReader(
await fixture(buildWxapkg(members, { aliasOffsets: true })),
);
try {
await expect(entries(reader)).rejects.toMatchObject({ reason: "limit" });
} finally {
await reader.close();
}
});
it("types a short later row after an overstated first name consumes its bytes", async () => {
const bytes = buildWxapkg([
{ path: "a".repeat(40), data: "x" },
{ path: "b", data: "y" },
]);
// The first row's declared name consumes most of the second row. The
// minimum total table-size check alone cannot establish each later row.
bytes.writeUInt32BE(50, 18);
// Keep the first member admissible at the position the parser reads after
// the inflated name, so the failure must come from the short later row.
const dataStart = 18 + bytes.readUInt32BE(5);
bytes.writeUInt32BE(dataStart, 18 + 4 + 50);
bytes.writeUInt32BE(1, 18 + 4 + 50 + 4);
const reader = new WxapkgArtifactReader(await fixture(bytes));
try {
await expect(entries(reader)).rejects.toMatchObject({
reason: "format",
message: expect.stringContaining("truncated"),
});
} finally {
await reader.close();
}
});
});
describe("wxapkg stream ownership", () => {
it("retains descriptor ownership after cleanup fails and permits retry", async () => {
const reader = new WxapkgArtifactReader(
await fixture(buildWxapkg([{ path: "x", data: "bytes" }])),
);
const [entry] = await entries(reader);
const stream = await reader.open(entry!);
const failure = new Error("descriptor close denied");
const close = vi
.spyOn(OwnedFileHandle.prototype, "close")
.mockRejectedValueOnce(failure);
try {
await expect(reader.close()).rejects.toBe(failure);
expect(stream.closed).toBe(true);
await expect(reader.open(entry!)).rejects.toMatchObject({
reason: "unavailable",
});
await reader.close();
expect(close).toHaveBeenCalledTimes(2);
} finally {
close.mockRestore();
await reader.close();
}
});
it("joins an outstanding positional read before close completes", async () => {
const path = await fixture(buildWxapkg([{ path: "x", data: "bytes" }]));
const admitted = await openRegularFile(path, { symlinks: "reject" });
const reader = new WxapkgArtifactReader(path, admitted);
const reached = createDeferred<void>(),
release = createDeferred<void>();
const nativeRead = admitted.handle.read.bind(admitted.handle);
const [entry] = await entries(reader);
const reads = vi.spyOn(admitted.handle, "read").mockImplementation((async (
...args: unknown[]
) => {
reached.resolve();
await release.promise;
return Reflect.apply(nativeRead, admitted.handle, args);
}) as typeof admitted.handle.read);
try {
const stream = await reader.open(entry!);
stream.read(0);
await awaitAcknowledgement(
reached.promise,
once(stream, "close"),
"positional member read",
);
let closed = false;
const closing = reader.close().then(() => {
closed = true;
});
await setImmediate();
expect(closed).toBe(false);
release.resolve();
await closing;
expect(stream.closed).toBe(true);
} finally {
release.resolve();
await reader.close();
reads.mockRestore();
await admitted.handle.close();
}
});
it("preserves cancellation inside a member read", async () => {
const data = Buffer.alloc(256 * 1024, 0x61);
const path = await fixture(buildWxapkg([{ path: "x", data }]));
const admitted = await openRegularFile(path, { symlinks: "reject" });
const reader = new WxapkgArtifactReader(path, admitted);
const controller = new AbortController();
const reason = new Error("cancel during member read");
const nativeRead = admitted.handle.read.bind(admitted.handle);
const [entry] = await entries(reader);
const reads = vi.spyOn(admitted.handle, "read").mockImplementation((async (
...args: unknown[]
) => {
const result = await Reflect.apply(nativeRead, admitted.handle, args);
globalThis.setImmediate(() => {
controller.abort(reason);
});
return result;
}) as typeof admitted.handle.read);
try {
await expect(
buffer(await reader.open(entry!, controller.signal)),
).rejects.toMatchObject({ reason: "cancelled", cause: reason });
} finally {
await reader.close();
reads.mockRestore();
await admitted.handle.close();
}
});
});
describe("wxapkg stream capacity and freshness", () => {
it("bounds active streams and releases capacity after abandonment", async () => {
const reader = new WxapkgArtifactReader(
await fixture(buildWxapkg([{ path: "x", data: "bytes" }])),
);
try {
const [entry] = await entries(reader);
const streams = await Promise.all(
Array.from({ length: 4 }, () => reader.open(entry!)),
);
await expect(reader.open(entry!)).rejects.toMatchObject({
reason: "limit",
boundary: "capacity",
});
const closed = new Promise<void>((resolve) => {
streams[0]!.once("close", resolve);
});
streams[0]!.destroy();
await closed;
expect(await text(await reader.open(entry!))).toBe("bytes");
} finally {
await reader.close();
}
});
it("bounds repeated member I/O operations, including empty members", async () => {
const reader = new WxapkgArtifactReader(
await fixture(
buildWxapkg([
{ path: "x", data: "b" },
{ path: "empty.wxss", data: "" },
]),
),
);
try {
const [entry, emptyEntry] = await entries(reader);
await expect(
(async () => {
for (let index = 0; index < 5000; index++)
await buffer(await reader.open(entry!));
})(),
).rejects.toMatchObject({ reason: "limit", boundary: "capacity" });
await expect(
(async () => {
for (let index = 0; index < 5000; index++)
await buffer(await reader.open(emptyEntry!));
})(),
).rejects.toMatchObject({ reason: "limit", boundary: "capacity" });
} finally {
await reader.close();
}
});
it("validates freshness before serving member bytes", async () => {
const bytes = buildWxapkg([{ path: "x", data: "before" }]);
const path = await fixture(bytes);
const reader = new WxapkgArtifactReader(path);
try {
const [entry] = await entries(reader);
expect(await text(await reader.open(entry!))).toBe("before");
await writeFile(path, Buffer.concat([bytes, Buffer.from([0])]));
await expect(reader.open(entry!)).rejects.toMatchObject({
reason: "integrity",
});
} finally {
await reader.close();
}
});
it("does not turn a paused inventory into complete coverage when closed", async () => {
const reader = new WxapkgArtifactReader(
await fixture(
buildWxapkg([
{ path: "x", data: "bytes" },
{ path: "y", data: "bytes" },
]),
),
);
const iterator = reader.entries()[Symbol.asyncIterator]();
try {
expect((await iterator.next()).done).toBe(false);
await reader.close();
await expect(iterator.next()).rejects.toMatchObject({
reason: "unavailable",
});
} finally {
await reader.close();
}
});
});
+468
View File
@@ -0,0 +1,468 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { text } from "node:stream/consumers";
import { describe, expect, it } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import {
buildWxapkg,
wxapkgEncryptedBytes,
type WxapkgMember,
} from "../../tests/fixtures/wxapkg.js";
import type {
ArtifactEntry,
ArtifactTraversalOptions,
} from "./ArtifactReader.js";
import { hasWxapkgHeader } from "../domain/wxapkgFormat.js";
import { WxapkgArtifactReader } from "./WxapkgArtifactReader.js";
const writePackage = async (bytes: Buffer): Promise<string> => {
const path = join(
await createTestTempDirectory("rea-wxapkg-"),
"package.wxapkg",
);
await writeFile(path, bytes);
return path;
};
const list = async (
reader: WxapkgArtifactReader,
options?: ArtifactTraversalOptions,
): Promise<ArtifactEntry[]> => {
const entries: ArtifactEntry[] = [];
for await (const entry of reader.entries(undefined, options))
entries.push(entry);
return entries;
};
const withReader = async <T>(
bytes: Buffer,
use: (reader: WxapkgArtifactReader) => Promise<T>,
): Promise<T> => {
const reader = new WxapkgArtifactReader(await writePackage(bytes));
try {
return await use(reader);
} finally {
await reader.close();
}
};
const members: readonly WxapkgMember[] = [
{ path: "app-config.json", data: "{}" },
{ path: "pages/index/index.js", data: "module.exports={};" },
{ path: "empty.wxss", data: "" },
];
describe("WxapkgArtifactReader", () => {
it("lists members and reads their declared bytes", async () => {
await withReader(buildWxapkg(members), async (reader) => {
const entries = await list(reader);
expect(entries.map(({ path }) => path)).toEqual([
"app-config.json",
"pages/index/index.js",
"empty.wxss",
]);
expect(await text(await reader.open(entries[0]!))).toBe("{}");
expect(await text(await reader.open(entries[1]!))).toBe(
"module.exports={};",
);
expect(await text(await reader.open(entries[2]!))).toBe("");
});
});
it("preserves member provenance on entries and metadata", async () => {
await withReader(buildWxapkg(members), async (reader) => {
const entries = await list(reader);
const entry = entries[1]!;
expect(entry).toMatchObject({
path: "pages/index/index.js",
sourcePath: "pages/index/index.js",
kind: "file",
declaredSize: 18,
compressedSize: null,
encrypted: false,
declaredIntegrity: null,
unpacked: false,
adapterKey: "wxapkg-entry:1",
formatMetadata: {
kind: "wxapkg-entry",
index: 1,
source_path: "pages/index/index.js",
},
});
// Member offsets are absolute carrier positions after the table.
expect(entry.byteOffset).toBeGreaterThanOrEqual(18);
expect(reader.formatMetadata()).toMatchObject({
kind: "wxapkg",
file_count: 3,
directory_status: "available",
});
});
});
it("reads a member larger than the chunk size", async () => {
const data = Buffer.alloc(200 * 1024, 0x61);
await withReader(
buildWxapkg([{ path: "big.bin", data }]),
async (reader) => {
const [entry] = await list(reader);
const chunks: Buffer[] = [];
for await (const chunk of await reader.open(entry!))
chunks.push(chunk as Buffer);
expect(Buffer.concat(chunks).equals(data)).toBe(true);
},
);
});
it("accepts consecutive dots and unicode names", async () => {
await withReader(
buildWxapkg([
{ path: "data..bin", data: "a" },
{ path: "页面/配置.json", data: "b" },
]),
async (reader) => {
const entries = await list(reader);
expect(entries[0]?.path).toBe("data..bin");
expect(entries[1]?.path).toBe("页面/配置.json");
expect(await text(await reader.open(entries[1]!))).toBe("b");
},
);
});
it("translates producer root prefixes while keeping declared names", async () => {
await withReader(
buildWxapkg([
{ path: "/app-config.json", data: "{}" },
{ path: "/pages/index/index.js", data: "x=1;" },
]),
async (reader) => {
const entries = await list(reader);
expect(entries.map((entry) => entry.path)).toEqual([
"app-config.json",
"pages/index/index.js",
]);
expect(entries[1]).toMatchObject({
sourcePath: "/pages/index/index.js",
formatMetadata: { source_path: "/pages/index/index.js" },
});
expect(await text(await reader.open(entries[0]!))).toBe("{}");
},
);
});
it("exposes no members for a V1MMWX-encrypted package", async () => {
await withReader(wxapkgEncryptedBytes(), async (reader) => {
const coverage: {
status?: string;
limitations?: readonly string[];
} = {};
const entries = await list(reader, {
onCoverage: (observation) => {
coverage.status = observation.status;
coverage.limitations = observation.limitations;
},
});
expect(entries).toEqual([]);
expect(coverage.status).toBe("unknown");
expect(coverage.limitations?.join(" ")).toContain("V1MMWX");
expect(reader.formatMetadata()).toMatchObject({
kind: "wxapkg",
directory_status: "encrypted",
file_count: null,
info: null,
});
});
});
});
describe("WxapkgArtifactReader malformed inputs", () => {
it.each([
["wrong first mark", { firstMark: 0x00 }],
["wrong last mark", { lastMark: 0x00 }],
["file count above the row count", { declaredFileCount: 4 }],
["index length below the rows", { declaredIndexLength: 10 }],
["declared name exceeds the row", { nameLength: 1024 }],
["zero-length declared name", { nameLength: 0 }],
])("rejects %s as a typed failure", async (_name, options) => {
await withReader(buildWxapkg(members, options), async (reader) => {
await expect(list(reader)).rejects.toMatchObject({
reason: expect.stringMatching(/^(format|limit)$/),
});
});
});
it("rejects a member offset into the header region", async () => {
await withReader(
buildWxapkg([{ path: "evil.bin", data: "x", offset: 4 }]),
async (reader) => {
await expect(list(reader)).rejects.toMatchObject({
reason: "format",
});
},
);
});
it("rejects a member extent beyond the file", async () => {
await withReader(
buildWxapkg([{ path: "evil.bin", data: "x", size: 1 << 20 }]),
async (reader) => {
await expect(list(reader)).rejects.toMatchObject({
reason: "format",
});
},
);
});
it("rejects a member offset into the index region", async () => {
// Offset 18 is inside the declared table, not the data region.
await withReader(
buildWxapkg([{ path: "evil.bin", data: "x", offset: 18, size: 1 }]),
async (reader) => {
await expect(list(reader)).rejects.toMatchObject({ reason: "format" });
},
);
});
});
describe("WxapkgArtifactReader data region", () => {
it("rejects a member range that covers unused index bytes", async () => {
const bytes = buildWxapkg([{ path: "a", data: "Z" }], {
tablePadding: 8,
});
// The row ends at byte 31; the next eight bytes are still the index.
bytes.writeUInt32BE(31, 23);
bytes.writeUInt32BE(1, 27);
await withReader(bytes, async (reader) => {
await expect(list(reader)).rejects.toMatchObject({ reason: "format" });
});
});
it("reads member bytes that follow unused index bytes", async () => {
await withReader(
buildWxapkg([{ path: "a", data: "Z" }], { tablePadding: 8 }),
async (reader) => {
const [entry] = await list(reader);
expect(entry).toMatchObject({ byteOffset: 39, declaredSize: 1 });
expect(await text(await reader.open(entry!))).toBe("Z");
},
);
});
it("rejects a member extent beyond the declared data region", async () => {
await withReader(
buildWxapkg([{ path: "x", data: "0123456789" }], {
declaredDataLength: 5,
}),
async (reader) => {
await expect(list(reader)).rejects.toMatchObject({ reason: "format" });
},
);
});
it("rejects a data region declared past the file", async () => {
await withReader(
buildWxapkg([{ path: "x", data: "ok" }], {
declaredDataLength: 1 << 24,
}),
async (reader) => {
await expect(list(reader)).rejects.toMatchObject({ reason: "format" });
},
);
});
});
describe("WxapkgArtifactReader index length", () => {
it("reads a one-byte member when indexLength counts only the rows", async () => {
const bytes = buildWxapkg([{ path: "a", data: "Z" }]);
expect(bytes).toHaveLength(32);
expect(bytes.readUInt32BE(5)).toBe(13);
await withReader(bytes, async (reader) => {
const [entry] = await list(reader);
expect(entry).toMatchObject({ byteOffset: 31, declaredSize: 1 });
expect(reader.formatMetadata()).toMatchObject({
index_length: 13,
data_length: 1,
file_count: 1,
});
expect(await text(await reader.open(entry!))).toBe("Z");
});
});
it("reads a one-byte member when indexLength includes the member count", async () => {
const bytes = buildWxapkg([{ path: "a", data: "Z" }], {
indexLengthIncludesFileCount: true,
});
expect(bytes).toHaveLength(32);
expect(bytes.readUInt32BE(5)).toBe(17);
await withReader(bytes, async (reader) => {
const [entry] = await list(reader);
expect(entry).toMatchObject({
path: "a",
byteOffset: 31,
declaredSize: 1,
});
expect(reader.formatMetadata()).toMatchObject({
index_length: 17,
data_length: 1,
file_count: 1,
});
expect(await text(await reader.open(entry!))).toBe("Z");
});
});
it("reads a count-inclusive member when exactly four bytes trail the package", async () => {
const bytes = Buffer.concat([
buildWxapkg([{ path: "a", data: "Z" }], {
indexLengthIncludesFileCount: true,
}),
Buffer.alloc(4, 0xee),
]);
expect(bytes).toHaveLength(36);
await withReader(bytes, async (reader) => {
const [entry] = await list(reader);
expect(entry).toMatchObject({ byteOffset: 31, declaredSize: 1 });
expect(await text(await reader.open(entry!))).toBe("Z");
});
});
it("rejects a count-inclusive member that covers trailing bytes", async () => {
const bytes = Buffer.concat([
buildWxapkg([{ path: "a", data: "Z" }], {
indexLengthIncludesFileCount: true,
}),
Buffer.alloc(4, 0xee),
]);
bytes.writeUInt32BE(5, 27);
await withReader(bytes, async (reader) => {
await expect(list(reader)).rejects.toMatchObject({ reason: "format" });
});
});
it("keeps a count-inclusive member inside the data region when bytes trail the package", async () => {
const bytes = Buffer.concat([
buildWxapkg([{ path: "a", data: "Z" }], {
indexLengthIncludesFileCount: true,
}),
Buffer.alloc(8, 0xee),
]);
await withReader(bytes, async (reader) => {
const [entry] = await list(reader);
expect(entry).toMatchObject({ byteOffset: 31, declaredSize: 1 });
expect(await text(await reader.open(entry!))).toBe("Z");
});
});
it("rejects an index byte of a count-inclusive package", async () => {
await withReader(
buildWxapkg([{ path: "a", data: "Z", offset: 18, size: 1 }], {
indexLengthIncludesFileCount: true,
}),
async (reader) => {
await expect(list(reader)).rejects.toMatchObject({ reason: "format" });
},
);
});
});
describe("WxapkgArtifactReader damaged packages", () => {
it("keeps a damaged package's format identity as a typed failure", async () => {
// Marks hold but the declared count cannot fit the declared table.
const bytes = buildWxapkg(members, {
declaredFileCount: 2,
declaredIndexLength: 12,
});
expect(hasWxapkgHeader(bytes)).toBe(true);
await withReader(bytes, async (reader) => {
await expect(list(reader)).rejects.toMatchObject({ reason: "format" });
});
});
it("rejects a truncated package before the declared table ends", async () => {
const bytes = buildWxapkg(members);
await withReader(bytes.subarray(0, 30), async (reader) => {
await expect(list(reader)).rejects.toMatchObject({ reason: "format" });
});
});
it("rejects a non-UTF-8 member name as a typed format failure", async () => {
const bytes = buildWxapkg(members);
bytes[26] = 0xff; // first byte of the first member name
await withReader(bytes, async (reader) => {
await expect(list(reader)).rejects.toMatchObject({ reason: "format" });
});
});
it("rejects a forged foreign entry with an integrity failure", async () => {
await withReader(buildWxapkg(members), async (reader) => {
const [entry] = await list(reader);
await expect(
reader.open({ ...entry!, adapterKey: "wxapkg-entry:9" }),
).rejects.toMatchObject({ reason: "integrity" });
});
});
});
describe("WxapkgArtifactReader lifecycle", () => {
it("aborts member reads when the caller cancels", async () => {
const data = Buffer.alloc(300 * 1024, 0x62);
await withReader(
buildWxapkg([{ path: "big.bin", data }]),
async (reader) => {
const [entry] = await list(reader);
const controller = new AbortController();
const stream = await reader.open(entry!, controller.signal);
let delivered = 0;
await expect(
(async () => {
for await (const chunk of stream) {
delivered += (chunk as Buffer).length;
controller.abort();
}
})(),
).rejects.toThrow();
expect(delivered).toBeLessThan(data.length);
},
);
});
it("survives a destroyed member stream and serves the next entry", async () => {
await withReader(buildWxapkg(members), async (reader) => {
const entries = await list(reader);
const abandoned = await reader.open(entries[0]!);
abandoned.destroy();
await expect(text(await reader.open(entries[1]!))).resolves.toBe(
"module.exports={};",
);
});
});
it("fails streams and later reads after close", async () => {
const reader = new WxapkgArtifactReader(
await writePackage(buildWxapkg(members)),
);
const entries: ArtifactEntry[] = [];
for await (const entry of reader.entries()) entries.push(entry);
const stream = await reader.open(entries[0]!);
const consumed = text(stream).catch((error: unknown) => error);
await reader.close();
expect(await consumed).toBeInstanceOf(Error);
await expect(reader.open(entries[1]!)).rejects.toMatchObject({
reason: "unavailable",
});
});
it("bounds concurrent member streams", async () => {
await withReader(buildWxapkg(members), async (reader) => {
const [entry] = await list(reader);
const streams = await Promise.all(
[0, 1, 2, 3].map(() => reader.open(entry!)),
);
await expect(reader.open(entry!)).rejects.toMatchObject({
reason: "limit",
});
for (const stream of streams) stream.destroy();
});
});
});
+311
View File
@@ -0,0 +1,311 @@
import { artifactExpansionBudgetBytes } from "../domain/artifactExpansionBudget.js";
import type { FileHandle } from "node:fs/promises";
import { setImmediate } from "node:timers/promises";
import { Readable } from "node:stream";
import {
ArtifactReaderFailure,
artifactFixedLimitFailure,
copyArtifactEntry,
sameArtifactEntry,
type ArtifactEntry,
type ArtifactReader,
type ArtifactTraversalOptions,
} from "./ArtifactReader.js";
import { ArtifactResourceScope } from "./ArtifactResourceScope.js";
import { assertArtifactReadActive } from "./ArtifactFileOperation.js";
import { readArtifactFileBytes } from "./ArtifactFileRange.js";
import type {
ArtifactFormatMetadata,
WxapkgMetadata,
} from "../domain/artifactFormatMetadata.js";
import type { StableRegularFileDescriptor } from "../filesystem/RegularFile.js";
import { ArtifactFileSource } from "./ArtifactFileSource.js";
import {
readWxapkgFileTable,
readWxapkgHeader,
wxapkgFormatFailure,
type WxapkgMember,
} from "./WxapkgStructures.js";
const CHUNK_BYTES = 64 * 1024;
/** Bound active member streams; each owns a file-range generator. */
const MAX_MEMBER_STREAMS = 4;
type WxapkgEntry = ArtifactEntry & {
readonly declaredSize: number;
readonly formatMetadata: Extract<
ArtifactFormatMetadata,
{ kind: "wxapkg-entry" }
>;
};
interface WxapkgEntryState {
readonly entry: WxapkgEntry;
readonly member: WxapkgMember;
}
/** One descriptor, one admitted member table and bounded raw-range member reads. */
export class WxapkgArtifactReader implements ArtifactReader {
readonly format = "wxapkg" as const;
readonly #scope = new ArtifactResourceScope();
readonly #entries = new Map<string, WxapkgEntryState>();
readonly #streams = new Map<Readable, AbortController>();
readonly #source: ArtifactFileSource;
#prepared: Promise<void> | undefined;
#header: WxapkgMetadata | undefined;
#remainingReadOperations = 0;
#admittedReadOperations = 0;
#closing = false;
constructor(path: string, admitted?: StableRegularFileDescriptor) {
this.#source = new ArtifactFileSource(path, "wxapkg package", admitted);
}
formatMetadata(): WxapkgMetadata | undefined {
return this.#header === undefined ? undefined : { ...this.#header };
}
async prepareContainer(
expectedSha256: string,
signal?: AbortSignal,
): Promise<void> {
await this.#scope.run(async () => {
this.#assertOpen(signal);
await this.#source.bindSha256(expectedSha256, signal);
});
}
async *entries(
signal?: AbortSignal,
options?: ArtifactTraversalOptions,
): AsyncIterable<ArtifactEntry> {
await this.#scope.run(async () => {
this.#assertOpen(signal);
this.#prepared ??= this.#prepare(signal);
await this.#prepared;
await this.#source.read(signal);
});
this.#assertOpen(signal);
if (this.#header?.directory_status === "encrypted") {
options?.onCoverage?.({
status: "unknown",
limitations: [
"wxapkg package is V1MMWX-encrypted; member identities and contents remain unknown.",
],
});
return;
}
let index = 0;
for (const { entry } of this.#entries.values()) {
if (index++ % 256 === 0) await setImmediate();
this.#assertOpen(signal);
yield copyArtifactEntry(entry);
}
// A close while paused at yield must not turn a cleared map into completion.
this.#assertOpen(signal);
}
async open(entry: ArtifactEntry, signal?: AbortSignal): Promise<Readable> {
return this.#scope.run(async () => {
this.#assertOpen(signal);
const state = this.#entries.get(entry.adapterKey);
if (state === undefined || !sameArtifactEntry(entry, state.entry))
throw new ArtifactReaderFailure(
"integrity",
"wxapkg entry does not match its owning inventory",
);
// Every logical open spends the work budget, including empty members
// whose streams perform only the carrier checks below.
this.#reserveMemberRead();
// Validate carrier freshness before handing out member bytes.
const { handle } = await this.#source.read(signal);
this.#assertOpen(signal);
if (this.#streams.size >= MAX_MEMBER_STREAMS)
throw new ArtifactReaderFailure(
"limit",
"wxapkg reader already owns four active member streams",
{ boundary: "capacity" },
);
const controller = new AbortController();
const output = Readable.from(
this.#memberChunks(handle, state, controller.signal),
{
objectMode: false,
highWaterMark: CHUNK_BYTES,
},
);
const onAbort = (): void => {
controller.abort(signal?.reason);
output.destroy(
new ArtifactReaderFailure(
"cancelled",
"wxapkg member read cancelled",
{ cause: signal?.reason },
),
);
};
output.on("error", () => undefined);
this.#streams.set(output, controller);
const completion = new Promise<void>((resolve) => {
output.once("close", () => {
signal?.removeEventListener("abort", onAbort);
this.#streams.delete(output);
resolve();
});
});
// Readable.from waits for its async iterator to retire before close. Keep
// that whole lifetime owned, including a suspended positional read.
void this.#scope
.run(() => completion)
.catch((cause: unknown) => {
controller.abort(cause);
output.destroy(
cause instanceof Error
? cause
: new Error("wxapkg stream ownership failed", { cause }),
);
});
signal?.addEventListener("abort", onAbort, { once: true });
if (signal?.aborted === true) onAbort();
return output;
});
}
async close(): Promise<void> {
this.#closing = true;
for (const [stream, controller] of this.#streams) {
const cause = new ArtifactReaderFailure(
"unavailable",
"wxapkg reader closed",
);
controller.abort(cause);
stream.destroy(cause);
}
await this.#scope.close();
await this.#source.close();
this.#entries.clear();
}
provenance(): readonly [] {
return [];
}
#assertOpen(signal?: AbortSignal): void {
assertArtifactReadActive(signal);
if (this.#closing)
throw new ArtifactReaderFailure("unavailable", "wxapkg reader is closed");
}
async #prepare(signal?: AbortSignal): Promise<void> {
const { handle, initial } = await this.#source.read(signal);
const header = await readWxapkgHeader(handle, initial.size, signal);
if (header === undefined)
throw wxapkgFormatFailure("no complete wxapkg header");
this.#header = {
kind: "wxapkg",
info: header.layout.status === "available" ? header.info : null,
index_length:
header.layout.status === "available" ? header.indexLength : null,
data_length:
header.layout.status === "available" ? header.dataLength : null,
file_count:
header.layout.status === "available" ? header.fileCount : null,
directory_status: header.layout.status,
};
if (header.layout.status !== "available") return;
const { members } = await readWxapkgFileTable(
handle,
header,
initial.size,
signal,
);
const budget = artifactExpansionBudgetBytes(initial.size);
let declaredBytes = 0n;
for (let index = 0; index < members.length; index++) {
if (index % 256 === 0) await setImmediate();
this.#assertOpen(signal);
const member = members[index]!;
// Overlapping member ranges are valid; each declared path causes a read.
declaredBytes += BigInt(member.size);
if (declaredBytes > BigInt(budget))
throw new ArtifactReaderFailure(
"limit",
`wxapkg members declare ${String(declaredBytes)} bytes, exceeding the ${String(budget)}-byte plaintext-work budget`,
{ boundary: "memory" },
);
const adapterKey = `wxapkg-entry:${String(index)}`;
this.#entries.set(adapterKey, {
entry: {
path: member.path,
sourcePath: member.declaredPath,
kind: "file",
declaredSize: member.size,
compressedSize: null,
executable: false,
encrypted: false,
// Member offsets are absolute carrier positions.
byteOffset: member.offset,
declaredIntegrity: null,
unpacked: false,
limitations: [],
adapterKey,
formatMetadata: {
kind: "wxapkg-entry",
index,
source_path: member.declaredPath,
declared_offset: member.offset,
},
},
member,
});
}
// Header, table and per-chunk member reads; a small-reader floor applies.
this.#admittedReadOperations =
4 *
Math.max(
1024,
members.length + Math.ceil(Number(declaredBytes) / CHUNK_BYTES),
);
this.#remainingReadOperations = this.#admittedReadOperations;
await this.#source.read(signal);
}
async *#memberChunks(
handle: FileHandle,
state: WxapkgEntryState,
signal: AbortSignal,
): AsyncGenerator<Buffer> {
const { offset, size } = state.member;
let delivered = 0,
chunks = 0;
for (let position = offset; position < offset + size;) {
if (chunks++ % 64 === 0) await setImmediate();
this.#assertOpen(signal);
this.#reserveMemberRead();
const length = Math.min(CHUNK_BYTES, offset + size - position);
const chunk = await readArtifactFileBytes(
handle,
position,
length,
offset + size,
signal,
);
position += chunk.length;
delivered += chunk.length;
yield chunk;
}
if (delivered !== size)
throw wxapkgFormatFailure("member ended before its declared size");
await this.#source.read(signal);
}
#reserveMemberRead(): void {
if (this.#remainingReadOperations <= 0)
throw artifactFixedLimitFailure(
"wxapkg directory-and-chunk I/O work",
this.#admittedReadOperations,
"operations",
);
this.#remainingReadOperations--;
}
}
+214
View File
@@ -0,0 +1,214 @@
import type { FileHandle } from "node:fs/promises";
import {
ArtifactReaderFailure,
artifactFixedLimitFailure,
} from "./ArtifactReader.js";
import { readArtifactFileBytes } from "./ArtifactFileRange.js";
import { assertArtifactReadActive } from "./ArtifactFileOperation.js";
import { setImmediate } from "node:timers/promises";
import {
hasWxapkgEncryptedMagic,
readWxapkgHeaderFacts,
resolveWxapkgIndexLayouts,
selectWxapkgIndexLayout,
WXAPKG_MIN_ROW_BYTES,
type WxapkgIndexLayout,
type WxapkgIndexLayoutFailure,
} from "../domain/wxapkgFormat.js";
/** The member table is a flat name/offset/size row list. */
const MAX_TABLE_BYTES = 16 * 1024 * 1024;
/** Encoded retained directory representation; paths are charged separately. */
const DIRECTORY_BUDGET_BYTES = 64 * 1024 * 1024;
/** Member names are relative package paths; larger claims are malformed. */
const MAX_MEMBER_NAME_BYTES = 64 * 1024;
/** Detecting prefix covers the fixed header, table head and encrypted magic. */
const HEADER_PREFIX_BYTES = 64;
const UTF8 = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
export const wxapkgFormatFailure = (detail: string): ArtifactReaderFailure =>
new ArtifactReaderFailure("format", `wxapkg is malformed: ${detail}`);
/** Reserve buffers, JS rows and escaped name projections before allocating. */
class WxapkgDirectoryBudget {
#remaining = DIRECTORY_BUDGET_BYTES;
reserve(bytes: number): void {
if (!Number.isSafeInteger(bytes) || bytes < 0 || bytes > this.#remaining)
throw artifactFixedLimitFailure(
"wxapkg directory representation",
DIRECTORY_BUDGET_BYTES,
"bytes",
);
this.#remaining -= bytes;
}
}
export interface WxapkgHeader {
readonly info: number;
readonly indexLength: number;
readonly dataLength: number;
readonly fileCount: number;
readonly layout:
| { readonly status: "encrypted" }
| {
readonly status: "available";
readonly indexLayouts: readonly WxapkgIndexLayout[];
};
}
export interface WxapkgMember {
readonly offset: number;
readonly size: number;
/** Package-relative path; declared root prefixes are translated here. */
readonly path: string;
/** Declared member name as stored in the table. */
readonly declaredPath: string;
}
export interface WxapkgFileTable {
readonly members: readonly WxapkgMember[];
}
/** Decode header fields once; encrypted variants and absent signatures are distinct. */
export const readWxapkgHeader = async (
handle: FileHandle,
size: number,
signal?: AbortSignal,
): Promise<WxapkgHeader | undefined> => {
const prefix = await readArtifactFileBytes(
handle,
0,
Math.min(size, HEADER_PREFIX_BYTES),
size,
signal,
);
if (hasWxapkgEncryptedMagic(prefix))
return {
info: 0,
indexLength: 0,
dataLength: 0,
fileCount: 0,
layout: { status: "encrypted" },
};
const facts = readWxapkgHeaderFacts(prefix);
if (facts === undefined) return undefined;
const indexes = resolveWxapkgIndexLayouts(facts, size);
if (!indexes.ok) throw wxapkgIndexLayoutFailure(indexes.error);
return {
...facts,
layout: { status: "available", indexLayouts: indexes.value },
};
};
const wxapkgIndexLayoutFailure = (
failure: WxapkgIndexLayoutFailure,
): ArtifactReaderFailure =>
wxapkgFormatFailure(
failure._tag === "DataRegionExceedsFile"
? "declared data region exceeds the file"
: "declared member table exceeds the file",
);
export const readWxapkgFileTable = async (
handle: FileHandle,
header: WxapkgHeader,
size: number,
signal?: AbortSignal,
): Promise<WxapkgFileTable> => {
const layout = header.layout;
if (layout.status !== "available")
throw new ArtifactReaderFailure(
"unavailable",
"wxapkg directory is encrypted",
);
const indexLayouts = layout.indexLayouts;
let readBytes = 0;
let tableStart: number | undefined;
for (const indexLayout of indexLayouts) {
if (tableStart === undefined) tableStart = indexLayout.tableStart;
if (indexLayout.tableBytes > readBytes) readBytes = indexLayout.tableBytes;
}
if (tableStart === undefined)
throw wxapkgFormatFailure("declared member table exceeds the file");
if (readBytes > MAX_TABLE_BYTES)
throw artifactFixedLimitFailure(
"wxapkg member table",
MAX_TABLE_BYTES,
"bytes",
);
const budget = new WxapkgDirectoryBudget();
budget.reserve(readBytes);
const table = await readArtifactFileBytes(
handle,
tableStart,
readBytes,
size,
signal,
);
// Loose lower bound already checked in facts; precise rows are checked here.
if (table.length < header.fileCount * WXAPKG_MIN_ROW_BYTES)
throw wxapkgFormatFailure("member table is truncated");
const members: WxapkgMember[] = [];
let cursor = 0;
for (let index = 0; index < header.fileCount; index++) {
if (index % 256 === 0) await setImmediate();
assertArtifactReadActive(signal);
if (cursor + 4 > table.length)
throw wxapkgFormatFailure("member row is truncated");
const nameLength = table.readUInt32BE(cursor);
cursor += 4;
if (
nameLength === 0 ||
nameLength > MAX_MEMBER_NAME_BYTES ||
cursor + nameLength + 8 > table.length
)
throw wxapkgFormatFailure("member row exceeds the table");
// Row storage plus the reader's owned entries/map; charge before decode.
budget.reserve(512 + nameLength * 24);
let declaredPath: string;
try {
declaredPath = UTF8.decode(table.subarray(cursor, cursor + nameLength));
} catch (cause: unknown) {
throw new ArtifactReaderFailure(
"format",
"wxapkg member path is not valid UTF-8",
{
cause,
},
);
}
cursor += nameLength;
const offset = table.readUInt32BE(cursor);
const sizeMember = table.readUInt32BE(cursor + 4);
cursor += 8;
// Producer names are root-prefixed; translate to package-relative once.
const path = declaredPath.startsWith("/")
? declaredPath.slice(1)
: declaredPath;
if (path.length === 0)
throw wxapkgFormatFailure("member name has no package-relative path");
members.push({ offset, size: sizeMember, path, declaredPath });
}
const candidates = indexLayouts.filter((index) => index.tableBytes >= cursor);
if (candidates.length === 0)
throw wxapkgFormatFailure("member table is truncated");
// Rows and member ranges choose the convention. EOF is not a signal: four
// trailing bytes can make the other convention end on the carrier.
const index = selectWxapkgIndexLayout(candidates, cursor, members);
if (index === undefined) {
const outside = members.find((member) => {
const extentEnd = member.offset + member.size;
return candidates.every(
(candidate) =>
member.offset < candidate.dataStart ||
extentEnd > candidate.dataEnd ||
extentEnd < member.offset,
);
});
throw wxapkgFormatFailure(
`member exceeds the data region: ${outside?.path ?? ""}`,
);
}
return { members };
};
+1
View File
@@ -30,6 +30,7 @@ type FormatsWithReaders =
| "ines"
| "godot-pck"
| "unityfs"
| "wxapkg"
| "uimage"
| "mach-o-universal"
| "directory"
+11
View File
@@ -40,6 +40,7 @@ import { PyInstallerArtifactReader } from "./PyInstallerArtifactReader.js";
import { Uf2ArtifactReader } from "./Uf2ArtifactReader.js";
import { UImageArtifactReader } from "./UImageArtifactReader.js";
import { UnityFsArtifactReader } from "./UnityFsArtifactReader.js";
import { WxapkgArtifactReader } from "./WxapkgArtifactReader.js";
import { ZipArtifactReader } from "./ZipArtifactReader.js";
/** How inventory describes a format when no reader was constructed. */
@@ -286,6 +287,16 @@ const NON_ZIP_FORMATS = {
createReader: (path, options) =>
new UnityFsArtifactReader(path, sourceOf(options)),
}),
wxapkg: container("wxapkg", {
roleContainer: true,
retainClassifiedSource: true,
extractable: true,
absentInventory: "unavailable",
readerLimitation:
"wxapkg inventory establishes packaged member names and stored bytes; wxml/wxss sources, compiled bundles and runtime behavior are not reconstructed. V1MMWX-encrypted packages are unsupported.",
createReader: (path, options) =>
new WxapkgArtifactReader(path, sourceOf(options)),
}),
uimage: container("uimage", {
roleContainer: true,
retainClassifiedSource: true,
@@ -215,6 +215,15 @@ export const materializeArtifactInventoryInScope = async (
input.selectedPath ?? input.inputPath,
`UnityFS directory is ${containerMetadata.directory_status}; its complete contents cannot be extracted`,
);
if (
containerMetadata?.kind === "wxapkg" &&
containerMetadata.directory_status !== "available"
)
throw new AnalysisUnsupportedTargetError(
"extract_artifact",
input.selectedPath ?? input.inputPath,
`wxapkg directory is ${containerMetadata.directory_status}; its complete contents cannot be extracted`,
);
const selectedOccurrences = snapshot.occurrences.filter(
(occurrence) =>
(occurrence.entry_kind === "file" || occurrence.entry_kind === "slice") &&
@@ -2,6 +2,10 @@ import { artifactFormatDescriptor } from "../artifactFormats.js";
import { hasGodotPckHeader } from "../../domain/godotPckFormat.js";
import { hasUImageMagic } from "../../domain/uimageFormat.js";
import { hasUnityFsHeader } from "../../domain/unityfsFormat.js";
import {
hasWxapkgEncryptedMagic,
hasWxapkgHeader,
} from "../../domain/wxapkgFormat.js";
import { detectHexRecordEncoding } from "../../domain/firmware/hexRecords.js";
import { recognizeEspImage } from "../../domain/firmware/espImage.js";
import { hasUf2Block } from "../../domain/firmware/uf2.js";
@@ -296,7 +300,8 @@ export const rootOccurrenceFor = (
executable: metadata.executable,
encrypted:
(metadata.formatMetadata?.kind === "godot-pck" ||
metadata.formatMetadata?.kind === "unityfs") &&
metadata.formatMetadata?.kind === "unityfs" ||
metadata.formatMetadata?.kind === "wxapkg") &&
metadata.formatMetadata.directory_status === "encrypted",
hash_status: "verified",
source_location: null,
@@ -607,6 +612,8 @@ export const classifyArtifactBytes = (
if (hasZipSignature(prefix)) return "zip";
if (hasGodotPckHeader(prefix)) return "godot-pck";
if (hasUnityFsHeader(prefix)) return "unityfs";
if (hasWxapkgEncryptedMagic(prefix) || hasWxapkgHeader(prefix))
return "wxapkg";
if (hasUImageMagic(prefix)) return "uimage";
const hexRecords = detectHexRecordEncoding(prefix);
if (hexRecords !== undefined) return hexRecords;
+1
View File
@@ -76,6 +76,7 @@ const formatSchema = z.enum([
"ines",
"godot-pck",
"unityfs",
"wxapkg",
"dmg",
"pkg",
"intel-hex",
@@ -277,6 +277,7 @@ export const projectArtifactInventoryView = (
view.kind === "inventory" ||
inventory.manifest.root_format === "godot-pck" ||
inventory.manifest.root_format === "unityfs" ||
inventory.manifest.root_format === "wxapkg" ||
inventory.manifest.root_format === "pyinstaller" ||
inventory.manifest.root_format === "node-sea" ||
inventory.manifest.root_format === "intel-hex" ||
+28
View File
@@ -63,6 +63,24 @@ export const unityFsEntryMetadataSchema = z.strictObject({
flags: uint32,
});
/** Parsed source fields owned by the wxapkg adapter, independent of member bytes. */
export const wxapkgMetadataSchema = z.strictObject({
kind: z.literal("wxapkg"),
info: uint32.nullable(),
index_length: uint32.nullable(),
data_length: uint32.nullable(),
file_count: uint32.nullable(),
directory_status: z.enum(["available", "encrypted"]),
});
export type WxapkgMetadata = z.infer<typeof wxapkgMetadataSchema>;
export const wxapkgEntryMetadataSchema = z.strictObject({
kind: z.literal("wxapkg-entry"),
index: uint32,
source_path: z.string(),
declared_offset: offset,
});
/** Source cookie fields and derived directory counts without Python execution. */
export const pyInstallerMetadataSchema = z.strictObject({
kind: z.literal("pyinstaller"),
@@ -765,6 +783,8 @@ export const artifactFormatMetadataSchema = z.discriminatedUnion("kind", [
godotPckEntryMetadataSchema,
unityFsMetadataSchema,
unityFsEntryMetadataSchema,
wxapkgMetadataSchema,
wxapkgEntryMetadataSchema,
pyInstallerMetadataSchema,
pyInstallerEntryMetadataSchema,
dotnetBundleEntryMetadataSchema,
@@ -837,6 +857,14 @@ const declarationsByKind: {
flags: metadata.flags,
}),
},
wxapkg: {
notCompared:
"Its fields come from the package's own bytes, which content comparison covers.",
},
"wxapkg-entry": {
notCompared:
"index is layout. source_path keeps the declared name, including a producer root prefix the occurrence path strips.",
},
pyinstaller: {
notCompared:
"Its fields come from the archive's own bytes, which content comparison covers.",
+2
View File
@@ -67,6 +67,7 @@ export const ARTIFACT_FORMAT_IDS = [
"ines",
"godot-pck",
"unityfs",
"wxapkg",
"uimage",
"javascript-bundle",
"source-map",
@@ -113,6 +114,7 @@ const artifactContentFormatSchema = artifactFormatSchema.extract([
"zip",
"godot-pck",
"unityfs",
"wxapkg",
"uimage",
"intel-hex",
"srec",
+1
View File
@@ -293,6 +293,7 @@ export type BinaryTarget =
| "ines"
| "godot-pck"
| "unityfs"
| "wxapkg"
| "uimage"
| "dmg"
| "pkg"
+1
View File
@@ -27,6 +27,7 @@ export const evidenceSubjectFormatSchema = z.enum([
"ines",
"godot-pck",
"unityfs",
"wxapkg",
"uimage",
"dmg",
"pkg",
+1
View File
@@ -73,6 +73,7 @@ export const liveBinaryTargetSummaryVariants = [
"ines",
"godot-pck",
"unityfs",
"wxapkg",
"uimage",
"dmg",
"pkg",
+198
View File
@@ -0,0 +1,198 @@
import { err, ok, type Result } from "./result.js";
/**
* WeChat mini-program package (wxapkg): an 18-byte big-endian header, a
* name/offset/size member table, then verbatim member data. Members are stored
* uncompressed at absolute file offsets. All multi-byte fields are big-endian.
*/
export const WXAPKG_FIRST_MARK = 0xbe;
export const WXAPKG_LAST_MARK = 0xed;
export const WXAPKG_HEADER_BYTES = 18;
/** Minimum fixed row fields: name length, offset and size without name bytes. */
export const WXAPKG_MIN_ROW_BYTES = 12;
/**
* The member count is stored in the last four header bytes. Some producers
* include those bytes in `indexLength`; others count only the rows that follow.
*/
const WXAPKG_MEMBER_COUNT_BYTES = 4;
/**
* Encrypted PC-client packages start with this ASCII magic instead of the
* plain header. Their member data is ciphertext; no decryption is supported.
*/
export const WXAPKG_ENCRYPTED_MAGIC = Buffer.from("V1MMWX", "latin1");
/** Whether bytes start with the encrypted PC-client magic. */
export const hasWxapkgEncryptedMagic = (bytes: Uint8Array): boolean =>
bytes.length >= WXAPKG_ENCRYPTED_MAGIC.length &&
Buffer.from(
bytes.buffer,
bytes.byteOffset,
WXAPKG_ENCRYPTED_MAGIC.length,
).equals(WXAPKG_ENCRYPTED_MAGIC);
/** Declared header fields; support and extent admission belong to callers. */
export interface WxapkgHeaderFacts {
/** Unknown header field after the first mark; producers write zero. */
readonly info: number;
/**
* Declared index length. This is either the row bytes or those rows plus
* the 4-byte member count; {@link resolveWxapkgIndexLayouts} admits both.
*/
readonly indexLength: number;
/** Declared byte length of the member data region. */
readonly dataLength: number;
readonly fileCount: number;
}
/**
* Decode declared fields after the caller admits a prefix covering the fixed
* header. Undefined means the admitted bytes cannot carry a plausible header;
* both mark checks and minimal row accounting must hold.
*/
export const readWxapkgHeaderFacts = (
bytes: Uint8Array,
): WxapkgHeaderFacts | undefined => {
if (
bytes.length < WXAPKG_HEADER_BYTES ||
bytes[0] !== WXAPKG_FIRST_MARK ||
bytes[13] !== WXAPKG_LAST_MARK
)
return undefined;
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
const info = view.getUint32(1);
const indexLength = view.getUint32(5);
const dataLength = view.getUint32(9);
const fileCount = view.getUint32(14);
// Every row carries at least its fixed fields; a table too small for the
// declared count cannot be a wxapkg index.
if (fileCount === 0 || indexLength < fileCount * WXAPKG_MIN_ROW_BYTES)
return undefined;
return { info, indexLength, dataLength, fileCount };
};
/**
* Whether bytes carry the plain wxapkg marks. Recognition is intentionally
* separate from admission: a damaged package keeps its format identity and the
* reader rejects impossible table accounting as a typed format failure.
*/
export const hasWxapkgHeader = (bytes: Uint8Array): boolean =>
bytes.length >= WXAPKG_HEADER_BYTES &&
bytes[0] === WXAPKG_FIRST_MARK &&
bytes[13] === WXAPKG_LAST_MARK;
/** Row table and declared data region for one index-length convention. */
export interface WxapkgIndexLayout {
/** Absolute offset of the first name/offset/size row. */
readonly tableStart: number;
/** Row bytes to read. The member count, when included, stays in the header. */
readonly tableBytes: number;
/** Inclusive start of the declared member-data region. */
readonly dataStart: number;
/** Exclusive end of the declared member-data region. */
readonly dataEnd: number;
}
export type WxapkgIndexLayoutFailure =
| { readonly _tag: "IndexExceedsFile" }
| { readonly _tag: "DataRegionExceedsFile" };
const spanEnd = (start: number, length: number): number | undefined => {
if (
!Number.isSafeInteger(start) ||
!Number.isSafeInteger(length) ||
length < 0
)
return undefined;
const end = start + length;
return Number.isSafeInteger(end) ? end : undefined;
};
const indexLayout = (
facts: WxapkgHeaderFacts,
fileSize: number,
includedCountBytes: number,
): WxapkgIndexLayout | "index-outside" | "data-outside" | "too-small" => {
if (
!Number.isSafeInteger(facts.indexLength) ||
!Number.isSafeInteger(facts.fileCount) ||
facts.fileCount < 0 ||
facts.indexLength < includedCountBytes
)
return "too-small";
const tableBytes = facts.indexLength - includedCountBytes;
if (tableBytes < facts.fileCount * WXAPKG_MIN_ROW_BYTES) return "too-small";
const dataStart = spanEnd(WXAPKG_HEADER_BYTES, tableBytes);
if (dataStart === undefined || dataStart > fileSize) return "index-outside";
const dataEnd = spanEnd(dataStart, facts.dataLength);
if (dataEnd === undefined || dataEnd > fileSize) return "data-outside";
return { tableStart: WXAPKG_HEADER_BYTES, tableBytes, dataStart, dataEnd };
};
/**
* Admit every index-length convention whose index and declared data region
* both lie inside the carrier. Trailing bytes can make both fit, so callers
* keep every candidate until the parsed rows and member ranges decide.
*/
export const resolveWxapkgIndexLayouts = (
facts: WxapkgHeaderFacts,
fileSize: number,
): Result<readonly WxapkgIndexLayout[], WxapkgIndexLayoutFailure> => {
if (!Number.isSafeInteger(fileSize) || fileSize < 0)
return err({ _tag: "IndexExceedsFile" });
const described = [WXAPKG_MEMBER_COUNT_BYTES, 0].map((included) =>
indexLayout(facts, fileSize, included),
);
const admissible = described.filter(
(layout): layout is WxapkgIndexLayout => typeof layout !== "string",
);
if (admissible.length === 0)
return err({
_tag: described.some((layout) => layout === "data-outside")
? "DataRegionExceedsFile"
: "IndexExceedsFile",
});
return ok(admissible);
};
const memberFitsIndex = (
layout: WxapkgIndexLayout,
member: { readonly offset: number; readonly size: number },
): boolean => {
const extentEnd = member.offset + member.size;
return (
member.offset >= layout.dataStart &&
extentEnd <= layout.dataEnd &&
extentEnd >= member.offset
);
};
/**
* Choose the admitted index that contains the parsed rows and every member
* range. An index that ends on the last parsed row wins a tie. Neither the
* carrier end nor the shorter table is enough on its own: four trailing bytes
* can make the other convention land on EOF.
*/
export const selectWxapkgIndexLayout = (
layouts: readonly WxapkgIndexLayout[],
consumedRowBytes: number,
members: readonly { readonly offset: number; readonly size: number }[],
): WxapkgIndexLayout | undefined => {
const fitting = layouts.filter(
(layout) =>
layout.tableBytes >= consumedRowBytes &&
members.every((member) => memberFitsIndex(layout, member)),
);
const exactRows = fitting.filter(
(layout) => layout.tableBytes === consumedRowBytes,
);
const pool = exactRows.length > 0 ? exactRows : fitting;
let selected: WxapkgIndexLayout | undefined;
for (const layout of pool) {
if (selected === undefined || layout.tableBytes < selected.tableBytes)
selected = layout;
}
return selected;
};
+87
View File
@@ -0,0 +1,87 @@
import { readFile, rm, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, it } from "vitest";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { artifactInspectionResultSchema } from "../../../src/domain/artifactInspection.js";
import { artifactExtractionResultSchema } from "../../../src/domain/artifactGraph.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js";
import { buildWxapkg, wxapkgEncryptedBytes } from "../../fixtures/wxapkg.js";
it.each(["full", "compact"] as const)(
"retains wxapkg facts and encrypted coverage through compiled MCP (%s)",
async (profile) => {
const root = await createTestTempDirectory("rea-wxapkg-mcp-");
const path = join(root, "package.wxapkg");
const payload = Buffer.from("module.exports={};");
await writeFile(
path,
buildWxapkg([
{ path: "app-config.json", data: "{}" },
{ path: "pages/index/index.js", data: payload },
]),
);
const { call } = await connectLocalToolsMcp({
REA_MCP_INPUT_SCHEMA_PROFILE: profile,
});
let outputRoot: string | undefined;
try {
expect((await call("open_binary", { path })).isError).not.toBe(true);
const reply = await call("inspect_artifact", {});
expect(reply.isError, JSON.stringify(reply)).not.toBe(true);
const inspection = artifactInspectionResultSchema.parse(
parseEvidence(reply.structuredContent).normalized_result,
);
const observed = inspection.substeps[0]!.evidence.normalized_result;
expect(observed.occurrences).toContainEqual(
expect.objectContaining({
logical_path: "pages/index/index.js",
hash_status: "verified",
source_location: expect.objectContaining({
offset: expect.any(Number),
}),
}),
);
const extracted = await call("extract_artifact", {});
expect(extracted.isError, JSON.stringify(extracted)).not.toBe(true);
const result = artifactExtractionResultSchema.parse(
parseEvidence(extracted.structuredContent).normalized_result,
);
outputRoot = result.output_root;
expect(
(await readFile(join(outputRoot, "pages/index/index.js"))).equals(
payload,
),
).toBe(true);
await call("close_binary", {});
// Encrypted PC-client packages remain represented without member guesses.
await writeFile(path, wxapkgEncryptedBytes());
await call("open_binary", { path });
const unknownReply = await call("inspect_artifact", {});
expect(unknownReply.isError, JSON.stringify(unknownReply)).not.toBe(true);
const unknown = artifactInspectionResultSchema.parse(
parseEvidence(unknownReply.structuredContent).normalized_result,
).substeps[0]!.evidence.normalized_result;
expect(unknown.coverage).toMatchObject({ status: "unknown" });
expect(unknown.occurrences).toMatchObject([
{
artifact_format: "wxapkg",
encrypted: true,
format_metadata: {
kind: "wxapkg",
directory_status: "encrypted",
file_count: null,
},
},
]);
const refused = await call("extract_artifact", {});
expect(refused.isError).toBe(true);
expect(JSON.stringify(refused)).toContain("cannot be extracted");
await call("close_binary", {});
} finally {
if (outputRoot !== undefined)
await rm(outputRoot, { recursive: true, force: true });
}
},
60_000,
);
+114
View File
@@ -0,0 +1,114 @@
/** Synthetic WeChat mini-program package (wxapkg) builders for reader tests. */
export interface WxapkgMember {
readonly path: string;
readonly data: string | Buffer;
/** Override the declared member offset (absolute file position). */
readonly offset?: number;
/** Override the declared member byte size. */
readonly size?: number;
}
export interface WxapkgOptions {
readonly firstMark?: number;
readonly lastMark?: number;
readonly info?: number;
readonly declaredFileCount?: number;
readonly declaredIndexLength?: number;
readonly declaredDataLength?: number;
/**
* Count the 4-byte member count inside `indexLength`. Row bytes and member
* offsets stay where they are; only the declared length grows by four.
*/
readonly indexLengthIncludesFileCount?: boolean;
/** Override the encoded row's name-length field for every member. */
readonly nameLength?: number;
/** Bytes appended to the table region beyond the declared rows. */
readonly tablePadding?: number;
/** Declare every member after the first at the first member's offset. */
readonly aliasOffsets?: boolean;
/** Replace the whole package prefix (encrypted variants). */
readonly magic?: Buffer;
/** Truncate the final package to this many bytes. */
readonly truncate?: number;
}
const utf8 = (value: string): Buffer => Buffer.from(value, "utf8");
const u32 = (value: number): Buffer => {
const out = Buffer.alloc(4);
out.writeUInt32BE(value);
return out;
};
/** Encrypted PC-client package prefix, without plaintext members. */
export const wxapkgEncryptedBytes = (): Buffer =>
Buffer.concat([Buffer.from("V1MMWX", "latin1"), Buffer.alloc(64)]);
/** Plain wxapkg: 18-byte header, name/offset/size rows, then verbatim data. */
export const buildWxapkg = (
members: readonly WxapkgMember[],
options: WxapkgOptions = {},
): Buffer => {
if (options.magic !== undefined)
return Buffer.concat([
options.magic,
...members.map(({ data }) =>
typeof data === "string" ? utf8(data) : data,
),
]);
const names = members.map(({ path }) => utf8(path));
// Encoded rows carry the real name bytes regardless of any declared override.
const rowBytes = names.reduce((sum, name) => sum + 12 + name.length, 0);
const rowRegion = rowBytes + (options.tablePadding ?? 0);
const indexLength =
options.declaredIndexLength ??
rowRegion + (options.indexLengthIncludesFileCount === true ? 4 : 0);
// A declared length override repositions the data. Including the member
// count changes only the length field; the rows still begin at byte 18.
const dataStart =
options.declaredIndexLength !== undefined
? 18 + indexLength
: 18 + rowRegion;
let cursor = dataStart;
const rows: Buffer[] = [];
const payload: Buffer[] = [];
let firstOffset: number | undefined;
members.forEach((member, index) => {
const name = names[index]!;
const data =
typeof member.data === "string" ? utf8(member.data) : member.data;
const aliased =
options.aliasOffsets === true && index > 0 && firstOffset !== undefined;
const offset = aliased ? firstOffset! : (member.offset ?? cursor);
const size = member.size ?? data.length;
firstOffset ??= offset;
rows.push(
u32(options.nameLength ?? name.length),
name,
u32(offset),
u32(size),
);
if (!aliased) {
payload.push(data);
if (offset === cursor) cursor = offset + size;
}
});
const dataLength =
options.declaredDataLength ?? Math.max(0, cursor - dataStart);
const header = Buffer.concat([
Buffer.from([options.firstMark ?? 0xbe]),
u32(options.info ?? 0),
u32(indexLength),
u32(dataLength),
Buffer.from([options.lastMark ?? 0xed]),
u32(options.declaredFileCount ?? members.length),
]);
const table = Buffer.concat([
...rows,
Buffer.alloc(options.tablePadding ?? 0),
]);
let bytes = Buffer.concat([header, table, ...payload]);
if (options.truncate !== undefined)
bytes = bytes.subarray(0, options.truncate);
return bytes;
};